Senior Data Software Engineer

We are looking for a Senior Data Software Engineer to join our team.
This role focuses on building and supporting data infrastructure that powers AI-driven products and intelligent agent systems. You'll have the opportunity to work with cutting-edge technologies and contribute to scalable, reliable platforms in a collaborative environment.
Responsibilities
Design, build, and maintain data ingestion and processing pipelines that feed RAG systems, including handling unstructured data, images, videos, metadata, and permissions
Administer and optimize vector database infrastructure, including Amazon Kendra with an ongoing migration to OpenSearch
Create evaluation datasets and performance measurement frameworks for agents
Develop monitoring and observability pipelines for AI workloads, covering latency, quality, and cost dashboards
Implement data governance, privacy safeguards, and quality controls for AI training and inference data
Support A/B testing and experimentation infrastructure for assessing agent iterations
Collaborate with Backend AI engineers on data schemas and embedding strategies
Requirements
A minimum of 3 years of data engineering experience, including direct exposure to AI/ML data infrastructure
Strong Python skills for building data pipelines, ETL processes, and backend automation scripting
Hands-on production experience with vector databases, including schema design and index management for Amazon Kendra or OpenSearch
Deep understanding of search and retrieval concepts, including embedding models, chunking strategies, and retrieval optimization
Practical knowledge of AWS services such as S3, Glue, Athena, and Kinesis (or equivalents), along with Docker and distributed data environments
Experience embedding data quality practices such as monitoring, validation, and lineage tracking as operational defaults
Background in designing AI/ML evaluation metrics and establishing systematic tracking through evaluation frameworks
English language proficiency (written and spoken) at B2+ level or higher
Nice to have
Experience with LangSmith, RAGAS, or custom evaluation framework solutions
Background in multi-modal data processing covering unstructured text, images, and videos, along with associated governance
Hands-on involvement with LLM fine-tuning data preparation
Familiarity with observability tooling deeply integrated with AI calls, such as Langfuse or Arize
Experience building streaming data pipelines using technologies such as Kafka or Kinesis