Senior Data Engineer

Location: While this position is posted in a specific location, all of Oyster’s positions are fully remote and you can work from home. Forever. To create the best experience for our new hire, this role requires you to be based within UTC−6 to UTC+3.

Oyster’s Data Engineering team is building the foundational AI platform layer that will enable teams across Oyster to develop, deploy, and operate secure, production-grade AI capabilities. As a Senior Data Engineer focused on AI Platform, you’ll work at the intersection of data engineering, platform engineering, and applied AI. You’ll build the data pipelines, services, integrations, and reusable platform capabilities that power AI use cases across Oyster.

You’ll leverage Oyster’s existing AWS and Snowflake data platform to make enterprise data accessible, reliable, secure, and useful for AI, supporting capabilities such as LLMs, RAG, embeddings, vector search, AI agents, tool calling, and AI workflows.

This is not a traditional data engineering role focused primarily on analytics pipelines or reporting. You’ll help define and build the data and platform foundations that let AI systems move from experimentation to reliable production, working closely with Engineering, Product, IT, and other teams across Oyster.

Key Responsibilities

• Design and build data pipelines and platform capabilities that support AI applications, knowledge systems, and AI workflows.
• Build and maintain data and knowledge pipelines for ingestion, transformation, chunking, embeddings, retrieval, vector search, metadata, and knowledge management.
• Develop secure, reusable ways for AI systems to access enterprise data and systems using APIs, MCP, tool calling, and similar patterns.
• Build reusable services, libraries, frameworks, and developer tooling that make it easier for engineering teams to build and productionize AI capabilities.
• Apply strong data engineering practices around data modeling, data quality, lineage, access, reliability, and governance to AI-related data and systems.
• Help establish patterns for AI deployment, observability, evaluation, and lifecycle management, including quality, latency, reliability, security, and cost.
• Partner with engineers and stakeholders to take AI use cases from prototype to production, ensuring the underlying data and infrastructure are scalable and maintainable.
• Ensure AI platform capabilities meet Oyster’s security, privacy, access control, and data governance requirements.
• Evaluate emerging AI and data technologies and determine where they can create meaningful value for Oyster.

Core Requirements

• 5+ years of experience in data engineering, platform engineering, backend engineering, ML engineering, or a related discipline, with experience building and operating production systems.
• Strong Python and SQL skills.
• Strong data engineering fundamentals, including data pipelines, data modeling, data quality, orchestration, and secure data access.
• Experience with Snowflake, Databricks, or comparable modern data platforms, and tools such as dbt, Airflow, or similar.
• Hands-on experience building or supporting production systems using LLMs / generative AI.
• Practical experience with one or more of RAG, embeddings, vector search, tool/function calling, AI agents, or enterprise knowledge systems.
• Experience building data or platform capabilities that are reusable across teams, rather than only developing one-off solutions.
• Experience taking systems from experimentation to reliable production, including considerations around scalability, observability, performance, and maintainability.
• Solid understanding of security, authentication/authorization, privacy, and data governance, particularly in the context of enterprise data.
• Strong communicator who can work across Engineering, Product, and IT to turn ambiguous AI and data problems into practical engineering solutions.

Bonus

• Experience with AWS Bedrock or other managed foundation-model platforms.
• Hands-on experience with MCP (Model Context Protocol) or similar approaches for connecting AI systems to enterprise tools and data.
• Experience with LLM evaluation, observability, monitoring, or AI quality.
• Experience with LangChain, LangGraph, or similar frameworks.
• Experience with vector databases or search technologies.
• Experience implementing AI security, governance, or responsible AI practices.
• Experience building internal developer platforms, SDKs, APIs, or reusable infrastructure for engineering teams.