Senior Data Engineer responsible for building and maintaining scalable data ingestion systems for healthcare data. Optimizing data pipelines and collaborating with analytics engineers in a fast-growing environment.
Responsibilities
The Senior Data Engineer is responsible for building and maintaining scalable data ingestion infrastructure and operational systems.
Develop and optimize scalable data ingestion pipelines from platform sources (RDS, DynamoDB) into Snowflake.
Building event-driven pipelines using Kinesis, Airbyte, or other open-source frameworks to handle high-volume healthcare data.
Implementing and maintaining a staging-layer architecture that supports the broader medallion (staging → intermediate → marts) structure.
Creating configuration-driven, containerized toolsets (Docker/Kubernetes) to ensure data solutions are portable and maintainable.
Ensuring data reliability by building comprehensive monitoring, alerting, and automated testing for all ingestion processes.
Collaborating with analytics engineers to streamline the flow of data for dbt transformation.
Applying software engineering best practices, including modular design and test-driven development, to all data infrastructure.
Refactoring existing ingestion processes to improve performance, cost-efficiency, and scalability.
Mentoring mid-level and junior engineers through code reviews and sharing best practices in data operations.
Requirements
4-6+ years of professional experience in data engineering with a focus on data ingestion and infrastructure.
Proficiency in Python and SQL, with a track record of building production-grade data pipelines.
Strong experience with ingestion tools such as Kinesis, Airbyte, Kafka, or similar frameworks.
Hands-on experience with Snowflake and moving data from operational databases (RDS, DynamoDB) to cloud data warehouses.
Solid understanding of AWS services (S3, Lambda, Step Functions, RDS).
Experience with containerization (Docker) and deploying maintainable systems.
Knowledge of ELT patterns, specifically supporting analytics engineering workflows and dbt.
Experience with CDC (Change Data Capture) and incremental processing methodologies.
Detail-oriented mindset regarding data privacy and compliance (HIPAA experience is a plus).
Strong communication skills, with the ability to collaborate effectively across data science and engineering teams.
Senior Data Engineer developing and maintaining data solutions while ensuring data governance for the organization. Collaborating with teams using Microsoft Azure technologies in a hybrid work environment.
Lead Data Engineer in charge of designing and optimizing cloud - based data platforms for healthcare. Collaborating with teams while implementing scalable, secure solutions.
Senior Fabric Data Engineer designing and optimizing data solutions using Microsoft Fabric for diverse clients. Collaborating with teams and mentoring junior engineers in data platform initiatives.
Senior Data Architect responsible for defining scalable data architectures, supporting analytics, and collaborating with engineering, cloud, and security teams for a client.
Enterprise Data Architect at Togetherwork responsible for AWS - native data strategy and architecture. Design and implement scalable data solutions to support multi - product SaaS portfolio.
Senior Data Engineer managing end - to - end Business Intelligence solutions at a Lyon - based company specializing in e - commerce. Leading technical developments and ensuring operational continuity of reporting systems.
Senior Lead Data Engineer developing and implementing data solutions for Capital One. Collaborating with Agile teams to enhance data - driven technology and cloud solutions.
Senior Manager Data Engineer leading full - stack development and driving transformation at Capital One. Collaborating with Agile teams to enhance cloud - based solutions for customers' financial empowerment.
Distinguished Data Engineer at Capital One defining AI systems' strategic vision and overseeing development. Providing technical leadership and mentoring teams to drive engineering excellence.