Data Engineer specializing in Azure Databricks and healthcare data pipelines. Responsible for designing high-throughput ingestion pipelines and ensuring compliance with data standards.
Responsibilities
This role requires a highly technical Data Engineer with expert-level proficiency in Azure Databricks, distributed data pipelines, and large-scale healthcare data processing.
This role focuses on designing and implementing high-throughput ingestion pipelines, transactional lakehouse layers, and secure PHI data flows using Azure-native services and Databricks runtime optimizations.
You will build and operate production-grade data pipelines that meet rigorous requirements for security, lineage, compliance (HIPAA), observability, and operational SLAs, supporting analytics, AI, and clinical insights across the organization.
Requirements
5+ years of experience in modern data engineering roles
Expert-level proficiency in: PySpark and Spark SQL
Databricks (Jobs, Workflows, Repos, Delta Live Tables)
Delta Lake architecture and transactional design patterns
Azure Data Factory or Azure Synapse Pipelines
Cloud-native data security (RBAC, ABAC, privilege boundary enforcement)
Strong experience working with healthcare data formats and standards: FHIR (JSON)
HL7 v2/v3
X12 EDI claims data
Deep understanding of distributed systems, data partitioning strategies, concurrency, and cluster resource tuning
Benefits
Comprehensive health, dental, and vision insurance
Health Savings Account with an employer contribution
Data Architect designing and implementing data architectures supporting analytics and ML for federal clients. Collaborating with teams to translate mission needs into robust data solutions.
IT Data Engineer developing data pipelines and integrations for Scanfil Group's global IT organization. Collaborating across teams to enhance data solutions and reporting capabilities.
Data Engineer developing Azure data solutions at PwC New Zealand. Responsibilities include data quality monitoring, pipeline development, and collaboration with stakeholders in a supportive environment.
Senior Data Engineer designing and implementing the Enterprise Data Platform at Stellix. Focusing on analytics and insights with a growth path to Principal Data Engineer or Data Architect.
R&D Data Engineer at DXC, transforming complex data into digital assets for global analytics and Smart Lab solutions. Collaborating on ELN and LIMS tools for enhanced data management.
Data Engineer role focusing on data pipelines and processing at 42dot, a mobility AI company. Responsibilities include data collection, schema management, and pipeline monitoring.
Senior Data Engineer at mobility AI company designing large - scale data processing pipelines. Leading technical decisions and mentoring junior engineers in data architecture.
Senior Data Engineer at Booz Allen building advanced tech solutions for mission - driven projects. Utilizing data engineering activities, pipelines, and platforms for impactful data insights.
Senior Software Engineer contributing to Workday's AI/MLOps cloud ops platform. Involves data ingestion, computation, and generation of curated data sets with modern technologies.
Data Engineer role at Citi designing and maintaining scalable data solutions. Seeking a skilled professional with extensive data engineering experience and expertise in various technologies.