Data Engineer specializing in Azure Databricks and healthcare data pipelines. Responsible for designing high-throughput ingestion pipelines and ensuring compliance with data standards.
Responsibilities
This role requires a highly technical Data Engineer with expert-level proficiency in Azure Databricks, distributed data pipelines, and large-scale healthcare data processing.
This role focuses on designing and implementing high-throughput ingestion pipelines, transactional lakehouse layers, and secure PHI data flows using Azure-native services and Databricks runtime optimizations.
You will build and operate production-grade data pipelines that meet rigorous requirements for security, lineage, compliance (HIPAA), observability, and operational SLAs, supporting analytics, AI, and clinical insights across the organization.
Requirements
5+ years of experience in modern data engineering roles
Expert-level proficiency in: PySpark and Spark SQL
Databricks (Jobs, Workflows, Repos, Delta Live Tables)
Delta Lake architecture and transactional design patterns
Azure Data Factory or Azure Synapse Pipelines
Cloud-native data security (RBAC, ABAC, privilege boundary enforcement)
Strong experience working with healthcare data formats and standards: FHIR (JSON)
HL7 v2/v3
X12 EDI claims data
Deep understanding of distributed systems, data partitioning strategies, concurrency, and cluster resource tuning
Benefits
Comprehensive health, dental, and vision insurance
Health Savings Account with an employer contribution
Lead Data Platform architecting data infrastructure for Ankorstore’s wholesale operations. Managing data engineers and ensuring reliable data flows for business intelligence and insights.
Lead Cloud Data Engineer developing applications on Azure data platform initiative at LSEG. Collaborating with data teams to implement scalable and efficient data solutions.
Senior Enterprise Data Architect shaping data flow across Lundbeck by unifying enterprise data platforms. Leading architectural design and governance for scalable and sustainable data solutions.
Data Engineer supporting the development and implementation of enterprise - wide data governance practices at a climate technologies company focused on sustainability. Collaborating cross - functionally to enhance data quality and compliance processes.
Data Engineer III at Hanger, Inc. designing and maintaining data solutions using Microsoft Azure. Collaborating with stakeholders and optimizing ETL processes for enterprise analytics.
Data Engineer building scalable data pipelines for analytics at UOL EdTech. Collaborating with data teams and supporting data - driven culture in education technology.
Data Engineering Intern helping build and maintain data pipelines using Python and SQL. Assisting the Data and Analytics team on various data processes and projects.
Senior Data Engineer designing and maintaining data processing pipelines for analytics and machine learning in a fast - paced startup. Collaborating with cross - functional teams to ensure data accuracy and security.
Data Engineer developing data pipelines and ETL processes for Stefanini's data architecture modernization. Involves data migration from AS400 to Microsoft Fabric Lakehouse.
Senior Data Engineer responsible for overseeing data ingestion and delivery at Kpler. Leading engineering best practices and collaborating with teams on client - facing data solutions.