Data Engineer with expertise in Databricks, SQL, and Python for scalable data solutions. Focused on ETL/ELT pipeline development, auditability, data quality, and automated testing.
Responsibilities
Lead the design, development, and optimization of ETL/ELT pipelines using Databricks, Python, Spark, and Delta Lake.
Architect scalable data solutions using Medallion architecture (Bronze, Silver, Gold layers).
Design and implement data models and transformations using SQL and Python.
Build and maintain audit frameworks to ensure traceability, compliance, and data lineage.
Develop data quality monitoring and automated testing frameworks for pipeline reliability.
Perform data analysis to support operational data requests and user queries.
Collaborate with clinical data teams to analyse IRT/RTSM datasets.
Create and maintain dashboards and reports using BI tools (e.g., Superset Power BI, Tableau, Qlik, or similar).
Help manage CICD and automated code branching/deployment.
Ensure compliance with GxP, CDISC, and other regulatory standards.
Mentor junior engineers and promote engineering best practices.
Requirements
8–10 years of experience in data engineering, with leadership or team lead responsibilities.
Strong hands-on experience with Databricks, Apache Spark, and Delta Lake.
Advanced proficiency in SQL and Python for data transformation and automation.
Experience with ETL/ELT orchestration, job optimization, and performance tuning.
Proven experience designing and implementing audit, data quality, and testing frameworks.
Hands-on experience with IRT/RTSM clinical trial data systems.
Strong data analysis skills and ability to interpret complex datasets.
Experience with BI/reporting tools such as Power BI, Tableau, or Qlik.
Knowledge of clinical data standards (e.g., CDISC, SDTM, ADaM).
Experience with cloud platforms (Azure, AWS, or GCP) and CI/CD pipelines.
Program Manager leading development of AI - driven data platform to enhance revenue intelligence across global business functions. Collaborating across teams and regions in a hybrid work environment.
Project Manager overseeing payroll system implementation with global outsourcing partner. Leading cross - functional teams and stakeholder engagement for project success.
Data Engineer 3 optimizing Market Place processes for Walmart Global Tech's Chennai team. Developing data pipelines and ensuring efficient utilization of Market Place systems.
Data Engineer at CBTW handling data pipelines and ETL processes using SAS. Collaborating with business stakeholders and ensuring data governance within SAS environments.
Data Engineer I at Catalyst Brands developing and optimizing data pipelines for cross - functional teams. Designing next generation data platform architecture to meet increasing data demands in a retail environment.
Data Engineer at Grupo Iter responsible for data pipelines and architecture in Azure. Collaborating on data governance and integrating analytics with Power BI.
Full Stack Data Architect for Concurrency designing Azure data - intensive applications. Leading complex data architecture initiatives and mentoring engineering teams in a high - performance environment.
AHEAD builds digital business platforms; seeking a Data Engineer in a development program. Join us to grow into a technical leader emphasizing skills across various practices.