Data Engineer building modern Data Lake architecture on AWS and implementing scalable ETL/ELT pipelines. Collaborating across teams for analytics and reporting on gaming platforms.
About the role
- Senior Data Engineer responsible for building and optimizing ETL pipelines for a leading travel data platform. Collaborating with software and data engineers to modernize data ingestion and processing.
Responsibilities
- Design, build, and optimize scalable ETL and Structured Streaming pipelines in Azure Databricks for real-time and batch ingestion of Flight Status data
- Design and implement data ingestion and processing pipelines that consolidate heterogeneous data sources, including APIs, event streams, and file-based feeds, into the OAG lakehouse (Azure Databricks + Delta Lake), ensuring data consistency, reliability, and scalability
- Implement and monitor data quality using automated validation, alerting, and observability practices
- Develop and maintain orchestration workflows in Apache Airflow, coordinating ingestion and transformation processes across multiple data flows
- Build reusable frameworks for schema evolution, error handling, deduplication, and auditing
- Collaborate with data platform, analytics, and product teams to define SLAs, data contracts, and performance targets
- Optimize Spark and Delta Lake performance for scalability, latency, and cost efficiency
- Implement CI/CD pipelines and automation for data workflows using Azure DevOps or equivalent tools
- Mentor engineers, review code, contribute to platform design discussions and planning, and help grow data engineering competencies in the team and across OAG
Requirements
- Proven track record in data engineering with a strong focus on ETL development and streaming data architectures
- Experience with Azure Databricks, Apache Spark (Structured Streaming), and Delta Lake
- Proficiency in Python (PySpark) and SQL, with experience transforming large-scale, complex datasets
- Hands-on experience in data orchestration and workflow automation (e.g., Apache Airflow or similar)
- Experience working in a cloud data environment (preferably Azure) across storage, compute, and pipeline services
- Familiarity with streaming or messaging technologies (e.g., Kafka, Event Hubs)
- Strong understanding of data quality, validation, and observability practices
- Ability to deliver production-grade solutions with a results-oriented and ownership-driven mindset
- Experience implementing CI/CD and version-control practices using Azure DevOps, GitHub Actions, or similar tools
- Excellent analytical, communication, and collaboration skills
- Strong understanding of modern data engineering patterns and ability to design scalable, modular, and reliable data systems
Benefits
- Company-provided free lunch every day
- Private health insurance
- Company bonus scheme
- Voluntary participation in a company-supported retirement scheme
- Generous annual leave policy that grows with each year of service
- Day off during your birthday month
- Participation in team-building activities
- Participation in team workshops
- Participation in group learning sessions
Job title
Job type
Experience level
Salary
Degree requirement
Tech skills
Location requirements
Chief Data Engineer leading Scania’s Commercial Data Engineering team for growing sustainable transport solutions. Focused on data products and pipelines for BI, analytics, and AI.
Data Engineer designing and building scalable ETL/ELT pipelines for enterprise - grade analytics solutions. Collaborating with product teams to deliver high - quality, secure, and discoverable data.
Entry - Level Data Engineer at GM, focusing on building large scale data platforms in cloud environments. Collaborating with data engineers and scientists while migrating systems to cloud solutions.
Data Engineer responsible for data integrations with AWS technology stack for Adobe's Digital Experience. Collaborating with multiple teams to conceptualize solutions and improve data ecosystem.
People Data Architect designing and managing people data analytics for Gen, delivering actionable insights for HR. Collaborating across teams to enhance data - driven decision - making.
Data Engineer role focused on shaping future connectivity for customers at Vodafone. Involves solving complex challenges in a diverse and inclusive environment.
VP, Senior Data Engineer responsible for designing and developing cloud data solutions for insider risk in Information Security at SMBC. Collaborating with multiple teams to enhance cybersecurity data platform.
Data Engineer responsible for architecting, developing, and maintaining Allegiant’s enterprise data infrastructure. Overseeing transition to cloud hosted data warehouse and developing next - generation data tools.
Senior Data Engineer developing Azure - based data solutions for clients in the Data & AI department. Collaborating with architects and consultants to enhance automated decision making.