Snowflake Data Engineer optimizing data pipelines using Snowflake for a global life science company. Collaborate with cross - functional teams for data solutions and performance improvements in Madrid.
About the role
- Senior Data Engineer focusing on ETL developments in Azure Databricks at OAG, a travel data platform. Collaborating with multi-skilled teams to modernize Flight Status data processing.
Responsibilities
- Design, build, and optimize scalable ETL and Structured Streaming pipelines in Azure Databricks for real-time and batch ingestion of Flight Status data
- Design and implement data ingestion and processing pipelines that consolidate heterogeneous data sources, including APIs, event streams, and file-based feeds, into the OAG lakehouse (Azure Databricks + Delta Lake), ensuring data consistency, reliability, and scalability
- Implement and monitor data quality using automated validation, alerting, and observability practices
- Develop and maintain orchestration workflows in Apache Airflow, coordinating ingestion and transformation processes across multiple data flows
- Build reusable frameworks for schema evolution, error handling, deduplication, and auditing
- Collaborate with data platform, analytics, and product teams to define SLAs, data contracts, and performance targets
- Optimize Spark and Delta Lake performance for scalability, latency, and cost efficiency
- Implement CI/CD pipelines and automation for data workflows using Azure DevOps or equivalent tools
- Mentor engineers, review code, contribute to platform design discussions and planning, and help grow data engineering competencies in the team and across OAG
Requirements
- Proven track record in data engineering, with a strong focus on ETL development and streaming data architectures
- Experience with Azure Databricks, Apache Spark (Structured Streaming), and Delta Lake
- Proficiency in Python (PySpark) and SQL, with experience transforming large-scale, complex datasets
- Hands-on experience in data orchestration and workflow automation (e.g., Apache Airflow or similar)
- Experience working in a cloud data environment (preferably Azure) across storage, compute, and pipeline services
- Familiarity with streaming or messaging technologies (e.g., Kafka, Event Hubs)
- Strong understanding of data quality, validation, and observability practices
- Ability to deliver production-grade solutions with a results-oriented and ownership-driven mindset
- Experience implementing CI/CD and version-control practices using Azure DevOps, GitHub Actions, or similar tools
- Excellent analytical, communication, and collaboration skills
- Strong understanding of modern data engineering patterns and ability to design scalable, modular, and reliable data systems
Benefits
- Company-provided free lunch every day
- Private health insurance
- Company bonus scheme
- Voluntary participation in a company-supported retirement scheme
- A generous annual leave policy, growing with each year of service, and a day off during your birthday month
- Participation in team-building activities, team workshops, and group learning sessions
Job title
Job type
Experience level
Salary
Degree requirement
Tech skills
Location requirements
Data Engineer designing and implementing big data solutions at DATAIS. Collaborating with clients to deliver actionable business insights and innovative data products in a hybrid environment.
SAP Data Engineer supporting MERKUR GROUP in becoming a data - driven company. Responsible for data integration, ETL processes, and collaboration with various departments.
Big Data Engineer designing and managing data applications on Google Cloud. Join Vodafone’s global tech team to optimize data ingestion and processing for machine learning.
Data Engineer building and maintaining data pipelines for Farfetch’s data platform. Collaborating with the Data team to improve data reliability and architecture in Porto.
Senior Data Engineer at Razer leading initiatives in data engineering and AI infrastructure. Collaborating across teams to develop robust data solutions and enhancing AI/ML projects.
Data Engineering Intern working with data as Jua builds AI for climate and geospatial datasets. Contributing to the integration and validation of new datasets with experienced mentors.
Data Engineer supporting a fintech company in building and maintaining data pipelines. Collaborating with tech teams and enhancing data processing in a high - volume environment.
Senior Data Engineer developing and optimizing data pipelines for Scene+’s cloud - native platform in Toronto. Collaborating across teams to enhance data governance and analytics capabilities.
Staff Engineer developing innovative data solutions for dentsu's B2B marketing vision. Collaborating using cutting - edge cloud technologies and mentoring engineers in their careers.