Azure Data Engineer developing scalable data pipelines and collaborating with data teams for EU institutions. Working remotely from EU locations with a focus on high-quality data management.
Responsibilities
Develop, deploy, and maintain scalable and incremental data pipelines from REST APIs and databases using Python, PySpark, Azure Synapse, Knime, SQL, and ETL tools to ingest, transform, and prepare data.
Process and transform complex JSON and GIS data into structured datasets optimized for analysis and reporting.
Load, organize, and manage data in Azure Data Lake Storage and Microsoft Fabric OneLake, ensuring accessibility, performance, and efficient storage using lakehouse and Delta Lake patterns.
Document ETL processes, metadata definitions, data lineage, and technical specifications to ensure transparency and reusability.
Collaborate with data analysts, BI developers, and business stakeholders to understand data requirements and deliver reliable, well-documented datasets aligned with organizational needs.
Implement data quality checks, logging, monitoring, and automated incremental load mechanisms within data pipelines to support maintainability, observability, and troubleshooting.
Requirements
Bachelor degree - EQF 6
Ability to understand, speak and write English (C1/C2), French (B2) will be an advantage
Excellent knowledge of data engineering tools Azure Synapse Analytics, Microsoft Fabric, PySpark and Python.
Excellent knowledge of working with REST APIs, including ingestion and parsing of JSON and GIS data
Excellent knowledge of Azure data lake storage and Oracle database
Experience designing incremental loads, CDC processes, and automated schema evolution
Ability to implement robust data quality checks, logging, and monitoring in ETL processes
Ability to document ETL workflows, metadata, and technical specifications clearly and consistently
Familiarity with DevOps and version control best practices.
Experience with CI/CD pipelines
Experience working in an Agile and Scrum framework
Analysis and problem solving skills
Ability to participate in technical meetings and good communication skills
Ability to participate in multilingual meetings
Ability to work in multi-cultural environment, on multiple large projects; Excellent Team Player
Specific expertise (mandatory) at least 5 years of excellent knowledge in Azure Data Lake Storage, Microsoft Fabric OneLake, and Oracle databases
at least 5 years of excellent expertise in developing data pipelines from REST APIs and on integration (such as Azure Synapse, PySpark, Microsoft Fabric, Python, SQL, KNIME)
at least 5 years of excellent expertise in processing JSON and GIS data
Certificates (mandatory) Microsoft Azure Data Engineer Associate
Following certifications are a plus: Microsoft Certified: Azure Solutions Architect Expert
Microsoft Certified: Azure Developer Associate
Microsoft Certified: Azure Database Administrator Associate
Associate Delivery Manager leading CDW and MDM solutions at Beghou Consulting, improving commercialization strategies for life sciences clients. Ensuring high - quality delivery of data solutions and mentoring junior staff.
Data Engineer/BI Analyst developing ETL processes and analyzing datasets at highly growing software solutions company in Athens. Focus on Azure Databricks and BI tools for data - driven insights.
Senior Data Engineer for iKnowHow S.A. designing scalable data pipelines and mentoring a team. Leading technical decisions and ensuring data quality in the software and robotics sector.
Vice President of Data Engineering leading the Reference Data Master team in transforming data strategies. Driving cloud - native solutions and modernizing party reference data management practices.
Cloud Data Engineer at BCBSNE using advanced technologies to enhance healthcare solutions. Collaborating with stakeholders and delivering scalable data pipelines with a focus on automation.
Data Engineer participating in a 1 - month intensive academy focused on Snowflake and Databricks. Opportunity to work on enterprise - level projects after training in Monterrey, Mexico.
Data Engineering professional enhancing skills through a specialized academy on Snowflake and Databricks. Program leads to permanent contract with a big client post - completion.
Senior Data Engineer responsible for data platform management in a Stockholm - based public transport company. Working with Azure and AWS tech in a hybrid role.
Data Migration Engineer creating and maintaining valuable datasets supporting health service technology team. Collaborate with cross - functional teams to ensure data integrity and project success.
Senior Data Engineer leading the Data Engineering program for generative AI technology at Docuvera. Design and build scalable data foundations while ensuring data quality and security compliance.