Data Engineer at Keyrus responsible for building and optimizing data pipelines for major projects. Contributing to data solutions and ensuring data quality in a growing team.
About the role
- Data Engineer designing and building data systems and pipelines. Collaborating with data scientists and analysts to meet data requirements and resolve issues.
Responsibilities
- Design and implement data pipelines.
- Optimize data processing and storage.
- Ensure data solutions meet performance standards.
- Provide technical support.
- Collaborate with stakeholders.
Requirements
- 8+ years in data engineering/ETL roles, with at least 4+ years in Azure cloud ETL leadership.
- Azure Data Sources: Azure Data Lake Storage (ADLS), Blob Storage, Azure SQL Database, Synapse Analytics.
- Tools: Azure Data Factory (ADF) for orchestration, Databricks connectors.
- Apache Spark: Strong knowledge of Spark (PySpark, Spark SQL) for distributed processing.
- Data Cleaning & Normalization: Handling nulls, duplicates, schema evolution.
- Performance Optimization: Partitioning, caching, broadcast joins.
- Delta Lake: Implementing ACID transactions, time travel, and schema enforcement.
- Azure Data Factory (ADF): Building pipelines to orchestrate Databricks notebooks.
- Azure Key Vault: Secure credential management.
- Azure Monitor & Logging: For ETL job monitoring and alerting.
- Networking & Security: VNET integration, private endpoints.
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Data Architect designing and implementing scalable data architectures for Keyrus in Bordeaux. Leading client transitions and contributing to the tech ecosystem with innovative data solutions.
Lead Data Engineer responsible for leading data engineering projects within the AI & Data Engineering team in Pune, India.
Data Engineer utilizing Databricks and AWS for scalable data solutions. Collaborating on analytics with business teams at Travelers.
Manager leading coordination between teams for data engineering at DPR Construction. Supporting core markets and account management through data analytics and technical delivery.
Program Manager leading enterprise - wide data migration efforts for Boeing's transition to modern data platforms. Overseeing complex processes to ensure secure and effective migrations across multiple systems.
Senior Data Engineer responsible for developing data products for Disney's immersive digital experiences. Collaborating with teams to ensure data quality and operational efficiency in a fast - paced environment.
Senior Data Engineer crafting and developing data products for analytical insights at Zendesk. Collaborating in an Agile environment with a focus on data warehousing and process optimization.
Senior Data Engineer designing and building data warehouse solutions with Snowflake for a fintech company. Collaborating with cross - functional teams to facilitate data insights and analytics.
Data Engineer developing and maintaining data pipelines and applications at EvidenceCare. Collaborating across teams to generate actionable insights from healthcare data for better decision - making.