Contract Data Engineer at Envitia tasked with building scalable data solutions for public sector clients. Design and maintain secure systems using cutting - edge data engineering tools and methodologies.
About the role
- Associate Manager, D&T, Data Engineering responsible for building data pipelines on Databricks using PySpark and Structured Streaming. Collaborating with Data Scientists to enhance data quality and model deployment.
Responsibilities
- Design and maintain scalable batch and real-time data pipelines on Databricks using PySpark, Structured Streaming, DBT, and Delta Live Tables.
- Manage cloud infrastructure with Terraform on AWS and deploy workflows using Databricks Asset Bundles with CI/CD best practices.
- Optimize Spark performance and costs while implementing data governance through Unity Catalog.
- Collaborate with Data Scientists to produce ML models using MLflow and ensure data quality through automated testing and robust deployment pipelines.
- Develop and manage robust transformation layers using DBT to deliver clean, tested, and well-documented data models.
- Enforce data governance and quality standards using Unity Catalog (row-level security) and support MLOps initiatives through MLflow-based model tracking and production deployment.
Requirements
- 10+ years of experience in Data Engineering, with strong expertise in the Databricks platform.
- Deep hands-on experience with Unity Catalog, Delta Live Tables (DLT), and Databricks Asset Bundles (DAB).
- Strong proficiency in AWS services (S3, Lambda, Kinesis) and Infrastructure as Code using Terraform.
- Expert-level programming skills in Python (PySpark) and SQL for scalable data processing.
- Proven experience in data transformation and modeling using DBT.
- Solid understanding of data modeling concepts, including Star Schema, Dimensional Modeling, and Data Vault, with the ability to contribute to architectural design decisions.
Benefits
- A team of diverse employees who aren’t afraid to think outside of the box.
- A truly global and collaborative team that cares about the experience of our employees.
- The encouragement you need to develop and achieve personal growth.
- A role that is crucial on projects and allows you to build your brand.
- A caring and supportive environment where you’re empowered to grow and share your ideas.
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
No Education Requirement
Tech skills
Location requirements
Senior Data Engineer shaping and delivering advanced data - driven solutions for clients. Leading delivery of tailored solutions in a team focused on data empowerment.
Data Engineer specializing in Snowflake solutions at LUZA Group in Portugal. Responsible for designing data models and ensuring data quality across platforms.
Senior PDM Data Architect at Ford focusing on 3D model creation for digital marketing. Responsible for managing complex CAD data and collaboration with creative studios.
Specialist PDM Data Architect managing the creation and maintenance of 3D models for digital marketing at Ford. Bridging technical gaps between Engineering and Marketing for digital asset accuracy.
Senior Consultant in SAP Data Migration at Scheer Group advising clients on data migration strategies. Leading migration projects and developing innovative migration solutions in a collaborative environment.
Junior/Medior SAP/SAC Data Engineer at BAM working on SAP data transformation for analytics and reporting. Collaborating within a DevOps team to optimize data pipelines and reporting structures.
Data Engineer working remotely with cross - functional teams to solve data challenges through coding. Analyzing large datasets and ensuring collaboration for effective implementation of insights.
Data Engineer at Capital One focusing on innovative technology solutions and mentoring talent. Collaborating on enterprise - wide initiatives and implementing cutting - edge practices.
Azure Data Engineer designing and implementing robust data solutions leveraging Microsoft Azure technologies. Collaborating with stakeholders to create efficient data pipelines and ensure data integrity.