Senior Data Engineer at Assembly working on data integration, transformation, and analytics collaboration. Handling cloud services and data quality across data projects with cross - functional teams.
About the role
- Senior Software Engineer developing scalable data processing solutions for data engineering team. Implementing data models and building ETL pipelines with Python, Spark, and SQL.
Responsibilities
- Design and develop scalable data processing solutions using Spark and PySpark
- Implement robust data models and optimize SQL queries
- Collaborate with data analysts and stakeholders
- Build and maintain ETL pipelines leveraging Python, Hive, and SQL
- Conduct thorough code reviews, performance tuning, and debugging
- Monitor, troubleshoot, and resolve issues in production data workflows
- Document technical processes, data models, and workflow architectures
- Stay updated with industry trends in big data technologies
Requirements
- 2 - 4 years of experience in software development
- Advanced proficiency in Spark and PySpark
- Strong knowledge of SQL (basic and advanced)
- Expertise in Python programming for data processing
- Experience with Hive for data warehousing solutions
- Solid understanding of data modelling fundamentals
- Ability to design and optimize ETL pipelines
- Hands-on experience with large-scale data processing
- Proficient in performance tuning of SQL and Spark jobs
- Familiarity with distributed computing concepts
- Competence in debugging and troubleshooting data workflows
- Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field
- Relevant certifications in big data technologies or data engineering are advantageous
Job title
Job type
Experience level
Salary
Degree requirement
Tech skills
Location requirements
Senior Data Engineer developing scalable data pipelines and collaborating with cross - functional teams at Technis. Technical guidance in a hybrid work environment based in Lausanne, Switzerland.
Data Engineer designing and maintaining data pipelines for CIEE, a philanthropic institution supporting youth development. Collaborating with Data Analysts for data quality and reliability.
Senior Data Engineer responsible for designing and implementing data solutions at Harambee. Collaborating with various stakeholders to enhance technology supporting work - seekers' journeys.
Senior Manager Data Engineer at Squarcle delivering technical leadership in data engineering and compliance with business objectives. Leading teams to optimize and develop data platforms for clients.
Senior Consultant Data Engineer in a consultancy firm focusing on data engineering and platform development. Collaborating with diverse teams to deliver high - quality data solutions.
Data Engineer at Mobileye building robust data pipelines for data infrastructure. Collaborating with teams to deliver high - quality data solutions for dynamic environments.
Cloud Data Engineer at Shift, focusing on building and operating data pipelines on Azure for Australian SMEs. Collaborating across teams to enhance data integration and quality.
AI Systems & Data Engineer at HyperFi designing Databricks pipelines and managing AI systems. Collaborating on data engineering tasks and optimizing workflows in a flexible tech environment.
Engenheiro de Dados especializado em PowerBI e Lakehouse no Campos Thomaz Advogados. Foco em preparar dados para dashboards e estruturação de ambientes com Microsoft.