Senior Data Engineer optimizing and industrializing data pipelines for AI products, ensuring data quality and performance across the lifecycle. Collaboration with cross - functional teams is crucial for this role.
About the role
- Staff Data Engineer leading scalable data solutions for analytics and reporting at Asurion. Designing data pipelines and ensuring data quality across cloud platforms.
Responsibilities
- Lead the design and development of scalable data pipelines and data products using Databricks, Spark, and Delta Lake
- Develop and optimize data transformations and ELT workflows using SQL and Python
- Design and implement data models and curated datasets to support analytics and reporting use cases
- Ensure data quality, consistency, and reliability through validation, monitoring, and testing practices
- Optimize pipeline performance, scalability, and cost efficiency within AWS environments
- Apply best practices for data partitioning, storage optimization, and query performance tuning
- Collaborate with product, analytics, and business teams to translate requirements into efficient data solutions
- Provide technical leadership and mentorship to engineers, including code reviews and design guidance
- Leverage AI tools for coding, validation, and documentation assistance to enhance productivity and code quality
- Troubleshoot and resolve data pipeline failures, latency issues, and data inconsistencies
- Continuously evaluate and improve data engineering workflows and tooling
Requirements
- 8+ years of experience in data engineering or data pipeline development
- Strong hands-on experience with Databricks, Apache Spark, and Delta Lake
- Advanced proficiency in SQL and Python for building and optimizing data pipelines
- Experience developing robust ETL/ELT pipelines and handling complex data transformations
- Hands-on experience with AWS cloud services (e.g., S3, EMR, Lambda, Glue, Redshift, Kinesis)
- Strong understanding of data modeling and data warehousing concepts
- Experience working with large-scale datasets (TB+ or greater) in distributed environments
- Knowledge of data quality frameworks, validation techniques, and monitoring practices
- Familiarity with CI/CD pipelines and modern development workflows
- Experience using AI-assisted development tools for code generation, validation, or documentation
- Strong problem-solving skills with the ability to debug complex data issues
- Bachelor's degree in Computer Science, Engineering, or related field
Benefits
- Health insurance
- Professional development opportunities
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Senior Data Engineer supporting critical banking insights at Smile. Driving data solutions and reporting in a collaborative environment across Brussels' financial sector.
Data Engineer I responsible for supporting AI solutions in Consumer applications at Bank of America. Involved in Hadoop ecosystem, big data technologies, and ensuring system stability.
Senior Data Architect defining comprehensive data strategy and architecture for AI. Delivering organization’s data vision and ensuring governance and technical oversight of enterprise data architecture.
Data Engineer at Booz Allen utilizing data to impact critical missions like fraud detection and cancer research. Collaborating with analysts and developers on advanced technology solutions.
Technical Product Manager for Data Engineering at Betclic. Owning product roadmaps, driving data infrastructure evolution, and ensuring alignment across engineering teams.
Senior Data Engineer maturing strategic data assets and delivering business analytics in a regulated financial environment. Collaborating with stakeholders to advance business data strategy on cloud platforms.
Lead Architect at Travelers shaping enterprise database and data platform solutions. Collaborating across technology and business units to drive digital transformation and modernization efforts.
Senior Data Engineer at Porto Bank responsible for developing robust data architecture and ensuring data quality across teams. Collaborating with various data solutions for effective data management.
Data Engineer II at Dun & Bradstreet collaborating with teams to enhance data quality standards and practices. Driving best - in - class data management across diverse disciplines.