Lead Data Engineer overseeing engineers and advancing the data platform at American Family Insurance. Creating tools and infrastructure to empower teams across the company.
About the role
- Manager of Data Engineering developing and optimizing data platforms for enterprise analytics. Leading a team and collaborating with business stakeholders for high-quality data solutions.
Responsibilities
- Lead, mentor, and grow a team of Data Engineers, including recruitment, onboarding, performance management, and continuous skill development.
- Oversee the design, development, and optimization of data pipelines, architectures, and data sets for batch and streaming use cases.
- Collaborate with business units and analysts to gather requirements and translate them into scalable data solutions.
- Drive best practices in data modeling (conceptual, logical, and physical) to support analytics and reporting.
- Establish and maintain metadata standards, data cataloging, and lineage to improve data discovery and governance.
- Ensure timely delivery of projects by managing priorities, scope, and resources in an Agile framework.
- Partner with vendors and managed service providers as required to support delivery and operations.
- Define, track, and report on team KPIs such as data pipeline uptime, data freshness, and adoption metrics.
- Contribute to capacity planning, cloud cost optimization, and vendor performance review.
- Promote standardization and reusability of data models, pipelines, and integration approaches.
- Serve as a subject matter expert, communicating technical solutions to both technical and non-technical audiences.
- Maintain compliance with organizational standards for security, architecture, and development practices.
- Encourage experimentation with automation, AI/ML integration, and modern data engineering practices.
- Provide input into enterprise architecture and align data initiatives.
Requirements
- Bachelor’s degree in computer science, Engineering, or related technical discipline.
- 5+ years of experience in data engineering, including 2–3 years in a leadership or managerial role.
- Proficiency in Python, SQL, and Spark (batch and streaming).
- Experience developing and maintaining data lakes and cloud-based platforms (preferably Azure/Databricks).
- Strong background in data architecture, including data modeling (conceptual, logical, dimensional) and relational databases.
- Familiarity with data mesh or federated architectures for distributed data ownership.
- Demonstrated ability to quantify business impact of data initiatives (e.g., cost savings, reduced downtime, increased adoption).
- Knowledge of data governance, data quality, and master data management practices.
- Proven ability to mentor and develop technical talent.
- Hands-on experience with Agile/Scrum methodologies.
- Excellent communication, problem-solving, and collaboration skills with a strong delivery-focused mindset.
- Knowledge of orchestration tools (SSIS, ADF, DBT or equivalent).
- Familiarity with DevOps/DataOps practices—CI/CD, version control, automated testing for data pipelines.
- Exposure to streaming technologies like Kafka/Event Hubs beyond Spark Streaming.
- Understanding of security and compliance frameworks (PII, GDPR, SOC2, HIPAA depending on industry).
Benefits
- Job Benefits
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Working Student contributing to data engineering projects for AI/ML workflows while gaining hands - on experience with data pipelines and tools.
Data Architect designing end - to - end Snowflake data solutions and collaborating with technical stakeholders at Emerson. Supporting the realization of Data and Digitalization Strategy.
Manager of Data Engineering leading data assets and infrastructure initiatives at CLA. Collaborating with teams to enforce data quality standards and drive integration efforts.
Data Engineer building modern Data Lake architecture on AWS and implementing scalable ETL/ELT pipelines. Collaborating across teams for analytics and reporting on gaming platforms.
Chief Data Engineer leading Scania’s Commercial Data Engineering team for growing sustainable transport solutions. Focused on data products and pipelines for BI, analytics, and AI.
Entry - Level Data Engineer at GM, focusing on building large scale data platforms in cloud environments. Collaborating with data engineers and scientists while migrating systems to cloud solutions.
Data Engineer designing and building scalable ETL/ELT pipelines for enterprise - grade analytics solutions. Collaborating with product teams to deliver high - quality, secure, and discoverable data.
Data Engineer responsible for data integrations with AWS technology stack for Adobe's Digital Experience. Collaborating with multiple teams to conceptualize solutions and improve data ecosystem.
People Data Architect designing and managing people data analytics for Gen, delivering actionable insights for HR. Collaborating across teams to enhance data - driven decision - making.