Lead Data Engineer overseeing engineers and advancing the data platform at American Family Insurance. Creating tools and infrastructure to empower teams across the company.
About the role
- Calix Cloud Data Engineer involved in architecture design, data ingestion, and analytics for service provider transformation. Collaborating with cross-functional teams in a flexible hybrid work model.
Responsibilities
- Work closely with Cloud product owners to understand, analyze product requirements and provide feedback
- Develop conceptual, logical, physical models and meta data solutions
- Design and manage an array of data design deliverables including data models, data diagrams, data flows and corresponding data dictionary documentations
- Determine database structural requirements by analyzing client operations, applications, and data from existing systems
- Technical leadership of software design in meeting requirements of service stability, reliability, scalability, and security
- Guiding technical discussions within engineer group and making technical recommendations
- Design review and code review with peer engineers
- Guiding testing architecture for large scale data ingestion and transformations
- Customer facing engineering role in debugging and resolving field issues
Requirements
- 10-12 years of software engineering experience delivering quality products
- 10+ years of development experience performing Data modeling, master data management and building ETL/data pipeline implementations
- Proficiency in both Google Cloud Platform (GCP) services (BigQuery, Dataflow, Dataproc, PubSub/Kafka, Cloud Storage) and AWS services (Redshift, Glue, Kinesis, S3)
- Proven experience in designing, building, and maintaining scalable data pipelines across GCP and AWS
- Knowledge of big data processing frameworks such as Apache Spark, Flink and Beam
- Proficient in using dbt/Dataform for data transformation and modeling within the data warehouse environment
- Strong knowledge of SQL and at least one programming language (Python, Java, or Scala)
- Understanding of Docker and Kubernetes for deploying data applications
- Knowledge of data catalog tools (e.g., DataHub, Collibra, Alation) to ingest and maintain metadata
- Strong analytical and troubleshooting skills, particularly in complex data scenarios
- Ability to work effectively in a team environment and engage with cross-functional teams
- Proficient in conveying complex technical concepts to stakeholders
- Knowledge of data governance, security best practices, and compliance regulations in both GCP and AWS environments
- Bachelor’s degree in Computer Science, Information Technology, or a related field
- Relevant certifications (e.g., Google Cloud Professional Data Engineer, AWS Certified Data Analytics – Specialty)
Benefits
- Flexible hybrid work model - work from Bangalore office for 20 days in a quarter
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Working Student contributing to data engineering projects for AI/ML workflows while gaining hands - on experience with data pipelines and tools.
Data Architect designing end - to - end Snowflake data solutions and collaborating with technical stakeholders at Emerson. Supporting the realization of Data and Digitalization Strategy.
Manager of Data Engineering leading data assets and infrastructure initiatives at CLA. Collaborating with teams to enforce data quality standards and drive integration efforts.
Data Engineer building modern Data Lake architecture on AWS and implementing scalable ETL/ELT pipelines. Collaborating across teams for analytics and reporting on gaming platforms.
Chief Data Engineer leading Scania’s Commercial Data Engineering team for growing sustainable transport solutions. Focused on data products and pipelines for BI, analytics, and AI.
Entry - Level Data Engineer at GM, focusing on building large scale data platforms in cloud environments. Collaborating with data engineers and scientists while migrating systems to cloud solutions.
Data Engineer designing and building scalable ETL/ELT pipelines for enterprise - grade analytics solutions. Collaborating with product teams to deliver high - quality, secure, and discoverable data.
Data Engineer responsible for data integrations with AWS technology stack for Adobe's Digital Experience. Collaborating with multiple teams to conceptualize solutions and improve data ecosystem.
People Data Architect designing and managing people data analytics for Gen, delivering actionable insights for HR. Collaborating across teams to enhance data - driven decision - making.