Lead Data Engineer overseeing engineers and advancing the data platform at American Family Insurance. Creating tools and infrastructure to empower teams across the company.
About the role
- Data Engineer leading data foundation architecture and optimization for a Kenyan startup. Constructing data pipelines that fuel machine learning models and internal analytics.
Responsibilities
- Architect and sustain scalable ETL workflows, guaranteeing consistency and accuracy across diverse data origins.
- Refine and optimize data models and database structures specifically tailored for reporting and analytics.
- Enforce industry best practices regarding data warehousing and storage methodologies.
- Fine-tune data systems to handle the demands of both real-time streams and batch processing.
- Manage the cloud data environment, utilizing platforms such as AWS, Azure, or GCP.
- Coordinate with software engineers to embed data solutions directly into our product suite.
- Design robust processes for ingesting both structured and unstructured datasets.
- Script automated quality checks and deploy monitoring instrumentation to instantly detect data anomalies.
- Build APIs and services that ensure seamless data interoperability between systems.
- Continuously monitor pipeline health, troubleshooting bottlenecks to maintain an uninterrupted data flow.
- Embed data governance and security protocols that meet rigorous industry standards.
- Collaborate with data scientists and analysts to maximize the usability and accessibility of our data assets.
- Maintain comprehensive documentation covering schemas, transformations, and pipeline architecture.
- Keep a pulse on emerging trends in cloud tech, analytics, and data engineering to drive continuous improvement.
Requirements
- A minimum of 3 years of professional experience in Data Engineering or a similar technical role.
- Bachelor’s or Master’s degree in Engineering, Computer Science, Data Science, or a relevant discipline.
- Expert-level command of SQL and management systems like PostgreSQL or MySQL.
- Hands-on proficiency with pipeline tools such as Luigi, DBT, or Apache Airflow.
- Practical experience with heavy-lifting technologies like Hadoop, Spark, or Kafka.
- Proven skills with cloud data stacks, specifically Google BigQuery, AWS Redshift, or Azure Data Factory.
- Strong programming logic in Java, Scala, or Python for data processing tasks.
- Familiarity with data integration frameworks and API utilization.
- Understanding of security best practices and compliance frameworks.
Benefits
- Flexible work arrangements
- Professional development opportunities
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Working Student contributing to data engineering projects for AI/ML workflows while gaining hands - on experience with data pipelines and tools.
Data Architect designing end - to - end Snowflake data solutions and collaborating with technical stakeholders at Emerson. Supporting the realization of Data and Digitalization Strategy.
Manager of Data Engineering leading data assets and infrastructure initiatives at CLA. Collaborating with teams to enforce data quality standards and drive integration efforts.
Data Engineer building modern Data Lake architecture on AWS and implementing scalable ETL/ELT pipelines. Collaborating across teams for analytics and reporting on gaming platforms.
Chief Data Engineer leading Scania’s Commercial Data Engineering team for growing sustainable transport solutions. Focused on data products and pipelines for BI, analytics, and AI.
Entry - Level Data Engineer at GM, focusing on building large scale data platforms in cloud environments. Collaborating with data engineers and scientists while migrating systems to cloud solutions.
Data Engineer designing and building scalable ETL/ELT pipelines for enterprise - grade analytics solutions. Collaborating with product teams to deliver high - quality, secure, and discoverable data.
Data Engineer responsible for data integrations with AWS technology stack for Adobe's Digital Experience. Collaborating with multiple teams to conceptualize solutions and improve data ecosystem.
People Data Architect designing and managing people data analytics for Gen, delivering actionable insights for HR. Collaborating across teams to enhance data - driven decision - making.