Senior Data Engineer on CNN's AI Enablement & Machine Learning team optimizing ML and AI experiences. Collaborating with engineers to enhance data pipelines and integrate features into platforms.
About the role
- GCP Data Engineer designing and developing data processing modules for Ki, an algorithmic insurance carrier. Working closely with multiple teams to optimize data pipelines and reporting.
Responsibilities
- Work with both the business teams (finance and actuary initially), data scientists and engineers to design, build, optimise and maintain production grade data pipelines and reporting from an internal Data warehouse solution, based on GCP/Big Query
- Work with finance, actuaries, data scientists and engineers to understand how we can make best use of new internal and external data sources
- Work with our delivery partners at EY/IBM to ensure robustness of Design and engineering of the data model/ MI and reporting which can support our ambitions for growth and scale
- BAU ownership of data models, reporting and integrations/pipelines
- Create frameworks, infrastructure and systems to manage and govern Ki’s data asset
- Produce detailed documentation to allow ongoing BAU support and maintenance of data structures, schema, reporting etc.
- Work with the broader Engineering community to develop our data and MLOps capability infrastructure
- Ensure data quality, governance, and compliance with internal and external standards.
- Monitor and troubleshoot data pipeline issues, ensuring reliability and accuracy.
Requirements
- Experience designing data models and developing industrialised data pipelines
- Strong knowledge of database and data lake systems
- Hands on experience in Big Query, dbt, GCP cloud storage
- Proficient in Python, SQL and Terraform
- Knowledge of Cloud SQL, Airbyte, Dagster
- Comfortable with shell scripting with Bash or similar
- Experience provisioning new infrastructure in a leading cloud provider, preferably GCP
- Proficient with Tableau Cloud for data visualization and reporting
- Experience creating DataOps pipelines
- Comfortable working in an Agile environment, actively participating in approaches such as Scrum or Kanban
- Desirable Skills
- Experience of streaming data systems and frameworks would be a plus
- Experience working in regulated industry, especially financial services would be a plus
- Experience creating MLOps pipelines is a plus
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Data Engineer at Sand Cherry Associates with expertise in ETL, Python, and SQL. Responsible for designing data structures and maintaining accuracy for client projects.
Data Engineer responsible for creating and maintaining ETL pipelines at Sand Cherry Associates. Requires strong expertise in Redshift SQL, Python, and DBT with hybrid work structure.
Data Engineer responsible for creating and maintaining ETL pipelines at Sand Cherry Associates with expertise in Redshift SQL, Python, and DBT.
Data Engineer responsible for the end - to - end data infrastructure at AVK. Designing ETL pipelines and collaborating with teams on data solutions.
Migration Engineer leading data migrations to GiG’s CoreX platform with a focus on data integrity and collaboration.
Data Platforms Manager overseeing technical delivery of engineering initiatives for Disney's data solutions. Leading project teams, innovation, and multi - stakeholder collaborations in a fast - paced environment.
Senior Data Engineer with expert - level database skills at FreedomPay. Collaborating on data pipelines and products across SQL Server and Snowflake.
Migration Engineer responsible for migrating brands to CoreX platform at GiG. Collaborating on data pipelines and integrity with cross - functional teams.
Migration Engineer for GiG’s Migrations Team managing and executing data migrations and ensuring data integrity. Collaborating with various stakeholders on the CoreX platform migrations.