Data Engineer focused on developing and sustaining Machine Learning solutions for ICBC's data needs. Collaborating with Data Scientists and Statistical Analysts on data - driven projects.
About the role
- Full Stack Data Engineer responsible for building and optimizing data pipelines and lakehouse architectures. Collaborating with teams to ensure data quality and delivering analytics-ready datasets.
Responsibilities
- Design, develop, and maintain end-to-end data pipelines using **Python, PySpark, and SQL**
- Build and optimize data transformation workflows using **dbt on Snowflake**
- Develop scalable **lakehouse architectures** for structured and semi-structured data
- Implement reliable data ingestion frameworks using **Kafka, AWS Glue, and custom connectors**
- Orchestrate workflows and manage dependencies using **Apache Airflow**
- Manage cloud infrastructure on **AWS** (S3, Glue, EMR, Redshift/Snowflake integrations)
- Implement Infrastructure as Code (IaC) using **Terraform**
- Collaborate with cross-functional teams to deliver analytics-ready datasets
- Ensure data quality, performance optimization, and cost efficiency
- Use **GitLab** for version control, CI/CD, and collaborative development
- Monitor, troubleshoot, and resolve data pipeline issues in production environments
Requirements
- 4+ years of experience with **AWS** data services and cloud-based data engineering
- Strong programming skills in **Python** and **PySpark**
- Hands-on experience with **Snowflake** and **dbt** for data modeling and transformations
- Solid understanding of **SQL** for complex analytical queries
- Experience with **Apache Airflow** for workflow orchestration
- Proficiency in **Kafka** for real-time/streaming data ingestion
- Experience with **AWS Glue/ Airflow** for ETL development
- Experience with **Terraform** for infrastructure automation
- Strong experience with **GitLab** and CI/CD pipelines
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Domain Data Architect at Farmers designing and developing IT architecture solutions aligned with enterprise standards. Requires extensive experience in Domain Architecture with a focus on business and technology domains.
Senior Data Engineer at Tripledot focusing on AI/ML initiatives for mobile game development. Collaborating with cross - functional teams to enhance player experience and company efficiency.
Senior Data Engineer working on data pipelines to enhance AI/ML capabilities in mobile gaming. Collaborating with cross - functional teams to drive data - driven insights and solutions.
Senior Data Engineer developing scalable data solutions within AI group at Tripledot. Collaborating with cross - functional teams to enhance game performance and player experience.
Lead Data Engineer at Verisk Analytics designing and optimizing AWS data platforms. Responsible for mentoring a team and delivering solutions for insurance and financial services clients.
Senior Data Architect role at Humana contributing to data architecture, data integration, and interoperability for CenterWell's OneHome team. Responsible for ensuring data solutions align with business and user needs.
Senior Data Engineer providing database engineering support for Fidelity’s API development team. Involves database design and modern observability tools, requiring advanced SQL and programming skills.
Senior Data Engineer within AI group at Tripledot, focusing on scalable data pipelines to enhance mobile gaming experiences. Collaborating with teams to integrate AI/ML solutions across games.
Senior Data Engineer working within AI group at Tripledot, a leading mobile games company. Collaborating with various teams to enhance AI/ML capabilities across all studios.