Data Engineer developing Azure data solutions at PwC New Zealand. Responsibilities include data quality monitoring, pipeline development, and collaboration with stakeholders in a supportive environment.
About the role
- Data Engineer building and maintaining scalable data pipelines for Flowcode using modern technologies. Collaborating with cross-functional teams to ensure efficient data flow and infrastructure optimization.
Responsibilities
- Design, develop, and maintain data pipelines for ingesting, processing, and transforming large datasets using Python.
- Leverage Snowflake to build and manage robust data warehouses, ensuring efficient storage and retrieval of data.
- Build, test, and maintain modular analytics models using DBT to ensure documented, versioned, and reliable transformations in Snowflake.
- Implement ETL workflows using Python, Fivetran and Amazon DMS, ensuring data integrity, quality, and timely delivery.
- Develop and integrate APIs to facilitate seamless data exchange between internal systems and external data sources.
- Implement data ingestion solutions from streaming platforms, including Kafka, to handle real-time data processing.
- Use Apache Airflow to schedule and monitor data workflows, ensuring consistent and reliable pipeline execution.
- Deploy and manage applications using Docker and Kubernetes to create scalable, containerized solutions.
- Continuously monitor, optimize, and troubleshoot data processes and infrastructure for performance improvements and reliability.
Requirements
- 3+ years of experience in data engineering or software development
- Bachelor’s or Master’s degree in Computer Science, Engineering, Information Systems, or a related field, or equivalent practical experience in data engineering.
- Proficient in Python.
- Hands-on experience with Snowflake, Fivetran, DBT, and Amazon DMS.
- Experience with RESTful/GraphQL APIs and Kafka.
- Skilled with Apache Airflow, Docker, and Kubernetes.
- Strong problem-solving abilities, with attention to detail and a focus on data quality.
- Proven experience working collaboratively in agile, cross-functional teams.
Benefits
- Health insurance
- 401(k) matching
- Flexible work hours
- Paid time off
- Professional development opportunities
Job title
Job type
Experience level
Salary
Degree requirement
Tech skills
Location requirements
Senior Data Engineer designing and implementing the Enterprise Data Platform at Stellix. Focusing on analytics and insights with a growth path to Principal Data Engineer or Data Architect.
R&D Data Engineer at DXC, transforming complex data into digital assets for global analytics and Smart Lab solutions. Collaborating on ELN and LIMS tools for enhanced data management.
Senior Data Engineer at mobility AI company designing large - scale data processing pipelines. Leading technical decisions and mentoring junior engineers in data architecture.
Data Engineer role focusing on data pipelines and processing at 42dot, a mobility AI company. Responsibilities include data collection, schema management, and pipeline monitoring.
Senior Data Engineer at Booz Allen building advanced tech solutions for mission - driven projects. Utilizing data engineering activities, pipelines, and platforms for impactful data insights.
Senior Software Engineer contributing to Workday's AI/MLOps cloud ops platform. Involves data ingestion, computation, and generation of curated data sets with modern technologies.
Data Engineer role at Citi designing and maintaining scalable data solutions. Seeking a skilled professional with extensive data engineering experience and expertise in various technologies.
Data Engineer responsible for designing and maintaining enterprise data warehouse for various projects. Collaborating with stakeholders to ensure efficient data flow and integration.
Data Engineer developing and evolving BI platform for Thomson Reuters in India. Building architectural solutions and collaborating in all development lifecycle aspects.