Azure Data Engineer responsible for creating scalable solutions on Microsoft Azure. Involves data ingestion, transformation, and collaboration with cross-functional teams.
Responsibilities
Design, develop, and implement scalable and reliable data solutions on the Microsoft Azure platform.
Collaborate with cross-functional teams to gather and analyze data requirements.
Design and implement data ingestion pipelines to collect data from various sources, ensuring data integrity and reliability.
Perform data integration and transformation activities, ensuring data quality and consistency.
Implement data storage and retrieval mechanisms, utilizing Azure services such as Azure SQL Database, Azure Data Lake, and Azure Blob Storage.
Monitor data pipelines and troubleshoot issues to ensure smooth data flow and availability.
Implement data quality measures and data governance practices to ensure data accuracy, consistency, and privacy.
Collaborate with data scientists and analysts to support their data needs and enable data-driven insights.
Requirements
Bachelor's degree in Computer Science, Engineering, or a related field.
5+ years of experience with Big Data technologies
Strong knowledge and experience with Microsoft Azure cloud platform
Proficiency in SQL and experience with SQL-based database systems.
Experience with batch and data streaming.
Proficiency in data processing frameworks such as Apache Spark, Apache Hadoop, or cloud-native data processing services (Azure Data Lake, Azure Data factory, Azure Databricks, Azure Synapse, Snowflake, CosmosDB)
Hands-on experience with Azure data services, such as Azure SQL Database, Azure Data Lake, and Azure Blob Storage.
Experience using Azure Databricks in real-world scenarios.
Experience with data integration and ETL (Extract, Transform, Load) processes.
Strong analytical and problem-solving skills.
Good understanding of data engineering principles and best practices.
Experience with programming languages such as Python or Scala.
Relevant certifications in Azure data services or data engineering are a plus.
Program Manager leading enterprise - wide data migration efforts for Boeing's transition to modern data platforms. Overseeing complex processes to ensure secure and effective migrations across multiple systems.
Senior Data Engineer responsible for developing data products for Disney's immersive digital experiences. Collaborating with teams to ensure data quality and operational efficiency in a fast - paced environment.
Senior Data Engineer crafting and developing data products for analytical insights at Zendesk. Collaborating in an Agile environment with a focus on data warehousing and process optimization.
Senior Data Engineer designing and building data warehouse solutions with Snowflake for a fintech company. Collaborating with cross - functional teams to facilitate data insights and analytics.
Data Engineer developing and maintaining data pipelines and applications at EvidenceCare. Collaborating across teams to generate actionable insights from healthcare data for better decision - making.
Data Engineer managing and expanding enterprise business intelligence and data platform. Focusing on Tableau development and administration with a strong engineering background.
Lead Data Engineer overseeing engineers and advancing the data platform at American Family Insurance. Creating tools and infrastructure to empower teams across the company.
Data Architect designing end - to - end Snowflake data solutions and collaborating with technical stakeholders at Emerson. Supporting the realization of Data and Digitalization Strategy.
Manager of Data Engineering leading data assets and infrastructure initiatives at CLA. Collaborating with teams to enforce data quality standards and drive integration efforts.