Data Engineer Lead developing and maintaining data applications and pipelines in Azure for State Street. Collaborating with teams to ensure data integrity and implementing data processing principles.
Responsibilities
Design, develop, and maintain data pipelines in Azure for ingesting, transforming, and loading data from various sources into centralized Azure data lakes and Databricks Delta Lake
Ensure data quality and integrity throughout the process
Implement efficient ELT/ETL processes to ensure data quality, consistency, and reliability
Develop transformation processes to clean, aggregate, and enrich raw data, ensuring it is in the appropriate format for downstream analysis and consumption
Integrate data from diverse sources to provide a unified view of information
Design and implement efficient data models and database schemas that support the storage and retrieval of structured and unstructured data
Optimize data storage and access for performance and scalability
Implement knowledge of modern data processing principles to streamline data import/transformation processes
Collaborate with stakeholders to define and enforce data governance standards and policies
Requirements
Bachelor’s Degree level qualification in a computer or IT related subject
10+ years of overall IT industry experience
8+ years of overall Bigdata data pipeline experience
8+ years of experience as a Data Engineer, with a focus on designing and implementing data solutions on the Azure Databricks
8+ years of experience on cloud-based development including Azure Services, Azure Devops, Kubernetes, Docker
Experience on Snowflake is plus
Azure certifications such as Azure Data Engineer Associate or Azure Solutions Architect is a plus
Benefits
generous medical care
insurance coverage including basic life, medical, dental, vision, long-term disability, and other optional additional coverages
retirement savings plan (401K) with company match
paid-time off including vacation, sick leave, short term disability, and family care responsibilities
access to our Employee Assistance Program
incentive compensation including eligibility for annual performance-based awards (excluding certain sales roles subject to sales incentive plans)
Data Architect designing and implementing data architectures for pharmaceutical clients. Collaborating with teams to create efficient data ecosystems that drive business value.
Working student in Automation & Data Engineering at Kelvion supporting internal tools and automations using Microsoft Power Platform. Involvement in systems configuration, reports, dashboards, and data structure maintenance.
Senior AWS Data Engineer supporting HHS grant award management, providing technical leadership and maintaining cloud data systems. Ideal for candidates excelling in modernization and operational environments.
Data Engineer at Booz Allen developing technology solutions for clients analyzing large datasets. Collaborating with teams on mission - driven projects using data engineering best practices.
Data Engineer supporting data pipelines for business insights and GenAI applications at Assurity Trusted Solutions. Collaborating with teams on data workflows in a hybrid environment.
Senior Data Engineer leveraging big data and cloud expertise to build data pipelines at Alight. Ensuring reliability, governance, and operational excellence across data platforms.
Data & AI Engineer at Decathlon Colombia designing and optimizing data ecosystems. Involves building data pipelines and deploying machine learning models.
Senior hands - on engineering lead at Electric Mind focusing on data platforms, AI, and analytics. Collaborating with teams on complex, mission - critical transformations in technology solutions.
AI Data Engineer (Azure + Copilot Studio) developing data pipelines and AI agent solutions. Involves working with Azure Data Factory, Copilot Studio, and data integration in a hybrid role.
Senior Data Engineer in Porto Bank developing and managing scalable data architectures and systems. Focused on integrating diverse data solutions while identifying and managing risks.