Senior Data Engineer designing and improving software for business capabilities at Barclays. Collaborating with teams to build a data and intelligence platform for Equity Derivatives.
About the role
- Senior Data Engineer with 3–5 years experience in building scalable Azure data pipelines. Collaborating with architects and business teams for analytics and ML use cases in AI & Data Engineering department.
Responsibilities
- Design, build, and maintain ETL/ELT pipelines using Synapse or Azure Data Factory.
- Implement data models and optimize storage for Azure Data Lake Storage (ADLS), Azure SQL, and Azure Synapse SQL.
- Write clean, performant PySpark/SQL transformations; manage job orchestration, dependencies, and retries.
- Develop and maintain Azure Functions for lightweight data transformations, event-driven ingestion, and custom pipeline activities.
- Use Azure Logic Apps to build automated workflows for data movement, validations, notifications, and approvals.
- Develop batch pipelines and streaming jobs using Azure Event Hubs / Kafka and Structured Streaming or Azure Stream Analytics.
- Contribute to data pipeline deployments using Azure DevOps (Repos, Pipelines), version control (Git), and environment management.
- Implement data validation, profiling, and monitoring (e.g., ADF/Synapse job monitoring, alerting).
- Apply best practices for RBAC, Key Vault, Managed Identities, and data governance (cataloging, lineage) using Microsoft Purview.
- Produce clear technical documentation and collaborate with BI/ML teams to support use cases.
Requirements
- 3–5 years of hands-on experience building reliable, scalable data pipelines on Microsoft Azure
- 2–3 years experience in data engineering with Azure.
- Strong problem-solving, data analysis, and root cause analysis skills
- Strong knowledge of Azure Data Factory or Azure Synapse Analytics.
- Proficiency in PySpark, SQL, and performance tuning.
Benefits
- Azure ADF Data Engineers.
- Health insurance
- Retirement plans
- Flexible work arrangements
- Professional development
- Stock options
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Senior AI & Data Engineer developing and implementing AI solutions in collaboration with clients and teams. Working on projects involving generative AI, predictive analytics, and data mastery.
Consultant driving IA business growth in Deloitte's Artificial Intelligence & Data team. Delivering innovative solutions using data analytics and automation technologies.
Data Engineer responsible for managing data architecture and pipelines at Snappi, a neobank. Collaborating with teams to enable data processing and analysis in innovative banking solutions.
Data Engineer at Destinus developing the data platform to support production and analytics needs. Involves migrating Excel sources to Lakehouse and integrating ERP systems in a hybrid role.
Senior Data Engineer developing solutions within the Global Specialty portfolio at an insurance company. Engaging with diverse business partners to ensure high quality data reporting.
Data Engineer at UBDS Group focusing on designing and optimizing modern data platforms. Collaborating in a multidisciplinary team to develop reliable data assets for analytics and operational use cases.
Data Engineer (dbt) at SDG Group involved in all phases of data projects. Collaborate on data ingestion, transformation, and visualization in a hybrid environment.
Data Consultant at SDG Group specializing in Data & Analytics projects. Collaborate on technical - functional definitions, ETL, data modeling, and visualization for cloud solutions.
Senior Data Engineer responsible for growing customer - defined targeting calculations and developing key/value databases for real - time data processing.