Data Engineer in Veepee's Data Factory working on data ingestion pipelines and improving data quality. Collaborative environment utilizing Kubernetes, Python, Java, and modern data architectures.
About the role
- Senior Data Engineer at Barclays building and maintaining data pipelines and warehouses. Collaborating with data scientists and ensuring data accuracy, accessibility, and security.
Responsibilities
- Build and maintain systems that collect, store, process, and analyze data, such as data pipelines, data warehouses, and data lakes
- Ensure data accuracy, accessibility, and security
- Design and implement data architectures and processing algorithms
- Collaborate with data scientists to build and deploy machine learning models
- Advise stakeholders on functional and cross-functional areas of impact and alignment
- Manage and mitigate risks through assessment in support of control and governance agenda
- Create solutions based on sophisticated analytical thought and research
Requirements
- Advanced Apache Spark expertise (API design and performance optimization)
- Software engineering skills in Java, Scala, and Python, including unit testing best practices
- Experience with data storage and serialization formats (Parquet, Avro) and open table formats/catalogs (Hive Metastore, AWS Glue, Unity Catalog, Iceberg)
- Proficient in packaging and build tooling (sbt, Nexus) and version control using GitLab
- Ample DevOps and automation skills, including GitOps, CI/CD pipelines in GitLab, Bash scripting, and Linux environments
- Experience with the Databricks platform and AWS infrastructure/architecture design
- Proficient in Infrastructure as Code (Terraform) for cloud provisioning and environment management
- Experience with streaming and queuing technologies, including Apache Kafka, AWS SNS, AWS SQS, and MQ
- Framework development experience focused on scalable and maintainable solutions
Benefits
- Join Barclays as a Senior Data Engineer
- Professional network with industry mentors and experts
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Data Architect designing and maintaining enterprise data architecture at Envalior. Driving enterprise - wide impact ensuring scalability and reliability of systems, reporting, and AI initiatives.
Data Engineer role at Valmont focused on data analytics and technology for sustainable agricultural practices. Collaborating with cross - functional teams to enhance data management and analytics tools.
Lead Data Engineer guiding a team in designing scalable data solutions for iKnowHow S.A. Overseeing development of data pipelines while collaborating with cross - functional teams.
Data Engineer at LPL Financial developing Python - based ETL pipelines. Collaborating with cross - functional teams to ensure reliable data delivery and optimizing pipeline performance.
Senior Data Engineer at Keyrus focusing on data solutions and projects to drive performance. Collaborating with teams globally to enhance data transformation and governance processes.
Data Engineer developing scalable data pipelines for ETL/ELT processes using GCP services. Collaborating with team members to optimize data workflows and ensure data integrity.
Data Governance Engineer in Fintech developing a formal cyber data governance framework. Collaborating with cyber security, analytics, and platform engineering teams on metadata and lineage capabilities.
Junior Data Engineer role at Allegro, focusing on developing ETL/ELT pipelines and processing large datasets. Collaborate with cross - functional teams for data quality and reporting.
Data Engineer at Concept Reply developing innovative data - driven solutions in IoT. Collaborating with teams to unlock the potential of data and cloud computing.