Senior Data Engineer designing scalable data pipelines and supporting ML workloads at Trainline. Collaborating with cross - functional teams in a hybrid environment based in London.
About the role
- Senior Data Engineer optimizing scalable data pipelines at Matrix for the Brazilian energy market. Involved in ETL processes and data architecture design with a collaborative team environment.
Responsibilities
- Develop and optimize high-performance, scalable batch data pipelines for ingesting structured, semi-structured, and unstructured data into our Data Lake (GCS);
- Develop and manage ETL/ELT jobs for data processing using Spark/SQL to populate the Trusted (Silver) and Refined (Gold) layers of our Data Warehouse (BigQuery);
- Consolidate and model data from internal and external sources to enable Analytics, Business Intelligence, Data Science, GenAI initiatives, and Data Products;
- Develop online data integrations (streaming) for data ingestion and processing on GCP;
- Design and implement efficient data architectures, optimizing development time, performance, and compute cost;
- Reproduce business rules within data transformation processes to generate datasets;
- Create and maintain Data Quality, Data Assurance, Lineage, and Data Versioning processes;
- Serve as the Technical Reference for Matrix Energia's Data & Analytics team, supporting the most innovative and ambitious initiatives in our day-to-day work.
Requirements
- Bachelor's degree in Computer Science, Systems, Technology, Data, or a related field;
- Strong experience with Python and Spark/SQL;
- Experience with CI/CD, preferably using GitHub;
- Experience with Google Cloud Platform (GCP), Google Cloud Storage (GCS), and Google BigQuery;
- Experience with data orchestration tools such as Airflow or Prefect;
- Experience with data streaming technologies such as Pub/Sub or Azure Functions;
- Knowledge of Data Modeling using the Medallion methodology.
Benefits
- High-quality health coverage with Omint: access to a top-tier medical and dental network with nationwide coverage.
- Alelo Pod card for flexible meal and grocery expenses, accepted at major restaurant chains and supermarkets.
- Flexible and sustainable working hours.
- Mobility: choose the option that works best for you — transit voucher, fuel allowance, or parking.
- Total Pass: an online platform dedicated to promoting quality of life, health, and well-being. Plans provide access to gyms, studios, and online services with exclusive Matrix discounts, covering you and up to 3 dependents.
- Group life insurance.
- Profit Sharing (PLR), subject to target achievement and performance.
- Work with a fantastic team in a collaborative environment focused on building a brighter, more sustainable future.
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Performance Data Engineer providing data modeling expertise and engineering a cloud - based Data Lakehouse platform. Support Federal agency ETL applications with ongoing development and maintenance responsibilities.
Manager for Data & AI team at Valorem Reply focused on modern data platforms with Microsoft technologies. Leading technical direction and collaborating with clients on data governance frameworks.
Manager of Data & AI team focused on building modern data platforms using Microsoft technologies. Collaborating with clients and team members for successful delivery and governance.
Senior Staff Data Engineer at DeepL working on enterprise - wide data engineering standards and cloud solutions. Leading technical initiatives and mentoring engineers to support data capabilities across the organization.
Data Engineer contributing to advanced analytics and machine learning solutions in aviation at Boeing. Collaborating within a data science team to produce industry - leading insights and build cloud - based tools.
Data Engineer designing and maintaining scalable data solutions in GCP and Snowflake environments. Collaborating with clients and stakeholders to ensure data quality and functionality.
Data Architect leading data architecture and engineering projects within GCP and Snowflake environments for AI consulting company. Collaborate with clients to define data strategies and guide implementation teams.
Implementation Engineer guiding customers in data architecture and integration solutions at MotherDuck. Collaborating with teams and leading technical discussions to ensure customer success.
Data Engineer/Senior Data Engineer developing scalable ETL/ELT pipelines and architecting data systems at Manulife. Collaborating with data professionals to ensure data quality and compliance.