Senior Data Engineer developing high - impact data solutions in a collaborative financial team. Integrating data systems and ensuring performance with innovative technologies.
About the role
- Data Engineer at Logcomex, developing robust data pipelines and ensuring data quality for trade solutions. Collaborating with various teams to optimize data governance and performance.
Responsibilities
- Develop and maintain robust, scalable, and testable data pipelines using PySpark and Airflow.
- Design and implement data ingestion and transformation processes to populate the data lake using a layered architecture (Bronze, Silver, Gold).
- Work on data quality control, documentation, and lineage management using OpenMetadata.
- Collaborate with product and capture squads to ensure data consistency and coverage.
- Ensure data governance, versioning, and auditing of production pipelines.
- Optimize ETL/ELT routines and query performance in relational databases, data warehouses, and engines such as Elasticsearch and Athena.
Requirements
- We are looking for people with knowledge in...
- PySpark
- Apache Airflow
- AWS S3, Glue, Athena, EC2
- SQL (Athena, PostgreSQL)
- Elasticsearch/OpenSearch
- Docker
- Pandas
- Jupyter
- Unix (Linux), Bash
- DBT
- Nice to have:
- Glue
- Delta Lake
- Kubernetes
- NoSQL
- Elasticsearch
- Airbyte
Benefits
- Swile benefits card with a fixed monthly value of R$2,540.00 (food, mobility, multi-balance, and home office allowance);
- National health plan (Unimed or Amil);
- Dental plan (Odontoprev);
- Life insurance (MetLife);
- TotalPass;
- Starbem (health services platform for your physical, mental, and emotional well-being);
- Pharmacy discount program with Panvel;
- Extended maternity and paternity leave through the Empresa Cidadã program;
- Subsidy for professional development in partnership with Unico Skill, offering various options for undergraduate, postgraduate, language courses, mentorships, etc.;
- Private English lessons for leadership and specialists level II and above;
- School/Education assistance;
- Fresh fruit, cookies, coffee, tea, and energy drinks available at any time;
- Celebrations, integration events, and team building activities;
- Partnership with KÜK Station to provide the best for our Loggers on office days;
- In-company massage;
- Birthday day-off;
- Birthday gift;
- Service anniversary gifts;
- Recruta Loggers (employee referral program with bonuses).
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
No Education Requirement
Tech skills
Location requirements
Senior Data Engineer developing data pipelines and infrastructure on Google Cloud Platform for WorkWhile's staffing marketplace. Collaborating with Data Science and Engineering teams to enhance data quality and availability.
Data Engineer developing data platforms for a consulting firm focused on quality solutions. Collaborating within a small team to deliver robust infrastructure and systems.
Senior Data Engineer designing and maintaining data pipelines within a fast - growing social impact startup. Collaborate cross - functionally to enhance products and analytics capabilities.
Senior Associate in data engineering at PwC focusing on designing robust data solutions. Leading complex data pipeline projects and collaborating with cross - functional teams to support automation and analytics.
Data Engineering & Warehousing Manager overseeing data engineering and warehousing operations at Hastings Insurance. Leading pipelines, platforms, and technical teams for enterprise data insights.
Senior Data Engineer delivering scalable data solutions in data engineering team at fintech startup. Building and maintaining data pipelines, collaborating with cross - functional teams for accurate data delivery.
Software Engineer contributing to Machine Learning initiatives and data infrastructure at AKASA. Working in a hybrid setup between South San Francisco and NYC.
Senior Enterprise Data Architect at Fresenius Kabi shaping data governance and architecture strategies across the enterprise for data - driven decision - making.
Database Engineer I focused on acquisition and integration of data into Data Lake. Responsible for management of databases and leading company’s data strategy.