Engenheiro(a) de Dados responsável por desenvolvimento de pipelines e otimização de dados em Azure. Focando na qualidade de dados e na arquitetura Medallion.
Responsibilities
Pipeline Development: Build and maintain robust data pipelines using PySpark in the Azure Databricks environment, ensuring high availability and continuous data flow.
Medallion Architecture: Implement and manage the data lifecycle following a layered structure (Bronze, Silver, Gold), ensuring organization and quality from ingestion through consumption.
Ingestion and Integration: Configure ETL/ELT workflows from heterogeneous sources (SQL Server, APIs, streaming) using Azure Data Factory and Azure Data Lake Storage (ADLS).
Optimization and Monitoring: Oversee cluster and process performance with a focus on scalability, latency reduction, and cost-efficiency in the cloud.
Data Quality: Develop data cleansing, processing and enrichment routines to turn raw data into assets ready for business analysis.
Security and Governance: Document technical processes and ensure all pipelines comply with data security and privacy policies.
Requirements
Education: Bachelor's degree in Computer Science, Engineering, Data Science or related IT fields.
Azure Cloud: Hands-on experience with the Microsoft data ecosystem.
Technical Stack: Advanced proficiency in Databricks, PySpark, Python and SQL.
Architecture: Solid experience with Medallion Architecture, Data Lake concepts and Delta Lake technology.
Modeling: Knowledge of data modeling (Dimensional/Star schema).
Visualization: Basic knowledge of Power BI to support the consumption (Gold) layer.
System Protection Engineer Intern working on transmission and distribution protection engineering projects at PG&E. Supporting project and maintenance activities with hands - on exposure under supervision.
Brake Test & Development Engineer developing North America brake systems through testing for Hyundai/KIA/Genesis vehicles. Collaborating with various departments and conducting benchmarking activities.
Chassis Dynamics Development Engineer II at Hyundai developing ESC systems and chassis dynamics control systems. Conducting tests and collaborating with engineers to enhance vehicle performance.
Chassis Control Calibration Engineer developing ESC systems through testing and collaboration with engineers at Hyundai Motor Company. Includes supporting vehicle performance and safety standards.
SCADA Engineer responsible for development and engineering of SCADA solutions for offshore wind projects. Collaborating on SCADA, network, and telecommunications concepts adhering to standards.
Graduate Engineer assisting site management from project start to handover at BAM Ireland based in Donegal. Engaging in civil engineering projects while collaborating with site teams.
Process Engineer responsible for defining, planning, executing, and validating manufacturing equipment launches. Collaborating cross - functionally to ensure machines are installed, commissioned, and integrated into production.
Senior Signal Integrity Engineer ensuring the performance and reliability of high - speed hardware designs at General Motors. Collaborating with teams to develop next - generation automotive systems.
Lab Informatics Engineer at Lonza developing lab informatics solutions to empower researchers and enhance scientific workflows. Collaborating with cross - functional teams to drive digital transformation in R&D.
Lab Informatics Engineer at Lonza designing scalable lab informatics systems. Collaborating with R&D to improve scientific workflows and enhance data management standards.