Engenheiro(a) de Dados responsável por desenvolvimento de pipelines e otimização de dados em Azure. Focando na qualidade de dados e na arquitetura Medallion.
Responsibilities
Pipeline Development: Build and maintain robust data pipelines using PySpark in the Azure Databricks environment, ensuring high availability and continuous data flow.
Medallion Architecture: Implement and manage the data lifecycle following a layered structure (Bronze, Silver, Gold), ensuring organization and quality from ingestion through consumption.
Ingestion and Integration: Configure ETL/ELT workflows from heterogeneous sources (SQL Server, APIs, streaming) using Azure Data Factory and Azure Data Lake Storage (ADLS).
Optimization and Monitoring: Oversee cluster and process performance with a focus on scalability, latency reduction, and cost-efficiency in the cloud.
Data Quality: Develop data cleansing, processing and enrichment routines to turn raw data into assets ready for business analysis.
Security and Governance: Document technical processes and ensure all pipelines comply with data security and privacy policies.
Requirements
Education: Bachelor's degree in Computer Science, Engineering, Data Science or related IT fields.
Azure Cloud: Hands-on experience with the Microsoft data ecosystem.
Technical Stack: Advanced proficiency in Databricks, PySpark, Python and SQL.
Architecture: Solid experience with Medallion Architecture, Data Lake concepts and Delta Lake technology.
Modeling: Knowledge of data modeling (Dimensional/Star schema).
Visualization: Basic knowledge of Power BI to support the consumption (Gold) layer.
Development Engineer focusing on PEM electrolysis and mechanical properties of membrane materials. Collaborating on characterization methods and material qualification within a multidisciplinary team.
Roaming IREG Engineer managing international roaming processes and procedures for a global mobile communications provider. Responsible for technical setup, testing, and maintenance of roaming services.
Component Engineer managing the electronic components portfolio for b - plus in Deggendorf. Ensuring data accuracy in the PLM system and supporting technical evaluations.
PLC Engineer leading automation control operations for a leading airport logistics provider. Involves system upgrades, troubleshooting, and enhancing project performance in baggage handling.
Senior Software Engineer at citema systems GmbH developing software systems for satellite ground systems and cybersecurity applications. Collaborating in agile projects while also focusing on software architecture and interface design.
Entry level engineer at DTE Electric Company responsible for routine engineering assignments. Collaborates with field employees and assists in the operation and maintenance of company assets.
Project Engineer delivering engineering projects in support of ESB’s strategic objectives. Collaborating with teams to ensure technical specifications are met and managing contractor engagements.
Junior Engineer supporting Central Health and Safety Team at Bridgestone during maternity leave. Collaborating across various business environments and geographic regions in Rome, Italy.
Molding Engineer designing and defining efficient processes for plastic injection molds at Newell Brands. Leading continuous improvement projects while ensuring production meets quality specifications.
Process Engineer at Newell Brands involved in troubleshooting and improving manufacturing processes. Requires engineering degree and injection molding background for efficiency and productivity.