Data Engineer focused on analytics and data pipeline development for network optimisation. Collaborating with teams to deliver high - quality data solutions with Python and SQL.
About the role
- Data Engineer developing data pipelines and stream processing solutions for Leonardo in the Cyber & Security Solutions area. Supporting data ingestion, processing, and analytics for large-scale datasets.
Responsibilities
- Sviluppare data pipelines per ingestion, processing e transformation di grandi volumi di dati
- Implementare batch processing jobs con Apache Spark (PySpark, Scala)
- Sviluppare real-time data pipelines con Apache Kafka e Apache Flink
- Implementare stream processing applications per event transformation, enrichment e aggregation
- Orchestrare workflows complessi con Apache Airflow (DAG design, dependencies, scheduling)
- Sviluppare trasformazioni analitiche con SQL avanzato e dbt per analytics layers
- Sviluppare streaming aggregations con windowing operations (tumbling, sliding, session windows)
- Integrare stream processing con batch layers per unified analytics
- Implementare exactly-once processing semantics e state management in Flink
- Sviluppare Kafka consumers e producers con optimal configuration for throughput
- Implementare data quality testing e validation frameworks
- Integrare con data lakehouse (Delta Lake, Iceberg) e object storage per data persistence
- Implementare stream-to-lake integration per data persistence in lakehouse
- Sviluppare data modeling (dimensional, star schema) per analytics e reporting
- Collaborare con analytics teams per requirements gathering e data modeling
- Ottimizzare performance di Spark jobs, query execution plans e streaming applications per low-latency processing
- Implementare incremental processing patterns per efficiency
- Implementare monitoring e alerting per streaming pipelines health
- Gestire backpressure e failure recovery in streaming applications
- Supportare integration con BI tools (Tableau, PowerBI) per reporting
- Contribuire a DataOps practices (CI/CD for data pipelines, testing, monitoring) e best practices per stream processing
Requirements
- Laurea Magistrale in Ingegneria Informatica, Matematica, Statistica, Fisica, Informatica o equivalente
- 2 a 5 anni di esperienza nel ruolo, o più di 5 anni di esperienza in ruoli analoghi
- Data processing con Apache Spark (PySpark, Scala APIs) per batch workloads
- Stream processing con Apache Flink (DataStream API, Table API, SQL)
- Apache Kafka per event streaming (producers, consumers, Kafka Streams API, Kafka Connect)
- Real-time data pipelines con windowing operations e event-time processing
- State management e fault tolerance in streaming applications (checkpointing, savepoints)
- Orchestration con Apache Airflow (DAG design, custom operators, sensors, XComs)
- SQL avanzato (window functions, CTE, recursive queries, query optimization)
- dbt per analytics transformations e data modeling.
- Data modeling (dimensional modeling, star schema, data vault basics)
- Data lakehouse platforms (Delta Lake, Apache Iceberg) con ACID transactions
- Data quality frameworks (Great Expectations, dbt tests)
- Exactly-once semantics e watermarking for late data handling
- Performance tuning for low-latency and high-throughput.
- Conoscenze linguistiche: Italiano madrelingua, Inglese professionale (B2)
Benefits
- Disponibilità a brevi trasferte su territorio nazionale
- Certificazioni data engineering (Databricks, Snowflake), streaming (Confluent Certified Developer for Apache Kafka, Flink) sono titoli preferenziali
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Senior Product Manager defining platform capabilities for Data Cloud in Salesforce. Collaborating with R&D teams while shaping product strategy for Data 360 integration.
Senior Data Engineer at Goodwin enhancing data platforms and fostering data - driven culture across teams. Collaborating with IT and Finance on technology solutions and data governance practices.
Director, Data Platform Design and Strategy at MedImpact leading data platform and AI innovations to enhance healthcare services. Overseeing enterprise projects and managing teams to meet strategic goals.
Data Engineer delivering AI - and data - driven solutions for Honeywell’s industrial customers. Architecting and implementing scalable data pipelines and platforms focused on IoT and real - time data processing.
Data Engineering Associate focusing on data quality control and management for distribution platform. Collaborates on large scale data projects to ensure data accuracy and availability for users.
Data Architect managing enterprise data platform built on Microsoft Fabric at Johnstone Supply. Leading architectural standards and collaborating with business and IT leaders for strategic data - driven insights.
Data Engineer at Studyportals responsible for data pipelines and infrastructure. Join a team ensuring accurate and trustworthy data for analytics and business decisions.
AI/ML Engineer designing and refining prompts and workflows using large language models. Responsible for developing data pipelines and delivering scalable AI solutions in a hybrid work environment.
AWS Data Architect at Fractal designing and operationalizing AWS data solutions at enterprise scale. Collaborating with clients and mentoring engineers in best practices.