Senior Engineer responsible for data lake management and processing pipeline design using Python, Spark, and Azure. Ensuring high data quality and operational continuity in a growing organization.
Responsibilities
Maintain, develop, and optimize the global datalake environment, ensuring high data quality, reliability, and overall performance
Design and implement high-performance data processing pipelines and workflows utilizing Python, Spark, and Databricks
Manage scalable and efficient data integration and processing solutions using Azure Data Factory, Databricks, and Event Hub
Establish robust data management processes and execute advanced data querying and quality control with SQL and big data technologies
Automate operational tasks and support infrastructure maintenance by leveraging shell scripting and foundational UNIX knowledge
Support data modeling initiatives for coherent structure design and document all technical processes to ensure operational continuity
Requirements
Strong background and understanding of database data management (general SQL knowledge, data querying, data quality control)
Strong programming knowledge in Python (pandas, numpy)
Advanced level Big Data / data lake framework (Spark, Databricks)
Strong cloud knowledge and handling of Azure Data Factory, Databricks, Event Hub
Basic familiarity with UNIX operating system, especially shell scripting
Basic understanding of network level problems and connectivity requirements
Basic Understanding data modelling principes
Data-centric mindset
Structured, analytical thinking
Can work in a multi-shift operation (Monday to Friday, 07:00 AM - 22:00 PM) + on-call (weeknights, weekend)
Benefits
Possibility to improve yourself in a constantly growing organization
SW Developer in Mobileye's Automotive Software team focusing on AUTOSAR self - driving platforms. High debugging skills and AUTOSAR experience required for tackling complex software challenges.
FullStack Developer contributing to Smart Farming solutions in aquaculture at HydroNeo. Collaborate on mobile and backend development while learning through an innovative startup environment.
Senior Software Engineer building and maintaining payment infrastructure for Wellfit fintech. Expanding payment capabilities and ensuring secure, scalable solutions in a collaborative team environment.
Tech Lead to design and develop significant digital products at Theodo, leading a team while ensuring high - quality delivery and client satisfaction in Paris.
Tech Lead at Theodo designing and developing innovative digital products while ensuring high delivery standards. Collaborating with teams to modernize legacy systems using AI technology.
Tech Lead at Theodo leading the design and development of digital products. Collaborating with teams to ensure quality and performance for client solutions.
Software Engineer Fullstack developing effective and durable digital products at Theodo. Collaborating with small teams and focusing on client value across the entire application lifecycle.
Tech Lead designing and building digital products for clients in Theodo HealthTech. Leading project delivery and fostering team growth in a collaborative environment.
Associate Software Engineer developing web applications that support manufacturing operations and data visualization. Focused on building scalable software solutions connecting digital technology with real - world manufacturing systems.