Senior Engineer responsible for data lake management and processing pipeline design using Python, Spark, and Azure. Ensuring high data quality and operational continuity in a growing organization.
Responsibilities
Maintain, develop, and optimize the global datalake environment, ensuring high data quality, reliability, and overall performance
Design and implement high-performance data processing pipelines and workflows utilizing Python, Spark, and Databricks
Manage scalable and efficient data integration and processing solutions using Azure Data Factory, Databricks, and Event Hub
Establish robust data management processes and execute advanced data querying and quality control with SQL and big data technologies
Automate operational tasks and support infrastructure maintenance by leveraging shell scripting and foundational UNIX knowledge
Support data modeling initiatives for coherent structure design and document all technical processes to ensure operational continuity
Requirements
Strong background and understanding of database data management (general SQL knowledge, data querying, data quality control)
Strong programming knowledge in Python (pandas, numpy)
Advanced level Big Data / data lake framework (Spark, Databricks)
Strong cloud knowledge and handling of Azure Data Factory, Databricks, Event Hub
Basic familiarity with UNIX operating system, especially shell scripting
Basic understanding of network level problems and connectivity requirements
Basic Understanding data modelling principes
Data-centric mindset
Structured, analytical thinking
Can work in a multi-shift operation (Monday to Friday, 07:00 AM - 22:00 PM) + on-call (weeknights, weekend)
Benefits
Possibility to improve yourself in a constantly growing organization
Software Engineer developing AI - based solutions for social work. Involves innovation cycle participation and real user - centric technological implementation.
Specialist Software Engineer at L3Harris Technologies contributing to planning and implementation of software components and applications. Responsible for technical designs, mentoring, and maintaining corporate standards.
Principal Software Engineer leading technical direction for Thanks’ advertising platform. Managing projects and mentoring engineers while ensuring high - quality production code is delivered.
Software Engineer needed for dynamic project team specializing in development of embedded software. Position based in Saint - Mandrier - sur - Mer with occasional travel.
Software Developer creating innovative embedded solutions for products and customer projects in Deggendorf or Regensburg. Focus on high - performance software with clear APIs for efficient system integration.
Senior Data Processing Engineer at ABSOLUT SENSING developing geophysical products from hyperspectral images. Focused on satellite monitoring to reduce global methane emissions.
Automation Test Lead with strong expertise in selenium, TestNG, and Cucumber. Responsible for automation of web - based applications and test methodologies in testing services.
SAP Tosca Sr Technical Lead using MySQL for automation and testing. Collaborating on test scripting and executing diverse test types to ensure software quality.
SAP ABAP Technical Lead involved in gathering user requirements and developing enterprise applications. Collaborating with various teams to ensure effective implementation of technical designs and documentation.
Selenium test automation expert with 8 - 10 years experience in QA methodologies. Developing automation scripts and collaborating with teams on web - based applications.