Senior Data Engineer role at AI & Data Engineering focused on Databricks engineering.
About the role
- Data Engineer building analytical tools for Dry Bulk market data operations at Kpler. Join a team of over 700 experts transforming data into actionable strategies.
Responsibilities
- Create scripts, CLI utilities, and web-based UIs (using tools like Streamlit) that empower analysts to perform data quality interventions more efficiently and at greater scale
- Query and analyse substantial datasets to surface insights, identify data quality issues, and inform both tactical interventions and strategic improvements
- Work alongside software engineers to develop backend tools and data pipelines that improve the overall quality and reliability of our tracking systems
- Understand the day-to-day challenges analysts face and translate those pain points into technical solutions that reduce manual effort
- Create clear documentation for tools, data processes, and analytical approaches to ensure knowledge sharing across the team
- Develop automated checks and monitoring systems that proactively flag data anomalies before they require manual intervention
- Rapidly test new approaches to data quality challenges, learning from what works and what doesn't, and scaling successful solutions
Requirements
- Experience extracting, transforming and analysing data from databases and APIs
- Comfortability working with large, unstructured datasets too complex for standard spreadsheet tools
- Experience tackling problems that have required learning new tools or technical approaches
- Background working as part of a multidisciplinary team, particularly with mixed technical/non-technical colleagues
- Proficiency in Python (or demonstrable ability to become proficient quickly)
- Experience building CLI tools, or scripts that automate processes and improve team efficiency
- Familiarity with Git-based workflows and code reviews
- A mindset that prioritises delivering practical, workable solutions over chasing perfect ones that never ship
- The inclination to take ownership of problems, using initiative to tackle issues, without waiting to be asked
Benefits
- Inclusive and diverse work environment
- Supportive colleagues and clients
- Opportunities for personal and professional growth
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Senior Data Engineer at Yü Group responsible for leading data engineering team and optimizing data solutions within the energy sector.
Data Engineer managing payment processing and data accuracy while collaborating with financial teams. Building and optimizing data pipelines for transactional data in a hybrid work environment.
Data Engineer developing tools for maintaining data integrity in cargo tracking at Kpler. Collaborating with analysts and engineers to enhance data quality management.
Lead Azure Data Engineer designing and optimizing data ecosystems on Microsoft Cloud. Responsible for building scalable data platforms and pipelines for analytics and reporting.
Data Engineer providing support for IBM DataStage ETL jobs at Callibrity. Collaborating with stakeholders and working to modernize technology solutions in a hybrid work environment.
Cloud Data Engineer implementing tailored solutions for Volkswagen Group data processing. Building ETL/ELT pipelines while collaborating with technical experts.
Data Engineer responsible for building scalable data infrastructure that supports data - driven decisions. Collaborating with team to maintain systems and unlock data value for organizations.
Data Engineer designing and optimizing data pipelines using Databricks and Google Cloud Platform. Collaborating with analysts and scientists to deliver high - quality data products.
Associate Data Engineer supporting privacy engineering controls and executing privacy impact assessments in a financial services company. Collaborating across business units to ensure alignment with privacy regulations.