Data Engineer designing and optimizing data pipelines using Databricks and Google Cloud Platform. Collaborating with analysts and scientists to deliver high - quality data products.
About the role
- Senior Data Engineer responsible for data pipelines and backend systems at Kpler. Working with a diverse team to deliver insights on global refinery operations and economics.
Responsibilities
- Design, build, and evolve batch and streaming data pipelines that power refinery modeling, analytics, and customer-facing products.
- Own complex data ingestion, transformation, validation, and delivery workflows across multiple data sources.
- Drive improvements in pipeline reliability, scalability, and observability, including retries, backfills, data quality checks, and monitoring.
- Lead schema design, versioning, and evolution strategies to support stable, long-lived data contracts.
- Build and maintain backend components and APIs used to serve data to downstream systems and applications.
- Partner closely with data scientists, product managers, and other engineers to translate domain requirements into robust technical solutions.
- Continuously improve existing systems as data volume, complexity, and product expectations grow.
Requirements
- Deliver high-quality, well-tested, and maintainable code, setting a strong example for engineering best practices.
- Own significant parts of the data platform end-to-end, from ingestion to production delivery.
- Make architectural contributions to data processing, storage, and delivery patterns.
- Contribute to and improve CI/CD pipelines, automation, and operational tooling.
- Instrument services and pipelines with metrics, logs, and alerts, and help define operational standards.
- Play an active role in incident response, root-cause analysis, and long-term system improvements.
- Review code, mentor other engineers, and help reinforce shared coding and architectural standards across the team.
- Exposure to Kafka, Spark, or streaming architectures.
- Experience with Kubernetes.
- Familiarity with event-driven or microservices architectures.
- Exposure to analytical datastores (e.g. Elasticsearch).
- Full-stack awareness (e.g. ability to read, review, and provide feedback on frontend or API-layer pull requests, without being a primary frontend contributor).
- Prior experience working on data products in energy, commodities, or industrial domains.
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Data Engineer responsible for building scalable data infrastructure that supports data - driven decisions. Collaborating with team to maintain systems and unlock data value for organizations.
Associate Data Engineer supporting privacy engineering controls and executing privacy impact assessments in a financial services company. Collaborating across business units to ensure alignment with privacy regulations.
Data Engineer at CVS Health optimizing data pipelines and analytical models. Driving data - driven decisions with healthcare data for improved business outcomes.
Senior Data Engineer at CVS Health developing robust data pipelines for healthcare data. Collaborating with teams to provide actionable insights and integrate them with consumer touchpoints.
Senior Data Engineer supporting AI - enabled financial compliance initiative with data pipelines and ingestion processes. Collaborating with diverse teams in a mission - critical regulated environment.
Data Architect leading the definition and construction of cloud data architecture for Kyndryl. Participating in significant technological modernization initiatives, focusing on Google Cloud Platform.
Senior Data Engineer driving data intelligence requirements and scalable data solutions for a global consulting firm. Collaborating across functions to enhance Microsoft architecture and analytics capabilities.
Experienced AI Engineer designing and building production - grade agentic AI systems using generative AI and large language models. Collaborating with data engineers, data scientists in a tech - driven company.
Intermediate Data Engineer designing and building data pipelines for travel industry data management. Collaborating across teams to ensure reliable data for analytics and reporting.