Data Engineer role focused on creating and monitoring data pipelines in an innovative energy company. Collaborate with IT and departments to ensure quality data availability in a hybrid work environment.
About the role
- Lead Data Engineer responsible for designing and building AI data foundations in Ad Tech at Disney. Collaborate with cross-functional teams to enhance data architecture and pipeline reliability.
Responsibilities
- Design and build streaming and batch pipelines for AI applications, ensuring low-latency and high-throughput data flows.
- Implement embedding pipelines and vector store integrations to support retrieval, semantic search, and AI assistants.
- Optimize storage and retrieval patterns for scalability, compliance, and cost efficiency.
- Own the design of AI-optimized data stores that align with enterprise data architecture.
- Partner with the AI Core Engineering team to provide reliable pipelines into shared agents, registries, and services.
- Develop reusable data ingestion and transformation frameworks for cross-team adoption.
- Build observability into data pipelines with monitoring, alerting, and dashboards for latency, drift, and failures.
- Implement data quality checks, schema validation, and governance controls.
- Ensure pipelines meet enterprise standards for compliance, audit, and resilience.
- Mentor engineers, conduct design/code reviews, and enforce data engineering best practices.
- Collaborate with infra, product, and governance teams to align on priorities and safe adoption of AI.
- Drive delivery of high-profile AI applications by balancing execution speed with system reliability.
Requirements
- 7+ years of data engineering experience, with at least 2 years in a lead or senior technical role.
- Experience building and scaling streaming data pipelines in large-scale, distributed environments.
- Strong skills in Python, Java and SQL with expert level skill in either Python or Java.
- Proven experience building streaming data pipelines (e.g., Kafka, Flink, Spark, Kinesis).
- Experience with embedding pipelines and vector stores (e.g., Pinecone, Weaviate, FAISS, pgvector).
- Strong knowledge of data modeling, storage optimization, and retrieval patterns for large-scale systems.
- Hands-on experience with workflow orchestration tools (Airflow, Dagster, etc.).
- Strong collaboration and communication skills, able to partner across AI engineering, infra, and product teams.
Benefits
- A bonus and/or long-term incentive units may be provided as part of the compensation package
- Full range of medical, financial, and/or other benefits, dependent on the level and position offered
Job title
Job type
Experience level
Salary
Degree requirement
Tech skills
Location requirements
SQL Migration Data Engineer at Auxo Solutions focusing on Azure SQL/Fabric Lakehouse migrations and building data pipelines. Collaborating on technical designs and data governance for modernization initiatives.
Data Engineer developing cloud solutions and software tools on Microsoft Azure big data platform. Collaborating with various teams for data analysis and visualization in healthcare.
Boomi Integration Architect designing and leading integration solutions for data warehouses. Collaborating with cross - functional teams to implement scalable integration patterns using Boomi technologies.
Seeking a Boomi Integration Architect specializing in Data Warehouse and Master Data Hub implementations. Responsible for designing high - performance integration solutions across enterprise platforms.
Principal Data Engineer at Serko enhancing global travel tech through data - driven solutions. Collaborating across teams in Bengaluru to drive innovative engineering and best practices.
Data Engineer at Keyrus responsible for building and optimizing data pipelines for major projects. Contributing to data solutions and ensuring data quality in a growing team.
Data Architect designing and implementing scalable data architectures for Keyrus in Bordeaux. Leading client transitions and contributing to the tech ecosystem with innovative data solutions.
Lead Data Engineer responsible for leading data engineering projects within the AI & Data Engineering team in Pune, India.
Data Engineer utilizing Databricks and AWS for scalable data solutions. Collaborating on analytics with business teams at Travelers.