Senior Data Engineer at Yü Group responsible for leading data engineering team and optimizing data solutions within the energy sector.
About the role
- BI Data Engineer II supporting scalable Lakehouse data pipelines at Boston Beer Company. Collaborating with stakeholders to drive data ingestion and maintain enterprise data quality.
Responsibilities
- Serve as a Databricks-first data engineer, designing, building, and supporting scalable Lakehouse data pipelines (ETL/ELT) using Spark and Delta Lake.
- Develop and maintain Databricks notebooks, workflows, and jobs, with secondary support for On-Premise SQL Server integrations.
- Build analytics-ready data models in Databricks, supporting limited dimensional and reporting models on SQL Server as needed.
- Drive data ingestion, migration, and integration between Databricks and On-Premise systems as part of platform modernization.
- Collaborate with analysts, data scientists, business stakeholders, and all teams under the Data Enterprise organization (including Data Operation and MDM) to translate requirements into scalable, maintainable solutions
- Optimize Spark jobs and Databricks workflows for performance, reliability, and cost efficiency.
- Implement enterprise data quality checks, security best practices, and governance standards across Databricks and SQL environments.
- Support production pipelines with CI/CD and automated deployment for reliable, repeatable releases.
Requirements
- Bachelor's Degree in Computer Science, or a closely related field, or the equivalent
- Databricks and Spark: 2–5 years of hands‑on experience with Databricks notebooks, Spark transformations, and Delta Lake for scalable
- Strong Python, Scala, and SQL skills for building and maintaining data pipelines.
- Exposure to automated deployment pipelines using tools such as Azure DevOps and Databricks Asset Bundles (DABs) for reliable, production-ready workflows.
- Understanding of enterprise data quality checks, security practices, and governance frameworks.
- Ability to work effectively with analysts, stakeholders, and cross-functional teams to troubleshoot and optimize pipelines.
Benefits
- Generous healthcare on day one
- Stock purchase plan
- 401k
- Tuition reimbursement
- Fertility/adoption support
- Free financial coaching
- Health & wellness program and discounts
- Professional development & training
- Free beer!
Job title
Job type
Experience level
Salary
Degree requirement
Tech skills
Location requirements
Data Engineer managing payment processing and data accuracy while collaborating with financial teams. Building and optimizing data pipelines for transactional data in a hybrid work environment.
Data Engineer building analytical tools for Dry Bulk market data operations at Kpler. Join a team of over 700 experts transforming data into actionable strategies.
Data Engineer developing tools for maintaining data integrity in cargo tracking at Kpler. Collaborating with analysts and engineers to enhance data quality management.
Lead Azure Data Engineer designing and optimizing data ecosystems on Microsoft Cloud. Responsible for building scalable data platforms and pipelines for analytics and reporting.
Data Engineer providing support for IBM DataStage ETL jobs at Callibrity. Collaborating with stakeholders and working to modernize technology solutions in a hybrid work environment.
Cloud Data Engineer implementing tailored solutions for Volkswagen Group data processing. Building ETL/ELT pipelines while collaborating with technical experts.
Data Engineer responsible for building scalable data infrastructure that supports data - driven decisions. Collaborating with team to maintain systems and unlock data value for organizations.
Data Engineer designing and optimizing data pipelines using Databricks and Google Cloud Platform. Collaborating with analysts and scientists to deliver high - quality data products.
Associate Data Engineer supporting privacy engineering controls and executing privacy impact assessments in a financial services company. Collaborating across business units to ensure alignment with privacy regulations.