Dedicate your time to developing, testing, and maintaining high-quality, scalable ETL/ELT data pipelines using Python, PySpark, and Databricks on AWS
You will lead the development of new data features and services across the full product lifecycle, from architectural design to deployment including technologies like Terraform, spark structured streaming, dlt, etc.
Create CI/CD controlled data engineering deployments using Databricks Asset Bundles and leverage Databricks serverless compute with cost optimization techniques to deliver performant data products
Work within a fast-paced agile team, collaborating closely with Product Managers, Data Scientists, and fellow engineers to solve complex data challenges. You will help ensure the quality, integrity, and security of data throughout the data lifecycle, implementing best practices in data governance
Take technical ownership of major components of our data platform architecture. You will lead design discussions, ensuring our data systems are secure, scalable, and maintainable
Analyse workloads, queries, clusters/warehouses, jobs, and services to identify inefficiencies and savings opportunities
Implement continuous monitoring and intelligence dashboards for cost and performance optimization
Stay ahead of new Snowflake/Databricks features and translate them into marketable product differentiators
Requirements
Extensive experience with data engineering; **DataBricks & DBT** (Mandatory)
Proven expertise in platform internals:
**Databricks**: clusters, jobs, Delta Lake, Spark optimization, autoscaling, Unity Catalog
Strong **SQL + Python skills** for automation, monitoring, and recommendation logic
Experience with frameworks like **Terraform & Airflow**
Track record of delivering performance gains and cost savings through platform-level optimizations
Senior Associate Data Engineer contributing to Travelers' analytics landscape by building and operationalizing data solutions. Collaborating with teams to ensure reliable data delivery across the enterprise.
Salesforce Data Engineer serving as a subject matter expert in the State of Tennessee. Designing scalable data pipelines and collaborating on cross - agency initiatives.
Data Engineer Senior responsible for building data architecture and optimizing pipelines for Business Intelligence. Collaborating with analysts to develop insights using Power BI and Azure technologies.
Principal Data Engineer driving modernization from legacy systems to cloud - native platforms at Mastercard. Architecting and developing ETL platforms with AI integration and establishing data - driven strategies.
Principal Data Engineer modernizing cloud - native platforms for AI - powered solutions at Mastercard. Leading teams to enhance data processing efficiency and reliability across global operations.
Data Engineer creating data pipelines for Santander's card transactions. Collaborating with an agile team in strategic projects involving Databricks and PySpark.
Data Engineer designing, implementing, and maintaining data pipelines at Sabiá Gaming. Focused on high - quality data access and integration for enhanced decision - making.
Quantitative Data Engineer developing data solutions and automations for MassMutual's investment management. Working with data orchestration tools within a collaborative team environment.
Senior Data Engineer designing and scaling data infrastructure for analytics, machine learning, and business intelligence in a software supply chain security company.