R&D Engineer overseeing Biologics and Regenerative Medicine project plans at Smith+Nephew. Collaborating across teams for innovation and product development in healthcare technology.
TL
Hybrid Senior Azure Databricks Engineer
Posted last month
About the role
- Senior Azure Databricks Engineer at AHEAD building platforms for digital business. Architecting and optimizing Azure Databricks solutions and data pipelines for enterprise clients.
Responsibilities
- Architect, build, and optimize **Azure Databricks** solutions (batch + streaming) using **Apache Spark**.
- Design Lakehouse data models and curation layers using **Delta Lake** (Bronze/Silver/Gold), including SCD patterns where applicable.
- Lead end-to-end development of **ETL/ELT** pipelines integrating sources like APIs, RDBMS, SaaS apps, files, and event streams.
- Implement orchestration and scheduling using **Databricks Workflows** and/or **Azure Data Factory** (or Airflow where relevant).
- Drive performance and cost optimization: cluster policies, autoscaling, photon/runtime selection, partitioning, file sizing, job tuning.
- Establish best practices for code quality: modular PySpark/SQL, reusable libraries, unit/integration testing, and peer reviews.
- Implement CI/CD for Databricks (Repos/Git), automated deployments, environment promotion, and release governance.
- Partner with security/governance teams to implement access controls, secrets, encryption, and data governance (e.g., **Unity Catalog**).
- Create operational excellence: monitoring, alerting, runbooks, incident response, and root-cause analysis.
- Mentor engineers, set technical direction, and contribute to roadmap planning and estimation.
Requirements
- 6+ years of data engineering experience, including strong hands-on delivery with **Azure Databricks**.
- Expert-level **Spark** experience (PySpark and/or Scala) and strong **SQL**.
- Proven experience building **production-grade Delta Lake / Lakehouse** architectures and data pipelines.
- Strong experience with Azure services such as **ADLS Gen2**, **Azure Data Factory**, **Event Hubs/Kafka**, **Synapse**, **Key Vault**, and Azure networking/security concepts.
- Demonstrated ability to tune Spark workloads (joins/shuffles, partitioning, caching, skew handling, file compaction).
- Experience with **Git-based development** and CI/CD (Azure DevOps or GitHub Actions), plus automated testing approaches.
- Strong communication skills and ability to lead technical discussions, influence decisions, and translate requirements into designs.
Benefits
- Employment Benefits include:
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
No Education Requirement
Tech skills
Location requirements
CPU verification engineer at Intel developing validation strategies for CPU logic designs. Collaborating with architects and designers to ensure rigorous pre - silicon verification methodologies.
Senior CPU Verification Engineer at Intel driving and executing comprehensive verification plans for CPU logic designs. Collaborating with architects and designers to ensure functional correctness of designs in a rigorous environment.
Senior CPU Pre - Silicon Verification Engineer at Intel creating world - changing technology. Driving functional verification of CPU logic for advanced semiconductor solutions.
Installation Engineer delivering Mechanical Fire Suppression Systems across commercial and industrial environments. Providing high - quality installations and exceptional customer service with a proactive approach to safety.
Staff RF Engineer developing hardware solutions including RF design and device verification at NETGEAR. Involves complex telecommunications and wireless systems work with extensive experience required.
Engineer delivering scalable systems and refactoring monolithic architectures for millions of users. Collaborating with cross - functional teams in a hybrid working environment.
Ingénieur méthodes et logistique H/F dans un groupe logistique, assisté de l'ingénieur méthodes sur des projets d'amélioration continue.
Process Engineer planning and optimizing machining processes in aerospace and automotive sectors. Responsible for generating engineering documents and improving manufacturing efficiency with minimal supervision.
Field Service Engineer responsible for installation and maintenance of medical technologies in France. Provide technical support and user training for innovative healthcare solutions.