Seeking a Senior/Lead Platform Engineer responsible for architecting and implementing scalable data and ML platforms. Focusing on AWS and Databricks, while leading DevSecOps practices.
Responsibilities
Architect and implement end-to-end data and ML platforms: data lakes, warehouses, streaming and batch pipelines, model training/deployment infrastructure, on AWS + Databricks.
Lead DevSecOps and DataOps practices: infrastructure as code (IaC), CI/CD pipelines for data & ML workflows, secure multi-account/multi-region cloud operations.
Integrate AWS services (e.g., S3, Redshift, Kinesis, Lambda, EKS/ECS) with Databricks runtime, Delta Lake, Unity Catalog etc to build scalable, performant pipelines.
Build and operate ML infrastructure: training clusters, model versioning, MLOps toolchain (e.g., MLflow), model monitoring and observability, automatic retraining workflows.
Establish data governance, lineage, quality, observability standards across data pipelines and ML workflows.
Mentor engineering teams, define architectural best practices and guide implementation of high-scale data/ML systems.
Optimize system performance, cost and scalability; diagnose and resolve large-scale production issues.
Continuously evaluate new tools and technologies in the areas of cloud, data platform, DevSecOps, ML infrastructure and apply them to drive innovation.
Requirements
7+ years of experience in data platform architecture, cloud/ML infrastructure engineering or related roles.
Deep technical expertise in **Databricks and AWS**: demonstrated ability to design, integrate and operate solutions spanning both platforms.
Strong hands-on implementation skills: you will not just design but build, deploy and operate the platform.
Proven track record of building and operating scalable ML/AI platforms in production (model training & deployment).
Expertise in Apache Spark, Delta Lake, modern data pipeline frameworks (batch + streaming).
Strong background in infrastructure as code (Terraform, CloudFormation), CI/CD for data/ML, and DevSecOps practices.
Proficiency in Python and SQL; familiarity with Scala or equivalent is a plus.
Experience with data governance, data lineage, observability and MLOps frameworks (e.g., MLflow, Airflow, dbt).
Bonus: Experience in fintech, regulated industries or high-security environments.
Support with architecture, design, and implementation of Kubernetes environments. Involving in CI/CD pipelines, multi - cloud orchestration, and providing relevant content for clients.
Platform Engineering Manager leading engineering of Anglian Water’s hybrid digital platforms. Focusing on secure and scalable cloud and on - premise infrastructure while enabling digital service delivery.
Platform Engineer responsible for maintaining uptime and stability of robot testing platforms. Collaborating with teams for high reliability in testing environments for autonomous vehicles.
Platform Engineer for Pfizer’s Data and AI Platforms team, developing Azure solutions and pro - code AI agent applications. Leading engineering and operations for a scalable enterprise - grade platform.
Senior Embedded Platform Engineer developing low - level embedded software for Ford's Electric Vehicles and the future of transportation. Collaborating with agile teams to ensure functionality and efficiency.
Senior IT Engineer enhancing cloud platform and infrastructure reliability at Xcel Energy. Collaborating with teams to influence platform strategy and deliver high - impact capabilities.
Platform Engineer developing Kubernetes solutions supporting multi - tenant platforms at Bundesdruckerei in Berlin. Collaborating on innovative digital solutions for identity and data protection.
Lead Platform Engineer at PGIM Private Capital focusing on cloud modernization. Collaborate with cross - functional teams to develop cloud - based applications in a hybrid work environment.
AI Platform Engineer designing and deploying AI and ML platforms at Utica National Insurance Group. Collaborating with internal teams to implement AI solutions and establish observability and telemetry.
Head of Platform Engineering at Flutter leading core infrastructure services and cloud platforms. Focused on modernizing systems and improving reliability for critical operations.