Product Engineer working on air handling units at Johnson Controls. Engaging in engineering work and recommending solutions for product design and development.
About the role
- Senior Software Engineer developing enterprise-scale data platforms with Apache Airflow and Apache Spark for global sports technology company.
Responsibilities
- Partner with data engineering teams to understand requirements and translate them into data platform capabilities
- Build tooling and automation to improve developer experience and reduce operational overhead
- Establish monitoring, alerting, and observability practices for platform health and performance
- Drive platform upgrades, migrations, and adoption of new technologies
- Own and maintain the Apache Airflow orchestration platform, ensuring reliable DAG execution for data teams across the organization
- Manage and optimize the Apache Spark platform on EMR on EKS, supporting large-scale data processing workloads
- Design and implement platform improvements to enhance reliability, scalability, and performance
- Troubleshoot complex distributed systems issues and provide timely resolution for platform incidents
- Optimize resource utilization and cost efficiency across AWS infrastructure
- Create and maintain technical documentation, runbooks, and best practices
- Participate in on-call rotation and incident response to ensure platform availability
- Conduct capacity planning and scaling to accommodate organizational growth
Requirements
- 5+ years of software engineering experience with a focus on data infrastructure, platform engineering, and distributed systems
- Strong proficiency in at least one programming language - Python/Go/Scala
- Deep understanding of Apache Spark architecture and experience running Spark workloads in production
- Solid experience with AWS services, particularly EMR, EKS, S3 etc.
- Experience with Data Platform infrastructure on Kubernetes and container orchestration
- Strong understanding of data engineering concepts, workflows, and best practices
- Proven ability to debug complex issues in distributed systems
- Knowledge of data quality frameworks and tooling (Great Expectations, Deequ, Monte Carlo, etc.)
- Experience with data lineage tools and metadata management (Apache Atlas, DataHub, OpenLineage, etc.)
- Understanding of data governance principles and compliance requirements
- Experience with Apache Airflow setup
- AWS Certification is a plus (but not required)
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Staff Software Engineer leading the design and development of an AI - powered Banker Workbench feature for CBA. Focused on front - end leadership and modernizing banking technology.
Intern role in software engineering at Airwallex providing hands - on project experience and personal mentorship while collaborating with innovative team.
Software Engineer developing and implementing automation systems at Actemium Controlmatic. Collaborating in interdisciplinary teams and supporting project execution in Berlin.
Software Engineer developing software for thermal management systems on GM’s electric vehicles. Collaborating in an Agile team responsible for control and diagnostics software development.
Senior Software Engineer developing APIs and microservices for GM's Loyalty systems. Leading design and project implementation while ensuring best practices in software development.
Lead Development Engineer collaborating on full stack solutions for control room software at Barco. Drive quality product increments and mentor team members across global locations.
Staff Software Engineer leading technical direction for patient interactions in a healthcare startup. Growing with a mission - driven team focused on transforming patient care through technology.
Embedded Software Developer at Capgemini Engineering focusing on OS configuration for Qualcomm SoCs. Responsibilities include device driver implementation and hardware diagnostics in a collaborative environment.
Senior Software Engineer leading EDI integrations and API projects at Capgemini, empowering organizations by building sustainable technology solutions.