Hybrid Principal Software Engineer – Data Platform Engineering

Posted 2 weeks ago

Apply now

About the role

  • Design and implement scalable data pipelines for high-volume IoT telemetry data
  • Analyze and optimize Spark jobs using execution plans, caching strategies, and memory tuning
  • Develop Python-based micro-services and reusable libraries to support data processing workflows
  • Work in a DevOps Agile/Scrum environment
  • Collaborate with product managers, data scientists, and software engineers to define system requirements and data integration needs
  • Build and manage infrastructure as code using Terraform
  • Create CI/CD pipelines for deploying and testing data workflows and services
  • Package and deploy applications in Docker containers
  • Apply strong testing practices
  • Participate in architectural discussions and technical design reviews

Requirements

  • Minimum of 10 years of software engineering experience
  • at least 3 years focused on data platform or data engineering roles
  • Proficient in Python
  • strong knowledge of PySpark and Apache Spark
  • Hands-on experience with Databricks and Spark SQL
  • Strong understanding of Spark performance tuning and query execution plans
  • Experience with Azure / AWS cloud services
  • Experience writing infrastructure-as-code using Terraform
  • Working knowledge of Docker and containerized deployment
  • Experience with CI/CD pipelines
  • Bachelor’s degree in Computer Science, Engineering, or related field

Benefits

  • financial, social, health and wellbeing programs
  • paid vacation
  • 401k matching
  • employee stock purchase program
  • hybrid work schedule
  • annual bonus program

Job title

Principal Software Engineer – Data Platform Engineering

Job type

Experience level

Lead

Salary

Not specified

Degree requirement

Bachelor's Degree

Location requirements

Report this job

See something inaccurate? Let us know and we'll update the listing.

Report job