Hybrid Senior Data Engineer

at Acuity

• Job title: Senior Data Engineer
• Job type: Full Time
• Experience level: Senior
• Salary: $120,800 - $217,400 per year
• Degree requirement: Bachelor's Degree
• Tech skills: Akka, Apache, Azure, Cloud, Docker, IoT, Java, Kafka, Kubernetes, Postgres, PySpark, Python, PyTorch, Scala, Spark, Tensorflow, Terraform
• Location requirements: Hybrid, Atlanta, United States

About the role

• Senior Data Engineer focusing on building intelligent data platforms for IoT analytics at Acuity Brands. Collaborating with cross-functional teams to enhance AI/ML applications and optimize operations.

Responsibilities

• Play a critical role in architecting and developing pipelines that transform raw IoT data into actionable intelligence
• Build robust data models that ensure data quality and consistency, enabling downstream analytics and AI/ML applications that optimize operations, enhance user experiences, and unlock new business opportunities
• Design and implement scalable data engineering pipelines for ingesting, transforming, and storing IoT telemetry data using Apache Flink, Apache Spark, and Databricks
• Build and maintain time-series data solutions using PostgreSQL and TimescaleDB to support high-resolution telemetry analytics
• Integrate data governance frameworks for metadata management, lineage tracking, and compliance within a data lake ecosystem
• Apply performance tuning techniques in Databricks to optimize batch processing speed and resource utilization
• Design and implement Infrastructure as Code (IaC) solutions using Terraform and Azure Bicep to provision and manage cloud resources across multi-cloud environments
• Optimize and monitor pipeline performance and resource utilization across distributed environments such as Kubernetes and Databricks clusters
• Leverage telemetry and diverse data sources to design, test, and deploy AI/ML solutions, collaborating with data scientists and engineers to build deep learning capabilities within the platform for prediction, early alerting, and prescriptive recommendations
• Define appropriate business metrics to measure the impact of AI/ML solutions within the platform
• Write scalable, distributed, and highly efficient code, in languages such as Python, Java, PySpark, Scala, and R

Requirements

• A BS in Computer Science, Statistics, Mathematics, or a related field
• 5+ years of experience in building end-to-end analytics solutions, including designing and implementing real-time and batch data pipelines for IoT telemetry or time-series data
• Excellent problem-solving, critical thinking, and communication skills
• Demonstrated initiative to find solutions to complex problems at scale and operationalize them
• Demonstrated ability to work in ambiguous situations and across organizational boundaries
• Lead with respect, accountability, integrity, and a positive can-do attitude
• Experience working in a data-intensive environment and translating business needs into data requirements
• 2+ years of experience in building and operationalizing analytics pipelines and services, adopting the container ecosystem of Docker and Kubernetes
• 2+ years of demonstrated AI/ML pipeline development with relevant code experience
• 2+ years’ experience using one or more of the following: TensorFlow, MLFlow, PyTorch, SparkML, etc.
• 2+ years’ experience using one or more of the following frameworks: Apache Flink, Spark, Spark (structured) Streaming, Akka, Kafka, etc.
• 2+ years’ experience building scalable batch data pipelines on cloud-based platforms, such as Databricks

Benefits

• Generous benefits including health care
• Dental coverage
• Vision plans
• 401K benefits
• Commissions/incentive compensation depending on the role