Director of Software Engineering at Acuity leading AI - enabled digital commerce platform development and transforming user experience with modern architecture.
About the role
- Software Engineer in CrowdStrike's Data Engineering team, focused on building Spark ecosystems and scalable infrastructure. Collaborating with engineers to create solutions for cybersecurity challenges.
Responsibilities
- Design and implement extremely scalable large scale distributed Spark cluster on Kubernetes platform optimized for Data center environment.
- Perform and oversee a variety of functions to ensure that our Spark infrastructure on Kubernetes is available, reliable, stable and secure
- Applies judgment in analyzing and selecting technologies, installation and maintenance of software and hardware systems that allow multiple Engineering and DataScience teams to interact with this system
- Design, writes and own APIs, automate and generate reports to visualize cluster health.
- Has responsibility for building and maintaining the Spark infrastructure such that software engineers, data analysts and data scientists can run jobs to gather data and insights.
- Work on Query systems, like Apache Presto or Trino, Jupyter Notebooks
Requirements
- Strong in one or both of field i.e., Spark ecosystem or Kubernetes ecosystem
- Strong analytical skills and with deep understanding of Distributed Systems
- Strong programming skills in languages as Go, Java or Python
- Understanding of Apache Spark ecosystem technologies (Deltalake/Iceberg, YARN, HDFS, Kafka, Zookeeper, Hive, Jupyter notebooks)
- Experience with large-scale business critical platforms with Spark on Kubernetes on data center or cloud.
- Experience with continuous deployment on K8 with Helm, fluxCD etc
- Solid understanding of Spark(data) and K8 storage systems (Object stores S3/Minio, Spark ephemeral storage, Persistent volumes mapping and claims)
- Understanding or spark memory management and experience with spark internals.
- Familiarity with Chef is preferred.
- Proven ability to work with both local and remote teams
- Strong communication skills both verbal and written.
Benefits
- Market leader in compensation and equity awards
- Comprehensive physical and mental wellness programs
- Competitive vacation and holidays for recharge
- Paid parental and adoption leaves
- Professional development opportunities for all employees regardless of level or role
- Employee Networks, geographic neighborhood groups, and volunteer opportunities to build connections
- Vibrant office culture with world class amenities
- Great Place to Work Certified™ across the globe
Job title
Job type
Experience level
Salary
Degree requirement
Tech skills
Location requirements
Senior Product Engineer leading application and integration of protection and control solutions by Hubbell. Collaborating with engineering, sales, and customer support to deploy tailored technical solutions.
Software Engineer leading a team to develop high quality software solutions for DoD training systems. Supporting the JTSE program at Joint Staff Complex in Suffolk, VA.
Lead Principal Engineer Specialist at SAE facilitating aviation standards through technical management and collaboration. Recruiting and mentoring volunteers while driving continuous improvement initiatives in a hybrid work environment.
Product Engineer overseeing the technical lifecycle of screening and biomass handling products for Valmet. Collaborating with global teams and providing engineering expertise across the product lifecycle.
Lead ETL Developer responsible for ETL solutions involving data integration and automation. Working in a hybrid environment at Canada Life with a strong emphasis on collaboration.
Senior Software Engineer developing high - quality software solutions for Savanta. Collaborating with cross - functional teams in a hybrid work environment to deliver impactful products.
Technical Lead developing and evolving iTakeControl, a clinical trial patient engagement platform at Red Nucleus. Leading in - house product development with a focus on compliance and mentoring engineers.
Principal Software Engineer developing and enhancing secure software systems for Northrop Grumman's CHORD portfolio. Focused on collaboration, team empowerment, and personal responsibility in a complex technical environment.
Software Engineer developing Python applications on Linux for Northrop Grumman's Space Sector. Collaborating with cross - functional teams to deliver secure, scalable software in a SCIF environment.