Platform Engineer responsible for building and managing Kubernetes platforms for Spark workloads at Comcast. Collaborating with data teams to ensure efficient data processing and cost optimization.
Responsibilities
Architecting and managing platforms where Spark runs, such as Kubernetes clusters or cloud services
Packaging Spark workloads and integrating with orchestration systems like Apache Flyte
Deploying Infrastructure via Terraform/Ansible
Troubleshooting job failures and execution anomalies
Optimizing Spark configurations to reduce costs
Building automation and tools in languages like Python, Java, or Scala
Writing medium to complex SQL Queries
Implementing monitoring systems for platform stability
Working closely with data engineers and scientists
Engaging with open-source communities to discuss technical challenges
Creating and maintaining documentation for Kubernetes infrastructure
Providing training and support to team members
Requirements
Bachelor's degree in computer science or a related field, or equivalent experience
Expertise in Apache Spark
Proficiency in Python, Pyspark, and Scala/Java
Proficient in Linux Scripting (Bash)
Proficient in writing SQL
Experience in CI/CD tools, Github
Experience in using observability tools like Prometheus, Grafana
Knowledge of networking concepts like TCP/IP, DNS, Load Balancer
Automation via Terraform/Ansible
Hands-on experience with cloud providers (AWS, Azure, GCP)
Familiarity with technologies like Delta Lake, Apache Iceberg, Apache Kafka, Hadoop
Strong analytical skills for debugging complex issues
Software Developer enhancing the MERKURflow tool for adp MERKUR GmbH. Involvement in frontend and backend development, user interaction, and feature conception.
Lead Engineer for Product Development in Subsea Production Systems. Develop innovative engineering solutions and manage project timelines and budgets at Baker Hughes.
Full Stack Developer at Finnet, innovating financial solutions through web and mobile applications, collaborating with product teams and improving systems.
Lead design and development activities in a goal - oriented organization for West's healthcare product development. Demonstrate strategic leadership and innovation in medical device design and management.
Tech Lead focusing on AI and software engineering for strategic digital products at Livelo. Leading team evolution and quality delivery of impactful solutions.
Software Engineer developing NVLINK switch OS features for NVIDIA's next generation networking technologies. Engaging in coding, debugging, and customer integration in High - Performance Computing.
SDET responsible for building solutions to test applications across Web, Windows, Mobile, and API platforms. Collaborating with cross - functional teams to enhance product quality and delivery in an Agile environment.
Software Engineer II developing innovative software solutions for Vistra’s Global Expansion Platform team. Collaborating with stakeholders and engineers using Next.js, TypeScript, Node.js, and MySQL.
Software Engineer III developing software solutions for business operations in a hybrid role. Works independently on software development and collaborates with Agile teams to realize company goals.
Staff Fullstack Engineer enhancing complex architectural solutions for women's healthcare. Leading technical direction and mentoring engineers in a hybrid environment.