Hybrid Senior Manager, AI Inference

at Red Hat

• Job title: Senior Manager, AI Inference
• Job type: Full Time
• Experience level: Senior
• Salary: $210,730 - $358,070 per year
• Degree requirement: Bachelor's Degree
• Tech skills: Kubernetes, Open Source, Python, PyTorch
• Location requirements: Hybrid, Boston, United States

About the role

• Senior Engineering Manager at Red Hat leading machine learning engineering teams and setting strategic objectives. Fostering innovation and collaboration in AI Inference and open-source technologies.

Responsibilities

• Setting the overarching vision, objectives, and strategies for a group of engineering teams and engineering managers.
• Lead and inspire a distributed team of individual contributors and managers, fostering a collaborative and innovative work environment.
• Engage with the AI and machine learning open source communities such as vLLM, llm-d, and other open source communities.
• Work with product management and engineering teams to develop technology roadmaps and schedules, and communicate these schedules externally.
• Work with cross-functional engineering managers and teams on documentation, product management, and quality assurance to coordinate tasks necessary for releasing enterprise-quality MLOps software.
• Working closely with the technical leads and scrum leads to direct the team in agile development.
• Meet and present technical presentations to customers and partners as necessary.
• Mentor and nurture team members in their career development, and professional growth.
• Recruit and build a world class engineering team.

Requirements

• 10+ years of significant hands-on software development and system design experience
• 5+ years of experience in managing software engineering teams
• Proven experience in leading machine learning engineering teams, with a track record of successful project delivery and development of software engineers and engineering managers
• Experience in machine learning frameworks and tools, such as PyTorch, and HuggingFace.
• Extremely proficient in running and analyzing LLM performance and accuracy benchmarks across a variety of accelerators.
• Excellent programming skills in languages like Python, and C++/CUDA.
• Experience with developing and scaling applications with Kubernetes.
• Experience using AI-assisted development tools (e.g., GitHub Copilot, Cursor, Claude Code) for code generation, and a multi-agent driven orchestration and automation to accelerate development cycles and enhance code quality.
• Excellent written and verbal communication skills.
• Ability to lead and work with diverse and distributed teams from multiple countries and cultures.

Benefits

• Comprehensive medical, dental, and vision coverage
• Flexible Spending Account - healthcare and dependent care
• Health Savings Account - high deductible medical plan
• Retirement 401(k) with employer match
• Paid time off and holidays
• Paid parental leave plans for all new parents
• Leave benefits including disability, paid family medical leave, and paid military leave
• Additional benefits including employee stock purchase plan, family planning reimbursement, tuition reimbursement, transportation expense account, employee assistance program, and more!