Hybrid AI Agent Software Engineer – Agent Performance Engineering

at Assembled Products Corporation

• Job title: AI Agent Software Engineer – Agent Performance Engineering
• Job type: Full Time
• Experience level: Mid level, Senior
• Salary: $135,000 - $280,000 per year
• Degree requirement: Bachelor's Degree
• Location requirements: Hybrid, San Francisco, United States

About the role

• AI Agent Software Engineer developing intelligence systems for AI agents in customer support. Building foundational evaluation infrastructure and optimizing performance for enterprise-scale solutions.

Responsibilities

• Build foundational evaluation infrastructure: Develop comprehensive evaluation systems from the ground up, including golden dataset creation, automated benchmarking, and model comparison tools.
• Automate knowledge generation: Design and implement systems that automatically create synthetic guides, documentation, and metadata to improve agent knowledge bases.
• Optimize AI agent accuracy: Enhance our retrieval systems, implement advanced prompt optimization techniques, and build tools that continuously improve agent responses through automated evaluation and refinement.
• Develop intelligence infrastructure: Architect systems that enable rapid model upgrades, A/B testing of different AI approaches, and scalable evaluation pipelines that support enterprise deployment.
• Drive volume automation: Focus on maximizing automated resolutions across voice, chat, and email by building the intelligence systems that make it possible.

Requirements

• Have 5+ years of experience in software engineering as an individual contributor
• Have experience with AI evaluation systems, data pipelines, or AI model optimization
• Are passionate about building systems that measure and improve AI performance
• Have worked with retrieval systems, knowledge bases, or information extraction
• Enjoy building tools and infrastructure that enable other engineers and AI systems to perform better
• Are highly ambitious and driven, setting high goals for yourself and others
• Put customers first, focusing on solving real problems that impact support quality
• Enjoy fast-paced environments and can quickly adjust when new insights emerge
• Have a bit of a maverick streak that helps you come up with creative solutions
• Have made a noticeable impact on small teams and have solid experience in startups or smaller companies
• Stay humble and open to feedback, value teamwork, and are always ready to learn and grow

Benefits

• Generous medical, dental, and vision benefits
• Paid company holidays, sick time, and unlimited time off
• Monthly credits to spend on each: professional development, general wellness, Assembled customers, and commuting
• Paid parental leave
• Hybrid work model with catered lunches everyday (M-F), snacks, and beverages in our SF & NY offices
• 401(k) plan enrollment