Hybrid Software Engineer, Gen AI Platform

at Abridge

• Job title: Software Engineer, Gen AI Platform
• Job type: Full Time
• Experience level: Mid level, Senior
• Salary: $162,000 - $234,000 per year
• Degree requirement: Bachelor's Degree
• Tech skills: Java, Kubernetes, Python, Go
• Location requirements: Hybrid, San Francisco, United States

About the role

• AI Platform Engineer collaborating closely with researchers and product engineers on healthcare AI-powered products. Designing and building runtime, orchestration engine, and evaluation platform for workflows.

Responsibilities

• Design and build GenAI systems that turn LLMs into composable, dependable tools—leveraging retrieval, tool use, agentic reasoning, and structured outputs.
• Design and implement a highly reliable and scalable agent runtime: orchestration, shared state and memory, tool-calling interfaces, and scheduling for cost, latency, and quality.
• Build secure, sandboxed execution for agent actions and code; optimize cold start, isolation, and observability.
• Ship unified interfaces for multiple model sizes and providers; integrate with open tool ecosystems such as MCP-style connectors for data and actions.
• Develop an evaluation platform for online and offline assessments, A/B tests, safety checks, and regression gates that improve agent reliability over time.
• Partner with Research to deliver new agent capabilities end to end—from prototype to production.

Requirements

• Experience building agent applications with tool-calling, context engineering, or open connector integrations.
• Fluency with LLM APIs, prompting strategies, and orchestration patterns (e.g., LangChain, LlamaIndex, or custom pipelines).
• Experience with retrieval systems (e.g., semantic and lexical retrieval, vector DBs, efficient kNN), function calling, tool use, or agentic workflows.
• Strong coding skills in one or more of: Python, Java, Go. Comfortable with service design, APIs, and data models for high-throughput systems.
• Working knowledge of containers and kubernetes concepts. Familiarity with metrics, tracing, on-call rotations, and incident response practices.
• Self-motivated with a willingness to take ownership.
• Strong communication skills and ability to work collaboratively in a team environment.

Benefits

• Generous Time Off: 13 paid holidays, flexible PTO for salaried employees, and accrued time off for hourly employees.
• Comprehensive Health Plans: Medical, Dental, and Vision plans for all full-time employees. Abridge covers 100% of the premium for you and 75% for dependents. If you choose a HSA-eligible plan, Abridge also makes monthly contributions to your HSA.
• Paid Parental Leave: 16 weeks paid parental leave for all full-time employees.
• 401k and Matching: Contribution matching to help invest in your future.
• Pre-tax Benefits: Access to Flexible Spending Accounts (FSA) and Commuter Benefits.
• Learning and Development Budget: Yearly contributions for coaching, courses, workshops, conferences, and more.
• Sabbatical Leave: 30 days of paid Sabbatical Leave after 5 years of employment.
• Compensation and Equity: Competitive compensation and equity grants for full time employees.
• ... and much more!