Hybrid Staff AI Engineer – Orchestration

at Heidi Health

• Job title: Staff AI Engineer – Orchestration
• Job type: Full Time
• Experience level: Lead
• Salary: Not specified
• Degree requirement: Bachelor's Degree
• Tech skills: Kubernetes, Python, PyTorch, SQL
• Location requirements: Hybrid, Melbourne, Australia

About the role

• Staff AI Engineer designing and delivering clinician-grade AI solutions for healthcare. Collaborating across teams to enhance patient care and optimize workflows.

Responsibilities

• Define the end‑to‑end architecture for literature and guideline ingestion, normalization, metadata extraction, de‑duplication, and versioning.
• Build hybrid search and retrieval: lexical + vector + re‑ranking, with tight latency budgets and cost controls.
• Design grounding and answer synthesis that cite sources, preserve provenance, and expose confidence and abstention.
• Lead model work across prompting, fine‑tuning, distillation, and tool use to improve faithfulness, coverage, and utility.
• Stand up gold‑standard evaluation: offline IR metrics (nDCG, MAP, recall), factuality/faithfulness audits, and human review with adjudication.
• Run online experiments at scale. Define guardrails, KPIs, and ship A/Bs to measure impact on clinician workflows.
• Productionize services with observability, tracing, canaries, rollbacks, and incident playbooks.
• Set data governance for medical content: access control, PHI handling, audit logs, and retention policies.
• Partner with clinicians to define intents, schemas, and acceptance criteria. Convert ambiguous questions into testable specs.
• Coach engineers and scientists. Raise the technical bar through design docs, reviews, and reusable components.

Requirements

• Staff‑level track record shipping search, NLP, or LLM systems that serve real users at scale.
• Mastery of Python and SQL. Strong software engineering fundamentals, testing strategy, and API/service design.
• Depth in modern IR/NLP: embeddings, ANN indexes, re‑rankers, retrieval‑augmented generation, and prompt/program synthesis.
• Experience building data pipelines: parsing PDFs/HTML, OCR when needed, metadata extraction, and content hashing/versioning.
• Familiarity with PyTorch, plus distributed training/inference patterns.
• MLOps and reliability: containers, Kubernetes, feature/model registries, experiment tracking, monitoring, and alerting.
• Evidence of rigorous evaluation design: offline metrics, human‑in‑the‑loop judging, power analysis for online tests.
• Clear thinking on safety: hallucination controls, calibration, abstention, red‑teaming, and privacy/security by design.
• Ability to lead cross‑functional initiatives and make crisp decisions with incomplete information.

Benefits

• Flexible hybrid working environment, with 3 days in the office.
• A generous personal development budget of $500 per annum
• Learn from some of the best engineers and creatives, joining a diverse team
• Become an owner, with shares (equity) in the company, if Heidi wins, we all win
• The rare chance to create a global impact as you immerse yourself in one of Australia’s leading healthtech startups
• If you have an impact quickly, the opportunity to fast track your startup career!