AI Lead at Aspire Software identifying and building AI - powered solutions. Responsible for customer discovery, product validation, and delivering measurable impact.
About the role
- AI Engineer developing evaluation frameworks and observability for LLMs at Mistral AI. Collaborating with product teams to enhance AI models and ensure quality improvements.
Responsibilities
- Build and maintain an LLM evaluation framework (reference tests, heuristics, model-graded checks).
- Define and track metrics: task success, helpfulness, hallucination proxies, safety flags, latency/cost.
- Run A/B tests for prompts, models, and system prompts, analyze results, recommend rollout or rollback.
- Set up observability for LLM calls: structured logging, tracing, dashboards, alerts.
- Operate the model release: canary and shadow traffic, sign-offs, SLO-based rollback criteria, regression detection.
- Improve core behaviors: memory write/retrieve policies and evals, intent classification, follow-ups, routing, tool-call reliability.
- Create templates and docs so other teams can author evals and ship safely.
- Partner with Science, diagnose regressions and lead post-mortems.
Requirements
- Strong TypeScript or Python skills
- Production LLM experience: prompts, tool/function calling, and system prompts.
- Hands-on with evals and A/B testing, you can design metrics and make rollout decisions from data.
- Observability: logging, tracing, dashboards, alerting
- Product mindset: form hypotheses, run experiments, interpret results, iterate.
- Clear written and spoken communication, autonomous; and product-oriented.
- Now it would be ideal if you have experience with
- Safety systems: moderation, PII handling/redaction, guardrails.
- Release operations: canary/shadowing, automated rollbacks, experiment platforms.
Benefits
- 💰 Competitive salary and equity
- 🧑⚕️ Health insurance
- 🚴 Transportation allowance
- 🥎 Sport allowance
- 🥕 Meal vouchers
- 💰 Private pension plan
- 🍼 Parental : Generous parental leave policy
- 🌎 Visa sponsorship
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
AI Engineer developing and deploying AI/ML solutions using Azure technologies. Collaborating with cross - functional teams and implementing CI/CD pipelines in a responsible AI environment.
Principal AI Engineer at Humana focused on enterprise platform strategy and AI systems design. Leading development of agentic AI platforms and influencing architectural direction.
AI Developer Technology Engineer designing and optimizing GPU - accelerated workloads for financial markets at NVIDIA. Collaborating with technical experts to analyze and improve performance of complex AI applications.
AI Engineer at Maki building AI - powered talent acquisition and management solutions. Collaborating on AI features, evaluating models, and ensuring compliance in a hybrid role.
Head of ML/AI Platform at ZEIT Verlagsgruppe leading a new team in ML/AI development efforts. Requires expertise in cloud infrastructure, software development, and agile environments.
Director leading AI platform strategy and development at Cargill, serving employees globally with AI solutions. Overseeing the technical capabilities and guiding organizational innovation for AI.
Forward Deployed AI Engineer deploying generative models for pharmaceutical and biotech customers. Ensuring seamless integration and advocating for customer success through technical excellence and collaboration.
Senior AI Architect leading AI architecture strategy for Honeywell. Shape scalable AI platforms across ML, GenAI and agentic systems.
Associate Principal AI Engineer evolving AI platform capabilities and supporting AI adoption at Aegon. Building production - ready AI solutions while collaborating with diverse teams on business transformation initiatives.