Hybrid AI Evaluation Engineer

at Factorial

• Job title: AI Evaluation Engineer
• Job type: Full Time
• Experience level: Mid level, Senior
• Salary: Not specified
• Degree requirement: Postgraduate Degree
• Tech skills: Python, PyTorch, Tensorflow
• Location requirements: Hybrid, Ghent, Belgium

About the role

• AI Evaluation Engineer developing methodologies for assessing advanced AI systems' performance and reliability. Collaborating in a hybrid role in Ghent, Belgium.

Responsibilities

• Design and Develop Evaluation Frameworks: Create scalable, reproducible evaluation pipelines for large-scale AI systems, including LLMs and multi-agent architectures, covering both automated and human-in-the-loop testing strategies.
• Metric Innovation: Define and implement novel evaluation metrics that capture model capabilities beyond traditional benchmarks.
• Benchmarking & Performance Analysis: Conduct benchmarking of AI models across domains, tasks modalities, analyzing their skills and behavior under different setups.
• Safety, Reliability & Alignment Testing: Develop tools and experiments to probe model safety, robustness, interpretability, and bias.
• Cross-functional Collaboration: Work closely with model finetuning and optimization teams to evaluate end-to-end system effectiveness, efficiency. Identify trade-offs between model performance, latency, and energy footprint.
• Continuous Improvement & Reporting: Monitor model performance over time, automate regression detection, and contribute to the continuous evaluation infrastructure that supports Openchip’s AI research and product roadmap.

Requirements

• MSc or PhD in Computer Science, Artificial Intelligence, Machine Learning, Statistics, or a related field.
• A publication record in ML evaluation, benchmarking, or interpretability is a plus.
• 3+ years of experience developing, evaluating, or optimizing AI systems.
• Strong programming skills in Python, with experience using PyTorch, TensorFlow, or JAX.
• Experience in designing evaluation protocols for LLMs, multi-agent systems, or reinforcement learning environments.
• Deep understanding of ML metrics, evaluation methodologies, and statistical analysis.
• Experience with data quality, annotation workflows, and benchmark dataset creation is a plus.
• Fluent in English; proficiency in additional European languages (German, Dutch, Spanish, French, or Italian) is a plus.

Benefits

• The opportunity to build a cloud AI deployment platform that will power next generation AI systems.
• A collaborative, innovation-driven environment with significant autonomy and ownership.
• Hybrid work model with flexible scheduling.
• A chance to join one of Europe’s most ambitious companies at the intersection of AI and silicon engineering.