Quality Assurance Employee performing tests and quality checks for high - quality IC substrates at AT&S in Fehring. Involves document preparation and inspection tasks.
About the role
- Lead AI Quality Engineer responsible for quality, reliability, and trustworthiness of AI product experiences. Collaborating and designing tests for AI-powered solutions at Wolters Kluwer.
Responsibilities
- Design and implement evaluation harnesses to measure retrieval accuracy, citation correctness, response quality, and overall system behavior
- Develop automated tests for APIs, ingestion pipelines, and chat workflows
- Collaborate with developers and product managers to define quality metrics (accuracy, latency, cost, hallucination rate)
- Analyze logs, traces, and feedback signals to identify root causes of failures in AI-driven responses
- Create regression suites to ensure changes to prompts, chunking, or embeddings don’t break existing behavior
- Validate REST APIs and service integrations for resilience, correctness, and security
- Contribute to observability by instrumenting metrics and dashboards for system performance
- Participate in sprint planning and retrospectives, ensuring testability is built into features from day one
Requirements
- Bachelors Degree in Computer Science or equivalent
- 5+ years of experience in software testing, quality engineering, or equivalent engineering roles with a focus on validation and reliability.
- Experience with AI evaluation frameworks (e.g. LlamaIndex evals, OpenAI Evals, Ragas, TruLens, or custom harnesses)
- Strong skills in Python testing frameworks (Pytest, unittest, or equivalent)
- Experience testing web applications and APIs
- Familiarity with AI/ML or non-deterministic system testing
- Knowledge of CI/CD pipelines, Git, and automated regression testing
- Strong analytical skills: able to define metrics and success criteria where outputs aren’t deterministic
- Comfortable working in a fast-paced Agile environment with weekly sprints, pairing, and close collaboration with PM/UX/Dev
Benefits
- Medical, Dental, & Vision Plans
- 401(k)
- FSA/HSA
- Commuter Benefits
- Tuition Assistance Plan
- Vacation and Sick Time
- Paid Parental Leave
Job title
Job type
Experience level
Salary
Degree requirement
Tech skills
Location requirements
Mid - Level Quality Engineer at Boeing focusing on quality assurance and process improvement. Collaborating with various engineering functions to ensure compliance and improve overall quality performance.
Senior Manager Software Quality Engineering managing compliance and quality assurance for defense systems. Overseeing evaluations, risk assessments, and ensuring contractual adherence for digital products.
QA Engineer developing automated test scripts and supporting software releases in a global educational company. Collaborating with teams to ensure reliable applications and improve testing processes in hybrid work environment.
Quality Engineer responsible for leading quality assurance activities and compliance for medical devices. Collaborating across functions to uphold high standards of quality and regulatory compliance.
QA Manager overseeing Quality Assurance and Quality Control in a chemical manufacturing plant. Leading a team and ensuring compliance with quality regulations and standards.
Argus Japan Tester responsible for testing safety data management systems. Collaborating with cross - functional teams to maintain regulatory compliance and ensure high - quality deliverables.
Working Student in Quality Assurance at ANDRITZ Metals Forming supporting documentation of quality measures and collaborating with teams on quality assurance. Engaging in engineering practices for sustainable technologies.
Automation Tester at Jade Global developing and maintaining automated test scripts using Selenium and Java. Collaborating with teams to enhance test coverage and efficiency in a fast - paced environment.
Quality Assurance Tester contributing to GovTech solutions at SINC. Conducting software tests and documenting results for product quality enhancement.