Primary Metallurgical QA Professional ensuring compliance with metallurgical processes at Metallus. Overseeing chemical and testing application aspects of primary steelmaking for various products in a manufacturing plant.
About the role
- Senior QA Analyst focused on testing AI and ML-driven features ensuring quality and performance standards. Collaborates closely with engineering teams for cybersecurity software.
Responsibilities
- Design and execute test plans for AI and ML-driven features, including classification, summarization, anomaly detection and autofill systems
- Develop Python-based automation and evaluation scripts to validate model performance, reliability and output quality
- Test LLM-based features for accuracy, guardrail effectiveness, hallucination risk and behavior under edge cases
- Validate data pipelines, input/output transformations and model integration points across Keeper’s platform
- Conduct regression testing on ML models and AI workflows to ensure reliability across releases
- Test enterprise-scale behavior analysis features driven by data from ARAM, PAM and session recordings
- Validate security, privacy and zero-trust requirements for AI model interactions and data handling
- Collaborate with ML engineers and cross-functional teams to refine model behavior, identify defects and improve output quality
- Analyze logs, metrics and model outputs to detect anomalies, performance issues or silent failures
Requirements
- 3+ years of experience in QA, test automation or evaluation of complex systems
- Hands-on experience writing Python automation or evaluation scripts for testing APIs, model outputs or data pipelines
- Understanding of machine learning concepts such as classification, summarization, embeddings, anomaly detection or behavior analysis
- Experience testing LLM-integrated features or AI-driven workflows
- Familiarity with ML tooling such as Pytest, Hugging Face libraries, Jupyter Notebooks or similar
- Experience with API testing, JSON validation and test data generation
- Ability to test nondeterministic systems and evaluate model quality with structured methodologies
- Strong analytical, troubleshooting and communication skills
- Experience testing cloud-integrated applications or systems deployed on AWS
- Bachelor’s degree in Computer Science, Information Systems or related field, or equivalent practical experience
Benefits
- Medical, Dental & Vision (inclusive of domestic partnerships)
- Employer Paid Life Insurance & Employee/Spouse/Child Supplemental life
- Voluntary Short/Long Term Disability Insurance
- 401K (Roth/Traditional)
- A generous PTO plan that celebrates your commitment and seniority (including paid Bereavement/Jury Duty, etc)
- Above market annual bonuses**
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Senior QA engineer at Barco responsible for testing applications and ensuring quality in a collaborative team environment. Requires strong skills in automation and CI/CD tools.
Supplier Quality Engineer providing quality engineering support during product launch processes. Collaborating with suppliers and the USA quality engineer to ensure product quality and compliance.
QA Chapter Lead developing skills in testing, guiding a community of experts at Vodafone in high impact digital and core IT platforms. Inspiring knowledge transfer and innovation in testing practices.
Quality Assurance Associate at Lilly providing quality support for drug product manufacturing and batch disposition in pharmaceuticals. Ensuring adherence to quality systems for external manufacturers with a focus on compliance and continuous improvement.
Quality Engineer at Brink responsible for maintaining ISO 9001 standards and improving production processes. Work collaboratively within the quality team to support and advise on critical processes.
Quality Assurance Specialist overseeing production quality and compliance at Berghof Fluoroplastic Technology. Conducting inspections and continuous improvement initiatives in manufacturing.
Quality Assurance Engineer leading automation testing efforts for the Analytics Scrum Team at Hitachi Energy. Focused on developing frameworks and ensuring quality for analytical algorithms.
Quality Manager responsible for regular quality checks and problem resolution in flat glass production. Collaborating with teams to enhance product quality and ensure compliance with standards.
Quality Engineer responsible for implementing and supervising quality management systems at Tenneco in Swarzedz - Jasin, Poland. Conducting audits, maintaining documentation, and supporting training activities.