Hybrid Senior QA Engineer – AI

Posted 2 weeks ago

Apply now

About the role

  • Design and execute comprehensive testing strategies specifically for AI and ML models, focusing on accuracy, robustness, and performance
  • Perform extensive testing to identify and mitigate biases or discriminatory outcomes in AI models, ensuring they adhere to ethical AI principles
  • Balance the use of manual and automated testing methods
  • Perform manual exploratory testing on complex, non-deterministic AI outputs (e.g., Large Language Models) while building and maintaining automated suites for consistent regression testing
  • Develop, maintain, and scale automated AI-based test cases using scripting languages such as Python or JavaScript
  • Integrate automated tests into CI/CD pipelines to enable continuous model evaluation and rapid delivery
  • Test data pipelines, feature engineering, and data quality to ensure models are trained on clean and reliable data
  • Create and execute test cases to challenge the AI model with edge cases and adversarial inputs to ensure safety and stability
  • Identify, document, and track software and AI model defects with detailed, reproducible steps
  • Work with development teams to prioritize and resolve issues
  • Work with DevOps, QA leadership to configure monitoring tools and define metrics for tracking post-deployment model health and detecting performance degradation or "model drift"
  • Stay up to date with emerging QA automation trends, AI testing methodologies, and tools

Requirements

  • Bachelor’s degree in computer science, Data Science, or related technical field
  • 10 years+ experience in a hybrid QA role, with a strong background in both manual and automated testing
  • Direct, hands-on experience testing AI applications or ML models in a production environment (preferred)
  • Strong skills with automation frameworks and tools such as Playwright, Selenium, or similar (preferred)
  • Proficiency in typescript, as well as Python being highly preferred due to its prevalence in ML (preferred)
  • Solid understanding of software quality assurance methodologies, testing tools, and processes within an Agile development lifecycle (preferred)
  • Excellent analytical and problem-solving skills with a keen eye for detail
  • Strong written and verbal communication skills, with the ability to articulate complex issues to both technical and non-technical stakeholders

Benefits

  • Medical, Dental, and Vision Insurance Options
  • Life and Disability Insurance
  • Paid Time-Off
  • Parental Benefits
  • Compassionate Care Leave
  • 401k with Company Match
  • Employee Stock Purchase Plan

Job title

Senior QA Engineer – AI

Job type

Experience level

Senior

Salary

$100,000 - $120,000 per year

Degree requirement

Bachelor's Degree

Location requirements

Report this job

See something inaccurate? Let us know and we'll update the listing.

Report job