AI Research Intern at NVIDIA focusing on multi - modal AI and vision - language model development. Collaborating with engineers to advance cutting - edge machine learning research in Vietnam.
About the role
- Lead the design, implementation, and execution of robust frameworks to evaluate the performance of generative AI systems, including text and multi-modal models
- Establish and refine metrics and benchmarks for model quality, including output fidelity, diversity, creativity, and bias detection
- Perform technical AI evaluations, benchmarking and “red-team” tests on large language models to assess robustness, embedded biases, vulnerabilities
- Work with clients and junior team members to design custom evaluation approaches
- Develop a suite of technical and analytical AI evaluation frameworks and tools assessing robustness, explainability, fairness, privacy, safety, and security of AI
- Lead design and implementation of evaluation frameworks for Large Language Models (LLMs)
- Define and refine metrics for evaluating model performance
- Curate and manage large, high-quality datasets for evaluating LLMs
- Mentor junior data scientists in best practices for LLM evaluation
- Stay up-to-date with the latest advancements in Natural Language Processing (NLP) and LLM evaluation
Requirements
- Extensive experience as a data scientist training or deploying deep learning based natural language models/large language models in real-world contexts
- About 5-8 years of working experience or a relevant postgraduate degree with 2+ years of working experience building and deploying LLMs
- Strong experience in evaluating LLMs using metrics such as perplexity, BLEU, ROUGE, and human-centered evaluation techniques
- Proven track record of managing and analyzing large, complex language datasets, including text preprocessing and tokenization
- Excellent written and verbal communication skills, with the ability to clearly explain complex technical concepts to diverse audiences, including non-technical stakeholders
- Solid programming skills in Python and experience building automated pipelines for continuous model evaluation
- Passion and interest in applied research on the safe and responsible use of AI and with large language models.
Benefits
- Professional development opportunities
- Flexible work arrangements
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
AI Research Intern at NVIDIA developing generative models for biotechnology and computational science. Collaborating with a team to enhance AI model performance in drug discovery.
PI
PiecesApplied AI Scientist
Applied AI Scientist enhancing healthcare AI products with a focus on responsible AI and collaborative development. Utilizing advanced AI techniques to support healthcare providers' needs and streamline processes in patient care.
NO
Join Nokia Bell Labs' AI Accelerator Group as an intern. Contribute to innovative AI solutions for networks under expert guidance.
AI Research Engineer developing innovative learning systems at Campus. Leading research and collaboration to enhance student engagement through AI solutions.
RE
ResaroSenior AI Scientist, LLM
Senior AI Scientist at Resaro, developing AI evaluation frameworks for generative AI systems. Engaging in deep learning applications, mentoring, and enhancing model performance metrics.
TI
Tri-global Solutions Group Inc.Senior AI Research Scientist – Human Behavior, Carbon Neutrality
AI Research Scientist at Toyota Research Institute developing generative AI models for understanding human behavior. Focused on machine learning and behavioral science for carbon neutrality research.
Senior Applied AI Scientist at Messagepoint developing generative AI solutions using large language models. Focused on transforming research into production systems for customer communications.
CL
Causeway Geotech LtdLead Data and AI Scientist
Lead Data & AI Scientist at Causeway building ML models and data platforms for construction software. Drive insights, implement data architectures, and collaborate with product, architecture, and engineering teams.
TS
Texas A&M University SystemAI Research Innovation Director
AI Research Innovation Director leading innovation and collaboration in AI research at TTI. Driving strategic partnerships and advancing AI applications in transportation domains.