AI Research Intern at NVIDIA focusing on multi - modal AI and vision - language model development. Collaborating with engineers to advance cutting - edge machine learning research in Vietnam.
About the role
- Lead the design, implementation, and execution of robust frameworks to evaluate the performance of generative AI systems, including text and multi-modal models.
- Establish and refine metrics and benchmarks for model quality, including output fidelity, diversity, creativity, and bias detection.
- Perform technical AI evaluations, benchmarking and “red-team” tests on large language models, including assessing them for robustness in performance, embedded biases, vulnerability to jailbreaks and prompt injection attacks.
- Work with clients and more junior team members to design custom evaluation approaches using the latest scientific research that address the client’s needs.
- Work with the product management team to develop a suite of technical and analytical AI evaluation frameworks and tools that are backed by scientific research and methods.
- Lead the design and implementation of evaluation frameworks for Large Language Models (LLMs), including but not limited to GPT-based models, BERT, T5, and other state-of-the-art architectures.
- Define and refine metrics for evaluating model performance, such as perplexity, BLEU, ROUGE, accuracy, coherence, factual consistency, and bias detection.
- Lead efforts in curating and managing large, high-quality datasets for evaluating LLMs, ensuring data is representative, unbiased, and ethically sourced.
- Mentor junior data scientists, guiding them in best practices for LLM evaluation and the latest advancements in NLP.
- Stay up-to-date with the latest advancements in Natural Language Processing (NLP) and LLM evaluation, applying cutting-edge methods and tools to improve model performance.
Requirements
- Extensive experience as a data scientist training or deploying deep learning based natural language models/large language models in real-world contexts.
- About 5-8 years of working experience or a relevant postgraduate degree with 2+ years of working experience building and deploying LLMs.
- Strong experience in evaluating LLMs using metrics such as perplexity, BLEU, ROUGE, and human-centered evaluation techniques.
- Proven track record of managing and analyzing large, complex language datasets, including text preprocessing and tokenization.
- Excellent written and verbal communication skills, with the ability to clearly explain complex technical concepts to diverse audiences, including non-technical stakeholders.
- Solid programming skills in Python and experience building automated pipelines for continuous model evaluation.
- Passion and interest in applied research on the safe and responsible use of AI and with large language models.
- NICE TO HAVE: Published research in the field of generative AI or model evaluation.
- Hands-on experience with model explainability tools and methods.
- Familiarity with cloud-based platforms (e.g., AWS, GCP) for scalable model evaluation and deployment.
Benefits
- Health insurance
- Professional development
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
AI Research Intern at NVIDIA developing generative models for biotechnology and computational science. Collaborating with a team to enhance AI model performance in drug discovery.
PI
PiecesApplied AI Scientist
Applied AI Scientist enhancing healthcare AI products with a focus on responsible AI and collaborative development. Utilizing advanced AI techniques to support healthcare providers' needs and streamline processes in patient care.
NO
Join Nokia Bell Labs' AI Accelerator Group as an intern. Contribute to innovative AI solutions for networks under expert guidance.
AI Research Engineer developing innovative learning systems at Campus. Leading research and collaboration to enhance student engagement through AI solutions.
RE
ResaroAI Scientist, LLM
Senior Data Scientist leading design and execution of evaluation frameworks for generative AI systems at Resaro. Focusing on large language models, applying scientific methods to ensure AI safety and effectiveness.
TI
Tri-global Solutions Group Inc.Senior AI Research Scientist – Human Behavior, Carbon Neutrality
AI Research Scientist at Toyota Research Institute developing generative AI models for understanding human behavior. Focused on machine learning and behavioral science for carbon neutrality research.
Senior Applied AI Scientist at Messagepoint developing generative AI solutions using large language models. Focused on transforming research into production systems for customer communications.
CL
Causeway Geotech LtdLead Data and AI Scientist
Lead Data & AI Scientist at Causeway building ML models and data platforms for construction software. Drive insights, implement data architectures, and collaborate with product, architecture, and engineering teams.
TS
Texas A&M University SystemAI Research Innovation Director
AI Research Innovation Director leading innovation and collaboration in AI research at TTI. Driving strategic partnerships and advancing AI applications in transportation domains.