Where is the Research Scientist - Vision Language Model role at Ifm Us located?

The Research Scientist - Vision Language Model position at Ifm Us is based in Sunnyvale, CA. This is an on-site position.

What type of employment is the Research Scientist - Vision Language Model role at Ifm Us?

The Research Scientist - Vision Language Model position at Ifm Us is a Full time role.

What skills are required for the Research Scientist - Vision Language Model role at Ifm Us?

Key skills for the Research Scientist - Vision Language Model position at Ifm Us include: deep learning, machine learning, natural language processing.

About the Institute of Foundation Models

We are a dedicated research lab for building, understanding, using, and risk-managing foundation models. Our mandate is to advance research, nurture the next generation of AI builders, and drive transformative contributions to a knowledge-driven economy.

As part of our team, you’ll have the opportunity to work on the core of cutting-edge foundation model training, alongside world-class researchers, data scientists, and engineers, tackling the most fundamental and impactful challenges in AI development. You will participate in the development of groundbreaking AI solutions that have the potential to reshape entire industries. Strategic and innovative problem-solving skills will be instrumental in establishing MBZUAI as a global hub for high-performance computing in deep learning, driving impactful discoveries that inspire the next generation of AI pioneers.

Position Summary

As a Research Scientist in the Vision Language Model (VLM) team, your role will be central to advancing state-of-the-art multimodal foundation models that integrate visual understanding, reasoning, and agentic capabilities. You will work on the research and development of large-scale VLM systems, spanning model architectures, data recipes for pre-training and post-training, and evaluation benchmarks. The role combines cutting-edge research with practical engineering, emphasizing large-scale data processing, filtering, and weighting pipelines, distributed training systems, and reinforcement learning algorithms and systems for multimodal reasoning and agent development.

Key Responsibilities

Research and development of next-generation Vision Language Models across pre-training, instruction tuning, reasoning, and agents.
Develop novel architectures and training methodologies for integrating visual understanding, language reasoning, and tool-use capabilities.
Research efficient multimodal learning techniques, including data-efficient training, long-context modeling, model modularity, and inference optimization.
Build and improve large-scale multimodal datasets, synthetic data generation pipelines, and evaluation benchmarks for VLM capabilities.
Investigate multimodal reasoning, agentic behavior, OCR, grounding, document understanding, chart understanding, and visual question answering capabilities.
Contribute to technical reports, research publications, and open-source software.
Represent MBZUAI at research conferences and industry events, showcasing advancements in multimodal foundation models and large-scale AI systems.
Mentor junior researchers and collaborate across teams to drive impactful research initiatives.

Academic Qualifications

PhD or equivalent research experience in Machine Learning, Computer Vision, Natural Language Processing, or Multimodal AI.

Research Scientist - Vision Language Model

Job description