VLA, RFM, WFM, and VLM play different roles in robot AI. A VLM understands a scene. A VLA takes a scene and an instruction and directly outputs robot actions. RFM is a broad category for general-purpose robot models pretrained across multiple robots and tasks. WFM predicts or generates how the world may change as a result of an action.

These four terms often appear together in robotics news, but their inputs and outputs differ, which means their training data requirements differ as well. This guide compares the four by input, output, role, and training data, then explains how they fit together in a robot system. This article reflects the state of the field as of October 2026.

📌 Key Takeaways

• The clearest way to distinguish them is by their outputs. VLM outputs text, VLA outputs actions, and WFM outputs future visual states. RFM describes general-purpose capability rather than a specific output format, and many RFMs today take the form of VLAs.
• Training data follows the output. VLM requires image-text pairs, VLA requires episodes that record observations and actions together, and WFM requires large volumes of video, often conditioned on actions.
• In real robot systems, these model types can work together. WFM can expand training data or predict outcomes, VLA generates actions, and VLM supports planning and verification.

What Goes Into Each Model, and What Comes Out?

The diagram below shows the inputs and outputs. The table summarizes each model by role, training data, and public examples.

The four model types have different inputs and outputs, which means they also require different training data

VLM: A Model That Sees and Answers

A VLM learns jointly from images and text so it can describe a scene or answer questions about what it sees. In a robot system, a VLM can interpret a situation such as, "There are two cups on the table, and one has fallen over," break a task into steps, and check whether the result after an action matches the intended outcome. It does not directly output robot actions. That is the key distinction between VLM and VLA. To move a robot using a VLM, the system needs a separate action module such as a motion planner or VLA.

VLA: A Model That Sees and Acts

VLA extends vision-language modeling with action outputs. It can take camera input together with an instruction such as "Put the cup in the sink" and directly output the robot arm's next joint targets or motion commands. Google DeepMind's RT-2 brought broad attention to this approach by continuing the training of a large VLM on robot data. Open-weight models such as OpenVLA followed.

Training a VLA requires episodes in which observations and actions are recorded along the same timeline. Image-text pairs alone cannot teach the model how to act. That is also why Superb AI is developing robot training episodes through three different paths.

RFM: A Category Defined by General-Purpose Capability

RFM does not refer to a specific model architecture. It is a broad term for robot models pretrained at scale across multiple robot embodiments and tasks, then adapted to a new robot or task using a relatively small amount of data. It is similar to how the term "foundation model" is used in language AI. Many of the models currently described as RFMs take the form of VLAs, which is why the two terms often overlap.

A useful way to remember the distinction is:

VLA describes the model form. RFM describes its general-purpose capability.

WFM: A Model That Predicts the Next Scene

WFM takes the current scene together with an action, or another condition such as text, and generates what the world may look like next. In robotics, it can be used in two main ways.

First, it can support planning by predicting the likely result of an action before the robot executes it. Second, it can support data generation by creating variations of simulator-rendered video with different weather, lighting, textures, and other visual conditions.

NVIDIA Cosmos is a representative example in this category. WFM does not directly output robot actions. Instead, it predicts what may happen as a result of an action, making it complementary to VLA.

Related article: NVIDIA Cosmos Analysis.

Does WFM Eliminate the Need for Real-World Data?

No.

WFM is effective at generating variations from existing video and increasing data volume, but real-world data is still needed to capture the actual distribution of the target environment. In practice, the roles are complementary: real-world data provides realism, while generated data provides scale.

The specific workflow for combining the two is covered in our Synthetic Data Pipeline Guide.

How Do the Four Fit Together in a Real Robot System?

The four concepts can appear together within the same robot system, although RFM describes a category of general-purpose models rather than a separate functional layer.

A typical system can be viewed in four layers:

1. Data layer. Episodes are created through real-world collection and simulation, while WFM can generate additional visual variations.
2. Planning layer. VLM interprets the scene and breaks the task into steps.
3. Action layer. VLA, often an RFM adapted to a specific robot, outputs the actions required for each step.
4. Verification layer. WFM can predict the outcome of an action to help screen risky behavior, while VLM can check the result after execution.

Which layers to build in-house and which to source from public models is a strategic choice for robotics companies.

As more pretrained RFMs and WFMs become publicly available, differentiation is increasingly shifting toward the data itself: how many high-quality episodes a company can collect from its own operating environment.

The Model Type Determines the Data Specification

The model you plan to train determines what data you need to procure. To adapt a VLA, camera video alone is not enough. You need episodes in which camera observations, joint states, action commands, and language instructions are synchronized along the same timeline. Fine-tuning a WFM requires video paired with action conditions. If you only plan to use a VLM, image-text data may be sufficient. That is why the first question before requesting a data quote should be: Which model layer are you planning to train?

Detailed data specifications are covered in the Physical AI Dataset Development Guide and the Robot Training Data Sourcing Guide.

How Much Data Does a VLA Need for My Robot?

If you are fine-tuning a pretrained RFM, you can typically start with far fewer episodes than you would need to train a model from scratch. The amount required depends on the number of tasks, diversity of environments, and target success rate. A practical approach is to start with a smaller dataset, evaluate the success rate, and expand the data where performance falls short.

For more on how to interpret robot success rates, see Why Doesn't Robot AI Have an ImageNet?.

Superb AI is a Vision Intelligence company that transforms visual data from industrial sites into intelligence enterprises can act on. In its Physical AI Data Factory, Superb AI builds 3D spatial, action, and object assets and creates robot training data through simulation and real-world collection.

Related Content

• Physical AI Glossary: 40 Terms for Robot Training Data, Simulation, and 3D Reconstruction
• ⑧ The Breakthrough for Physical AI? Robotics Training Reimagined by NVIDIA Cosmos
• Physical AI Dataset Development Guide: 4 Data Types and a 3-Step Pipeline

💬 Deciding which model layer to train for your robot? Tell us the robot type and task below. Rather than starting with a sales call, we'll begin by reviewing the data specifications you need.