WFM (World Foundation Model) at a Glance

Definition

World Foundation Model (WFM) is a large-scale model that models, predicts, and generates real-world states and changes as sequences from diverse inputs such as text, images, and video.

WFM is used for synthetic data generation and environment prediction in Physical-AI domains such as robotics and autonomous driving via physical simulation (e.g., NVIDIA Cosmos). This line of work represents the “World-as-Physics” direction, incorporating real sensors, robot actions, and autonomous-driving environments.

Meanwhile, DeepMind’s Genie 3 and OpenAI’s Sora2 learn not only physical rules of the world but also visual causality, cognitive patterns, and linguistic context—the “World-as-Perception / Understanding” family of WFM.

These models simulate environmental changes from text or video inputs alone and train agents to interpret situations cognitively.

Meta V-JEPA 2 is a self-supervised video model that masks parts of frames and predicts future scenes. By inferring spatial and temporal patterns, it improves both scene understanding and action planning—representing the “World-as-Prediction / Understanding” direction within WFM.

Broader Meaning of WFM (Precise Concept)

Why It Matters

• Collecting real-world data is costly and risky.
• We need to rapidly generate and test diverse settings in virtual spaces.
• This allows robots and AV models to be trained sufficiently in simulation and
• reduces risk and cost in Sim-to-Real transfer.

Core Capabilities

• Scene / World Understanding & Prediction (video level) — future-frame prediction from past frames, learning video causality (e.g., V-JEPA family)
• World / Environment Generation (text → interactive world / video) — generate environments from text prompts (e.g., Genie 3)
• Synthetic Data Generation & Supply — large-scale data pipelines for robotics / autonomy / vision systems

Representative Models / Platforms (as of 2025)

Side-by-Side Snapshot (2025, official benchmarks)

Scope and Role Separation

A WFM is more than a video generator.

It integrates generation + understanding + prediction + synthetic-data provision. This definition is also reflected in NVIDIA’s Cosmos documentation.

Reference:NVIDIA Announces Major Release of Cosmos World Foundation Models— NVIDIA Newsroom

WFM and RFM (Robot Foundation Model) have distinct roles:

• WFM: builds a digital twin of the environment/world, enabling simulation.
• RFM: learns robot policy and control to act within that environment.

This role split is stated in the Cosmos paper as a “world model + policy model” architecture.

The figure above (from the Cosmos paper) presents the basic WFM structure: given past observations x0:tx_{0:t}x0:t and the robot’s control inputs ctc_tct, it predicts the next world state x^t+1\hat{x}_{t+1}x^t+1.

This visually clarifies the relationship: WFM predicts the environment; RFM decides actions.

(Reference paper: Cosmos World Foundation Model Platform for Physical AI — arXiv Link: arXiv)

This separation is useful to emphasize WFM’s generation/prediction role vs. RFM’s control role.

Connection to Omniverse / NVIDIA Ecosystem

WFM—particularly NVIDIA Cosmos—ships with Omniverse libraries and supports:

• Real-world capture & reconstruction → digital twin creation
• Large-scale synthetic data generation
• Robot simulation and AI agent training environments

(Article: NVIDIA Opens Portals to World of Robotics With New Omniverse Libraries and Cosmos Physical AI Models — NVIDIA Newsroom)

Omniverse Blueprints are also reported to connect with Cosmos WFM, enabling robot-ready facilities and mass synthetic data generation.

(Article: NVIDIA Omniverse Physical AI Operating System Expands to More Industries and Partners)

Global WFM Research Trends (Beyond NVIDIA Cosmos)

WFM is not a single company’s technology but a global research trend toward AI that “understands, predicts, and generates the world.” Beyond physics-centric WFM, the field expands to cognitive, visual, and language-based world modeling.

1) World-as-Generation (world creation-centric)

2. World-as-Perception / Understanding (world understanding-centric)

3. World-as-Physics (world simulation-centric)

Multi-Layer Data Structure Learned by WFM

WFM spans more than visual or physical information.

In practice, WFM integrates World / Automation / Robot / Asset / Analysis Data, evolving into an Integrated World Model that predicts world state and change.

These five data layers define WFM’s input domain. Cosmos, Genie, V-JEPA, etc., each focus on specific domains (physics, vision, cognition) while sharing the common goal of understanding and generating the world.

This layered structure forms the input foundation for WFM to understand and simulate the world, enabling AI to perform contextual world modeling.