Technology / World Action Model

让机器人理解动作，也理解动作之后的世界 Understand actions — and what happens next

世界动作模型将当前状态、任务目标、视觉与深度信息、机器人本体状态和人类输入统一到同一个交互式模型中，在生成下一步动作的同时预测动作执行后的状态变化。 The World Action Model brings current state, task goals, visual and depth observations, robot state, and human input into one interactive model. It generates the next action while predicting the resulting state transition.

多模态上下文 Multimodal context 动作生成 Action generation 状态预测 State prediction
Why WAM

从动作输出走向世界状态建模 From action output to world-state modeling

机器人面对的是连续变化的物理世界。模型不仅需要决定“下一步做什么”，还需要理解动作会如何改变环境、物体与机器人自身状态。 Robots operate in a continuously changing physical world. A model must decide what to do next and understand how that action changes the environment, objects, and robot state.

01

理解任务上下文 Understand task context

将目标、场景观测、执行进度与历史信息组织为统一上下文。 Organize goals, observations, progress, and history into one context.

02

生成可执行动作 Generate executable actions

结合环境和机器人本体约束，形成面向真实执行的动作输出。 Produce actions for real execution under environment and embodiment constraints.

03

预测执行后状态 Predict resulting states

学习“当前世界—执行动作—未来状态”之间的动态关系。 Learn the dynamics between the current world, an action, and its resulting state.

Multimodal context

把不同来源的信息放进同一个任务上下文 Bring every signal into one task context

任务目标、环境感知、接触信息、机器人本体状态与人类输入分别经过编码，再进入统一的时空建模与上下文融合过程。 Task goals, environment observations, contact signals, robot state, and human input are encoded before entering a unified temporal-spatial modeling process.

任务与目标 Tasks & goals 任务指令 · 视觉子目标 · 执行进度 Instructions · Visual goals · Progress
环境感知 Environment RGB · 深度 · 点云 RGB · Depth · Point clouds
Unified Context
接触与本体 Contact & embodiment 力触觉 · 关节 · 机器人状态 Force · Joints · Robot state
人类输入 Human input 文本 · 语音 · 人工反馈 Text · Speech · Human feedback
Action-state joint modeling

同时生成动作与预测下一状态 Generate actions and predict the next state

视觉、深度点云、音频、机器人状态和文本分别进入对应编码器。模型完成时空建模和上下文融合，同时输出可执行动作与动作后的环境和机器人状态。 Vision, depth and point clouds, audio, robot state, and text enter dedicated encoders. The model performs temporal-spatial fusion and produces executable actions together with the resulting environment and robot state.

当前状态 Current state 场景 · 目标 · 机器人 Scene · Goal · Robot
WAM 时空建模 Temporal-spatial modeling
下一步动作 Next action 面向真实执行 For real execution
未来状态 Future state 环境与机器人变化 Environment and robot transition
Interactive loop

在真实交互中形成闭环 Close the loop through real interaction

模型通过真实执行结果、状态变化和人类反馈比较预测与现实，将有效经验和失败样本回流到后续训练与评估。 The model compares predictions with real execution, state transitions, and human feedback, returning both valid experience and failures to later training and evaluation.

01 任务规划 Plan
02 动作执行 Act
03 环境感知 Observe
04 预测比较 Compare
05 反馈评估 Evaluate
Data flywheel

从数据采集到真实部署反馈 From capture to deployment feedback

第一人称采集、UMI演示采集与真实机器人Rollout共同提供数据来源，经过管理、质量评估和训练后重新进入真实执行。 Egocentric capture, UMI demonstrations, and real-robot rollouts provide data that is managed, evaluated, trained, and returned to real execution.

第一人称数据采集设备 / Egocentric data capture device
第一人称采集 Egocentric capture 记录面向任务执行的视角与交互过程 Record task-oriented viewpoints and interactions
UMI动作演示采集装置 / UMI demonstration capture device
UMI演示采集 UMI demonstration capture 获取人类演示并支持动作重定向 Capture human demonstrations for action retargeting
01 数据采集 Capture 02 数据管理 Manage 03 质量评估 Evaluate 04 模型训练 Train 05 真机 Rollout Real rollout 06 反馈回流 Feedback
Technology relationship

模型、运行底座与工程系统各司其职 A clear role for every layer

运行底座 Runtime layer Dora OS 连接数据、算法与设备 Connect data, algorithms, and devices 工程载体 Engineering layer Oasis WAM 承载具身决策与动作生成 Embodied decisions and action generation 数据资产 Data layer Oasis Data 治理、版本化并供给训练数据 Govern and version training data