Choose teleoperation when target-robot action fidelity is the priority, UMI-style collection when you need portable and scalable manipulation demonstrations, and egocentric capture when the objective is broad human activity, semantic context or pretraining. Most serious embodied AI programs use a designed mixture rather than forcing one method to do every job.
The collection interface determines what the dataset can teach. It shapes action labels, operator behavior, sensor geometry, cost per episode and the gap between demonstration and deployment. The wrong interface can produce technically clean data that is poorly matched to the model.
The three methods answer different questions
Direct robot teleoperation
Teleoperation records observations and actions on the target or a closely related robot. Systems range from leader-follower arms to VR controllers and whole-body interfaces. Mobile ALOHA, for example, was designed as a low-cost whole-body teleoperation system for collecting bimanual mobile manipulation data.
Best for: behavior cloning on a known embodiment, contact-rich tasks, precise gripper timing, whole-body coordination and data that must match an existing control stack.
Main trade-offs: robot time is occupied during collection; resets and safety supervision add cost; operator ergonomics affect throughput; and hardware faults can contaminate the action stream.
UMI-style human demonstration
The Universal Manipulation Interface uses portable hand-held grippers and carefully designed representations to transfer in-the-wild human demonstrations to robot policies. Its central advantage is operational: useful manipulation data can be collected without bringing the target robot into every environment.
Best for: dynamic, bimanual or long-horizon manipulation; collecting across many natural settings; early exploration of task feasibility; and programs where robot fleet availability is the limiting resource.
Main trade-offs: human-handled tools do not reproduce every robot constraint; calibration and trajectory reconstruction require discipline; and final deployment normally still benefits from target-embodiment validation or fine-tuning.
Egocentric human activity capture
Egocentric data records a first-person view, often with head pose, audio, IMU and optional wrist or hand sensors. Large research efforts show its value for understanding human activity. Ego-Exo4D pairs synchronized first- and third-person views and provides time-indexed language and rich annotations for skilled activities.
Best for: learning task semantics, pretraining perception, discovering procedural structure, grounding language, studying natural human behavior and covering settings where a robot cannot yet operate.
Main trade-offs: video alone does not provide robot actions; head motion is not end-effector motion; hands may occlude objects; and translating human kinematics into robot-compatible control requires an additional modeling layer.
A decision matrix
| Decision factor | Teleoperation | UMI-style | Egocentric |
|---|---|---|---|
| Target action fidelity | High | Medium, representation-dependent | Low without added tracking |
| Natural-environment coverage | Medium | High | Very high |
| Robot fleet required | Yes | No during human capture | No |
| Semantic and procedural context | Medium | High | Very high |
| Contact and gripper-state fidelity | High | High with instrumented interface | Limited unless instrumented |
| Scaling across locations | Operationally heavy | Portable | Most portable |
The table is a starting point, not a verdict. “High” fidelity is useful only if the stream is synchronized and the operator can execute the task consistently.
Start from the training target, not the hardware you own
A common mistake is to choose the interface first because it is available. Reverse the order. Write down:
- Model input: which camera views, language fields, proprioception and history windows will be consumed?
- Model output: joint targets, end-effector deltas, poses, action chunks or higher-level skills?
- Deployment envelope: fixed workcell, mobile platform, home, warehouse or outdoor setting?
- Generalization requirement: new objects, new scenes, new embodiments or new task language?
- Failure tolerance: can the system stop and ask for help, or must it recover autonomously?
These answers determine what must be observable in the dataset. If the planned policy outputs end-effector deltas at a fixed rate, a beautiful human video without a compatible trajectory representation is insufficient. If the primary problem is understanding a long household procedure, direct robot teleoperation may be an unnecessarily expensive first step.
The best program is often a data ladder
Rather than mixing methods randomly, assign each a role:
- Egocentric discovery: capture how people naturally perform the task, including object choice, ordering and environmental variation.
- UMI-style skill capture: convert the most valuable manipulation segments into portable, information-rich demonstrations.
- Target-robot teleoperation: anchor the learned behavior to the actual embodiment, controller and safety constraints.
- Closed-loop correction: collect targeted interventions and recoveries from model rollouts.
This ladder avoids two expensive extremes: trying to deploy directly from human video, or using scarce robot time to discover basic task semantics that people could capture more quickly.
Five hidden costs to include in the comparison
- Reset labor: placing objects back into controlled or varied starting states.
- Calibration downtime: camera, tracker, robot-frame and gripper checks.
- Operator learning curve: early episodes may reflect interface adaptation rather than task expertise.
- Data repair: dropped frames, clock drift, mislabeled outcomes and incomplete trajectories.
- Embodiment conversion: the engineering required to map captured motion into the policy’s action space.
Cost per recorded hour ignores all five. Cost per accepted and useful episode is a better comparison.
How MovraWorks scopes collection interfaces
For UMI data collection, we focus on portable task execution, trajectory reconstruction, synchronized views and environment diversity. For teleoperation data collection, the priority shifts to embodiment fidelity, control signals, operator consistency and safe resets. For egocentric data collection, the design centers on natural behavior, view quality, privacy, procedural metadata and optional multimodal sensors.
The choice should survive a simple test: after one thousand episodes, can the engineering team explain exactly what new capability the method is expected to add? If the answer is only “more data,” the collection design is not finished.