How does a robot learn a new job from one video?

One demonstration replaces months of data collection

A factory robot usually needs a new dataset and a full retraining run every time the product line changes. Skild AI's S1 model skips that cycle. An operator records a short video of the desired task — potting a plant, brewing pour-over coffee — and feeds it to the model as a prompt. The model interprets the intent, the objects, and the sequence, then maps them into actions for the robot in front of it. No weight updates. No task-specific post-training. The technique is called in-context learning.

In a plant-potting test the team moved from recording the demonstration to autonomous execution on hardware in 11 minutes. The robot can also adjust when objects move, recover from errors, and combine skills in sequences that were never explicitly programmed. Tasks can last up to 10 minutes and span dozens of manipulation steps. Kit assembly is another example the company demonstrated.

The numbers behind the claim

On new multi-step tasks S1 succeeded about 66 percent of the time at each step. A similar AI system scored 9 percent. That is a more than sevenfold improvement. Skild also estimates that showing the robot one short video example can be as useful as giving it roughly 380 hands-on training examples. A person collecting those examples manually could take 50 to 100 hours.

The company reached a $100 million annual revenue run rate 10 months after its first commercial deployment. In that time it built more than 60 deployment partnerships. The work spans manufacturing, logistics, inspection, security, and food preparation.

From lab to Blackwell assembly line

Foxconn and NVIDIA are already deploying the Skild Brain on dual-arm manipulators for high-precision assembly of NVIDIA Blackwell systems. In one demonstrated workflow a robot installs a busbar and limit block, fastens 16 screws, and adapts to disturbances across a multi-step task. The work requires precise motion, contact-aware control, sequence tracking, and recovery when the scene differs from the plan.

Where customer agreements permit, experience from commercial deployments feeds back into the broader model and accelerates future deployments.

The NVIDIA stack underneath

NVIDIA accelerated computing gives Skild the scale to train its shared robot brain using simulation, human video, teleoperation, and deployment data. NVIDIA Cosmos open world foundation models help diversify training data and turn video into structured descriptions. Cosmos Curator annotates, filters, and organizes data at scale.

Omniverse libraries and the Isaac Sim framework provide physically based virtual environments that generate data, test edge cases, and validate behaviors. Reinforcement learning runs in Isaac Lab, an open modular robot learning framework powered by the Newton physics engine. Newton helps engineers accurately model forces, contact, collision, and pressure, reducing the simulation-to-reality gap.

Skild and NVIDIA are jointly developing new GPU-accelerated simulation solvers that quickly and accurately model how robots physically touch, grip, and manipulate solid objects. They will soon be available to all developers as part of Newton. As models move toward production, NVIDIA Nsight tools find performance bottlenecks during training, and the TensorRT software development kit optimizes inference so robots can respond quickly in the physical world. These technologies connect data, simulation, training, and deployment instead of treating them as separate systems.