Encord, the data-tooling firm building training sets for humanoid and warehouse robots, is now strapping brain-wave headsets onto its human trainers to see whether neural signals can sharpen the data feeding robotics models. Inside a warehouse in San Leandro, California, one of the company's pilots plays Jenga while a headset from German neuroscience startup Zander Labs tracks his brain activity alongside his field of view. Encord's head of robot learning, Vineeth Velmurugan, estimates the industry needs a corpus roughly five times the size of YouTube's video library to break through the physical AI data bottleneck.
That number reframes the whole robotics race. Large language models were trained on a near-free scrape of the internet; robot brains have no equivalent. Self-driving companies collect their own data at enormous cost, video-only training lacks fidelity, and the raw material for teaching a neural network to pour coffee or plug in an ethernet cable has to be manufactured rather than harvested. Encord runs about a dozen pilots at the San Leandro facility, producing egocentric video, teleoperated robot demonstrations, and now brain-wave-tagged sessions to test whether the extra signal justifies the cost.
Encord started as an annotation and evaluation platform for machine-vision customers. As those customers, described by Velmurugan as many of the leading robotics firms though he is not authorized to name them, pivoted to end-to-end learning for manipulation, they hit the same wall.
“The data simply does not exist”— Vineeth Velmurugan, Head of robot learning at Encord
Key facts
- 01Encord estimates the field needs a training corpus five times the size of YouTube's video library to crack physical AI.
- 02Densely annotated robot data is worth roughly 100x more than raw egocentric footage but costs 20x more to produce.
- 03Encord runs about a dozen pilots at its San Leandro, California facility, generating data for leading but unnamed humanoid firms.
- 04The brain-wave headsets are built by Zander Labs, a German neuroscience startup measuring error, intent, and surprise during tasks.
- 05Encord's data lead Vineeth Velmurugan previously worked at OpenAI's robot lab and warehouse automation firm Berkshire Grey.
Velmurugan, a veteran of OpenAI's robot lab and warehouse automation firm Berkshire Grey, was hired to build the internal data-creation team. The pitch is that Encord no longer just manages customer data, it produces the training sets those customers cannot source anywhere else. The company draws egocentric video from several factories around the globe and uses San Leandro for experimental modalities and skill-specific fine-tuning sets.
The Zander Labs collaboration is still a trial. Encord plans to build an initial brain-wave-tagged data set, run it through customer models, and evaluate whether it actually improves performance before scaling. Zander neuroscientist Lucas Gehrke says the amount of brain activity used at any point during a task offers clues for model builders trying to figure out when they need to deploy their highest-effort models, a kind of biological signal for compute allocation.
At other stations, pilots use leader-follower rigs, paired robotic arms where one is controlled by a human and the other mimics its movements, to generate demonstrations for tasks like pouring coffee and stacking poker chips. Storage racks are stocked with fake flowers, plastic vegetables, kitty litter trays, and bundles of wires, the raw props for training household manipulators. Pilot Sofia Infante works on plugging and unplugging ethernet cables from server backs, exactly the kind of data-center task operators want automated.
“Every humanoid company has asked us for these pieces”— Vineeth Velmurugan, Head of robot learning at Encord
Another modality under development uses forearm sensors to detect electrical signals in muscles. Egocentric video routinely fails to capture the full hand, and Velmurugan wants to reconstruct a 3D representation of the hand's position from arm signals to give models a more robust picture of manipulation. Each video clip is densely annotated with physical descriptions, phrases like right hand tightens bolt, to help LLM-based models parse what they are seeing.
The economics are the catch. Velmurugan estimates dense annotation is worth 100 times more than junky egocentric footage for training specific tasks and costs about 20 times more to produce, a favorable trade on paper. But 20 times more is real money, and that is where the LLM analogy breaks. Scraping Stack Overflow and the rest of the web cost frontier labs almost nothing. Manufacturing physical training data changes the unit economics of building robot foundation models entirely, and it means the companies that control data pipelines may hold as much leverage as the companies training the models.
Encord's pilots have their own trajectory. Both Infante and pilot Andrew Ceja came from Scale, the AI annotation firm; Ceja previously kept a robotic trash sorter running at a waste-management company. Their job is now to generate the building blocks for humanoid intelligence, one Jenga tower and one pot of coffee at a time.
The counterweight is that none of this is proven at scale. Encord's brain-wave work is a trial that has not yet been validated against customer benchmarks, muscle-sensor data has not been shown to move the needle on manipulation accuracy, and the five-times-YouTube figure is an estimate from a company that would benefit commercially from the field agreeing with it. Robotics has spent a decade chasing generalist manipulation and repeatedly discovered that new sensor modalities produce marginal gains rather than step changes.
The story to watch is whether physical AI settles into a data-brokerage market that looks structurally different from the LLM race. If the ceiling on humanoid performance is data rather than compute, the firms sitting between many robotics customers, seeing which techniques work across dozens of programs, capture disproportionate value. Encord is betting that vantage point is worth more than any single model, and if the five-times-YouTube number is even directionally right, the physical AI economy will be built on infrastructure the LLM boom never needed.
Working on something we should cover, or seeing a story we missed? Send leads, documents, or feedback to hello@aichatdaily.com. For sensitive tips, see our secure tips page for Signal and PGP options.
Spotted an error? Email hello@aichatdaily.com with the URL and the issue, or read our full corrections policy.
