Brain Waves and Robot Training: Inside the Physical AI Data Bottleneck
Encord and Zander Labs are trialling brain wave headsets to build richer robot training data. Here's why physical AI still can't scale the way LLMs did.
At a warehouse in San Leandro, California, a company called Encord is filming workers pull Jenga blocks while wearing headsets that record their brain waves. The headsets come from Zander Labs, a German neuroscience startup. The experiment is an early trial: Encord wants to tag robot training data with mental states like error, intent, and surprise, then test whether that extra signal actually improves model performance. It is one of the more striking examples of how physical AI is running into a data wall that no one has solved yet.
What happened
| Detail | Fact |
|---|---|
| Location | Warehouse in San Leandro, California |
| Companies involved | Encord (data tooling), Zander Labs (German neuroscience startup) |
| Brain wave headset purpose | Detect mental states: error, intent, surprise |
| Current status | Trial run; scale-up depends on whether performance improves |
| Annotation value estimate | Dense annotation is ~100x more useful than low-quality egocentric video |
| Annotation cost premium | ~20x more expensive to produce than basic egocentric data |
| Data scale needed | ~5x the size of YouTube’s entire video corpus |
Encord was originally built to help machine-vision teams annotate data and evaluate models. As its robotics customers moved toward end-to-end learning (training neural networks directly on raw sensor data rather than hand-coded rules), the company concluded that training data simply did not exist at the volumes needed. That insight pushed Encord into manufacturing data itself, not just managing it.
Vineeth Velmurugan, Encord’s head of robot learning and a former member of OpenAI’s robot lab and warehouse automation firm Berkshire Grey, leads the internal data-creation team. He told TechCrunch plainly: “The data simply does not exist.”
Why does physical AI need so much more data than a chatbot?
Large language models were built on text scraped from the entire internet, essentially for free. Physical AI has no equivalent. Self-driving companies collect their own data, but that approach is hard to replicate at scale. Training from video is possible but lacks the precision of real-world sensor data. Velmurugan estimates the field needs something like five times YouTube’s total video corpus to make a serious breakthrough.
That comparison puts the cost gap in perspective. Web scraping cost frontier labs next to nothing. Generating physical training data requires human operators, specialized equipment, careful annotation, and physical props ranging from fake flowers to ethernet cables. It has to be manufactured, not harvested.
What Encord is actually building
The San Leandro facility runs two main data collection methods:
- Egocentric video: Workers wear head cameras while performing tasks. Encord collects this from several factories around the world.
- Leader-follower rigs: Paired robotic arms where a human controls one and the other mimics it, capturing tasks like pouring coffee or stacking poker chips.
A newer modality uses forearm sensors to detect electrical muscle signals. Because hand-camera footage often misses parts of the hand, Velmurugan wants to use those signals to build a 3D map of hand position at any moment, giving models a richer spatial understanding of manipulation.
All video is annotated with plain-language descriptions of what is happening, for example “right hand tightens bolt.” Velmurugan estimates that level of dense annotation is worth 100 times as much as raw, unstructured egocentric footage for training specific tasks. It costs about 20 times more to produce. On paper that is a good return, but “20 times more” still represents real money, and that cost has no equivalent in the LLM world.
Where brain waves fit in
Lucas Gehrke, a Zander Labs neuroscientist supervising the trial, says the amount of brain activity at any point during a task tells model builders something useful: when the operator is working hard, making an error, or encountering something unexpected. That metadata could help robotics models decide when to deploy their most computationally expensive processing versus when a lighter approach is fine.
The brain wave work is still experimental. Encord’s plan is to build an initial tagged dataset, run it through customer models, and measure whether performance actually goes up before committing to a larger rollout. Velmurugan called it the “bleeding edge” of the data bottleneck effort.
Our take
The brain wave angle is genuinely interesting, but the more important story is the economics buried in the details. The LLM analogy for physical AI gets repeated constantly, yet it breaks down at exactly the point that matters: cost of raw materials. Text was free. Physical manipulation data is not, and every new modality (brain waves, muscle sensors, multi-angle rigs) adds more cost before it adds more value.
For businesses thinking about integrating AI into operations, this is a useful reality check. Warehouse and humanoid robotics are not one architecture update away from being deployable. The constraint is data, and solving the data problem is an industrial operation, not a software release.
We covered a related tension in the US-China AI race earlier this year, where model capability is no longer the only axis of competition. Data pipelines and data quality are becoming equally strategic. Encord’s bet is that whoever builds the best data manufacturing operation wins, independent of which model architecture becomes dominant. That is a defensible position, though it depends heavily on whether brain wave tagging and muscle-sensor mapping actually move the needle on model benchmarks, which remains unproven.
What to do about it
- Watch Encord’s published results from the Zander Labs trial before drawing conclusions about brain wave data as a training signal.
- If you are evaluating warehouse automation vendors, ask specifically about their training data provenance and annotation density, not just model architecture.
- Track the cost per useful training sample as a metric, not just dataset size, when assessing physical AI readiness claims.
- Treat “physical AI will scale like LLMs” as a red flag in vendor pitches until the data cost problem has a clearer answer.
The honest bottom line: physical AI is real and advancing, but anyone telling you it is close to LLM-scale progress is skipping over the part where the training data has to be built by hand, one task at a time.
Frequently asked questions
What is the physical AI data bottleneck?
Physical AI models need vast amounts of real-world sensor and video data to learn manipulation tasks. Unlike LLMs, which were trained on free text scraped from the web, physical training data must be manufactured by human operators using specialized equipment, making it expensive and slow to produce at scale.
How is Encord using brain waves to train robots?
Encord is trialling headsets from German startup Zander Labs that record brain activity while human operators perform physical tasks. The goal is to tag training data with mental states like error, intent, and surprise, giving robot models a signal for when high-effort processing is needed. The trial is still in an early evaluation phase.
How much robot training data is needed for physical AI to advance?
According to Encord's head of robot learning, Vineeth Velmurugan, breaking through the current ceiling would require a dataset roughly five times the size of YouTube's entire video corpus.
Is dense annotation of robot training data worth the cost?
Encord estimates that densely annotated physical training data is about 100 times more useful for training specific tasks than raw egocentric video, but costs roughly 20 times more to produce. The economics look good on paper, though the absolute cost is still significant compared to LLM training data.
