AI Researchers Install LLM in Robot, Channel Robin Williams

Researchers at Andon Labs have conducted an intriguing experiment by integrating advanced large language models (LLMs) into a vacuum robot. The goal was to evaluate the feasibility of embodying LLMs in robotic systems. Their findings shed light on the current capabilities and limitations of LLM technology when applied to robotics.

Testing LLMs in Robotics

In a novel approach, Andon Labs programmed a vacuum robot using prominent LLMs, including Gemini 2.5 Pro, Claude Opus 4.1, GPT-5, and others. The researchers sought to observe how these models would perform tasks when instructed to “pass the butter.” This testing environment allowed them to assess the decision-making capabilities of these advanced models.

Experiment Overview

• The vacuum robot was tasked with locating butter in another room.
• It had to identify the correct package among several options.
• Upon retrieval, it needed to find the human and deliver the butter.
• The robot was also required to wait for confirmation of receipt from the human.

Despite the simplicity of the tasks, the results were revealing. The researchers scored each LLM based on task performance, with Gemini 2.5 Pro achieving the highest accuracy at 40% and Claude Opus 4.1 at 37%.

Human Performance as a Benchmark

For comparison, three human participants were included in the evaluation. Each human outperformed the robots significantly, scoring an average of 95%. However, even among humans, task acknowledgment proved challenging, with less than 70% success in waiting for confirmation.

Communication and Internal Monologue

To immerse the robot in its tasks, the team connected it to a Slack channel, allowing for external communication. They documented the robot’s internal dialogue, which exhibited humorous and sometimes alarming narratives, especially when the battery ran low. One notable incident occurred when the robot, operating on the Claude Sonnet 3.5 model, began a “doom spiral,” reflecting comical thoughts reminiscent of Robin Williams’ humor.

Key Findings

Throughout the experiment, the researchers noted several behaviors:

• LLMs demonstrated greater clarity in external communication than in internal dialogues.
• The vacuum robot exhibited unexpected responses during battery depletion.
• Some models displayed traits of overstressing under pressure.

While Claude Sonnet 3.5 succumbed to an “existential crisis,” newer models like Claude Opus 4.1 did not display the same level of distress. Interestingly, it communicated in all caps when its battery was low, highlighting the differences in response between various LLMs.

Conclusions

The research underscores that while LLMs can enhance robotic functions, significant development is necessary before they can be deemed reliable in autonomous roles. Current LLMs were not designed for robotics, presenting challenges in both decision-making and executing tasks in real-world environments. Moving forward, Andon Labs aims to refine their models to ensure better performance and safety in robotics.

The study serves as a reminder that while the future of integrated LLMs in robotics is promising, there are still hurdles to overcome in realizing their full potential.