Embodied Interaction

The interaction an agent has with people, objects, and its surroundings in a physical or simulated space.

In embodied AI, embodied interaction refers to an agent, a real robot or a virtual body in simulation, interacting with people and the environment in a physical or simulated space: this includes actions that change the environment, such as moving, grasping, pushing, and pulling, as well as acting on spoken instructions and answering questions. The 2024 embodied-AI survey from Sun Yat-sen University and others lists it as one of four research directions, alongside embodied perception, embodied agents, and sim-to-real, using embodied question answering and language-guided grasping as typical tasks. It differs from pure perception in that interaction changes the environment, so the agent must decide both what to do and when it has gathered enough information. The term has a separate meaning in human-computer interaction (HCI): Paul Dourish's 2001 book Where the Action Is uses it to describe how people interact with computing systems through their bodies, physical objects, and social context, which belongs to interface-design theory — a distinction worth keeping in mind when reading the literature.

ExampleA user says, “I'm thirsty, get me something to drink.” The robot has to infer the intent, find a drink, and hand it over securely, not just recognize that there is a cup somewhere in the image.

Related

Embodied Perception · Embodied Question Answering · Human-Robot Interaction · Physical Human-Robot Interaction · Interactive Perception · Language Grounding

Sources

Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AI (Liu et al., 2024)
Paul Dourish (Wikipedia, on Where the Action Is: The Foundations of Embodied Interaction)