Prompt / Prompt Engineering

A prompt is the text fed into a model; prompt engineering is designing it to get the output you want.

A prompt is the text given as input to a large language model or multimodal model, which can include task instructions, background context, examples, and formatting requirements for the output. Prompt engineering is the practice of iterating on and testing prompts so the model reliably produces the desired result, without changing the model's parameters; common techniques include giving a few examples (few-shot), having the model write out its reasoning first (chain-of-thought), and specifying the output format. In embodied AI it shows up in two main ways. First, when using a large model for task planning, the prompt tells the model what skills the robot has and what's in the scene — SayCan and Code as Policies both depend on carefully designed prompts. Second, the natural-language instruction a VLA receives is also often just called the 'prompt' in the code. By contrast, a soft prompt is a trainable vector rather than text.

ExampleCalling π0.5 through openpi, the input includes camera images plus a prompt field reading 'pick up the fork'; the model outputs an action chunk for picking up the fork based on it.

Also called

Prompt, Prompting

Related

Large Language Model · Chain-of-Thought · In-Context Learning · Code as Policies · SayCan · Prompt Tuning / Soft Prompt

Sources

Claude Docs: Prompt engineering overview
openpi (Physical Intelligence) README