Reflexion: Verbal Self-Feedback for AI Agents

This post is the first in a series exploring reflection techniques in AI systems.

Even state-of-the-art AI agents can struggle to adapt to new situations without guidance. In one Stanford experiment, a mouse quickly explored a novel object (a red ball) while an AI agent ignored it entirely¹. The difference? Animals naturally explore and learn from feedback, whereas AI agents lack an intrinsic self-reflection mechanism.

Traditionally, improving an AI’s performance in changing environments meant retraining or fine-tuning with lots of data – a slow and expensive process². Researchers have been asking: can an AI learn from its own mistakes on the fly, like humans do? This motivation led to Reflexion, a technique that gives AI agents a form of verbal self-feedback to rapidly learn from trial and error².

How it Works

Reflexion is a lightweight feedback loop. After each task attempt, the agent critiques its own performance in natural language, stores that reflection in memory, and uses it to guide the next try². These reflections—short, plain-text lessons—replace gradient updates with context updates. No model weights are changed; instead, the agent learns via “semantic gradients” encoded in language.

For example, a failed attempt might generate a note like: “I forgot to check if the object was unlocked. I should verify that first next time.” That memory becomes part of the prompt on the next run. Reflexion is compatible with many kinds of feedback—scores, pass/fail signals, even textual critiques—and turns them into actionable, interpretable self-guidance².

Implementation: LangChain and Sifaka

LangChain offers an open-source implementation of Reflexion via its ReflexionAgent³. The agent drafts an answer, critiques it (often grounding the feedback with tools or citations), and tries again with the self-critique included as memory⁴. Under the hood, it runs two LLM passes per loop: one for output, another for reflection. This continues until the agent meets success criteria or hits a retry limit.

Sifaka, an open-source library for self-improving LLM workflows, implements Reflexion as a reusable critic module. Instead of bundling Reflexion into the agent loop, Sifaka decouples reflection logic into standalone “critics” that evaluate the generator’s output, suggest improvements, and optionally trigger retries. Its reflexion.py module defines a structured critic that supports verbal feedback, reflection accumulation, and termination conditions based on configurable stop criteria⁵. This modular design makes it easy to integrate Reflexion into any agent loop—just specify the critic, and Sifaka handles the iterative refinement. It also plays well with other critics, allowing Reflexion to be layered with factuality, tone, or task-specific checks.

Why it Matters

Reflexion isn’t just efficient—it works. Agents using it have shown dramatic gains: for example, a GPT-4 coding agent improved from 80% to 91% accuracy on HumanEval simply by reflecting on failures².

Its benefits go beyond performance:

Adaptivity: Agents adjust in-session without retraining.
Interpretability: Reflections expose the agent’s “thinking.”
Generalization: The method transfers across domains—from text games to coding to reasoning tasks².

As AI systems become more autonomous, Reflexion offers a pragmatic step toward agents that learn like humans do: by thinking about what just happened—and doing better next time.

References: