ReAct Prompting

ReAct (short for Reason + Act) is a prompting paradigm for large language models (LLMs) that enables them to solve complex tasks by interleaving verbal reasoning (thoughts) and actions that interact with an external environment. The method was proposed in 2022 by researchers from Google Research and Princeton University and has become a fundamental approach in the development of intelligent agents.

The key innovation of ReAct is the integration of logical reasoning, characteristic of techniques like Chain-of-Thought (CoT), with the execution of external actions, such as information retrieval or interaction with software interfaces (APIs). This allows the model to dynamically adjust its plan, verify facts, and overcome the limitations of its own often outdated or incomplete knowledge.

Background and History

Before the advent of ReAct, there were two primary but separate approaches to using LLMs for solving complex problems:

• Chain-of-Thought (CoT): The model generates step-by-step logical reasoning to solve a task, but it does so in an "informational vacuum," relying solely on its internal knowledge. This often leads to factual errors and "hallucinations".
• Act-Only: The model generates a sequence of actions to interact with an external environment (e.g., in text-based games), but without explicit high-level planning. Such systems struggle with errors and cannot adapt their strategy when faced with unforeseen circumstances.

ReAct was created to synergistically combine the strengths of both approaches. It allows the model not only to reason but also to ground its reasoning in facts from the external world and adapt its actions based on the feedback received.

How It Works: The Thought–Action–Observation Cycle

ReAct operates within an iterative cycle that includes three key components. The model sequentially generates a trajectory consisting of Thought, Action, and Observation steps.

Example prompt structure:

Question: What is the capital city where the inventor known for alternating current was born?

Thought 1: I need to find out who invented alternating current. Then, I need to find their birthplace and check if that city is a capital.

Action 1: Search[inventor of alternating current]

Observation 1: Alternating current (AC) was developed and promoted by Nikola Tesla.

Thought 2: Now I need to find out where Nikola Tesla was born.

Action 2: Search[Nikola Tesla's birthplace]

Observation 2: Nikola Tesla was born in the village of Smiljan, Austrian Empire (modern-day Croatia).

Thought 3: Smiljan is not a capital city. Thus, the question does not have a straightforward answer as phrased. However, Tesla lived and worked for a long time in the US. Perhaps the question refers to the capital of the country where he made his major discoveries. That is a more complex question. I should state the fact based on his birthplace. The city of his birth is not a capital.

Final Answer: The inventor associated with alternating current, Nikola Tesla, was born in Smiljan (Croatia), which is not a capital city.

Thought

The model generates internal reasoning in which it analyzes the task, decomposes it into sub-tasks, forms or adjusts a plan of action, and assesses the current state. This step does not affect the external environment but serves for internal planning.

Action

Based on its reasoning, the model performs an external action by calling one of the available tools. Examples of actions include:

• Search [query] — to search for information in a knowledge base (e.g., Wikipedia).
• Click [element] — to navigate a web page.
• API_call [parameters] — to call an external API.

Observation

The system or an external tool executes the action and returns a result (observation). This could be a text snippet, an API response, or an error message. The model uses this observation to form its next thought, closing the feedback loop.

Experimental Results and Effectiveness

The authors of ReAct tested the method on a wide range of tasks, demonstrating its superiority over CoT and Act-Only approaches.

ReAct results compared to baseline methods on key benchmarks

Benchmark	Task	ReAct (Success/Accuracy)	Chain-of-Thought (Success/Accuracy)	Act-Only (Success/Accuracy)
FEVER	Fact Verification	87.6%	82.8%	70.1%
HotpotQA	Multi-hop Question Answering	31.8%	36.3%	17.0%
ALFWorld	Text-based Game (Planning)	71%	10%	13%
WebShop	Web Navigation	40%	-	30.1%

• In tasks requiring fact verification (FEVER), ReAct significantly outperforms CoT, as it can verify information.
• In interactive planning tasks (ALFWorld, WebShop), ReAct demonstrates a huge advantage, as it can adapt to a changing environment.
• In pure reasoning tasks (HotpotQA), CoT can show comparable or even better results. However, a hybrid ReAct+CoT approach showed the best performance, achieving 35% on HotpotQA.

Criticism and Limitations

Despite its effectiveness, ReAct has several limitations and vulnerabilities.

• Brittleness: Later research has shown that ReAct's success may be attributed not so much to a genuine synergy of reasoning and action but to the syntactic similarity between examples in the prompt and the current task. The model might follow the `Thought-Action-Observation` template without fully understanding the semantics of the reasoning.
• Computational Costs: Each ReAct cycle requires at least one LLM call and one call to an external tool, making it slow and expensive compared to standard prompting.
• Dependence on Tools: The agent's effectiveness is directly dependent on the quality and reliability of the available tools. An incorrect or noisy response from an API can steer the model's reasoning down a false path.

Significance and Further Development

ReAct became a significant milestone in AI development, marking a shift from generative systems to agentic systems. It laid the conceptual and practical foundation for most modern frameworks for creating agents, such as:

• LangChain
• LlamaIndex
• AutoGen

The ideas behind ReAct served as a starting point for more complex reasoning architectures, such as Reflexion (which adds a self-reflection cycle after failures) and Tree of Thoughts (ToT) (which explores multiple reasoning paths in parallel).

External links

• Official ReAct Project Page
• ReAct description in the Learn Prompting guide
• Google Research (2022). ReAct: Synergizing Reasoning and Acting in Language Models (blog post). Google Research Blog.
• Prompt engineering — Wikipedia

Literature

• Yao, S. et al. (2022). ReAct: Synergizing Reasoning and Acting in Language Models. arXiv:2210.03629.
• Yao, S. et al. (2023). Tree of Thoughts: Deliberate Problem Solving with Large Language Models. arXiv:2305.10601.
• Shinn, N. et al. (2023). Reflexion: Language Agents with Verbal Reinforcement Learning. arXiv:2303.11366.
• Verma, V. et al. (2024). On the Brittle Foundations of ReAct Prompting for Agentic Large Language Models. arXiv:2405.13966.
• Yao, S. et al. (2022). WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents. arXiv:2207.01206.
• Shridhar, M. et al. (2020). ALFWorld: Aligning Text and Embodied Environments for Interactive Learning. arXiv:2010.03768.
• Qin, L. et al. (2023). AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Collaboration. arXiv:2308.08155.
• Thorne, J. et al. (2018). FEVER: A Large-Scale Dataset for Fact Extraction and Verification. arXiv:1803.05355.
• Yang, Z. et al. (2018). HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering. arXiv:1809.09600.