Back to AI Research

AI Research

An Agent-Oriented Pluggable Experience-RAG Skill fo... | AI Research

Key Takeaways

  • An Agent-Oriented Pluggable Experience-RAG Skill for Experience-Driven Retrieval Strategy Orchestration Retrieval-augmented generation (RAG) systems typicall...
  • We present Experience-RAG Skill, an agent-oriented pluggable retrieval orchestration layer positioned between the agent and the retriever pool.
  • The proposed skill analyzes the current scene, consults an experience memory, selects an appropriate retrieval strategy, and returns structured evidence to the agent.
  • The results suggest that retrieval strategy selection can be productively encapsulated as a reusable agent skill rather than being hard-coded in the upper workflow.
  • An Agent-Oriented Pluggable Experience-RAG Skill for Experience-Driven Retrieval Strategy Orchestration
Paper AbstractExpand

Retrieval-augmented generation systems often assume that one fixed retrieval pipeline is sufficient across heterogeneous tasks, yet factoid question answering, multi-hop reasoning, and scientific verification exhibit different retrieval preferences. We present Experience-RAG Skill, an agent-oriented pluggable retrieval orchestration layer positioned between the agent and the retriever pool. The proposed skill analyzes the current scene, consults an experience memory, selects an appropriate retrieval strategy, and returns structured evidence to the agent. Under a fixed candidate pool, Experience-RAG Skill achieves an overall nDCG@10 of 0.8924 on BeIR/nq, BeIR/hotpotqa, and BeIR/scifact, outperforming fixed single-retriever baselines and remaining competitive with Adaptive-RAG-style routing. The results suggest that retrieval strategy selection can be productively encapsulated as a reusable agent skill rather than being hard-coded in the upper workflow.

An Agent-Oriented Pluggable Experience-RAG Skill for Experience-Driven Retrieval Strategy Orchestration
Retrieval-augmented generation (RAG) systems typically rely on a single, fixed retrieval method to gather information for an AI agent. However, different tasks—such as answering simple facts, performing multi-hop reasoning, or verifying scientific claims—often require different search strategies to be effective. This paper introduces "Experience-RAG Skill," a new architectural layer that sits between an AI agent and its pool of retrieval tools. Instead of forcing one strategy on every request, this system analyzes the task at hand and uses a memory of past experiences to select the most appropriate retrieval method, effectively turning strategy selection into a reusable skill for the agent.

How the Experience-RAG Skill Works

The system functions as an orchestration layer with four primary responsibilities: analyzing the current scene, consulting an experience memory, routing the query to the best retriever, and packaging the results for the agent. The "experience memory" is a key component that stores records of past queries, including task types, question complexity, and which retrieval strategies performed best under those specific conditions. By treating this process as a pluggable skill, the agent can interact with a unified interface while the underlying complexity of choosing and executing the right search method remains hidden and modular.

Performance and Results

The researchers tested the system across three standard benchmarks: BeIR/nq, BeIR/hotpotqa, and BeIR/scifact. The results showed that the Experience-RAG Skill achieved an overall nDCG@10 score of 0.8924, outperforming traditional fixed-retriever baselines like BM25 and dense retrieval. When compared to other modern, complex routing methods, the Experience-RAG Skill remained highly competitive, proving that its approach to orchestration is effective for handling diverse, heterogeneous workloads.

Key Takeaways and Limitations

The study suggests that retrieval strategy selection should be treated as a distinct, reusable agent capability rather than being hard-coded into the system's workflow. While the results are promising, the authors note several limitations. The current version of the system relies on a fixed candidate pool and uses rule-based routing, as their experiments with learned routing did not yet outperform simple rules. Additionally, the current evaluation is based on sampled data rather than a full-scale, end-to-end interactive agent benchmark. Future work is expected to focus on dynamic candidate onboarding and more advanced, automated routing techniques.

Comments (0)

No comments yet

Be the first to share your thoughts!