Skip to content

Back to the catalog

From Knowledge Access to Source Learning: Developing Source-Specific Competence
arxiv.orgpaper

From Knowledge Access to Source Learning: Developing Source-Specific Competence

This paper introduces "source learning," the idea that an LLM agent repeatedly consulting the same authoritative external source should progressively build reusable competence in understanding it, rather than treating each access as independent retrieval. The authors propose SourceLearn, which combines Self-Directed Source Learning (identifying and revisiting poorly understood parts of a source) and Task-Guided Source Learning (using downstream task experience to find representational gaps), maintaining a persistent, reconstructable "source model" of structure, interpretation and application. Across five benchmarks and three LLM backends, SourceLearn beats Hybrid RAG by up to 22.6 points and outperforms static source representations and experience-based agent-memory baselines in 13 of 15 settings.

TL;DR A new LLM-agent learning paradigm, SourceLearn, builds persistent source-specific competence over repeatedly used external sources, beating Hybrid RAG and memory-based baselines across five benchmarks.
Open original ↗
Document

What it claims

SourceLearn builds an explicit, persistent source model of a single authoritative source (a document collection, code repository or API docs) and refines it over time. At inference, the model is supplied to the solver alongside normal retrieval; it does not replace retrieval. The method's main constraint is that learning signals decide what to revisit, but every persistent update is rebuilt from the source text. Raw observations, study questions and task answers are never written into memory. The paper reports evaluation on document QA, code QA, tool use and interactive environments.

How it works

  1. Initial construction (M0). The method splits large sources into coherent regions, using native structure such as code modules, symbols or document sections when available. It reads each region entity by entity, guided by a generic instruction. That instruction favours major entities, relations, procedures, governing conditions and distinctions, and avoids redundant low-level detail. M0 is deliberately provisional. Easily retrieved details stay in the source.
  2. Self-directed learning (M0 → MS) runs an Inspect–Study–Consolidate cycle: - Inspect. For each entity, the system rereads its source text against the current representation and records what that representation fails to explain. These observations do not modify the model. - Study. An adaptive planner picks a small set of actions per round. DEEPEN(e, q) probes one entity's unresolved mechanism or condition. CONNECT(ei, ej, q) studies a suspected dependency between two entities. Each action uses a temporary study question that is discarded afterwards. - Consolidate. Updates are committed once, after study ends ("read-many, write-once").
  3. Grounded reconstruction gate. An update is committed only if source evidence supports it. Previously supported meaning must be preserved unless a stronger representation subsumes it. Reconstruction can merge fragments, make conditions explicit or remove redundancy.
  4. Task-guided learning (MS → MT) uses a set of guidance tasks G = {(q, y)} from the same source. Failures expose local representational gaps, and recurring demands shape how knowledge is organised. Updates go through the same reconstruction gate.
  5. Activation. If the model fits within a token budget B, the solver receives all of it. Otherwise, task-relevant regions are selected and their internal structure is kept.
flowchart TD
  D[Authoritative source] --> C[Construct M0]
  C --> I[Inspect vs. model]
  I --> P[Plan DEEPEN / CONNECT]
  P --> S[Study source]
  S -->|more rounds| P
  S --> G{Supported by source?}
  G -->|no| X[Discard observation]
  G -->|yes| M[Commit to source model]
  T[Guidance task failures] --> G
  M --> A{Fits budget B?}
  A -->|yes| F[Full model]
  A -->|no| R[Activate relevant regions]
  F --> Q[Solver]
  R --> Q
  D -->|direct retrieval| Q

Evidence

The fetched text contains only the abstract-level results:

  • Five benchmarks and three LLM backends, giving 15 settings.
  • SourceLearn is best in 13 of the 15.
  • Gains reach up to 22.6 points over Hybrid RAG.
  • The paper also claims "substantial" overall improvements over static source representations and experience-based memory baselines. Related work names RAPTOR, GraphRAG, HippoRAG and A-MEM as candidates for these baselines.
  • Both the self-directed and task-guided stages are said to contribute.

The fetched text does not show per-benchmark numbers, metrics, backend names or ablation tables.

How to use it

The fetched text gives no install commands, licence or cost figures.

Limits

  • Truncated page. The fetched text stops at the start of §4.4, so check the full paper for:
  • task-guided details,
  • results tables,
  • which 2 of the 15 settings it loses,
  • construction and study cost in LLM calls or tokens.
  • Labelled tasks needed. Task-guided learning requires labelled guidance tasks (q, y) on the same source, which you may not have.
  • Static sources only. The method assumes a persistent, stable source. The text does not describe how the model handles source updates.
  • Up-front cost. The method front-loads repeated LLM reading of the whole source before any task arrives, and that cost is not quantified here.

Organisations named here

  • Hybrid RAG

Techniques named here

  • SourceLearn
  • Self-Directed Source Learning
  • Task-Guided Source Learning
  • Persistent source model

Related in research