What you will learn How to give a model your internal material, why "just paste everything" stops working, and why a citation on the answer is the decisive part.
A language model predicts the next word from what it saw during training. So it structurally cannot answer this:
This is where hallucination most commonly appears. You asked about something it does not know.
Before you ask, find the material that would justify the answer and pass it in alongside the question.
The industry calls this RAG (retrieval-augmented generation). The name is harder than the idea.
HyperTeamsgraph TD
A["Question: what was the Q4 return rate?"] --> B["1. Search internal material"]
B --> C["2. Pass 3-5 retrieved chunks + the question to the model"]
C --> D["3. Model answers from those chunks only"]
D --> E["Answer + source (which document, which page)"]You are not retraining the model's weights. You are fetching material each time you ask. That is why updating a document takes effect on the very next question.
This is still learning — it just accumulates in the material rather than in the model. Which kind of learning? splits that four ways.
The most common question, and the answer is yes, when there is little of it. For a single policy handbook, pasting the whole thing beats building search.
The problem is growth.
| Paste everything | Search, then paste |
|---|---|
| Beyond the context window it simply doesn't fit | only the chunks needed |
| Every question costs the full volume | a few chunks' worth |
| Irrelevant content dilutes accuracy | only what is relevant |
| Breaks as documents grow | unchanged as documents grow |
The third row surprises people. More material does not mean better answers. Bury the right paragraph among fifty irrelevant documents and the model misses it exactly like a person would.
Two approaches, and in practice you want both.
| Approach | Basis | Good at | Bad at |
|---|---|---|---|
| Keyword search | matching words | product codes, IDs, proper nouns | "refund policy" won't find "return terms" |
| Semantic search | closeness in meaning | finds it despite different wording | weak on exact codes and figures |
Semantic search turns text into a list of numbers (an embedding) and groups what lands close together. "Refund policy" and "return terms" share no words, but as numbers they sit near each other.
Combining the two is the standard. Pick only one and there is a class of question you will always miss.
Documents are not stored whole — they are split into chunks. And a chunk torn out of its page loses its context.
An experiment Anthropic published shows the size of this problem. Prepending one line of context — which document, which section — before storing each chunk cut the retrieval failure rate from 5.7% to 3.7%, a 35% reduction. Adding keyword search brought it to 49%, and re-ranking the retrieved chunks brought it to 67%.
"We added search but it doesn't find things" is usually step 1 missing.
This is the part that matters most in practice.
The second one can be checked. Someone opens page 7 and confirms it in five minutes. The first one has to be re-derived from scratch to verify.
An answer without a source is unusable in an organisation. When it's right you cannot show why; when it's wrong you cannot tell where. The real reason to add retrieval is verifiability, more than accuracy.
| What it does | Where |
|---|---|
| Uploaded files become answer material | Connect Drive |
| PDFs and documents converted to searchable text | turning documents into material |
| Picking only the relevant tools when there are many | when there are too many tools |
The last row is the interesting one. Choosing tools runs on the same principle — tool descriptions are stored as numbers, and only the ones close to the question are pulled in.
It reduces it; it does not remove it. Three cases survive: retrieval fetched the wrong document, the fetched document is itself stale or wrong, and the material has no answer so the model fills the gap with something nearby.
The last one is largely fixable by instruction. State explicitly: "if the provided material does not support an answer, say so."
Decide who is allowed to see what first. Retrieval does not know about permissions. Put HR files in the same store and HR files will surface in anyone's question. Same argument as tools are permissions.
1. Why does the model invent a number when asked for last year's return rate?
You asked about something absent from its training data. It predicts the next word from what it learned, so in an unknown area a plausible number comes out more naturally than "I don't know." Fetching the material and passing it in is the fix.
2. What changes when you prepend one line of context to each chunk?
Retrieval failures drop sharply. In the published experiment, 5.7% → 3.7%, a 35% reduction — and 67% with keyword search and re-ranking added. Without it, "this limit" in an isolated chunk no longer says a limit on what.
3. Why does a citation matter more than accuracy?
Because it makes the answer checkable. With a source, someone confirms it in five minutes. Without one, you cannot show your basis when right or locate the error when wrong. For organisational use, verifiability comes first.
Now spend thirty minutes doing it. The next chapter has you see the difference for yourself → Try attaching sources
"What was our return rate in Q4 last year?"→ our return rate is not in the training data→ yet a plausible-looking number comes out more easily than "I don't know"Page 3: "...for grade B customers this limit is 3 million won..."→ as a chunk: "for grade B customers this limit is 3 million won"→ limit on what? "monthly credit limit" was in the paragraph above, now gone1. Chunk with context attached → 35% fewer failures2. Add keyword search → 49%3. Re-rank what was retrieved → 67%✗ "The Q4 return rate was 3.2%."✓ "The Q4 return rate was 3.2%. (2024_Q4_quality_report.pdf, p.7)"