Skip to content

Commit

Agents recall Docs passages only when they are about the question: the meaning floor is 0.7, measured on real pages, so "hello" or an unrelated question no longer pulls in runbooks

syntaqxcommitted Parentbee5a81Browse files
2 files+9−50/2 viewed
+1−1
4646 });
4747
4848 test("required spaces come first, then the rest", () => {
49− const picked = pickPassages([c("a:0", "s1", 0.9), c("b:0", "s2", 0.65), c("c:0", "s1", 0.85)], { allowed: new Set(["s1", "s2"]), required: ["s2"], limit: 2 });
49+ const picked = pickPassages([c("a:0", "s1", 0.9), c("b:0", "s2", 0.75), c("c:0", "s1", 0.85)], { allowed: new Set(["s1", "s2"]), required: ["s2"], limit: 2 });
5050 assert.deepEqual(
5151 picked.map((p) => p.id),
5252 ["b:0", "a:0"],
+8−4
88
99 /**
1010 * The least cosine similarity a passage needs to count as being about the
11− * query. bge-base-en-v1.5 scores unrelated English around 0.4 to 0.55 and
12− * a passage on the asked-about topic from about 0.65; 0.6 keeps recall
13− * quiet when the docs say nothing about it.
11+ * query. Measured on g1t's own docs folder (606 passages, chunked and
12+ * embedded as here, bge-base-en-v1.5 with Workers AI's mean pooling): the
13+ * best passage for 16 questions the docs answer scored 0.72 to 0.84, while
14+ * the best for 16 they don't ("hello", a recipe, Postgres tuning, a
15+ * vacation policy, SSO with Okta) scored 0.54 to 0.67. At 0.6, 11 of those
16+ * 16 pulled in passages (up to 89 above it); 0.7 keeps every answer and
17+ * keeps recall quiet when the docs say nothing about the question.
1418 */
15−export const MEANING_FLOOR = 0.6;
19+export const MEANING_FLOOR = 0.7;
1620 /** The score a passage found only by its words carries: below the floor, so callers can tell. */
1721 export const WORDS_SCORE = 0.5;
1822 /** Nearest passages asked of the index. */