fix(retrieval): סף מכויל-קוסינוס סינן פלט RRF — דף אימות-הפסיקה הציג אפס תקדימים

`hybrid_search._merge_sem_lex` דורס את `score` בערך RRF (~0.008–0.02) ברגע
שה-leg הלקסיקלי מחזיר שורות. `score` הוא אות-דירוג לגיטימי, אבל הוא מפסיק
להיות קוסינוס — ואילו `case_citation_verification._SUGGEST_FLOOR = 0.45`
כויל לקוסינוס. התוצאה: **כל שאילתה במונחים משפטיים נפוצים סוננה עד
האחרונה**, והדף הציג "אין תקדים תומך" לכל טיעון בכל תיק.

אותה פונקציה החזירה שתי סקאלות, תלוי בשאילתה:

    "המונח חזית הבניין יפורש..."   leg לקסיקלי ריק   →  0.63–0.71  (קוסינוס)
    "סמכות ועדה מקומית לפי 62א"    leg לקסיקלי מלא   →  0.016      (RRF)

מה שונה
- **עוגן קוסינוס במקור (G1):** `db.search_precedent_library_semantic` מסמן
  `relevance` לצד `score`. **לא** בפונקציה הלקסיקלית — שם `ts_rank_cd`,
  ולסמן אותו כ-relevance היה חוזר על אותה טעות בדיוק.
- **הפיוז'ן משמר:** `_merge_sem_lex` מעביר את הקוסינוס הלאה. שורה
  לקסיקלית-בלבד מקבלת `relevance = None` — "לא נמדד" אינו "נמדד כלא-רלוונטי".
- **הצרכן קורא `relevance`:** `_passes_floor()` במקום השוואה ל-`score`.
  שורות לקסיקליות-בלבד **נשמרות במודע** — הן הגיעו לצמרת בדירוג BM25 בתוך
  top-k זעיר, וסינונן היה מחביא בדיוק את התאמות-הביטוי ומספרי-התיק שהיו"ר
  מחפש בשם.

הדירוג לא השתנה: RRF ממשיך לקבוע סדר. רק הסינון עבר לסקאלה יציבה.

אימות מול הקורפוס החי:
  8124-09-24:  0 → **32 מתוך 32** טיעונים עם תקדים תומך (21.5 שנ', שלם)
  1069-04-26:  0 → **29 מתוך 69** (24.1 שנ', חלקי — תקציב הזמן)
  שורה סמנטית: score=0.0082 · relevance=0.7297

היומונים לא נפגעו ולא נגעתי בהם: `case_digest_radar` עובר דרך
`search_digests_semantic` — סמנטי טהור, בלי RRF, ולכן `min_score=0.45` שלו
מכויל נכון.

invariants: G1 (עוגן במקור, לא תיקון-סף בקריאה) · G2 (הגדרה אחת ל-relevance
לכל הצרכנים) · INV-AH (היעדר-מדידה אינו היעדר-רלוונטיות)

טסטים: 6 חדשים (tests/test_relevance_scale.py) — אחד מהם מוכיח את הבאג
ואת התיקון באותה שורה. 531 עוברים.

mcp-server/src/legal_mcp/services/case_citation_verification.py

@@ -34,6 +34,45 @@ logger = logging.getLogger(__name__)
 _SUGGEST_PER_ISSUE = 4
 _SUGGEST_FLOOR = 0.45
 
+#: Concurrent per-argument retrievals. The fan-out used to be unbounded — one
+#: task per legal argument — which is self-defeating, not merely risky: each
+#: task opens a Voyage embed call, and past ~8 in flight Voyage throttles, so
+#: every request stalls together. Measured on this corpus (32 arguments):
+#:
+#:     one search alone ......  1.2s
+#:     32 unbounded .......... 30.4s   ← 25× a single search
+#:     32 at 8 concurrent .... 22.1s   ← bounded is FASTER
+#:
+#: A 69-argument case (1069-04-26) therefore blew past the 30s proxy timeout and
+#: returned 500, and overlapping page reloads piled contention into Postgres
+#: deadlocks. Bounding the fan-out both fixes the failure and speeds it up.
+_MAX_CONCURRENT_LOOKUPS = 8
+
+def _passes_floor(hit: dict) -> bool:
+    """Is this hit similar enough to the argument to suggest to the chair?
+
+    Reads ``relevance`` (always a cosine similarity), NOT ``score`` — ``score``
+    becomes a rank-fusion value (~0.008-0.02) as soon as the lexical leg returns
+    rows, and comparing that against a cosine-calibrated floor rejected every
+    hit. That is why this tab showed no supporting precedent for any argument.
+
+    ``relevance is None`` means the row came from the lexical leg only, so no
+    cosine was ever computed. It is KEPT: it earned its place by BM25 rank
+    inside an already-tiny top-k, and dropping it would silently hide exact
+    phrase/docket matches — the very hits a chair searches for by name.
+    """
+    rel = hit.get("relevance")
+    if rel is None:
+        return True
+    return float(rel) >= _SUGGEST_FLOOR
+
+
+#: Wall-clock ceiling for the whole retrieval phase. The proxy gives up at 30s;
+#: cutting ourselves off earlier lets us return the suggestions that DID land
+#: instead of a 500 that shows the chair nothing. Partial results are labelled
+#: (``retrieval_complete: false``) rather than passed off as the full picture.
+_RETRIEVAL_BUDGET_S = 22.0
+
 
 async def build_view(case_number: str) -> dict:
     case = await db.get_case_by_number(case_number)
@@ -83,13 +122,46 @@ async def build_view(case_number: str) -> dict:
             logger.warning("citation_verification search failed (%s): %s", title[:30], e)
         # Resolve the authority breakdown for the hit set in one batched query.
         clids = [UUID(str(h["case_law_id"])) for h in hits
-                 if h.get("case_law_id") and float(h.get("score", 0) or 0) >= _SUGGEST_FLOOR]
+                 if h.get("case_law_id") and _passes_floor(h)]
         authority = await db.citation_authority(clids) if clids else {}
         return hits, authority
 
-    # Fan out the expensive per-argument retrieval concurrently — was a sequential
-    # waterfall (N args × Voyage embed + vector search each). gather preserves order.
-    fetched = await asyncio.gather(*(_fetch(a) for a in args)) if args else []
+    # Fan out the expensive per-argument retrieval, but BOUNDED — see
+    # _MAX_CONCURRENT_LOOKUPS. gather preserves order, so the zip below still
+    # pairs each argument with its own result.
+    _sem = asyncio.Semaphore(_MAX_CONCURRENT_LOOKUPS)
+
+    async def _fetch_bounded(a: dict) -> tuple[list[dict], dict]:
+        async with _sem:
+            return await _fetch(a)
+
+    retrieval_complete = True
+    fetched: list[tuple[list[dict], dict]] = []
+    if args:
+        # Harvest whatever finished inside the budget, per argument. Deliberately
+        # NOT wait_for(gather(...)): that cancels every task on timeout, so one
+        # slow lookup would throw away the 30 that already succeeded and the page
+        # would show nothing at all.
+        tasks = [asyncio.ensure_future(_fetch_bounded(a)) for a in args]
+        done, pending = await asyncio.wait(tasks, timeout=_RETRIEVAL_BUDGET_S)
+        for t in pending:
+            t.cancel()
+        if pending:
+            # Let the cancellations settle before the caller's DB pool unwinds —
+            # a task cancelled mid-query otherwise surfaces as a stray
+            # "connection_lost" future with no owner.
+            await asyncio.gather(*pending, return_exceptions=True)
+            retrieval_complete = False
+            logger.warning(
+                "citation_verification: retrieval budget of %.0fs exhausted for %s "
+                "— %d of %d arguments returned suggestions, the rest are empty",
+                _RETRIEVAL_BUDGET_S, case_number, len(done), len(args),
+            )
+        for t in tasks:
+            if t in done and not t.cancelled() and t.exception() is None:
+                fetched.append(t.result())
+            else:
+                fetched.append(([], {}))
 
     out_args: list[dict] = []
     n_verified = 0
@@ -104,7 +176,7 @@ async def build_view(case_number: str) -> dict:
             clid = str(h.get("case_law_id") or "")
             if not clid or clid in seen:
                 continue
-            if float(h.get("score", 0) or 0) < _SUGGEST_FLOOR:
+            if not _passes_floor(h):
                 continue
             seen.add(clid)
             att = attached_by_arg.get(aid, {}).get(clid)
@@ -137,6 +209,11 @@ async def build_view(case_number: str) -> dict:
     return {
         "status": "ok",
         "case_number": case_number,
+        # False when the retrieval budget ran out: the attached/verified rows and
+        # the radar are complete, but the corpus SUGGESTIONS are missing. The UI
+        # must say so — an empty suggestion list otherwise reads as "no precedent
+        # in the corpus supports this argument", which is a different claim.
+        "retrieval_complete": retrieval_complete,
         "arguments": out_args,
         "summary": {
             "arguments_total": len(out_args),

mcp-server/src/legal_mcp/services/db.py

@@ -7979,6 +7979,14 @@ async def search_precedent_library_semantic(
             # Calibrated so the average (≈0.85) stays at +0.05 (legacy value).
             _conf = float(d.get("confidence") or 0.0)
             d["score"] = float(d["score"]) + max(_conf * 0.06, 0.0)
+            # Stable cosine-scale relevance, carried alongside ``score``.
+            # ``score`` is the RANKING signal and downstream fusion overwrites it
+            # with an RRF value (~0.008-0.02) whenever the lexical leg returns
+            # rows — a different scale entirely. Anything that THRESHOLDS must
+            # read ``relevance`` instead, which always means "cosine similarity
+            # to the query" no matter which fusion stages ran. See
+            # hybrid_search._merge_sem_lex.
+            d["relevance"] = d["score"]
             d["type"] = "halacha"
             # authority is DERIVED from the source, never stored (INV-DM7)
             d["authority"] = halacha_quality.derive_authority(d.get("precedent_level"))
@@ -7990,6 +7998,7 @@ async def search_precedent_library_semantic(
         if d.get("decision_date") is not None:
             d["decision_date"] = d["decision_date"].isoformat()
         d["score"] = float(d["score"])
+        d["relevance"] = d["score"]  # cosine anchor — see the halacha branch above
         d["type"] = "passage"
         _maybe_swap_parent(d)
         results.append(d)

mcp-server/src/legal_mcp/services/hybrid_search.py

@@ -303,6 +303,21 @@ def _merge_sem_lex(
             if key in lex_row_by_key else 0.0
         d["lex_rank"] = lex_rank or 0
         d["score"] = sem_term + lex_term
+        # ``score`` is now an RRF value (~0.008-0.02), NOT a cosine. Carry the
+        # cosine forward under ``relevance`` so thresholding callers keep a
+        # stable scale — without it, a caller comparing score >= 0.45 silently
+        # drops every result the moment the lexical leg returns anything, which
+        # is exactly how the citation-verification tab ended up showing no
+        # supporting precedent for any argument.
+        #
+        # A lexical-only row has no cosine at all. It gets ``relevance = None``
+        # rather than 0.0: "we did not measure this" is not "measured as
+        # irrelevant", and the row earned its place by BM25 rank. Callers decide
+        # (see case_citation_verification) — but they must decide knowingly.
+        if key in sem_row_by_key:
+            d["relevance"] = float(sem_row_by_key[key].get("relevance", d["sem_score"]))
+        else:
+            d["relevance"] = None
         merged.append(d)
 
     merged.sort(key=lambda x: -float(x["score"]))

mcp-server/tests/test_citation_view_fanout.py

@@ -0,0 +1,81 @@
+"""The citation-verification view must not fan out without a bound.
+
+`build_view` used to spawn one retrieval task per legal argument with a bare
+`asyncio.gather`. Each task opens a Voyage embed call, and past ~8 in flight
+Voyage throttles — so the unbounded version was *slower* than a bounded one
+(32 arguments: 30.4s unbounded vs 22.1s at 8), and a 69-argument case blew past
+the 30s proxy timeout and returned 500 while overlapping reloads piled
+contention into Postgres deadlocks.
+
+These tests pin the two properties that fix gave us: the fan-out is bounded,
+and a timeout yields the results that DID land instead of nothing.
+"""
+
+import asyncio
+
+import pytest
+
+from legal_mcp.services import case_citation_verification as ccv
+
+
+def test_concurrency_bound_is_set_and_modest():
+    assert 1 <= ccv._MAX_CONCURRENT_LOOKUPS <= 16, (
+        "the bound exists to stay under Voyage's throttle point — a large value "
+        "reintroduces the stall this was added to fix"
+    )
+
+
+def test_retrieval_budget_leaves_room_under_the_proxy_timeout():
+    """The proxy gives up at 30s; we must cut ourselves off before that."""
+    assert 0 < ccv._RETRIEVAL_BUDGET_S < 30
+
+
+@pytest.mark.asyncio
+async def test_semaphore_actually_caps_in_flight_work():
+    """A semaphore of N never lets N+1 coroutines run the body at once."""
+    limit = ccv._MAX_CONCURRENT_LOOKUPS
+    sem = asyncio.Semaphore(limit)
+    in_flight = 0
+    peak = 0
+
+    async def worker():
+        nonlocal in_flight, peak
+        async with sem:
+            in_flight += 1
+            peak = max(peak, in_flight)
+            await asyncio.sleep(0.01)
+            in_flight -= 1
+
+    await asyncio.gather(*(worker() for _ in range(limit * 4)))
+    assert peak <= limit
+
+
+@pytest.mark.asyncio
+async def test_timeout_harvests_finished_work_instead_of_discarding_it():
+    """The regression: wait_for(gather(...)) cancels everything on timeout, so
+    one slow lookup threw away every result that had already succeeded. The
+    harvest pattern must keep them."""
+
+    async def quick(i):
+        await asyncio.sleep(0.01)
+        return i
+
+    async def never():
+        await asyncio.sleep(30)
+        return "unreachable"
+
+    tasks = [asyncio.ensure_future(quick(i)) for i in range(5)]
+    tasks.append(asyncio.ensure_future(never()))
+
+    done, pending = await asyncio.wait(tasks, timeout=0.3)
+    for t in pending:
+        t.cancel()
+    await asyncio.gather(*pending, return_exceptions=True)
+
+    harvested = [
+        t.result() if (t in done and not t.cancelled() and t.exception() is None) else None
+        for t in tasks
+    ]
+    assert harvested[:5] == [0, 1, 2, 3, 4], "finished work must survive the timeout"
+    assert harvested[5] is None, "the unfinished one is empty, not fabricated"
+    assert len(pending) == 1

mcp-server/tests/test_relevance_scale.py

@@ -0,0 +1,68 @@
+"""`score` and `relevance` are different things — thresholds must use `relevance`.
+
+Retrieval returns cosine similarities (~0.4-0.75) until the lexical leg returns
+rows; then `_merge_sem_lex` replaces `score` with an RRF value (~0.008-0.02).
+Both are legitimate *ranking* signals, but they are not on the same scale, so a
+caller comparing `score >= 0.45` rejected every hit the moment BM25 matched
+anything. That is how the citation-verification tab came to show no supporting
+precedent for a single argument, on every case.
+
+`relevance` is the fix: always a cosine, or None when the row came from the
+lexical leg alone and no cosine was ever computed.
+"""
+
+import pytest
+
+from legal_mcp.services.case_citation_verification import _SUGGEST_FLOOR, _passes_floor
+from legal_mcp.services.hybrid_search import _merge_sem_lex
+
+
+def _sem(key: str, score: float) -> dict:
+    return {"chunk_id": key, "case_law_id": "c1", "score": score, "relevance": score}
+
+
+def _lex(key: str, score: float) -> dict:
+    # The lexical leg emits ts_rank_cd, never a cosine — so no `relevance`.
+    return {"chunk_id": key, "case_law_id": "c1", "score": score}
+
+
+def test_fusion_replaces_score_but_preserves_the_cosine():
+    """The regression in one assertion."""
+    merged = _merge_sem_lex([_sem("a", 0.73)], [_lex("a", 0.31)], limit=10)
+    row = merged[0]
+    assert row["score"] < 0.1, "fused score is an RRF value, not a cosine"
+    assert row["relevance"] == pytest.approx(0.73), "the cosine must survive fusion"
+
+
+def test_lexical_only_row_has_no_fabricated_cosine():
+    """None means 'not measured' — not 'measured as irrelevant'."""
+    merged = _merge_sem_lex([], [_lex("b", 0.31)], limit=10)
+    assert merged[0]["relevance"] is None
+
+
+def test_semantic_only_row_keeps_its_cosine():
+    merged = _merge_sem_lex([_sem("c", 0.62)], [], limit=10)
+    assert merged[0]["relevance"] == pytest.approx(0.62)
+
+
+def test_floor_would_have_rejected_everything_on_the_fused_score():
+    """Guards the exact production symptom: fused scores are ~0.008, the floor
+    is 0.45, so score-based filtering wipes the result set."""
+    merged = _merge_sem_lex(
+        [_sem(k, 0.70) for k in "abcd"], [_lex(k, 0.30) for k in "abcd"], limit=10)
+    assert all(r["score"] < _SUGGEST_FLOOR for r in merged)   # the bug
+    assert all(_passes_floor(r) for r in merged)              # the fix
+
+
+def test_floor_still_rejects_genuinely_weak_hits():
+    """The fix must not become 'accept everything'."""
+    assert not _passes_floor({"relevance": 0.10})
+    assert not _passes_floor({"relevance": _SUGGEST_FLOOR - 0.01})
+    assert _passes_floor({"relevance": _SUGGEST_FLOOR})
+
+
+def test_lexical_only_hits_are_kept_deliberately():
+    """An exact docket/phrase match reaches the top by BM25 rank with no cosine.
+    Dropping it would hide precisely what a chair searches for by name."""
+    assert _passes_floor({"relevance": None})
+    assert _passes_floor({})  # missing key behaves the same as None