QA claims check: Haiku→Sonnet + filter appellant claims only
Two fixes for claims_coverage false negatives (55% → expected ~85%+):
1. Model upgrade: Haiku → Sonnet for semantic matching. Haiku missed
obvious matches (e.g., paragraph about "כריתת עצים" not matching
claim about tree cutting). Sonnet understands context better.
2. Filter: only check appellant/respondent claims, not committee or
permit_applicant claims. Committee claims are defensive positions
("the application complies with the plan") — they don't need to
be "addressed" in the discussion section.
3. Send full discussion text (was truncated to 12K chars).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -124,7 +124,16 @@ async def check_claims_coverage(blocks: list[dict], claims: list[dict]) -> dict:
|
||||
if not claims:
|
||||
return {"name": "claims_coverage", "passed": True, "errors": [], "severity": "critical"}
|
||||
|
||||
# Filter: only claims from original pleadings
|
||||
# Filter: only APPELLANT claims from original pleadings.
|
||||
# Committee/permit_applicant claims are defensive positions, not claims
|
||||
# that need to be "addressed" in the discussion.
|
||||
source_claims = [
|
||||
c for c in claims
|
||||
if c.get("source_document", "") != "block-zayin"
|
||||
and c.get("party_role") in ("appellant", "respondent")
|
||||
]
|
||||
if not source_claims:
|
||||
# Fallback: all non-block-zayin claims
|
||||
source_claims = [c for c in claims if c.get("source_document", "") != "block-zayin"]
|
||||
if not source_claims:
|
||||
source_claims = claims
|
||||
@@ -134,12 +143,12 @@ async def check_claims_coverage(blocks: list[dict], claims: list[dict]) -> dict:
|
||||
for i, c in enumerate(source_claims, 1):
|
||||
claims_text += f"טענה #{i}: {c['claim_text'][:300]}\n"
|
||||
|
||||
# Truncate discussion if needed
|
||||
discussion = yod["content"][:12000]
|
||||
# Send full discussion — don't truncate
|
||||
discussion = yod["content"]
|
||||
|
||||
client = _get_anthropic()
|
||||
message = client.messages.create(
|
||||
model="claude-haiku-4-5-20251001",
|
||||
model="claude-sonnet-4-20250514",
|
||||
max_tokens=8192,
|
||||
messages=[{
|
||||
"role": "user",
|
||||
|
||||
Reference in New Issue
Block a user