Improve signal quality (Exa news category) and remove mock fallback

Exa harvester:
- EXA_CATEGORY=news (default) restricts to news articles, excluding evergreen
  careers/marketing landing pages that were producing irrelevant signals.
- Event-focused query (acquisition/earnings/launch/leadership), dropped
  hiring/career terms that pulled recruiting pages.
- Domain scoping now OFF by default (broad-web news about the account beats
  first-party careers pages); add EXA_EXCLUDE_DOMAINS and loop --domain-filter.

LLM layer:
- Remove the mock reasoner entirely (no provider => loop refuses to run).
- On HTTP 429, WAIT (LLM_RATELIMIT_WAIT_SECONDS, default 900s; honors
  Retry-After) and retry up to LLM_RATELIMIT_MAX_RETRIES (24) instead of
  degrading to mock output.
- If the LLM is ultimately unavailable for an account, skip it and leave its
  signals unconsumed (retried next run) — never fabricate briefs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-24 10:41:41 -04:00
parent 2374ef137d
commit 9131e83c08
9 changed files with 169 additions and 173 deletions

View File

@@ -84,18 +84,18 @@ def _count_request() -> None:
def _build_query(account: dict, domain_scoped: bool) -> str:
# Event-focused: target datable business events, NOT evergreen pages. We avoid
# generic words like "hiring"/"careers" that pull recruiting landing pages.
name = account["accountName"]
if domain_scoped:
# Domain is already restricting results, so keep the query topical/broad.
return (
f"{name} announcements: funding, acquisition, merger, leadership change, "
f"strategy, restructuring, product launch, hiring, earnings, partnership"
)
# No domain restriction -> anchor harder on the company name to limit drift.
return (
f'"{name}" company news: funding, acquisition, merger, leadership change, '
f"strategy, restructuring, product launch, hiring, earnings, partnership"
events = (
"acquisition, merger, earnings results, product launch, partnership, "
"expansion, restructuring, layoffs, leadership change, executive appointment, "
"data center, cloud, modernization, regulatory, strategy"
)
if domain_scoped:
return f"{name} news and announcements: {events}"
# No domain restriction -> anchor on the exact company name to limit drift.
return f'"{name}" news: {events}'
def harvest_news(account: dict) -> list[RawSignal] | None:
@@ -117,8 +117,12 @@ def harvest_news(account: dict) -> list[RawSignal] | None:
"startPublishedDate": start,
"contents": {"text": {"maxCharacters": 1000}},
}
if config.EXA_CATEGORY:
payload["category"] = config.EXA_CATEGORY
if domain_scoped:
payload["includeDomains"] = [domain]
if config.EXA_EXCLUDE_DOMAINS:
payload["excludeDomains"] = config.EXA_EXCLUDE_DOMAINS
req = urllib.request.Request(
_ENDPOINT,
data=json.dumps(payload).encode("utf-8"),
@@ -162,7 +166,8 @@ def harvest_news(account: dict) -> list[RawSignal] | None:
)
)
remaining = usage_status()["monthly_remaining"]
cat = config.EXA_CATEGORY or "any"
scope = f"domain={domain}" if domain_scoped else "broad web"
print(f" [exa] {len(signals)} article(s) for {account['accountName']} ({scope}) "
print(f" [exa] {len(signals)} {cat} article(s) for {account['accountName']} ({scope}) "
f"(monthly budget left: {remaining}/{config.EXA_MONTHLY_REQUEST_CAP})")
return signals