Improve signal quality (Exa news category) and remove mock fallback

Exa harvester:
- EXA_CATEGORY=news (default) restricts to news articles, excluding evergreen
  careers/marketing landing pages that were producing irrelevant signals.
- Event-focused query (acquisition/earnings/launch/leadership), dropped
  hiring/career terms that pulled recruiting pages.
- Domain scoping now OFF by default (broad-web news about the account beats
  first-party careers pages); add EXA_EXCLUDE_DOMAINS and loop --domain-filter.

LLM layer:
- Remove the mock reasoner entirely (no provider => loop refuses to run).
- On HTTP 429, WAIT (LLM_RATELIMIT_WAIT_SECONDS, default 900s; honors
  Retry-After) and retry up to LLM_RATELIMIT_MAX_RETRIES (24) instead of
  degrading to mock output.
- If the LLM is ultimately unavailable for an account, skip it and leave its
  signals unconsumed (retried next run) — never fabricate briefs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-24 10:41:41 -04:00
parent 2374ef137d
commit 9131e83c08
9 changed files with 169 additions and 173 deletions

View File

@@ -20,7 +20,7 @@ and iterate. Deliberate v1 simplifications (we'll revisit each tomorrow):
| Postgres + Redis + pgvector | **JSON files** under `output/state/` | Hash dedup works; semantic dedup is a documented v2 hook. |
| Vector ANN capability pre-filter | **Keyword-overlap pre-filter** | Same interface, no embedding service. |
| Slack Bolt + SendGrid | **Console + markdown files** | Briefs render Slack-style; digest writes to `output/digest.md`. |
| LLM relevance + synthesis | **Gemini or Anthropic when a key is set, else a deterministic mock reasoner** | Provider-agnostic (`LLM_PROVIDER`); loop runs offline without a key. |
| LLM relevance + synthesis | **OpenAI-compatible / Gemini / Anthropic** (required) | Provider-agnostic (`LLM_PROVIDER`). **No mock fallback** — on a rate limit it waits and retries; without a key it refuses to run. |
## Run it
@@ -75,9 +75,16 @@ LLM_MODEL=claude-sonnet-4-6
```
If `LLM_PROVIDER` is blank it auto-detects from whichever key is present (OpenAI-
compatible preferred). With no key, a deterministic mock reasoner produces grounded
briefs so the full pipeline still runs offline. All providers use plain `urllib`
no SDKs. LLM calls are batched (one relevance call + one synthesis call per account).
compatible preferred). All providers use plain `urllib` — no SDKs. LLM calls are
batched (one relevance call + one synthesis call per account).
**No mock fallback.** Quality is the priority: the loop refuses to run without a
configured provider, and on HTTP 429 it **waits and retries** (`LLM_RATELIMIT_WAIT_
SECONDS`, default 900s; honors `Retry-After`) up to `LLM_RATELIMIT_MAX_RETRIES`
(default 24) rather than emitting low-quality output. If the LLM is ultimately
unavailable for an account, that account is skipped and its signals are left
unconsumed (retried next run) — never fabricated. Degenerate/unparseable responses
(e.g. a model repetition loop) are re-sampled `LLM_CONTENT_RETRIES` times.
## Exa.ai news connector & free-plan protection
@@ -100,23 +107,26 @@ request per loop run (so the default 3 sample accounts = 3 requests/run).
python3 run.py exa-usage # show remaining monthly budget
```
### Domain scoping (default on)
### Signal quality: news category & domain scoping
By default, news search is **scoped to each account's own `domain`** (via Exa
`includeDomains`) — newsroom, press releases, investor-relations — which cuts out
stock-ticker and analyst noise from a bare company-name search. Toggle it:
The single biggest lever on signal quality is **`EXA_CATEGORY=news`** (default): it
restricts results to news articles and keeps out evergreen pages (careers, marketing,
generic corporate). The query is also event-focused (acquisition, earnings, product
launch, leadership change, ...) rather than pulling recruiting pages.
Domain scoping is **off by default** — with `category=news`, broad-web search finds
actual coverage *about* the account (PRNewswire, CNBC, trade press). Scope to the
company's own newsroom only if you want first-party press releases:
```bash
python3 run.py loop --no-domain-filter # widen to broad third-party web coverage
# or set EXA_DOMAIN_FILTER=0 in .env to default to broad
python3 run.py loop --domain-filter # account's own domain (newsroom/press only)
python3 run.py loop --no-domain-filter # broad-web news about the account (default)
# or set EXA_DOMAIN_FILTER / EXA_CATEGORY / EXA_EXCLUDE_DOMAINS in .env
```
Note: domain scoping only returns results when the account's real domain has indexed
content. The fictional sample accounts return 0 domain-scoped articles by design —
real target-account domains will return their first-party news.
Tune all limits in `.env` (`EXA_MONTHLY_REQUEST_CAP`, `EXA_PER_RUN_REQUEST_CAP`,
`EXA_RESULTS_PER_QUERY`, `EXA_LOOKBACK_DAYS`). Set `EXA_ENABLED=0` to force fixtures.
`EXA_RESULTS_PER_QUERY`, `EXA_LOOKBACK_DAYS`, `EXA_CATEGORY`). Set `EXA_ENABLED=0` to
force fixtures.
## How the loop maps to the design