ProspectingAgentLoop/sales-agent/README.md

# Sales Prospecting Agent Loop — MVP (v1)

A runnable first cut of the closed-loop account-intelligence system described in
[`../sales-prospecting-agent-loop.md`](../sales-prospecting-agent-loop.md). It
harvests signals per account, evaluates them against a capability library, scores
and synthesizes opportunity briefs, delivers them, and feeds rep feedback back into
scoring — the full loop, end to end.

## ⚠️ What's real vs. placeholder in this version

This MVP is built to **run today with zero setup** so we can review output quality
and iterate. Deliberate v1 simplifications (we'll revisit each tomorrow):

| Design calls for | v1 MVP uses | Why |
|---|---|---|
| TypeScript / Node + BullMQ | **Python 3, stdlib only** | Node isn't installed on this machine; concepts map 1:1. Easy to port. |
| Proxycurl/Apify, JSearch, SEC EDGAR | **Fixture file** (`data/fixtures.json`) | No API keys needed; swap each connector for the real API. |
| Exa.ai news/web search | **LIVE** when `EXA_API_KEY` set (else fixtures) | Real connector with hard request ceilings — see below. |
| Salesforce/HubSpot | **`data/accounts.json`** + console write-back | Read context + log-activity stub. |
| Postgres + Redis + pgvector | **JSON files** under `output/state/` | Hash dedup works; semantic dedup is a documented v2 hook. |
| Vector ANN capability pre-filter | **Keyword-overlap pre-filter** | Same interface, no embedding service. |
| Slack Bolt + SendGrid | **Console + markdown files** | Briefs render Slack-style; digest writes to `output/digest.md`. |
| LLM relevance + synthesis | **OpenAI-compatible / Gemini / Anthropic** (required) | Provider-agnostic (`LLM_PROVIDER`). **No mock fallback** — on a rate limit it waits and retries; without a key it refuses to run. |

## Run it

```bash
cd sales-agent

# 1. Run the loop over all target accounts (harvest -> deliver)
python3 run.py loop

# Single account or one tier:
python3 run.py loop --account acct_meridian
python3 run.py loop --tier strategic

# 2. See the per-rep daily digest
python3 run.py digest
python3 run.py digest --rep rep_sarah

# 3. Record rep feedback (closes the loop)
python3 run.py feedback --brief <BRIEF_ID> --action act_on
#   actions: act_on | snooze | reject | won | lost   (reject also suppresses 14d)

# 4. Recompute scoring weights from feedback (the weekly recalibration job)
python3 run.py recalibrate

# Helpers
python3 run.py accounts   # list target accounts
python3 run.py briefs     # list all generated briefs + status
```

### LLM reasoning (provider-agnostic)

The relevance filter and synthesizer call whichever provider is configured. Set in
`.env`:

```bash
# OpenAI-compatible (default) — OpenAI, OpenRouter, Together, Groq, vLLM, Ollama, ...
LLM_PROVIDER=openai
OPENAI_BASE_URL=https://api.openai.com/v1   # any /chat/completions endpoint
OPENAI_API_KEY=...
OPENAI_MODEL=gpt-4o-mini
OPENAI_JSON_MODE=1                          # set 0 if the endpoint rejects json_object

# or Gemini
LLM_PROVIDER=gemini
GEMINI_API_KEY=...
GEMINI_MODEL=gemini-2.5-flash

# or Anthropic
LLM_PROVIDER=anthropic
ANTHROPIC_API_KEY=...
LLM_MODEL=claude-sonnet-4-6
```

If `LLM_PROVIDER` is blank it auto-detects from whichever key is present (OpenAI-
compatible preferred). All providers use plain `urllib` — no SDKs. LLM calls are
batched (one relevance call + one synthesis call per account).

**No mock fallback.** Quality is the priority: the loop refuses to run without a
configured provider, and on HTTP 429 it **waits and retries** (`LLM_RATELIMIT_WAIT_
SECONDS`, default 900s; honors `Retry-After`) up to `LLM_RATELIMIT_MAX_RETRIES`
(default 24) rather than emitting low-quality output. If the LLM is ultimately
unavailable for an account, that account is skipped and its signals are left
unconsumed (retried next run) — never fabricated. Degenerate/unparseable responses
(e.g. a model repetition loop) are re-sampled `LLM_CONTENT_RETRIES` times.

## Exa.ai news connector & free-plan protection

The news harvester ([`src/connectors/exa.py`](src/connectors/exa.py)) calls the real
Exa search API (`POST https://api.exa.ai/search`) when `EXA_API_KEY` is set, and
falls back to fixtures otherwise. **Two hard ceilings keep you inside the free plan**
(Exa's free tier is ~20,000 requests/month; we default far below that):

- `EXA_MONTHLY_REQUEST_CAP` (default **1000**) — a persistent counter in
  `output/state/exa_usage.json` that resets at the start of each calendar month.
  Once hit, the connector refuses to call Exa and falls back to fixtures.
- `EXA_PER_RUN_REQUEST_CAP` (default **25**) — a per-process cap so a single loop
  can't burn through the budget.

A request is only counted when a call actually reaches Exa (a billable call);
network failures that never hit Exa are not counted. Each account uses **one** Exa
request per loop run (so the default 3 sample accounts = 3 requests/run).

```bash
python3 run.py exa-usage      # show remaining monthly budget
```

### Signal quality: news category & domain scoping

The single biggest lever on signal quality is **`EXA_CATEGORY=news`** (default): it
restricts results to news articles and keeps out evergreen pages (careers, marketing,
generic corporate). The query is also event-focused (acquisition, earnings, product
launch, leadership change, ...) rather than pulling recruiting pages.

Domain scoping is **off by default** — with `category=news`, broad-web search finds
actual coverage *about* the account (PRNewswire, CNBC, trade press). Scope to the
company's own newsroom only if you want first-party press releases:

```bash
python3 run.py loop --domain-filter      # account's own domain (newsroom/press only)
python3 run.py loop --no-domain-filter   # broad-web news about the account (default)
# or set EXA_DOMAIN_FILTER / EXA_CATEGORY / EXA_EXCLUDE_DOMAINS in .env
```

Tune all limits in `.env` (`EXA_MONTHLY_REQUEST_CAP`, `EXA_PER_RUN_REQUEST_CAP`,
`EXA_RESULTS_PER_QUERY`, `EXA_LOOKBACK_DAYS`, `EXA_CATEGORY`). Set `EXA_ENABLED=0` to
force fixtures.

## How the loop maps to the design

```
run.py loop
  └─ src/pipeline.run_account_cycle           (design §7.3)
       1. agents/harvester      Phase 1  signal harvest  (connectors/*)
       2. agents/normalizer     Phase 2  normalize + hash dedup
       3. agents/relevance      Phase 3  LLM vs. capability library
       4. agents/synthesizer    Phase 4  opportunity briefs
       5. agents/scorer         Phase 5  composite score + priority
       6. delivery/render       Phase 7  deliver (console/md) + CRM log
       7. memory/store          Phase 6  account memory + suppression
  feedback ─ memory/feedback    §6       feedback store + recalibration
```

## Project layout

```
sales-agent/
├── run.py                 CLI entrypoint
├── data/
│   ├── accounts.json      target accounts (stand-in for CRM)
│   ├── capabilities.json  the Capability Library (the "brain")
│   └── fixtures.json      placeholder harvested signals
├── src/
│   ├── config.py          paths, thresholds, .env loader
│   ├── types.py           RawSignal / NormalizedSignal / Brief dataclasses
│   ├── llm.py             Anthropic call (urllib) + mock fallback
│   ├── capabilities.py    library loader + keyword pre-filter
│   ├── connectors/        linkedin, news, jobs, crm (fixture-backed)
│   ├── agents/            harvester, normalizer, relevance, synthesizer, scorer
│   ├── memory/            store (dedup/memory/briefs) + feedback (recalibration)
│   ├── delivery/          render (briefs) + digest
│   └── pipeline.py        the per-account orchestration
└── output/                generated briefs, digest, and JSON state (created on run)
```

## Suggested review focus for tomorrow

1. **Capability Library** (`data/capabilities.json`) — now seeded with 40 real
   Broadcom/VMware capabilities across Mainframe (13), Enterprise Software (9),
   Identity & Access Management (6), and VMware by Broadcom (12). Review wording of
   buying-signals/anti-signals and add any missing SKUs. The prior placeholder library
   is kept at `data/capabilities.placeholder.json`.
2. **Accounts** (`data/accounts.json`) — sample BSG target accounts; swap for a real
   target-account list + CRM pull.
3. **Scoring weights** — the composite formula in `agents/scorer.py` uses the design's
   weights; tune the tier/seniority/timing tables.
4. **Connectors** — Exa.ai news is now **live** (with request caps). Improve query
   precision (a bare company name pulls in stock-ticker noise — add domain filtering
   via the account `domain`), then wire the next source (jobs/LinkedIn).
5. **Reasoning quality** — set an API key and compare Claude briefs vs. the mock.
```