Add OpenAI-compatible LLM provider (URL/model/key configurable)

- New provider 'openai' calls any OpenAI-compatible /chat/completions endpoint
  (OpenAI, OpenRouter, Together, Groq, NVIDIA, vLLM, Ollama, ...), set via
  OPENAI_BASE_URL / OPENAI_API_KEY / OPENAI_MODEL. Now the default; auto-detects.
- Robustness for models that degenerate (e.g. Kimi repetition loops): anti-
  repetition sampling (temperature + frequency penalty), and a schema-aware
  retry — call_json(require_keys=...) retries a fresh sample on degenerate or
  unparseable output (LLMRetryable). Relevance requires 'signals', synth 'briefs'.
- response_format=json_object (toggle via OPENAI_JSON_MODE); base URL accepts a
  full /chat/completions URL or a /v1 base.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-24 10:19:52 -04:00
parent b4e3ed9d9b
commit 2374ef137d
6 changed files with 135 additions and 22 deletions

View File

@@ -56,7 +56,14 @@ The relevance filter and synthesizer call whichever provider is configured. Set
`.env`:
```bash
# Gemini (current default)
# OpenAI-compatible (default) — OpenAI, OpenRouter, Together, Groq, vLLM, Ollama, ...
LLM_PROVIDER=openai
OPENAI_BASE_URL=https://api.openai.com/v1 # any /chat/completions endpoint
OPENAI_API_KEY=...
OPENAI_MODEL=gpt-4o-mini
OPENAI_JSON_MODE=1 # set 0 if the endpoint rejects json_object
# or Gemini
LLM_PROVIDER=gemini
GEMINI_API_KEY=...
GEMINI_MODEL=gemini-2.5-flash
@@ -67,9 +74,10 @@ ANTHROPIC_API_KEY=...
LLM_MODEL=claude-sonnet-4-6
```
If `LLM_PROVIDER` is blank it auto-detects from whichever key is present (Gemini
preferred). With no key, a deterministic mock reasoner produces grounded briefs so
the full pipeline still runs offline. Both providers use plain `urllib` no SDKs.
If `LLM_PROVIDER` is blank it auto-detects from whichever key is present (OpenAI-
compatible preferred). With no key, a deterministic mock reasoner produces grounded
briefs so the full pipeline still runs offline. All providers use plain `urllib`
no SDKs. LLM calls are batched (one relevance call + one synthesis call per account).
## Exa.ai news connector & free-plan protection