feat: add shadow AI benchmarking for admin model comparison

Add a new admin-only feature that allows the platform owner to benchmark
the production AI model against up to 2 alternate models (any OpenAI-compatible
API) using real tenant data, without impacting users.

Backend:
- Shared AI caller utility (ai-caller.ts) for OpenAI-compatible endpoints
- Shadow AI module with service, controller, and 3 entities
- 6 admin API endpoints for model config CRUD, run trigger, and history
- Auto-creates shadow_ai_models, shadow_runs, shadow_run_results tables
- Exposes health-scores and investment-planning prompt builders for reuse

Frontend:
- New admin page at /admin/shadow-ai with 3 tabs:
  - Model Configuration (production + 2 alternate slots)
  - Run Comparison (tenant select, feature select, side-by-side results)
  - History (filterable run log with detail drill-down)
- Full side-by-side output display with diff highlighting
- Sidebar navigation link for AI Benchmarking

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
JoeBot
2026-04-05 07:16:38 -04:00
parent 629d112850
commit 4797669591
16 changed files with 2461 additions and 6 deletions

View File

@@ -30,6 +30,7 @@ import { UserPreferencesPage } from './pages/preferences/UserPreferencesPage';
import { OrgMembersPage } from './pages/org-members/OrgMembersPage';
import { AdminPage } from './pages/admin/AdminPage';
import { AdminIdeasPage } from './pages/admin/AdminIdeasPage';
import { AdminShadowAiPage } from './pages/admin/AdminShadowAiPage';
import { AssessmentGroupsPage } from './pages/assessment-groups/AssessmentGroupsPage';
import { CashFlowForecastPage } from './pages/cash-flow/CashFlowForecastPage';
import { MonthlyActualsPage } from './pages/monthly-actuals/MonthlyActualsPage';
@@ -135,6 +136,7 @@ export function App() {
>
<Route index element={<AdminPage />} />
<Route path="ideas" element={<AdminIdeasPage />} />
<Route path="shadow-ai" element={<AdminShadowAiPage />} />
</Route>
{/* Main app routes (require auth + org) */}