8 Commits

Author SHA1 Message Date
JoeBot
4797669591 feat: add shadow AI benchmarking for admin model comparison
Add a new admin-only feature that allows the platform owner to benchmark
the production AI model against up to 2 alternate models (any OpenAI-compatible
API) using real tenant data, without impacting users.

Backend:
- Shared AI caller utility (ai-caller.ts) for OpenAI-compatible endpoints
- Shadow AI module with service, controller, and 3 entities
- 6 admin API endpoints for model config CRUD, run trigger, and history
- Auto-creates shadow_ai_models, shadow_runs, shadow_run_results tables
- Exposes health-scores and investment-planning prompt builders for reuse

Frontend:
- New admin page at /admin/shadow-ai with 3 tabs:
  - Model Configuration (production + 2 alternate slots)
  - Run Comparison (tenant select, feature select, side-by-side results)
  - History (filterable run log with detail drill-down)
- Full side-by-side output display with diff highlighting
- Sidebar navigation link for AI Benchmarking

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 07:50:59 -04:00
629d112850 Merge pull request 'Add k6 load testing suite and CLAUDE.md' (#9) from claude/beautiful-gauss into main
Reviewed-on: #9
2026-04-02 17:42:35 -04:00
32506d6a2e Merge branch 'main' into claude/beautiful-gauss 2026-04-02 17:42:24 -04:00
9a60970837 Updated Version 2026-04-02 17:41:49 -04:00
1ade446187 Merge pull request 'ideation-feature' (#11) from ideation-feature into main
Reviewed-on: #11
2026-04-02 17:39:32 -04:00
JoeBot
d430b96b51 feat: add admin ideas management page with private notes
Adds a dedicated super admin page for managing idea submissions across
all tenants. Includes status summary cards, filterable/searchable table,
detail modal with status updates, and private admin notes for internal
tracking (sprint refs, thoughts, follow-ups). Notes are not visible to
tenant users.

- Database: admin_note column on shared.ideas (019 migration)
- Backend: PUT /admin/ideas/:id/note endpoint
- Frontend: AdminIdeasPage with table, filters, detail modal
- Sidebar: "Idea Submissions" nav link in admin sections
- Routing: /admin/ideas route under SuperAdminRoute guard

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-02 17:35:30 -04:00
JoeBot
140cd7acb7 feat: add ideation feature with per-tenant toggle
Adds idea submission capability gated by a per-tenant feature flag.
Super admins can enable/disable ideation for specific tenants via the
admin tenant detail drawer. Users see a lightbulb icon in the header
when enabled, opening a modal to submit ideas (title + description).
Ideas are stored in shared schema for cross-tenant backlog querying.

- Database: shared.ideas table (018-ideas.sql migration)
- Backend: Ideas NestJS module (entity, service, controller)
- Admin API: GET /admin/ideas, PUT /admin/ideas/:id/status,
  PUT /admin/organizations/:id/settings
- Frontend: IdeaModal component, lightbulb ActionIcon in header
- Admin UI: Feature Toggles card with ideation Switch in drawer

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-02 17:20:37 -04:00
06bc0181f8 feat: add k6 load testing suite, NRQL query library, and CLAUDE.md
Add comprehensive load testing infrastructure:
- k6 auth-dashboard flow (login → profile → dashboard KPIs → widgets → refresh → logout)
- k6 CRUD flow (units, vendors, journal entries, payments, reports)
- Environment configs with staging/production/local thresholds
- Parameterized user pool CSV matching app roles
- New Relic NRQL query library (25+ queries for perf analysis)
- Empty baseline.json structure for all tested endpoints
- CLAUDE.md documenting full stack, auth, route map, and conventions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 15:49:22 -04:00
37 changed files with 4429 additions and 8 deletions

229
CLAUDE.md Normal file
View File

@@ -0,0 +1,229 @@
# CLAUDE.md HOA Financial Platform (HOALedgerIQ)
## Project Overview
Multi-tenant SaaS platform for HOA (Homeowners Association) financial management. Handles chart of accounts, journal entries, budgets, invoices, payments, reserve planning, and board scenario planning.
---
## Stack & Framework
| Layer | Technology |
| --------- | --------------------------------------------------- |
| Backend | **NestJS 10** (TypeScript), runs on port 3000 |
| Frontend | **React 18** + Vite 5 + Mantine UI + Zustand |
| Database | **PostgreSQL** via **TypeORM 0.3** |
| Cache | **Redis** (BullMQ for queues) |
| Auth | **Passport.js** JWT access + httpOnly refresh |
| Payments | **Stripe** (checkout, subscriptions, webhooks) |
| Email | **Resend** |
| AI | NVIDIA API (Qwen model) for investment advisor |
| Monitoring| **New Relic** APM (app name: `HOALedgerIQ_App`) |
| Infra | Docker Compose (dev + prod), Nginx reverse proxy |
---
## Auth Pattern
- **Access token**: JWT, 1-hour TTL, payload `{ sub, email, orgId, role, isSuperadmin }`
- **Refresh token**: 64-byte random, SHA256-hashed in DB, 30-day TTL, sent as httpOnly cookie `ledgeriq_rt`
- **MFA**: TOTP via `otplib`, challenge token (5-min TTL), recovery codes
- **Passkeys**: WebAuthn via `@simplewebauthn/server`
- **SSO**: Google OAuth 2.0, Azure AD
- **Password hashing**: bcryptjs, cost 12
- **Rate limiting**: 100 req/min global (Throttler), custom per endpoint
### Guards & Middleware
- `TenantMiddleware` extracts `orgId` from JWT, sets tenant schema (60s cache)
- `JwtAuthGuard` Passport JWT guard on all protected routes
- `WriteAccessGuard` blocks write ops for `viewer` role and `past_due` orgs
- `@AllowViewer()` decorator exempts read endpoints from WriteAccessGuard
### Roles
`president`, `treasurer`, `secretary`, `member_at_large`, `manager`, `homeowner`, `admin`, `viewer`
---
## Multi-Tenant Architecture
- **Shared schema** (`shared`): users, organizations, user_organizations, refresh_tokens, invite_tokens, login_history, cd_rates
- **Tenant schemas** (dynamic, per org): accounts, journal_entries, budgets, invoices, payments, units, vendors, etc.
- Schema name stored in `shared.organizations.schema_name`
---
## Route Map (180+ endpoints)
### Auth (`/api/auth`)
| Method | Path | Purpose |
| ------ | ----------------------- | -------------------------------- |
| POST | /login | Email/password login |
| POST | /refresh | Refresh access token (cookie) |
| POST | /logout | Revoke refresh token |
| POST | /logout-everywhere | Revoke all sessions |
| GET | /profile | Current user profile |
| POST | /register | Register (disabled by default) |
| POST | /activate | Activate invited user |
| POST | /forgot-password | Request password reset |
| POST | /reset-password | Reset with token |
| PATCH | /change-password | Change password (authed) |
| POST | /switch-org | Switch active organization |
### Auth MFA (`/api/auth/mfa`)
| POST | /setup | POST /enable | POST /verify | POST /disable | GET /status |
### Auth Passkeys (`/api/auth/passkeys`)
| POST /register-options | POST /register | POST /login-options | POST /login | GET / | DELETE /:id |
### Admin (`/api/admin`) superadmin only
| GET /metrics | GET /users | GET /organizations | PUT /organizations/:id/subscription | POST /impersonate/:userId | POST /tenants |
### Organizations (`/api/organizations`)
| POST / | GET / | PATCH /settings | GET /members | POST /members | PUT /members/:id/role | DELETE /members/:id |
### Accounts (`/api/accounts`)
| GET / | GET /trial-balance | POST / | PUT /:id | PUT /:id/set-primary | POST /bulk-opening-balances | POST /:id/opening-balance | POST /:id/adjust-balance |
### Journal Entries (`/api/journal-entries`)
| GET / | GET /:id | POST / | POST /:id/post | POST /:id/void |
### Budgets (`/api/budgets`)
| GET /:year | PUT /:year | GET /:year/vs-actual | POST /:year/import | GET /:year/template |
### Invoices (`/api/invoices`)
| GET / | GET /:id | POST /generate-preview | POST /generate-bulk | POST /apply-late-fees |
### Payments (`/api/payments`)
| GET / | GET /:id | POST / | PUT /:id | DELETE /:id |
### Units (`/api/units`)
| GET / | GET /:id | POST / | PUT /:id | DELETE /:id | GET /export | POST /import |
### Vendors (`/api/vendors`)
| GET / | GET /:id | POST / | PUT /:id | GET /export | POST /import | GET /1099-data |
### Reports (`/api/reports`)
| GET /dashboard | GET /balance-sheet | GET /income-statement | GET /cash-flow | GET /cash-flow-sankey | GET /aging | GET /year-end | GET /cash-flow-forecast | GET /quarterly |
### Board Planning (`/api/board-planning`)
Scenarios CRUD, scenario investments, scenario assessments, projections, budget plans 28 endpoints total.
### Other Modules
- `/api/fiscal-periods` list, close, lock
- `/api/reserve-components` CRUD
- `/api/capital-projects` CRUD
- `/api/projects` CRUD + planning + import/export
- `/api/assessment-groups` CRUD + summary + default
- `/api/monthly-actuals` GET/POST /:year/:month
- `/api/health-scores` latest + calculate
- `/api/investment-planning` snapshot, market-rates, recommendations
- `/api/investment-accounts` CRUD
- `/api/attachments` upload, list, download, delete (10MB limit)
- `/api/onboarding` progress get/patch
- `/api/billing` trial, checkout, webhook, subscription, portal
---
## Database
- **Connection pool**: min 5, max 30, 30s idle, 5s connect timeout
- **Migrations**: SQL files in `db/migrations/` (manual execution, no ORM runner)
- **Init script**: `db/init/00-init.sql` (shared schema DDL)
---
## Key File Paths
| Purpose | Path |
| ---------------------- | ------------------------------------------------- |
| NestJS bootstrap | `backend/src/main.ts` |
| Root module | `backend/src/app.module.ts` |
| Auth controller | `backend/src/modules/auth/auth.controller.ts` |
| Auth service | `backend/src/modules/auth/auth.service.ts` |
| Refresh token svc | `backend/src/modules/auth/refresh-token.service.ts` |
| JWT strategy | `backend/src/modules/auth/strategies/jwt.strategy.ts` |
| Tenant middleware | `backend/src/database/tenant.middleware.ts` |
| Write-access guard | `backend/src/common/guards/write-access.guard.ts` |
| DB schema init | `db/init/00-init.sql` |
| Env example | `.env.example` |
| Docker compose (dev) | `docker-compose.yml` |
| Frontend entry | `frontend/src/main.tsx` |
| Frontend pages | `frontend/src/pages/` |
---
## Environment Variables (critical)
```
DATABASE_URL PostgreSQL connection string
REDIS_URL Redis connection
JWT_SECRET JWT signing key
INVITE_TOKEN_SECRET Invite token signing
STRIPE_SECRET_KEY Stripe API key
STRIPE_WEBHOOK_SECRET Stripe webhook verification
RESEND_API_KEY Email service
NEW_RELIC_APP_NAME "HOALedgerIQ_App"
NEW_RELIC_LICENSE_KEY New Relic license
APP_URL Base URL for email links
```
---
## New Relic
- **App name**: `HOALedgerIQ_App` (env: `NEW_RELIC_APP_NAME`)
- Enabled via `NEW_RELIC_ENABLED=true`
- NRQL query library: `load-tests/analysis/nrql-queries.sql`
---
## Load Testing
### Run k6 scenarios
```bash
# Auth + Dashboard flow (staging)
k6 run --env TARGET_ENV=staging load-tests/scenarios/auth-dashboard-flow.js
# CRUD flow (staging)
k6 run --env TARGET_ENV=staging load-tests/scenarios/crud-flow.js
# Local dev
k6 run --env TARGET_ENV=local load-tests/scenarios/auth-dashboard-flow.js
```
### Conventions
- Scenarios live in `load-tests/scenarios/`
- Config in `load-tests/config/environments.json` (staging/production/local thresholds)
- Test users parameterized from `load-tests/config/user-pool.csv`
- Baseline results stored in `load-tests/analysis/baseline.json`
- NRQL queries for New Relic in `load-tests/analysis/nrql-queries.sql`
- All k6 scripts use `SharedArray` for user pool, `http.batch()` for parallel requests
- Custom metrics: `*_duration` trends + `*_error_rate` rates per journey
- Thresholds: p95 latency + error rate per environment
### User Pool CSV Format
```
email,password,orgId,role
```
Roles match the app: `treasurer`, `admin`, `president`, `manager`, `member_at_large`, `viewer`, `homeowner`
---
## Fix Conventions
- Backend tests: `npm run test` (Jest, `*.spec.ts` co-located with source)
- E2E tests: `npm run test:e2e`
- Backend build: `npm run build` (NestJS CLI)
- Frontend dev: `npm run dev` (Vite, port 5173)
- Frontend build: `npm run build`
- Always run `npm run build` in `backend/` after changes to verify compilation
- TypeORM entities use decorators (`@Entity`, `@Column`, etc.)
- Multi-tenant: any new module touching tenant data must use `TenantService` to get the correct schema connection
- New endpoints need `@UseGuards(JwtAuthGuard)` and should respect `WriteAccessGuard`
- Use `@AllowViewer()` on read-only endpoints

View File

@@ -33,6 +33,8 @@ import { BoardPlanningModule } from './modules/board-planning/board-planning.mod
import { BillingModule } from './modules/billing/billing.module';
import { EmailModule } from './modules/email/email.module';
import { OnboardingModule } from './modules/onboarding/onboarding.module';
import { IdeasModule } from './modules/ideas/ideas.module';
import { ShadowAiModule } from './modules/shadow-ai/shadow-ai.module';
import { ScheduleModule } from '@nestjs/schedule';
@Module({
@@ -88,6 +90,8 @@ import { ScheduleModule } from '@nestjs/schedule';
BillingModule,
EmailModule,
OnboardingModule,
IdeasModule,
ShadowAiModule,
ScheduleModule.forRoot(),
],
controllers: [AppController],

View File

@@ -0,0 +1,101 @@
/**
* Shared utility for calling OpenAI-compatible chat completion APIs.
* Used by both production AI features and shadow AI benchmarking.
*/
export interface AICallerParams {
apiUrl: string;
apiKey: string;
model: string;
messages: Array<{ role: string; content: string }>;
temperature: number;
maxTokens: number;
timeoutMs?: number;
}
export interface AICallerResult {
content: string;
usage?: { prompt_tokens: number; completion_tokens: number; total_tokens: number };
responseTimeMs: number;
rawResponse: string;
}
export async function callOpenAICompatible(params: AICallerParams): Promise<AICallerResult> {
const { apiUrl, apiKey, model, messages, temperature, maxTokens, timeoutMs = 600000 } = params;
const requestBody = {
model,
messages,
temperature,
max_tokens: maxTokens,
};
const bodyString = JSON.stringify(requestBody);
const startTime = Date.now();
const { URL } = await import('url');
const https = await import('https');
const aiResult = await new Promise<{ status: number; body: string }>((resolve, reject) => {
const url = new URL(`${apiUrl}/chat/completions`);
const options = {
hostname: url.hostname,
port: url.port || 443,
path: url.pathname,
method: 'POST',
headers: {
'Authorization': `Bearer ${apiKey}`,
'Content-Type': 'application/json',
'Content-Length': Buffer.byteLength(bodyString, 'utf-8'),
},
timeout: timeoutMs,
};
const req = https.request(options, (res) => {
let data = '';
res.on('data', (chunk) => { data += chunk; });
res.on('end', () => {
resolve({ status: res.statusCode!, body: data });
});
});
req.on('error', (err) => reject(err));
req.on('timeout', () => {
req.destroy();
reject(new Error(`Request timed out after ${timeoutMs / 1000}s`));
});
req.write(bodyString);
req.end();
});
const responseTimeMs = Date.now() - startTime;
if (aiResult.status >= 400) {
throw new Error(`AI API returned ${aiResult.status}: ${aiResult.body}`);
}
const data = JSON.parse(aiResult.body);
const content = data.choices?.[0]?.message?.content || null;
if (!content) {
throw new Error('AI model returned empty content');
}
// Clean response: strip markdown fences and thinking blocks
let cleaned = content.trim();
if (cleaned.startsWith('```')) {
cleaned = cleaned.replace(/^```(?:json)?\s*\n?/, '').replace(/\n?```\s*$/, '');
}
cleaned = cleaned.replace(/<think>[\s\S]*?<\/think>\s*/g, '').trim();
const usage = data.usage || undefined;
return {
content: cleaned,
usage,
responseTimeMs,
rawResponse: content,
};
}

View File

@@ -5,6 +5,7 @@ import { AuthService } from './auth.service';
import { UsersService } from '../users/users.service';
import { OrganizationsService } from '../organizations/organizations.service';
import { AdminAnalyticsService } from './admin-analytics.service';
import { IdeasService } from '../ideas/ideas.service';
import * as bcrypt from 'bcryptjs';
@ApiTags('admin')
@@ -17,6 +18,7 @@ export class AdminController {
private usersService: UsersService,
private orgService: OrganizationsService,
private analyticsService: AdminAnalyticsService,
private ideasService: IdeasService,
) {}
private async requireSuperadmin(req: any) {
@@ -196,4 +198,45 @@ export class AdminController {
return { success: true, organization: org };
}
// ── Ideation ──
@Get('ideas')
async listAllIdeas(@Req() req: any) {
await this.requireSuperadmin(req);
return this.ideasService.findAll();
}
@Put('ideas/:id/status')
async updateIdeaStatus(
@Req() req: any,
@Param('id') id: string,
@Body() body: { status: string },
) {
await this.requireSuperadmin(req);
const idea = await this.ideasService.updateStatus(id, body.status);
return { success: true, idea };
}
@Put('ideas/:id/note')
async updateIdeaNote(
@Req() req: any,
@Param('id') id: string,
@Body() body: { adminNote: string },
) {
await this.requireSuperadmin(req);
const idea = await this.ideasService.updateNote(id, body.adminNote);
return { success: true, idea };
}
@Put('organizations/:id/settings')
async updateOrgSettings(
@Req() req: any,
@Param('id') id: string,
@Body() body: Record<string, any>,
) {
await this.requireSuperadmin(req);
const org = await this.orgService.updateSettings(id, body);
return { success: true, organization: org };
}
}

View File

@@ -17,11 +17,13 @@ import { JwtStrategy } from './strategies/jwt.strategy';
import { LocalStrategy } from './strategies/local.strategy';
import { UsersModule } from '../users/users.module';
import { OrganizationsModule } from '../organizations/organizations.module';
import { IdeasModule } from '../ideas/ideas.module';
@Module({
imports: [
UsersModule,
OrganizationsModule,
IdeasModule,
PassportModule,
JwtModule.registerAsync({
imports: [ConfigModule],

View File

@@ -6,5 +6,6 @@ import { HealthScoresScheduler } from './health-scores.scheduler';
@Module({
controllers: [HealthScoresController],
providers: [HealthScoresService, HealthScoresScheduler],
exports: [HealthScoresService],
})
export class HealthScoresModule {}

View File

@@ -180,7 +180,7 @@ export class HealthScoresService {
// ── Data Readiness Checks ──
private async checkDataReadiness(qr: any, scoreType: string): Promise<string[]> {
async checkDataReadiness(qr: any, scoreType: string): Promise<string[]> {
const missing: string[] = [];
if (scoreType === 'operating') {
@@ -249,7 +249,7 @@ export class HealthScoresService {
// ── Data Gathering ──
private async gatherOperatingData(qr: any) {
async gatherOperatingData(qr: any) {
const year = new Date().getFullYear();
const [accounts, budgets, assessments, cashFlow, recentTransactions, actualsMonths] = await Promise.all([
@@ -520,7 +520,7 @@ export class HealthScoresService {
};
}
private async gatherReserveData(qr: any) {
async gatherReserveData(qr: any) {
const year = new Date().getFullYear();
const currentMonth = new Date().getMonth(); // 0-indexed
const monthNames = ['jan','feb','mar','apr','may','jun','jul','aug','sep','oct','nov','dec_amt'];
@@ -787,7 +787,7 @@ export class HealthScoresService {
// ── AI Prompt Construction ──
private buildOperatingPrompt(data: any): Array<{ role: string; content: string }> {
buildOperatingPrompt(data: any): Array<{ role: string; content: string }> {
const today = new Date().toISOString().split('T')[0];
const systemPrompt = `You are an HOA financial health analyst. You evaluate the operating fund health of homeowners associations on a scale of 0-100.
@@ -927,7 +927,7 @@ Projected Year-End Cash: $${data.projectedYearEndCash.toFixed(0)}`;
];
}
private buildReservePrompt(data: any): Array<{ role: string; content: string }> {
buildReservePrompt(data: any): Array<{ role: string; content: string }> {
const today = new Date().toISOString().split('T')[0];
const systemPrompt = `You are an HOA reserve fund analyst. You evaluate reserve fund health on a scale of 0-100, assessing whether the HOA is adequately prepared for future capital expenditures.

View File

@@ -0,0 +1,12 @@
import { IsString, IsNotEmpty, IsOptional, MaxLength } from 'class-validator';
export class CreateIdeaDto {
@IsString()
@IsNotEmpty()
@MaxLength(255)
title: string;
@IsString()
@IsOptional()
description?: string;
}

View File

@@ -0,0 +1,49 @@
import {
Entity,
PrimaryGeneratedColumn,
Column,
CreateDateColumn,
UpdateDateColumn,
ManyToOne,
JoinColumn,
} from 'typeorm';
import { Organization } from '../../organizations/entities/organization.entity';
import { User } from '../../users/entities/user.entity';
@Entity({ schema: 'shared', name: 'ideas' })
export class Idea {
@PrimaryGeneratedColumn('uuid')
id: string;
@Column({ name: 'org_id' })
orgId: string;
@Column({ name: 'user_id' })
userId: string;
@Column({ length: 255 })
title: string;
@Column({ type: 'text', nullable: true })
description: string;
@Column({ length: 20, default: 'new' })
status: string;
@Column({ name: 'admin_note', type: 'text', nullable: true })
adminNote: string;
@CreateDateColumn({ name: 'created_at', type: 'timestamptz' })
createdAt: Date;
@UpdateDateColumn({ name: 'updated_at', type: 'timestamptz' })
updatedAt: Date;
@ManyToOne(() => Organization)
@JoinColumn({ name: 'org_id' })
organization: Organization;
@ManyToOne(() => User)
@JoinColumn({ name: 'user_id' })
user: User;
}

View File

@@ -0,0 +1,27 @@
import { Controller, Get, Post, Body, Req, UseGuards } from '@nestjs/common';
import { ApiTags, ApiBearerAuth } from '@nestjs/swagger';
import { JwtAuthGuard } from '../auth/guards/jwt-auth.guard';
import { IdeasService } from './ideas.service';
import { CreateIdeaDto } from './dto/create-idea.dto';
@ApiTags('ideas')
@Controller('ideas')
@ApiBearerAuth()
@UseGuards(JwtAuthGuard)
export class IdeasController {
constructor(private ideasService: IdeasService) {}
@Post()
async create(@Req() req: any, @Body() dto: CreateIdeaDto) {
const orgId = req.user.orgId;
const userId = req.user.userId || req.user.sub;
const idea = await this.ideasService.create(orgId, userId, dto);
return { success: true, idea };
}
@Get()
async findByOrg(@Req() req: any) {
const orgId = req.user.orgId;
return this.ideasService.findByOrg(orgId);
}
}

View File

@@ -0,0 +1,14 @@
import { Module } from '@nestjs/common';
import { TypeOrmModule } from '@nestjs/typeorm';
import { Idea } from './entities/idea.entity';
import { Organization } from '../organizations/entities/organization.entity';
import { IdeasController } from './ideas.controller';
import { IdeasService } from './ideas.service';
@Module({
imports: [TypeOrmModule.forFeature([Idea, Organization])],
controllers: [IdeasController],
providers: [IdeasService],
exports: [IdeasService],
})
export class IdeasModule {}

View File

@@ -0,0 +1,89 @@
import { Injectable, ForbiddenException, NotFoundException, BadRequestException } from '@nestjs/common';
import { InjectRepository } from '@nestjs/typeorm';
import { Repository } from 'typeorm';
import { Idea } from './entities/idea.entity';
import { Organization } from '../organizations/entities/organization.entity';
import { CreateIdeaDto } from './dto/create-idea.dto';
@Injectable()
export class IdeasService {
constructor(
@InjectRepository(Idea)
private ideasRepository: Repository<Idea>,
@InjectRepository(Organization)
private orgRepository: Repository<Organization>,
) {}
async create(orgId: string, userId: string, dto: CreateIdeaDto): Promise<Idea> {
const org = await this.orgRepository.findOne({ where: { id: orgId } });
if (!org) {
throw new NotFoundException('Organization not found');
}
if (org.settings?.ideationEnabled !== true) {
throw new ForbiddenException('Ideation is not enabled for this organization');
}
const idea = this.ideasRepository.create({
orgId,
userId,
title: dto.title,
description: dto.description,
});
return this.ideasRepository.save(idea);
}
async findByOrg(orgId: string): Promise<Idea[]> {
return this.ideasRepository.find({
where: { orgId },
order: { createdAt: 'DESC' },
});
}
async findAll(): Promise<any[]> {
return this.ideasRepository
.createQueryBuilder('idea')
.leftJoin('idea.organization', 'org')
.leftJoin('idea.user', 'user')
.select([
'idea.id AS id',
'idea.title AS title',
'idea.description AS description',
'idea.status AS status',
'idea.createdAt AS "createdAt"',
'idea.adminNote AS "adminNote"',
'org.id AS "orgId"',
'org.name AS "orgName"',
'user.id AS "userId"',
'user.email AS "userEmail"',
'user.firstName AS "userFirstName"',
'user.lastName AS "userLastName"',
])
.orderBy('idea.createdAt', 'DESC')
.getRawMany();
}
async updateStatus(id: string, status: string): Promise<Idea> {
const validStatuses = ['new', 'reviewed', 'accepted', 'rejected'];
if (!validStatuses.includes(status)) {
throw new BadRequestException(`Invalid status. Must be one of: ${validStatuses.join(', ')}`);
}
const idea = await this.ideasRepository.findOne({ where: { id } });
if (!idea) {
throw new NotFoundException('Idea not found');
}
idea.status = status;
return this.ideasRepository.save(idea);
}
async updateNote(id: string, adminNote: string): Promise<Idea> {
const idea = await this.ideasRepository.findOne({ where: { id } });
if (!idea) {
throw new NotFoundException('Idea not found');
}
idea.adminNote = adminNote;
return this.ideasRepository.save(idea);
}
}

View File

@@ -5,5 +5,6 @@ import { InvestmentPlanningService } from './investment-planning.service';
@Module({
controllers: [InvestmentPlanningController],
providers: [InvestmentPlanningService],
exports: [InvestmentPlanningService],
})
export class InvestmentPlanningModule {}

View File

@@ -877,7 +877,7 @@ export class InvestmentPlanningService {
// ── Private: AI Prompt Construction ──
private buildPromptMessages(
buildPromptMessages(
snapshot: any,
allRates: { cd: MarketRate[]; money_market: MarketRate[]; high_yield_savings: MarketRate[] },
monthlyForecast: any,
@@ -1059,6 +1059,285 @@ Based on this complete financial picture INCLUDING the 12-month cash flow foreca
];
}
// ── Schema-Based Prompt Building (for shadow AI benchmarking) ──
/**
* Build investment recommendation prompt messages for a specific tenant schema.
* Bypasses request-scoped TenantService by using DataSource directly.
*/
async buildPromptForSchema(schemaName: string): Promise<Array<{ role: string; content: string }>> {
const qr = this.dataSource.createQueryRunner();
try {
await qr.connect();
await qr.query(`SET search_path TO "${schemaName}"`);
const year = new Date().getFullYear();
const currentMonth = new Date().getMonth() + 1;
const monthNames = ['jan','feb','mar','apr','may','jun','jul','aug','sep','oct','nov','dec_amt'];
const monthLabels = ['Jan','Feb','Mar','Apr','May','Jun','Jul','Aug','Sep','Oct','Nov','Dec'];
// ── Gather financial snapshot data ──
const [accountBalances, investmentAccounts, budgets, projects] = await Promise.all([
qr.query(`
SELECT a.id, a.account_number, a.name, a.account_type, a.fund_type, a.interest_rate,
CASE
WHEN a.account_type IN ('asset', 'expense')
THEN COALESCE(SUM(jel.debit), 0) - COALESCE(SUM(jel.credit), 0)
ELSE COALESCE(SUM(jel.credit), 0) - COALESCE(SUM(jel.debit), 0)
END as balance
FROM accounts a
LEFT JOIN journal_entry_lines jel ON jel.account_id = a.id
LEFT JOIN journal_entries je ON je.id = jel.journal_entry_id
AND je.is_posted = true AND je.is_void = false
WHERE a.is_active = true AND a.account_type IN ('asset', 'liability', 'equity')
GROUP BY a.id, a.account_number, a.name, a.account_type, a.fund_type, a.interest_rate
ORDER BY a.account_number
`),
qr.query(`
SELECT id, name, institution, investment_type, fund_type,
principal, interest_rate, maturity_date, purchase_date, current_value
FROM investment_accounts WHERE is_active = true
ORDER BY maturity_date NULLS LAST
`),
qr.query(
`SELECT b.fund_type, a.account_type, a.name, a.account_number,
(b.jan + b.feb + b.mar + b.apr + b.may + b.jun +
b.jul + b.aug + b.sep + b.oct + b.nov + b.dec_amt) as annual_total
FROM budgets b JOIN accounts a ON a.id = b.account_id
WHERE b.fiscal_year = $1 ORDER BY a.account_type, a.account_number`,
[year],
),
qr.query(`
SELECT name, estimated_cost, target_year, target_month, fund_source,
status, priority, current_fund_balance, funded_percentage
FROM projects WHERE is_active = true AND status IN ('planned', 'approved', 'in_progress')
ORDER BY target_year, target_month NULLS LAST, priority
`),
]);
// Cash flow context
const [opCashResult, resCashResult, assessmentIncome] = await Promise.all([
qr.query(`
SELECT COALESCE(SUM(sub.bal), 0) as total FROM (
SELECT COALESCE(SUM(jel.debit), 0) - COALESCE(SUM(jel.credit), 0) as bal
FROM accounts a
JOIN journal_entry_lines jel ON jel.account_id = a.id
JOIN journal_entries je ON je.id = jel.journal_entry_id AND je.is_posted = true AND je.is_void = false
WHERE a.account_type = 'asset' AND a.fund_type = 'operating' AND a.is_active = true
GROUP BY a.id
) sub
`),
qr.query(`
SELECT COALESCE(SUM(sub.bal), 0) as total FROM (
SELECT COALESCE(SUM(jel.debit), 0) - COALESCE(SUM(jel.credit), 0) as bal
FROM accounts a
JOIN journal_entry_lines jel ON jel.account_id = a.id
JOIN journal_entries je ON je.id = jel.journal_entry_id AND je.is_posted = true AND je.is_void = false
WHERE a.account_type = 'asset' AND a.fund_type = 'reserve' AND a.is_active = true
GROUP BY a.id
) sub
`),
qr.query(`
SELECT COALESCE(SUM(ag.regular_assessment *
(SELECT COUNT(*) FROM units u WHERE u.assessment_group_id = ag.id AND u.status = 'active')), 0) as monthly_assessment_income
FROM assessment_groups ag WHERE ag.is_active = true
`),
]);
const operatingCash = accountBalances
.filter((a: any) => a.fund_type === 'operating' && a.account_type === 'asset')
.reduce((sum: number, a: any) => sum + parseFloat(a.balance || '0'), 0);
const reserveCash = accountBalances
.filter((a: any) => a.fund_type === 'reserve' && a.account_type === 'asset')
.reduce((sum: number, a: any) => sum + parseFloat(a.balance || '0'), 0);
const operatingInvestments = investmentAccounts
.filter((i: any) => i.fund_type === 'operating')
.reduce((sum: number, i: any) => sum + parseFloat(i.current_value || i.principal || '0'), 0);
const reserveInvestments = investmentAccounts
.filter((i: any) => i.fund_type === 'reserve')
.reduce((sum: number, i: any) => sum + parseFloat(i.current_value || i.principal || '0'), 0);
const snapshot = {
summary: {
operating_cash: operatingCash,
reserve_cash: reserveCash,
operating_investments: operatingInvestments,
reserve_investments: reserveInvestments,
total_operating: operatingCash + operatingInvestments,
total_reserve: reserveCash + reserveInvestments,
total_all: operatingCash + reserveCash + operatingInvestments + reserveInvestments,
},
account_balances: accountBalances,
investment_accounts: investmentAccounts,
budgets,
projects,
cash_flow_context: {
current_operating_cash: parseFloat(opCashResult[0]?.total || '0'),
current_reserve_cash: parseFloat(resCashResult[0]?.total || '0'),
budget_summary: await qr.query(
`SELECT b.fund_type, a.account_type,
SUM(b.jan + b.feb + b.mar + b.apr + b.may + b.jun +
b.jul + b.aug + b.sep + b.oct + b.nov + b.dec_amt) as annual_total
FROM budgets b JOIN accounts a ON a.id = b.account_id
WHERE b.fiscal_year = $1 GROUP BY b.fund_type, a.account_type`,
[year],
),
monthly_assessment_income: parseFloat(assessmentIncome[0]?.monthly_assessment_income || '0'),
},
};
// ── Build monthly forecast ──
const [opCashRows2, resCashRows2, opInvRows, resInvRows] = await Promise.all([
qr.query(`SELECT COALESCE(SUM(sub.bal), 0) as total FROM (
SELECT COALESCE(SUM(jel.debit), 0) - COALESCE(SUM(jel.credit), 0) as bal
FROM accounts a JOIN journal_entry_lines jel ON jel.account_id = a.id
JOIN journal_entries je ON je.id = jel.journal_entry_id AND je.is_posted = true AND je.is_void = false
WHERE a.account_type = 'asset' AND a.fund_type = 'operating' AND a.is_active = true GROUP BY a.id
) sub`),
qr.query(`SELECT COALESCE(SUM(sub.bal), 0) as total FROM (
SELECT COALESCE(SUM(jel.debit), 0) - COALESCE(SUM(jel.credit), 0) as bal
FROM accounts a JOIN journal_entry_lines jel ON jel.account_id = a.id
JOIN journal_entries je ON je.id = jel.journal_entry_id AND je.is_posted = true AND je.is_void = false
WHERE a.account_type = 'asset' AND a.fund_type = 'reserve' AND a.is_active = true GROUP BY a.id
) sub`),
qr.query(`SELECT COALESCE(SUM(current_value), 0) as total FROM investment_accounts WHERE fund_type = 'operating' AND is_active = true`),
qr.query(`SELECT COALESCE(SUM(current_value), 0) as total FROM investment_accounts WHERE fund_type = 'reserve' AND is_active = true`),
]);
let runOpCash = parseFloat(opCashRows2[0]?.total || '0');
let runResCash = parseFloat(resCashRows2[0]?.total || '0');
let runOpInv = parseFloat(opInvRows[0]?.total || '0');
let runResInv = parseFloat(resInvRows[0]?.total || '0');
const assessmentGroups = await qr.query(`
SELECT ag.frequency, ag.regular_assessment, ag.special_assessment,
(SELECT COUNT(*) FROM units u WHERE u.assessment_group_id = ag.id AND u.status = 'active') as unit_count
FROM assessment_groups ag WHERE ag.is_active = true
`);
const getAssessmentIncome = (month: number): { operating: number; reserve: number } => {
let operating = 0, reserve = 0;
for (const g of assessmentGroups) {
const units = parseInt(g.unit_count) || 0;
const regular = parseFloat(g.regular_assessment) || 0;
const special = parseFloat(g.special_assessment) || 0;
const freq = g.frequency || 'monthly';
let applies = false;
if (freq === 'monthly') applies = true;
else if (freq === 'quarterly') applies = [1,4,7,10].includes(month);
else if (freq === 'annual') applies = month === 1;
if (applies) { operating += regular * units; reserve += special * units; }
}
return { operating, reserve };
};
const budgetsByYearMonth: Record<string, { opIncome: number; opExpense: number; resIncome: number; resExpense: number }> = {};
for (const yr of [year, year + 1]) {
const budgetRows = await qr.query(
`SELECT b.fund_type, a.account_type,
b.jan, b.feb, b.mar, b.apr, b.may, b.jun, b.jul, b.aug, b.sep, b.oct, b.nov, b.dec_amt
FROM budgets b JOIN accounts a ON a.id = b.account_id WHERE b.fiscal_year = $1`, [yr],
);
for (let m = 0; m < 12; m++) {
const key = `${yr}-${m + 1}`;
if (!budgetsByYearMonth[key]) budgetsByYearMonth[key] = { opIncome: 0, opExpense: 0, resIncome: 0, resExpense: 0 };
for (const row of budgetRows) {
const amt = parseFloat(row[monthNames[m]]) || 0;
if (amt === 0) continue;
const isOp = row.fund_type === 'operating';
if (row.account_type === 'income') { if (isOp) budgetsByYearMonth[key].opIncome += amt; else budgetsByYearMonth[key].resIncome += amt; }
else if (row.account_type === 'expense') { if (isOp) budgetsByYearMonth[key].opExpense += amt; else budgetsByYearMonth[key].resExpense += amt; }
}
}
}
const maturities = await qr.query(`
SELECT fund_type, current_value, maturity_date, interest_rate, purchase_date
FROM investment_accounts WHERE is_active = true AND maturity_date IS NOT NULL AND maturity_date > CURRENT_DATE
`);
const maturityIndex: Record<string, { operating: number; reserve: number }> = {};
for (const inv of maturities) {
const d = new Date(inv.maturity_date);
const key = `${d.getFullYear()}-${d.getMonth() + 1}`;
if (!maturityIndex[key]) maturityIndex[key] = { operating: 0, reserve: 0 };
const val = parseFloat(inv.current_value) || 0;
const rate = parseFloat(inv.interest_rate) || 0;
const purchaseDate = inv.purchase_date ? new Date(inv.purchase_date) : new Date();
const matDate = new Date(inv.maturity_date);
const daysHeld = Math.max((matDate.getTime() - purchaseDate.getTime()) / 86400000, 1);
const interestEarned = val * (rate / 100) * (daysHeld / 365);
const maturityTotal = val + interestEarned;
if (inv.fund_type === 'operating') maturityIndex[key].operating += maturityTotal;
else maturityIndex[key].reserve += maturityTotal;
}
const projectExpenses = await qr.query(`
SELECT estimated_cost, target_year, target_month, fund_source
FROM projects WHERE is_active = true AND status IN ('planned', 'in_progress')
AND target_year IS NOT NULL AND estimated_cost > 0
`);
const projectIndex: Record<string, { operating: number; reserve: number }> = {};
for (const p of projectExpenses) {
const yr2 = parseInt(p.target_year);
const mo = parseInt(p.target_month) || 6;
const key = `${yr2}-${mo}`;
if (!projectIndex[key]) projectIndex[key] = { operating: 0, reserve: 0 };
const cost = parseFloat(p.estimated_cost) || 0;
if (p.fund_source === 'operating') projectIndex[key].operating += cost;
else projectIndex[key].reserve += cost;
}
const datapoints: any[] = [];
for (let i = 0; i < 12; i++) {
const fYear = year + Math.floor((currentMonth - 1 + i) / 12);
const fMonth = ((currentMonth - 1 + i) % 12) + 1;
const key = `${fYear}-${fMonth}`;
const label = `${monthLabels[fMonth - 1]} ${fYear}`;
const assessments = getAssessmentIncome(fMonth);
const budget = budgetsByYearMonth[key] || { opIncome: 0, opExpense: 0, resIncome: 0, resExpense: 0 };
const maturity = maturityIndex[key] || { operating: 0, reserve: 0 };
const project = projectIndex[key] || { operating: 0, reserve: 0 };
const opIncomeMonth = budget.opIncome > 0 ? budget.opIncome : assessments.operating;
const resIncomeMonth = budget.resIncome > 0 ? budget.resIncome : assessments.reserve;
runOpCash += opIncomeMonth - budget.opExpense - project.operating + maturity.operating;
runResCash += resIncomeMonth - budget.resExpense - project.reserve + maturity.reserve;
if (maturity.operating > 0) runOpInv = Math.max(0, runOpInv - (maturity.operating * 0.96));
if (maturity.reserve > 0) runResInv = Math.max(0, runResInv - (maturity.reserve * 0.96));
datapoints.push({
month: label,
operating_cash: Math.round(runOpCash * 100) / 100,
operating_investments: Math.round(runOpInv * 100) / 100,
reserve_cash: Math.round(runResCash * 100) / 100,
reserve_investments: Math.round(runResInv * 100) / 100,
op_income: Math.round(opIncomeMonth * 100) / 100,
op_expense: Math.round(budget.opExpense * 100) / 100,
res_income: Math.round(resIncomeMonth * 100) / 100,
res_expense: Math.round(budget.resExpense * 100) / 100,
project_cost_op: Math.round(project.operating * 100) / 100,
project_cost_res: Math.round(project.reserve * 100) / 100,
maturity_op: Math.round(maturity.operating * 100) / 100,
maturity_res: Math.round(maturity.reserve * 100) / 100,
});
}
const assessmentSchedule = assessmentGroups.map((g: any) => ({
frequency: g.frequency || 'monthly',
regular_per_unit: parseFloat(g.regular_assessment) || 0,
special_per_unit: parseFloat(g.special_assessment) || 0,
units: parseInt(g.unit_count) || 0,
total_regular: (parseFloat(g.regular_assessment) || 0) * (parseInt(g.unit_count) || 0),
total_special: (parseFloat(g.special_assessment) || 0) * (parseInt(g.unit_count) || 0),
}));
const monthlyForecast = { datapoints, assessment_schedule: assessmentSchedule };
const allRates = await this.getMarketRates();
return this.buildPromptMessages(snapshot, allRates, monthlyForecast);
} finally {
await qr.release();
}
}
// ── Private: AI API Call ──
private async callAI(messages: Array<{ role: string; content: string }>): Promise<AIResponse> {

View File

@@ -0,0 +1,37 @@
import {
Entity,
PrimaryGeneratedColumn,
Column,
CreateDateColumn,
UpdateDateColumn,
} from 'typeorm';
@Entity({ schema: 'shared', name: 'shadow_ai_models' })
export class ShadowAiModel {
@PrimaryGeneratedColumn('uuid')
id: string;
@Column({ type: 'varchar', length: 10, unique: true })
slot: string;
@Column({ type: 'varchar', length: 100 })
name: string;
@Column({ name: 'api_url', type: 'varchar', length: 500 })
apiUrl: string;
@Column({ name: 'api_key', type: 'varchar', length: 500 })
apiKey: string;
@Column({ name: 'model_name', type: 'varchar', length: 200 })
modelName: string;
@Column({ name: 'is_active', type: 'boolean', default: true })
isActive: boolean;
@CreateDateColumn({ name: 'created_at', type: 'timestamptz' })
createdAt: Date;
@UpdateDateColumn({ name: 'updated_at', type: 'timestamptz' })
updatedAt: Date;
}

View File

@@ -0,0 +1,52 @@
import {
Entity,
PrimaryGeneratedColumn,
Column,
CreateDateColumn,
ManyToOne,
JoinColumn,
} from 'typeorm';
import { ShadowRun } from './shadow-run.entity';
@Entity({ schema: 'shared', name: 'shadow_run_results' })
export class ShadowRunResult {
@PrimaryGeneratedColumn('uuid')
id: string;
@Column({ name: 'run_id', type: 'uuid' })
runId: string;
@Column({ name: 'model_role', type: 'varchar', length: 20 })
modelRole: string;
@Column({ name: 'model_name', type: 'varchar', length: 200 })
modelName: string;
@Column({ name: 'api_url', type: 'varchar', length: 500 })
apiUrl: string;
@Column({ name: 'raw_response', type: 'text', nullable: true })
rawResponse: string;
@Column({ name: 'parsed_response', type: 'jsonb', nullable: true })
parsedResponse: any;
@Column({ name: 'response_time_ms', type: 'integer', nullable: true })
responseTimeMs: number;
@Column({ name: 'token_usage', type: 'jsonb', nullable: true })
tokenUsage: any;
@Column({ type: 'varchar', length: 20, default: 'pending' })
status: string;
@Column({ name: 'error_message', type: 'text', nullable: true })
errorMessage: string;
@CreateDateColumn({ name: 'created_at', type: 'timestamptz' })
createdAt: Date;
@ManyToOne(() => ShadowRun, (run) => run.results, { onDelete: 'CASCADE' })
@JoinColumn({ name: 'run_id' })
run: ShadowRun;
}

View File

@@ -0,0 +1,44 @@
import {
Entity,
PrimaryGeneratedColumn,
Column,
CreateDateColumn,
OneToMany,
} from 'typeorm';
import { ShadowRunResult } from './shadow-run-result.entity';
@Entity({ schema: 'shared', name: 'shadow_runs' })
export class ShadowRun {
@PrimaryGeneratedColumn('uuid')
id: string;
@Column({ name: 'tenant_id', type: 'uuid' })
tenantId: string;
@Column({ type: 'varchar', length: 30 })
feature: string;
@Column({ type: 'varchar', length: 20, default: 'running' })
status: string;
@Column({ name: 'triggered_by', type: 'uuid', nullable: true })
triggeredBy: string;
@Column({ name: 'prompt_messages', type: 'jsonb' })
promptMessages: any;
@Column({ name: 'started_at', type: 'timestamptz', default: () => 'NOW()' })
startedAt: Date;
@Column({ name: 'completed_at', type: 'timestamptz', nullable: true })
completedAt: Date;
@CreateDateColumn({ name: 'created_at', type: 'timestamptz' })
createdAt: Date;
@OneToMany(() => ShadowRunResult, (result) => result.run, { eager: true })
results: ShadowRunResult[];
// Virtual field populated via JOIN
tenantName?: string;
}

View File

@@ -0,0 +1,118 @@
import {
Controller,
Get,
Put,
Post,
Delete,
Body,
Param,
Query,
UseGuards,
Req,
ForbiddenException,
BadRequestException,
NotFoundException,
} from '@nestjs/common';
import { ApiTags, ApiBearerAuth } from '@nestjs/swagger';
import { JwtAuthGuard } from '../auth/guards/jwt-auth.guard';
import { UsersService } from '../users/users.service';
import { ShadowAiService } from './shadow-ai.service';
@ApiTags('admin/shadow-ai')
@Controller('admin/shadow-ai')
@ApiBearerAuth()
@UseGuards(JwtAuthGuard)
export class ShadowAiController {
constructor(
private shadowAiService: ShadowAiService,
private usersService: UsersService,
) {}
private async requireSuperadmin(req: any) {
const user = await this.usersService.findById(req.user.userId || req.user.sub);
if (!user?.isSuperadmin) {
throw new ForbiddenException('Superadmin access required');
}
return user;
}
// ── Model Configuration ──
@Get('models')
async getModels(@Req() req: any) {
await this.requireSuperadmin(req);
return this.shadowAiService.getModels();
}
@Put('models/:slot')
async upsertModel(
@Req() req: any,
@Param('slot') slot: string,
@Body() body: { name: string; apiUrl: string; apiKey: string; modelName: string; isActive?: boolean },
) {
await this.requireSuperadmin(req);
if (!['A', 'B'].includes(slot)) {
throw new BadRequestException('Slot must be A or B');
}
if (!body.name || !body.apiUrl || !body.apiKey || !body.modelName) {
throw new BadRequestException('name, apiUrl, apiKey, and modelName are required');
}
return this.shadowAiService.upsertModel(slot, body);
}
@Delete('models/:slot')
async deleteModel(@Req() req: any, @Param('slot') slot: string) {
await this.requireSuperadmin(req);
if (!['A', 'B'].includes(slot)) {
throw new BadRequestException('Slot must be A or B');
}
return this.shadowAiService.deleteModel(slot);
}
// ── Shadow Runs ──
@Post('runs')
async triggerRun(
@Req() req: any,
@Body() body: { tenantId: string; feature: string },
) {
const user = await this.requireSuperadmin(req);
const validFeatures = ['operating_health', 'reserve_health', 'investment_recommendations'];
if (!validFeatures.includes(body.feature)) {
throw new BadRequestException(`Feature must be one of: ${validFeatures.join(', ')}`);
}
if (!body.tenantId) {
throw new BadRequestException('tenantId is required');
}
return this.shadowAiService.triggerRun(
body.tenantId,
body.feature as any,
user.id,
);
}
@Get('runs')
async getRunHistory(
@Req() req: any,
@Query('page') page?: string,
@Query('limit') limit?: string,
@Query('tenantId') tenantId?: string,
@Query('feature') feature?: string,
) {
await this.requireSuperadmin(req);
return this.shadowAiService.getRunHistory({
page: page ? parseInt(page) : undefined,
limit: limit ? parseInt(limit) : undefined,
tenantId,
feature,
});
}
@Get('runs/:id')
async getRunDetail(@Req() req: any, @Param('id') id: string) {
await this.requireSuperadmin(req);
const detail = await this.shadowAiService.getRunDetail(id);
if (!detail) throw new NotFoundException('Shadow run not found');
return detail;
}
}

View File

@@ -0,0 +1,26 @@
import { Module, OnModuleInit } from '@nestjs/common';
import { TypeOrmModule } from '@nestjs/typeorm';
import { ShadowAiController } from './shadow-ai.controller';
import { ShadowAiService } from './shadow-ai.service';
import { ShadowAiModel } from './entities/shadow-ai-model.entity';
import { ShadowRun } from './entities/shadow-run.entity';
import { ShadowRunResult } from './entities/shadow-run-result.entity';
import { HealthScoresModule } from '../health-scores/health-scores.module';
import { UsersModule } from '../users/users.module';
@Module({
imports: [
TypeOrmModule.forFeature([ShadowAiModel, ShadowRun, ShadowRunResult]),
HealthScoresModule,
UsersModule,
],
controllers: [ShadowAiController],
providers: [ShadowAiService],
})
export class ShadowAiModule implements OnModuleInit {
constructor(private shadowAiService: ShadowAiService) {}
async onModuleInit() {
await this.shadowAiService.ensureTables();
}
}

View File

@@ -0,0 +1,723 @@
import { Injectable, Logger } from '@nestjs/common';
import { ConfigService } from '@nestjs/config';
import { DataSource } from 'typeorm';
import { HealthScoresService } from '../health-scores/health-scores.service';
import { callOpenAICompatible } from '../../common/utils/ai-caller';
type Feature = 'operating_health' | 'reserve_health' | 'investment_recommendations';
interface ModelConfig {
role: string;
name: string;
apiUrl: string;
apiKey: string;
modelName: string;
}
@Injectable()
export class ShadowAiService {
private readonly logger = new Logger(ShadowAiService.name);
constructor(
private dataSource: DataSource,
private configService: ConfigService,
private healthScoresService: HealthScoresService,
) {}
// ── Model Configuration CRUD ──
async getModels() {
const rows = await this.dataSource.query(
`SELECT id, slot, name, api_url, api_key, model_name, is_active, created_at, updated_at
FROM shared.shadow_ai_models ORDER BY slot`,
);
return rows.map((r: any) => ({
...r,
api_key: r.api_key ? `****${r.api_key.slice(-4)}` : null,
}));
}
async upsertModel(slot: string, dto: { name: string; apiUrl: string; apiKey: string; modelName: string; isActive?: boolean }) {
const isActive = dto.isActive !== undefined ? dto.isActive : true;
// Check if model exists for this slot
const existing = await this.dataSource.query(
`SELECT id, api_key FROM shared.shadow_ai_models WHERE slot = $1`,
[slot],
);
if (existing.length > 0) {
// If apiKey is masked (starts with ****), keep the existing key
const apiKey = dto.apiKey.startsWith('****') ? existing[0].api_key : dto.apiKey;
await this.dataSource.query(
`UPDATE shared.shadow_ai_models
SET name = $1, api_url = $2, api_key = $3, model_name = $4, is_active = $5, updated_at = NOW()
WHERE slot = $6`,
[dto.name, dto.apiUrl, apiKey, dto.modelName, isActive, slot],
);
} else {
await this.dataSource.query(
`INSERT INTO shared.shadow_ai_models (slot, name, api_url, api_key, model_name, is_active)
VALUES ($1, $2, $3, $4, $5, $6)`,
[slot, dto.name, dto.apiUrl, dto.apiKey, dto.modelName, isActive],
);
}
return { slot, status: 'saved' };
}
async deleteModel(slot: string) {
await this.dataSource.query(
`DELETE FROM shared.shadow_ai_models WHERE slot = $1`,
[slot],
);
return { slot, status: 'deleted' };
}
// ── Shadow Run Execution ──
async triggerRun(tenantId: string, feature: Feature, userId: string) {
// Look up tenant schema
const orgs = await this.dataSource.query(
`SELECT schema_name, name FROM shared.organizations WHERE id = $1`,
[tenantId],
);
if (!orgs.length) throw new Error('Tenant not found');
const schemaName = orgs[0].schema_name;
// Build prompt messages for the feature
const messages = await this.buildPromptMessages(schemaName, feature);
// Create shadow run record
const runRows = await this.dataSource.query(
`INSERT INTO shared.shadow_runs (tenant_id, feature, status, triggered_by, prompt_messages, started_at)
VALUES ($1, $2, 'running', $3, $4, NOW())
RETURNING id`,
[tenantId, feature, userId, JSON.stringify(messages)],
);
const runId = runRows[0].id;
// Get model configs
const modelConfigs = await this.getModelConfigs();
// Create pending result rows
for (const config of modelConfigs) {
await this.dataSource.query(
`INSERT INTO shared.shadow_run_results (run_id, model_role, model_name, api_url, status)
VALUES ($1, $2, $3, $4, 'pending')`,
[runId, config.role, config.modelName, config.apiUrl],
);
}
// Fire-and-forget: run all models in parallel
this.executeModels(runId, messages, modelConfigs, feature).catch((err) => {
this.logger.error(`Shadow run ${runId} failed: ${err.message}`);
});
return { runId, status: 'running' };
}
// ── Run History ──
async getRunHistory(query: { page?: number; limit?: number; tenantId?: string; feature?: string }) {
const page = query.page || 1;
const limit = Math.min(query.limit || 20, 100);
const offset = (page - 1) * limit;
let where = '';
const params: any[] = [];
let paramIdx = 1;
if (query.tenantId) {
where += ` AND sr.tenant_id = $${paramIdx++}`;
params.push(query.tenantId);
}
if (query.feature) {
where += ` AND sr.feature = $${paramIdx++}`;
params.push(query.feature);
}
const [rows, countRows] = await Promise.all([
this.dataSource.query(
`SELECT sr.id, sr.tenant_id, sr.feature, sr.status, sr.started_at, sr.completed_at, sr.created_at,
o.name as tenant_name,
(SELECT COUNT(*) FROM shared.shadow_run_results rr WHERE rr.run_id = sr.id) as result_count,
(SELECT COUNT(*) FROM shared.shadow_run_results rr WHERE rr.run_id = sr.id AND rr.status = 'success') as success_count
FROM shared.shadow_runs sr
LEFT JOIN shared.organizations o ON o.id = sr.tenant_id
WHERE 1=1 ${where}
ORDER BY sr.created_at DESC
LIMIT $${paramIdx++} OFFSET $${paramIdx++}`,
[...params, limit, offset],
),
this.dataSource.query(
`SELECT COUNT(*) as total FROM shared.shadow_runs sr WHERE 1=1 ${where}`,
params,
),
]);
return {
runs: rows,
total: parseInt(countRows[0]?.total || '0'),
page,
limit,
};
}
async getRunDetail(runId: string) {
const [runs, results] = await Promise.all([
this.dataSource.query(
`SELECT sr.*, o.name as tenant_name
FROM shared.shadow_runs sr
LEFT JOIN shared.organizations o ON o.id = sr.tenant_id
WHERE sr.id = $1`,
[runId],
),
this.dataSource.query(
`SELECT * FROM shared.shadow_run_results
WHERE run_id = $1
ORDER BY CASE model_role
WHEN 'production' THEN 1
WHEN 'alternate_a' THEN 2
WHEN 'alternate_b' THEN 3
END`,
[runId],
),
]);
if (!runs.length) return null;
return {
...runs[0],
results,
};
}
// ── Private Helpers ──
private async buildPromptMessages(
schemaName: string,
feature: Feature,
): Promise<Array<{ role: string; content: string }>> {
if (feature === 'operating_health' || feature === 'reserve_health') {
const qr = this.dataSource.createQueryRunner();
try {
await qr.connect();
await qr.query(`SET search_path TO "${schemaName}"`);
const scoreType = feature === 'operating_health' ? 'operating' : 'reserve';
const data = scoreType === 'operating'
? await this.healthScoresService.gatherOperatingData(qr)
: await this.healthScoresService.gatherReserveData(qr);
return scoreType === 'operating'
? this.healthScoresService.buildOperatingPrompt(data)
: this.healthScoresService.buildReservePrompt(data);
} finally {
await qr.release();
}
}
// investment_recommendations — build prompt directly via DataSource
return this.buildInvestmentPromptForSchema(schemaName);
}
/**
* Build investment recommendation prompts for a given tenant schema.
* Self-contained: uses DataSource directly, no request-scoped dependencies.
*/
private async buildInvestmentPromptForSchema(schemaName: string): Promise<Array<{ role: string; content: string }>> {
const qr = this.dataSource.createQueryRunner();
try {
await qr.connect();
await qr.query(`SET search_path TO "${schemaName}"`);
const year = new Date().getFullYear();
const currentMonth = new Date().getMonth() + 1;
const monthNames = ['jan','feb','mar','apr','may','jun','jul','aug','sep','oct','nov','dec_amt'];
const monthLabels = ['Jan','Feb','Mar','Apr','May','Jun','Jul','Aug','Sep','Oct','Nov','Dec'];
// ── Financial snapshot ──
const [accountBalances, investmentAccounts, budgets, projects] = await Promise.all([
qr.query(`
SELECT a.id, a.account_number, a.name, a.account_type, a.fund_type, a.interest_rate,
CASE WHEN a.account_type IN ('asset', 'expense')
THEN COALESCE(SUM(jel.debit), 0) - COALESCE(SUM(jel.credit), 0)
ELSE COALESCE(SUM(jel.credit), 0) - COALESCE(SUM(jel.debit), 0)
END as balance
FROM accounts a
LEFT JOIN journal_entry_lines jel ON jel.account_id = a.id
LEFT JOIN journal_entries je ON je.id = jel.journal_entry_id AND je.is_posted = true AND je.is_void = false
WHERE a.is_active = true AND a.account_type IN ('asset', 'liability', 'equity')
GROUP BY a.id, a.account_number, a.name, a.account_type, a.fund_type, a.interest_rate ORDER BY a.account_number
`),
qr.query(`SELECT id, name, institution, investment_type, fund_type, principal, interest_rate, maturity_date, purchase_date, current_value
FROM investment_accounts WHERE is_active = true ORDER BY maturity_date NULLS LAST`),
qr.query(`SELECT b.fund_type, a.account_type, a.name, a.account_number,
(b.jan+b.feb+b.mar+b.apr+b.may+b.jun+b.jul+b.aug+b.sep+b.oct+b.nov+b.dec_amt) as annual_total
FROM budgets b JOIN accounts a ON a.id = b.account_id WHERE b.fiscal_year = $1 ORDER BY a.account_type, a.account_number`, [year]),
qr.query(`SELECT name, estimated_cost, target_year, target_month, fund_source, status, priority, current_fund_balance, funded_percentage
FROM projects WHERE is_active = true AND status IN ('planned','approved','in_progress') ORDER BY target_year, target_month NULLS LAST, priority`),
]);
const [opCashResult, resCashResult, budgetSummary, assessmentIncome] = await Promise.all([
qr.query(`SELECT COALESCE(SUM(sub.bal),0) as total FROM (SELECT COALESCE(SUM(jel.debit),0)-COALESCE(SUM(jel.credit),0) as bal FROM accounts a JOIN journal_entry_lines jel ON jel.account_id=a.id JOIN journal_entries je ON je.id=jel.journal_entry_id AND je.is_posted=true AND je.is_void=false WHERE a.account_type='asset' AND a.fund_type='operating' AND a.is_active=true GROUP BY a.id) sub`),
qr.query(`SELECT COALESCE(SUM(sub.bal),0) as total FROM (SELECT COALESCE(SUM(jel.debit),0)-COALESCE(SUM(jel.credit),0) as bal FROM accounts a JOIN journal_entry_lines jel ON jel.account_id=a.id JOIN journal_entries je ON je.id=jel.journal_entry_id AND je.is_posted=true AND je.is_void=false WHERE a.account_type='asset' AND a.fund_type='reserve' AND a.is_active=true GROUP BY a.id) sub`),
qr.query(`SELECT b.fund_type, a.account_type, SUM(b.jan+b.feb+b.mar+b.apr+b.may+b.jun+b.jul+b.aug+b.sep+b.oct+b.nov+b.dec_amt) as annual_total FROM budgets b JOIN accounts a ON a.id=b.account_id WHERE b.fiscal_year=$1 GROUP BY b.fund_type, a.account_type`, [year]),
qr.query(`SELECT COALESCE(SUM(ag.regular_assessment*(SELECT COUNT(*) FROM units u WHERE u.assessment_group_id=ag.id AND u.status='active')),0) as monthly_assessment_income FROM assessment_groups ag WHERE ag.is_active=true`),
]);
const operatingCash = accountBalances.filter((a: any) => a.fund_type === 'operating' && a.account_type === 'asset').reduce((s: number, a: any) => s + parseFloat(a.balance || '0'), 0);
const reserveCash = accountBalances.filter((a: any) => a.fund_type === 'reserve' && a.account_type === 'asset').reduce((s: number, a: any) => s + parseFloat(a.balance || '0'), 0);
const operatingInvestments = investmentAccounts.filter((i: any) => i.fund_type === 'operating').reduce((s: number, i: any) => s + parseFloat(i.current_value || i.principal || '0'), 0);
const reserveInvestments = investmentAccounts.filter((i: any) => i.fund_type === 'reserve').reduce((s: number, i: any) => s + parseFloat(i.current_value || i.principal || '0'), 0);
const snapshot = {
summary: { operating_cash: operatingCash, reserve_cash: reserveCash, operating_investments: operatingInvestments, reserve_investments: reserveInvestments,
total_operating: operatingCash + operatingInvestments, total_reserve: reserveCash + reserveInvestments, total_all: operatingCash + reserveCash + operatingInvestments + reserveInvestments },
account_balances: accountBalances, investment_accounts: investmentAccounts, budgets, projects,
cash_flow_context: {
current_operating_cash: parseFloat(opCashResult[0]?.total || '0'), current_reserve_cash: parseFloat(resCashResult[0]?.total || '0'),
budget_summary: budgetSummary, monthly_assessment_income: parseFloat(assessmentIncome[0]?.monthly_assessment_income || '0'),
},
};
// ── 12-month forecast ──
const [opInvRows, resInvRows] = await Promise.all([
qr.query(`SELECT COALESCE(SUM(current_value),0) as total FROM investment_accounts WHERE fund_type='operating' AND is_active=true`),
qr.query(`SELECT COALESCE(SUM(current_value),0) as total FROM investment_accounts WHERE fund_type='reserve' AND is_active=true`),
]);
let runOpCash = parseFloat(opCashResult[0]?.total || '0'), runResCash = parseFloat(resCashResult[0]?.total || '0');
let runOpInv = parseFloat(opInvRows[0]?.total || '0'), runResInv = parseFloat(resInvRows[0]?.total || '0');
const assessmentGroups = await qr.query(`SELECT ag.frequency, ag.regular_assessment, ag.special_assessment,
(SELECT COUNT(*) FROM units u WHERE u.assessment_group_id=ag.id AND u.status='active') as unit_count FROM assessment_groups ag WHERE ag.is_active=true`);
const getAssessmentInc = (month: number) => {
let op = 0, res = 0;
for (const g of assessmentGroups) {
const units = parseInt(g.unit_count) || 0, reg = parseFloat(g.regular_assessment) || 0, spec = parseFloat(g.special_assessment) || 0;
const freq = g.frequency || 'monthly';
let applies = freq === 'monthly' || (freq === 'quarterly' && [1,4,7,10].includes(month)) || (freq === 'annual' && month === 1);
if (applies) { op += reg * units; res += spec * units; }
}
return { operating: op, reserve: res };
};
const budgetsByYM: Record<string, { opIncome: number; opExpense: number; resIncome: number; resExpense: number }> = {};
for (const yr of [year, year + 1]) {
const bRows = await qr.query(`SELECT b.fund_type, a.account_type, b.jan,b.feb,b.mar,b.apr,b.may,b.jun,b.jul,b.aug,b.sep,b.oct,b.nov,b.dec_amt FROM budgets b JOIN accounts a ON a.id=b.account_id WHERE b.fiscal_year=$1`, [yr]);
for (let m = 0; m < 12; m++) {
const k = `${yr}-${m+1}`;
if (!budgetsByYM[k]) budgetsByYM[k] = { opIncome: 0, opExpense: 0, resIncome: 0, resExpense: 0 };
for (const r of bRows) {
const amt = parseFloat(r[monthNames[m]]) || 0;
if (!amt) continue;
const isOp = r.fund_type === 'operating';
if (r.account_type === 'income') { if (isOp) budgetsByYM[k].opIncome += amt; else budgetsByYM[k].resIncome += amt; }
else if (r.account_type === 'expense') { if (isOp) budgetsByYM[k].opExpense += amt; else budgetsByYM[k].resExpense += amt; }
}
}
}
const maturities = await qr.query(`SELECT fund_type, current_value, maturity_date, interest_rate, purchase_date FROM investment_accounts WHERE is_active=true AND maturity_date IS NOT NULL AND maturity_date>CURRENT_DATE`);
const matIdx: Record<string, { operating: number; reserve: number }> = {};
for (const inv of maturities) {
const d = new Date(inv.maturity_date), k = `${d.getFullYear()}-${d.getMonth()+1}`;
if (!matIdx[k]) matIdx[k] = { operating: 0, reserve: 0 };
const val = parseFloat(inv.current_value) || 0, rate = parseFloat(inv.interest_rate) || 0;
const pDate = inv.purchase_date ? new Date(inv.purchase_date) : new Date();
const days = Math.max((d.getTime() - pDate.getTime()) / 86400000, 1);
const total = val + val * (rate/100) * (days/365);
if (inv.fund_type === 'operating') matIdx[k].operating += total; else matIdx[k].reserve += total;
}
const projExp = await qr.query(`SELECT estimated_cost, target_year, target_month, fund_source FROM projects WHERE is_active=true AND status IN ('planned','in_progress') AND target_year IS NOT NULL AND estimated_cost>0`);
const projIdx: Record<string, { operating: number; reserve: number }> = {};
for (const p of projExp) {
const k = `${parseInt(p.target_year)}-${parseInt(p.target_month)||6}`;
if (!projIdx[k]) projIdx[k] = { operating: 0, reserve: 0 };
const c = parseFloat(p.estimated_cost) || 0;
if (p.fund_source === 'operating') projIdx[k].operating += c; else projIdx[k].reserve += c;
}
const datapoints: any[] = [];
for (let i = 0; i < 12; i++) {
const fY = year + Math.floor((currentMonth-1+i)/12), fM = ((currentMonth-1+i)%12)+1;
const k = `${fY}-${fM}`, label = `${monthLabels[fM-1]} ${fY}`;
const asmt = getAssessmentInc(fM), bud = budgetsByYM[k] || { opIncome: 0, opExpense: 0, resIncome: 0, resExpense: 0 };
const mat = matIdx[k] || { operating: 0, reserve: 0 }, proj = projIdx[k] || { operating: 0, reserve: 0 };
const opInc = bud.opIncome > 0 ? bud.opIncome : asmt.operating, resInc = bud.resIncome > 0 ? bud.resIncome : asmt.reserve;
runOpCash += opInc - bud.opExpense - proj.operating + mat.operating;
runResCash += resInc - bud.resExpense - proj.reserve + mat.reserve;
if (mat.operating > 0) runOpInv = Math.max(0, runOpInv - mat.operating * 0.96);
if (mat.reserve > 0) runResInv = Math.max(0, runResInv - mat.reserve * 0.96);
datapoints.push({ month: label, operating_cash: Math.round(runOpCash*100)/100, operating_investments: Math.round(runOpInv*100)/100,
reserve_cash: Math.round(runResCash*100)/100, reserve_investments: Math.round(runResInv*100)/100,
op_income: Math.round(opInc*100)/100, op_expense: Math.round(bud.opExpense*100)/100,
res_income: Math.round(resInc*100)/100, res_expense: Math.round(bud.resExpense*100)/100,
project_cost_op: Math.round(proj.operating*100)/100, project_cost_res: Math.round(proj.reserve*100)/100,
maturity_op: Math.round(mat.operating*100)/100, maturity_res: Math.round(mat.reserve*100)/100 });
}
const asmtSchedule = assessmentGroups.map((g: any) => ({
frequency: g.frequency || 'monthly', regular_per_unit: parseFloat(g.regular_assessment) || 0,
special_per_unit: parseFloat(g.special_assessment) || 0, units: parseInt(g.unit_count) || 0,
total_regular: (parseFloat(g.regular_assessment) || 0) * (parseInt(g.unit_count) || 0),
total_special: (parseFloat(g.special_assessment) || 0) * (parseInt(g.unit_count) || 0),
}));
// ── Market rates from shared schema ──
const fetchLatest = async (rateType: string) =>
qr.query(`SELECT bank_name, apy, min_deposit, term, term_months, rate_type, fetched_at
FROM shared.cd_rates WHERE rate_type=$1 AND fetched_at=(SELECT MAX(fetched_at) FROM shared.cd_rates WHERE rate_type=$1)
ORDER BY apy DESC LIMIT 25`, [rateType]);
const [cdRates, mmRates, hysRates] = await Promise.all([fetchLatest('cd'), fetchLatest('money_market'), fetchLatest('high_yield_savings')]);
const allRates = { cd: cdRates, money_market: mmRates, high_yield_savings: hysRates };
// ── Build prompt (replicates InvestmentPlanningService.buildPromptMessages) ──
const { summary, investment_accounts: invAccts, cash_flow_context: cfc } = snapshot;
const today = new Date().toISOString().split('T')[0];
const systemPrompt = `You are a financial advisor specializing in HOA (Homeowners Association) reserve fund management and conservative investment strategy. You provide fiduciary-grade investment recommendations.
CRITICAL RULES:
1. HOAs are legally required to maintain adequate reserves. NEVER recommend depleting reserve funds below safe levels.
2. HOA investments must be conservative ONLY: CDs, money market accounts, treasury bills, and high-yield savings. NO stocks, bonds, mutual funds, or speculative instruments.
3. Liquidity is paramount: always ensure enough cash to cover at least 3 months of operating expenses AND any capital project expenses due within the next 12 months.
4. CD laddering is the preferred strategy for reserve funds — it balances yield with regular liquidity access.
5. Operating funds should remain highly liquid (money market or high-yield savings only).
6. Respect the separation between operating funds and reserve funds. Never suggest commingling.
7. Base your recommendations ONLY on the available market rates (CDs, Money Market, High Yield Savings) provided. Do not reference rates or banks not in the provided data.
8. CRITICAL: Use the 12-MONTH CASH FLOW FORECAST to understand future liquidity. The forecast includes projected income (regular assessments AND special assessments collected from homeowners), budgeted expenses, investment maturities, and capital project costs. Do NOT flag liquidity shortfalls if the forecast shows sufficient income arriving before the expense is due.
9. When recommending money market or high yield savings accounts, focus on their liquidity advantages for operating funds. When recommending CDs, focus on their higher yields for longer-term reserve fund placement.
10. Compare current account rates against available market rates. If better rates are available, suggest specific moves with the potential additional interest income that could be earned.
RESPONSE FORMAT:
Respond with ONLY valid JSON (no markdown, no code fences) matching this exact schema:
{
"recommendations": [
{
"type": "cd_ladder" | "new_investment" | "reallocation" | "maturity_action" | "liquidity_warning" | "general",
"priority": "high" | "medium" | "low",
"title": "Short action title (under 60 chars)",
"summary": "One sentence summary of the recommendation",
"details": "Detailed explanation with specific dollar amounts and timeframes",
"fund_type": "operating" | "reserve" | "both",
"suggested_amount": 50000.00,
"suggested_term": "12 months",
"suggested_rate": 4.50,
"bank_name": "Bank name from market rates (if applicable)",
"rationale": "Financial reasoning for why this makes sense",
"components": [
{
"label": "Component label (e.g. '6-Month CD at Marcus')",
"amount": 6600.00,
"term_months": 6,
"rate": 4.05,
"bank_name": "Marcus",
"investment_type": "cd"
}
]
}
],
"overall_assessment": "2-3 sentence overview of the HOA's current investment position and opportunities",
"risk_notes": ["Array of risk items or concerns to flag for the board"]
}
IMPORTANT ABOUT COMPONENTS:
- For cd_ladder recommendations, you MUST include a "components" array with each individual CD as a separate component. Each component should have its own label, amount, term_months, rate, and bank_name. The suggested_amount should be the total of all component amounts.
- For other multi-part strategies (e.g. splitting funds across multiple accounts), also include a "components" array.
- For simple single-investment recommendations, omit the "components" field entirely.
IMPORTANT: Provide 3-7 actionable recommendations. Prioritize high-priority items (liquidity risks, maturing investments) before optimization opportunities. Include specific dollar amounts wherever possible. When there are opportunities for better rates on existing positions, quantify the additional annual interest that could be earned.`;
const investmentsList = invAccts.length === 0 ? 'No current investments.'
: invAccts.map((i: any) => `- ${i.name} | Type: ${i.investment_type} | Fund: ${i.fund_type} | Principal: $${parseFloat(i.principal).toFixed(2)} | Rate: ${parseFloat(i.interest_rate||'0').toFixed(2)}% | Maturity: ${i.maturity_date ? new Date(i.maturity_date).toLocaleDateString() : 'N/A'}`).join('\n');
const budgetLines = budgets.length === 0 ? 'No budget data available.'
: budgets.map((b: any) => `- ${b.name} (${b.account_number}) | ${b.account_type}/${b.fund_type}: $${parseFloat(b.annual_total).toFixed(2)}/yr`).join('\n');
const projectLines = projects.length === 0 ? 'No upcoming capital projects.'
: projects.map((p: any) => `- ${p.name} | Cost: $${parseFloat(p.estimated_cost).toFixed(2)} | Target: ${p.target_year||'?'}/${p.target_month||'?'} | Fund: ${p.fund_source} | Status: ${p.status} | Funded: ${parseFloat(p.funded_percentage||'0').toFixed(1)}%`).join('\n');
const budgetSummaryLines = (cfc.budget_summary || []).length === 0 ? 'No budget summary available.'
: cfc.budget_summary.map((b: any) => `- ${b.fund_type} ${b.account_type}: $${parseFloat(b.annual_total).toFixed(2)}/yr (~$${(parseFloat(b.annual_total)/12).toFixed(2)}/mo)`).join('\n');
const formatRates = (rates: any[], label: string) => rates.length === 0
? `No ${label} rate data available. Rate fetcher may not have been run yet.`
: rates.map((r: any) => `- ${r.bank_name} | APY: ${parseFloat(String(r.apy)).toFixed(2)}%${r.term !== 'N/A' ? ` | Term: ${r.term}` : ''} | Min Deposit: ${r.min_deposit ? '$'+parseFloat(String(r.min_deposit)).toLocaleString() : 'N/A'}`).join('\n');
const asmtLines = asmtSchedule.length === 0 ? 'No assessment schedule available.'
: asmtSchedule.map((a: any) => `- ${a.frequency} collection | ${a.units} units | Regular: $${a.regular_per_unit.toFixed(2)}/unit ($${a.total_regular.toFixed(2)} total) → Operating | Special: $${a.special_per_unit.toFixed(2)}/unit ($${a.total_special.toFixed(2)} total) → Reserve`).join('\n');
const forecastLines = datapoints.map((dp: any) => {
const d: string[] = [];
if (dp.op_income > 0) d.push(`OpInc:$${dp.op_income.toFixed(0)}`);
if (dp.op_expense > 0) d.push(`OpExp:$${dp.op_expense.toFixed(0)}`);
if (dp.res_income > 0) d.push(`ResInc:$${dp.res_income.toFixed(0)}`);
if (dp.res_expense > 0) d.push(`ResExp:$${dp.res_expense.toFixed(0)}`);
if (dp.project_cost_res > 0) d.push(`ResProjCost:$${dp.project_cost_res.toFixed(0)}`);
if (dp.project_cost_op > 0) d.push(`OpProjCost:$${dp.project_cost_op.toFixed(0)}`);
if (dp.maturity_op > 0) d.push(`OpMaturity:$${dp.maturity_op.toFixed(0)}`);
if (dp.maturity_res > 0) d.push(`ResMaturity:$${dp.maturity_res.toFixed(0)}`);
return `- ${dp.month} | OpCash: $${dp.operating_cash.toFixed(0)} | ResCash: $${dp.reserve_cash.toFixed(0)} | OpInv: $${dp.operating_investments.toFixed(0)} | ResInv: $${dp.reserve_investments.toFixed(0)} | Drivers: ${d.join(', ') || 'none'}`;
}).join('\n');
const userPrompt = `Analyze this HOA's financial position and provide investment recommendations.
TODAY'S DATE: ${today}
=== CURRENT CASH POSITIONS ===
Operating Cash (bank accounts): $${summary.operating_cash.toFixed(2)}
Reserve Cash (bank accounts): $${summary.reserve_cash.toFixed(2)}
Operating Investments: $${summary.operating_investments.toFixed(2)}
Reserve Investments: $${summary.reserve_investments.toFixed(2)}
Total Operating Fund: $${summary.total_operating.toFixed(2)}
Total Reserve Fund: $${summary.total_reserve.toFixed(2)}
Grand Total: $${summary.total_all.toFixed(2)}
=== CURRENT INVESTMENTS ===
${investmentsList}
=== ASSESSMENT INCOME SCHEDULE ===
${asmtLines}
Note: "Regular" assessments fund Operating. "Special" assessments fund Reserve. Both are collected from homeowners per the frequency above.
=== ANNUAL BUDGET (${new Date().getFullYear()}) ===
${budgetLines}
=== BUDGET SUMMARY (Annual Totals by Category) ===
${budgetSummaryLines}
=== MONTHLY ASSESSMENT INCOME ===
Recurring monthly regular assessment income: $${cfc.monthly_assessment_income.toFixed(2)}/month (operating fund)
=== UPCOMING CAPITAL PROJECTS ===
${projectLines}
=== 12-MONTH CASH FLOW FORECAST (Projected) ===
This forecast shows month-by-month projected balances factoring in ALL income (regular assessments, special assessments, budgeted income), ALL expenses (budgeted expenses, capital project costs), and investment maturities.
${forecastLines}
=== AVAILABLE MARKET RATES ===
--- CD Rates ---
${formatRates(allRates.cd, 'CD')}
--- Money Market Rates ---
${formatRates(allRates.money_market, 'Money Market')}
--- High Yield Savings Rates ---
${formatRates(allRates.high_yield_savings, 'High Yield Savings')}
Based on this complete financial picture INCLUDING the 12-month cash flow forecast, provide your investment recommendations. Consider:
1. Is there excess cash that could earn better returns in CDs, money market accounts, or high-yield savings?
2. Are any current investments maturing soon that need reinvestment planning?
3. Is the liquidity position adequate for upcoming expenses and projects? USE THE FORECAST to check — if income (including special assessments) arrives before expenses are due, the position may be adequate even if current cash seems low.
4. Would a CD ladder strategy improve the yield while maintaining access to funds?
5. Are operating and reserve funds properly separated in the investment strategy?
6. Could any current money market or savings accounts earn better rates at a different bank? Quantify the potential additional annual interest.
7. For operating funds that need to stay liquid, are money market or high-yield savings accounts being used optimally?`;
return [
{ role: 'system', content: systemPrompt },
{ role: 'user', content: userPrompt },
];
} finally {
await qr.release();
}
}
private async getModelConfigs(): Promise<ModelConfig[]> {
const configs: ModelConfig[] = [];
// Production model from env vars
const prodApiUrl = this.configService.get<string>('AI_API_URL') || 'https://integrate.api.nvidia.com/v1';
const prodApiKey = this.configService.get<string>('AI_API_KEY');
const prodModel = this.configService.get<string>('AI_MODEL') || 'qwen/qwen3.5-397b-a17b';
if (prodApiKey) {
configs.push({
role: 'production',
name: 'Production',
apiUrl: prodApiUrl,
apiKey: prodApiKey,
modelName: prodModel,
});
}
// Alternate models from DB
const alternates = await this.dataSource.query(
`SELECT slot, name, api_url, api_key, model_name
FROM shared.shadow_ai_models
WHERE is_active = true
ORDER BY slot`,
);
for (const alt of alternates) {
configs.push({
role: alt.slot === 'A' ? 'alternate_a' : 'alternate_b',
name: alt.name,
apiUrl: alt.api_url,
apiKey: alt.api_key,
modelName: alt.model_name,
});
}
return configs;
}
private getFeatureParams(feature: Feature): { temperature: number; maxTokens: number } {
if (feature === 'investment_recommendations') {
return { temperature: 0.3, maxTokens: 4096 };
}
return { temperature: 0.1, maxTokens: 2048 };
}
private async executeModels(
runId: string,
messages: Array<{ role: string; content: string }>,
configs: ModelConfig[],
feature: Feature,
) {
const { temperature, maxTokens } = this.getFeatureParams(feature);
const promises = configs.map(async (config) => {
// Mark as running
await this.dataSource.query(
`UPDATE shared.shadow_run_results SET status = 'running' WHERE run_id = $1 AND model_role = $2`,
[runId, config.role],
);
try {
const result = await callOpenAICompatible({
apiUrl: config.apiUrl,
apiKey: config.apiKey,
model: config.modelName,
messages,
temperature,
maxTokens,
});
// Try to parse the response as JSON
let parsedResponse: any = null;
try {
parsedResponse = JSON.parse(result.content);
} catch {
// Store raw content if not valid JSON
parsedResponse = { raw_text: result.content };
}
await this.dataSource.query(
`UPDATE shared.shadow_run_results
SET status = 'success', raw_response = $1, parsed_response = $2,
response_time_ms = $3, token_usage = $4
WHERE run_id = $5 AND model_role = $6`,
[
result.rawResponse,
JSON.stringify(parsedResponse),
result.responseTimeMs,
result.usage ? JSON.stringify(result.usage) : null,
runId,
config.role,
],
);
this.logger.log(`Shadow run ${runId} - ${config.role} (${config.modelName}) completed in ${result.responseTimeMs}ms`);
} catch (error: any) {
this.logger.error(`Shadow run ${runId} - ${config.role} (${config.modelName}) failed: ${error.message}`);
await this.dataSource.query(
`UPDATE shared.shadow_run_results
SET status = 'error', error_message = $1
WHERE run_id = $2 AND model_role = $3`,
[error.message, runId, config.role],
);
}
});
await Promise.allSettled(promises);
// Determine overall run status
const results = await this.dataSource.query(
`SELECT status FROM shared.shadow_run_results WHERE run_id = $1`,
[runId],
);
const allSuccess = results.every((r: any) => r.status === 'success');
const allError = results.every((r: any) => r.status === 'error');
const status = allSuccess ? 'completed' : allError ? 'failed' : 'partial';
await this.dataSource.query(
`UPDATE shared.shadow_runs SET status = $1, completed_at = NOW() WHERE id = $2`,
[status, runId],
);
this.logger.log(`Shadow run ${runId} finished with status: ${status}`);
}
// ── Table Creation (for initial setup) ──
async ensureTables() {
const qr = this.dataSource.createQueryRunner();
try {
await qr.connect();
await qr.query(`
CREATE TABLE IF NOT EXISTS shared.shadow_ai_models (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
slot VARCHAR(10) NOT NULL UNIQUE CHECK (slot IN ('A', 'B')),
name VARCHAR(100) NOT NULL,
api_url VARCHAR(500) NOT NULL,
api_key VARCHAR(500) NOT NULL,
model_name VARCHAR(200) NOT NULL,
is_active BOOLEAN DEFAULT TRUE,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
)
`);
await qr.query(`
CREATE TABLE IF NOT EXISTS shared.shadow_runs (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
tenant_id UUID NOT NULL,
feature VARCHAR(30) NOT NULL CHECK (feature IN ('operating_health', 'reserve_health', 'investment_recommendations')),
status VARCHAR(20) NOT NULL DEFAULT 'running' CHECK (status IN ('running', 'completed', 'partial', 'failed')),
triggered_by UUID,
prompt_messages JSONB NOT NULL,
started_at TIMESTAMPTZ DEFAULT NOW(),
completed_at TIMESTAMPTZ,
created_at TIMESTAMPTZ DEFAULT NOW()
)
`);
await qr.query(`
CREATE INDEX IF NOT EXISTS idx_shadow_runs_tenant ON shared.shadow_runs(tenant_id)
`);
await qr.query(`
CREATE INDEX IF NOT EXISTS idx_shadow_runs_created ON shared.shadow_runs(created_at DESC)
`);
await qr.query(`
CREATE TABLE IF NOT EXISTS shared.shadow_run_results (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
run_id UUID NOT NULL REFERENCES shared.shadow_runs(id) ON DELETE CASCADE,
model_role VARCHAR(20) NOT NULL CHECK (model_role IN ('production', 'alternate_a', 'alternate_b')),
model_name VARCHAR(200) NOT NULL,
api_url VARCHAR(500) NOT NULL,
raw_response TEXT,
parsed_response JSONB,
response_time_ms INTEGER,
token_usage JSONB,
status VARCHAR(20) NOT NULL DEFAULT 'pending' CHECK (status IN ('pending', 'running', 'success', 'error')),
error_message TEXT,
created_at TIMESTAMPTZ DEFAULT NOW(),
UNIQUE(run_id, model_role)
)
`);
await qr.query(`
CREATE INDEX IF NOT EXISTS idx_shadow_results_run ON shared.shadow_run_results(run_id)
`);
this.logger.log('Shadow AI tables ensured');
} finally {
await qr.release();
}
}
}

View File

@@ -0,0 +1,15 @@
-- Ideation feature: shared ideas table for cross-tenant idea submissions
CREATE TABLE IF NOT EXISTS shared.ideas (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
org_id UUID NOT NULL REFERENCES shared.organizations(id) ON DELETE CASCADE,
user_id UUID NOT NULL REFERENCES shared.users(id) ON DELETE CASCADE,
title VARCHAR(255) NOT NULL,
description TEXT,
status VARCHAR(20) NOT NULL DEFAULT 'new',
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_ideas_org_id ON shared.ideas(org_id);
CREATE INDEX IF NOT EXISTS idx_ideas_status ON shared.ideas(status);
CREATE INDEX IF NOT EXISTS idx_ideas_created_at ON shared.ideas(created_at DESC);

View File

@@ -0,0 +1,2 @@
-- Add private admin note column to ideas table
ALTER TABLE shared.ideas ADD COLUMN IF NOT EXISTS admin_note TEXT;

View File

@@ -0,0 +1,275 @@
# Shadow AI Benchmarking Feature
## Context
The platform uses a single AI model (Qwen 3.5 via NVIDIA NIM) for three features: Operating Health Score, Reserve Health Score, and Investment Recommendations. The platform owner needs a way to evaluate alternate models (different providers, different versions) against the production model using real tenant data — without impacting users. This enables informed model migration decisions by comparing outputs side-by-side.
## Architecture Overview
- **New admin page** at `/admin/shadow-ai` with model configuration, run trigger, and history
- **New backend module** `shadow-ai` with controller, service, and 3 entities
- **3 new DB tables** in the `shared` schema for model configs, runs, and results
- **Shared AI caller utility** to avoid duplicating HTTP logic
- **Minimal changes** to existing services: make prompt-building methods public and export modules
---
## Phase 1: Shared AI Caller Utility
### New file: `backend/src/common/utils/ai-caller.ts`
Extract the HTTP POST logic (currently duplicated in both `callAI()` methods) into a reusable function:
```typescript
export async function callOpenAICompatible(params: {
apiUrl: string;
apiKey: string;
model: string;
messages: Array<{ role: string; content: string }>;
temperature: number;
maxTokens: number;
timeoutMs?: number; // default 600000
}): Promise<{
content: string; // cleaned JSON string (fences + <think> stripped)
usage?: { prompt_tokens: number; completion_tokens: number; total_tokens: number };
responseTimeMs: number;
}>
```
Handles: HTTPS POST to `{apiUrl}/chat/completions`, timeout, markdown fence stripping, `<think>` block removal, timing.
## Phase 2: Expose Existing Prompt Builders
### `backend/src/modules/health-scores/health-scores.service.ts`
- Change `private``public` on:
- `gatherOperatingData(qr)` (line 252)
- `gatherReserveData(qr)` (line 523)
- `buildOperatingPrompt(data)` (line 790)
- `buildReservePrompt(data)` (line 930)
- `checkDataReadiness(qr, scoreType)` (used to validate data exists)
### `backend/src/modules/health-scores/health-scores.module.ts`
- Add `exports: [HealthScoresService]`
### `backend/src/modules/investment-planning/investment-planning.service.ts`
- Add new public method `buildPromptForSchema(schemaName: string)` that:
1. Creates a query runner, sets `search_path` to the tenant schema
2. Runs the same data-gathering queries (financial snapshot, market rates, monthly forecast) using the query runner directly (bypassing request-scoped `TenantService`)
3. Calls the existing `buildPromptMessages()` with gathered data
4. Returns `Array<{ role: string; content: string }>`
- Change `buildPromptMessages()` from `private``public` (line 880)
### `backend/src/modules/investment-planning/investment-planning.module.ts`
- Add `exports: [InvestmentPlanningService]`
## Phase 3: Database Tables & Entities
### 3 new tables in `shared` schema
**`shared.shadow_ai_models`** — Alternate model configurations (slots A and B)
| Column | Type | Notes |
|--------|------|-------|
| id | UUID PK | |
| slot | VARCHAR(10) | CHECK IN ('A', 'B'), UNIQUE |
| name | VARCHAR(100) | Display label |
| api_url | VARCHAR(500) | OpenAI-compatible endpoint |
| api_key | VARCHAR(500) | Bearer token |
| model_name | VARCHAR(200) | Model identifier |
| is_active | BOOLEAN | Default true |
| created_at | TIMESTAMPTZ | |
| updated_at | TIMESTAMPTZ | |
**`shared.shadow_runs`** — One row per comparison execution
| Column | Type | Notes |
|--------|------|-------|
| id | UUID PK | |
| tenant_id | UUID FK | → shared.organizations |
| feature | VARCHAR(30) | CHECK IN ('operating_health', 'reserve_health', 'investment_recommendations') |
| status | VARCHAR(20) | CHECK IN ('running', 'completed', 'partial', 'failed') |
| triggered_by | UUID FK | → shared.users |
| prompt_messages | JSONB | Exact messages sent to all models (proof of identical input) |
| started_at | TIMESTAMPTZ | |
| completed_at | TIMESTAMPTZ | |
| created_at | TIMESTAMPTZ | |
**`shared.shadow_run_results`** — One row per model per run (up to 3 per run)
| Column | Type | Notes |
|--------|------|-------|
| id | UUID PK | |
| run_id | UUID FK | → shadow_runs ON DELETE CASCADE |
| model_role | VARCHAR(20) | CHECK IN ('production', 'alternate_a', 'alternate_b'), UNIQUE(run_id, model_role) |
| model_name | VARCHAR(200) | Snapshot of model used |
| api_url | VARCHAR(500) | Snapshot of endpoint used |
| raw_response | TEXT | Unprocessed AI response |
| parsed_response | JSONB | Validated structured output |
| response_time_ms | INTEGER | |
| token_usage | JSONB | { prompt_tokens, completion_tokens, total_tokens } |
| status | VARCHAR(20) | CHECK IN ('pending', 'running', 'success', 'error') |
| error_message | TEXT | |
| created_at | TIMESTAMPTZ | |
### Entity files
- `backend/src/modules/shadow-ai/entities/shadow-ai-model.entity.ts`
- `backend/src/modules/shadow-ai/entities/shadow-run.entity.ts`
- `backend/src/modules/shadow-ai/entities/shadow-run-result.entity.ts`
All use `@Entity({ schema: 'shared', name: '...' })` pattern.
## Phase 4: Shadow AI Backend Module
### New directory: `backend/src/modules/shadow-ai/`
### `shadow-ai.service.ts`
**Model CRUD:**
- `getModels()` — Return both slots, mask API keys (show last 4 chars)
- `upsertModel(slot, dto)` — INSERT/UPDATE config for slot A or B
- `deleteModel(slot)` — Remove model config
**Run Execution:**
- `triggerRun(tenantId, feature, userId)`:
1. Look up tenant `schema_name` from `shared.organizations`
2. Build prompt messages by calling the appropriate exposed method:
- `operating_health`: Create query runner → set search_path → `healthScoresService.gatherOperatingData(qr)``healthScoresService.buildOperatingPrompt(data)`
- `reserve_health`: Same pattern with reserve methods
- `investment_recommendations`: `investmentPlanningService.buildPromptForSchema(schemaName)`
3. Insert `shadow_runs` row with `prompt_messages` stored as JSONB
4. Get production config from env vars, alternate configs from DB
5. Insert 1-3 `shadow_run_results` rows as 'pending' (production + active alternates)
6. Return `{ runId }` immediately
7. Fire-and-forget: call all models in parallel using `callOpenAICompatible()`
- Per feature: operating/reserve use temp 0.1, max_tokens 2048; investment uses temp 0.3, max_tokens 4096
8. Update each result row as it completes (success/error, parsed response, timing)
9. Update run status when all complete
**History:**
- `getRunHistory(page, limit, tenantFilter?, featureFilter?)` — Paginated list with tenant name JOIN
- `getRunDetail(runId)` — Full run + all results
### `shadow-ai.controller.ts`
All endpoints use `@UseGuards(JwtAuthGuard)` + `requireSuperadmin(req)` pattern from `admin.controller.ts`.
| Method | Path | Body/Params |
|--------|------|-------------|
| GET | `/admin/shadow-ai/models` | — |
| PUT | `/admin/shadow-ai/models/:slot` | `{ name, apiUrl, apiKey, modelName, isActive }` |
| DELETE | `/admin/shadow-ai/models/:slot` | — |
| POST | `/admin/shadow-ai/runs` | `{ tenantId, feature }` |
| GET | `/admin/shadow-ai/runs` | `?page&limit&tenantId&feature` |
| GET | `/admin/shadow-ai/runs/:id` | — |
### `shadow-ai.module.ts`
```typescript
@Module({
imports: [
TypeOrmModule.forFeature([ShadowAiModel, ShadowRun, ShadowRunResult]),
HealthScoresModule,
InvestmentPlanningModule,
UsersModule,
],
controllers: [ShadowAiController],
providers: [ShadowAiService],
})
```
### Register in `backend/src/app.module.ts`
- Add `import { ShadowAiModule }` and include in the `imports` array
## Phase 5: Frontend — Admin Shadow AI Page
### New file: `frontend/src/pages/admin/AdminShadowAiPage.tsx`
**Layout**: Mantine `Tabs` with 3 tabs
#### Tab 1: "Model Configuration"
- Three `Card` components in a `SimpleGrid cols={3}`:
- **Production** (read-only): Shows model name, API URL from a dedicated endpoint or hardcoded label "From environment config"
- **Alternate A**: Form with `TextInput` (name, API URL, model name), `PasswordInput` (API key), `Switch` (active), Save/Delete buttons
- **Alternate B**: Same form
- Fetches via `GET /api/admin/shadow-ai/models`
- Saves via `PUT /api/admin/shadow-ai/models/A` or `/B`
#### Tab 2: "Run Comparison"
- `Select` dropdown for tenant (reuse `GET /api/admin/organizations` already used by AdminPage)
- `Select` for feature type (Operating Health / Reserve Health / Investment Recommendations)
- `Button` "Run Shadow Comparison"
- On trigger: `POST /api/admin/shadow-ai/runs` → get `runId`
- Poll `GET /api/admin/shadow-ai/runs/:id` every 3s via `refetchInterval` until status !== 'running'
- Show per-model progress indicators during run
- Once complete, render results using shared comparison component (below)
#### Tab 3: "History"
- `Table` with columns: Date, Tenant, Feature, Status (Badge), Duration
- Filter controls: tenant Select, feature Select
- Click row → expand detail or modal showing full comparison
- Pagination
#### Shared Component: Side-by-Side Results Display
- `SimpleGrid cols={3}` (or fewer columns if only some models were configured)
- Each column:
- Header: model name + response time `Badge`
- **For health scores**: Score with `RingProgress`, label `Badge`, summary text, factors list (color-coded by impact), recommendations list (color-coded by priority)
- **For investment**: Overall assessment text, recommendation cards with type/priority badges, risk notes
- Collapsible raw JSON via `Accordion`
- **Diff highlighting**: Where parsed values differ across models, apply subtle background highlight (e.g., `yellow.0` in Mantine theme). Simple recursive comparison of JSON keys/values.
### Route addition: `frontend/src/App.tsx`
Within the `/admin` route group (after `<Route index element={<AdminPage />} />`):
```tsx
<Route path="shadow-ai" element={<AdminShadowAiPage />} />
```
### Sidebar nav: `frontend/src/components/layout/Sidebar.tsx`
In the `isAdminOnly` section (after the "Admin Panel" NavLink, around line 134):
```tsx
<NavLink
label="AI Benchmarking"
leftSection={<IconScale size={18} />}
active={location.pathname === '/admin/shadow-ai'}
onClick={() => go('/admin/shadow-ai')}
color="violet"
/>
```
## Implementation Order
1. **`ai-caller.ts`** — Shared utility (no dependencies)
2. **Health scores + investment planning** — Make methods public, add exports, add `buildPromptForSchema`
3. **Entities** — 3 TypeORM entity files
4. **Service + Controller + Module** — Shadow AI backend
5. **Register module** in `app.module.ts`
6. **Frontend page**`AdminShadowAiPage.tsx`
7. **Route + Sidebar** — Wire up navigation
## Verification
1. **Backend**: Start server, confirm no TypeORM errors for new entities
2. **Model config**: Use admin UI to save/load/delete alternate model configs
3. **Run comparison**: Select a tenant, trigger a run, verify all 3 models are called with identical prompts
4. **Results display**: Confirm side-by-side output renders correctly for all 3 feature types
5. **History**: Verify past runs are persisted and browsable
6. **Auth**: Confirm non-superadmin users get 403 on all shadow-ai endpoints
7. **Production safety**: Verify no changes to production AI behavior — shadow runs are completely isolated
## Key Files to Modify
- `backend/src/modules/health-scores/health-scores.service.ts` — Make 5 methods public
- `backend/src/modules/health-scores/health-scores.module.ts` — Add exports
- `backend/src/modules/investment-planning/investment-planning.service.ts` — Add `buildPromptForSchema()`, make `buildPromptMessages()` public
- `backend/src/modules/investment-planning/investment-planning.module.ts` — Add exports
- `backend/src/app.module.ts` — Register ShadowAiModule
- `frontend/src/App.tsx` — Add route
- `frontend/src/components/layout/Sidebar.tsx` — Add nav item
## New Files
- `backend/src/common/utils/ai-caller.ts`
- `backend/src/modules/shadow-ai/shadow-ai.module.ts`
- `backend/src/modules/shadow-ai/shadow-ai.service.ts`
- `backend/src/modules/shadow-ai/shadow-ai.controller.ts`
- `backend/src/modules/shadow-ai/entities/shadow-ai-model.entity.ts`
- `backend/src/modules/shadow-ai/entities/shadow-run.entity.ts`
- `backend/src/modules/shadow-ai/entities/shadow-run-result.entity.ts`
- `frontend/src/pages/admin/AdminShadowAiPage.tsx`

View File

@@ -29,6 +29,8 @@ import { SettingsPage } from './pages/settings/SettingsPage';
import { UserPreferencesPage } from './pages/preferences/UserPreferencesPage';
import { OrgMembersPage } from './pages/org-members/OrgMembersPage';
import { AdminPage } from './pages/admin/AdminPage';
import { AdminIdeasPage } from './pages/admin/AdminIdeasPage';
import { AdminShadowAiPage } from './pages/admin/AdminShadowAiPage';
import { AssessmentGroupsPage } from './pages/assessment-groups/AssessmentGroupsPage';
import { CashFlowForecastPage } from './pages/cash-flow/CashFlowForecastPage';
import { MonthlyActualsPage } from './pages/monthly-actuals/MonthlyActualsPage';
@@ -133,6 +135,8 @@ export function App() {
}
>
<Route index element={<AdminPage />} />
<Route path="ideas" element={<AdminIdeasPage />} />
<Route path="shadow-ai" element={<AdminShadowAiPage />} />
</Route>
{/* Main app routes (require auth + org) */}

View File

@@ -0,0 +1,69 @@
import { useState } from 'react';
import { Modal, TextInput, Textarea, Button, Stack } from '@mantine/core';
import { notifications } from '@mantine/notifications';
import { useMutation } from '@tanstack/react-query';
import api from '../../services/api';
interface IdeaModalProps {
opened: boolean;
onClose: () => void;
}
export function IdeaModal({ opened, onClose }: IdeaModalProps) {
const [title, setTitle] = useState('');
const [description, setDescription] = useState('');
const submitIdea = useMutation({
mutationFn: async () => {
const { data } = await api.post('/ideas', { title, description });
return data;
},
onSuccess: () => {
notifications.show({ message: 'Idea submitted — thank you!', color: 'green' });
setTitle('');
setDescription('');
onClose();
},
onError: (err: any) => {
notifications.show({
message: err.response?.data?.message || 'Failed to submit idea',
color: 'red',
});
},
});
const handleClose = () => {
setTitle('');
setDescription('');
onClose();
};
return (
<Modal opened={opened} onClose={handleClose} title="Submit an Idea" size="md">
<Stack>
<TextInput
label="Title"
placeholder="Brief summary of your idea"
required
value={title}
onChange={(e) => setTitle(e.currentTarget.value)}
maxLength={255}
/>
<Textarea
label="Description"
placeholder="Describe your idea in more detail (optional)"
minRows={4}
value={description}
onChange={(e) => setDescription(e.currentTarget.value)}
/>
<Button
onClick={() => submitIdea.mutate()}
loading={submitIdea.isPending}
disabled={!title.trim()}
>
Submit Idea
</Button>
</Stack>
</Modal>
);
}

View File

@@ -11,6 +11,7 @@ import {
IconEyeOff,
IconSun,
IconMoon,
IconBulb,
} from '@tabler/icons-react';
import { Outlet, useNavigate, useLocation } from 'react-router-dom';
import { useAuthStore } from '../../stores/authStore';
@@ -18,6 +19,7 @@ import { usePreferencesStore } from '../../stores/preferencesStore';
import { Sidebar } from './Sidebar';
import { AppTour } from '../onboarding/AppTour';
import { OnboardingWizard } from '../onboarding/OnboardingWizard';
import { IdeaModal } from '../ideas/IdeaModal';
import logoSrc from '../../assets/logo.png';
export function AppLayout() {
@@ -28,6 +30,10 @@ export function AppLayout() {
const location = useLocation();
const isImpersonating = !!impersonationOriginal;
// ── Ideation State ──
const [ideaModalOpened, { open: openIdeaModal, close: closeIdeaModal }] = useDisclosure(false);
const ideationEnabled = currentOrg?.settings?.ideationEnabled === true;
// ── Onboarding State ──
const [showTour, setShowTour] = useState(false);
const [showWizard, setShowWizard] = useState(false);
@@ -121,6 +127,13 @@ export function AppLayout() {
{currentOrg && (
<Text size="sm" c="dimmed">{currentOrg.name}</Text>
)}
{ideationEnabled && (
<Tooltip label="Submit an idea">
<ActionIcon variant="default" size="lg" onClick={openIdeaModal} aria-label="Submit idea">
<IconBulb size={18} />
</ActionIcon>
</Tooltip>
)}
<Tooltip label={colorScheme === 'dark' ? 'Light mode' : 'Dark mode'}>
<ActionIcon
variant="default"
@@ -209,6 +222,9 @@ export function AppLayout() {
{/* ── Onboarding Components ── */}
<AppTour run={showTour} onComplete={handleTourComplete} />
<OnboardingWizard opened={showWizard} onComplete={handleWizardComplete} />
{/* ── Ideation Modal ── */}
<IdeaModal opened={ideaModalOpened} onClose={closeIdeaModal} />
</AppShell>
);
}

View File

@@ -20,6 +20,7 @@ import {
IconCalculator,
IconGitCompare,
IconScale,
IconBulb,
} from '@tabler/icons-react';
import { useAuthStore } from '../../stores/authStore';
@@ -132,6 +133,20 @@ export function Sidebar({ onNavigate }: SidebarProps) {
onClick={() => go('/admin')}
color="red"
/>
<NavLink
label="Idea Submissions"
leftSection={<IconBulb size={18} />}
active={location.pathname === '/admin/ideas'}
onClick={() => go('/admin/ideas')}
color="yellow"
/>
<NavLink
label="AI Benchmarking"
leftSection={<IconScale size={18} />}
active={location.pathname === '/admin/shadow-ai'}
onClick={() => go('/admin/shadow-ai')}
color="violet"
/>
{organizations && organizations.length > 0 && (
<>
<Divider my="sm" />
@@ -230,6 +245,20 @@ export function Sidebar({ onNavigate }: SidebarProps) {
onClick={() => go('/admin')}
color="red"
/>
<NavLink
label="Idea Submissions"
leftSection={<IconBulb size={18} />}
active={location.pathname === '/admin/ideas'}
onClick={() => go('/admin/ideas')}
color="yellow"
/>
<NavLink
label="AI Benchmarking"
leftSection={<IconScale size={18} />}
active={location.pathname === '/admin/shadow-ai'}
onClick={() => go('/admin/shadow-ai')}
color="violet"
/>
</>
)}
</ScrollArea>

View File

@@ -0,0 +1,308 @@
import { useState } from 'react';
import {
Title, Text, Card, Table, Group, Stack, Badge, Loader, Center,
Select, TextInput, Textarea, Button, Modal, SimpleGrid, ActionIcon,
Tooltip, Paper,
} from '@mantine/core';
import { useDisclosure } from '@mantine/hooks';
import { notifications } from '@mantine/notifications';
import {
IconBulb, IconSearch, IconNote, IconFilter,
} from '@tabler/icons-react';
import { useQuery, useMutation, useQueryClient } from '@tanstack/react-query';
import api from '../../services/api';
interface AdminIdea {
id: string;
title: string;
description: string | null;
status: string;
createdAt: string;
adminNote: string | null;
orgId: string;
orgName: string;
userId: string;
userEmail: string;
userFirstName: string;
userLastName: string;
}
const statusColor: Record<string, string> = {
new: 'blue',
reviewed: 'yellow',
accepted: 'green',
rejected: 'red',
};
const statusOptions = [
{ value: 'new', label: 'New' },
{ value: 'reviewed', label: 'Reviewed' },
{ value: 'accepted', label: 'Accepted' },
{ value: 'rejected', label: 'Rejected' },
];
function formatDate(dateStr: string | null | undefined): string {
if (!dateStr) return '—';
return new Date(dateStr).toLocaleDateString();
}
function formatDateTime(dateStr: string | null | undefined): string {
if (!dateStr) return '—';
return new Date(dateStr).toLocaleString();
}
export function AdminIdeasPage() {
const [search, setSearch] = useState('');
const [statusFilter, setStatusFilter] = useState<string | null>(null);
const [selectedIdea, setSelectedIdea] = useState<AdminIdea | null>(null);
const [detailOpened, { open: openDetail, close: closeDetail }] = useDisclosure(false);
const [noteText, setNoteText] = useState('');
const queryClient = useQueryClient();
const { data: ideas, isLoading } = useQuery<AdminIdea[]>({
queryKey: ['admin-ideas'],
queryFn: async () => { const { data } = await api.get('/admin/ideas'); return data; },
});
const updateStatus = useMutation({
mutationFn: async ({ id, status }: { id: string; status: string }) => {
await api.put(`/admin/ideas/${id}/status`, { status });
},
onSuccess: () => {
queryClient.invalidateQueries({ queryKey: ['admin-ideas'] });
notifications.show({ message: 'Status updated', color: 'green' });
},
});
const updateNote = useMutation({
mutationFn: async ({ id, adminNote }: { id: string; adminNote: string }) => {
await api.put(`/admin/ideas/${id}/note`, { adminNote });
},
onSuccess: () => {
queryClient.invalidateQueries({ queryKey: ['admin-ideas'] });
notifications.show({ message: 'Note saved', color: 'green' });
},
});
const openIdeaDetail = (idea: AdminIdea) => {
setSelectedIdea(idea);
setNoteText(idea.adminNote || '');
openDetail();
};
const handleSaveNote = () => {
if (selectedIdea) {
updateNote.mutate({ id: selectedIdea.id, adminNote: noteText });
}
};
const filtered = (ideas || []).filter((idea) => {
const matchesSearch = !search ||
idea.title.toLowerCase().includes(search.toLowerCase()) ||
idea.description?.toLowerCase().includes(search.toLowerCase()) ||
idea.orgName.toLowerCase().includes(search.toLowerCase()) ||
idea.userEmail.toLowerCase().includes(search.toLowerCase());
const matchesStatus = !statusFilter || idea.status === statusFilter;
return matchesSearch && matchesStatus;
});
const counts = {
total: ideas?.length || 0,
new: ideas?.filter(i => i.status === 'new').length || 0,
reviewed: ideas?.filter(i => i.status === 'reviewed').length || 0,
accepted: ideas?.filter(i => i.status === 'accepted').length || 0,
rejected: ideas?.filter(i => i.status === 'rejected').length || 0,
};
if (isLoading) {
return <Center h={400}><Loader /></Center>;
}
return (
<Stack>
<Group justify="space-between">
<Group>
<IconBulb size={28} />
<Title order={2}>Idea Submissions</Title>
</Group>
<Badge size="lg" variant="light">{counts.total} total</Badge>
</Group>
{/* Summary cards */}
<SimpleGrid cols={{ base: 2, sm: 4 }}>
<Paper withBorder p="md" radius="md">
<Text size="xs" c="dimmed" tt="uppercase" fw={700}>New</Text>
<Text size="xl" fw={700} c="blue">{counts.new}</Text>
</Paper>
<Paper withBorder p="md" radius="md">
<Text size="xs" c="dimmed" tt="uppercase" fw={700}>Reviewed</Text>
<Text size="xl" fw={700} c="yellow">{counts.reviewed}</Text>
</Paper>
<Paper withBorder p="md" radius="md">
<Text size="xs" c="dimmed" tt="uppercase" fw={700}>Accepted</Text>
<Text size="xl" fw={700} c="green">{counts.accepted}</Text>
</Paper>
<Paper withBorder p="md" radius="md">
<Text size="xs" c="dimmed" tt="uppercase" fw={700}>Rejected</Text>
<Text size="xl" fw={700} c="red">{counts.rejected}</Text>
</Paper>
</SimpleGrid>
{/* Filters */}
<Group>
<TextInput
placeholder="Search ideas, tenants, users..."
leftSection={<IconSearch size={16} />}
value={search}
onChange={(e) => setSearch(e.currentTarget.value)}
style={{ flex: 1 }}
/>
<Select
placeholder="All statuses"
leftSection={<IconFilter size={16} />}
data={statusOptions}
value={statusFilter}
onChange={setStatusFilter}
clearable
w={160}
/>
</Group>
{/* Ideas table */}
<Card withBorder p={0}>
<Table striped highlightOnHover>
<Table.Thead>
<Table.Tr>
<Table.Th>Date</Table.Th>
<Table.Th>Tenant</Table.Th>
<Table.Th>Submitted By</Table.Th>
<Table.Th>Title</Table.Th>
<Table.Th>Status</Table.Th>
<Table.Th w={40}></Table.Th>
</Table.Tr>
</Table.Thead>
<Table.Tbody>
{filtered.length === 0 ? (
<Table.Tr>
<Table.Td colSpan={6}>
<Text ta="center" c="dimmed" py="lg">
{ideas?.length === 0 ? 'No ideas submitted yet' : 'No ideas match your filters'}
</Text>
</Table.Td>
</Table.Tr>
) : (
filtered.map((idea) => (
<Table.Tr
key={idea.id}
style={{ cursor: 'pointer' }}
onClick={() => openIdeaDetail(idea)}
>
<Table.Td>
<Text size="xs">{formatDate(idea.createdAt)}</Text>
</Table.Td>
<Table.Td>
<Text size="sm" fw={500}>{idea.orgName}</Text>
</Table.Td>
<Table.Td>
<Text size="sm">{idea.userFirstName} {idea.userLastName}</Text>
<Text size="xs" c="dimmed">{idea.userEmail}</Text>
</Table.Td>
<Table.Td>
<Text size="sm" fw={500} lineClamp={1}>{idea.title}</Text>
</Table.Td>
<Table.Td>
<Badge size="sm" variant="light" color={statusColor[idea.status]}>
{idea.status}
</Badge>
</Table.Td>
<Table.Td>
{idea.adminNote && (
<Tooltip label="Has admin note">
<IconNote size={16} color="gray" />
</Tooltip>
)}
</Table.Td>
</Table.Tr>
))
)}
</Table.Tbody>
</Table>
</Card>
{/* Detail Modal */}
<Modal
opened={detailOpened}
onClose={closeDetail}
title={<Text fw={600}>Idea Detail</Text>}
size="lg"
>
{selectedIdea && (
<Stack>
<Card withBorder>
<SimpleGrid cols={2} spacing="xs">
<Text size="xs" c="dimmed">Tenant</Text>
<Text size="sm" fw={500}>{selectedIdea.orgName}</Text>
<Text size="xs" c="dimmed">Submitted By</Text>
<Text size="sm">{selectedIdea.userFirstName} {selectedIdea.userLastName} ({selectedIdea.userEmail})</Text>
<Text size="xs" c="dimmed">Date</Text>
<Text size="sm">{formatDateTime(selectedIdea.createdAt)}</Text>
</SimpleGrid>
</Card>
<Card withBorder>
<Text fw={600} mb="xs">Title</Text>
<Text size="sm">{selectedIdea.title}</Text>
{selectedIdea.description && (
<>
<Text fw={600} mt="md" mb="xs">Description</Text>
<Text size="sm" style={{ whiteSpace: 'pre-wrap' }}>{selectedIdea.description}</Text>
</>
)}
</Card>
<Card withBorder>
<Text fw={600} mb="xs">Status</Text>
<Select
data={statusOptions}
value={selectedIdea.status}
onChange={(val) => {
if (val && val !== selectedIdea.status) {
updateStatus.mutate({ id: selectedIdea.id, status: val }, {
onSuccess: () => {
setSelectedIdea({ ...selectedIdea, status: val });
},
});
}
}}
w={200}
/>
</Card>
<Card withBorder>
<Group justify="space-between" mb="xs">
<Text fw={600}>Private Admin Note</Text>
<Text size="xs" c="dimmed">Only visible to super admins</Text>
</Group>
<Textarea
placeholder="Add internal notes — sprint reference, thoughts, follow-up actions..."
minRows={3}
value={noteText}
onChange={(e) => setNoteText(e.currentTarget.value)}
/>
<Button
size="xs"
variant="light"
mt="xs"
onClick={handleSaveNote}
loading={updateNote.isPending}
disabled={noteText === (selectedIdea.adminNote || '')}
>
Save Note
</Button>
</Card>
</Stack>
)}
</Modal>
</Stack>
);
}

View File

@@ -11,7 +11,7 @@ import {
IconCrown, IconPlus, IconArchive, IconChevronDown,
IconCircleCheck, IconBan, IconArchiveOff, IconDashboard,
IconHeartRateMonitor, IconSparkles, IconCalendar, IconActivity,
IconCurrencyDollar, IconClipboardCheck, IconLogin, IconEye,
IconCurrencyDollar, IconClipboardCheck, IconLogin, IconEye, IconBulb,
} from '@tabler/icons-react';
import { useQuery, useMutation, useQueryClient } from '@tanstack/react-query';
import { useNavigate } from 'react-router-dom';
@@ -211,6 +211,16 @@ export function AdminPage() {
},
});
const toggleIdeation = useMutation({
mutationFn: async ({ orgId, enabled }: { orgId: string; enabled: boolean }) => {
await api.put(`/admin/organizations/${orgId}/settings`, { ideationEnabled: enabled });
},
onSuccess: () => {
queryClient.invalidateQueries({ queryKey: ['admin-tenant-detail', selectedOrgId] });
queryClient.invalidateQueries({ queryKey: ['admin-orgs'] });
},
});
const impersonateUser = useMutation({
mutationFn: async (userId: string) => {
const { data } = await api.post(`/admin/impersonate/${userId}`);
@@ -782,6 +792,27 @@ export function AdminPage() {
</SimpleGrid>
</Card>
<Card withBorder>
<Text fw={600} mb="xs">Feature Toggles</Text>
<Group justify="space-between">
<Group gap="xs">
<IconBulb size={16} />
<div>
<Text size="sm">Ideation</Text>
<Text size="xs" c="dimmed">Allow users to submit feature ideas</Text>
</div>
</Group>
<Switch
checked={tenantDetail.organization.settings?.ideationEnabled === true}
onChange={(e) => {
if (selectedOrgId) {
toggleIdeation.mutate({ orgId: selectedOrgId, enabled: e.currentTarget.checked });
}
}}
/>
</Group>
</Card>
<Card withBorder>
<Text fw={600} mb="xs">Subscription</Text>
<Stack gap="xs">

View File

@@ -0,0 +1,780 @@
import { useState, useEffect } from 'react';
import {
Title, Text, Card, SimpleGrid, Group, Stack, Badge, Loader, Center,
Tabs, TextInput, Button, PasswordInput, Select, Table, Accordion,
Switch, Paper, RingProgress, Divider, Alert, Code, ScrollArea, Box,
Tooltip, ActionIcon,
} from '@mantine/core';
import {
IconScale, IconSettings, IconPlayerPlay, IconHistory,
IconCheck, IconX, IconAlertTriangle, IconClock, IconTrash,
IconRefresh, IconArrowRight, IconChevronDown,
} from '@tabler/icons-react';
import { useQuery, useMutation, useQueryClient } from '@tanstack/react-query';
import api from '../../services/api';
// ── Interfaces ──
interface ShadowModel {
id: string;
slot: string;
name: string;
api_url: string;
api_key: string;
model_name: string;
is_active: boolean;
created_at: string;
updated_at: string;
}
interface ShadowRunResult {
id: string;
run_id: string;
model_role: string;
model_name: string;
api_url: string;
raw_response: string;
parsed_response: any;
response_time_ms: number;
token_usage: any;
status: string;
error_message: string;
created_at: string;
}
interface ShadowRun {
id: string;
tenant_id: string;
tenant_name: string;
feature: string;
status: string;
prompt_messages: any;
started_at: string;
completed_at: string;
created_at: string;
results: ShadowRunResult[];
result_count?: string;
success_count?: string;
}
interface AdminOrg {
id: string;
name: string;
status: string;
}
// ── Helper Functions ──
const featureLabels: Record<string, string> = {
operating_health: 'Operating Health',
reserve_health: 'Reserve Health',
investment_recommendations: 'Investment Recommendations',
};
const roleLabels: Record<string, string> = {
production: 'Production',
alternate_a: 'Alternate A',
alternate_b: 'Alternate B',
};
const statusColor: Record<string, string> = {
running: 'blue',
completed: 'green',
partial: 'yellow',
failed: 'red',
pending: 'gray',
success: 'green',
error: 'red',
};
function formatDuration(ms: number | null): string {
if (!ms) return '-';
if (ms < 1000) return `${ms}ms`;
return `${(ms / 1000).toFixed(1)}s`;
}
function formatDate(d: string): string {
if (!d) return '-';
return new Date(d).toLocaleString();
}
// ── Model Configuration Tab ──
function ModelConfigTab() {
const queryClient = useQueryClient();
const { data: models, isLoading } = useQuery<ShadowModel[]>({
queryKey: ['shadow-ai-models'],
queryFn: () => api.get('/admin/shadow-ai/models').then((r) => r.data),
});
const modelA = models?.find((m) => m.slot === 'A');
const modelB = models?.find((m) => m.slot === 'B');
return (
<Stack>
<Text size="sm" c="dimmed">
Configure alternate AI models to benchmark against the production model.
Each model can use any OpenAI-compatible API endpoint.
</Text>
<SimpleGrid cols={{ base: 1, md: 3 }}>
<ProductionModelCard />
<ModelSlotCard slot="A" model={modelA} isLoading={isLoading} />
<ModelSlotCard slot="B" model={modelB} isLoading={isLoading} />
</SimpleGrid>
</Stack>
);
}
function ProductionModelCard() {
return (
<Card withBorder shadow="sm">
<Stack gap="sm">
<Group justify="space-between">
<Text fw={600}>Production Model</Text>
<Badge color="green" variant="light">Active</Badge>
</Group>
<Divider />
<Text size="sm" c="dimmed">Configured via environment variables</Text>
<TextInput label="Model" value="(from AI_MODEL env var)" readOnly disabled size="sm" />
<TextInput label="API URL" value="(from AI_API_URL env var)" readOnly disabled size="sm" />
<Text size="xs" c="dimmed" mt="xs">
Production model settings are managed through server environment
variables and cannot be changed from the UI.
</Text>
</Stack>
</Card>
);
}
function ModelSlotCard({ slot, model, isLoading }: { slot: string; model?: ShadowModel; isLoading: boolean }) {
const queryClient = useQueryClient();
const [name, setName] = useState('');
const [apiUrl, setApiUrl] = useState('');
const [apiKey, setApiKey] = useState('');
const [modelName, setModelName] = useState('');
const [isActive, setIsActive] = useState(true);
useEffect(() => {
if (model) {
setName(model.name);
setApiUrl(model.api_url);
setApiKey(model.api_key);
setModelName(model.model_name);
setIsActive(model.is_active);
}
}, [model]);
const saveMutation = useMutation({
mutationFn: () => api.put(`/admin/shadow-ai/models/${slot}`, { name, apiUrl, apiKey, modelName, isActive }),
onSuccess: () => queryClient.invalidateQueries({ queryKey: ['shadow-ai-models'] }),
});
const deleteMutation = useMutation({
mutationFn: () => api.delete(`/admin/shadow-ai/models/${slot}`),
onSuccess: () => {
setName(''); setApiUrl(''); setApiKey(''); setModelName(''); setIsActive(true);
queryClient.invalidateQueries({ queryKey: ['shadow-ai-models'] });
},
});
if (isLoading) return <Card withBorder shadow="sm"><Center h={200}><Loader size="sm" /></Center></Card>;
return (
<Card withBorder shadow="sm">
<Stack gap="sm">
<Group justify="space-between">
<Text fw={600}>Alternate {slot}</Text>
{model ? (
<Badge color={isActive ? 'blue' : 'gray'} variant="light">
{isActive ? 'Active' : 'Inactive'}
</Badge>
) : (
<Badge color="gray" variant="light">Not configured</Badge>
)}
</Group>
<Divider />
<TextInput label="Display Name" placeholder="e.g. GPT-4o" value={name} onChange={(e) => setName(e.target.value)} size="sm" />
<TextInput label="API URL" placeholder="https://api.openai.com/v1" value={apiUrl} onChange={(e) => setApiUrl(e.target.value)} size="sm" />
<PasswordInput label="API Key" placeholder="sk-..." value={apiKey} onChange={(e) => setApiKey(e.target.value)} size="sm" />
<TextInput label="Model Name" placeholder="gpt-4o" value={modelName} onChange={(e) => setModelName(e.target.value)} size="sm" />
<Switch label="Active" checked={isActive} onChange={(e) => setIsActive(e.currentTarget.checked)} />
<Group>
<Button
size="sm"
onClick={() => saveMutation.mutate()}
loading={saveMutation.isPending}
disabled={!name || !apiUrl || !apiKey || !modelName}
>
Save
</Button>
{model && (
<Button size="sm" color="red" variant="light" onClick={() => deleteMutation.mutate()} loading={deleteMutation.isPending}>
<IconTrash size={16} />
</Button>
)}
</Group>
{saveMutation.isError && <Text size="xs" c="red">Failed to save</Text>}
{saveMutation.isSuccess && <Text size="xs" c="green">Saved</Text>}
</Stack>
</Card>
);
}
// ── Run Comparison Tab ──
function RunComparisonTab() {
const queryClient = useQueryClient();
const [tenantId, setTenantId] = useState<string | null>(null);
const [feature, setFeature] = useState<string | null>(null);
const [activeRunId, setActiveRunId] = useState<string | null>(null);
const { data: orgs } = useQuery<AdminOrg[]>({
queryKey: ['admin-orgs'],
queryFn: () => api.get('/admin/organizations').then((r) => r.data),
});
const triggerMutation = useMutation({
mutationFn: () => api.post('/admin/shadow-ai/runs', { tenantId, feature }),
onSuccess: (res) => {
setActiveRunId(res.data.runId);
},
});
const { data: activeRun } = useQuery<ShadowRun>({
queryKey: ['shadow-ai-run', activeRunId],
queryFn: () => api.get(`/admin/shadow-ai/runs/${activeRunId}`).then((r) => r.data),
enabled: !!activeRunId,
refetchInterval: (query) => {
const run = query.state.data;
return run?.status === 'running' ? 3000 : false;
},
});
const orgOptions = (orgs || [])
.filter((o) => o.status === 'active')
.map((o) => ({ value: o.id, label: o.name }));
const featureOptions = [
{ value: 'operating_health', label: 'Operating Health Score' },
{ value: 'reserve_health', label: 'Reserve Health Score' },
{ value: 'investment_recommendations', label: 'Investment Recommendations' },
];
return (
<Stack>
<Card withBorder shadow="sm">
<Stack gap="md">
<Text fw={600}>Run Shadow Comparison</Text>
<SimpleGrid cols={{ base: 1, sm: 3 }}>
<Select
label="Tenant"
placeholder="Select a tenant"
data={orgOptions}
value={tenantId}
onChange={setTenantId}
searchable
/>
<Select
label="AI Feature"
placeholder="Select feature"
data={featureOptions}
value={feature}
onChange={setFeature}
/>
<Stack justify="flex-end">
<Button
leftSection={<IconPlayerPlay size={16} />}
onClick={() => triggerMutation.mutate()}
loading={triggerMutation.isPending}
disabled={!tenantId || !feature}
>
Run Comparison
</Button>
</Stack>
</SimpleGrid>
{triggerMutation.isError && (
<Alert color="red" icon={<IconAlertTriangle size={16} />}>
Failed to start comparison. Ensure at least one alternate model is configured.
</Alert>
)}
</Stack>
</Card>
{activeRun && (
<Card withBorder shadow="sm">
<Stack gap="md">
<Group justify="space-between">
<Group>
<Text fw={600}>
{featureLabels[activeRun.feature] || activeRun.feature}
</Text>
<Badge color={statusColor[activeRun.status]}>{activeRun.status}</Badge>
</Group>
{activeRun.tenant_name && (
<Text size="sm" c="dimmed">Tenant: {activeRun.tenant_name}</Text>
)}
</Group>
{activeRun.status === 'running' && (
<Center py="md">
<Stack align="center" gap="xs">
<Loader size="md" />
<Text size="sm" c="dimmed">Running models... This may take a few minutes.</Text>
<Group gap="xs">
{(activeRun.results || []).map((r) => (
<Badge key={r.model_role} color={statusColor[r.status]} variant="light">
{roleLabels[r.model_role]}: {r.status}
</Badge>
))}
</Group>
</Stack>
</Center>
)}
{activeRun.status !== 'running' && activeRun.results && (
<ComparisonResults results={activeRun.results} feature={activeRun.feature} />
)}
</Stack>
</Card>
)}
</Stack>
);
}
// ── History Tab ──
function HistoryTab() {
const [selectedRunId, setSelectedRunId] = useState<string | null>(null);
const [tenantFilter, setTenantFilter] = useState<string | null>(null);
const [featureFilter, setFeatureFilter] = useState<string | null>(null);
const { data: orgs } = useQuery<AdminOrg[]>({
queryKey: ['admin-orgs'],
queryFn: () => api.get('/admin/organizations').then((r) => r.data),
});
const { data: historyData, isLoading } = useQuery({
queryKey: ['shadow-ai-runs', tenantFilter, featureFilter],
queryFn: () => {
const params = new URLSearchParams();
if (tenantFilter) params.set('tenantId', tenantFilter);
if (featureFilter) params.set('feature', featureFilter);
params.set('limit', '50');
return api.get(`/admin/shadow-ai/runs?${params}`).then((r) => r.data);
},
});
const { data: selectedRun } = useQuery<ShadowRun>({
queryKey: ['shadow-ai-run', selectedRunId],
queryFn: () => api.get(`/admin/shadow-ai/runs/${selectedRunId}`).then((r) => r.data),
enabled: !!selectedRunId,
});
const orgOptions = [
{ value: '', label: 'All Tenants' },
...(orgs || []).map((o) => ({ value: o.id, label: o.name })),
];
const featureOptions = [
{ value: '', label: 'All Features' },
{ value: 'operating_health', label: 'Operating Health' },
{ value: 'reserve_health', label: 'Reserve Health' },
{ value: 'investment_recommendations', label: 'Investment Recommendations' },
];
const runs: ShadowRun[] = historyData?.runs || [];
return (
<Stack>
<Group>
<Select
size="sm"
placeholder="Filter by tenant"
data={orgOptions}
value={tenantFilter || ''}
onChange={(v) => setTenantFilter(v || null)}
clearable
w={200}
/>
<Select
size="sm"
placeholder="Filter by feature"
data={featureOptions}
value={featureFilter || ''}
onChange={(v) => setFeatureFilter(v || null)}
clearable
w={200}
/>
</Group>
{isLoading ? (
<Center py="xl"><Loader /></Center>
) : runs.length === 0 ? (
<Text c="dimmed" ta="center" py="xl">No shadow runs found.</Text>
) : (
<Table striped highlightOnHover>
<Table.Thead>
<Table.Tr>
<Table.Th>Date</Table.Th>
<Table.Th>Tenant</Table.Th>
<Table.Th>Feature</Table.Th>
<Table.Th>Status</Table.Th>
<Table.Th>Models</Table.Th>
<Table.Th>Duration</Table.Th>
<Table.Th></Table.Th>
</Table.Tr>
</Table.Thead>
<Table.Tbody>
{runs.map((run) => {
const duration = run.completed_at && run.started_at
? new Date(run.completed_at).getTime() - new Date(run.started_at).getTime()
: null;
return (
<Table.Tr
key={run.id}
style={{ cursor: 'pointer' }}
onClick={() => setSelectedRunId(run.id)}
bg={selectedRunId === run.id ? 'var(--mantine-color-blue-light)' : undefined}
>
<Table.Td>{formatDate(run.created_at)}</Table.Td>
<Table.Td>{run.tenant_name || '-'}</Table.Td>
<Table.Td>{featureLabels[run.feature] || run.feature}</Table.Td>
<Table.Td><Badge color={statusColor[run.status]} size="sm">{run.status}</Badge></Table.Td>
<Table.Td>{run.success_count || '0'}/{run.result_count || '0'}</Table.Td>
<Table.Td>{formatDuration(duration)}</Table.Td>
<Table.Td><IconArrowRight size={14} /></Table.Td>
</Table.Tr>
);
})}
</Table.Tbody>
</Table>
)}
{selectedRun && selectedRun.results && (
<Card withBorder shadow="sm" mt="md">
<Stack gap="md">
<Group justify="space-between">
<Group>
<Text fw={600}>{featureLabels[selectedRun.feature] || selectedRun.feature}</Text>
<Badge color={statusColor[selectedRun.status]}>{selectedRun.status}</Badge>
</Group>
<Text size="sm" c="dimmed">
{selectedRun.tenant_name} | {formatDate(selectedRun.created_at)}
</Text>
</Group>
<ComparisonResults results={selectedRun.results} feature={selectedRun.feature} />
</Stack>
</Card>
)}
</Stack>
);
}
// ── Comparison Results Component ──
function ComparisonResults({ results, feature }: { results: ShadowRunResult[]; feature: string }) {
const isHealthScore = feature === 'operating_health' || feature === 'reserve_health';
// Collect all parsed values for diff highlighting
const parsedValues = results
.filter((r) => r.status === 'success' && r.parsed_response)
.map((r) => r.parsed_response);
return (
<SimpleGrid cols={{ base: 1, md: Math.min(results.length, 3) }}>
{results.map((result) => (
<ResultCard
key={result.model_role}
result={result}
isHealthScore={isHealthScore}
allParsed={parsedValues}
/>
))}
</SimpleGrid>
);
}
function ResultCard({
result,
isHealthScore,
allParsed,
}: {
result: ShadowRunResult;
isHealthScore: boolean;
allParsed: any[];
}) {
const roleColor: Record<string, string> = {
production: 'green',
alternate_a: 'blue',
alternate_b: 'violet',
};
return (
<Card withBorder shadow="xs" padding="md">
<Stack gap="sm">
<Group justify="space-between">
<Group gap="xs">
<Badge color={roleColor[result.model_role] || 'gray'} variant="filled">
{roleLabels[result.model_role]}
</Badge>
</Group>
<Badge
color={statusColor[result.status]}
variant="light"
leftSection={result.status === 'success' ? <IconCheck size={12} /> : result.status === 'error' ? <IconX size={12} /> : <IconClock size={12} />}
>
{result.status}
</Badge>
</Group>
<Text size="xs" c="dimmed" truncate>{result.model_name}</Text>
{result.response_time_ms && (
<Badge color="gray" variant="light" size="sm">
{formatDuration(result.response_time_ms)}
</Badge>
)}
{result.token_usage && (
<Text size="xs" c="dimmed">
Tokens: {result.token_usage.prompt_tokens || '?'} prompt / {result.token_usage.completion_tokens || '?'} completion
</Text>
)}
<Divider />
{result.status === 'error' && (
<Alert color="red" icon={<IconAlertTriangle size={16} />}>
<Text size="sm">{result.error_message || 'Unknown error'}</Text>
</Alert>
)}
{result.status === 'success' && result.parsed_response && (
isHealthScore
? <HealthScoreDisplay data={result.parsed_response} allParsed={allParsed} />
: <InvestmentDisplay data={result.parsed_response} allParsed={allParsed} />
)}
{result.status === 'success' && (
<Accordion variant="contained">
<Accordion.Item value="raw">
<Accordion.Control>
<Text size="xs">Raw JSON Response</Text>
</Accordion.Control>
<Accordion.Panel>
<ScrollArea h={300}>
<Code block style={{ fontSize: 11 }}>
{JSON.stringify(result.parsed_response, null, 2)}
</Code>
</ScrollArea>
</Accordion.Panel>
</Accordion.Item>
</Accordion>
)}
</Stack>
</Card>
);
}
// ── Health Score Display ──
function HealthScoreDisplay({ data, allParsed }: { data: any; allParsed: any[] }) {
const score = data.score ?? data.raw_text;
const label = data.label || '';
const summary = data.summary || '';
const factors = data.factors || [];
const recommendations = data.recommendations || [];
// Check if score differs from other models
const scores = allParsed.map((p) => p.score).filter((s) => typeof s === 'number');
const scoreDiffers = scores.length > 1 && !scores.every((s) => s === scores[0]);
const labelColor: Record<string, string> = {
Excellent: 'green', Good: 'teal', Fair: 'yellow',
'Needs Attention': 'orange', 'At Risk': 'red', Critical: 'red',
};
return (
<Stack gap="sm">
{typeof score === 'number' && (
<Group justify="center">
<Box bg={scoreDiffers ? 'yellow.0' : undefined} p="xs" style={{ borderRadius: 8 }}>
<RingProgress
size={100}
thickness={10}
roundCaps
sections={[{ value: score, color: labelColor[label] || 'blue' }]}
label={
<Text ta="center" fw={700} size="lg">{score}</Text>
}
/>
</Box>
</Group>
)}
{label && (
<Group justify="center">
<Badge color={labelColor[label] || 'gray'} size="lg">{label}</Badge>
</Group>
)}
{summary && <Text size="sm">{summary}</Text>}
{factors.length > 0 && (
<>
<Text size="xs" fw={600} c="dimmed" tt="uppercase">Factors</Text>
{factors.map((f: any, i: number) => (
<Group key={i} gap="xs" wrap="nowrap">
<Badge
size="xs"
variant="light"
color={f.impact === 'positive' ? 'green' : f.impact === 'negative' ? 'red' : 'gray'}
>
{f.impact}
</Badge>
<Text size="xs" style={{ flex: 1 }}><b>{f.name}:</b> {f.detail}</Text>
</Group>
))}
</>
)}
{recommendations.length > 0 && (
<>
<Text size="xs" fw={600} c="dimmed" tt="uppercase">Recommendations</Text>
{recommendations.map((r: any, i: number) => (
<Group key={i} gap="xs" wrap="nowrap">
<Badge
size="xs"
variant="light"
color={r.priority === 'high' ? 'red' : r.priority === 'medium' ? 'yellow' : 'blue'}
>
{r.priority}
</Badge>
<Text size="xs" style={{ flex: 1 }}>{r.text}</Text>
</Group>
))}
</>
)}
</Stack>
);
}
// ── Investment Display ──
function InvestmentDisplay({ data, allParsed }: { data: any; allParsed: any[] }) {
const recommendations = data.recommendations || [];
const overall = data.overall_assessment || '';
const riskNotes = data.risk_notes || [];
const recCounts = allParsed.map((p) => (p.recommendations || []).length);
const countDiffers = recCounts.length > 1 && !recCounts.every((c) => c === recCounts[0]);
const typeColors: Record<string, string> = {
cd_ladder: 'violet', new_investment: 'blue', reallocation: 'teal',
maturity_action: 'orange', liquidity_warning: 'red', general: 'gray',
};
return (
<Stack gap="sm">
{overall && (
<Paper p="xs" bg="gray.0" radius="sm">
<Text size="sm">{overall}</Text>
</Paper>
)}
{recommendations.length > 0 && (
<>
<Group gap="xs">
<Text size="xs" fw={600} c="dimmed" tt="uppercase">
Recommendations
</Text>
<Badge
size="xs"
variant="light"
color={countDiffers ? 'yellow' : 'gray'}
>
{recommendations.length}
</Badge>
</Group>
{recommendations.map((rec: any, i: number) => (
<Card key={i} withBorder padding="xs" radius="sm">
<Stack gap={4}>
<Group gap="xs">
<Badge size="xs" color={typeColors[rec.type] || 'gray'}>{rec.type}</Badge>
<Badge size="xs" variant="light" color={rec.priority === 'high' ? 'red' : rec.priority === 'medium' ? 'yellow' : 'blue'}>
{rec.priority}
</Badge>
{rec.fund_type && <Badge size="xs" variant="outline">{rec.fund_type}</Badge>}
</Group>
<Text size="sm" fw={600}>{rec.title}</Text>
<Text size="xs">{rec.summary}</Text>
{rec.suggested_amount && (
<Text size="xs" c="dimmed">
Amount: ${rec.suggested_amount.toLocaleString()}
{rec.suggested_rate ? ` | Rate: ${rec.suggested_rate}%` : ''}
{rec.suggested_term ? ` | Term: ${rec.suggested_term}` : ''}
</Text>
)}
</Stack>
</Card>
))}
</>
)}
{riskNotes.length > 0 && (
<>
<Text size="xs" fw={600} c="dimmed" tt="uppercase">Risk Notes</Text>
{riskNotes.map((note: string, i: number) => (
<Group key={i} gap="xs" wrap="nowrap">
<IconAlertTriangle size={14} color="orange" />
<Text size="xs">{note}</Text>
</Group>
))}
</>
)}
</Stack>
);
}
// ── Main Page ──
export function AdminShadowAiPage() {
return (
<Stack gap="lg" p="md">
<Group>
<IconScale size={28} />
<Title order={2}>AI Benchmarking</Title>
</Group>
<Text c="dimmed" size="sm">
Compare AI model outputs side-by-side using real tenant data.
Configure alternate models, run shadow comparisons, and review historical results.
</Text>
<Tabs defaultValue="run">
<Tabs.List>
<Tabs.Tab value="config" leftSection={<IconSettings size={16} />}>
Model Configuration
</Tabs.Tab>
<Tabs.Tab value="run" leftSection={<IconPlayerPlay size={16} />}>
Run Comparison
</Tabs.Tab>
<Tabs.Tab value="history" leftSection={<IconHistory size={16} />}>
History
</Tabs.Tab>
</Tabs.List>
<Tabs.Panel value="config" pt="md">
<ModelConfigTab />
</Tabs.Panel>
<Tabs.Panel value="run" pt="md">
<RunComparisonTab />
</Tabs.Panel>
<Tabs.Panel value="history" pt="md">
<HistoryTab />
</Tabs.Panel>
</Tabs>
</Stack>
);
}

View File

@@ -237,7 +237,7 @@ export function SettingsPage() {
</Group>
<Group justify="space-between">
<Text size="sm" c="dimmed">Version</Text>
<Badge variant="light">2026.03.18</Badge>
<Badge variant="light">2026.4.2</Badge>
</Group>
<Group justify="space-between">
<Text size="sm" c="dimmed">API</Text>

View File

@@ -0,0 +1,141 @@
{
"_meta": {
"capturedAt": null,
"environment": "staging",
"k6Version": null,
"vus": null,
"duration": null,
"notes": "Empty baseline populate after first stable load test run"
},
"auth": {
"POST /api/auth/login": {
"p50": null,
"p95": null,
"p99": null,
"errorRate": null,
"rps": null
},
"POST /api/auth/refresh": {
"p50": null,
"p95": null,
"p99": null,
"errorRate": null,
"rps": null
},
"POST /api/auth/logout": {
"p50": null,
"p95": null,
"p99": null,
"errorRate": null,
"rps": null
},
"GET /api/auth/profile": {
"p50": null,
"p95": null,
"p99": null,
"errorRate": null,
"rps": null
}
},
"dashboard": {
"GET /api/reports/dashboard": {
"p50": null,
"p95": null,
"p99": null,
"errorRate": null,
"rps": null
},
"GET /api/reports/balance-sheet": {
"p50": null,
"p95": null,
"p99": null,
"errorRate": null,
"rps": null
},
"GET /api/reports/income-statement": {
"p50": null,
"p95": null,
"p99": null,
"errorRate": null,
"rps": null
},
"GET /api/reports/aging": {
"p50": null,
"p95": null,
"p99": null,
"errorRate": null,
"rps": null
},
"GET /api/health-scores/latest": {
"p50": null,
"p95": null,
"p99": null,
"errorRate": null,
"rps": null
},
"GET /api/reports/cash-flow": {
"p50": null,
"p95": null,
"p99": null,
"errorRate": null,
"rps": null
},
"GET /api/reports/cash-flow-forecast": {
"p50": null,
"p95": null,
"p99": null,
"errorRate": null,
"rps": null
}
},
"crud": {
"units": {
"GET /api/units": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null },
"POST /api/units": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null },
"GET /api/units/:id": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null },
"PUT /api/units/:id": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null },
"DELETE /api/units/:id": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null }
},
"vendors": {
"GET /api/vendors": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null },
"POST /api/vendors": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null },
"GET /api/vendors/:id": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null },
"PUT /api/vendors/:id": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null }
},
"journalEntries": {
"GET /api/journal-entries": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null },
"POST /api/journal-entries": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null },
"GET /api/journal-entries/:id": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null },
"POST /api/journal-entries/:id/post": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null },
"POST /api/journal-entries/:id/void": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null }
},
"payments": {
"GET /api/payments": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null },
"POST /api/payments": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null },
"GET /api/payments/:id": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null },
"PUT /api/payments/:id": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null },
"DELETE /api/payments/:id": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null }
},
"accounts": {
"GET /api/accounts": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null },
"GET /api/accounts/trial-balance": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null },
"POST /api/accounts": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null },
"PUT /api/accounts/:id": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null }
},
"invoices": {
"GET /api/invoices": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null },
"GET /api/invoices/:id": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null },
"POST /api/invoices/generate-bulk": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null }
}
},
"boardPlanning": {
"GET /api/board-planning/scenarios": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null },
"POST /api/board-planning/scenarios": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null },
"GET /api/board-planning/scenarios/:id/projection": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null },
"GET /api/board-planning/budget-plans": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null }
},
"organizations": {
"GET /api/organizations": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null },
"GET /api/organizations/members": { "p50": null, "p95": null, "p99": null, "errorRate": null, "rps": null }
}
}

View File

@@ -0,0 +1,270 @@
-- =============================================================================
-- HOA Financial Platform (HOALedgerIQ) New Relic NRQL Query Library
-- App Name: HOALedgerIQ_App (see NEW_RELIC_APP_NAME env var)
-- =============================================================================
-- ---------------------------------------------------------------------------
-- 1. OVERVIEW & THROUGHPUT
-- ---------------------------------------------------------------------------
-- Overall throughput (requests/min) over past hour
SELECT rate(count(*), 1 minute) AS 'RPM'
FROM Transaction
WHERE appName = 'HOALedgerIQ_App'
SINCE 1 hour ago
TIMESERIES AUTO;
-- Throughput by HTTP method
SELECT rate(count(*), 1 minute) AS 'RPM'
FROM Transaction
WHERE appName = 'HOALedgerIQ_App'
FACET request.method
SINCE 1 hour ago
TIMESERIES AUTO;
-- Apdex score over time
SELECT apdex(duration, t: 0.5)
FROM Transaction
WHERE appName = 'HOALedgerIQ_App'
SINCE 1 hour ago
TIMESERIES AUTO;
-- ---------------------------------------------------------------------------
-- 2. AUTHENTICATION ENDPOINTS
-- ---------------------------------------------------------------------------
-- Login latency (p50, p95, p99)
SELECT percentile(duration, 50, 95, 99) AS 'Login Latency (s)'
FROM Transaction
WHERE appName = 'HOALedgerIQ_App'
AND request.uri = '/api/auth/login'
SINCE 1 hour ago
TIMESERIES AUTO;
-- Login error rate
SELECT percentage(count(*), WHERE httpResponseCode >= 400) AS 'Login Error %'
FROM Transaction
WHERE appName = 'HOALedgerIQ_App'
AND request.uri = '/api/auth/login'
SINCE 1 hour ago
TIMESERIES AUTO;
-- Token refresh latency
SELECT percentile(duration, 50, 95, 99) AS 'Refresh Latency (s)'
FROM Transaction
WHERE appName = 'HOALedgerIQ_App'
AND request.uri = '/api/auth/refresh'
SINCE 1 hour ago
TIMESERIES AUTO;
-- Auth endpoints overview
SELECT count(*), average(duration), percentile(duration, 95)
FROM Transaction
WHERE appName = 'HOALedgerIQ_App'
AND request.uri LIKE '/api/auth/%'
FACET request.uri
SINCE 1 hour ago;
-- ---------------------------------------------------------------------------
-- 3. DASHBOARD & REPORTS
-- ---------------------------------------------------------------------------
-- Dashboard KPI latency
SELECT percentile(duration, 50, 95, 99) AS 'Dashboard Latency (s)'
FROM Transaction
WHERE appName = 'HOALedgerIQ_App'
AND request.uri = '/api/reports/dashboard'
SINCE 1 hour ago
TIMESERIES AUTO;
-- All report endpoints performance
SELECT count(*), average(duration), percentile(duration, 95)
FROM Transaction
WHERE appName = 'HOALedgerIQ_App'
AND request.uri LIKE '/api/reports/%'
FACET request.uri
SINCE 1 hour ago;
-- Slowest report queries (> 2s)
SELECT count(*)
FROM Transaction
WHERE appName = 'HOALedgerIQ_App'
AND request.uri LIKE '/api/reports/%'
AND duration > 2
FACET request.uri
SINCE 1 hour ago;
-- ---------------------------------------------------------------------------
-- 4. CRUD OPERATIONS
-- ---------------------------------------------------------------------------
-- Units endpoint latency by method
SELECT percentile(duration, 50, 95) AS 'Units Latency'
FROM Transaction
WHERE appName = 'HOALedgerIQ_App'
AND request.uri LIKE '/api/units%'
FACET request.method
SINCE 1 hour ago;
-- Vendors endpoint latency by method
SELECT percentile(duration, 50, 95) AS 'Vendors Latency'
FROM Transaction
WHERE appName = 'HOALedgerIQ_App'
AND request.uri LIKE '/api/vendors%'
FACET request.method
SINCE 1 hour ago;
-- Journal entries performance
SELECT percentile(duration, 50, 95) AS 'JE Latency'
FROM Transaction
WHERE appName = 'HOALedgerIQ_App'
AND request.uri LIKE '/api/journal-entries%'
FACET request.method
SINCE 1 hour ago;
-- Payments performance
SELECT percentile(duration, 50, 95) AS 'Payments Latency'
FROM Transaction
WHERE appName = 'HOALedgerIQ_App'
AND request.uri LIKE '/api/payments%'
FACET request.method
SINCE 1 hour ago;
-- Accounts performance
SELECT percentile(duration, 50, 95) AS 'Accounts Latency'
FROM Transaction
WHERE appName = 'HOALedgerIQ_App'
AND request.uri LIKE '/api/accounts%'
FACET request.method
SINCE 1 hour ago;
-- Invoices performance
SELECT percentile(duration, 50, 95) AS 'Invoices Latency'
FROM Transaction
WHERE appName = 'HOALedgerIQ_App'
AND request.uri LIKE '/api/invoices%'
FACET request.method
SINCE 1 hour ago;
-- ---------------------------------------------------------------------------
-- 5. MULTI-TENANT / ORG OPERATIONS
-- ---------------------------------------------------------------------------
-- Organizations endpoint performance
SELECT percentile(duration, 50, 95)
FROM Transaction
WHERE appName = 'HOALedgerIQ_App'
AND request.uri LIKE '/api/organizations%'
FACET request.method
SINCE 1 hour ago;
-- Board planning (complex module) latency
SELECT count(*), percentile(duration, 50, 95, 99)
FROM Transaction
WHERE appName = 'HOALedgerIQ_App'
AND request.uri LIKE '/api/board-planning%'
FACET request.uri
SINCE 1 hour ago;
-- ---------------------------------------------------------------------------
-- 6. ERROR ANALYSIS
-- ---------------------------------------------------------------------------
-- Error rate by endpoint (top 20 offenders)
SELECT percentage(count(*), WHERE httpResponseCode >= 400) AS 'Error %',
count(*) AS 'Total'
FROM Transaction
WHERE appName = 'HOALedgerIQ_App'
FACET request.uri
SINCE 1 hour ago
LIMIT 20;
-- 5xx errors specifically
SELECT count(*)
FROM Transaction
WHERE appName = 'HOALedgerIQ_App'
AND httpResponseCode >= 500
FACET request.uri, httpResponseCode
SINCE 1 hour ago;
-- Error rate over time
SELECT percentage(count(*), WHERE httpResponseCode >= 500) AS 'Server Error %',
percentage(count(*), WHERE httpResponseCode >= 400 AND httpResponseCode < 500) AS 'Client Error %'
FROM Transaction
WHERE appName = 'HOALedgerIQ_App'
SINCE 1 hour ago
TIMESERIES AUTO;
-- ---------------------------------------------------------------------------
-- 7. DATABASE & EXTERNAL SERVICES
-- ---------------------------------------------------------------------------
-- Database call duration (TypeORM / Postgres)
SELECT average(databaseDuration), percentile(databaseDuration, 95)
FROM Transaction
WHERE appName = 'HOALedgerIQ_App'
SINCE 1 hour ago
TIMESERIES AUTO;
-- Slowest DB transactions
SELECT average(databaseDuration), count(*)
FROM Transaction
WHERE appName = 'HOALedgerIQ_App'
AND databaseDuration > 1
FACET request.uri
SINCE 1 hour ago
LIMIT 20;
-- External service calls (Stripe, Resend, NVIDIA AI)
SELECT average(externalDuration), count(*)
FROM Transaction
WHERE appName = 'HOALedgerIQ_App'
AND externalDuration > 0
FACET request.uri
SINCE 1 hour ago;
-- ---------------------------------------------------------------------------
-- 8. LOAD TEST COMPARISON
-- ---------------------------------------------------------------------------
-- Compare metrics between two time windows (baseline vs test)
-- Adjust SINCE/UNTIL for your test windows
SELECT percentile(duration, 50, 95, 99), count(*), percentage(count(*), WHERE httpResponseCode >= 500)
FROM Transaction
WHERE appName = 'HOALedgerIQ_App'
SINCE 2 hours ago
UNTIL 1 hour ago
COMPARE WITH 1 hour ago;
-- Per-endpoint comparison during load test window
SELECT average(duration), percentile(duration, 95), count(*)
FROM Transaction
WHERE appName = 'HOALedgerIQ_App'
FACET request.uri
SINCE 1 hour ago
LIMIT 50;
-- ---------------------------------------------------------------------------
-- 9. INFRASTRUCTURE (if NR Infrastructure agent is installed)
-- ---------------------------------------------------------------------------
-- CPU utilization
SELECT average(cpuPercent)
FROM SystemSample
WHERE hostname LIKE '%hoaledgeriq%'
SINCE 1 hour ago
TIMESERIES AUTO;
-- Memory utilization
SELECT average(memoryUsedPercent)
FROM SystemSample
WHERE hostname LIKE '%hoaledgeriq%'
SINCE 1 hour ago
TIMESERIES AUTO;
-- Connection pool saturation (custom metric requires NR custom events)
-- SELECT average(custom.db.pool.active), average(custom.db.pool.idle)
-- FROM Metric
-- WHERE appName = 'HOALedgerIQ_App'
-- SINCE 1 hour ago
-- TIMESERIES AUTO;

View File

@@ -0,0 +1,53 @@
{
"staging": {
"baseUrl": "https://staging.hoaledgeriq.com",
"stages": {
"rampUp": 10,
"steady": 25,
"rampDown": 0
},
"thresholds": {
"http_req_duration_p95": 2000,
"login_p95": 1500,
"dashboard_p95": 3000,
"crud_write_p95": 2000,
"crud_read_p95": 1500,
"error_rate": 0.05
},
"notes": "Staging environment smaller infra, relaxed thresholds"
},
"production": {
"baseUrl": "https://app.hoaledgeriq.com",
"stages": {
"rampUp": 20,
"steady": 50,
"rampDown": 0
},
"thresholds": {
"http_req_duration_p95": 1000,
"login_p95": 800,
"dashboard_p95": 1500,
"crud_write_p95": 1000,
"crud_read_p95": 800,
"error_rate": 0.01
},
"notes": "Production thresholds strict SLA targets"
},
"local": {
"baseUrl": "http://localhost:3000",
"stages": {
"rampUp": 5,
"steady": 10,
"rampDown": 0
},
"thresholds": {
"http_req_duration_p95": 3000,
"login_p95": 2000,
"dashboard_p95": 5000,
"crud_write_p95": 3000,
"crud_read_p95": 2000,
"error_rate": 0.10
},
"notes": "Local dev generous thresholds for single-machine testing"
}
}

View File

@@ -0,0 +1,11 @@
email,password,orgId,role
loadtest-treasurer-01@hoaledgeriq.test,LoadTest!Pass01,org-uuid-placeholder-1,treasurer
loadtest-treasurer-02@hoaledgeriq.test,LoadTest!Pass02,org-uuid-placeholder-1,treasurer
loadtest-admin-01@hoaledgeriq.test,LoadTest!Pass03,org-uuid-placeholder-1,admin
loadtest-admin-02@hoaledgeriq.test,LoadTest!Pass04,org-uuid-placeholder-2,admin
loadtest-president-01@hoaledgeriq.test,LoadTest!Pass05,org-uuid-placeholder-2,president
loadtest-manager-01@hoaledgeriq.test,LoadTest!Pass06,org-uuid-placeholder-2,manager
loadtest-member-01@hoaledgeriq.test,LoadTest!Pass07,org-uuid-placeholder-1,member_at_large
loadtest-viewer-01@hoaledgeriq.test,LoadTest!Pass08,org-uuid-placeholder-1,viewer
loadtest-homeowner-01@hoaledgeriq.test,LoadTest!Pass09,org-uuid-placeholder-2,homeowner
loadtest-homeowner-02@hoaledgeriq.test,LoadTest!Pass10,org-uuid-placeholder-2,homeowner
1 email password orgId role
2 loadtest-treasurer-01@hoaledgeriq.test LoadTest!Pass01 org-uuid-placeholder-1 treasurer
3 loadtest-treasurer-02@hoaledgeriq.test LoadTest!Pass02 org-uuid-placeholder-1 treasurer
4 loadtest-admin-01@hoaledgeriq.test LoadTest!Pass03 org-uuid-placeholder-1 admin
5 loadtest-admin-02@hoaledgeriq.test LoadTest!Pass04 org-uuid-placeholder-2 admin
6 loadtest-president-01@hoaledgeriq.test LoadTest!Pass05 org-uuid-placeholder-2 president
7 loadtest-manager-01@hoaledgeriq.test LoadTest!Pass06 org-uuid-placeholder-2 manager
8 loadtest-member-01@hoaledgeriq.test LoadTest!Pass07 org-uuid-placeholder-1 member_at_large
9 loadtest-viewer-01@hoaledgeriq.test LoadTest!Pass08 org-uuid-placeholder-1 viewer
10 loadtest-homeowner-01@hoaledgeriq.test LoadTest!Pass09 org-uuid-placeholder-2 homeowner
11 loadtest-homeowner-02@hoaledgeriq.test LoadTest!Pass10 org-uuid-placeholder-2 homeowner

View File

@@ -0,0 +1,189 @@
import http from 'k6/http';
import { check, group, sleep } from 'k6';
import { SharedArray } from 'k6/data';
import { Counter, Rate, Trend } from 'k6/metrics';
// ---------------------------------------------------------------------------
// Custom metrics
// ---------------------------------------------------------------------------
const loginDuration = new Trend('login_duration', true);
const refreshDuration = new Trend('refresh_duration', true);
const dashboardDuration = new Trend('dashboard_duration', true);
const profileDuration = new Trend('profile_duration', true);
const loginFailures = new Counter('login_failures');
const authErrors = new Rate('auth_error_rate');
// ---------------------------------------------------------------------------
// Test user pool parameterized from CSV
// ---------------------------------------------------------------------------
const users = new SharedArray('users', function () {
const lines = open('../config/user-pool.csv').split('\n').slice(1); // skip header
return lines
.filter((l) => l.trim().length > 0)
.map((line) => {
const [email, password, orgId, role] = line.split(',');
return { email: email.trim(), password: password.trim(), orgId: orgId.trim(), role: role.trim() };
});
});
// ---------------------------------------------------------------------------
// Environment config
// ---------------------------------------------------------------------------
const ENV = JSON.parse(open('../config/environments.json'));
const CONF = ENV[__ENV.TARGET_ENV || 'staging'];
const BASE = CONF.baseUrl;
// ---------------------------------------------------------------------------
// k6 options ramp-up / steady / ramp-down
// ---------------------------------------------------------------------------
export const options = {
scenarios: {
auth_dashboard: {
executor: 'ramping-vus',
startVUs: 0,
stages: [
{ duration: '1m', target: CONF.stages.rampUp }, // ramp-up
{ duration: '5m', target: CONF.stages.steady }, // steady state
{ duration: '1m', target: 0 }, // ramp-down
],
gracefulStop: '30s',
},
},
thresholds: {
http_req_duration: [`p(95)<${CONF.thresholds.http_req_duration_p95}`],
login_duration: [`p(95)<${CONF.thresholds.login_p95}`],
dashboard_duration: [`p(95)<${CONF.thresholds.dashboard_p95}`],
auth_error_rate: [`rate<${CONF.thresholds.error_rate}`],
http_req_failed: [`rate<${CONF.thresholds.error_rate}`],
},
};
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
function authHeaders(accessToken) {
return {
headers: {
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
},
};
}
function jsonPost(url, body, params = {}) {
return http.post(url, JSON.stringify(body), {
headers: { 'Content-Type': 'application/json', ...((params.headers) || {}) },
tags: params.tags || {},
});
}
// ---------------------------------------------------------------------------
// Default VU function login → dashboard journey
// ---------------------------------------------------------------------------
export default function () {
const user = users[__VU % users.length];
let accessToken = null;
// ── 1. Login ─────────────────────────────────────────────────────────
group('01_login', () => {
const res = jsonPost(`${BASE}/api/auth/login`, {
email: user.email,
password: user.password,
}, { tags: { name: 'POST /api/auth/login' } });
loginDuration.add(res.timings.duration);
const ok = check(res, {
'login status 200|201': (r) => r.status === 200 || r.status === 201,
'login returns accessToken': (r) => {
try { return !!JSON.parse(r.body).accessToken; } catch { return false; }
},
});
if (!ok) {
loginFailures.add(1);
authErrors.add(1);
return; // abort journey cannot continue without token
}
authErrors.add(0);
const body = JSON.parse(res.body);
accessToken = body.accessToken;
});
if (!accessToken) return; // guard
sleep(0.5); // think-time between login and dashboard load
// ── 2. Fetch profile ────────────────────────────────────────────────
group('02_profile', () => {
const res = http.get(`${BASE}/api/auth/profile`, authHeaders(accessToken));
profileDuration.add(res.timings.duration);
check(res, {
'profile status 200': (r) => r.status === 200,
});
});
sleep(0.3);
// ── 3. Dashboard KPIs ───────────────────────────────────────────────
group('03_dashboard', () => {
const res = http.get(`${BASE}/api/reports/dashboard`, authHeaders(accessToken));
dashboardDuration.add(res.timings.duration);
check(res, {
'dashboard status 200': (r) => r.status === 200,
});
});
sleep(0.3);
// ── 4. Parallel dashboard widgets (batch) ───────────────────────────
group('04_dashboard_widgets', () => {
const now = new Date();
const year = now.getFullYear();
const fromDate = `${year}-01-01`;
const toDate = now.toISOString().slice(0, 10);
const responses = http.batch([
['GET', `${BASE}/api/accounts?fundType=operating`, null, authHeaders(accessToken)],
['GET', `${BASE}/api/reports/income-statement?from=${fromDate}&to=${toDate}`, null, authHeaders(accessToken)],
['GET', `${BASE}/api/reports/balance-sheet?as_of=${toDate}`, null, authHeaders(accessToken)],
['GET', `${BASE}/api/reports/aging`, null, authHeaders(accessToken)],
['GET', `${BASE}/api/health-scores/latest`, null, authHeaders(accessToken)],
['GET', `${BASE}/api/onboarding/progress`, null, authHeaders(accessToken)],
]);
responses.forEach((res, i) => {
check(res, {
[`widget_${i} status 200`]: (r) => r.status === 200,
});
});
});
sleep(0.5);
// ── 5. Refresh token ───────────────────────────────────────────────
group('05_refresh_token', () => {
const res = http.post(`${BASE}/api/auth/refresh`, null, {
headers: { 'Content-Type': 'application/json' },
tags: { name: 'POST /api/auth/refresh' },
});
refreshDuration.add(res.timings.duration);
check(res, {
'refresh status 200|201': (r) => r.status === 200 || r.status === 201,
});
});
sleep(0.5);
// ── 6. Logout ──────────────────────────────────────────────────────
group('06_logout', () => {
const res = http.post(`${BASE}/api/auth/logout`, null, authHeaders(accessToken));
check(res, {
'logout status 200|201': (r) => r.status === 200 || r.status === 201,
});
});
sleep(1); // pacing between iterations
}

View File

@@ -0,0 +1,377 @@
import http from 'k6/http';
import { check, group, sleep } from 'k6';
import { SharedArray } from 'k6/data';
import { Counter, Rate, Trend } from 'k6/metrics';
// ---------------------------------------------------------------------------
// Custom metrics
// ---------------------------------------------------------------------------
const loginDuration = new Trend('crud_login_duration', true);
const createDuration = new Trend('crud_create_duration', true);
const readDuration = new Trend('crud_read_duration', true);
const updateDuration = new Trend('crud_update_duration', true);
const deleteDuration = new Trend('crud_delete_duration', true);
const listDuration = new Trend('crud_list_duration', true);
const crudErrors = new Rate('crud_error_rate');
// ---------------------------------------------------------------------------
// Test user pool
// ---------------------------------------------------------------------------
const users = new SharedArray('users', function () {
const lines = open('../config/user-pool.csv').split('\n').slice(1);
return lines
.filter((l) => l.trim().length > 0)
.map((line) => {
const [email, password, orgId, role] = line.split(',');
return { email: email.trim(), password: password.trim(), orgId: orgId.trim(), role: role.trim() };
});
});
// ---------------------------------------------------------------------------
// Environment config
// ---------------------------------------------------------------------------
const ENV = JSON.parse(open('../config/environments.json'));
const CONF = ENV[__ENV.TARGET_ENV || 'staging'];
const BASE = CONF.baseUrl;
// ---------------------------------------------------------------------------
// k6 options
// ---------------------------------------------------------------------------
export const options = {
scenarios: {
crud_flow: {
executor: 'ramping-vus',
startVUs: 0,
stages: [
{ duration: '1m', target: CONF.stages.rampUp },
{ duration: '5m', target: CONF.stages.steady },
{ duration: '1m', target: 0 },
],
gracefulStop: '30s',
},
},
thresholds: {
http_req_duration: [`p(95)<${CONF.thresholds.http_req_duration_p95}`],
crud_create_duration: [`p(95)<${CONF.thresholds.crud_write_p95}`],
crud_update_duration: [`p(95)<${CONF.thresholds.crud_write_p95}`],
crud_read_duration: [`p(95)<${CONF.thresholds.crud_read_p95}`],
crud_list_duration: [`p(95)<${CONF.thresholds.crud_read_p95}`],
crud_error_rate: [`rate<${CONF.thresholds.error_rate}`],
http_req_failed: [`rate<${CONF.thresholds.error_rate}`],
},
};
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
function authHeaders(token) {
return {
headers: {
Authorization: `Bearer ${token}`,
'Content-Type': 'application/json',
},
};
}
function jsonPost(url, body, token, tags) {
return http.post(url, JSON.stringify(body), {
headers: {
Authorization: `Bearer ${token}`,
'Content-Type': 'application/json',
},
tags: tags || {},
});
}
function jsonPut(url, body, token, tags) {
return http.put(url, JSON.stringify(body), {
headers: {
Authorization: `Bearer ${token}`,
'Content-Type': 'application/json',
},
tags: tags || {},
});
}
function login(user) {
const res = http.post(
`${BASE}/api/auth/login`,
JSON.stringify({ email: user.email, password: user.password }),
{ headers: { 'Content-Type': 'application/json' }, tags: { name: 'POST /api/auth/login' } },
);
loginDuration.add(res.timings.duration);
if (res.status !== 200 && res.status !== 201) return null;
try { return JSON.parse(res.body).accessToken; } catch { return null; }
}
// ---------------------------------------------------------------------------
// VU function CRUD journey across core entities
// ---------------------------------------------------------------------------
export default function () {
const user = users[__VU % users.length];
// ── 1. Login ─────────────────────────────────────────────────────────
const accessToken = login(user);
if (!accessToken) {
crudErrors.add(1);
return;
}
crudErrors.add(0);
sleep(0.5);
// ── 2. Units CRUD ───────────────────────────────────────────────────
let unitId = null;
group('units_crud', () => {
// List
group('list_units', () => {
const res = http.get(`${BASE}/api/units`, authHeaders(accessToken));
listDuration.add(res.timings.duration);
check(res, { 'list units 200': (r) => r.status === 200 });
});
sleep(0.3);
// Create
group('create_unit', () => {
const payload = {
unitNumber: `LT-${__VU}-${Date.now()}`,
address: `${__VU} Load Test Lane`,
ownerName: `Load Tester ${__VU}`,
ownerEmail: `lt-${__VU}@loadtest.local`,
squareFeet: 1200,
};
const res = jsonPost(`${BASE}/api/units`, payload, accessToken, { name: 'POST /api/units' });
createDuration.add(res.timings.duration);
const ok = check(res, {
'create unit 200|201': (r) => r.status === 200 || r.status === 201,
});
if (ok) {
try { unitId = JSON.parse(res.body).id; } catch { /* noop */ }
}
});
sleep(0.3);
// Read
if (unitId) {
group('read_unit', () => {
const res = http.get(`${BASE}/api/units/${unitId}`, authHeaders(accessToken));
readDuration.add(res.timings.duration);
check(res, { 'read unit 200': (r) => r.status === 200 });
});
sleep(0.3);
// Update
group('update_unit', () => {
const res = jsonPut(`${BASE}/api/units/${unitId}`, {
ownerName: `Updated Tester ${__VU}`,
}, accessToken, { name: 'PUT /api/units/:id' });
updateDuration.add(res.timings.duration);
check(res, { 'update unit 200': (r) => r.status === 200 });
});
sleep(0.3);
// Delete
group('delete_unit', () => {
const res = http.del(`${BASE}/api/units/${unitId}`, null, authHeaders(accessToken));
deleteDuration.add(res.timings.duration);
check(res, { 'delete unit 200': (r) => r.status === 200 });
});
}
});
sleep(0.5);
// ── 3. Vendors CRUD ─────────────────────────────────────────────────
let vendorId = null;
group('vendors_crud', () => {
group('list_vendors', () => {
const res = http.get(`${BASE}/api/vendors`, authHeaders(accessToken));
listDuration.add(res.timings.duration);
check(res, { 'list vendors 200': (r) => r.status === 200 });
});
sleep(0.3);
group('create_vendor', () => {
const payload = {
name: `LT Vendor ${__VU}-${Date.now()}`,
email: `vendor-${__VU}@loadtest.local`,
phone: '555-0100',
category: 'maintenance',
};
const res = jsonPost(`${BASE}/api/vendors`, payload, accessToken, { name: 'POST /api/vendors' });
createDuration.add(res.timings.duration);
const ok = check(res, {
'create vendor 200|201': (r) => r.status === 200 || r.status === 201,
});
if (ok) {
try { vendorId = JSON.parse(res.body).id; } catch { /* noop */ }
}
});
sleep(0.3);
if (vendorId) {
group('read_vendor', () => {
const res = http.get(`${BASE}/api/vendors/${vendorId}`, authHeaders(accessToken));
readDuration.add(res.timings.duration);
check(res, { 'read vendor 200': (r) => r.status === 200 });
});
sleep(0.3);
group('update_vendor', () => {
const res = jsonPut(`${BASE}/api/vendors/${vendorId}`, {
name: `Updated Vendor ${__VU}`,
}, accessToken, { name: 'PUT /api/vendors/:id' });
updateDuration.add(res.timings.duration);
check(res, { 'update vendor 200': (r) => r.status === 200 });
});
}
});
sleep(0.5);
// ── 4. Journal Entries workflow ─────────────────────────────────────
let journalEntryId = null;
group('journal_entries', () => {
group('list_journal_entries', () => {
const res = http.get(`${BASE}/api/journal-entries`, authHeaders(accessToken));
listDuration.add(res.timings.duration);
check(res, { 'list JE 200': (r) => r.status === 200 });
});
sleep(0.3);
// Fetch accounts first so we can build a valid entry
let accounts = [];
group('fetch_accounts_for_je', () => {
const res = http.get(`${BASE}/api/accounts`, authHeaders(accessToken));
check(res, { 'list accounts 200': (r) => r.status === 200 });
try { accounts = JSON.parse(res.body); } catch { /* noop */ }
});
sleep(0.3);
if (Array.isArray(accounts) && accounts.length >= 2) {
group('create_journal_entry', () => {
const payload = {
date: new Date().toISOString().slice(0, 10),
memo: `Load test JE VU-${__VU}`,
type: 'standard',
lines: [
{ accountId: accounts[0].id, debit: 100, credit: 0, memo: 'debit leg' },
{ accountId: accounts[1].id, debit: 0, credit: 100, memo: 'credit leg' },
],
};
const res = jsonPost(`${BASE}/api/journal-entries`, payload, accessToken, { name: 'POST /api/journal-entries' });
createDuration.add(res.timings.duration);
const ok = check(res, {
'create JE 200|201': (r) => r.status === 200 || r.status === 201,
});
if (ok) {
try { journalEntryId = JSON.parse(res.body).id; } catch { /* noop */ }
}
});
sleep(0.3);
if (journalEntryId) {
group('read_journal_entry', () => {
const res = http.get(`${BASE}/api/journal-entries/${journalEntryId}`, authHeaders(accessToken));
readDuration.add(res.timings.duration);
check(res, { 'read JE 200': (r) => r.status === 200 });
});
sleep(0.3);
// Post (finalize) the journal entry
group('post_journal_entry', () => {
const res = http.post(`${BASE}/api/journal-entries/${journalEntryId}/post`, null, authHeaders(accessToken));
updateDuration.add(res.timings.duration);
check(res, { 'post JE 200': (r) => r.status === 200 });
});
sleep(0.3);
// Void the journal entry (cleanup)
group('void_journal_entry', () => {
const res = http.post(`${BASE}/api/journal-entries/${journalEntryId}/void`, null, authHeaders(accessToken));
deleteDuration.add(res.timings.duration);
check(res, { 'void JE 200': (r) => r.status === 200 });
});
}
}
});
sleep(0.5);
// ── 5. Payments CRUD ────────────────────────────────────────────────
let paymentId = null;
group('payments_crud', () => {
group('list_payments', () => {
const res = http.get(`${BASE}/api/payments`, authHeaders(accessToken));
listDuration.add(res.timings.duration);
check(res, { 'list payments 200': (r) => r.status === 200 });
});
sleep(0.3);
group('create_payment', () => {
const payload = {
amount: 150.00,
date: new Date().toISOString().slice(0, 10),
method: 'check',
memo: `Load test payment VU-${__VU}`,
};
const res = jsonPost(`${BASE}/api/payments`, payload, accessToken, { name: 'POST /api/payments' });
createDuration.add(res.timings.duration);
const ok = check(res, {
'create payment 200|201': (r) => r.status === 200 || r.status === 201,
});
if (ok) {
try { paymentId = JSON.parse(res.body).id; } catch { /* noop */ }
}
});
sleep(0.3);
if (paymentId) {
group('read_payment', () => {
const res = http.get(`${BASE}/api/payments/${paymentId}`, authHeaders(accessToken));
readDuration.add(res.timings.duration);
check(res, { 'read payment 200': (r) => r.status === 200 });
});
sleep(0.3);
group('update_payment', () => {
const res = jsonPut(`${BASE}/api/payments/${paymentId}`, {
memo: `Updated payment VU-${__VU}`,
}, accessToken, { name: 'PUT /api/payments/:id' });
updateDuration.add(res.timings.duration);
check(res, { 'update payment 200': (r) => r.status === 200 });
});
sleep(0.3);
group('delete_payment', () => {
const res = http.del(`${BASE}/api/payments/${paymentId}`, null, authHeaders(accessToken));
deleteDuration.add(res.timings.duration);
check(res, { 'delete payment 200': (r) => r.status === 200 });
});
}
});
sleep(0.5);
// ── 6. Reports (read-heavy) ─────────────────────────────────────────
group('reports_read', () => {
const now = new Date();
const year = now.getFullYear();
const toDate = now.toISOString().slice(0, 10);
const fromDate = `${year}-01-01`;
const responses = http.batch([
['GET', `${BASE}/api/reports/balance-sheet?as_of=${toDate}`, null, authHeaders(accessToken)],
['GET', `${BASE}/api/reports/income-statement?from=${fromDate}&to=${toDate}`, null, authHeaders(accessToken)],
['GET', `${BASE}/api/reports/aging`, null, authHeaders(accessToken)],
['GET', `${BASE}/api/reports/cash-flow?from=${fromDate}&to=${toDate}`, null, authHeaders(accessToken)],
['GET', `${BASE}/api/accounts/trial-balance?asOfDate=${toDate}`, null, authHeaders(accessToken)],
]);
responses.forEach((res, i) => {
readDuration.add(res.timings.duration);
check(res, { [`report_${i} status 200`]: (r) => r.status === 200 });
});
});
sleep(1); // pacing
}