fix: improve AI health score accuracy and consistency #1

JoeBot · 2026-03-06T12:44:54-05:00

JoeBot commented

2026-03-06 12:44:54 -05:00

Summary

Reduce AI temperature from 0.3 → 0.1 for health score calculations to eliminate 16-40 point score volatility across consecutive runs
Add strict cash runway classification rules to the operating prompt, preventing the model from rating sub-3-month cash runway as "positive" (was inflating scores ~10-15 points)
Pre-compute total special assessment income in both operating and reserve prompts, eliminating the per-unit vs total confusion ($300/unit mistaken for $300 total instead of $20,100)
Make YTD budget comparisons actuals-aware: only compare months with posted journal entries, show current month budget separately, and add prompt guidance about month-end posting cadence

Full audit details in docs/AI_FEATURE_AUDIT.md.

Test plan

Rebuild backend and verify it starts without errors
Trigger operating health score recalculation — verify score is closer to 72-80 (not 88) given 2.4-month runway
Verify YTD section shows "through January" (last month with actuals) not "through March"
Verify special assessment totals are pre-computed in prompt data
Run 3 consecutive health score calculations and verify scores are within ±5 points of each other
Verify reserve health score stays in 40-50 range

🤖 Generated with Claude Code

## Summary - **Reduce AI temperature** from 0.3 → 0.1 for health score calculations to eliminate 16-40 point score volatility across consecutive runs - **Add strict cash runway classification rules** to the operating prompt, preventing the model from rating sub-3-month cash runway as "positive" (was inflating scores ~10-15 points) - **Pre-compute total special assessment income** in both operating and reserve prompts, eliminating the per-unit vs total confusion ($300/unit mistaken for $300 total instead of $20,100) - **Make YTD budget comparisons actuals-aware**: only compare months with posted journal entries, show current month budget separately, and add prompt guidance about month-end posting cadence Full audit details in `docs/AI_FEATURE_AUDIT.md`. ## Test plan - [ ] Rebuild backend and verify it starts without errors - [ ] Trigger operating health score recalculation — verify score is closer to 72-80 (not 88) given 2.4-month runway - [ ] Verify YTD section shows "through January" (last month with actuals) not "through March" - [ ] Verify special assessment totals are pre-computed in prompt data - [ ] Run 3 consecutive health score calculations and verify scores are within ±5 points of each other - [ ] Verify reserve health score stays in 40-50 range 🤖 Generated with [Claude Code](https://claude.com/claude-code)

JoeBot added 1 commit 2026-03-06 12:44:55 -05:00

fix: improve AI health score accuracy and consistency 07d15001ae

Address 4 issues identified in AI feature audit:

1. Reduce temperature from 0.3 to 0.1 for health score calculations
   to reduce 16-40 point score volatility across runs

2. Add explicit cash runway classification rules to operating prompt
   preventing the model from rating sub-3-month runway as "positive"

3. Pre-compute total special assessment income in both operating and
   reserve prompts, eliminating per-unit vs total confusion ($300
   vs $20,100)

4. Make YTD budget comparison actuals-aware: only compare months with
   posted journal entries, show current month budget separately, and
   add prompt guidance about month-end posting cadence

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

JoeBot added 1 commit 2026-03-06 14:43:00 -05:00

feat: async AI calls, 10-min timeout, and failure messaging 9146118df1

- Make all AI endpoints (health scores + investment recommendations)
  fire-and-forget: POST returns immediately, frontend polls for results
- Extend AI API timeout from 2-5 min to 10 min for both services
- Add "last analysis failed — showing cached data" message to the
  Investment Recommendations panel (matches health score widgets)
- Add status/error_message columns to ai_recommendations table
- Remove nginx AI timeout overrides (no longer needed)
- Users can now navigate away during AI processing without interruption

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

JoeBot merged commit c429dcc033 into main

2026-03-06 14:44:40 -05:00

JoeBot referenced this issue from a commit

2026-03-06 14:44:42 -05:00

Merge pull request 'fix: improve AI health score accuracy and consistency' (#1) from ai-improvements into main

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: JoeBot/HOA_Financial_Platform#1