fix: Improved duplicate prevention in cast iron scanner

- Better link normalization and checking
- Skip items already in seen_links with logging
- Clean up state file to last 500 items
- Always mark items as seen (deal or not)
- Added logging for skipped duplicates

Also: eBay scraping temporarily blocked/changed - investigating
This commit is contained in:
2026-04-10 16:16:35 -04:00
parent 30703bfd45
commit 4bd829ca8c
9 changed files with 573 additions and 1002 deletions

View File

@@ -1,9 +1,9 @@
# Self-Improving Heartbeat State
last_heartbeat_started_at: 2026-04-10T15:47:00Z
last_heartbeat_started_at: 2026-04-10T18:41:00Z
last_reviewed_change_at: 2026-03-26T12:20:00Z
last_heartbeat_result: HEARTBEAT_OK
## Last actions
- 2026-04-10 15:47Z: Heartbeat check - no changes in self-improving files since last review
- 2026-04-10 18:41Z: Heartbeat check - no changes in self-improving files since last review
- Sales-lead agent: ✅ Cron executed at 04:17 AM, 3 leads detected (john@example.com, jane@example123.com, smith@example.com)
- Marketing-content agent: ✅ Today's 9:00 AM run completed successfully. No new content produced since last check.