fix: Improved duplicate prevention in cast iron scanner

- Better link normalization and checking
- Skip items already in seen_links with logging
- Clean up state file to last 500 items
- Always mark items as seen (deal or not)
- Added logging for skipped duplicates

Also: eBay scraping temporarily blocked/changed - investigating

agents/sales-lead/monitor.log

@@ -2325,3 +2325,35 @@
 [2026-04-10T15:18:09Z] ROI Calc submissions response: 7791 bytes
 [2026-04-10T15:18:09Z] Processing calc submissions...
 [2026-04-10T15:18:09Z] Check complete. Next run at 2026-04-10T12:18:EDT
+[2026-04-10T16:00:00Z] Starting lead monitor check
+[2026-04-10T16:00:02Z] ROI Calc submissions response: 7791 bytes
+[2026-04-10T16:00:02Z] Processing calc submissions...
+[2026-04-10T16:00:02Z] Check complete. Next run at 2026-04-10T13:00:EDT
+[2026-04-10T16:19:22Z] Starting lead monitor check
+[2026-04-10T16:19:24Z] ROI Calc submissions response: 7791 bytes
+[2026-04-10T16:19:24Z] Processing calc submissions...
+[2026-04-10T16:19:24Z] Check complete. Next run at 2026-04-10T13:19:EDT
+[2026-04-10T17:00:00Z] Starting lead monitor check
+[2026-04-10T17:00:02Z] ROI Calc submissions response: 7791 bytes
+[2026-04-10T17:00:02Z] Processing calc submissions...
+[2026-04-10T17:00:02Z] Check complete. Next run at 2026-04-10T14:00:EDT
+[2026-04-10T18:00:00Z] Starting lead monitor check
+[2026-04-10T18:00:02Z] ROI Calc submissions response: 7791 bytes
+[2026-04-10T18:00:02Z] Processing calc submissions...
+[2026-04-10T18:00:02Z] Check complete. Next run at 2026-04-10T15:00:EDT
+[2026-04-10T18:15:34Z] Starting lead monitor check
+[2026-04-10T18:15:36Z] ROI Calc submissions response: 7791 bytes
+[2026-04-10T18:15:36Z] Processing calc submissions...
+[2026-04-10T18:15:36Z] Check complete. Next run at 2026-04-10T15:15:EDT
+[2026-04-10T19:00:01Z] Starting lead monitor check
+[2026-04-10T19:00:02Z] ROI Calc submissions response: 7791 bytes
+[2026-04-10T19:00:02Z] Processing calc submissions...
+[2026-04-10T19:00:02Z] Check complete. Next run at 2026-04-10T16:00:EDT
+[2026-04-10T19:24:10Z] Starting lead monitor check
+[2026-04-10T19:24:12Z] ROI Calc submissions response: 7791 bytes
+[2026-04-10T19:24:12Z] Processing calc submissions...
+[2026-04-10T19:24:12Z] Check complete. Next run at 2026-04-10T16:24:EDT
+[2026-04-10T20:00:00Z] Starting lead monitor check
+[2026-04-10T20:00:01Z] ROI Calc submissions response: 7791 bytes
+[2026-04-10T20:00:01Z] Processing calc submissions...
+[2026-04-10T20:00:01Z] Check complete. Next run at 2026-04-10T17:00:EDT