Why Most Daily AML Batch Pipelines Crumble at 10 Million Records – My 8‑Step Rescue
Photo by Mitchell Luo on Unsplash High‑Stakes Hook It was 02:13 am on a humid Thursday. My Slack pinged: "ALERT: 10 M transactions flagged in the last 24 h – SAR backlog at 98%" . The BFIU had just sent a surprise audit notice, demanding evidence of compliance for the past week. My team stared at a spreadsheet that looked like a city map of Dhaka – every cell a potential structuring ring. If we didn’t tame the flood, we’d face a BDT 25 million penalty and a possible licence suspension. The Hidden Problem Most fintechs in Bangladesh treat daily AML batch jobs like a nightly cron that “just works”. They copy a template from a vendor, slap a SELECT * FROM transactions into it, and hope the scheduler will magically sort the rest. In reality, three hidden forces grind those pipelines to a halt: Volume spikes – bKash’s promo days can double daily volume overnight. Regulatory quirks – BFIU’s 100 K BDT MFS threshold must be enforced per‑customer, per‑day, not just per‑transaction...