The following scenario is a composite illustration; the figures are drawn from documented BSA/AML program patterns and do not reflect a specific institution or examination. A compliance director at a $9 billion regional bank stands in front of her BSA (Bank Secrecy Act) committee and delivers a number that sounds like a win: the institution's new AI-assisted transaction monitoring system has reduced the monthly alert volume from 2,800 to 1,100, a reduction of 61 percent. Analyst headcount stayed flat. Cost per investigation dropped. The board wants a press release. The problem surfaces eight months later, during a targeted examination by the OCC (Office of the Comptroller of the Currency), when an examiner requests the institution's SAR (Suspicious Activity Report) filing history for the prior 18 months and begins comparing it against the alert volume chart. The filing rate has dropped by 48 percent since the AI system was deployed. The examiner asks a straightforward question: "Did you also reduce the number of suspicious transactions at your institution by 48 percent?" The answer is no. The alert reduction was real. The suspicious activity reduction was not. The institution had tuned its model to cut false positives aggressively, and in doing so had begun suppressing the true positives that the program existed to find. This is the most consequential risk in AI-assisted BSA/AML: the tuning failure that no one notices until an examination, because the alert queue looks cleaner and the analysts look less stressed and the metrics look like success. This lesson is about how to tune AML (Anti-Money Laundering) models to actually reduce the false-positive burden without quietly raising the true-positive miss rate, how to measure whether the balance is right, and what examiners look for when they evaluate a bank that has made this trade-off.

Understanding the False-Positive Problem and Why It Persists

The 90 to 95 percent false-positive rate in BSA/AML transaction monitoring is not an accident or a product of lazy scenario design. It is a structural consequence of the nature of money laundering and the nature of legitimate banking behavior. Cash-intensive small businesses, remittance senders, international students, seasonal workers, and cryptocurrency participants all engage in transaction patterns that overlap substantially with the patterns of structuring, layering, and placement that the monitoring rules are designed to detect. A rule-based system that catches financial criminals will also catch a large number of people doing nothing wrong, because the behavioral signatures of the two groups are similar.

The consequence of a persistently high false-positive rate is not merely an operational inconvenience. It is a program quality crisis. An analyst who spends the first six hours of an eight-hour day reviewing and dismissing false positives has two hours left for the cases that actually require investigation. If the institution files 20 SARs per month from a queue of 2,000 alerts, and the analysts have meaningful bandwidth to investigate only 400 of them, the institution is not monitoring its transactions. It is monitoring the transactions that happen to appear in the first 400 slots of the queue. The 20 true positives might be in those 400 slots, or they might be in slots 401 through 2,000. Without intelligent prioritization and tuning, the institution has no way to know.

AI-assisted tuning addresses this problem by replacing the one-size-fits-all threshold approach of traditional rule-based monitoring with a more differentiated approach that calibrates alert sensitivity to the actual risk profile of each customer segment, transaction type, and behavioral pattern. Instead of applying the same $9,800 cash deposit threshold to a coin dealer, a restaurant owner, and a currency exchange, a tuned model applies different effective sensitivity levels based on the historical risk experience of each profile, reducing false positives for low-risk patterns while maintaining or increasing sensitivity for high-risk ones.

The risk in tuning, as the compliance director's story illustrates, is that the metrics most visible to leadership (alert volume, cost per investigation, analyst workload) all move in a favorable direction when the model is tuned aggressively, regardless of whether the tuning is improving the program or degrading it. Alert volume falls. Analyst stress falls. Investigation costs fall. But if the tuning is suppressing true positives alongside false positives, the BSA program is systematically failing to find the suspicious activity it is required to detect and report. The failure is invisible until an examiner compares the filing rate trend against external evidence of financial crime activity in the institution's market.

The Two Error Types and Why Their Costs Are Not Symmetric

Tuning a BSA/AML model involves trading off between two types of errors: false positives (alerts generated for legitimate activity) and false negatives (suspicious activity that the model does not flag at all). Every change to a model's sensitivity settings affects both error types simultaneously: lowering sensitivity reduces false positives but also reduces true positives, potentially increasing false negatives; raising sensitivity increases true positives but also increases false positives, raising analyst workload.

The critical insight for BSA/AML governance is that these two error types do not have symmetric costs. A false positive costs the institution analyst time: the analyst investigates the alert, confirms it is not suspicious, closes it, and moves on. The direct cost is one to four hours of analyst labor. The regulatory cost is zero unless the pattern of false positives is so severe that it crowds out meaningful investigation of true positives, at which point the false-positive burden becomes a program quality finding.

A false negative, by contrast, creates the possibility that suspicious activity goes undetected and unreported. The BSA requires institutions to file SARs when they have reason to suspect financial crime. An institution that misses a SAR filing because its model failed to generate an alert for genuinely suspicious activity has a BSA program failure. Depending on the activity missed and the pattern of misses, the regulatory consequences range from examination findings and remediation orders to civil money penalties and, in extreme cases, criminal referral for willful BSA violations. The financial crime enabled by the missed reporting, whether drug trafficking, human trafficking, or sanctions evasion, is a harm that goes beyond the institution's own regulatory exposure.

This asymmetry means that institutions tuning BSA/AML models should never optimize purely for false-positive reduction. The correct optimization target is a reduction in false positives while holding the false-negative rate constant or improving it. If a tuning change reduces false positives by 30 percent and holds the true-positive detection rate steady, it is a success. If a tuning change reduces false positives by 60 percent but allows 20 percent of the previously detected true positives to fall below the investigation threshold, it is a program degradation that looks like an efficiency win.

The Precision-Recall Trade-Off Explained for BSA Analysts

The statistical framework that captures this trade-off is precision and recall, terms that data scientists use and that BSA analysts need to understand in operational terms. Precision is the fraction of alerts that turn out to be true positives: if the model generates 100 alerts and 10 of them result in SAR filings, the precision is 10 percent. Recall is the fraction of all actual true positives that were captured in the alert queue: if there are 10 truly suspicious cases in the portfolio this month and the model flagged 8 of them (with 2 falling below the alert threshold and never being investigated), the recall is 80 percent.

A high-precision model generates alerts that are frequently true positives (less analyst waste) but may miss some true positives by setting thresholds too high. A high-recall model catches most true positives but generates many false positives (more analyst work). The BSA/AML context demands high recall above all: the institution must not miss the real suspicious activity. Precision improvements (reducing false positives) are desirable but cannot come at the cost of recall.

The governance principle that follows from this is that any model tuning change that improves precision (reduces false positives) must be tested for its impact on recall (true-positive capture rate) before deployment. A precision improvement that degrades recall is not a BSA program improvement. It is a BSA program degradation with a favorable-looking cost metric attached.

Tuning Methods: What Actually Changes When You Adjust a Model

BSA/AML model tuning can operate at several levels, from simple threshold adjustments to sophisticated customer-segment-based calibration. Understanding what each type of tuning changes is essential for assessing its impact on both precision and recall.

Threshold tuning is the simplest and most common form. A rule-based monitoring scenario generates an alert when the monitored variable (cash deposit total, wire volume, structuring pattern) meets or exceeds a threshold. Raising the threshold reduces false positives (legitimate customers who were close to but not at the old threshold no longer generate alerts) but also risks suppressing true positives if the real suspicious activity happens to cluster just above the new threshold. The question to answer before raising any threshold is: where do the historical true positives (cases that resulted in SAR filings) cluster in the distribution of the monitored variable? If most true positives are well above the proposed new threshold, raising it is relatively safe. If many true positives are clustered near the proposed threshold, raising it will directly reduce true-positive detection.

Segmentation tuning applies different thresholds or scenario parameters to different customer segments based on their risk profiles. A coin dealer who regularly transacts in cash at volumes that would trigger a structuring alert for a wage-earner presents different risk than an individual with no documented cash-based business who exhibits the same transaction pattern. Segmentation-based tuning acknowledges this by applying higher thresholds to known cash-intensive business types while maintaining tighter thresholds for segments where the same transaction pattern is less explainable. Segmentation tuning requires careful documentation of which customers fall into which segments, clear criteria for segment assignment, and validation that the segments are being assigned correctly to prevent gaming and to justify the differentiated thresholds to an examiner.

Behavioral baseline tuning is used in more sophisticated AI models that generate alerts based on deviation from a customer's own historical baseline rather than absolute thresholds. A customer who normally deposits $20,000 per month in cash will not trigger a velocity alert at $22,000 under a baseline model, even though $22,000 would trigger an alert for a customer whose baseline is $5,000. Baseline models can significantly reduce false positives for known cash-intensive businesses while maintaining sensitivity for customers whose behavior changes unexpectedly. The tuning challenge with baseline models is drift: the model's estimate of a customer's baseline changes over time as new transactions are incorporated, and a gradual escalation of suspicious activity can cause the baseline to drift upward in a way that perpetually keeps the current activity just within the normal range.

Feature weight tuning in machine-learning scoring models adjusts the relative importance of different input features in the model's scoring function. If a risk analyst observes that alerts on accounts in certain industry codes are generating a high false-positive rate while adding little true-positive value, the model can be retrained with lower weight on that feature for those industry codes. Feature weight tuning is more surgical than threshold tuning but also more opaque: it is harder to explain to an examiner why one industry code is weighted differently than another without a clear, documented risk rationale.

Testing Tuning Changes Before Deployment

No tuning change should go live in a production BSA/AML model without a documented test that measures the change's impact on both precision and recall using historical case data. This is not a best practice. Under OCC Bulletin 2026-13, which superseded OCC 2011-12 in April 2026 and applies explicitly to AI and machine learning models used in BSA/AML programs, model changes require validation that is documented in the institution's model-risk management framework. A threshold change applied to a production BSA model without documented testing and validation is an OCC 2026-13 governance failure.

The standard test for a proposed tuning change is a retrospective simulation: applying the proposed new thresholds or model parameters to the historical transaction data from the past 12 to 24 months and counting how many of the historical SAR filings would still have generated alerts under the new configuration. If the institution filed 240 SARs over the past 24 months and the proposed tuning would have eliminated alerts on 35 of those cases (meaning the SAR would not have been filed because the alert would not have been generated), the tuning change has a 14.6 percent historical true-positive suppression rate. That number must be disclosed to the BSA compliance committee and weighed against the false-positive reduction the same tuning achieves.

The 14.6 percent figure is not automatically disqualifying. A tuning change that reduces the false-positive burden by 70 percent while suppressing 14.6 percent of historical true positives might be net positive for program quality if the suppressed cases were low-priority filings and the freed analyst capacity is redirected to higher-quality investigation. But the decision to accept that trade-off must be made consciously, documented explicitly, and approved by the appropriate governance body. The institution cannot make that choice by accident, and it cannot discover it has made that choice retrospectively during an examination.

For new AI models (rather than adjustments to existing rule-based systems), the testing framework is more involved. Model validation under OCC Bulletin 2026-13 requires that the institution confirm the model performs as intended across the full range of customer types and transaction patterns in the institution's portfolio. This includes testing against held-out case data that was not used in training, testing against cases from customer segments that may be underrepresented in the training data, and testing the model's performance at different threshold settings to produce a calibration curve that shows the precision-recall trade-off explicitly. The calibration curve allows governance decision-makers to choose a threshold setting that achieves an acceptable balance between false-positive reduction and true-positive retention, with documentation of why the chosen setting is appropriate for the institution's risk profile.

What Examiners Watch: The Metrics That Reveal Tuning Failures

The opening scenario in this lesson, the compliance director's 61 percent alert reduction followed by a 48 percent SAR filing reduction, represents exactly the pattern examiners are trained to look for. The OCC, FDIC, Federal Reserve, and FinCEN have all published examination guidance that explicitly identifies declining SAR filing rates as a potential indicator of BSA program degradation, particularly when the decline follows a model change or system deployment. Examiners will not simply accept an institution's explanation that improved AI reduced false positives. They will look at the data.

The primary metrics examiners evaluate in a BSA program that uses AI-assisted monitoring include the following.

SAR filing rate trends. An institution that was filing 25 SARs per month before an AI deployment and is filing 12 per month after it needs to explain the difference. The explanation must be grounded in evidence: either the institution's transaction volume declined, the mix of customers changed, or external intelligence indicates that financial crime activity in the institution's markets declined. "We reduced false positives" is not an explanation for a SAR filing rate decline; it is a description of false-positive reduction that does not address whether true positives were also reduced.

Coverage of known typologies. Examiners consult FinCEN typology guidance and law enforcement intelligence to identify the financial crime patterns prevalent in the institution's geography and customer segments. They then check whether the institution's monitoring system generates alerts on transactions that match those typologies. An institution whose AI model has been tuned in ways that systematically underweight known high-risk patterns will fail this coverage test. For example, if FinCEN has issued an advisory on healthcare billing fraud as a money laundering vehicle, an examiner at a bank with a large healthcare provider customer base will check whether the institution's model is configured to flag anomalies consistent with that typology.

Alert investigation quality. Examiners review a sample of closed false-positive alerts to confirm that the dismissal rationale is documented and defensible. If the analyst noted "reviewed, no issues" with no further analysis on a complex case, the examiner will find that inadequate. If AI-assisted triage led the analyst to spend 90 seconds on a case that warranted a 90-minute investigation because the model scored it low, the examination will identify that as a program quality failure.

Explainability of model scores. Under OCC Bulletin 2026-13, an institution must be able to explain how its AI models make scoring decisions. If an examiner asks why a specific account that later appeared in a law enforcement investigation was scored in the bottom 20 percent of the alert queue and never investigated, the institution must be able to produce an explanation: what features drove the low score, whether those features were accurate for the account, and whether the model's training data included cases similar to that account type. "The model gave it a low score" is not sufficient. The institution must be able to explain why.

Disparity analysis across customer segments. Examiners at institutions with diverse customer populations will check whether AI models produce systematically higher false-positive rates or systematically lower detection rates for specific customer segments, including segments that may overlap with protected classes. This is not purely a fair-lending question in BSA/AML: it is a program quality question. A model that generates 40 false positives per true positive for Latino-owned small businesses while generating 8 false positives per true positive for otherwise comparable Anglo-owned businesses is a model with a significant calibration problem that the institution should have identified and addressed in validation.

Building a Tuning Governance Program: The Right Structure

The compliance director in the opening scenario did not set out to degrade her institution's BSA program. She made a series of individually reasonable-seeming decisions (reduce false positives, cut costs, improve analyst efficiency) without a governance structure that required those decisions to be tested for their impact on true-positive detection. A tuning governance program is the structure that prevents that failure mode.

An effective tuning governance program for BSA/AML AI has four components:

Pre-deployment testing with documented true-positive impact analysis. Any proposed change to a monitoring model, whether a threshold adjustment, a scenario parameter change, or a model retrain, must be tested against historical case data before deployment. The test must produce a quantified estimate of the true-positive impact: how many historical SAR filings would have been suppressed by the proposed change? That number, along with the estimated false-positive reduction, must be presented to the BSA compliance committee before the change is approved. The committee approves or rejects the change based on the documented trade-off, and the approval is recorded in the model-risk file.

Post-deployment monitoring with trigger thresholds. After a tuning change is deployed, the institution should monitor SAR filing rates, alert dismissal rates by case type, and analyst override rates on a monthly basis. Threshold triggers should be defined in advance: if the SAR filing rate falls by more than 15 percent relative to the pre-change baseline without an identifiable external explanation, the governance program triggers a review of whether the tuning change has affected true-positive detection. If the filing rate for a specific SAR activity category (structuring, layering, fraud) falls by more than a defined percentage, that category receives targeted analysis. These triggers prevent the institution from drifting into the compliance director's situation without noticing.

Model validation on a defined cadence. Under OCC Bulletin 2026-13, AI models require periodic revalidation, not just initial validation at deployment. For BSA/AML scoring models, the revalidation cadence should be at least annual, and more frequently if the institution's customer mix, geographic footprint, or business strategy changes materially. Revalidation should include the same precision-recall analysis used in initial validation, applied to the most recent 12 months of case data, to confirm that model performance has not drifted from its initial calibration. It should also include segment-level analysis to identify any emerging disparity in model performance across customer types.

Documentation that reconstructs the governance history. The model-risk file for each BSA/AML AI component should be maintained as a living record that captures every tuning change made to the model, the testing performed before the change, the committee approval, and the post-deployment performance monitoring results. An examiner who asks "what changes have been made to this model in the past two years and what was the governance process for each change?" should be able to receive a complete and accurate answer from the model-risk file. An institution that cannot reconstruct its tuning history is an institution that cannot demonstrate to an examiner that its BSA program changes were governed appropriately.

The Human Oversight Layer: Why Governance Alone Is Not Enough

Governance frameworks, testing requirements, and monitoring metrics are necessary but not sufficient for managing the tuning trade-off. They work only if the humans who implement and oversee them have the BSA expertise to interpret what the metrics are saying and the institutional authority to act on that interpretation.

The compliance director in the opening scenario had access to the same metrics as everyone else. What she lacked was a structured decision process that required her to ask, before approving the AI deployment: "What happens to our SAR filing rate if this reduction in alert volume is not purely a reduction in false positives?" That question is the analyst's question, not the engineer's question. It requires the person reviewing the tuning to understand the difference between a false-positive rate and a true-positive rate, to know that a 61 percent alert reduction should produce a much smaller SAR filing rate reduction if the model is working correctly, and to recognize a 48 percent SAR decline as evidence of a problem rather than a benefit.

BSA compliance committees that oversee AI-assisted monitoring programs need members who understand the model-risk framework well enough to ask the right questions of the data scientists and vendors presenting tuning proposals. They need to know that "we reduced alerts by 60 percent" is a precision statement (fewer false positives per alert), not a recall statement (we are still finding the same true positives we were finding before). They need to be comfortable asking: "Show me the retrospective simulation. How many historical SARs would we have missed with this new configuration?"

Under OCC Bulletin 2026-13, the institution's model risk management function is responsible for validating that BSA/AML AI models perform as intended, and the BSA compliance function retains responsibility for the quality of the BSA program outcomes. When these two functions operate in silos, the validation can conclude that the model is technically accurate while the compliance program is quietly degrading because the model's optimization target does not align with the program's compliance obligation. The governance structure must ensure that these two functions communicate, that model validation reports are reviewed by BSA compliance leadership, and that BSA compliance performance metrics are reviewed by model risk governance. The space between those two functions is where the compliance director's problem lived.

Accountability for the BSA program's quality stays with the human leadership of that program. An AI vendor cannot be held responsible for a SAR filing rate decline. The transaction monitoring system's technical architect cannot be held responsible for a program quality failure. The named BSA compliance officer, who certifies the institution's annual BSA program assessment to its federal regulator, is the person responsible. That accountability is not transferred when AI is adopted. If anything, it is intensified, because the AI creates new ways to degrade the program invisibly, and the compliance officer's obligation to detect and address those degradations in the same.

Key Takeaways

• The 90 to 95 percent false-positive rate in BSA/AML transaction monitoring is a structural condition, not a program failure. But aggressively tuning an AI model to reduce false positives without measuring the impact on true-positive detection is how an institution converts an operational problem into a compliance failure. Alert volume reduction and SAR filing rate decline are not the same event and must not be treated as equivalent.
• False positives and false negatives have asymmetric costs in BSA/AML. A false positive costs analyst time. A false negative creates the possibility of an undetected, unreported SAR, with consequences ranging from examination findings to civil money penalties and, in extreme cases, criminal referral for willful BSA violations. BSA/AML model tuning should target precision improvement (fewer false positives) while holding recall (true-positive capture rate) constant or improving it.
• The precision-recall trade-off is the technical core of the tuning problem. High precision means fewer false positives; high recall means catching more true positives. BSA/AML governance demands high recall above all. Any tuning change that improves precision at the cost of recall is a program degradation with a favorable cost metric, not a program improvement.
• Every proposed tuning change must be tested against historical case data before deployment, using a retrospective simulation that quantifies the true-positive suppression rate: how many historical SAR filings would have been missed with the proposed new configuration? That number must be presented to and approved by the BSA compliance committee before the change is deployed, with the approval documented in the model-risk file.
• Examiners evaluate BSA programs with AI-assisted monitoring by comparing SAR filing rate trends against alert volume trends, checking coverage of known FinCEN typologies, reviewing investigation quality on a sample of dismissed alerts, and requiring the institution to explain model scoring decisions for specific accounts. "We reduced false positives" does not explain a SAR filing rate decline. The institution must demonstrate that true-positive detection was preserved.
• OCC Bulletin 2026-13, issued April 2026, superseded OCC 2011-12 and requires BSA/AML AI models to be documented in the model inventory, validated before and after deployment, and revalidated on at least an annual cadence. Tuning changes are model changes and require the same documentation and governance as initial model deployment.
• Human oversight of AI-assisted tuning requires BSA compliance leadership who understand the difference between precision and recall, who can interpret SAR filing rate trends in light of model changes, and who have the authority to halt deployments whose true-positive impact has not been adequately tested. The governance structure must ensure that model risk validation and BSA compliance performance monitoring are integrated, not siloed, because the failure mode lives in the gap between them.