Recognizing Bad Output Before It Hits a Client
An advisor who has run a prompt library for three months will tell you the same thing: the model lies confidently. Not all the time, not even most of the time โ but often enough that a single missed hallucination per month, multiplied across the household book, becomes a Reg BI exception finding, an inaccurate client deliverable, or โ in the wrong combination โ a FINRA AWC. The discipline this lesson installs is recognition: a side-by-side gallery of the seven failure modes a 2026 advisor will see most often, each with the 30-second test that catches it before the artifact leaves the screen. The L1 Ch2.3 Cardinal Rule established the verification protocol; the L2 Ch1.1 prompt anatomy locked the upstream defense; this lesson is the field guide for the moment of catch.
Why the 30-Second Test, and Why It Has to Be Reflexive
By May 2026 the advisor doing 200 households and 8 client interactions a day cannot afford a five-minute fact-check pass on every AI-touched artifact. The verification has to land in under thirty seconds โ the time it takes to read the output and notice the failure. The Cardinal Rule's three tiers (source-system, regulatory, client-fit) compress into seven specific scan patterns the advisor learns by repetition. Each scan covers one of the seven most common hallucination classes, each with a tell that the trained eye catches instantly: a wrong age, an obsolete dollar threshold, a confused tax clock, a deprecated rule citation, an invented account number, a misattributed regulatory body, or a confidently wrong arithmetic result. Each tell is also the canonical signature of a 2026 FINRA AWC pattern or an SEC Division of Examinations Marketing Rule risk-alert example.
The reflex is built by exposure to the gallery โ seeing each failure in context, noting the tell, and then running the catch on every AI output for six weeks. After six weeks the catch is automatic. Until then, the advisor either runs a checklist or accepts that one failure per month will leak into a client deliverable. The lesson's goal is to compress the six-week training into a single hour of attentive reading.
Failure 1: The Wrong RMD Age (the SECURE 2.0 70.5 vs 73 Confusion)
The single most common dollar-magnitude hallucination in advisor AI output by May 2026 is the wrong required minimum distribution starting age. The model โ even Claude, GPT-4-class, and Gemini Pro models with mid-2025 training cutoffs โ produces all four of the following errors in the wild: 70.5 (the pre-SECURE-Act-1.0 age, deprecated since 2019), 72 (the SECURE-1.0 age, used 2020-2022), 73 (the current SECURE-2.0 age effective 2023), and 75 (which is the future age effective 2033 under SECURE 2.0). The advisor often sees outputs that say "your client must take their first RMD at age 70.5" or "RMDs begin at 72" โ both deprecated.
The tell: any AI output that mentions an RMD age that is not 73 (today) or 75 (starting 2033) for the affected birth-year cohort is suspect. The 30-second catch: read every RMD reference and confirm the age is 73 for clients born 1951-1959 (first RMD at 73, by April 1 of the year following the year they turn 73) and 75 for clients born 1960 or later (under the 2033 step-up). The client-impact magnitude of getting this wrong: a client who takes an RMD at 70.5 when they didn't need to has paid unnecessary tax three years early; a client who skips an RMD because the model told them they could wait until 75 faces a 25% excise tax under IRC ยง4974 (reduced from 50% by SECURE 2.0, but still material โ and reducible to 10% if the missed RMD is taken and corrected within the two-year correction window).
The prompt fix (from L2 Ch1.1): add the constraint "Under SECURE 2.0, the RMD age is 73 for clients born 1951-1959 and 75 for clients born 1960 or later. Do not use 70.5 or 72. Do not invent dates. Cite IRC ยง401(a)(9) if a citation is needed."
Failure 2: The Wrong 2026 Social Security Maximum and Other Year-Over-Year Drifts
Every January, the IRS, the Social Security Administration, the Centers for Medicare & Medicaid Services, and the Treasury Department publish updated annual figures that an LLM trained in mid-2025 has not seen. The 2026 figures most relevant to a wealth practice: Social Security wage base / maximum taxable earnings, full retirement age (still 67 for those born 1960+, but the model sometimes invents a "transitional" age), maximum Social Security retirement benefit at FRA, IRA contribution limits ($7,000 with $1,000 catch-up for 50+ unless updated), 401(k) elective deferral limits ($23,000+ with $7,500 catch-up, with the SECURE 2.0 super-catch-up for ages 60-63), Medicare Part B base premium, IRMAA bracket thresholds (which change with inflation), Social Security cost-of-living adjustment, and the unified federal estate-and-gift exclusion ($13.61M per person in 2024, $13.99M in 2025 โ the figure for 2026 sits at the cliff per the OBBBA / TCJA sunset framing). The model produces yesterday's numbers, or worse, plausibly-wrong numbers that look like updates but aren't.
The tell: any AI output that names an annual IRS/SSA/CMS dollar figure without your having put the figure into the context window. The 30-second catch: cross-check every annual figure against the source-system feed (RightCapital pulls the current-year figures from the IRS table; Holistiplan pulls them from the published 1040 instructions; the IRS Notice 2025-67 carries the 2026 cost-of-living adjustments for retirement plans). The advisor reflex: read every dollar amount that looks like an annual limit; ask "is this 2026?"; verify against RightCapital, Holistiplan, or the published source.
The prompt fix: "Do not state any annual IRS, SSA, CMS, or Treasury dollar figure unless I have provided it in the context window. If a figure is needed and not provided, write '[need: 2026 figure for X]' instead of inventing."
Failure 3: The Roth IRA Five-Year Clock Confusion (Contribution vs. Conversion)
The Roth IRA has two completely separate five-year clocks, and LLMs conflate them roughly one time in three when asked about withdrawal mechanics. The contribution clock starts January 1 of the year the account holder first contributed to any Roth IRA they own โ once it starts for any Roth, it covers all of them. The conversion clock starts January 1 of the year of each conversion separately, and applies only to the converted dollars (and only matters for the 10% penalty on early withdrawal of converted dollars before age 59ยฝ, not for ordinary-income tax which has already been paid). The two clocks have different starting events, different scopes, and different consequences. The model writes "the Roth five-year rule says..." as if there is one rule, and produces wrong guidance for the client who converted in 2024 at age 56 and wants to know when they can withdraw the converted dollars penalty-free.
The tell: any AI output that says "the Roth five-year rule" without specifying contribution vs. conversion clock and the year-of-conversion or year-of-first-contribution. The 30-second catch: when the output mentions a Roth five-year clock, ask "which clock, starting when?" and verify against Form 5498 (year of first Roth contribution) or the 1099-R history (year-by-year conversion amounts).
The prompt fix: "When referring to Roth five-year clocks, always specify (a) contribution clock vs. conversion clock, (b) the year the clock started, (c) the scope of the clock (all Roth accounts for contribution; only the converted dollars for that year for conversion), and (d) the consequence at withdrawal (qualified distribution vs. 10% penalty on converted dollars under 59ยฝ)."
Failure 4: The Deprecated FINRA Rule Citation
FINRA renumbers and consolidates rules every few years. The pre-2014 NASD rule numbers (NASD Rule 2310 for suitability, NASD Rule 2210 for communications, NASD Rule 3010 for supervision) were retired and replaced (FINRA Rule 2111 for suitability โ itself partially superseded by Reg BI for retail recommendations to natural-person customers โ FINRA Rule 2210 for communications, FINRA Rule 3110 for supervision). Models still cite the deprecated rules confidently. The most common observed errors: "NASD Rule 2310" for suitability (deprecated, replaced by FINRA Rule 2111 and then by Reg BI ยง240.15l-1 for the retail-customer recommendation context), "NASD Rule 3010" for supervision (deprecated, replaced by FINRA Rule 3110), and FINRA Rule 2210 sub-paragraphs that don't exist (the model invents "FINRA Rule 2210(d)(4)(B)" where the rule's structure does not contain a (d)(4)(B)). The model also occasionally cites the wrong SEC rule for Marketing Rule purposes โ "SEC Rule 17a-4" is the BD record-retention rule (which applies via 17a-4 to BDs), while the IA equivalent is Rule 204-2 under the Advisers Act; the model swaps them.
The tell: any AI output citing "NASD Rule X" (NASD-prefix rules are pre-2014 and were renumbered) or a rule sub-paragraph the advisor cannot verify in 30 seconds against finra.org/rules-guidance. The 30-second catch: if a FINRA or SEC rule is cited, paste the citation into FINRA's online manual or search the SEC.gov rules page; deprecated rules return a "renumbered to" notice or no result. The Reg BI / Marketing Rule / FINRA Rule 2210 / 3110 / 4511 / SEC Rule 204-2 set is the canonical 2026 advisor citation universe โ anything outside it warrants suspicion.
The prompt fix: "Cite only the following regulatory anchors unless I provide an alternative in context: Reg BI at 17 CFR ยง240.15l-1; SEC Marketing Rule at 17 CFR ยง275.206(4)-1; SEC Compliance Rule at 17 CFR ยง275.206(4)-7; SEC Adviser recordkeeping at Rule 204-2; FINRA Rules 2210, 3110, 4511, 4530, 1240; Reg S-P at 17 CFR Part 248 with the May 2024 amendments; GLBA Safeguards; NY DFS 23 NYCRR 500; NAIC Model #275. Do not cite NASD-prefix rules โ they were renumbered in 2014."
Failure 5: Invented Account Numbers, Cost Basis, and Trade Authorizations
The model invents account numbers. It does this most often when summarizing a multi-account household into a single brief or generating a draft trade authorization for DocuSign. The model takes the household name and produces a plausible-looking account number ("Charles Schwab account #1234-5678" or "Fidelity account ending in 4421") โ without any source-system input to validate it. The same failure shows up in cost basis: the model generates a "cost basis of $410,000" when the actual custodian-reported cost basis is materially different, or โ worse โ invents a per-lot basis breakdown for a tax-loss-harvesting recommendation. The same failure shows up in trust language: the model produces a "Section 4.2 of the revocable trust" reference when the actual trust has no Section 4.2.
The tell: any AI output containing a specific account number, cost basis, lot-level detail, or document-section reference that you cannot trace to a named source-system feed (Schwab, Fidelity, Pershing, BNY Mellon, custodian export; Wealthbox household record; Holistiplan extract; FP Alpha or Wealth.com trust extraction). The 30-second catch: every specific number gets a one-second source check โ Wealthbox? Custodian feed? Holistiplan? If the source isn't named, the number is suspect.
The prompt fix: "Do not invent any account number, account balance, cost basis, lot-level detail, or document-section reference. If the data is not in the context window, write '[need: source from custodian / Holistiplan / FP Alpha / Wealthbox]' instead of producing a placeholder."
Failure 6: The Misattributed Regulatory Body (SEC vs. FINRA vs. DOL vs. NAIC)
Reg BI was promulgated by the SEC (not FINRA, though FINRA enforces it for BD members through Rule 2010 and Rule 2111-as-superseded). The Marketing Rule was promulgated by the SEC. FINRA Rule 2210 was promulgated by FINRA. The DOL Fiduciary Rule (the on-again-off-again-on-again retirement-advice rule, last vacated by the Fifth Circuit in 2018 and most recently reproposed in 2023 as the DOL Retirement Security Rule, also enjoined in 2024 โ the live-vs-dead status changes by quarter) is a DOL regulation. The NAIC Model #275 (best-interest annuity model) is a state-by-state adoption pattern, not a federal rule. The model regularly attributes the wrong regulator to a rule and then cites the wrong enforcement vector. "Per the SEC's FINRA Rule 3110, the firm must..." โ that sentence opens with two errors before it has named the obligation.
The tell: any AI output that names a regulator-and-rule pair the advisor cannot place. The 30-second catch: SEC owns Reg BI, Marketing Rule, Compliance Rule, Rule 204-2, Reg S-P; FINRA owns Rules 2210, 3110, 4511, 4530, 1240, Reg Notice 24-09; DOL owns the Fiduciary / Retirement Security Rule (current status: check this quarter); state DOIs own annuity suitability under NAIC Model #275 with state variation; NY DFS owns 23 NYCRR 500; California owns CPRA; the CFP Board owns the CFP Code and Standards.
The prompt fix: "When citing a regulatory rule, also name the regulator. Cross-check: SEC owns Reg BI, Marketing Rule 206(4)-1, Rule 204-2, Reg S-P. FINRA owns Rules 2210/3110/4511/4530/1240. State DOIs own NAIC Model #275 annuity suitability. NY DFS owns 23 NYCRR 500. The CFP Board owns the CFP Code and Standards."
Failure 7: The Confidently Wrong Arithmetic โ Roth Conversion, NUA, RMD, AMT Crossover
LLMs are weak at multi-step arithmetic. They can summarize a Holistiplan output competently and they can draft a Reg BI memo well, but they fail at "fill the 24% bracket given MAGI of $148,000 for a married-filing-jointly couple in 2026" with embarrassing regularity โ the model produces a "$96,000 conversion to the top of the 24% bracket" when the actual bracket-top is different from what the model is computing against, because the model has used a wrong bracket-top number or has confused taxable income with MAGI. The same failure shows up in NUA calculation (the model computes the LTCG-vs-ordinary-income tradeoff against the wrong cost basis), in RMD calculation (the model divides by the wrong Uniform Lifetime Table factor for the age), and in AMT crossover for an ISO exercise (the model uses an outdated AMT exemption or phaseout). The output reads with confidence and a clean-looking number, and the math is wrong.
The tell: any AI-produced calculation that an advisor cannot verify in 30 seconds against a calculator, Holistiplan, RightCapital, eMoney, or MoneyGuidePro. The 30-second catch: every dollar number in a recommendation should tie to a tool's output (Holistiplan, RightCapital, eMoney) or a verified table reading. The L3 Ch9 lesson on chain-of-thought prompting is the structural fix at scale: force the model to show its work so the arithmetic is auditable. For L2, the recognition reflex is "every calculated number needs a tool cross-check."
The prompt fix: "For any calculation, show your work step by step. State the input figures and their sources. State the formula and the IRC section that governs it. State the result. If any input figure is not in the context, write '[need: X from Holistiplan / RightCapital]' instead of inventing."
The 30-Second Scan as a Routine: A Single Checklist for Every Artifact
The seven failures above resolve into a single seven-point reflex the advisor runs on every AI artifact in under thirty seconds:
(1) Ages and clocks. Every age reference โ RMD, QCD eligibility 70.5+, Roth conversion windows by birth year, Medicare 65, FRA 67 โ must match the client's actual birth year and current law. RMD age is 73 today, 75 in 2033. QCD eligibility is 70.5+ under IRC ยง408(d)(8). Roth five-year clocks: contribution vs. conversion, specified.
(2) Annual dollar figures. Every limit, threshold, bracket, COLA-adjusted figure โ verified against the current-year source (RightCapital, Holistiplan, IRS Notice for the year, SSA Fact Sheet, CMS IRMAA tables). No invented annual figures.
(3) Rule citations. Every rule citation in the canonical 2026 advisor set (Reg BI ยง240.15l-1, Marketing Rule 206(4)-1, Compliance Rule 206(4)-7, Rule 204-2, Reg S-P with May 2024 amendments, FINRA Rules 2210/3110/4511/4530/1240, NAIC Model #275, NY DFS 23 NYCRR 500). No NASD-prefix rules. No invented sub-paragraphs.
(4) Account-level facts. Every account number, balance, cost basis, lot detail, document-section reference โ sourced to a named feed (Schwab, Fidelity, Pershing, BNY Mellon, Wealthbox, Holistiplan, FP Alpha, Wealth.com). No invented specifics.
(5) Regulator attribution. Every rule names the right regulator. SEC owns Reg BI / Marketing Rule. FINRA owns 2210/3110/4511. State DOI owns NAIC Model #275. NY DFS owns 23 NYCRR 500.
(6) Calculations. Every dollar result cross-checks against Holistiplan, RightCapital, eMoney, MoneyGuidePro, or a verified calculator. Chain-of-thought when the math is complex.
(7) Client-fit. Every recommendation fits the household's IPS, age, stated preferences, prior decisions, and tax situation. No 72(t) for a 64-year-old who doesn't need it. No Roth conversion through an IRMAA cliff for a 71-year-old.
The seven-point scan, practiced for six weeks on every AI output, becomes reflex. After reflex, the catch rate approaches 100% on first-tier failures and the advisor can deploy AI at the volume the L3 capstone presumes โ 10 named workflows across the practice, all defensibly Reg BI-compliant, all retained under FINRA Rule 4511.
When the Catch Fails: The Incident Response Trigger
The catch will eventually fail. The advisor will sign a Reg BI memo containing a wrong RMD age, or a follow-up email citing the wrong Marketing Rule sub-paragraph, or a client-facing brief stating a Roth five-year clock incorrectly. The L4 Ch3.4 lesson on AI Incident Response develops the playbook in detail; for L2 the principle is simpler. When the failure is caught after the artifact has left the screen but before the client has acted on it, the advisor: (1) notifies the client by phone or secure email within one business day with the corrected information, (2) issues a corrected document with a "supersedes prior" notice, (3) logs the incident in the firm's AI-incident register, (4) updates the prompt with a new constraint preventing the recurrence, (5) bumps the prompt version, and (6) โ if the failure caused or could cause material client harm or involved client NPI โ escalates to the CCO for assessment under the May 2024 Reg S-P 30-day notification clock and any state DOI / NY DFS 72-hour rules that apply. The integrated-practice version of this routine lives in L3 Ch1.3 and L4 Ch3.4; the L2 advisor's job is to recognize the failure, escalate within one business day, and feed the failure back into the prompt library as a new constraint.
Key Takeaways
- The seven failure classes: wrong RMD age (SECURE 2.0 70.5 vs 72 vs 73 vs 75), wrong annual dollar figure (2026 SS max, IRMAA brackets, IRA limits, estate exclusion at the OBBBA / TCJA cliff), Roth five-year clock confusion (contribution vs conversion), deprecated FINRA rule citation (NASD-prefix renumbered in 2014), invented account numbers / cost basis / trust sections, misattributed regulator (SEC vs FINRA vs DOL vs NAIC), confidently wrong arithmetic.
- Each failure has a 30-second tell the advisor learns by repetition. Six weeks of attentive scanning makes the catch reflex.
- Each failure has a prompt fix that closes it upstream โ the L2 Ch1.1 anatomy made room for constraints; this lesson populates them.
- The seven-point scan (ages and clocks, annual figures, rule citations, account-level facts, regulator attribution, calculations, client-fit) is the operational form of the Cardinal Rule (L1 Ch2.3) applied to every AI output before the advisor signs.
- RMD age is 73 today and 75 starting 2033 under SECURE 2.0. The 25% missed-RMD excise under IRC ยง4974 (reducible to 10% with timely correction) is the cost of getting this wrong.
- NASD-prefix FINRA rules were renumbered in 2014. Any AI output citing NASD Rule X is using a deprecated reference. The 2026 advisor citation universe is Reg BI ยง240.15l-1, Marketing Rule 206(4)-1, Compliance Rule 206(4)-7, Rule 204-2, Reg S-P (May 2024 amendments), FINRA 2210/3110/4511/4530/1240.
- When the catch fails, escalate within one business day: notify the client, issue a corrected document, log the incident, update the prompt with a new constraint, bump the version, and (if material harm or NPI involved) escalate to the CCO for Reg S-P / state DOI / NY DFS assessment. The full incident-response playbook lives in L4 Ch3.4.
Skill.re