After the scorecard (L4 Ch2 L1) and the 40-question DD (L4 Ch2 L2), the third deliverable in the vendor onboarding chain is the proof-of-concept pilot — a 60-day, structured, parallel-run validation with documented success metrics, kill-criteria, and a go/no-go decision framework. The POC is where the vendor's declared posture meets the practice's operational reality. It is also where the practice runs the total-cost-of-ownership analysis that the vendor's per-seat price obscures: integration time, training cost, compliance retrofit, and the three-year switching cost that decides whether the choice is durable. This lesson installs the 60-day pilot blueprint, the TCO model, and the go/no-go decision memo that closes the vendor onboarding chain and produces the regulatory artifact set L4 Ch2 was designed to ship.

Why 60 Days — Not a 14-Day Trial, Not a 6-Month Pilot

The vendor's typical "14-day free trial" is product marketing dressed as evaluation. Two weeks fits a single client review cycle, captures no recurring workflow patterns, exposes no integration friction, and produces no statistically meaningful accuracy data. A six-month pilot, conversely, is open-ended commitment: the practice deploys the tool, advisors learn it, the team forms habits, and the kill decision becomes politically and operationally difficult. Sixty days is the empirical fit: two months covers two monthly review cycles for most practices, allows integration testing across CRM, planning, archive, and LLM layers, generates enough accuracy data points (typically 100-300 documents or interactions depending on use case) to score the vendor's accuracy dimension reliably, and remains short enough that kill criteria can be enforced without sunk-cost contamination.

The 60-day pilot is not a soft trial. It is a structured evaluation with the same regulatory artifact rigor as the scorecard and DD: documented design, named stakeholders, signed pilot agreement, recorded kill-criteria, captured success metrics, and a written go/no-go decision memo at day 60. The pilot is part of the firm's compliance record under SEC Rule 204-2 and FINRA Rule 4511 — examined by SEC Division of Examinations and FINRA examiners as evidence of the firm's vendor oversight under Reg S-P (May 2024 amendments) and Compliance Rule 206(4)-7.

Step 1 — Pilot Charter (Days -7 to 0)

The week before the pilot starts, the practice writes the charter. Five mandatory elements:

Sponsor and stakeholders. Managing partner (or solo principal) as sponsor; CCO as compliance owner; head of advisory or lead advisor as operational owner; IT/MSP lead as integration owner. Each named with accountabilities documented.

Use case definition. Single, narrow, named use case — not "evaluate Holistiplan generally" but "Holistiplan extraction quality on Q4 1040s for the 28 households we serve in the equity-comp executive niche, with downstream sync to RightCapital." The narrower the use case, the cleaner the measurement.

Sample size and selection. Quantified target: 80-150 documents or 30-60 client interactions depending on use case. Selected to be representative (mix of household types, complexity bands, custodian profiles), not cherry-picked.

Success metrics with thresholds. Quantitative metrics tied to scorecard dimensions: accuracy threshold (e.g., 90% field-level extraction accuracy on 1040), integration latency (e.g., Holistiplan-to-RightCapital sync within 4 hours), advisor time savings (vs. baseline of manual workflow), compliance defensibility (Reg BI memo template alignment, Marketing Rule disclosure language preserved). Each with a specific numeric threshold.

Kill-criteria. Pre-defined conditions under which the pilot is terminated regardless of progress: accuracy below 70%, breach event during pilot, vendor SLA failure on critical issue, integration failure that blocks workflow, regulatory concern surfacing mid-pilot. Kill-criteria are documented and signed by the pilot sponsor; once triggered, the pilot ends, the decision memo records the termination rationale, and the vendor is moved to the L4 Ch2 L1 scorecard's Decline disposition.

Step 2 — Pilot Execution (Days 1-60)

The pilot's execution discipline is the parallel-run protocol. Every artifact the vendor produces (extracted 1040 fields, meeting summary, draft email, Reg BI memo, IPS section) is also produced by the practice's existing manual workflow (or held back from production review). The two outputs are compared on a documented schedule.

Days 1-7 — Integration and onboarding. Configure the vendor integration with the practice's stack (CRM, planning, archive, LLM). Walk through the L4 Ch2 L2 DD answers in production. Verify the SOC 2 Type II controls operationally (data residency confirmed, MFA enforced, encryption verified, audit log accessible). Onboard the pilot participants (typically 1-3 advisors plus the compliance and IT leads). The integration phase typically reveals 1-3 friction points not exposed in vendor demos; these are documented and either resolved or escalated to vendor support.

Days 8-50 — Production parallel run. Run the use case through the vendor while continuing the manual workflow in parallel. For each sample, capture the vendor output, the manual output, the time elapsed, the integration latency, any errors or exceptions. The practice's compliance lead reviews vendor output for Reg BI documentation discipline, Marketing Rule disclosure adequacy, FINRA Rule 4511 retention metadata, and Reg S-P data handling integrity. Daily 15-minute standups for the first two weeks; weekly checkpoints thereafter.

Days 51-60 — Final measurement and analysis. Compile the accuracy data, integration metrics, time-savings analysis, and compliance review findings. Compare against the charter's success thresholds. Prepare the go/no-go decision memo.

Step 3 — TCO Beyond the Seat Price

The vendor's $80/seat/month line item is rarely the actual cost. The TCO analysis runs in parallel with the pilot and quantifies five categories:

Direct vendor cost. Per-seat license, per-document fee if applicable, premium support tier, professional services for advanced configuration, annual vs. monthly commitment differential, volume discounts, ramp pricing. Year 1, Year 2, Year 3 each modeled.

Integration time and cost. Hours of internal time (IT, MSP, advisor) to configure the integration, plus any integration platform cost (Beacon, Hubly, custom). Initial deployment plus ongoing maintenance. The L4 Ch1 L1 multi-custodian archetype anticipates higher integration cost; solo and ensemble lower.

Training cost. Hours of advisor and ops time for initial training, ongoing training as the team grows, refresh training as the vendor changes features. The "free" vendor training has hidden cost in lost producer hours.

Compliance retrofit cost. WSP update (L4 Ch3 L1), ADV Part 2A amendment if material (L5 Ch7 off-cycle), Marketing Rule audit scope addition (L4 Ch7 L1), cyber inventory addition (L4 Ch4 L1), engagement letter language review, client communication if applicable. Recurring cost as the vendor adds features that require compliance reassessment.

Switching cost in Year 3. What if, two-and-a-half years from now, the practice wants to replace the vendor? Data extraction and export, parallel run with the replacement, retraining the team, contractual termination cost, downtime during transition. Year 3 switching cost is the hidden lock-in tax; vendors with weak data portability inflate it; vendors with strong portability and contractual data return reduce it.

The TCO model produces a three-year total cost. Compare against the vendor's stated price to confirm the multiplier (typically 1.5-3x the line item depending on archetype, integration complexity, and compliance retrofit depth). If the multiplier exceeds 3x, the deployment is structurally inefficient — either the practice's stack is incompatible (data plumbing first, per L4 Ch1 L1 multi-custodian), the use case is too narrow for the integration overhead, or the vendor's pricing model conceals real cost. Document the multiplier in the decision memo and use it as a negotiating lever.

Step 4 — Go/No-Go Decision Memo

At day 60, the pilot team produces the decision memo. Structure:

Section 1 — Pilot summary. Restate the charter, the use case, the sample size, the success thresholds, and the kill-criteria.

Section 2 — Quantitative results. For each success metric, the measured value against the threshold. Accuracy 92% (vs. 90% threshold = Pass). Integration latency average 3.2 hours (vs. 4-hour threshold = Pass). Advisor time savings 47 minutes per 1040 (vs. 30-minute threshold = Pass). Compliance review findings: zero Marketing Rule trips, zero Reg BI gaps, full Rule 4511 archive metadata.

Section 3 — Qualitative findings. Integration friction points discovered and resolved (or unresolved). Vendor support quality. Advisor adoption signal. Compliance team comfort. Edge cases the pilot surfaced (e.g., K-1 extraction quality lower than 1040, complex multi-page trusts produced false-positive findings).

Section 4 — TCO three-year total. Direct vendor cost + integration + training + compliance retrofit + Year 3 switching cost. Per-advisor-seat per-year and total.

Section 5 — Recommendation. One of: Go (full deployment), Conditional Go (deploy with documented caveats, e.g., 'K-1 extraction handled manually'), No-Go (decline). Each with specific rationale tied to results.

Section 6 — Next steps. If Go, the contract negotiation list (per L4 Ch2 L1 scorecard's contract-terms dimension), the WSP update (L4 Ch3 L1), the ADV amendment trigger assessment (L5 Ch7), the rollout plan with named milestones. If No-Go, the decline rationale documented and the alternative vendor sequence.

The memo is signed by the pilot sponsor (managing partner) and the CCO. Filed under the WSP per SEC Rule 204-2 and FINRA Rule 4511. Retained for the vendor relationship duration plus three years (or, in No-Go cases, three years from decision date to support the regulatory record of process).

Worked Example: A 60-Day Jump vs Zocks Pilot Blueprint

An 8-advisor ensemble RIA in early 2026 has narrowed meeting-AI to Jump and Zocks after the L4 Ch2 L1 scorecard and L4 Ch2 L2 DD passes. The two vendors score within 4 points of each other on the scorecard; the practice runs a head-to-head 60-day pilot to decide. The charter: sponsor = managing partner Jennifer Ortega; compliance owner = CCO Robert Klein; operational owner = head of advisory David Park; IT owner = outsourced MSP lead Sara Chen; pilot team = three advisors (one early-adopter Marcus, one skeptic Linda, one neutral William); use case = client review meeting capture + Reg BI rollover memo draft + follow-up email draft for the executive-comp niche households (28 households); sample size = 60 meetings split evenly (30 to Jump, 30 to Zocks) over the 60-day window; success thresholds = meeting transcript accuracy ≥95% (measured against a human-corrected baseline on 10 sampled meetings per vendor), Reg BI memo draft requiring ≤15 minutes of advisor edit time (vs baseline 45 minutes), follow-up email draft requiring ≤8 minutes of edit time, Wealthbox sync within 1 hour of meeting end, zero Reg S-P data-handling incidents during pilot; kill-criteria = transcript accuracy below 88% on any vendor at the 30-day midpoint check, any data-handling incident, Wealthbox sync failure rate exceeding 10%.

Execution: Days 1-7 = integration. Both vendors connect to Wealthbox via API. Jump connects to Smarsh archive directly; Zocks requires a Smarsh-specific webhook configured by Sara — 4 hours of MSP time. Both vendors configured to default to high-security transcript handling with no training-on-customer-data. Days 8-50 = parallel run. Three advisors rotate which vendor records each meeting; the practice's existing manual workflow (paraplanner transcribes notes + drafts follow-up) runs in parallel for the first 20 meetings to establish baseline; thereafter the AI output is the primary with the paraplanner spot-checking. Days 51-60 = measurement.

Results at day 60: Jump transcript accuracy 96.4%, Zocks 95.8%. Jump Reg BI memo edit time average 11 minutes, Zocks 13 minutes. Jump follow-up email edit time 6 minutes, Zocks 9 minutes. Jump Wealthbox sync latency average 18 minutes, Zocks 41 minutes. Jump zero data-handling incidents, Zocks zero. Both pass thresholds. The skeptic Linda notes that Zocks' interface is more usable for clients who join the meeting on their own device; the early-adopter Marcus prefers Jump's downstream integration. The decision: Go on Jump for the production deployment of meeting capture + Reg BI memo + follow-up; pilot Zocks for the specific use case of client-on-own-device meetings as a secondary tool. Decision memo signed; Jump contract negotiated; WSP updated; ADV not materially amended (Jump was already disclosed at category level); rollout plan kicks off with the broader advisor team.

Worked TCO Example — The Jump Deployment Across Three Years

The TCO model for the Jump deployment at the 8-advisor practice over a 3-year horizon:

Direct vendor cost. Jump pricing in 2026: approximately $80/advisor/month for the standard tier, plus $30/month per support seat. 8 advisors × $80 × 12 = $7,680/year; plus 4 support seats × $30 × 12 = $1,440/year; plus $4,000 one-time professional services for the initial deployment. Year 1 = $13,120. Years 2-3 = $9,120 each (no PS repeat). 3-year direct = $31,360.

Integration time and cost. Initial integration consumed approximately 24 hours of MSP time at $185/hour = $4,440; ongoing integration maintenance approximately 6 hours per year at the same rate = $1,110/year. Internal advisor time for initial integration validation: approximately 20 hours total across the team at blended $200/hour effective cost = $4,000 (one-time). 3-year integration = $4,440 + $4,000 + $3,330 = $11,770.

Training cost. Initial training: 8 advisors × 4 hours × $200 effective = $6,400; CCO + ops 6 hours × $150 = $900; total $7,300 initial. Ongoing refresh training 1 hour per advisor per year = $1,600/year. 3-year training = $7,300 + $3,200 = $10,500.

Compliance retrofit. WSP update approximately 6 hours of CCO time at $200 = $1,200; ADV review (concluded no material amendment needed): 2 hours = $400; Marketing Rule audit scope addition: 3 hours = $600; client communication template: 2 hours = $400. Year 1 total $2,600. Years 2-3 approximately $800 each for ongoing review. 3-year compliance = $4,200.

Year-3 switching cost (hypothetical). If the practice decided to switch vendors at end of Year 3: data export and review approximately 16 hours = $2,400; replacement vendor pilot (assume another full 60-day pilot) = approximately $8,000 fully-loaded; team retraining = $7,000; contractual termination cost — Jump's standard contract permits 90-day termination notice with no cancellation fee. Estimated 3-year-end switching cost: $17,400.

3-year TCO including switching reserve: $31,360 + $11,770 + $10,500 + $4,200 + $17,400 = $75,230. On a per-advisor-per-year basis: $75,230 / (8 advisors × 3 years) = $3,135. Compared to Jump's stated line item of $80/advisor/month = $960/year, the multiplier is approximately 3.3x — slightly above the 3x structural-inefficiency flag. Investigation reveals the multiplier is driven mostly by the switching reserve (which the practice may not actually incur) and the initial integration (which amortizes over the deployment life). Excluding the switching reserve, the multiplier drops to 2.4x — within the typical 1.5-3x band. The decision memo flags the multiplier with the explanation, and the practice proceeds with the deployment.

Archetype-Specific Pilot Design Notes

The pilot scales differently across the L4 Ch1 L1 archetypes.

Solo RIA. Pilot team of one (the solo) plus outsourced CCO plus IT/MSP. Sample size on the lower end (50-100 documents/interactions). 60-day timeline. The risk is solo bias in scoring; mitigation is documented scoring criteria and outsourced CCO independent review.

Ensemble RIA. Pilot team of 2-4 advisors (including one skeptic and one early-adopter), CCO, IT lead, head of advisory. Sample size 100-200 documents/interactions across multiple household types. 60-day timeline. The risk is "early-adopter halo" where the enthusiastic adopter's outcomes don't generalize; mitigation is the skeptic-included pilot team and use-case-specific scoring.

Multi-Custodian RIA. Pilot team includes representatives across the custodian footprint (Schwab, Fidelity, Pershing, BNY Mellon as relevant). Sample size 150-300 across custodian profiles. The custodian-by-custodian integration validation is mandatory — the L4 Ch1 L1 multi-custodian integration-debt failure mode is the canonical risk. If integration is solid on Schwab but weak on Pershing, the decision must address the gap explicitly.

Wirehouse FA. The wirehouse FA's "pilot" is functionally a workflow validation within the home-office-approved toolset. Pilot scope is the FA team's adoption and workflow optimization, not vendor selection. The decision memo recommends process changes and product-feedback to the home office, not vendor switch.

OSJ / BD Supervisor. Pilot is at the supervisor layer specifically — testing the vendor's support for Rule 2210 principal review, Reg BI memo audit, Rule 4511 supervisory log, AI first-pass screening for the Marketing Rule pre-use review queue (L4 Ch3 L2). The pilot must include the dual-sampling protocol that catches false-negative rates (sampling AI-cleared content, not just flagged content). Pilot team includes the OSJ principal, CCO, and at least one supervised rep representative.

Key Takeaways

• 60-day pilot, not 14-day trial, not 6-month pilot. Two months covers two review cycles, generates 100-300 sample data points, allows kill-criteria enforcement without sunk-cost contamination.
• Pilot charter (Days -7 to 0): sponsor, stakeholders, use case definition, sample size, success metrics with thresholds, kill-criteria.
• Pilot execution (Days 1-60): parallel-run protocol with manual baseline; daily standups first 14 days, weekly thereafter; integration verification in week 1, production parallel through day 50, measurement and analysis days 51-60.
• TCO analysis in five categories: direct vendor cost, integration time/cost, training cost, compliance retrofit cost, Year 3 switching cost. Multiplier of 1.5-3x line-item typical; over 3x is structurally inefficient.
• Go/No-Go decision memo with six sections (summary, quantitative results, qualitative findings, TCO total, recommendation, next steps). Signed by managing partner + CCO, filed under WSP, retained per SEC Rule 204-2 and FINRA Rule 4511 for vendor duration + 3 years (or 3 years post-decline for No-Go).
• Archetype-specific design: Solo (1-person team, outsourced CCO review), Ensemble (4-person team with skeptic + adopter), Multi-Custodian (custodian-by-custodian validation), Wirehouse FA (workflow validation not vendor selection), OSJ (supervisor-layer with dual-sampling).
• The pilot closes the L4 Ch2 vendor onboarding chain — scorecard (L4 Ch2 L1) + DD questionnaire (L4 Ch2 L2) + pilot (L4 Ch2 L3) produce the three-artifact defensible vendor selection record that survives SEC examination and supports the L4 Ch8 L2 M&A diligence pack.
• Kill-criteria are non-optional and load-bearing. A pilot the practice cannot kill is not a pilot. The L4 Ch1 L2 readiness audit's Culture sub-score "tolerance for documented failure" anchors the discipline.