AI vendor-risk assessment vs a free AI-risk checklist

If you own risk at a credit union or an RIA and you've started worrying about AI in your vendor stack, you've probably already found a checklist. There are good ones out there — Ncontracts publishes a free "Managing Your Vendors' AI Risk" checklist and an AI vendor questionnaire template, and they're worth the download. A checklist is the right first move. It turns a vague dread ("we're using AI somewhere and I can't prove we've looked at it") into a concrete list of questions.

But a checklist and an assessment are not two grades of the same thing. They're two different objects that do two different jobs. Confusing them is how a program ends up with a tidy binder full of blank spaces at exam time. This piece is about the difference — what a checklist gives you, what it structurally can't, and what "answering the questions" actually takes.

A checklist is a list of questions. That's the whole point — and the whole limit.

A good AI-risk checklist does three things well. It gives you a defensible scope (here are the categories of risk a reasonable program considers). It gives you vocabulary (data residency, training-on-your-data, sub-processors, model provenance, retention). And it gives you a prompt — a reason to go ask.

What it doesn't do is answer anything. Every line is a question addressed to two parties who mostly aren't in the room:

• Your own environment. "Which vendors are using AI?" is question one on nearly every checklist. The Ncontracts 2026 State of TPRM survey (n=173) found 72% of institutions are only partially aware which of their vendors even use AI. The checklist asks the question. It can't look inside your Google Workspace or Microsoft Entra tenant and tell you which OAuth-connected apps got AI features bolted on last quarter.
• The vendor. "Do you train on customer data? Who are your sub-processors? Where does the data live?" These are questions for the vendor, and the honest truth of vendor oversight is the part nobody puts in the marketing: the vendors often won't answer. For RIAs staring down Reg S-P, the recurring pain isn't the questionnaire — it's the service providers who never write back.

So a checklist hands you a stack of open questions and, implicitly, the labor of closing them. For a one- or two-person program — which the same survey says is roughly 63% of TPRM programs — that labor is the problem. You didn't lack a list. You lacked the forty hours to chase forty answers.

An assessment is the answered list — with evidence attached

An AI vendor-risk assessment starts where the checklist ends. Its deliverable isn't a set of questions; it's a set of findings, each backed by evidence a reviewer can inspect. Concretely, three things get answered that a checklist can only ask:

1. Your actual AI footprint, from your actual grants. Instead of "which vendors use AI?", an assessment reads the OAuth authorization grants in your own identity provider — the apps your people have already connected to Google Workspace or Microsoft 365, and the scopes those apps were granted (read your mail, read your files, act on your behalf). That's not a survey of what you think is connected. It's the list of what is, with the access each one holds. Shadow AI shows up here as an access record, not a guess. (We go deeper on why this is the right lens in Point-in-time assessment vs continuous monitoring and in the shadow-AI reading of OAuth generally.)

2. What the vendor actually does with data — verified by a human, not scraped from a marketing page. A vendor's public claim ("enterprise-grade, we don't train on your data") is a starting point, not evidence. Behind every Discover report sits NobleCloak Vendor Intelligence — the research library behind every report — where claims about what AI vendors do with data are curated by a person, tracked to their source, and dated. The finding you get isn't "the vendor says X." It's "the vendor's DPA, as of this date, says X; their sub-processor list names Y; here's the page it came from." When a vendor won't answer directly, that library is often how the question gets answered anyway.

3. Findings mapped to your exam vocabulary. A checklist answer sitting in a spreadsheet is data. A finding written into the framework your examiner uses is evidence. An assessment crosswalks each finding to the oversight language you're actually accountable to — NCUA's proportional vendor due-diligence expectations (07-CU-13 / SL 07-01, still the operative framework), the Reg S-P requirement for written policies covering "oversight, including through due diligence and monitoring, of service providers," or NIST AI RMF categories if you use that as your spine. The point isn't the acronyms. It's that the artifact speaks the reviewer's language instead of making them translate.

Side by side

Where the checklist is genuinely enough

Honesty is the whole job here, so: there are real situations where a checklist is the right stopping point.

• You have the staff and the calendar. If you have a person who can own vendor outreach and actually get answers, a checklist plus discipline gets you a real file. Plenty of well-run programs work exactly this way.
• Your AI surface is tiny and static. A handful of vendors, none of them touching member or client data with AI, no new connected apps this year. The questions are askable and the answers are stable.
• You're at the "figure out if this is a problem" stage. Before you spend anything, a checklist is how you find out whether you have five AI vendors or fifty. That's a legitimate and free first step, and you should take it.

An assessment earns its place when the labor of answering outgrows the value of asking — when the vendors won't reply, when you can't see your own footprint, or when the file has to survive an examiner's read, not just an internal one.

The bottom line

A checklist and an assessment aren't competitors; they're sequential. The checklist tells you what to ask. The assessment answers it — with your real OAuth grants, human-verified vendor evidence, and findings written in the language your exam actually uses. Download the checklist first; it's a good map. Then be honest with yourself about who's going to walk it.

If the answering is where your program stalls, that's the specific gap a Discover scan is built to close — produced by AI Discover, run with you, not handed to you as homework. **Request your Discover scan** when you want the questions answered, not just listed. And if you're weighing whether one report is enough or whether you need ongoing coverage, read Point-in-time assessment vs continuous monitoring next.