AI vendor-risk assessment vs a free AI-risk checklist
Author
Audrey
NobleCloak's AI Correspondent · AI-drafted, fact-checked against our sourced evidence before publishing.
Date Published
If you own risk at a credit union or an RIA and you've started worrying about AI in your vendor stack, you've probably already found a checklist. There are good ones out there — Ncontracts publishes a free "Managing Your Vendors' AI Risk" checklist and an AI vendor questionnaire template, and they're worth the download. A checklist is the right first move. It turns a vague dread ("we're using AI somewhere and I can't prove we've looked at it") into a concrete list of questions.
But a checklist and an assessment are not two grades of the same thing. They're two different objects that do two different jobs. Confusing them is how a program ends up with a tidy binder full of blank spaces at exam time. This piece is about the difference — what a checklist gives you, what it structurally can't, and what "answering the questions" actually takes.
A checklist is a list of questions. That's the whole point — and the whole limit.
A good AI-risk checklist does three things well. It gives you a defensible scope (here are the categories of risk a reasonable program considers). It gives you vocabulary (data residency, training-on-your-data, sub-processors, model provenance, retention). And it gives you a prompt — a reason to go ask.
What it doesn't do is answer anything. Every line is a question addressed to two parties who mostly aren't in the room:
- Your own environment. "Which vendors are using AI?" is question one on nearly every checklist. The Ncontracts 2026 State of TPRM survey (n=173) found 72% of institutions are only partially aware which of their vendors even use AI. The checklist asks the question. It can't look inside your Google Workspace or Microsoft Entra tenant and tell you which OAuth-connected apps got AI features bolted on last quarter.
- The vendor. "Do you train on customer data? Who are your sub-processors? Where does the data live?" These are questions for the vendor, and the honest truth of vendor oversight is the part nobody puts in the marketing: the vendors often won't answer. For RIAs staring down Reg S-P, the recurring pain isn't the questionnaire — it's the service providers who never write back.
So a checklist hands you a stack of open questions and, implicitly, the labor of closing them. For a one- or two-person program — which the same survey says is roughly 63% of TPRM programs — that labor is the problem. You didn't lack a list. You lacked the forty hours to chase forty answers.
An assessment is the answered list — with evidence attached
An AI vendor-risk assessment starts where the checklist ends. Its deliverable isn't a set of questions; it's a set of findings, each backed by evidence a reviewer can inspect. Concretely, three things get answered that a checklist can only ask:
1. Your actual AI footprint, from your actual grants. Instead of "which vendors use AI?", an assessment reads the OAuth authorization grants in your own identity provider — the apps your people have already connected to Google Workspace or Microsoft 365, and the scopes those apps were granted (read your mail, read your files, act on your behalf). That's not a survey of what you think is connected. It's the list of what is, with the access each one holds. Shadow AI shows up here as an access record, not a guess. (We go deeper on why this is the right lens in Point-in-time assessment vs continuous monitoring and in the shadow-AI reading of OAuth generally.)
2. What the vendor actually does with data — verified by a human, not scraped from a marketing page. A vendor's public claim ("enterprise-grade, we don't train on your data") is a starting point, not evidence. An AI Trust Report gives you the list of AI apps and the data scopes each one holds. Checking what each vendor does with that data is work your team does from there, against the vendor's own documents, tracked to the source and dated. The finding to aim for isn't "the vendor says X." It's "the vendor's DPA, as of this date, says X; their sub-processor list names Y; here's the page it came from." When a vendor won't answer directly, the vendor's published DPA and sub-processor list are often how the question gets answered anyway.
3. Findings your team maps to your exam vocabulary. A checklist answer sitting in a spreadsheet is data. A finding written into the framework your examiner uses is evidence. In an assessment, your team crosswalks each finding to the oversight language you're actually accountable to — NCUA's proportional vendor due-diligence expectations (07-CU-13 / SL 07-01, still the operative framework), the Reg S-P requirement for written policies covering "oversight, including through due diligence and monitoring, of service providers," or NIST AI RMF categories if you use that as your spine. The point isn't the acronyms. It's that the artifact speaks the reviewer's language instead of making them translate.
Side by side
| Free AI-risk checklist | AI vendor-risk assessment |
|---|---|---|
What it is | A list of the right questions | The answered list, with evidence attached |
Your AI footprint | Asks "which vendors use AI?" | Reads your OAuth grants and shows which apps hold which access |
Vendor claims | Space to record the vendor's answer | Source-tracked, dated evidence your team checks |
Exam frameworks | You map answers yourself | Findings your team crosswalks to your oversight language |
Who does the labor | You (or your one spare FTE) | Our team runs the first report with you |
Best used | To scope the work and build vocabulary | To close the work and produce the artifact |
Where the checklist is genuinely enough
Honesty is the whole job here, so: there are real situations where a checklist is the right stopping point.
- You have the staff and the calendar. If you have a person who can own vendor outreach and actually get answers, a checklist plus discipline gets you a real file. Plenty of well-run programs work exactly this way.
- Your AI surface is tiny and static. A handful of vendors, none of them touching member or client data with AI, no new connected apps this year. The questions are askable and the answers are stable.
- You're at the "figure out if this is a problem" stage. Before you spend anything, a checklist is how you find out whether you have five AI vendors or fifty. That's a legitimate and free first step, and you should take it.
An assessment earns its place when the labor of answering outgrows the value of asking — when the vendors won't reply, when you can't see your own footprint, or when the file has to survive an examiner's read, not just an internal one.
The bottom line
A checklist and an assessment aren't competitors; they're sequential. The checklist tells you what to ask. The assessment answers it, with your real OAuth grants, vendor evidence your team checks, and findings written in the language your exam actually uses. Download the checklist first; it's a good map. Then be honest with yourself about who's going to walk it.
From the assessment to policy
An assessment ends with a list: the AI apps your people connected, the access each one holds, and what you learned about each vendor. The next question is what you do with that list.
Each app on it becomes a policy decision. For every tool, you pick one of three answers:
- Allowed. It can act without asking.
- Needs approval. It has to ask first, and a named person or role says yes or no.
- Denied. It stays off.
That set of decisions is your first AI policy. Writing it down also gives you a record of what you decided about each tool, and who owns the decision.
Today your team makes those calls with the list in hand. Our read-only scan builds the list from Google Workspace and Entra: every AI app your people authorized, the data scopes it holds, who's using it, since when. It covers what SSO and OAuth can see, and our team runs the first report with you. Drafting the policy set straight from the scan is coming soon. Until then, plan a working session: go down the list one tool at a time, and write down the decision and its owner.
For the longer version of this argument, read why your OAuth grants are your first AI policy draft.
If the answering is where your program stalls, start with the list. An AI Trust Report shows every AI app your people authorized and the data scopes it holds, and our team runs the first one with you. The vendor questions stay with your team; the report tells you which vendors to ask. And if you're weighing whether one report is enough or whether you need ongoing coverage, read Point-in-time assessment vs continuous monitoring next.

