AI Vendor Due Diligence: Data, Cost and Exit Questions

An AI vendor review starts with the work the business wants to improve, not the most impressive demonstration. Define the input, expected output and consequence of an error. A draft marketing description and a decision affecting a customer’s eligibility have different risks. NIST’s AI Risk Management Framework provides a useful structure for examining those risks. Decide what data may enter the service and whether a safer sample can support the pilot. Do not upload private customer records simply to discover how a tool works. Assign a business owner who can explain the intended benefit and acceptable limits.

Review how the provider uses submitted data, whether it may train models on that information and what retention or deletion options apply. Ask about subprocessors, administrative access, incident communication and export. Test representative inputs that include ambiguity and missing information. Require the system to expose uncertainty where the workflow needs it, and keep a human review step proportionate to the consequences. A polished answer can still be incorrect. Record failure types and the time spent correcting them so that the business case reflects usable output rather than the number of responses generated.

Include subscription, usage, integration, review and support costs in the same calculation. A hypothetical tool saving an hour but requiring nearly as much checking may not improve the operating process. Set usage alerts and a spending owner. Before expanding, test an export of business records and identify how the workflow would continue if the vendor changed terms or became unavailable. Keep the pilot reversible and record a stop condition. The buying decision should be supported by a defined improvement and acceptable risk, not by a promise that automation will remove responsibility for the result.