Questions to Ask Your AI Vendor: A Checklist
October 17, 2024
Privacy Plus+
Privacy, Technology and Perspective
This week, we’re presenting a checklist of questions that you should ask potential AI vendors.
1. Understanding the Technology
Start by asking for a general overview of the AI technology. What data inputs are used to create and train the model? Are any third-party foundational models used? A vendor who can easily explain their technology and its purpose is a good sign. Be wary if the explanation is vague or overly technical without substance.
2. Data Sources and Privacy
Inquire about the sources of data used to train the models. Ideally, these should include publicly available information, appropriately licensed third-party data, or data obtained with clear consent from customers or end-users. Since this is still an evolving area of law given lawsuits such as the New York Times v. OpenAI, you should also inquire about the vendor’s ability to license all sources if required or ensure that if they have to remove some sources, there will be no degradation in the quality of the outputs.
Be particularly cautious about data scraped from the Internet. While the Internet is often seen as a public domain, indiscriminate scraping of online data for AI training can raise significant legal and ethical concerns. Many websites have terms of service that prohibit scraping, and the data may include copyrighted material or personal information that was not intended for such use.
Ask your vendor specifically about their policies on web scraping. Do they use scraped data? If so, how do they ensure compliance with copyright laws, website terms of service, and data protection regulations? A reputable vendor should have clear policies and safeguards in place if they use any scraped data.
Be wary of vendors who are unclear in their disclosure about data sources, or unwilling to indemnify for claims based on training data, including copyright claims. Remember, the legal landscape around web scraping for AI training is still evolving, and it's crucial to work with vendors who take a cautious and ethical approach to data collection.
3. Use of Customer Data
If the vendor uses customer or end-user data to train their models, ask for details on what data is used and how they obtain agreement. If they claim to use only de-identified data, request information about their de-identification method and their internal policies regarding de-identification, including prohibiting re-identification. Be cautious if the vendor reserves the right to use any customer data without limitations.
[Before submitting any data to an AI service, carefully consider its sensitivity and any applicable restrictions. Evaluate whether the data contains personal information, trade secrets, or other confidential details. Review relevant data protection laws, industry-specific regulations, and contractual obligations such as client confidentiality agreements or NDAs. Consider ethical obligations, particularly regarding client confidentiality and professional conduct rules. Discuss with your vendor their capabilities for handling sensitive data, their compliance with relevant regulations, and their processes for data segregation and deletion. [Remember, as the data controller, you remain responsible for ensuring compliance with all applicable laws, contracts, and ethical standards when processing sensitive or restricted data.]
4. AI Tool Guardrails and Review Processes
Inquire about the guardrails in place for their AI tools. Does the vendor have policies for reviewing AI tools for accuracy and bias? Look for detailed responses with specific guardrails, policies, and procedures, including regular review processes with clear KPIs. In addition, AI vendors should be implementing controls for safety and security related to their products. For example, if the technology includes robotics, what safeguards have they put in place to avoid injury or destruction of property?
5. Legal Compliance
Ask the vendor to list applicable laws, regulations, and judicial decisions related to their data collection, use, and AI processing. A comprehensive understanding of applicable privacy and AI regulations is crucial. A vendor that hesitates to provide such a list, or offers only a vague or incomplete response, may not be sufficiently mature in their compliance practices.
6. Insurance Coverage
Inquire about insurance coverage for AI-related losses or claims. While the absence of AI-specific insurance isn't necessarily a deal-breaker, it's important to understand what protections are in place.
[Equally important is a review of your own organization's insurance coverage considering your AI use case. Examine your current insurance policies to understand if and how they cover AI-related risks. This may include professional liability, cyber insurance, and general business liability policies. Depending on what gaps you find, you may wish to explore the possibility of adding AI-specific riders to your existing policies, and regularly reviewing and updating your coverage to ensure your organization keeps pace with its AI adoption.]
7. Terms of Service and Contractual Agreements
While vendor responses to requests for information (RFIs) and marketing materials can provide useful insights, it's important to remember that these are not legally binding. The actual terms of your relationship with an AI vendor are governed by the contractual agreements you sign. Often, there can be significant discrepancies between a vendor's marketing claims or RFI responses and their standard Terms of Service.
Therefore, it's essential to carefully review the vendor's Terms of Service, End User License Agreement (EULA), and any other contractual documents. Pay close attention to clauses relating to data usage, privacy, security measures, disclaimers, and limitations of liability. Don't assume the terms are non-negotiable; be prepared to discuss and modify terms to align with your organization's needs and risk tolerance.
Key areas to focus on include data ownership and usage rights, confidentiality and data protection measures, performance guarantees, liability limitations, and indemnification clauses. Ensure the contract includes appropriate indemnification to cover potential legal issues arising from the use of the AI service. If a vendor has made specific claims about their service, such as compliance with certain standards or unique features, ensure these are explicitly stated in the contract. Consider including terms that address potential future scenarios, such as changes in data protection laws or significant upgrades to the AI system.
In addition to the main contract, pay particular attention to the data processing addendum (DPA). This document is crucial when dealing with AI vendors who will be processing personal data on your behalf. The DPA should clearly outline the vendor's obligations as a data processor, including their commitment to data protection, details of subprocessors, data transfer mechanisms for international data flows, and procedures for handling data subject requests. Ensure that the DPA aligns with applicable data protection laws, such as GDPR or CCPA, depending on your jurisdiction and the nature of the data being processed.
8. Considerations for Responsible AI
A growing area of concern when using any vendor tool is the responsible use of AI which considers many of the areas highlighted above along with other considerations such as ethical use, discrimination, safety, and environmental impact. When working with a vendor, it may be good to engage in conversations related to their approach to responsible AI. Unethical vendor practices could reflect negatively on an organization, so having a clear understanding (and ideally documentation) of their approach could avoid future negative impacts.
Our Thoughts
This checklist provides a solid starting point for evaluating AI vendors, but it's important to remember that responses should be considered in context. Remember, the goal isn't to find a perfect vendor, but rather to make an informed decision about risk based on a comprehensive understanding of the vendor's practices, policies, and terms.
---
Hosch & Morris, PLLC is a boutique law firm dedicated to data privacy and protection, cybersecurity, the Internet and technology. Open the Future℠.