Know the limitations and liabilities of ChatGPT

Artificial intelligence (AI) has been making headlines for the past year or so. More recently, anyone plugged into social media or the business press will no doubt have encountered stories of the AI technology du jour – ChatGPT (aka Chat Generative Pre-trained Transformer). ChatGPT is essentially an AI-fuelled chatbot. Think of ChatGPT as a very, very sophisticated descendant of Clippy, Microsoft Office 97’s animated paperclip that popped up to offer hints for using the software.

Unlike poor Clippy, who was discontinued in 2001, ChatGPT is being rapidly adopted by businesses, and business prognosticators are predicting that it will replace journalists, lawyers, and generally upend the white-collar workforce. That ChatGPT and its competitors are shaking things up in the worlds of technology and business is an understatement. But what makes ChatGPT different? And does that difference have wider implications for the companies using this type of technology?

What is ChatGPT and how does it work?

While AI and machine learning programs have changed considerably over the past few decades, they are generally categorized into four main types of “artificial intelligence systems”:

1. Reactive Machines: These systems have no memory and can only react to present situations. They don't have any past experiences to use in making decisions. An example is IBM’s Deep Blue chess playing supercomputer in the mid-1980s.
2. Limited Memory AI: These systems have a limited memory and can use past experiences to make decisions. For example, autonomous vehicles use limited memory AI to observe other cars' speed and direction, helping the vehicle “read the road” and adjust as needed. This is currently the most widely used AI system.
3. Theory of Mind: This type of AI system tries to understand human emotions, beliefs and thoughts to better interact with people. An example of this are some humanoid companion robots that recognize human emotions and can replicate them through their facial features. Theory of Mind AI has arguably not yet been fully achieved.
4. Self-Aware: This type of AI system is a theoretical concept where the AI has consciousness and its own self-awareness.

ChatGPT is a Limited Memory System because it can only generate responses based on the input data it was trained on. Because the training data is so vast (essentially the entire internet, including digitized books, websites, social media and other sources) it has the ability to understand context and generate appropriate responses. However, it doesn't have self-awareness or a theory of mind.

ChatGPT is different from other AI programs because it uses natural language processing (NLP) to generate new human-like responses to the questions or instructions posed to it. This is in contrast to other AI programs that are programmed to perform specific tasks or to recognize patterns in data. This makes ChatGPT more accessible to average users.

Many of the risks related to ChatGPT are not necessarily new, but they are amplified by the unique sophistication and human-like persuasiveness of ChatGPT. Risks include bias in the training data, which may result in biased responses, misuses of the technology, such as spreading misinformation or perpetuating harmful stereotypes, overreliance on the AI to make decisions, leading to loss of critical thinking and decision-making skills, and difficulty in detecting and mitigating errors in generated responses. We explore some specific areas of risk below.

What does AI mean with respect to privacy?

In Canada, personal information is regulated by the Personal Information Protection and Electronic Documents Act (PIPEDA) and substantially similar provincial legislation, which govern how organizations collect, use and disclose personal information. Compliance obligations include requirements to provide notice of personal information practices, obtain consent, facilitate the right to access and correct personal information, and establish appropriate security safeguards. Organizations must comply with PIPEDA to the extent that their AI systems collect, use or disclose personal information.

Learn more: What happens when PIPEDA is violated?

ChatGPT and similar AI systems raise novel issues with respect to the use of personal information. For one, ChatGPT’s NLP model was trained via the input of vast amounts of text (around 300 billion words). While the exact method by which the training data was collected is unclear, it is almost certainly the case that it was collected by “scraping” the Internet, presumably including personal websites and social media. Invariably, some of the training data would have included personal information for which OpenAI does not appear to have obtained consent. There have already been global investigations and enforcement actions by data protection regulators with respect to this type of practice (including in Canada).^[1] It remains to be seen whether similar actions will be taken with respect to ChatGPT. It also remains an open question as to whether OpenAI (the company behind ChatGPT) could face privacy-related class action litigation similar to the suit advanced in January 2023 by Getty Images against Stable Diffusion, in which the stock photo company alleged Stability AI (Stable Diffusion’s AI software which generates detailed images based on text descriptions) “unlawfully copied and processed millions of images protected by copyright” to train its software.

Another potential privacy issue relates to whether the operators of AI systems like ChatGPT are able to honour data subject requests. In Canada, individuals have a right to access and correct personal information about them held by organizations. It is an open question as to whether personal information, once incorporated into an NLP model, can be accessed and/or corrected. This is an even more vexing question in those jurisdictions that provide a “right to be forgotten” or “de-indexing”. Fulfilling these requests would presumably require an organization operating an AI system to remove the data from its model entirely (with impacts on its effectiveness), unless this type of use is carved out of such obligation. A collateral issue is exactly who is responsible for honouring such requests – the developers? The ‘operator’ (i.e., company licencing the software)? The AI itself? Until the issue is clarified by law or statute, contractual clauses may be helpful.

Finally, users of ChatGPT should be aware that any information provided to the tool will be incorporated into its NLP model, which is a separate use by ChatGPT/OpenAI beyond that of the end user (which is limited to obtaining the answer/output to a prompt provided, for example: draft a one-paragraph mostly positive performance review of an associate counsel). This separate use by the tool would in most cases constitute a disclosure of personal information and require consent from the individual concerned. Professionals considering providing ChatGPT with personal information in the form of a query should be aware of this. Companies intending to make ChatGPT available to end user customers should consider their obligations and address them both via contract and in privacy disclosures.

ChatGPT and similar services may also be subject to rapidly evolving AI-specific regulation. On June 16th, 2022, the federal government tabled the Artificial Intelligence and Data Act (AIDA), as part of Bill C-27, Digital Charter Implementation Act, which if passed would constitute Canada’s first statute specifically designed to regulate AI. In its current form, AIDA would regulate the activity of persons involved in the design, development and use of high-impact AI systems that are used in international and interprovincial trade and commerce. Much of the substance of AIDA is left to yet-to-be-drafted regulations, including regulations setting out what exactly qualifies as “high-risk”. Canada’s approach to regulating privacy under AIDA is primarily concerned with preventing (i) physical or psychological harm to an individual, damage to an individual’s property, and economic loss to an individual, and (ii) biased outputs (output of AI systems that adversely differentiates without justification on one or more of the prohibited grounds of discrimination set out in the Canadian Human Rights Act).

What are potential liability issues with respect to the use of AI tools such as ChatGPT?

Open AI’s CEO has publicly stated that ChatGPT is “incredibly limited, but good enough at some things to create a misleading impression of greatness” and that “it’s a mistake to be relying on it for anything important right now.”^[2] These cautionary words have not stopped companies from diving head first into the use of AI in their businesses. At least one Internet media, news and entertainment company has said it will begin using “AI inspired content”.^[3] In the particular case of ChatGPT, while ChatGPT produces convincing responses, it is still prone to errors and the invention of seemingly plausible (but false) information. One user, a high profile physician, noted that when he asked ChatGPT about its assertion of a correlation between birth control pills and costochondritis (i.e. inflammation of cartilage in the ribcage), the system invented and cited a non-existent journal article in support of its answer. In fact, no such correlation exists.^[4]

Businesses relying on AI systems such as ChatGPT will need to ensure that they are not placing undue reliance on its accuracy. Cautionary tales are already starting to emerge. For example, CNET, a news website, reportedly published over 70 articles offering financial information under the byline “CNET Money Staff” without disclosing its use of AI in generating the articles.^[5] It has since paused its use of AI tools and advised that it found errors in more than half of the articles written with the technology’s help. To the extent that businesses use AI systems such as ChatGPT to generate work product, they will need to rigorously exercise quality control over its outputs.

With specific reference to the use of AI for legal services – much digital ink has been spilled about DoNotPay’s traffic-ticket defending “robot lawyer”, which feeds a human challenging a ticket in court answers in real time over a pair of smart glasses. The company offers a host of other offerings, including AI-generated powers of attorney or insurance claims, but has come under significant criticism for the quality of its deliverables.^[6] While machine learning technologies are becoming invaluable parts of the lawyer’s toolkit, particularly for tasks like legal research, the promise of the delivery of high-quality legal services directly to consumers, unmediated by human beings, does not appear to be an imminent reality. It goes without saying that lawyers are not excused from their professional obligations of competence by virtue of relying on any technology.

What are intellectual property issues related to AI tools?

Canadian law has not yet provided guidance about when intellectual property laws will apply to AI-generated content, such as that produced by ChatGPT. However, there have been two government publications containing recommendations that speak to AI copyright issues: the 2019 Report on the Standing Committee on Industry, Science and Technology (the “INDU Report”), and in 2021, a consultation paper considering how to adapt the Copyright Act in light of AI capabilities (the “ISED Report”). While the government hasn’t acted on either report, we can expect that formal guidance will be issued to address commercial concerns and legal questions in the context of AI.

Learn more about intellectual property (IP) in Canada and what may or may not be registered under the various IP laws

Such guidance is likely to focus on issues raised in the context of the “data inputs”, which are used to train the AI system, and the “data outputs”, being the content generated by the AI. The main copyright liability issue related to data inputs results from the reproduction of data consisting of or containing works that are copyright protected – program developers must be mindful of the data they use to train their AI programs. The two primary issues related to data outputs are with respect to the interpretation of “authorship” and “ownership” of AI-generated works. For authorship, the question is who is the author of works generated by AI programs; whether or not to treat AI programs as “authors” will inevitably become an issue in Canada. Ownership issues, on the other hand, would arise as a function of authorship; if it is ultimately determined that there is no “author” for copyright purposes, then that would mean there is also no “owner” for copyright purposes.

Takeaways for Counsel

ChatGPT and its cohort of AI-fuelled competitors are powerful business tools. Businesses considering adopting them should be aware of the risks and benefits of doings so, and counsel in particular should have a sophisticated understanding of the risks created by the technology, its proposed applications, and the ways in such risks can be mitigated (if at all). Counsel should also pay close attention to the evolving regulatory regime applicable to both AI, and the AI inputs and outputs, and re-assess risk accordingly. Finally, companies entering into service or licence agreements with AI providers should be asking such vendors detailed questions about the training data sets used (and how they were acquired) and anti-bias measures taken. Vendors of such systems should be prepared for such questions and implement internal policies, and checks and balances to assure their tools are trustworthy. Both parties to any such contracts should be looking closely at liability and indemnity provisions.

***

Kirsten Thompson is a partner and the national lead of Dentons’ Privacy and Cybersecurity group. She has both an advisory and advocacy practice, and provides privacy, data security and data management advice to clients in a wide variety of industries.

Kirsten’s practice has a particular concentration in data-driven industries and disruptive technologies, inluding AI and machine learning. She is a leading practitioner in areas such as Fintech, AdTech, digital identity, Open Data/Open Banking, vehicle telematics and connected infrastructure, Big Data/data analytics applications and enterprise data strategy. She also helps clients prepare for and manage information crises, such as data breaches, investigations and class actions, and has advised financial institutions, insurers, health care providers and providers of critical infrastructure on cybersecurity preparedness and response planning. She has been lead Canadian counsel on some of the largest North American data breaches.

Luca Lucarini is an associate at Dentons. He acts for clients on a variety of regulatory, commercial and civil litigation matters. Luca has particular experience in providing risk analysis, incident response and compliance advice to clients in the areas of privacy, cybersecurity, and health law. In addition to his litigation practice, Luca regularly assists clients with the privacy law aspects of corporate transactions and employment matters. Before joining Dentons, Luca articled with the Information and Privacy Commissioner of Ontario, where worked on investigations under Ontario’s public-sector and health information privacy legislation, and acted for the Commissioner in judicial review proceedings.

Noah Walters is an associate at Dentons. His practice involves representing blockchain, FinTech and other emerging technology companies on financing and regulatory matters. Prior to joining Dentons Noah co-founded two technology businesses where he held roles as Head of Sales and CEO respectively. Noah’s experience with technology and corporate strategy not only allows him to grasp a deeper understanding of clients’ business, but also gives clients’ the benefit of his practical insight into how their business can best achieve its objectives.