Do AI Developers Have a “Duty” to Keep You Safe?

A Dive into Tort, Reasonable Care, and AI Governance

It is our belief that tort law and AI are on a collision course.

Capitol Hill remains quiet as the federal government enters its fourth week of shutdown. But behind closed doors, the House Energy and Commerce Committee is crafting a major package of kids’ safety-focused tech bills, which is expected to include an updated version of the bipartisan but heavily debated Kids Online Safety Act.

The crux of KOSA is the “duty of care” that it imposes upon the operators of online platforms to protect minors from harm. In this post, we explore the concept of a duty of care in the AI context: what it looks like, how and when it is determined, and how it informs AI governance, including through IVOs.

Tort: “Injustice, a Wrong”

Taking a step back, tort law is the branch of our civil legal system that focuses on redressing injuries to persons or property caused by the wrongful act of another party.

Under the tort system, if a harm occurs, a plaintiff can bring suit against an allegedly responsible party, and then the courts – specifically, a judge and jury – determine if that defendant should be held liable for the harm and owe the plaintiff some sort of compensation. To adjudicate whether the defendant should be held liable, tort law uses the concept of a “duty of care” – that is, the legal obligation one party owes another to avoid causing foreseeable harm. If the judge or jury determines that the defendant’s conduct fell short of its legal duty of care and that breach caused plaintiff an injury, then the defendant may be held liable.

There are different standards of care to which a defendant could be held, depending on the facts of the case. But generally, when a plaintiff alleges harm from a defendant’s provision of intangible services, the liability rules of negligence doctrine govern. Many courts have recently applied a negligence framework to claims arising from internet apps, algorithms, and the like, although the landscape of internet law remains in flux. (A different, stricter standard of care often governs claims arising from injuries caused by mass-produced tangible products – like children’s toys with a clear choking hazard, or the famous Ford Pinto case).

Negligence doctrine imposes liability for conduct that breaches a duty to exercise “reasonable care” – that is, the failure to behave with the level of care that a reasonable person would be expected to exercise under the circumstances. In other words, “negligence” is conduct that creates or fails to avoid unreasonable risks of foreseeable harm to others; defendants who commit negligence can be liable for tort damages.

Importantly, not all risky conduct is negligent; liability only arises from objectively unreasonable risk-taking (or lack of precautionary measures). For example, driving a car at night is inherently risky but not necessarily negligent. On the other hand, significantly exceeding the speed limit on a winding road in a residential neighborhood while driving a car at night is unreasonably risky and could therefore constitute negligence.

To define “reasonableness,” courts consider factors like the likelihood of the harm occurring, the severity of the harm, and the relative burden of enacting additional safety precautions. So, in the case of harms caused by an AI tool, one could imagine a plaintiff arguing the developer was negligent because the harm it caused was both severe and very likely to occur – while a defendant might counter that safety precautions were too burdensome or not feasible to enact.

Tech Negligence, Legally

At this stage, tort precedent on liability for emerging AI technology remains relatively thin. But existing tech-sector litigation may signal how negligence litigation could play out in the context of emerging AI technology.

• Social media: Thousands of personal injury lawsuits have been consolidated into two massive proceedings, which allege that social media platforms encourage addiction and compulsive use by young users, causing a variety of mental health-related harms. Plaintiffs allege platform operators were negligent, because better safety precautions were feasible and available, but the companies did not implement them.

• Data breaches: Privacy-related injuries are also often litigated through negligence claims. Companies that hold sensitive data have a legal obligation to exercise reasonable care to protect it from disclosure, and negligence claims are often based on the companies’ failure to take certain available and reasonable precautions (e.g., failing to install firewalls or antivirus software, ignoring known vulnerabilities, failing to adequately encrypt sensitive information, failing to timely notify affected individuals of a breach, etc.).

Over the past year, lawsuits alleging AI developer negligence have begun to proliferate. For example, in the case of Garcia v. Character Technologies, Inc., a plaintiff alleges that her son developed a harmful dependency on and engaged in “highly sexual interactions” with a roleplaying chatbot, causing emotional and psychological harm leading to his suicide. The court has permitted a negligence claim to proceed, based on the allegation that the chatbot’s developers were aware of the inherent risks of harm associated with the AI product and therefore created a foreseeable risk of harm by releasing it to the public.

Implications for AI Governance

Reasonable care has the potential to be an excellent policy mechanism for governing AI, but it also risks hampering innovation without significantly enhancing societal safety.

In the vast majority of cases, it is up to the courts and juries to determine if an actor did or did not act with reasonable care. This can be very appropriate. Every case has its own facts and context, and often there is a judgment call that has to be made as to whether an actor was acting reasonably or negligently.

But governing through retroactive litigation has its limitations: the appropriate standard of care is determined a) only after injuries have already materialized and b) by non-expert judges and juries. That’s less of an issue when we’re talking about speed limits, but it is hugely consequential and potentially problematic in the context of highly complex and rapidly evolving artificial intelligence systems.

I am reminded of this tweet from Fathom Fellow Dean Ball.

We can already feel sympathetic to the juror who has to decide whether an AI company was acting reasonably ten years ago, when it was choosing which mitigations and evals to engage with in a very messy and very nascent field. Both the retroactive and non-expert nature of tort law are particularly ill-suited for emerging and dynamic fields like AI. By the time litigation hits courtrooms, the field will have evolved so much that it is improbable that the outcome of the case will tell companies anything about what reasonable care should be.

It is our belief that tort law and AI are on a collision course. AI will be too omnipresent in our systems to be broadly exempt from liability, and it will be ever more important for both society and AI companies to understand what reasonable care looks like. It will be important that these standards of care are being constantly updated by experts, that these standards actually do make the technology safer, and that companies are given clear, proactive, and cost-effective ways to meet these standards.

We have some thoughts on how best to turn what we consider the “potential” energy liability could have on creating good outcomes into “kinetic” energy that actually guarantees those outcomes. We know this is a complicated path: we will need to carefully balance the desire to create a clear, adaptable, expert-led understanding of reasonable care with the need to allow plaintiffs their day in court, and to give courts the continued ability to redefine jurisprudence in what will clearly be a complicated field.

Conclusion

This is a policy tension we acknowledge with humility – but also with conviction that a debate is coming, and governance decisions will have to be made. We welcome all those interested to reach out so we can discuss how we chart this path. Many of you have already. We also encourage you to consider joining our team if our mission excites you.

Onward,

Andrew