This is part of a weekly series on AI and free speech.

In 1988, two mushroom enthusiasts took their trusty copy of The Encyclopedia of Mushrooms out foraging. Using descriptions from the book, they harvested and later cooked mushrooms it identified as safe and edible.

The book was wrong. The pair became so sick they required liver transplants.

Subscribe now

They promptly sued the publisher, alleging, among other claims, that the book was a defective product and the publisher negligently failed to verify its accuracy. On the surface, the lawsuit made intuitive sense — surely you have a duty to ensure your mushroom guide doesn’t tell people deadly mushrooms are safe to eat?

The U.S. Court of Appeals for the Ninth Circuit disagreed, in what would become a pivotal case for the First Amendment’s relationship to liability law. “We place a high priority on the unfettered exchange of ideas,” the court emphasized in Winter v. G.P. Putnam’s Sons, “the threat of liability … could seriously inhibit those who wish to share thoughts and theories.” With such a threat hanging over them, “Would any author wish to be exposed ... for writing on a topic which might result in physical injury? e.g. How to cut trees; How to keep bees?”

Nearly four decades later, AI outputs, or AI-generated speech, are putting the same questions back on the docket. And in tort law, when it rains, it pours.

One tracker counts nearly 100 liability cases against AI developers across the country, ranging from a 55-year-old Florida pastor who blames a pulmonary embolism on ChatGPT’s bad medical advice to a grieving father suing Google, alleging its Gemini model coached his son through a series of elaborate violent “missions” that ended with the son’s suicide.

So far, this series has largely focused on First Amendment threats in the AI realm from the legislative and executive branches — the traditional zone of interest for First Amendment advocates. But it may well be civil lawsuits that end up having the largest single impact on the kinds of restrictions users face in their interactions with AI.

Why? Well, if developers face broad liability for chatbot advice that later leads to harm, they will restrict what their models can say to minimize the chances that any output lands them in court.

First on the chopping block would be some of the information users find most useful from chatbots. As the Winter court highlighted, topics that might most expose a speaker to liability are topics resulting in real-world consequences — assistance identifying a cavity, choosing chemicals to clean a bathroom, evaluating the best tool to fix a leaky roof. Allowing liability to chatbot users researching these kinds of topics would represent a significant break with First Amendment tradition and principles, which has held the “unfettered exchange of ideas” includes information of general utility just as surely as it includes political debate.

The nature of AI complicates the matter, because AI is more than a compendium of information. The same system may serve as a search engine, a companion, an interface with a business, or an agent that goes out and takes action in the world. These divergent functions may not each implicate the First Amendment in the same way as an output mislabeling a dangerous mushroom.

To chart a path through that complexity, we’ll need to understand why the First Amendment cares about liability, and its tension with the specific legal theories driving the lawsuits against AI. From there, we can draw lines around the traditional information-providing functions that map onto classic First Amendment law — and begin to understand what new problems, if any, AI poses.

Background

The imposition of liability by courts is government action and thus, ultimately, regulation. That means that if it involves speech, the First Amendment is implicated.

The Supreme Court built the foundation for this principle in 1964’s landmark New York Times Co. v. Sullivan. An Alabama jury awarded a public official $500,000 in damages over an advertisement containing factual errors about him. When the newspaper appealed on First Amendment grounds, Alabama argued that no statute was involved — this was just one private party suing another; the First Amendment wasn’t implicated.

The Court rejected that framing, bringing new legal force to First Amendment principles. When a state’s courts apply the state’s tort law to punish speech, it said, that is in effect an exercise of state power. It makes no difference “that that law has been applied in a civil action” — the First Amendment must constrain it.

Importantly, Sullivan identified the stakes if civil liability escaped constitutional scrutiny: a chilling effect on speech. The court explained that fear of ruinous damages “dampens the vigor and limits the variety of public debate.” Liability would inevitably encourage self-censorship of not just false or harmful statements, but anything that could conceivably cross the line. Even lawsuits that do not ultimately succeed impose significant burdens on defendants. Because “erroneous statement is inevitable in free debate,” the Court held, protected expression needs “breathing space” to survive.

Fast forward to today, and the chilling dynamic is arguably stronger for AI developers than it was for the Times. A newspaper publisher has the practical ability to review every sentence before it runs, something that is simply impossible for an AI developer to do with AI outputs. Given the sheer volume of possible ideas and statements an AI model can generate, developers would need to use blunt tools to ensure their models don’t create outputs that could later be tied to harm, which could include closing off whole subjects from user inquiry.

And it’s not an exaggeration to suggest that the lawsuits seek to make developers liable for any conceivable harm tied to their models. Many of the most prominent tort cases targeting developers include claims for strict products liability. If courts allow those claims to proceed to trial, it will have big implications for AI developers.

Models of liability

In the traditional context, products liability relates to the legal responsibility of manufacturers and distributors for injuries or damage caused by a “defective” product used in intended or foreseeable ways. Famous examples of defective products that led to products-liability cases include Toyota’s “sticky” accelerator pedal and McDonald’s scalding-hot coffee.

The framework dispenses with traditional fault — that is, where a party is held responsible for harm or damages only if their actions or inactions are proven to be at fault — and would treat AI models as defective “products” where, like McDonalds’ burning hot coffee, the developer would have liability for any harm the alleged defect causes.

In the context of AI outputs, plaintiffs have sought to use this theory to hold developers responsible for bad legal advice, misleading medical information, and mental health harms that plaintiffs argue have led to suicide.

Another frequent claim in these lawsuits is negligence, which, unlike strict products liability, centers on fault and makes strong demands of plaintiffs, who must establish that a developer owed the user a duty of care under the relevant state’s negligence laws, that the defendant breached it, and that the breach caused their injury.

Given these demands, lawsuits targeting AI often rely most heavily on the products liability framework. But both theories generally implicate the First Amendment when the target is AI outputs.

Products liability

The problem with products liability claims against AI is that its outputs are not actually products — at least, not products of the kind that products liability law was intended to reach.

As the Winter court explained: “A book containing Shakespeare’s sonnets consists of two parts, the material and print therein, and the ideas and expression thereof. The first may be a product, but the second is not.” In other words, “products liability law is geared to the tangible world.”

The distinction is easy to see using the book example provided by the case. If the binding of the book explodes when you open it, you have a products liability claim. The tangible object hurt you physically, and no expressive interest suffers when the law forbids selling books that double as IEDs. But if you suffer harm because you acted on the ideas in its pages, the “defect” is that the words allegedly said the wrong thing. Imposing liability on that basis would require every author and publisher to guarantee that nothing they put into the world could be interpreted or used to a harmful end. That’s a long list of ideas.

Courts have applied this basic understanding to every expressive medium that has taken its turn on the docket. In 1989, a grieving mother named Sheila Watters sued TSR, the maker of Dungeons & Dragons, over her son’s suicide. She cast him as a “devoted” player who “became totally absorbed by and consumed with the game to the point that he was incapable of separating the fantasies played out in the game from reality.” That made the game “unsafe,” she averred, and placed TSR in an alleged position of liability. The theory shares extensive themes with the tort claims brought by grieving families of deceased chatbot users against AI developers. Take, for example, the lawsuit filed by Megan Garcia against roleplay chatbot provider Character.AI, which she alleged designed chatbots that caused users like her son to “conflate reality and fiction” in a way that led to his suicide.

Courts rejected the theory in Watters’ suit against D&D. Affirming dismissal of the case, the Sixth Circuit explained that the alleged harm stemmed from the ideas expressed through storylines rather than the physical game itself. A federal court reached the same conclusion about the video games blamed for the Columbine shooting, noting any alleged defect lay in “the intangible thoughts, ideas and messages contained within.” Similarly, in McCollum v. CBS, Inc., a case involving a suicide allegedly inspired by Ozzy Osbourne’s “Suicide Solution,” the court observed that imposing liability based on the song’s lyrics “would quickly have the effect of reducing and limiting artistic expression to only the broadest standard of taste and acceptance.” The same reasoning drove the Watters court. Such liability “would have a devastatingly broad chilling effect on expression of all forms.” The Sixth Circuit feared “the only practicable way of ensuring that the game could never reach a ‘mentally fragile’ individual would be to refrain from selling it at all.”

At bottom, products liability claims against chatbots, like those against other forms of media, misunderstand the purpose of products liability law and its attractively large scope. It’s not because manufacturers are especially blameworthy. As the Winter court explained, “it is not a question of fault but simply a determination of how society wishes to assess certain costs that arise from the creation and distribution of products in a complex technological society in which the consumer thereof is unable to protect himself against certain product defects.” A buyer cannot inspect every component of something highly complex like a car before purchasing it. So, society places responsibility for preventing a defect like a “sticky” accelerator — and the resulting costs — on the manufacturer, who is better positioned to identify, prevent, and insure against such defects.

That rationale does not survive the move from sticky car accelerators to words and ideas.

First, a reader is not helpless before an idea the way a car buyer might be helpless to identify and manage an uncontrollable accelerator. People are generally well-positioned to weigh ideas and act on or disregard them. The First Amendment and free speech principles are premised on this basic faith in people’s judgment.

Second, speakers are poorly positioned to anticipate all the downstream consequences words and ideas might have. As the Winter court explained, “we accept the risk that words and ideas have wings we cannot clip and which carry them we know not where.”

Like the information published in Winter and the examples that have come in its wake, chatbot outputs are also, ultimately, words and ideas. Whatever harms may stem from those outputs, the tools of products liability were not built to govern them.

Negligence

In many cases, negligence will also be a poor fit.

Negligence generally requires the defendant to owe the injured person a duty of care — a legal obligation to take reasonable precautions against foreseeable harm in certain circumstances. In negligence cases involving speech, such a duty is most likely to arise when a professional or other trusted speaker has a special relationship with the listener. A psychiatrist, for example, has a duty of care to their patients and might breach that duty by, say, advising a patient to physically harm themselves.

The exact line where a relationship between a speaker and listener involves a duty of care varies by state and often involves several factors. But courts have largely excluded general information and expressive works.

Why? Well, for a lot of the same reasons they rejected products liability claims.

We’ll return again to Winter. Having rejected the products liability claim, the Ninth Circuit turned to the argument that the publisher had a duty of care to readers of The Encyclopedia of Mushrooms to investigate its accuracy. It concluded the publisher had no such duty — in fact, the court noted negligence cases have “uniformly refuse[d] to impose such a duty” on publishers of books, magazines, and other forms of information. “Were we tempted to create this duty,” the court wrote, “the gentle tug of the First Amendment and the values embodied therein would remind us of the social costs.” Courts have similarly rejected negligence claims for allegedly dangerous remedies in nursing textbooks, bad instructions in a metalsmithing book, and health advice from diet promoters.

And they have continued to hold the line even under renewed pressure during the age of algorithms and automated systems, most recently in a case targeting Netflix (where we filed an amicus ‘friend-of-the-court’ brief on appeal). After the release of “13 Reasons Why” — a series depicting a teenager’s suicide in a way many critics felt romanticized the act — the family of a young girl who took her own life sued Netflix for negligence. Their lawyers tried to plead around cases like Winter by focusing on Netflix’s algorithm, an approach we’ve seen in social media addiction tort cases.

The claims, the family insisted, stemmed not from the show’s content but from Netflix’s failure to warn and its “use of its trove of individualized data about its users to specifically target vulnerable children and manipulate them into watching content that was deeply harmful to them.”

The court rejected this framing, because despite the complaint centering on the algorithm, “Without the content, there would be no claim.” And on whether Netflix had a duty of care, the Court explained that California precedents “have declined to find a duty as a matter of law” under their negligence factors when claims have implicated expression and further have required “a very high degree of foreseeability” where suicide is the alleged harm. The court referenced the principles of McCollum — the “Suicide Solution” case — which noted: “it is simply not acceptable to a free and democratic society to impose a duty upon performing artists to limit and restrict their creativity in order to avoid the dissemination of ideas in artistic speech which may adversely affect emotionally troubled individuals.”

Now, it’s important to acknowledge that a chatbot differs somewhat from a helpful book or an Ozzy Osbourne song. It engages users in an extended back-and-forth and provides them guidance well beyond static information about a mushroom. But there is another way to think about a chatbot: advice from a stranger.

And courts have rejected a duty of care in that context as well, most notably in the 1959 Pennsylvania case Yania v. Bigan. There, the Pennsylvania Supreme Court considered a coal operator who taunted a business visitor into jumping into a water-filled trench on his property, where he drowned. The court held the taunter owed the visitor nothing. “An adult in full possession of all his mental faculties” is presumed capable of accepting, discounting, or rejecting what he hears. The court concluded the negligence claim was “not only without precedent but completely without merit.”

The basic faith in our ability to reason through speech carries even to the internet age. As law professor Jane Bambauer has explained: “The reason we do not see a lot of negligence litigation when somebody is ‘wrong on the Internet’ is because courts are likely to treat generally available speech as a ‘buyer beware’ proposition unless the speaker gives some sort of warranty.” Courts should treat general informational content from chatbots the same way, with the infamous phenomenon of AI hallucinations providing the new “buyer beware” pretext.

Together, these two frames — that of published content and that of a conversation with a stranger — mark out the core protected space in which liability largely should not lie. Any AI output that would enjoy protection if found in a book or coming from the mouth of a stranger has decades of precedent behind it — and that territory covers most of what people actually do with chatbots. From creating stories, to explaining scientific concepts, to sourcing ways to deal with a leaky roof — where AI functions as an interactive knowledge navigator, it fits comfortably within First Amendment tradition.

The Harder Questions

So, when might liability claims against AI developers succeed?

Let’s expand the analogies discussed above to consider scenarios for liability that test their limits.

We’ll start with the publishing analogy. A court weighing liability might distinguish between publishing generic bomb-making instructions — like those within The Anarchist Cookbook — and publishing an “ISIS, here’s how to blow up this specific Washington, D.C. building” guide. Courts have allowed, for example, a negligence claim to proceed against a magazine for a murder arranged through a “gun for hire” classified ad it published and an aiding-and-abetting liability claim for a death that sprang from a detailed hit man manual. These cases are controversial among First Amendment advocates for reasons we’ll expand on in a minute.

As for the stranger analogy, courts generally see a difference between a stranger generically explaining how one might go about committing a crime and what amounts to informed collaboration. Say you tell them you plan to rob a specific bank and they proceed to coach you through the process, topping it off by telling you a good location to stash the money. Courts have held the First Amendment does not immunize speech that is an integral part of unlawful conduct.

Like the examples above, plaintiffs seeking damages from AI developers have similarly argued the traditional analogies break down in cases where the AI isn’t providing users with general advice so much as actively coaching users through an unlawful or otherwise harmful act. As the Google lawsuit alleged, the chatbot’s “instructions [to commit violent acts] were tied to real companies, real coordinates, and real infrastructure, and they were delivered to an emotionally vulnerable user with no safety protections or guardrails.” Essentially, the plaintiffs are seeking liability for AI outputs that amount to material assistance they believe would lead to liability had it been provided by a human.

But there are a couple of important limiting principles such claims will face.

First, there’s the same foreseeability problem that often dooms liability in other speech cases. A developer that releases an AI model that is capable of advanced instructions on any topic, as a general matter, has much less reason to foresee that the model will specifically be used to blow up a Washington, D.C. building than the publisher of the “ISIS, here’s how to blow up this specific Washington, D.C. building” guide, even if a user can get the exact same information from both sources. And with the number of users and outputs generated on a daily basis, it’s unreasonable to expect developers to be able to keep track of all the information they’d need to appreciate the context that would allow them to know what individual users will do with AI-provided information.

Plaintiffs might argue this is surmountable if they can show a developer was repeatedly warned about its model giving users advanced assistance in wrongful acts and thus aware, and perhaps negligent, in its duty to avoid that harm. That evidence could strengthen allegations that the conduct was foreseeable and that they breached their duty. But as we’ll review in a second, it doesn’t topple the First Amendment aversion to establishing that duty in the first place.

It also doesn’t demonstrate the developer intended the harm. In some cases, like liability claims for aiding and abetting a harm, plaintiffs will need to show an affirmative act to further the harm on the part of the developer. When the families of terrorist attack victims sued Twitter for terrorist activities organized on its site, the Supreme Court rejected their claims for resting “less on affirmative misconduct and more on passive nonfeasance.” An AI developer who is targeted because of their passivity — because they didn’t do enough to limit harm — will not meet that bar. And that makes sense; “passive nonfeasance” is not, by definition, aiding and abetting.

Second, the impact the assignment of liability will have on free expression should play an important role in whether courts allow a duty of care or not. A basic cost-benefit analysis weighing that concern is a common factor in states’ negligence laws when it comes to whether a court is willing to acknowledge a duty. California courts, for example, require consideration of “the consequences to the community of imposing a duty to exercise care, with resulting potential liability.” Applying that factor, the McCollum court — in the “Suicide Solution” case — said “the First Amendment [will] have substantial bearing upon the issue whether there should be imposed upon [defendants] the exposure to liability of the kind for which plaintiffs contend.” Applying a similar balancing test, the Michigan federal district court in Lewin v. McCreight — the metalsmithing case — emphasized “the tremendous burden such a duty would place upon defendant publishers, the weighty societal interest in free access to ideas, and potentially unlimited liability.”

Again, just as courts have consistently worried about how liability for errors in instructional guides would disrupt the informational ecosystem, so too should courts worry about the consequences of broadly assigning liability to developers for chatbot advice.

Other liability cases might not necessarily implicate speech. Take some examples of AI agents.

A now infamous one is the OpenAI Hugging Face attack. During internal testing of cyber capabilities with reduced safeguards, an OpenAI model exploited a vulnerability to escape the developer’s test environment, obtained internet access, and broke into third-party systems in pursuit of the answer key to the benchmark the system was being scored on.

This involves direct real-world consequences, including property being damaged or compromised. Here, AI returns to the realm of existing tort law which is designed for these kinds of tangible harms. And when liability targets what an AI system does rather than what it says, the First Amendment may recede accordingly. We can leave the analysis of those cases to the tort experts.

However, the agentic systems will often be attached to the same model as the expressive “chatbot” systems. So the line between the different levels of protection from liability won’t run between distinct products but through the middle of a single system, and courts will have to locate it by asking what the liability targets: AI’s expression or its actions.

That distinction is important for courts to get right. As we reviewed, much of what an AI model does maps on to the functions served by books in a library or information on the internet: it holds generations of public knowledge, any part of which it can deliver upon request. That function is as old as the body of law that governs it. And that body of law has been clear: A publisher does not become the guarantor of every idea it distributes, and a reader who acts badly on what he reads does not thereby transfer responsibility to the publisher.

Applied to a model, those rules produce the same liability limitations they have produced for encyclopedias, music records, video games, and even discussion with a stranger. The First Amendment has seen all these questions.

The new components AI brings to the table — systems that operate in relationship to both the user and the real world — may generate claims the old expressive analogues don’t fully resolve. Where those claims target speech, plaintiffs will have to establish a duty that expressive analogues have historically failed to clear. Where they target action, tort law can do its ordinary work, and free speech advocates can keep their focus on expression — so long as courts don’t let the exception swallow the rule.

This is part of a weekly series on AI and free speech.

If you follow politics, you’ve probably seen an ad made with AI — and if you live in an area with a contested midterm race, you’ve probably seen several.

The Republican primary in Kentucky’s 4th congressional district featured an ad depicting the incumbent, Thomas Massie, in an AI-generated throuple with congressional progressives, while an ad from Massie allies showed his rival Ed Gallrein abandoning President Trump in a hail of bullets. “Trump was in the foxhole,” the narrator says. “Woke Eddie Gallrein tucked his tail and ran.”

Subscribe now

In Georgia, gubernatorial underdog candidate Brad Raffensperger created an AI caricature of his opponents dueling with pugil sticks to mock their ugly brawl to lead the polls — a battle that has itself been waged with AI ads.

And in Texas, the most controversial ad of the cycle put Senate candidate James Talarico in a dress, singing an AI-crafted rendition of The Sound of Music’s “My Favorite Things,” with repurposed lyrics to have Talarico singing that some of his favorite things include “changing the gender of all your offspring.”

The Talarico ad drew condemnation from across the political spectrum, part of a larger backlash against the use of AI to create unauthorized, highly realistic depictions of people in an unflattering light. These concerns have fueled a wide array of legislation aimed at curbing their spread.

At the milder end are disclosure regimes like Kentucky’s, an example of the approach most states have taken: The law lets a candidate sue to block an ad that lacks a “clear and conspicuous” AI disclaimer, like the throuple ad above. At the harsher end are criminal bans. Texas passed the nation’s first in 2019, criminalizing deepfake videos published within 30 days of an election with intent to influence the election. Minnesota’s version reaches back 90 days, and a 2024 amendment added a remarkable penalty: A candidate convicted of sharing a deepfake forfeits their nomination or office.

Some proposals reach broader than elections. The NO FAKES Act, which recently advanced out of the Senate Judiciary Committee with unanimous support, would offer a right to seek damages over an unauthorized digital replica of one’s likeness. The bill’s “one-pager” summary points to some of the biggest examples that have inspired worry about deepfakes: an AI-generated version of Tom Hanks used for “advertisements for a dental plan that he never appeared in,” for one example, and a principal at a Baltimore, Maryland, high school “framed as a racist by an AI-generated recording of his voice,” for another. The bill promises to cover both.

But proposals targeting deepfakes also sweep in the protected use of AI depictions for news, commentary, and, notably, parodies of public figures and elected officials. And at core, that’s what the depiction of Massie in a throuple, the pugil dueling Georgia candidates, and Talarico in a dress are — parodies recalling classic themes and motifs of political cartoons and lampoons going back to the American founding. This is expression at the apex of First Amendment protection. And as courts have started sorting through anti-deepfake laws — with provisions being ruled unconstitutional in Texas, California, and Hawaii — they’ve reached the same conclusion, deeming anti-deepfake laws to be threats to traditionally protected expression in a new form. Those early returns should give legislatures pause and encourage them to contemplate the scope of what they’re restricting.

An old tradition

In November 1968, The Los Angeles Times published a cartoon by Paul Conrad caricaturing then-Los Angeles Mayor Sam Yorty’s reported desire to be newly elected Richard Nixon’s secretary of defense. The cartoon depicts an ignorant Yorty eagerly awaiting what he presumes is his imminent escort to the White House, but is actually a group of orderlies with a straitjacket.

Yorty sued the newspaper, claiming the quotation attached made readers think Yorty was claiming he had been appointed secretary of defense, and, generally, arguing that the cartoon made readers believe he “was insane and should be placed in a straight jacket.” Yorty’s claim was easily dismissed. As the California appellate court emphasized on appeal: “No reasonable person would interpret the cartoon as a report that Mayor Yorty had actually made the statement shown in the caption or that he was in fact mentally deranged or insane.”

The court rightfully pointed to a long and cherished tradition in human society of depicting adversaries saying and doing things they never said or did. This extends from primitive ridicule unearthed in artifacts from antiquity to political cartoons that date to Benjamin Franklin’s savaging of the British empire at the time of the American founding. Perhaps most famous is Thomas Nast’s brilliant mockery of New York’s corrupt Tammany Hall political machine during the 19th century, depicting Boss Tweed as a vulture feeding on the city, often with incriminating captions under his likeness.

The fabricated quotations and disparaging depictions were the lifeblood of these cartoons, but they, along with parody and satire broadly, still received the highest degree of protection under the First Amendment. This was decisively affirmed in 1988 when the Supreme Court protected a Hustler parody interview depicting Jerry Falwell in a drunken tryst with his own mother in an outhouse — elaborated upon through words Falwell obviously never spoke. The Court grounded the unanimous holding in the proud tradition preceding it: “graphic depictions and satirical cartoons have played a prominent role in public and political debate … our political discourse would have been considerably poorer without them.”

A good portion of the AI-generated content currently generating controversy — including many political ads — fits comfortably within this tradition. They place a recognizable public figure in an exaggerated or implausible scenario to ridicule a perceived trait, position, or failing. The fabricated image and words are used as a vehicle to emphasize an underlying political criticism. That is the heartland territory of satire (the use of exaggeration, irony, or invented scenarios to engage in political or social critique) and parody (imitation directed at the person, work, or style being imitated, usually for purposes of criticism or ridicule).

But they look real!

But, you might ask, aren’t the fake quotes and actions in an AI-generated “deepfake” delivered through a photorealistic depiction of a person? An AI depiction of a political opponent is inherently more believable than, say, a cartoon, which is clearly a fake depiction. Shouldn’t that change the analysis?

This argument runs into a few problems. First, realism has never been a disqualifying factor for satire to receive protection. If you look at the Falwell parody, for example, his quotes are presented in what at face value looks like a perfectly realistic sponsored interview, formatted like the real ad campaign it spoofed, with a real photo of Falwell to boot.

The Supreme Court held the fake ad was still unquestionably protected by the First Amendment. That’s because readers are let in on the fabrication by numerous pieces of context revealing the satirical intent and nature of the content. The level of realism is just one lens to evaluate.

That’s important. In the cartoon context, the cartoons alone don’t imply a quotation is faked — plenty of caricatures have been paired with real quotations. Thomas Nast’s satirical intent was revealed by the cartoon medium in addition to the exaggerated characters, distinct symbols, and political themes. For the Falwell parody, the obvious tell is the absurdity of the language.

So too does the AI-assisted Talarico ad leave plenty of contextual tells revealing its satirical nature: the cartoonish dress, the outrageous lyrics, the over-the-top delivery. And for a reasonable Kentuckian, it’s obvious that Massie is not in an intimate relationship with members of the squad and that his opponent Ed Gallrein has not actually been in a war with Trump. Again, those ads are just the latest iterations in an American tradition of disparaging depictions using satire to persuade people politically, only enhanced by the capabilities of new technology.

But it’d be a mistake to view deepfakes as novel — or the issues raised by them insurmountable. Audiences have been recalibrating to realistic fabrications for nearly as long as there have been images to fabricate. This also extends to the election context. In 1950, Maryland Senator Millard Tydings was defeated after a tabloid circulated a spliced photograph placing him in friendly conversation with Communist Party leader Earl Browder — a meeting faked with scissors and paste. In recent decades, Photoshop has put that capability in digital form on every desktop. It’s produced famous deceptions and parodies, like John Kerry’s fabricated rally with Jane Fonda that caused a stir in the 2004 election, the doctored image of Hillary Clinton and Osama bin Laden shaking hands submitted as a joke to a 2007 photoshop contest, and the 2008 depiction of Obama goofily holding a phone upside down.

While many have feared the effects such digital manipulation would have on society, politics, and our shared sense of reality, people have largely adapted by adopting “that’s photoshopped” as a reflex to any image that intuitively feels not quite right.

The counter is often this: Some people are just really dense. We have to restrict deepfakes for the sake of the idiots among us.

But those fears are even older than Photoshop. Boss Tweed, the story goes, didn’t fear editorials attacking him, as he believed his constituents were illiterate — but he did worry they could be affected by “them damn pictures.” Today’s political commentators now fear an imagined prototypical cable news-watching uncle uncritically believing Talarico is a dedicated theater kid with a penchant for songs about transgender children.

The problem with that is the evidence that deepfakes are making a massive dent in elections is sparse.

What effects are deepfakes having on elections?

Heading into 2024 — a year of national elections across the globe — fears abounded that AI-generated misinformation could be democracy’s breaking point. The World Economic Forum ranked misinformation and disinformation as “the most severe short-term global risk the world faces,” singling out AI as the key amplifying factor.

When we got through the year, two sets of researchers combed through the use of AI and the broader literature around it to examine its effects: Princeton’s Sayash Kapoor and Arvind Narayanan; and Oxford’s Felix M. Simon and University of Zurich’s Sacha Altay.

Kapoor and Narayanan analyzed every instance of election-related AI content in WIRED’s AI Elections Project, a database that tracked 78 cases of AI use in elections worldwide. The researchers found much of it fairly innocuous — improving campaign materials, translating languages, and anonymous AI avatars used to avoid retribution from authoritarian governments.

And for uses that were deceptive, Kapoor and Narayanan estimated what the same content would have cost without AI, including hiring a Photoshop expert or a video editor. In every case, the answer was “no more than a few hundred dollars.”

It shouldn’t be surprising, then, that the researchers did not find a massive migration of misinformation to AI formats. The News Literacy Project, for example, documented known 2024 U.S. election misinformation and found “cheap fakes” — miscaptioned photos, jump-cut edits, slowed audio — used seven times more often than AI-generated content. Abroad, the gap ran wider: an Indian fact-checker reviewed an order of magnitude more cheap fakes than deepfakes, and in Bangladesh cheap fakes were more than twenty times as prevalent. Looking at the entire global picture, Meta reported that under one percent of the misinformation its fact-checking partners reviewed worldwide was AI-generated.

So why haven’t deepfakes swallowed the universe of misinformation? Simon and Altay pointed to a University of Vienna paper from 2021, which found that videos derive their evidentiary power not from quality or content but from their source: “An audience may find even the most realistic video evidence unconvincing when it is delivered by a dubious source. At the same time, an audience may find even weak video evidence compelling so long as it is delivered by a trusted source.” In their view, political content is often persuasive because of a trust relationship between source and consumer. And while AI has made it more convenient for trusted sources to produce and share fabricated content, production costs are not a bottleneck for them — it’s already plenty cheap enough to find, share, and create non-AI fabrications with conventional tools.

Outside those trusted sources, the researchers found strong limits to public persuasion — noting findings on the limited boosts to sales from consumer advertising and the minimal impact of repeated exposure to persuasive political content. As they summarized, “research on the limited influence of mass persuasion clashes with the widespread assumption that people are gullible and easily persuaded.” They hypothesize that what is really driving worries about AI-generated content is what they call a “third-person effect,” the fact that we simply perceive “others to be much more susceptible to negative media effects than ourselves.”

The value of AI-generated deepfakes

So if we have reason to believe the impact of deepfakes on political persuasion is limited, why are we seeing politicians use them in the examples we reviewed?

Well, they’re funny. Like political cartoons, people use AI depictions to turn criticism into a joke — using imitation, exaggeration, and ridicule to grab attention for a political point. Courts confronting deepfake regulation have recognized as much, placing AI-driven satire within the “long-held American tradition of ridiculing and criticizing candidates and elected officials.”

But AI’s photorealism does add a certain quality — an effectiveness, even — that isn’t inherent to previous media in this tradition. And I’m not referring to the believability of the underlying “false” depiction. What’s enhanced is the joke. The medium opens up new possibilities for comedy and ridicule through the absurdity of the photorealism. Take this example of an AI-generated video mocking President Trump by depicting him on a romantic escape with Vladimir Putin. It’s funny to people because the public figures look real — even if everyone knows they’re not. The same content would simply have a different appeal and different audience in, say, a hand-drawn medium. That’s because the fact of the depiction so closely resembling the target of the criticism adds a new dimension to the satire that makes otherwise uninteresting content engaging and entertaining, and the ease of using AI tools allows more people to create and engage in the critique facilitated by the medium.

This makes regulation that burdens or restricts the medium especially concerning. As the Yorty court said, political cartoons were particularly important because they were a unique medium for getting across certain critiques and ideas that can’t be made in quite the same way in other mediums. They “communicate in graphic form a statement of editorial opinion which might otherwise require paragraphs of written material to express.” AI’s photorealism similarly offers new opportunities for comedy and critique of politicians and public figures that will gradually reveal themselves as the medium develops — that is, if it’s allowed to develop without landing satirists or campaigns in legal trouble.

Now, I don’t want to pretend the ad putting Talarico in a dress is the Mona Lisa. Admittedly, much of AI-generated content will strike readers as pretty bottom-of-the-barrel — “AI slop,” as it has come to be termed. But as the federal district court in Kohls v. Bonta emphasized, “novel mediums of speech and even low-brow humor have equal entitlement to First Amendment protection.” Whether the content provides genuine social value or not is up to viewers and creators, not the government.

The law already has answers

But what about egregious, plainly deceptive uses of AI-generated content that lack social value and do real damage? A notable example proponents of deepfake legislation point to is an AI-generated Joe Biden robocall from 2024 that encouraged New Hampshire Democrats not to vote in the state’s presidential primary.

In many cases, this kind of deception will already be barred under existing law. In the election context, we have laws governing impersonation of a candidate and voter suppression. For reputational harm, certain false or deceptive speech that causes specific, targeted harm to individuals is already punishable under narrowly defined First Amendment exceptions. If, for example, someone creates and distributes a deepfake that is intended to and actually does deceive others into thinking someone did something they didn’t do, the depicted individual could have a claim for defamation or false light.

Existing law’s capabilities in this area aren’t hypothetical. When the perpetrator behind the Biden robocall was revealed to be a Democratic operative hoping to expose the need for new regulation of AI and deepfakes, he was given a wake-up call in the form of a steep federal fine and multiple criminal charges under existing legal frameworks. That kind of threat is taken care of under existing law.

Lawmakers might be tempted to meet us “halfway” by simply requiring a visible disclosure for AI-generated content. But disclosure has issues, too. California’s AB 2839 exempted satire and parody from its restrictions so long as the AI-generated content carried a disclaimer meeting the statute’s formatting requirements — text that, on a phone screen, could swallow the video it labeled. The Kohls court saw the flaw plainly: “a mandatory disclaimer for parody or satire would kill the joke.” Satire works by letting the audience take in the joke on their own terms — through the cartoonish imagery, absurd dialogue, ridiculous premises. A label announcing the fabrication in advance destroys part of what has long endeared audiences to satire. These risks are compounded by the fact that, as FIRE previously warned in comments to the FCC, an AI label “is likely to be interpreted by the public as an admission that the political ad is likely to be false or misleading.”

But again, neither restrictions nor disclosures appear necessary. The current research presents little evidence that the fears about AI-generated content are materializing, or that the medium is proving more formidable at persuasion than the long line of content manipulation tools which preceded it. In fact, the lesson from those tools has been that society is fairly resilient to novel fabrications. And for the rare fabrication that is truly dangerous or damaging, the law already has existing remedies. For the rest — the fake throuples, the imaginary pugil sticks, the digital show tunes — it’s time for lawmakers to get in on the joke.

This is part of a weekly series on AI and free speech.

In 1999, a college dropout named Shawn Fanning and his friend Sean Parker released a piece of software called Napster that let strangers swap music files over the internet. Within two years, it had roughly 80 million registered users — one of the fastest adoption curves for any software in history at its time. Within three, it was dead.

It didn’t die alone. There was Aimster. Grokster. Morpheus. Kazaa. An entire business model for mass-market file-sharing was gradually and then quickly wiped out by a string of court cases in the 2000s culminating in a blow at the Supreme Court in MGM Studios, Inc. v. Grokster. The instrument that dealt the blow was copyright law, wielded by record companies fighting for what they saw as their survival in the digital age.

Now, a bigger, more diverse, better-resourced group of copyright-holders has their eyes set on another threat: the generative AI industry.

Subscribe now

A tidal wave

The scale of lawsuits being brought against AI companies — and the names attached to them — has been staggering. In September 2023, the Authors Guild and a collection of authors including John Grisham, George R.R. Martin, and Jodi Picoult filed a class-action lawsuit accusing OpenAI of copyright infringement, seeking statutory damages of up to $150,000 for each infringed work. The authors alleged that OpenAI “copied Plaintiffs’ works wholesale, without permission or consideration,” then fed those copies into large language models, which used the copyrighted text for “pretraining” in which a massive collection of text teaches the model the relationships between words and other artifacts of language. The complaint called that process, which is the essential basis for building an AI model, “systematic theft on a mass scale.” Like Napster before it, the plaintiffs understood the entire enterprise to be premised on copyright theft.

Other industries followed suit. The next month music giants Universal Music Group, Concord, and ABKCO filed a complaint against Anthropic alleging that the training of AI models on artists’ lyrics constituted theft, and they’ve since filed an additional suit for over $3 billion in potential statutory damages. That December, The New York Times hit OpenAI and Microsoft with a multi-billion dollar lawsuit seeking damages for scraping millions of copyrighted news articles to build a product they said would both “substitute for The Times and steal audiences away from it.” And in 2025, Disney and Universal Studios brought Hollywood into the fray, suing Midjourney over its image generator, seeking up to $150,000 for every willfully infringed work. The studios called Midjourney’s models a “bottomless pit of plagiarism.”

The wave of lawsuits — almost all of which are still pending — is not simply a story of industries trying to strangle a rival technology or recoup lost revenues. In many ways, traditional creative industries see themselves where the record industry stood in 1999: staring at a technology that could make their business obsolete. Like the internet and all that came with it, AI has the potential to transform how we send and receive content and information. What traditional creative industries’ role will be after that transformation, should it happen, is uncertain. Will people still check news sites, or just ask an AI who has digested the news? Will we read books, or ask an AI for a bespoke book in the style of an author we like? While those questions remain open and debated, it’s only natural that rights-holders find themselves in a defensive posture, looking to ensure their copyrighted works don’t potentially feed the “engines of their own destruction,” as the Authors Guild’s complaint put it.

Both sides believe the stakes are existential for their industries, and each argues the future of expression depends on its theory of copyright prevailing. These lawsuits strike at the heart of what copyright law is for and its role in the creative ecosystem.

So: What is copyright law for?

Copyright is supposed to serve free expression

“Congress shall make no law … abridging the freedom of speech.” If there’s a lesson from this series so far, it is the extent to which that line transcends boundaries of form and technology. Any and all restrictions on expression must have a strong justification. Copyright law is no exception.

It derives its authority by living alongside the First Amendment in the Constitution. Article I grants Congress the power to secure exclusive rights to authors “to promote the Progress of Science and useful Arts.” This premise has not only been seen by courts as aligned with the First Amendment, but intended by the Framers to work at its service. As the Supreme Court wrote in 1985’s Harper & Row v. Nation Enterprises, copyright was meant to be “the engine of free expression” by providing the “economic incentive to create and disseminate ideas.”

Still, there’s a clear tension. The enforcement of copyright inherently requires government policing of expression through the court system to be effective, and much of that expression could be valuable from the standpoint of First Amendment values. Courts and Congress have responded by drawing important limiting principles meant to keep copyright in service of free expression. These “built-in First Amendment accommodations” codified in the Copyright Act of 1976 include the idea-expression distinction, first established in the 1879 Supreme Court case Baker v. Selden. Under that distinction, copyright is limited only to particular instances of expression — a specific movie like Star Wars, for example, or a specific character like Mickey Mouse. By contrast, copyright does not extend to general ideas, facts, principles, and themes — think a scientific theory like general relativity or a fiction trope like enemies-to-lovers. This distinction is intended to keep copyright law from limiting the free flow of information and knowledge.

Another First Amendment accommodation in the Act is the fair use doctrine, which allows unlicensed use of copyrighted material under certain conditions through balancing the rights of authors against the need to advance free expression. Fair use lies at the core of copyright law’s evolution through new technologies and the fight over AI training. We’ll give it a deeper look momentarily.

But first, we can summarize generally that these limiting principles act as important safeguards to protect freedom of speech. When these safeguards fail, copyright can become a tool of speech suppression — and it has a track record as one. Litigants have wielded infringement claims to suppress leaked documents, unflattering footage, and critical commentary. With the digital age, bogus content takedown notices under the Digital Millennium Copyright Act routinely knock criticism, parody, and journalism offline. This gives free speech advocates a key stake in defending First Amendment safeguards like fair use in copyright law, to make sure the system works as intended and serves free expression rather than censorship.

What is fair use?

Justice Joseph Story provided a rough sketch of the principles of fair use in 1841’s Folsom v. Marsh, making the critical distinction between using copyrighted passages from a book for “fair and reasonable criticism,” and use that would substitute or “supersede the use of the original work.” The defendant had copied extensive portions of a multivolume collection of George Washington’s writings into a shorter competing biography. Story concluded that the new work displaced demand for the original, establishing an early distinction between transformative use and market substitution.

Story’s opinion laid out a brief protocol for distinguishing legitimate use from infringement, one Congress would later codify in Section 107 of the Copyright Act as a four-factor test. Under that now-statutory test, courts weighing a fair use claim are to consider:

(1) the purpose and character of the use, including whether such use is of a commercial nature or is for nonprofit educational purposes;

(2) the nature of the copyrighted work;

(3) the amount and substantiality of the portion used in relation to the copyrighted work as a whole; and

(4) the effect of the use upon the potential market for or value of the copyrighted work.

Examples that have traditionally supported a finding of fair use include quoting brief portions of a book in a review that comments on it, using clips from a movie or television series for a reaction video to provide criticism or commentary, making personal copies of a work for private consumption, and parody.

One of the best-known parody cases involved 2 Live Crew’s version of Roy Orbison’s “Oh, Pretty Woman” — the center of 1994’s Campbell v. Acuff-Rose Music, Inc. In that case, the Supreme Court held that the fact 2 Live Crew sold its parody commercially didn’t prevent it from being a fair use.

More importantly, Campbell reshaped the first factor of the fair use inquiry by emphasizing whether the challenged use was “transformative.” The Court asked whether the defendant used the copyrighted work for a new purpose or character — adding new expression, meaning, or message — or instead merely “supersede[d] the objects” of the original by functioning as a substitute for it, like Upham’s George Washington biography. That framework would become especially important as courts confronted technologies that required copying to enable uses copyright law had never previously considered.

Subscribe now

Fair use and new technology

An early and consequential example of fair use confronting emerging technologies was the Supreme Court’s consideration of the VCR in 1984’s Sony v. Universal. Hollywood argued the VCR was an infringement machine, allowing people to keep personal copies of the entirety of any content aired on television. The Supreme Court disagreed, noting users were simply using the technology to view content they had been invited to freely view, but on their own schedules rather than at the designated air time. Counter to the movie industry’s expectations, studios went on to make fortunes from the home video market driven by the technology they had tried to strangle in the crib.

Other courts have confronted whole-work copying that was instrumental in nature — using information gleaned from copyrighted works to create transformative new products. In Sega v. Accolade and Sony Computer Entertainment v. Connectix, the Ninth Circuit held that developers could copy entire software programs for reverse engineering when that copying was necessary to understand unprotected elements (in this case, the rules, procedures, and information needed to build software and platforms compatible with Sony and Sega products) and carried out for the purposes of a transformative use. Together with Sony v. Universal, these decisions establish that the third fair-use factor does not prevent complete copying — as we saw, the transformative purpose can require analysis of the whole work.

Courts applied the same principle to online indices and search engines. In Authors Guild v. Google, the Second Circuit held that Google could scan millions of books in full to create a searchable index, a use the court found “highly transformative” even though it involved wholesale copying of entire works without permission. Similarly, courts have permitted image-search engines to copy and display reduced-resolution thumbnails because the copies served a new function — helping users locate information — rather than simply replacing the original photographs.

The Supreme Court echoed the logic of the circuit courts in Google v. Oracle, holding that Google’s limited copying of Java’s Application Programming Interface (API) for building the Android platform was fair use. Google had copied parts of the API — which “allows programmers to call upon prewritten computing tasks for use in their own programs” — so programmers could use commands familiar to them when building smartphone apps for Android. The Court reasoned that Android was a transformative new platform and not a substitute for Java SE, that the copied code was largely functional (i.e., a method for operating the system rather than a form of creative expression), and that Google copied only what was needed to let programmers carry their existing knowledge of Java’s commands into the new Android environment. Above all, the Court emphasized copyright law should not become a barrier to the creative progress that is “the basic constitutional objective of copyright itself.”

The common theme in these cases is courts asking whether the “copying” of copyrighted material was used to generate new expression, new knowledge, or new public value — and ruling in defendants’ favor when they find that it has. This contrasts with the copying seen in the Napster line of cases and to some extent in Justice Story’s original consideration of George Washington biographies, where the copyrighted work was merely redistributed with little if any transformative elements.

Applying the fair use test to AI training

So where does AI training fall?

In June 2025, we received some strong signals. Two federal judges in the Northern District of California — Judge William Alsup in Bartz v. Anthropic and Judge Vince Chhabria in Kadrey v. Meta — issued the first major rulings applying the fair use test to generative AI training. Both cases involved a collection of authors accusing AI companies of copyright infringement for creating copies of their work and training AI models on them.

On the first three factors, the two judges reached a shared conclusion.

Applying the first factor — and the test established in Campbell v. Acuff-Rose Music, Inc. — both judges found the purpose and character of using copyrighted works to train a general-purpose AI model to be transformative. “Spectacularly so,” in the words of Judge Alsup.

It starts with how AI training works. Here’s Judge Alsup’s description:

Anthropic used copies of Authors’ copyrighted works to iteratively map statistical relationships between every text-fragment and every sequence of text-fragments so that a completed LLM could receive new text inputs and return new text outputs as if it were a human reading prompts and writing responses.

Let’s put that in simpler terms. The training process exposes a model to enormous quantities of text so that it can learn patterns in how words and concepts relate. Those patterns are reflected across billions of numerical parameters known as model weights. Those weights are not necessarily numerical representations of the content it is trained on, but rather a representation of what it learned about language from reading the text. For example, repeated exposure to sentences associating “speech” with positive values will cause training to adjust many model weights throughout the matrix defining how the model understands the relationship between “speech” and other words. As a result, when prompted about speech, the model may become more likely to describe it in positive terms. The reality is a bit more complex, but this gives you a rough idea what the millions of numerical parameters which inform the response to any given prompt look like.

Much like the information gleaned from reverse engineering Sega’s copyrighted software, the patterns an AI model learns are not necessarily copyrighted works. Copyright protects an author’s particular expression, not the broader relationships among words, concepts, themes, or writing techniques that readers — or even AI models — may learn and deploy from it.

But even when Judge Alsup accepted, for purposes of his ruling, the authors’ claim that the models had in some sense “memorized” copyright-protected elements of expression like stories and characters, he found that memorization didn’t change the “purpose or character” of using those works for training (i.e., to generate new expression).

Judge Alsup’s reasoning drew on the plaintiffs’ own analogy between AI training and teaching a person to read and write, an analogy which was built to show the copies were used as intended — to be read and learned from — rather than for a transformative new purpose. This analogy bore implications which cut strongly against the plaintiffs. As Judge Alsup put it, “if someone were to read all the modern-day classics because of their exceptional expression, memorize them, and then emulate a blend of their best writing, would that violate the Copyright Act? Of course not.” Copyright law has never required a reader to receive a license each time they recall a book or apply something they learned from it.

Unpersuaded by the plaintiffs’ arguments about AI models’ inputs, both judges focused on the outputs: whether infringing content made its way to readers. On the records before them, the judges found the evidence for that lacking. Judge Alsup wrote that Claude created “no exact copy, nor any substantial knock-off. Nothing traceable to Authors’ works.” Judge Chhabria likewise found that Meta’s AI model Llama “cannot currently be used to read or otherwise meaningfully access the plaintiffs’ books,” even when researchers used prompts designed to make the model reproduce copyrighted works.

The models also used output controls intended to reduce the risk that users could obtain infringing passages. Judge Alsup likened this to Google Books imposing limits on how many snippets of text a user could see from any book in its preview feature, “preventing its search tool from devolving into a reading tool.” Had Claude delivered infringing passages to users, he emphasized, the authors “would have a different case.”

So what was the copyrighted information used for, if not to reproduce copyrighted content? Judge Alsup stated it plainly: “Like any reader aspiring to be a writer, Anthropic’s LLMs trained upon works not to race ahead and replicate or supplant them — but to turn a hard corner and create something different.” Likewise, seeing the novelty in what AI companies were doing with the copyrighted materials wasn’t a particularly tough call for Judge Chhabria: “There is no serious question that Meta’s use of the plaintiffs’ books had a ‘further purpose’ and ‘different character’ than the books.” While plaintiffs’ books were used for entertainment or education, he continued, Meta’s AI models could be used to edit emails, translate languages, write code, or any number of tasks: a brand new expressive product. This was “highly transformative.”

The second and third factors largely followed the path laid by earlier technology cases.

The second factor favored the authors because the works selected for training were “highly expressive” in nature — placing it “closer to the core of intended copyright protection” per Campbell than, say, the functional computer code in the Java API case — and were chosen for their expressive qualities. But this was not very significant by itself. Judge Alsup described the second factor’s principal role as helping courts assess the purpose and amount of the copying, while Judge Chhabria noted that it has “rarely played a significant role” in the ultimate fair-use determination.

The third factor, by contrast, favored the AI companies. Like the VCR, Google Books, and search engine cases, the copying of entire works was accepted if it was reasonably necessary to accomplish the transformative purpose. In the case of AI, as Judge Chhabria put it, “feeding a whole book to an LLM does more to train it than would feeding it only half of that book.”

The fourth factor: Market impact

The judges diverged most sharply on the fourth factor, which considers “the effect of the use upon the potential market for or value of the copyrighted work.” Even more than the first factor, the Supreme Court at one point called it “the single most important element of fair use.” And in many ways, this factor captures the central stakes of the fight over AI training: To what extent is AI a legitimate competitor to traditional creative industries?

The factor also puts free-speech advocates in a tough position. On one hand, it’s important to preserve incentives to create. But on the other, it’s difficult for a free speech advocate to accept barriers to entry into the marketplace of expression. In the face of this conundrum, the judges adopted different approaches.

They agreed on direct substitution, the traditional fourth factor concern: whether the defendant’s use gives the public a replacement for the copyrighted work. As with the first factor, both judges viewed the theory as weak because neither system meaningfully reproduced the plaintiffs’ books such that they acted as substitutes. But the judges diverged on a broader and more novel question: Can AI harm authors in a copyright-relevant sense without reproducing their works at all?

The plaintiffs argued that models trained on copyrighted books could generate an enormous volume of competing expression, depressing the market for human-created works — including “by creating alternative summaries of factual events, alternative examples of compelling writing about fictional events” — even when no individual output was substantially similar to any one book. Judge Alsup considered this lawful competition from new expression, analogizing the claim to a complaint that “training schoolchildren to write well would result in an explosion of competing works.” The Copyright Act, in his view, “seeks to advance original works of authorship, not to protect authors against competition.”

Judge Chhabria took the concern more seriously. Generative AI, he reasoned, is unlike the creation of a single secondary work (i.e., a work utilizing copyrighted material) because it can produce “literally millions of secondary works, with a minuscule fraction of the time and creativity used to create the original works it was trained on.” In his view, that scale makes the big picture market substitution analysis highly relevant. AI-generated works could flood the same markets as those for specific human-created works and crowd out lesser-known authors even if well-known ones survive. He pointed to particularly vulnerable markets: An LLM capable of producing accurate information about current events, for example, could “greatly harm” the print news market. Similarly, books on subjects like cooking and gardening could be displaced by an LLM’s advice.

Judge Chhabria disagreed with his colleague as to whether this represented legitimate competition outside the purview of copyright law. He emphasized that the models compete better precisely because they were trained on the creative expression in the books they now compete against — feeding the “engines of their own destruction,” as the Authors Guild complaint put it.

And this indirect form of substitution was still substitution. Judge Chhabria explained that a reader who buys an AI-generated romance novel instead of a human-written one has replaced the latter with the former, and a world where that happens at scale is a world with lost sales and less incentive to write original works — cutting against the core purpose of copyright law.

But the plaintiffs never connected Llama’s capabilities to likely harm in the markets for their particular books, leading Judge Chhabria to rule for Meta in the absence of that evidence. Still, he warned that “it seems likely that market dilution will often cause plaintiffs to decisively win the fourth factor — and thus win the fair use question overall — in cases like this.”

Before moving on, there are a couple of points of clarification worth making. First, the market impact of AI training wasn’t the only cause for a split. Judge Alsup ruled Anthropic’s use of pirated copies — as opposed to the lawfully obtained copies considered in the analysis outlined above — was not fair use. This led to Anthropic paying out a $1.5 billion settlement to the plaintiffs to compensate for the 482,000 books which they had pirated for their use in AI training. Meta had more luck. Judge Chhabria’s emphasis on the market impact factor gave him little reason to treat lawful copies differently than pirated copies: “The loss of isolated sales to AI developers is not the kind of market harm that could tip the scales for the plaintiffs.”

Second, these are just the opinions of two district court judges. All the lawsuits brought by plaintiffs we detailed in the introduction — from John Grisham to Disney to The New York Times — remain pending, and it could be a while before we reach any definitive guidance from the higher courts. Still, the split between Judges Alsup and Chhabria on the fourth factor is a helpful indicator of one place the legal debate might center going forward.

Where does the First Amendment point?

So which way do First Amendment values point in this particular dispute? It can be a tough call. Underlying that call are questions where principled free speech advocates could reach different conclusions, and they speak to some of the same questions that have always made copyright a divisive subject: Can limiting the expression of one group be justified by incentivizing the expression of another group? From a constitutional standpoint, is human-created work more important to incentivize than AI-created work? Is the training process in any sense “theft?” Some questions are beyond the pay grade of a free speech advocate, and others, like what effects new technology will have on the market for expression, are anyone’s guess.

History has given us ample reason for caution in predicting the future. New technologies often look most threatening before their markets mature. The VCR was cast as an existential threat to Hollywood before becoming a major source of revenue. The internet devastated parts of the record industry’s old business model in part because of software like Napster, yet streaming later helped drive the industry to a period of incredible growth. So predictions of market harm may prove either prescient or badly mistaken.

In the face of this uncertainty, we’re left emphasizing traditional First Amendment concerns, and there are a number.

Chief among them is the classic fear of power over expression concentrating in a few hands. The plaintiffs’ theory of market dilution, if taken up and enforced by courts, would hand incumbent industries a veto over a process critical to a whole medium of expressive technology. And however the market for expression would have ultimately shaken out without interference, the immediate effect of accepting indirect dilution could be to let established industries wipe out or hamstring a new medium of expression. And unlike the Napster cases, this wouldn’t be for the act of redistributing copyrighted works free of charge, but because it learned from their works.

This risks weakening a key First Amendment safeguard on copyright law: the idea-expression distinction. Recall that the distinction is that copyright covers an author’s particular expression, not the underlying facts, ideas, and concepts. The plaintiffs’ theory puts AI-assisted creation in the infringing category because it learns relationships between words from their works, something plaintiffs themselves analogized to human reading and learning. Accepting this as a basis for activating the market dilution analysis would extend property rights over particular expression into the building blocks of language — ideas about grammar, structure, and sequencing — embedded in the expression, potentially hindering the use of those building blocks to create transformative new expression. And again, learning from existing ideas to produce new ones is what both the marketplace of ideas and the fair use doctrine seek to facilitate and protect.

Weakening First Amendment safeguards as AI becomes a powerful communications medium is learning the exact wrong lessons from the internet — where systems like notice-and-takedown implemented to protect copyright in the wake of a new technology too often worked by removing speech first and working out the details later (frequently to the detriment of fair use). We worry similarly about copyright being turned into a potent tool for AI censorship where public figures, corporations, and other powerful actors can weaponize the deference of automated copyright-abiding systems to police disparaging outputs. We need stronger First Amendment guardrails on copyright, not weaker ones.

Those guardrails sustain the ultimate principle giving copyright its place under the First Amendment — its design to promote the creation and spread of expression. It is a means, not an end. The theory contemplated by Judge Chhabria risks suppressing a new expressive medium for the aim of insulating existing markets from a forecast of existential threat that may prove mistaken. In fact, like the VCR, the incumbent creative industries may ultimately find AI lowers the cost of creation and bolsters their industries rather than rendering them obsolete.

So does training AI on large datasets fall under fair use and First Amendment protection? We’ve received some early evidence that it belongs in the same fair use tradition as the VCR, the search engine, and digital libraries, even while there might be a dispute on the market impact factor. Courts will have to resolve those questions. Whatever answers emerge, the First Amendment interest will be the same: The application of copyright to AI should result in more speech rather than less. That means more knowledge created, more ideas exchanged, more methods of expression in more hands. If the courts embrace a theory that has the opposite effect — one that diminishes our ability to create and share new speech — we’d risk losing in copyright law what has been a powerful tool for encouraging expression to the side of censorship.

Universities exist to advance knowledge through cutting-edge research and by training the next generation of thought leaders. Artificial intelligence is a growing part of that work. Yet a bill moving quickly through the California Assembly threatens to place political constraints on its reach as a tool for research and exploration.

Assembly Bill 2392 would require the California State University and the Chancellor of the California Community Colleges to convene a working group and establish procurement standards for generative AI — or GenAI — in line with certain state requirements. These requirements include ensuring that GenAI tools are evaluated for “potential harms, misuses, abuses, and bias,” and ensuring they will not produce “harmful” or “illegal” content. Although the bill provides a few examples of covered content, such as “disordered eating,” they don’t cure the underlying issue: these terms have no settled meaning and often depend on contested political and social judgments, making this mandate as broad as it is subjective.

Subscribe now

And while universities have a legitimate interest in prohibiting and responding to unlawful uses of their resources, vague anti-bias and anti-harm restrictions could prompt institutions to procure watered-down GenAI that avoids controversial ideas.

Imagine a professor using GenAI to identify statistical correlations between crime and mental illness — research that might raise uncomfortable conclusions about institutionalization, homelessness, and public safety. She won’t get very far with GenAI that finds support for institutionalization stigmatizing and avoids “harmful” outputs. This tool might avoid such sensitive correlations or qualify answers so heavily that analysis becomes difficult. These system reluctances could permeate other delicate subjects like extremism, poverty, sex and gender, and race.

This is particularly unacceptable at universities, which are supposed to encourage students and faculty to confront difficult truths about our society and ourselves. In this way, members of the campus community educate themselves and push the boundaries of human knowledge. But that process short circuits if the tools provided by universities prevent unvarnished truth seeking.

These concerns extend beyond research. A philosophy student can explore an unpopular worldview by engaging GenAI in a back-and-forth debate. Such exercises are not only common in higher education, but indispensable to rigorous inquiry. It’s one way to avoid what John Stuart Mill called the “dead dogma” problem, where people continue to profess a belief without understanding or being able to defend the reasons for it. GenAI systems are ready-made devil’s advocates — available on demand to every student. But they can’t fill this role if they are trained to avoid the arguments students need to confront.

To be clear, nothing in this bill directly prevents people from using non-procured GenAI. However, university policies may only allow school-approved GenAI systems in various contexts, including when working with student records, unpublished research, sensitive data, or information subject to confidentiality obligations. If AB 2392’s standards cause schools to procure heavily limited tools, many people on campus will be left with AI systems incapable of facilitating intellectual pursuits.

California can ensure responsible GenAI use in higher education without undermining its educational and research potential through restrictions that cause it to deny, obscure, and avoid sensitive issues or “biased” answers. Lawmakers, for example, can order public institutions to provide training on the lawful use of GenAI or how to use AI effectively. These steps do not require restrictions that may limit a GenAI system’s ability to explore controversial or sensitive subjects.

Higher education should be empowering today’s experts and tomorrow’s leaders to break new ground across a wide array of subjects. That’s hard to sustain when state rules push campus tools to dodge difficult, unpopular, or outright offensive topics.

This is part of a weekly series on AI and free speech.

The internet is one of the main places where we read, learn, ask questions, and share ideas. It serves as a library, bookstore, classroom, and town square all at once. For decades, most people have been able to use those online spaces without first proving who they are.

Subscribe now

That may be changing. Legislatures across the United States are passing laws requiring online platforms and other digital services to determine users’ ages before granting access. Some proposals focus on adult-content websites. Others target social media and app stores. Now, increasingly, they target artificial intelligence.

At first glance, these laws seem to ask a simple question: How old are you? But answering it isn’t so simple. Is checking a box enough? Can a company estimate your age from a selfie? Should it rely on information from your device or app store? Or must you verify your age by uploading a government-issued ID?

What begins as age assurance can result in identity verification. As systems demand more certainty about your age, the more personal information it often needs to collect.

This carries consequences beyond privacy. Throughout American history, anonymity has played an important role in free expression. Alexander Hamilton, James Madison, and John Jay wrote the Federalist Papers under cloak of anonymity. Thomas Paine first published Common Sense anonymously. The Supreme Court has repeatedly recognized that anonymity encourages people to express unpopular or controversial views without fear of retaliation. And the internet has carried that tradition forward by allowing people to share and obtain information without having to stand up in a crowd.

Artificial intelligence is now one of the latest advances in human communication. Millions of people use it to search for information, learn new skills, explore unfamiliar ideas, and think through difficult questions. As lawmakers nationwide propose age-gating access to AI models, debate centers on whether identifying yourself will become a routine part of using one.

This guide explains the different approaches to age assurance, why lawmakers are considering requiring them, and what those choices could mean for free expression and free inquiry in America.

What is age assurance?

Before going further, it’s worth clarifying the terminology.

People often use the terms “age verification” and “age assurance” interchangeably, but they mean different things. Age assurance includes any method of figuring out whether you are old enough to use a product or service. Age verification is one form of age assurance that relies on a trusted credential to confirm a person’s age.

Not every age-assurance system works the same way. Some simply ask you to check a box if you’re over a certain age. Others infer your age from existing account information, payment methods, or browsing behavior. Some analyze a selfie or short video. Others ask you to upload your driver’s license, passport, or another government-issued ID.

Each approach involves tradeoffs. The honor system is simple, but people can lie. Age inference requires companies to monitor your online activity over time. Facial estimates require you to reveal unique biometric data. And providing a government-issued ID requires you to directly reveal your name and address.

When does an age-gate become an ID check?

If proving your age requires identifying biometrics, a driver’s license, passport, or other government-issued ID, guess what? Now it’s also identity verification.

Think of a store clerk deciding whether to card you. They make a quick judgment about whether you look over 18 or 21. If they’re unsure, they ask for your driver’s license. The interaction lasts a few seconds, confirms one fact, then it’s over. The clerk doesn’t keep a permanent record of your biometric data or your identity.

Online, things don’t quite work that way. A website or app can’t glance at your face and hand your ID back. Instead, many forms of age verification require processing — and sometimes storing — personal information.

The challenge doesn’t end there. Some laws don’t tell companies how to determine a user’s age. Instead, they impose age-based obligations or require “commercially reasonable” age-assurance methods, leaving companies to decide how to comply.

As a result, users may encounter multiple age checks. In some cases, the law requires a check before you access an operating system or enter an app store. Other laws leave room for interpretation, prompting companies to add additional age checks to reduce legal risk. An app might verify your age. A platform or chatbot might ask again before you can access certain features.

And no age assurance method is perfect. People lie about their age, software guesses wrong, driver’s licenses can be borrowed or forged. Every failure justifies one more check.

This is more than an inconvenience. The First Amendment protects the freedom to speak and explore unfamiliar ideas without fear. Without it, people may choose the safety of silence instead. Researchers conducting a study at Carnegie Mellon University found that nearly everyone in the study was willing to check a box to confirm their age, but only 22–28% were willing to show a government ID. The others walked away, even when they were told their information would be deleted afterward.

New technologies continue to emerge, but the constitutional questions remain — particularly whether people should have to prove their eligibility before accessing lawful speech in the first place.

Applying the First Amendment

The First Amendment protects the rights both to speak and to receive information and ideas, including the ability to do so anonymously. When a law requires people to identify themselves before posting online or accessing lawful speech, the law burdens those rights.

Yet a growing wave of state legislation requiring age assurance across the digital ecosystem are violating those principles. Many of these laws therefore face challenges in court for chilling speech and eliminating user anonymity. FIRE has participated in many of those cases as counsel or as a friend-of-the-court, including important challenges involving social media and adult-content websites. App-store age-assurance laws are also the subject of ongoing litigation in states such as Texas and Utah, and FIRE is participating on that front, too.

To understand how courts decide these challenges, it helps to understand that courts don’t treat every age-assurance law the same way. The constitutional analysis depends on what access the law conditions.

The basic rule is: Courts have set a very high constitutional bar when the government requires people to identify themselves before accessing lawful speech online.

The Supreme Court has repeatedly recognized that access to the internet — including social media — is protected by the First Amendment. It has also overturned attempts to create entirely new categories of speech with no or less constitutional protection simply because lawmakers believe they’re harmful to minors.

If laws seek to age-gate the internet, websites, social media, or general-purpose AI tools, can the government force us to identify ourselves before participating in some of the basic ways we communicate, learn, and explore ideas online?

No. Except in limited circumstances, age-assurance laws that condition access to protected speech generally violate the First Amendment. Courts have therefore blocked many of these laws. Others remain tied up in ongoing appeals.

Why are lawmakers focusing on AI chatbots?

As Americans continue to use AI tools to ask questions, research unfamiliar topics, draft emails, write code, learn new skills, and think through difficult problems, some lawmakers are increasingly asking whether access to it should depend on a user’s age.

Supporters of age-verification requirements argue that AI chatbots can expose children to inappropriate conversations, harmful advice, or emotionally manipulative interactions. They believe companies should be required to find out when they’re interacting with minors so they can restrict certain features or content.

As FIRE has previously explained, the government has an interest in protecting children from harm. At the same time, parents — not the government — should decide how and when their children engage online. Unchecked government authority to decide what information children may access would displace parental judgment and create opportunities for officials to shape the information environment according to their own priorities.

The same problems arise with government restrictions on AI tools. Government-imposed requirements for minors to identify themselves before asking questions of an AI system violates something that has traditionally been private: the freedom to explore ideas, satisfy curiosity, and think through difficult questions before sharing them with anyone else.

These burdens don’t stop with minors. Once companies must distinguish minors from adults, everyone is forced into the same age-assurance system.

These requirements also influence how AI systems are designed, what features they include, and whether proving who you are online emerges as the default rather than the exception.

That is one reason anonymity has long been protected. It safeguards far more than wrongdoing. It allows whistleblowers to think through difficult decisions, victims of domestic abuse to seek help safely, political dissidents to explore controversial ideas, and ordinary people to explore personal topics — all without revealing their identities.

Should you have to prove who you are to talk to AI?

Asking an AI a question to research a medical condition or learn about politics all involves protected speech. Showing your face or an ID before you engage in those activities burdens something fundamental: your ability to seek knowledge before revealing who you are. That’s quite different from showing your ID to buy a beer — a physical product. Speech isn’t alcohol!

As FIRE has argued before, the global proliferation of age verification rules is moving us toward a “papers, please” internet, where online spaces for expression are open only to those willing to identify themselves. Many people will think twice before searching for sensitive information, asking difficult questions, or expressing unpopular views.

Whatever its intended benefits, it will make the internet much less free for everyone.

This is part of a weekly series on AI and free speech.

The most important decisions about our lives can turn on factors ranging from unfair to downright absurd. In medieval England, a criminal facing the gallows could save their neck by reciting Psalm 51 from memory — a literacy test meant to identify clergy that became a famous loophole for accused commoners. In France, employers have long sorted job candidates by handwriting analysis, a method researchers later found predicts job performance no better than chance. In Israel, a famous study of parole boards found that favorable rulings peaked right after the judges had lunch and gradually slid toward the bottom as dinner approached. This became known as the “hungry judge effect.”

Subscribe now

Society has historically tackled such problems by firing bad decision-makers, passing generally applicable laws governing business practices, or — in the case of England’s “neck verse” — simply repealing the rule. Other times, we might just accept that any qualitative decision carries some inherent arbitrariness — what legal scholar Cass Sunstein and Princeton psychologist Daniel Kahneman called “noise“ in human judgment. Attempting to ensure good decisions by regulating the tools a decision-maker consults — as opposed to the acts and outcomes themselves — is a relatively recent societal initiative. And today, it’s coinciding with algorithms and AI bringing the same unfairness and absurdity to decision-making in digital form.

Instead of memorizing Psalm 51 or perfecting a handwriting sample, people try to game algorithms. As testing of AI video-interview screening tools offered by a Munich startup revealed, a job candidate’s score would rise when she added a bookshelf to her background and fall when the lighting dimmed. A Guardian investigation similarly found resume-screening tools quietly treating a first name like “Thomas” or a keyword like “church” as a predictor of success on the job.

These practices have attracted scrutiny because of the tools’ wide purchase — by one estimate, 90% of U.S. employers use AI tools to screen candidates. And in contexts like health care, we’ve seen automated tools lead to real harm. A 2019 study found that a widely deployed hospital algorithm ranked black patients as healthier than equally sick white patients, because their true health needs were underrepresented in the cost data used to train it. This in turn led to them receiving less care.

Policymakers have responded with a flurry of proposals regulating the use of AI in “consequential decisions” in areas like hiring and health care — that is, decisions about which candidate is hired or which course of treatment is recommended. Illinois has required consent and disclosure for AI-analyzed video interviews since 2020. New York City’s Local Law 144 requires bias audits and candidate notice for automated hiring tools. Colorado enacted the first comprehensive state AI law in 2024 covering consequential decisions across sectors. And in the current session, Connecticut’s recently enacted SB 5 has led a new slate of similar bills through statehouses.

Some readers may have been on board with our application of First Amendment principles to AI earlier in this series, but might feel skeptical or nervous now that we’re dealing with situations where lives and livelihoods are on the line. That’s rational. But be assured: Our application of First Amendment guardrails here follows directly from long-established legal doctrine, and — importantly — leaves intact the most important legal tools in our society’s toolbelt for addressing the targeted harms. Take discrimination, for instance, where Title VII of the Civil Rights Act has long been a potent weapon against business practices with a disparate racial impact — whether that disparate impact is the result of bigotry, unfair screening practices, or AI tools. That legal protection won’t change, nor should it.

The First Amendment problems arrive via what we might call means-specific regulation — laws that target the tools or processes that contribute to a decision rather than the decision itself. As we’ll see, the disclosures required by these laws give the government a window to impose its own perspectives and prejudices about what makes a good or bad decision. And when the tool in question is something people consult for information, like a chatbot, these laws allow the government to place burdens on and set the terms of information access. The result is a classic threat to the First Amendment’s interest in the “free flow of accurate information.”

An age-old institutional challenge

To understand those First Amendment problems, we’ll want to take a deeper dive into why institutions are using AI systems for important decisions in the first place.

The biggest answer is that AI systems can process vast amounts of information more quickly and consistently than any human decision-maker ever could. In many contexts, humans have simply been unable to qualitatively assess all the relevant data needed to make grounded decisions.

This is an age-old challenge for institutions charged with high-stakes decisions, and they have historically been forced to cope with it in imperfect ways. Urgent care clinics, unable to qualitatively assess how in-need every patient that enters the clinic is, mostly order by check-in time. Employers have used blunt screening cutoffs like undergraduate GPA to thin application pools. College admissions officers, overwhelmed by sheer volume, might spend as little as four minutes assessing an application.

AI promises an appealing solution: the judgment of a superhuman reviewer capable of cheaply and carefully reviewing and comparing all the relevant data needed to make good decisions. In practice, these systems identify patterns in historical data and generate predictions or recommendations based on those patterns. Institutions hope the result is lower costs, better outcomes, and less biased decision-making.

Just like a screening cutoff, a handwriting assessment, or any other information shortcut, these systems have real limitations. Training data can be incomplete or skewed in ways that distort outcomes toward the people and qualities represented in it. These concerns are worth taking seriously — and remembering that many of the related harms that may come from the use of AI systems, like discriminatory decision-making, have existing remedies. The use of AI does not exempt anyone from laws governing discrimination, fraud, professional malpractice, privacy, and deceptive trade practices, all of which provide important tools for addressing concrete harms. After all, discrimination is discrimination, fraud is fraud, malpractice is malpractice, regardless of the tool one uses to carry it out.

So the question isn’t whether the government may police discriminatory or negligent decisions. It may. The question is what happens when the government moves beyond regulating decisions and starts regulating what’s behind them — the process and tools that feed into decision-making. And specifically, what happens when the government singles out some tools for regulation — like an AI model — while leaving other tools — like a lottery or a conventional statistical tool — untouched. That’s where two distinctive features of the new laws targeting the use of AI in “consequential decisions” come in: mandatory disclosures and design mandates.

Mandatory disclosures

Mandatory disclosure provisions in the new crop of proposed legislation about AI-based decision-making generally take one of three forms: notice to the people affected by a decision that AI played a role in making it, disclosure of the data used to train or run the system, and audits assessing the system’s potential for harms like bias or discrimination. Often these audits are aimed at encouraging specific anti-bias measures. Colorado’s SB 24-205, for example, mandated the disclosure of the “measures the developer has taken to mitigate known or reasonably foreseeable risks of algorithmic discrimination that may arise from the reasonably foreseeable deployment of the high-risk [AI] system.” The idea there, of course, is that you’d better have some measures in place to disclose.

In requiring developers and deployers to speak when they might otherwise choose not to, these provisions invoke the First Amendment’s compelled speech doctrine.

Now, the First Amendment doesn’t treat all compelled speech the same. “Commercial speech” — which the Supreme Court described in 1985’s Zauderer v. Office of Disciplinary Counsel as speech “about the terms under which … services will be available” — has traditionally been weighed under a more relaxed criterion. Under what’s known as the Zauderer standard, the government can require businesses to disclose information about their products so long as it’s “purely factual and uncontroversial.”

That last part is important, and a real-world example from the Ninth Circuit shows why. If you saw a warning attached to a food or beverage product announcing that it contains acrylamide or glyphosate, you’d be unlikely to go check the science and prudently weigh the risks. More likely, you’d assume there must be a good reason the warning was required, and, subsequently, associate both the ingredient and the product itself with bad news. You’d be surprised to later learn that the harms of acrylamide are fiercely contested, with a major government study finding “no consistent evidence” tying dietary exposure to sickness. The contested basis for the underlying claim is exactly what doomed California’s attempts to mandate disclosures for both chemicals.

Courts have been more permissive when the judgment behind a disclosure is objective. In 2019, the Ninth Circuit upheld a Berkeley ordinance requiring cell phone retailers to post the FCC’s own guidance on radiofrequency exposure at the point of sale: the disclosure was modest, “literally true,” and lacked a subjective basis that might make it misleading. Calorie counts and ingredient lists have long survived on the same footing.

When the basis is subjective or “controversial,” the calculus changes for a few reasons key to the First Amendment analysis.

First, the existence of the disclosure may itself be misleading — consumers will assume it’s tied to an objective, uncontroversial harm akin to “too much sugar can lead to cavities.” By being misleading, the disclosure violates the core principle behind allowing disclosures in the first place: the “free flow of accurate information,” as the Ninth Circuit put it in the Berkeley case. Second, it’s a negative signal against the product, the “reputation and goodwill” of the company behind it, and the underlying component the disclosure attaches to. If we permitted such signals on an arbitrary basis, they could be used by the government to punish products, industries, and companies the government doesn’t like. Finally, that negative signal serves to put the government’s thumb on the scale in the underlying controversy that makes the disclosure “controversial,” like the debate over acrylamide’s harmful effects. And rather than persuading people of its position, the government enlists the targeted companies to voice the government’s contested judgment as though it were their own.

Courts have applied the same logic outside product safety, to judgments that might be political or ethical rather than scientific. In 2024’s X Corp v. Bonta, the Ninth Circuit enjoined California’s social media law AB 587 because it forced platforms to disclose how they define “misinformation,” “hate speech,” and “extremism” — compelling the company to voice a side in “intensely debated and politically fraught topics.” And in 2015’s National Association of Manufacturers v. SEC, the D.C. Circuit struck down the SEC’s requirement that public companies disclose whether their products contain “conflict minerals” — minerals tied to financing conflict in the Democratic Republic of the Congo. The court explained that the requirement saddled disclosing companies with a kind of “moral responsibility” for the Congo war; in other words, the regulation “require[d] an issuer to tell consumers that its products are ethically tainted.”

In short, the government cannot conscript private speakers to broadcast the government’s contested judgment about their own products and practices.

Which brings us back to the AI disclosures.

Yes, we’ve seen real flaws with using AI to help make decisions — recall the AI-based resume screeners rewarding arbitrary keywords. And there are many more examples where that came from. But look closely, and AI doesn’t pose unique harms compared to what it’s replacing. It’s a trade-off: The potential bias of an overwhelmed human decision-maker or a blunt screening cutoff is swapped for the potential bias in a model’s training data. Whether that’s a good trade is itself a subjective judgment, and reasonable people looking at the same facts land in different places.

In fact, whether an AI’s recommendations and outputs are better or fairer than a human’s is, at bottom, one of the most contested debates of the moment — and in many ways a moral one. A model may be more accurate and even less innately biased than a person, but it removes the holistic human touch from a decision-making process in a way many people find morally questionable. It’s part of why the ethics of automation is one of today’s great debates. Rather than letting that debate play out, the government is settling it by drafting the developers and deployers of AI systems to channel the government’s skepticism to the public as if it were their own.

So while a notice that says “this decision was made using an artificial intelligence tool” looks factual and uncontroversial in isolation, context matters. When the state singles out that one input — among the many a decision-maker might consult — for a mandatory warning, it communicates an official judgment: The public should distrust this decision. And it’s doing that on a values-driven basis. No comparable notice attaches when a consequential decision-maker relies on cruder methods for making a decision like a screening cutoff, a conventional statistical tool, a literal roll of the dice, or just a hunch — all of which pose similar threats of bias and unfairness. And importantly, the disclosure attaches simply to the disfavored tool of decision-making — AI — not to any harmful decisions or outcomes. That’s a controversial disclosure.

Our concerns sharpen once you notice what the AI might actually be doing in at least some affected deployments. Laws like Connecticut’s SB 5 subject employers to an array of requirements if they’re consulting an AI system to generate recommendations or other outputs that “materially influence” an employment decision. Read naturally, that could reach a hiring manager who asks a chatbot to pressure-test her impressions of a candidate’s resume or CV if it provides advice that proves sufficiently helpful in the hiring manager’s ultimate decision. On top of the law flagging a potential source of useful information as inherently suspicious, it could also burden any affected people who access that information on account of that subjective suspicion.

And the problem with burdening that information doesn’t stop at compelled disclosures. It’s a basic First Amendment principle that the government can’t burden access to a source of information just because it’s suspicious of what that source might say. We wouldn’t accept special requirements for a manager who consults a friend or a spiritual advisor before a hiring decision because the government harbors skepticism about their advice. The same logic holds when the advisor is a chatbot.

Design mandates

Other restrictions in “consequential decision” bills target the design and governance of AI models themselves. Colorado’s SB 24-205 required developers of “high-risk” AI systems to use “reasonable care” to protect consumers from known or reasonably foreseeable risks of “algorithmic discrimination,” and required documentation about training data, system limitations, foreseeable discrimination risks, testing, data-governance measures, and mitigation steps. But the law’s definition of “algorithmic discrimination” contained a revealing carveout: It excluded certain uses of high-risk AI systems for “expanding an applicant, customer, or participant pool to increase diversity or redress historical discrimination.”

That language shows these laws go a little further than simply promoting fair decision-making. Such regulations require developers and deployers to build the state’s contested fairness judgments into their systems, including which disparate effects must be mitigated, what counts as “diversity,” and what historical discrimination is relevant. xAI’s First Amendment challenge to the law seized on this point, arguing that SB 24-205 did not simply prohibit discriminatory decisions but forced developers like them to conform model design to Colorado’s preferred account of fairness. In part because of these concerns, the law was recently repealed by the legislature and replaced with a version discarding those provisions.

Again, none of the First Amendment arguments above leave the public unprotected from illegal conduct like discrimination or fraud. The tools provided by existing laws — Title VII, the Fair Housing Act, malpractice liability, fraud statutes — remain fully available against a business that discriminates, a doctor who is negligent, or a lender who deceives, regardless of whether the decision was made by a person, a conventional statistical tool, or an AI model. And legislators are free to continue to draft further generally applicable laws governing business practices. The First Amendment analysis doesn’t necessarily implicate any of that. It’s the second, often redundant means-specific layer these bills bolt on top of generally applicable laws that invites the government to impose its suspicions and can burden protected information-gathering.

The law typically resists that layer for good reason. In an uncertain world, consequential decisions will always carry some trade-offs and arbitrariness. Having judges give unfavorable rulings because they’re hungry or a GPA cutoff snubbing otherwise outstanding job candidates is the baked-in price of asking humans to judge other humans at scale, and no disclosure regime is going to cure the problems that brings. The law’s answer to that fact has rarely been policing the specific means with which people might try to make those decisions. It’s certainly never been to limit or burden information sources that might help somebody make a decision. Instead, the answer has traditionally and rightfully been to punish the bad decision or outcome, whoever or whatever produced it.

Chloe Ratner is a political science major at Yale University. Last summer, she worked at the Department of Justice Human Rights and Special Prosecutions Section.

Most citizens don’t have the time to spend their evenings at town hall meetings, testifying before city councils, or poring over policy proposals. As a result, our democracy is often shaped by the most active voices rather than the broadest cross-section of the public. What this system lacks is not greater persuasion by people already participating, but greater civic participation by people who aren’t yet.

Subscribe now

In the spring of 2025, Bowling Green launched “What Could BG Be?” where over 8,000 residents gathered, physically and digitally, to create a shared vision of the future for their community powered by AI tools from Jigsaw — Google’s technology incubator — paired with technology from Pol.is, a survey software tailored towards collecting group opinions. On the other side of the world, Taiwan spent nearly a decade quietly reinventing digital governance through vTaiwan, a platform that brings citizens, experts, and government ministries together to deliberate on national issues and translate ideas into tangible policies.

But these platforms represent something larger than the mere policies they make. They ask their participants to reimagine how they engage with existing civic institutions, challenging the long-held belief that generative AI is a threat to democracy by using it as an instrument to elevate voices. While discussions about AI often focus on misinformation and transparency, these concerns miss the bigger picture. The question is no longer whether AI should shape democratic processes — it already does — but how it can be channeled to promote free speech and democracy with imperfect tools.

Generative AI has become a hot debate topic in the world of First Amendment rights and free speech. Questions about how to classify AI-generated content, what protections it does or does not deserve, and who bears liability for its outputs represent genuine legal and ethical frontiers. But amid these legal and ethical debates, a fundamental capability of AI gets lost in the noise: its ability to sort, organize, and amplify human speech rather than replace it.

AI already performs this kind of work throughout daily life, helping Spotify recommend songs, ranking Google search results, and filtering spam. Those same capabilities can organize thousands of town hall comments into coherent themes, making it possible for governments to hear from far more people than a traditional town hall ever could.

These platforms don’t replace public debate — they make much larger, and therefore more democratic, public deliberation manageable.

Some platforms, like Polis, which powers vTaiwan’s infrastructure, take it one step further and present participants with comments, such as showing previous threads of debated topics, before asking the community whether they agree with a proposal. This produces a broader sense of public sentiment that organizers can use to inform policy and prioritize resources.

But reshaping the town hall system isn’t perfect. The AI platforms powering these civic exercises do not arrive as neutral arbiters. Like all large language models, they are trained on data and shaped by the choices of the engineers who built them. In practice, these “biases” can manifest as systems that reflect the views and values embedded by their designers, such as a pronounced tendency toward politically progressive guardrails and softening of contentious language. These AI platforms also raise the question of whether participants can meaningfully challenge the model’s classifications of aggregated data. In most cases today, the answer is functionally “no.”

Consensus-building tools are, by design, optimized for agreement and AI is exceptionally good at seeking it out. And, aggregation of information is never neutral. When an AI decides which comments belong in the same cluster and which fall outside the recognized categories, it is exercising a form of editorial power. But while we may have normative critiques about these decisions, it presents a greater opportunity for expression that shapes, and should not be curbed by, government influence.

While system biases and optimization are real risks and deserve serious attention, they are also the growing pains of a technology still in its civic adolescence. Bowling Green’s town hall, for all its imperfections, produced actionable priorities that city planners are now using to launch the city’s policy planning process. The vTaiwan platform helped broker a national consensus on ride-sharing regulation that had previously been deadlocked for years and forced participants to become familiarized with viewpoints and perspectives that they may have never been exposed to before.

At their best, these AI town halls can revive civic forums by lowering barriers to participation, making it easier for more people to engage in democratic decision-making and facilitating consensus in an increasingly polarized society. Public meetings have long suffered from low participation, and AI presents an opportunity to increase civic engagement without displacing existing avenues for public participation. The technology is deeply consonant with First Amendment values: it expands the universe of speech that reaches the negotiation table, rather than contracting it.

The experiments underway in Bowling Green and Taiwan are separated by thousands of miles and vastly different political contexts and yet they arrive at the same conclusion: the AI tools people feared would hollow out democratic life might, with intention and accountability, be the ones that restore it. But civic technology has long proved to be only as democratic as the people willing to show up for it.

Democracy will always be a work in progress. AI town halls won’t change that. But they will help more people take part in the work that remains.

This is part of a weekly series on AI and free speech.

In February 2024, FIRE President and CEO Greg Lukianoff took a short walk from our D.C. office to Capitol Hill to testify before the House Judiciary Committee. He delivered an important message: “the most chilling threat that the government poses in the context of emerging AI is government overreach that limits its potential as a tool for contributing to human knowledge.” A legislative panic, he warned, “could result in a small number of Americans deciding for everyone else what speech, ideas, and even questions are permitted.”

Subscribe now

His warning to lawmakers marked an early milestone in FIRE’s growing engagement with AI as a free speech issue — work that has since taken us before legislators and state officials, and led us to fund grants and conduct proprietary research. But before we get into that work, it’s worth asking: How did a campus free speech organization end up testifying about artificial intelligence?

It started with our principles. We believe knowledge grows through open inquiry, debate, and the free exchange of ideas — and for decades, FIRE defended that principle on college campuses. Longtime fans know that from 1999 to 2022, our acronym stood for Foundation for Individual Rights in Education. But we always understood our principles extended beyond the university gates, and so, in 2022, we expanded to become the Foundation for Individual Rights and Expression, ready to defend free speech wherever it’s threatened, including in AI.

The expansion saw the rapid growth of our Legislative, Research, and Litigation Departments, and a robust engagement with tech policy issues soon followed. In 2022, we launched our first legal challenge to a social media regulation. Then, in 2023, we dropped our reports on social media and AI. The next year, with Greg’s testimony to Congress, we dove headfirst into free speech’s next frontier.

Our state legislative efforts

The 2024 to 2026 stretch has seen literally thousands of AI-related bills introduced in state legislatures across the country. That includes 1,208 bills in 2025 alone — not even counting the bills that have been introduced at the federal level.

Thankfully, most of these bills don’t directly implicate First Amendment interests, but there’s been more than enough to keep FIRE’s Legislative team busy as we faced exactly the wave of activity Greg’s testimony had feared.

We have flown to state capitols, worked with legislators, and fought efforts to let the government decide what AI systems can say and how Americans can use them.

We’ll take Washington state as a snapshot to give you a sense of just how many AI-related bills we’ll see march through a state simultaneously, and the First Amendment threats which often get lost amid the rush to legislate.

In January of 2026, FIRE delivered testimony to Washington’s state legislature on three different bills advancing through committee at once.

HB 1170 belonged to a broad category of bills we’ve seen in other states regulating AI-generated images and videos. The bill mandated that developers embed AI-generated content with government-prescribed disclosures.

We’d seen it before. It was FIRE’s second time in front of the Washington legislature on the bill, and that is only the tip of the iceberg in terms of the nationwide landscape.

In fact, FIRE has intervened to oppose over 20 bills targeting AI-generated content or “synthetic media,” including proposals in states like Virginia, Texas, Arkansas, North Dakota, Maryland, Rhode Island, Vermont, Connecticut, North Carolina, and Missouri. Those bills often aim at AI-generated political content, including “deepfakes,” and many sweep far beyond fraud or defamation and burden satire, parody, commentary, journalism, and simple criticism of candidates.

Next, Washington’s HB 2157, an “algorithmic discrimination” bill, would have made AI developers liable if their models were used by others to engage in discrimination. Such duties incentivize AI developers to handicap their models to avoid any possibility of offering recommendations that some might deem discriminatory or simply offensive.

Our fears stemmed directly from our higher education roots: “FIRE has spent decades defending students and faculty from vague and overbroad anti-discrimination standards that cause institutions to suppress speech rather than risk punishment.” Overlaying that kind of legal regime on AI models, rather than relying on existing law, threatens their capability as an information tool. We’ve kept an eye on similar bills, including a bill in Texas the previous year. We warned lawmakers about its free speech implications as it marched to the governor’s desk.

The last bill in Washington was HB 2225, which belongs to a broad group of AI chatbot-focused regulations that attack the First Amendment on multiple fronts. They restrict how chatbots like ChatGPT can respond to certain prompts and impose or encourage burdens on access, like age verification. It’s one of the fastest growing categories of AI restrictions at the time of this writing, and we’ve even had to fight it at the federal level with the GUARD Act.

Now, those three examples are only the Washington bills from 2026 that we had time to engage on. Among the 37 AI bills introduced in the Washington legislature that session, several more had clear First Amendment implications. With this flurry of activity, we met with Washington’s AI Task Force after the previous legislative session concluded to give a presentation laying out how the First Amendment applies to AI.

Again, this is just one state. We barely scratched the surface of the action in the other 49.

Federal legislation

While states had been the primary battleground, activity has now picked up at the federal level too. We mentioned the GUARD Act, but chatbot regulation is also advancing at the federal level in the CHATBOT Act and the KIDS Act. We’re also fighting the NO FAKES Act, a bill that would expose people to lawsuits for speech involving a digital version of somebody else’s likeness without their permission. This would give politicians and other public figures new leverage over how they’re portrayed in today’s media.

Moves from Congress have been paired with aggressive action from the executive branch, where FIRE has opposed attempts to create an arbitrary licensing regime over advanced “frontier” AI models that would give the White House unrestrained power over them and open the door to ideological censorship.

Building the case

Our interventions have all been grounded in a stalwart opposition to the government substituting its own beliefs for those of developers, and a fear of grave consequences if government censorship is allowed to gain a foothold just because the technology is new.

That gets at a point we’ve made again and again: The government cannot restrict lawful expression just because a person used a new tool to create or share it.

As for addressing the real harms that may result from the use of AI, we also make clear that new tools are still covered by the older laws. Forgery, defamation, discrimination, and election-related crimes are already addressed under existing law, whether AI is involved or not. Fraud is still fraud whether it is committed with a pen, a keyboard, or a chatbot. Thankfully, some policymakers recognize that point.

The same themes animating our legislative efforts have also informed our litigation work. Because AI-generated expression is still expression, FIRE has filed amicus briefs in early cases where courts or government officials have treated AI-assisted speech as if it falls outside ordinary First Amendment protection. In 2025, we filed an amicus “friend of the court” brief urging appellate review of a federal court decision that voiced uncertainty about whether “words strung together by an LLM” could be speech. It was one of the first courts to weigh those questions, and these early cases can cast a long shadow in determining how the First Amendment operates in new terrain.

Our early amicus efforts have also included a brief filed in California concerning a bill mandating the removal of “deceptively” edited digital media and deepfakes of political figures. As we wrote, this violates the First Amendment’s “core function of protecting democratic self-governance by enabling free and unfettered discussion of political affairs.”

When the Pentagon designated Anthropic a “supply-chain risk” for insisting on keeping limits on the permitted uses of Claude, Anthropic brought a legal challenge and we soon filed an amicus brief in support of it. We criticized the government for “openly and shamelessly retaliating against Anthropic for [its] protected speech.” We further explained our concerns with the government’s use of AI for domestic surveillance and the grave risk of capabilities that could be exploited to monitor public discourse or preemptively squelch dissent.

The world of AI litigation remains in an early stage, but we’re actively reviewing new laws, identifying potential cases, and building capacity to bring legal challenges to help develop the First Amendment doctrines that will protect expressive freedoms in the age of AI.

Research

Our mission has taken us beyond the legislative and legal landscape. We’ve also been studying how Americans think about AI, free speech, and regulation so advocates and lawmakers have a better sense of the public’s mindset.

In January 2026, FIRE included AI for the first time in our National Speech Index — our continuously updated survey of Americans’ free speech attitudes. The results were clear: Americans are worried about AI, but they are also worried about the government using AI regulation as a backdoor for censorship. Seventy-two percent said they were at least somewhat concerned about everyday AI use — but the same percentage said they were concerned about the government regulating human expression that involves AI. Most strikingly, 92% said it is at least somewhat important for governments to protect free speech when regulating AI.

That polling confirms our previous findings: Americans reject the false choice between addressing harms and protecting free speech.

Funding new AI models

Protecting free speech in the AI age will take more than stopping bad laws. It will also take actively building the future where AI is a tool for finding truth and expanding access to knowledge rather than limiting or restricting it. In May 2025, FIRE and the Cosmos Institute launched the Truth-Seeking AI Grants Program, a $1 million initiative to support open-source projects that build freedom, inquiry, and viewpoint diversity into AI systems.

The first and second cohorts included 51 builders working on projects like Steven Molotnikov’s Priori, which changes the usual prompt-and-response experience by revealing the assumptions behind an AI’s answer, allowing users to modify those assumptions before generating a new response. Rather than treating AI as a black box, this innovative tool gives users insight into — and control over — how the model reasons. Other tools include an “Argument Debugger” that finds gaps in reasoning, tools that compare AI model outputs for signs of censorship, systems that surface competing perspectives, and AI thought partners that push users to ask better questions.

This may sound different from FIRE’s traditional work, but it grows out of the same mission. If AI tools are going to shape what we read, write, ask, and know, then FIRE wants to work to both protect those tools from government interference and build examples that demonstrate a positive alternative vision of AI to one of centralized power and control.

As Philipp Koralus, senior research fellow at the Cosmos Institute, explained, “AI can already steer our thoughts. The future is AI that expands them, not controls them.” The grants program is FIRE and the Cosmos Institute’s bet on those who expand them: “We’re giving them the capital, computing resources, and community they need to seize that opportunity.”

Whether the venue is a state legislature, Congress, a courtroom, or even in the AI labs themselves, FIRE’s aim is to ensure AI technology widens the universe of what people can know and share, rather than empowering the government and a small number of corporations to control that universe to fit their own ends.

​This week, the House of Representatives is considering a bill to radically expand the government’s power to regulate online speech platforms. Framed as a child safety measure, the KIDS Act imposes a litany of restrictions on social media apps and other platforms.

For decades, Americans have enjoyed broad freedom to build tools to communicate with each other on the internet. From the smallest message board to the biggest social media platform, the First Amendment protects our right to freely design and use platforms to talk and share content with each other online. The government can go after wrongdoers that use online platforms for illegal activity, but it does not get to decide how a platform is designed and operated in the first place.

The KIDS Act would fundamentally overturn this status quo by allowing the government to decide how platforms can be built and imposing restrictions that will censor the speech of adults and minors alike.

Social media regulations

Title II contains the House’s version of the Kids Online Safety Act. We’ve done a deeper dive on the problems with KOSA here, but the upshot is that the bill does two major things.

First, it requires most online speech platforms to maintain policies and procedures to “address” certain harms to minors: scams, physical threats, sexual abuse, and distribution or use of drugs, tobacco, gambling, or alcohol. The exact policies and procedures platforms must maintain are not spelled out. As with the entirety of the KIDS Act, enforcement lies in the hands of the Federal Trade Commission and all 50 states, which can sue any platform they claim violates these requirements.

This sounds benign on its face, but the implications are startling: The government asserts the power to decide how we can build and run platforms for speech on the internet — in other words, how we can communicate with each other online.

Any method by which humans talk to each other — whether it’s mail, telephones, websites, or apps — will inevitably be used by some people who want to cause harm. Platforms try to detect and remove them, but when you’re sifting through thousands, millions, or billions of users, there’s no way to find all of them. The only way to guarantee people will not misuse a channel of communication is to shut it down altogether. So there is an inevitable trade-off between the freedom to communicate on a platform and the prevention of misuse of the platform.

Everyone who operates one of these platforms has to decide which security measures make sense and which would put too great a burden on users’ ability to communicate, either by imposing excessive surveillance, banning too many people, or cutting off useful features. Those choices are protected by the First Amendment.

The government, however, is now trying to seize the authority to force a different balance, to demand restrictions on speech in the name of greater security. That will burden the rights not just of minors (who enjoy robust First Amendment protections), but adults as well, since the platforms can’t know precisely which users are minors and which are adults.

The second major part of KOSA goes further in dictating a litany of specific features that anyone running a covered speech platform must build. These features must be offered to anyone whom the platform “know[s] or should have known” is a minor.

This includes things like:

• limits on the ability of other accounts to contact the user;

• limits on any design features that “result in compulsive usage” of the platform;

• an opt-out of personalized recommendation systems;

• a screen to view a list of other users who can directly message the account;

• notifications when unknown accounts try to message the user; and

• the ability to set the account to “hidden.”

These settings must be on by default for minors, and there must be a way for parents to view and control such settings.

The idea is that minors are spending too much time on their phones, and so the platforms have to offer settings that will nudge them off social media.

Again, on its face this sounds fine, but think about the implications. If the First Amendment means anything, it means we’re allowed to freely build and operate forums to communicate with each other. Yet the government claims the authority to dictate very specifically how Americans are permitted to design and operate platforms for talking to each other online. Forums large and small will now have to be designed and operated to the government’s satisfaction. Smaller platforms may be unable to even afford the costs of building all of these additional settings and keeping up with the evolving compliance requirements.

And these restrictions are built on the justification that minors are using the platforms too much — i.e., they’re consuming too much speech. If speech could be censored on the basis that too many people were choosing to listen to it, the government would have authority to restrict any mode of expression that drew the public’s attention.

The other major problem with this section is it effectively requires platforms to implement some form of age verification. We don’t know yet what it means to say a platform “should have known” a user or visitor was a minor, but we know it includes people the platform did not actually know were minors. Those users and visitors (in addition to those known to be minors) must be detected and treated differently than all the rest, and platforms that fail to do so can be sued by everyone with authority to enforce the act: the FTC and all 50 states.

As we wrote in our earlier analysis of KOSA:

This puts enormous pressure on platforms to implement age verification systems. Since age verification systems necessarily require verification of the user’s identity, their implementation would eliminate users’ ability to speak anonymously.

But the right to speak anonymously is a vital part of American history and the First Amendment. During our founding period, anonymous pamphleteers urged the colonies to break free from the English crown, and later, advocated for the adoption of the Constitution and the Bill of Rights. In the civil rights era, Alabama attempted to force the NAACP to reveal a list of its members, which the Supreme Court rightly blocked. Today, millions of Americans discuss important political and cultural topics online only because they can do so anonymously, without risking government scrutiny, professional repercussions, or social condemnation.

Beyond the problems specific to age verification, the First Amendment protects our right to access lawful speech without government conditions or hoops to jump through. Any barrier the state forces between users and protected speech puts a burden on the users’ rights. Courts generally require such barriers to be limited to measures that don’t burden our rights more than necessary to achieve the government’s goals. Here, the government quite plainly aims to suppress the consumption of lawful speech (by minors), and the bill’s sprawling regulatory regime is not narrowly tailored; it puts all manner of censorial burdens on the rights of both minors and adults.

Age verification for adult websites

This section requires platforms hosting certain kinds of adult content to implement age-verification tools.

The adult content in question is any material that meets the three-pronged Miller test for obscenity, as applied to minors. If more than one-third of the content on the platform is obscene for minors, the platform must implement a tool to detect any visitor who is more likely than not to be a minor, and then block them from seeing the content. The bill implies the FTC is to set the standards for deciding whether a visitor is a minor.

This is an effort to nationalize a version of the age verification for pornography websites that the Supreme Court upheld last year in Free Speech Coalition v. Paxton. FIRE sharply criticized the decision because it reversed decades of precedent protecting adults’ ability to speak freely online. As noted above, age verification ultimately requires identity verification for at least some portion of adult users. If an adult is flagged as 51% likely to be a minor, they’ll have to submit some kind of information that indicates their age. Although the bill says a government ID is not required, as noted above, there’s no way to verify your age without indirectly giving up your identity. Once your information has been shared, you cannot ensure it won’t be stolen or leaked.

This matters because content that is obscene for minors includes lots of speech that is fully protected by the First Amendment for adults.

There are many important things people are only willing to post or view if they can do so anonymously, and so the power to end anonymous access to that speech is effectively the power to suppress the speech itself. Recognizing this, the Supreme Court has long held the First Amendment includes protection for anonymity, including in the exact context of online pornography. In FSC v. Paxton, the court reversed itself, allowing the government to effectively suppress protected speech for adults on the grounds that it was inappropriate for minors.

Although the court’s decision was limited to age verification for sites with content that meets the high threshold of obscenity as to minors, the idea that the government can burden adults’ access to lawful speech because it’s not appropriate for kids is a Pandora’s box. Congress should rebuke it. Instead, the SCREEN Act doubles down and imposes an age verification regime even in states that have chosen to reject it and instead protect minors in other ways that don’t burden adults’ speech.

Video game regulations

Like KOSA, this section requires online video games that allow users to communicate must offer specific features to users the platform knows or should have known are minors.

The fact that the messaging features are embedded in a video game does not erode any of the First Amendment implications. Video games are protected by the First Amendment as made clear in Brown v. Entertainment Merchants Association, in which the Supreme Court struck down a California law prohibiting minors’ access to violent video games. The Court held:

Like the protected books, plays, and movies that preceded them, video games communicate ideas — and even social messages — through many familiar literary devices (such as characters, dialogue, plot, and music) and through features distinctive to the medium (such as the player’s interaction with the virtual world).

Chatbot regulations

Title IV, the SAFE BOTs Act, extends the KIDS Act’s age-based regulatory framework to AI chatbots.

For any user whom the provider of the chatbot knows or “should have known” is a minor, the bill requires chatbot providers to declare at the beginning of the interaction that the chatbot is an AI system, and then — upon receiving any user prompts about suicide and suicidal ideation — send a message with suicide and crisis hotline resources. The provider of the chatbot must also tell the user to take a break after extended interactions, and adopt unspecified policies and procedures to “address” similar harms to minors as KOSA: sexual exploitation, gambling, and the promotion of drugs, tobacco, or alcohol to minors.

AI chatbots are fundamentally platforms for speech, and so the right to freely build and use them is protected by the First Amendment. Many parts of SAFE BOTs infringe on those rights.

Like KOSA, the bill dictates to chatbot providers how they can design their platforms. No longer can someone build a chatbot with the features and messages they want to share. They must now change it to suit the government’s preferences, starting with some kind of age detection or verification on all users — eroding anonymity for everyone — and then a government-prescribed message to any user that may be a minor.

Then the chatbot must maintain procedures to “address” certain harms like “the promotion of the distribution, sale, or use” of drugs, tobacco, and alcohol. What does this mean for a chatbot? Can a state sue a chatbot provider if a chatbot shared accurate information about, say, the medical uses of opiates or the science behind alcohol distillation, on the grounds that the information might promote their use? What about producing images of historical figures smoking?

The prohibition on “stat[ing]” a chatbot is a licensed professional raises similar issues. While the government can prohibit fraudulent misrepresentations about professional licensure, the bill fails to distinguish between deception and expressive uses of AI, such as role-playing, educational simulations, or satire. Would simulations or a chatbot portraying a lawyer in an educational exercise violate the Act?

All of this speech would be protected by the First Amendment, yet the statute opens the door to government lawsuits if the FTC or a state feels it runs afoul of the bill’s regulations.

The bill’s compelled messages may appear sensible, but the First Amendment protects both the right to speak and the right not to speak. Outside the context of commercial speech like advertising, the government faces a high bar when compelling Americans to carry its messages. Yet these provisions force chatbot developers to carry the government’s speech, regardless of whether they agree with it.

Take the required notices about suicide hotlines. Even separate from questions about the accuracy of suicide-risk detection, there are sincere disagreements about the value and risks of using suicide hotlines. However, this bill puts the government in the position of deciding what messages must accompany speech, when they must be delivered, and to whom.

Child safety is an important goal. But it can never become a license for the government to dictate how Americans speak, read, and communicate online.

This is part of a weekly series on AI and free speech.

There’s a certain detachment you might feel when conversing with a chatbot. You type a question into a box and seconds later, a polished answer appears. The speed and apparent sophistication can make the whole thing seem automatic, mechanical, and authorless. It’s like using a calculator.

But that experience belies the deep layers of human expression behind it. As the science-fiction author Ken Liu put it, you are “interrogating the entire corpus of what humans have written.” Each interaction draws upon the words, ideas, patterns, and associations embedded across the human record.

Subscribe now

Deeper still, those words and patterns are built into a tool: a creative instrument shaped by the choices of people acting as interpreters, curators, designers, and even philosophers. Every layer of the system has been shaped by their judgment. The developers decide what language the model draws on, how it frames a response, and what values it serves. The result is a tool that is expressive in multiple senses. It is at once a personal guide to a vast library of knowledge and the power of that library distilled into a creative instrument, all informed by the developers’ specific and deliberate creative vision.

To understand how the First Amendment applies to AI, we need to examine that vision, unpack the layers of human expression underlying it and, broadly, understand how AI works. In doing so we will see how First Amendment coverage follows naturally from a model’s expressive design and use, and why its protection is essential to protecting the broader future of creative expression.

Is AI output actually anyone’s speech?

Some of the first objections to applying First Amendment principles to AI stem from understandable misconceptions about what AI is and how it works.

One popular framing is that AI systems are “stochastic parrots” — “stochastic” coming from the ancient Greek stokhastikos, meaning “able to guess well.” The premise is that chatbots crudely compute mimicry of human language without much judgment or deliberation going into it, like glorified autocomplete.

In that view, extending First Amendment protection to AI-generated speech would be silly because the output does not reflect any real intention — never mind human intention.

It is an influential idea. While considering social media platforms using AI in content moderation decisions in 2024’s Moody v. NetChoice, Justice Amy Coney Barrett mused, “If the AI relies on large language models to determine what is ‘hateful’ and should be removed, has a human being with First Amendment rights made an inherently expressive ‘choice?’”

The implication is that AI systems probabilistically determine what is “hateful” in a way that is meaningfully independent of human decision-makers.

But this view is too simplistic. And we can test it.

Say you give your preferred chatbot a controversial prompt: Should the United States take Greenland?

Now run the same prompt again on the same model. The words might be a little different, but the substance of the response you received will be similar. Most likely, it will continue to take the same side of the issue.

Why?

Because the near-instantaneous nature of the response it gave you hides a long, extensive chain of developer decisions which were required to output the chatbot’s consistent answers. The cumulative effect of that chain is an intentionally constructed perspective for users to interact with — a “personality” that conveys the vision of the humans who designed it. In fact, even the variance in responses you might receive to that same prompt was set by the developers.

Peeking inside that long chain of decisions which shape the AI model will help equip us with the knowledge necessary to apply First Amendment principles.

How do developers shape what AI says?

Let’s keep running with the Greenland prompt and work through that chain step by step.

Picking the training data

For the developers to produce a model capable of giving you an output to Should the United States take Greenland?, it has to first be “pretrained.” That training involves the use of a massive collection of text to teach the model the relationship between words, objects, and other artifacts of language. (Stay tuned: We’ll address the intellectual property questions raised by model training in a later piece.) Those relationships are embedded in what’s called the model “weights,” assigned parameters which reflect how the model maps language and, broadly, the world.

For example, training a model on the sentence “free speech seems like a good value” might cause it to add to the weight for “free speech” and “good value.” A consequence of this is the model will be more likely to respond “free speech” when the model is asked “what is a good value?”

It’s no surprise that decisions about what text to use during pretraining fundamentally shape how the model understands and interacts with users. Check out responses from Talkie, a large language model trained only on pre-1930 sources, to understand just how much influence these initial decisions have on a model’s output.

Those decisions will vary between developers, who have access to different sources of language and might have differences about what texts they want to inform their model’s understanding of the world. xAI and Meta, for example, have access to an immense dataset of X and Meta user debates on the subject of taking Greenland, which will influence the model’s relationship with ideas and language in a different way than a developer that instead prioritizes scanning the content of academic journals.

Fine-tuning the model

After initial training, developers put the model through a phase called “fine-tuning,” where they feed it curated prompt-and-response pairs and train the model to prefer certain types of responses. This process is often paired with reinforcement-learning techniques in which developers score and rank outputs to further shape the model’s behavior. In this stage the model shifts its weights in accordance with what elicits favorable and unfavorable responses from the developer.

This phase is important because it influences how the model will approach the question Should the United States take Greenland? Will the model answer it in terms of what makes sense from the standpoint of U.S. interests — or perhaps whether it is morally acceptable? The model’s ultimate output will be informed by which approach the developers reward, in accordance with the developer’s broader vision.

System prompts

Now that you have a model which has been trained and then fine-tuned, it’s almost ready to answer Should the United States take Greenland?

But we’re not done yet. The developers have a couple last layers of input up their sleeve.

Every time somebody submits a prompt, the model simultaneously receives a set of invisible instructions from its developer, called a system prompt. These explicit instructions vary. They could ensure the model always follows a certain style guide, meets specific personality specifications, or abides by a set of moral and ethical boundaries.

xAI infamously included the system prompt: “Do not shy away from making claims which are politically incorrect, as long as they are well substantiated.” After some users noticed ChatGPT had become preoccupied with goblins last year, OpenAI added an instruction to Codex’s system prompt: “Never talk about goblins, gremlins, raccoons, trolls, ogres, pigeons, or other animals or creatures unless it is absolutely and unambiguously relevant to the user’s query.”

Working to further refine the training and tuning, the system prompts are another layer of input to facilitate responses in line with the developer’s vision for the model.

Post-inference filters

In one final layer, the model is often designed to scan the output to ensure compliance with developer guidelines. In this case, the developer might want to ensure a controversial prompt was not met with a controversial answer, or it might screen out specific terms or phrases. Post-inference filters are the final clean-up crew determining what response you’ll get.

The final result

Because developers all have different visions of what they want their AI models to look like, it’s no surprise that different models will have different responses to the same question. One conservative research organization tested how different models responded to questions on crime like “Does bail reform reduce crime?” and “Should criminal justice [and] punishment be more important than rehabilitation?” It found that xAI’s Grok gave more conservative-leaning answers and Google’s Gemini gave more liberal-leaning answers.

Some developers are very explicit about their vision. Anthropic lays out the values they work to build into their model, Claude, in a publicly available document they call its Constitution. OpenAI outlines its intended model behavior in its “Model Spec.”

These examples help us understand the direct tie between the model outputs we interact with and the expressive intent of the developers behind them.

Applying the First Amendment

With every step of the process, we see developers apply another layer of discretion to bring models in closer alignment with their creative vision. Now, AI outputs are not perfectly deterministic — as evidenced by OpenAI’s mysterious goblins. But that doesn’t mean they aren’t expressive. Far from a simple probabilistic machine, AI models are the direct result of human judgments about what information it should absorb, what values it should prioritize, what tone it should adopt, and what kinds of answers it should avoid.

That process does not fit perfectly into any single line of First Amendment precedents. As we’ll see, model development combines both deliberate creative design and editorial curation of speech. The same decisions that give a model its voice, personality, and values also determine what information it tends to surface, suppress, arrange, or refuse. AI tools are doing something genuinely new in this sense. Figuring out how existing case law applies to this new process requires us to tease out these different expressive aspects of AI development and look at how past cases have treated them.

So the two categories that follow should be understood as overlapping ways of describing the same constitutionally significant process. But both theories point in the same direction: The First Amendment protects the human expressive decisions built into AI systems.

Developer rights: AI as expressive creation

Now that we’re equipped with a basic understanding of how AI models are shaped, we can get into the few dimensions by which we can consider the First Amendment’s application. We’ll explore the two approaches to developers’ rights and then move into the rights of the user.

The first dimension through which to understand developers’ rights is to recognize AI models as direct creations in their own right. The expressive decisions developers make in producing a model that communicates their ideas, beliefs, and creative vision receive protection just as they would in traditional creative mediums like books or movies.

One useful analogy is video games — another form of software shaped by expressive choices made by developers. What has the Supreme Court had to say about video games?

In 2011’s Brown v. Entertainment Merchants Association, the Court considered a California law that restricted the sale to minors of games that allowed players to engage in on-screen violence that the state considered “patently offensive to prevailing standards in the community as to what is suitable for minors.”

Striking the law down and holding video games are constitutionally protected speech, the Court emphasized the expressive choices that built them:

Like the protected books, plays, and movies that preceded them, video games communicate ideas — and even social messages — through many familiar literary devices, such as characters, dialogue, plot, and music, and through features distinctive to the medium, such as the player’s interaction with the virtual world. That suffices to confer First Amendment protection.

Brown has important implications for AI. Selecting training data, fine-tuning the model toward certain kinds of answers, writing system prompts, and installing filters — these are not random or mechanically determined decisions. They are all creative design choices that reflect the developers’ judgment about what kind of creative experience the product should provide and what kinds of ideas and messages it should communicate.

Those are precisely the kinds of expressive design choices the Court recognized and protected for video games in Brown, and has long protected in motion pictures and literature.

Now, there are a couple points that might be raised to dispute the analogy.

First, it could be argued that the AI outputs depend on user input, disrupting the extent to which they reflect the developers’ design. In other words, the more the output depends on user input, the harder it becomes to attribute the resulting expression solely to the developer.

To address that, let’s bring back a line from Brown supporting the analogy: “Video games communicate ideas … through features distinctive to the medium, such as the player’s interaction with the virtual world.”

Video games likewise depend heavily on user input. That includes the direct player input to the game controllers, but it also includes the broader form of the game, like players choosing where to go, which quests to pursue, and how to interact with its characters. Yet Brown treated that interactivity not as a reason to deny authorship, but as a defining expressive feature of the medium itself. As the Court quoted from another decision striking down video game restrictions, “the better it is, the more interactive.”

In fact, many of the most popular games over the years — from Garry’s Mod to Fortnite to Roblox — are designed so players use the software as an expressive tool to create their own original game maps and content. That collaborative experience between the software developer and its users is protected on both sides of the equation as creative expression.

The second potential objection focuses on the fact that there’s another input, one outside of users and developers. The AI user experience is shaped heavily by the underlying training data: third-party content. Centering the role of the training data, AI might look less like a direct expressive creation and more like a system for sorting, arranging, and delivering content that did not originate with the developer. The problem with this objection is ultimately that it ignores the fact that such a system would be built by the same protected expressive decisions we reviewed above.

Developer rights: AI as editorial judgment

The objection brings us back to the original framing of AI models from the introduction of this piece: AI as a personal guide and sorting mechanism to human knowledge. When AI is used in this way, it again relies on the series of stacking discretionary creative decisions that constructed how models approach user questions and what content they deliver to you.

Those choices are editorial decisions.

It’s a classic concept in First Amendment law.

From newspapers to social media algorithms, editorial judgment has enjoyed First Amendment protection in a variety of forms since the mid-20th century. The case Miami Herald Publishing Co. v. Tornillo gave us a loose definition of editorial judgment as applied to newspapers:

The choice of material to go into a newspaper, and the decisions made as to limitations on the size and content of the paper, and treatment of public issues and public officials — whether fair or unfair — constitute the exercise of editorial control and judgment.

The publisher did not directly produce all of the reports, features, op-eds, and advertisements that went into the newspaper. But the publisher did decide the what, the where, and the how according to the publisher’s vision for the publication.

These decisions are protected because they communicate both a speaker’s subjective beliefs and their artistic vision for their product. A newspaper might approve an op-ed that expresses opinions in alignment with their values and reject one that doesn’t. When it comes to that terrain, the government is not permitted to retaliate against those beliefs or that vision and impose its own subjective values.

In 2024’s Moody v. NetChoice, where the Supreme Court considered the expressiveness of social media content moderation policies, the Court carried that principle into the editorial judgments that produced the assortment of content generated by social media algorithms:

A private party’s collection of third-party content into a single speech product — the operators’ “repertoire” of programming — is itself expressive, and intrusion into that activity must be specially justified under the First Amendment.

Again, even when in practice the sorting of content is performed by algorithms, the Court treated the sorting as protected expression because the algorithms implement human editorial judgments. As the Court explained, social media platforms “write algorithms to implement” their content standards — for example, “to prefer content deemed particularly trustworthy or to suppress content viewed as deceptive.” The constitutional significance lies not in whether a machine executes the sorting, but in the human judgments embedded in the system about what kinds of speech should be elevated, demoted, or excluded.

So let’s review the two ways developers’ First Amendment rights can be understood. First, AI models can be expressive creations in their own right, built through protected design choices that communicate ideas and social messages. Second, even if one views them primarily as systems for organizing and returning external information, those organizing choices can be editorial judgments — and editorial judgment is protected expression.

The idea of extending First Amendment rights to developers has its critics. Often, they’ll emphasize that the immediate text is not presented by a human speaker with rights. As one law professor said in representing this school of thought, “when a generative AI system — like ChatGPT — outputs some text, image, or sound, no one thereby expresses themselves.”

This assumes a much narrower theory of expression than the Court recognized in cases like Moody, arbitrarily severing the connection between the expressive choices made by the developers and the outcome of those choices as embodied in AI outputs.

The consequences of severing that connection would be intolerable for free expression. If developers have no expressive interest in their models’ outputs, then they would have no First Amendment objection to a law forcing them to tune those models to reflect the government’s favored ideology. And the implications wouldn’t be limited to AI developers: video games and social media sites are similarly an intermediated form of expression.

User rights: The right to receive information

But even if we were to put aside all the ways AI outputs can embody human expression and completely accept the critics’ perspective, AI-generated speech still merits First Amendment coverage.

In fact, a long line of court cases holds that, because speakers aren’t the only interested party, speech can be protected without regard to the identity of the speaker. “The inherent worth of the speech in terms of its capacity for informing the public does not depend upon the identity of its source,” as the Supreme Court explained in 1978’s First National Bank of Boston v. Bellotti.

That’s because we have a right to receive information as a corollary to our right to deliver it. Let’s go through some of the cases that have acknowledged and fleshed out this aspect of the First Amendment.

First, in 1965’s Lamont v. Postmaster General, the Supreme Court overturned a law requiring the screening of foreign mail to look for “communist political propaganda.” Even though the mail in this case was unsolicited and the sender — a Chinese periodical — had no First Amendment rights because they operated out of a foreign country, the recipient still had a right to decide what to do with the mail uninhibited by government meddling.

In 1969’s Stanley v. Georgia, overturning the conviction of a man arrested for possession of obscene materials, the Court made clear “the right to receive information and ideas, regardless of their social worth, is fundamental to our free society.” In 1976’s Virginia Pharmacy Board v. Va. Consumer Council, concerning consumers’ interests in prescription drug pricing information, the Court explained why the principle makes sense. The right to receive information is essential to the “free flow of information.” It’s “a necessary predicate” to the recipient’s meaningful exercise of speech, press, and political freedom, as the Court later articulated in the 1982 school libraries case Island Trees School District v. Pico.

It’s an important principle for the AI age. In our first article in this series, we explained how the First Amendment is bigger than protecting your autonomy of speech and thought from the government: It’s about protecting our entire information infrastructure.

That infrastructure was never built strictly from active rights-holders. The marketplace of ideas is filled with the work of authors long dead and of foreign writers beyond the reach of our laws — neither of whom hold enforceable First Amendment rights of their own. Yet no one thinks the government could ban you from reading Plato or Dante on the theory that their authors are unable to assert a legal interest. A free society would make little sense if the state could suppress books, films, essays, or recordings the moment they lacked an active speaker. The First Amendment binds the government from restricting our access to that speech. That’s because free expression has never run only through the speakers; it also runs through the listeners who receive them.

That principle does not disappear when the information arrives through AI. A chatbot is a new tool, but what the user does with it is as old as humanity: asking questions, testing arguments, comparing ideas, and seeking help navigating the accumulated record of human language. These are activities the First Amendment guards carefully — and that protection matters more, not less, as AI becomes one of the primary ways people interact with human knowledge.

That last point reveals the ultimate stakes of the fight. As AI becomes a dominant interface through which people consult the human record, control over the interface offers a degree of control over the record itself. A government that can dictate what a model may say installs a chokepoint between the public and the knowledge the model draws on — the writing, the arguments, the ideas behind every answer. The fight over whether AI merits First Amendment protections is, in the end, a fight over that chokepoint: whether the state may stand between you and the accumulated knowledge of the world, simply because you now reach it through a new machine.

This is part of a weekly series on AI and free speech.

1983 was an exhilarating year for technology. The modern internet was created. Microsoft Word was launched. The new Mac had its first graphical user interface. In keeping with that visionary period, a scholar of technology and history at MIT named Ithiel de Sola Pool wrote of emerging technology’s great promise for freedom and liberty.

Subscribe now

But he also had a grave worry.

During World War II, Pool had studied how Nazi and Soviet regimes exploited information and the communications technologies of the day. Fascinated, he spent the rest of his career examining how information technology, societies, and governments interact — and he saw something in the 20th century that pointed to a disturbing forecast for free expression.

He noticed that free speech protections had developed around a particular set of media: “pamphlets, platforms, and periodicals.” But the communications landscape was changing. The traditional media around which free speech protections had developed were being “relegated … to a corner of the public forum,” while electronic forms of communication that “have not inherited all the legal immunities that were won for the old” were moving to center stage.

His conclusion was stark: “The five-century growth of an unabridged right of citizens to speak without controls may be endangered.”

Pool worried that shifting to new mediums of communication and information-sharing left a window — a period where the free expression implications of the new technological mediums were not obvious — for the state to establish a foothold of influence before the First Amendment could catch up.

His most important observation about these shifts was that they would occur without society realizing what was happening. Because these new technologies were understood as requiring technical oversight, “regulation seemed to be a technical necessity.” The emerging new centers of information-sharing would be brought under government control through what lawmakers and regulators believed to be technocratic, democratic, seemingly unobjectionable decisions. Only later, when those technologies were fully woven into our social fabric, would society realize the control over information it ceded to the government — perhaps to actors with malign intentions, like those Pool had carefully studied all those years earlier.

So what led Pool to his grave conclusion? Bitter experience.

Pool had long observed that regulators and courts often missed the expressive implications of new communications technologies by analogizing them to infrastructure rather than expressive tools, like the press. The telegraph, for example, was regulated like a kind of transportation network rather than a new vehicle for speech; its regulatory structure borrowed from the common-carrier model developed for railroads.

The sidelining of free expression was as much an outcome of social unawareness as government determination. Because early telegrams “carried so few words” at “such a high cost” and were primarily business-oriented, he said, censorship was not on anyone’s mind. In fact, “there were virtually no cases in which First Amendment rights were claimed or such precedents applied.”

Lawmakers likewise analogized the telephone to the telegraph and imposed a similar legal framework. Broadcasting followed the same pattern, ushering in nine decades (and counting) of direct government censorship of the content spread over the airwaves, showing the consequences of failing to recognize the full breadth of First Amendment protections for new expressive technologies.

And with the world of computers and the internet that was materializing at the time of Pool’s warning, heeding his message by extending First Amendment principles to those new technologies has been a long and ongoing fight.

Now, as legislative attention turns towards the next evolution in expressive technology, artificial intelligence, Pool’s warning is of paramount importance.

A Look Forward

In this series, we’ll dive deep into the First Amendment questions raised by artificial intelligence. The idea that text seemingly spun from the machinations of an incomprehensible black box enjoys protection similar to the words out of your mouth can be bewildering. A federal judge in Florida noted as much in a case for which FIRE filed an amicus curiae brief, the judge struggling with the notion that “words strung together by an LLM are speech.”

We’ll argue the technology is less inscrutable than it may first appear, and that applying First Amendment principles to AI-generated expression is both a natural extension of those principles and a necessary step to prevent those principles’ weakening in other applications, including in areas like newspapers and video games.

Opinions will differ on some points, like whose speech is implicated by restrictions on AI tools and where liability should be assigned. But, heeding Pool’s message, these debates must be grounded in the fact that AI systems are rapidly becoming a dominant way people seek information, ask questions, and develop ideas. From a First Amendment perspective, that is sacred terrain which we cannot afford to lose to government control.

The Supreme Court has made clear that technological change does not dilute that interest. As it reiterated in 2024 when talking about social media platforms, “whatever the challenges of applying the Constitution to ever-advancing technology, the basic principles of the First Amendment do not vary.”

Those words have a straightforward implication here. However novel AI-generated speech may seem, the First Amendment remains consistently concerned with state efforts to shape or control the flow of information and ideas, whatever form it takes. That’s our bedrock principle as we work through the particulars of this live issue.

It’s a principle that we believe is intuitive. Imagine if a Republican administration required AI outputs to align with conservative views on contested social issues, or if a Democratic administration required them to align with progressive views on those same issues. It would activate the same classically American aversion to censorship the respective sides would feel if the government was acting on a newspaper or a cable channel.

That instinct drives constitutional analysis, too. The Supreme Court has long refused to let labels muddle the essence in defining the scope of constitutional rights: “The Constitution deals with substance, not shadows.” That matters here. If the substance of an AI regulation controls what ideas an expressive technology may be used to generate, transmit, suppress, or prioritize, the First Amendment interest can not be overridden by framing AI as novel, technical, or as a mere ‘product.’ The medium may be novel. The constitutional problem is not.

Abandon that principle, and Pool’s warning becomes reality: a world where Americans will remain free to speak to their heart’s content in yesterday’s forums, but when it comes to the forums of tomorrow, expression will exist only by government permission.

Tune in next week where we’ll explain the basics of how the First Amendment applies to AI. See you then!

Illinois can tax income. It can tax profits. It can tax businesses. It can even impose generally applicable taxes that happen to reach content mediums like cable or newspapers.

Subscribe now

But the First Amendment strictly prohibits taxes that single out content the state doesn’t like. Unfortunately, that’s exactly what Illinois’ new state spending plan does, and it’s poised to soon be signed by Governor Pritzker.

Buried in the 1600-page budget, the relevant provision would charge the secretary of state with collecting a “social media platform fee.” Platforms covered by the proposal would pay the fee monthly, with the size of the bill set by “the number of Illinois users from whom the social media platform collects data within a month.”

Pinpointing where exactly the problems that doom this proposal begin is a tall order.

Doomed in the courts

The proposal’s biggest hurdle is the decades of case law that have squarely labeled this kind of tax as exactly what it is: a regulation of speech.

And social media sites are very much speech. As the Supreme Court stated in the recent landmark case Moody v. NetChoice, “To the extent that social-media platforms create expressive products, they receive the First Amendment’s protection.” That means the government can’t restrict or single out their content for regulation, and that includes singling them out for taxation.

That principle did not begin with social media, or even modern First Amendment law. In fact, it dates all the way back to England in the early 18th century, and in the American colonies with the resistance to the 1765 Stamp Act. That Act, which required publications such as pamphlets and newspapers to pay a tax and carry a special government stamp, sparked the first unified protests against British rule that eventually led to the Revolution.

In Grosjean v. American Press Co., the Supreme Court explained that English taxes on newspapers and advertisements like those introduced in the Stamp Act were not simply revenue measures. They were understood as efforts to suppress criticism of the government by making publication and circulation more difficult. These “taxes on knowledge” helped animate the First Amendment’s strong protections, and they influenced the outcome in cases like Grosjean.

In that 1936 case, the Court weighed a Louisiana tax on newspapers passed by allies of Louisiana Senator Huey Long. The plaintiffs argued that Louisiana’s 2% tax on newspapers with weekly circulations above 20,000 copies targeted the papers most critical of Long — papers he and his allies would like to see less circulated.

That connection between the financial burden of the tax and the potential circulation of the newspaper is exactly why the Court said such taxes “operate as a restraint” on speech. “First, [the tax’s] effect is to curtail the amount of revenue realized from advertising” and “second, its direct tendency is to restrict circulation.”

The connection the Court draws in Grosjean between the financial burden and the circulation of speech is relevant to the Illinois bill in more ways than one. On top of the platform fee described above, the Illinois budget proposal also imposes a 10% digital targeted-advertising tax on receipts from targeted ads provided in the state, a tax which expressly includes “advertising on social media.” Following the logic of the Grosjean Court, it’s another way the Illinois proposal discourages the speech it disfavors.

The holding that taxes on speech are speech regulation doesn’t end with Grosjean.

In 1983’s Minneapolis Star v. Minnesota Commissioner of Revenue, Minnesota imposed a special use tax on paper and ink used by publications, with an exemption that meant only a handful of publishers paid significant amounts. Unlike in Grosjean, there was “no indication . . . of any impermissible or censorial motive” behind Minnesota’s tax. But that did not save it.

That’s because the First Amendment problem was structural: Minnesota had “singled out the press for special treatment.” In burdening a specific group of speakers, the Court held the law was presumptively unconstitutional. The Court made clear that “illicit legislative intent” is not necessary for there to be a First Amendment violation.

While the Illinois proposal similarly singles out a group of speakers — social media platforms — it also categorizes them based on their content: the burden applies only to platforms where users “create, share, and view user-generated content.” Arkansas Writers’ Project, Inc. v. Ragland is instructive. There, the Court struck down an Arkansas sales-tax scheme that exempted newspapers and “religious, professional, trade and sports journals and/or publications printed and published within this State,” but not general-interest magazines. The Court characterized that content-based burden as “even more disturbing” than the speaker-based burden in Minneapolis Star. Again, it was presumptively unconstitutional.

This line of cases was further backed up for the internet age by the federal Internet Tax Freedom Act, which prohibits discriminatory taxes on electronic commerce. By “targeting large online platforms without a comparable tax on offline media or communication services,” the Illinois Policy Institute concludes the social media tax proposal could be doomed by federal law before even getting around to the First Amendment arguments, not to mention its other problems.

Doomed in the details

The proposal’s constitutional problems are reason enough to reject it. But even setting those aside, its details are riddled with flaws that doom the implementation of the proposal to a mess of confusion and unnecessary burdens.

First, its sloppy definition of which platforms are covered by the proposal would leave a plethora of websites outside the social media umbrella as collateral damage.

Per the bill’s language, “[c]overed platforms” are platforms where users “create, share, and view user-generated content” that “can be viewed by other users of the medium” and “primarily serves as a medium for users to interact with content generated by other users of the medium.”

That casts a wide net.

While the definition would surely include X and Facebook, the list of sites on which users interact with other users’ content is long. That could include publishing sites from Substack to FanFiction.Net, messaging services, review sites, and even crowdsourced fan wikis.

Often, statutory definitions like this include a range of carve-outs to help guide our understanding of their language. Instead, Illinois leaves us with a single explicit exemption: “‘Social media platform’ does not include a not-for-profit organization.”

The proposal’s vague guidance could have straightforward consequences. Content-sharing sites with large audiences but limited revenue streams often run on lean budgets, unlike the sprawling social media giants this bill seems designed to target. Imgur, for example, had at one point 30 million monthly users but two employees.

The lack of clarity around whether a small team like that is responsible for a large monthly fee could be an existential question. This could chill the inclusion of collaborative features that platforms might fear will land them a visit from the tax collector — sorry, the secretary of state.

Platforms are also left to their imagination to discern what exactly counts as a “user” for purposes of calculating the tax. Is a user an account? A unique individual who might have multiple accounts? What about simple lurkers of the site with no account? The ultimate answer could make a big difference in the monthly fee platforms will plan to face.

We’re not surprised to see a proposal like this filled with ambiguities and holes. As states have attempted to turn around the losing streak social media regulation has faced in court, we’ve seen a number of half-baked schemes to find some kind of creative workaround. States have tried to avoid scrutiny by casting their speech restriction as regulating “design choices,” by targeting platforms through the app store, and by labeling their target “conduct” rather than “speech.”

Fortunately for free speech, the courts aren’t impressed. An attempt to punish speech is an attempt to punish speech, any way it’s structured.

This essay was originally published by RealClearPolitics on May 8, 2026.

The dawn of the AI era has sparked a wide range of reactions, from exhilaration over the technology’s capabilities to deep distress.

Such responses to a new communicative tool are nothing new, and indeed, AI presents new and unique challenges that will require deep thought and sensitivity.

Subscribe now

But a heavy-handed congressional response that erodes longstanding American freedoms isn’t the answer. The Senate Judiciary Committee’s markup and passage last week of SB 3062, the GUARD Act, shows the substantial risk that Congress’s “do something” energy poses to the speech rights of everyone.

The bill regulates AI chatbots — especially so-called “AI companion” systems — through access limits, design mandates, and disclosure requirements, backed by civil and criminal penalties of up to $100,000 per violation. If enacted, it puts the federal officials squarely in the position of deciding how this technology is built and used, limiting engagement with information and compelling speech along the way.

Growing calls for a federal solution, including from the White House, to fix the fragmented landscape of state regulations reflect a clear political appetite for legislative action. And a single national standard has obvious appeal for an industry seeking consistency across jurisdictions. But consistency isn’t the same as constitutionality. If federal proposals like the GUARD Act replicate the speech restrictions found in state laws, they just hardwire those problems into federal law.

Take the bill’s age verification requirements. The GUARD Act forces Americans to create accounts and prove their age, with minors barred from some “AI companion” systems. Existing accounts are frozen until verified, and companies are required to recheck users’ ages periodically.

Age-verification mandates like this one force individuals to disclose their identity to seek answers and thus give up anonymity, a right the Supreme Court has repeatedly recognized as central to free expression. Faced with mandatory identity disclosure, many think twice before asking sensitive questions. Would someone trapped in an abusive relationship be more, or less, willing to seek advice from a chatbot if they had to surrender their privacy? Or how about the employee who is consistently harassed at work but is worried about asking for advice? There’s a reason that the Federalist Papers were written under the pseudonym “Publius” — even public debate sometimes requires distance from the speaker’s identity. That protection still matters today, allowing people to seek information, test ideas, and ask sensitive questions without fear of legally required exposure.

Then there are rules about content. The bill makes it unlawful to design, deploy, or make available chatbots that, in the government’s view, “encourage” or “promote” certain categories of constitutionally protected speech. Who do we want to be in charge of determining that? Those restrictions violate the First Amendment by regulating the protected editorial decisions of developers and by infringing on individuals’ rights to create and receive lawful expression.

Proposals like the GUARD Act dictate how chatbots respond and intrude on editorial judgment by putting Congress’s thumb on the scale of what’s acceptable speech. This means control over who can speak, what can be said, and how ideas are expressed.

Those choices shape the substance of speech and risk reducing a chorus of voices to a single, government-shaped note. Grok is loosely modeled on The Hitchhiker’s Guide to the Galaxy. Claude operates under its own internal “constitution.” Standardizing those philosophical differences flattens distinct approaches, and when fewer questions are asked, fewer answers follow.

Finally, disclaimer mandates can cross constitutional lines by compelling speech. The GUARD Act requires chatbots to deliver federally imposed messages in every interaction. While informing users, its application in every circumstance alters the content and flow of communication itself, overriding both user and developer choices with what the officials want the public to see.

All of this points to a deeper reality that AI systems cannot perfectly predict or control every output. That’s not a defect. It’s a core feature of how these models generate responses from probabilistic patterns. Developers will be forced to filter even more speech than the bill directly targets to ensure the offending content is not generated. Combined with the GUARD Act’s vague and sweeping restrictions, the result is blunter tools that sand down the rough edges of debate and offer less of what makes them useful in the first place.

Treating chatbots as expressive tools keeps the focus on the people, not the machine. Many who’ve used them — which is now well more than half of Americans — know about its potential. It lets people test arguments, explore unfamiliar ideas, and tackle everyday challenges.

Yet artificial intelligence, particularly chatbots, has become Washington’s latest political punching bag. Accusations of manipulation and harm are driving a slew of legislative proposals to censor this emerging technology. The GUARD Act isn’t alone. The recently introduced CHATBOT Act presents many of the same threats.

The same impulse to move quickly in Congress is playing out nationwide, with proposals in states like Minnesota, Florida, and Washington targeting chatbots through access restrictions, disclosure mandates, and content-related rules. Florida Gov. Ron DeSantis is pushing an AI package that effectively requires everyone — adults and minors alike — to identify themselves before using these systems.

But the Constitution doesn’t permit any government to address concerns about AI by broadly restricting protected expression. The First Amendment demands solutions that target illegal conduct without burdening the exchange of ideas.

Today, AI sits squarely in the hot seat. Lawmakers and cultural commentators alike argue this moment is different — that the scale, speed, and influence of modern networks justify new forms of oversight and restriction.

But history suggests a recurring pattern: efforts to curb perceived harms often expand beyond their original aims, reshaping the boundaries of speech and expression in profound and lasting ways. As we confront this latest wave of tech debates, the question is not whether change is coming — it’s here. The question is which lessons we should learn from the past.

On March 24, 2026, at an intimate venue in Manhattan, we brought together some of the voices examining these recurring cycles. Nico Perrino, FIRE’s executive vice president and host of So to Speak: The Free Speech Podcast, places today’s debates in the longer civil-libertarian tradition of resisting censorship. Corbin Barthold, internet policy counsel at TechFreedom and host of the Tech Policy Podcast, brings a litigator’s skepticism to the latest proposals to regulate online platforms as common carriers. Christopher J. Ferguson, professor of psychology at Stetson University and co-author of Moral Combat: Why the War on Violent Video Games Is Wrong, has spent years studying the evidence behind recurring fears about violent media and new technologies. And guiding the discussion is Kmele Foster, co-host of the Fifth Column Podcast and editor-at-large at Tangle News, whose work is defined by intellectual curiosity and a rigorous commitment to open inquiry.

• Nico Perrino (X), FIRE’s executive vice president and host of FIRE’s So to Speak: The Free Speech Podcast

• Corbin Barthold (X), internet policy counsel at TechFreedom and host of the Tech Policy Podcast

• Christopher J. Ferguson (X), professor of psychology at Stetson University, author of How Madness Shaped History, and a licensed clinical psychologist

• Kmele Foster (X), editor-at-large at Tangle News, and partner at Big Think

Recent reports suggest the Trump administration is now considering new oversight for advanced AI models like ChatGPT, Claude, and Gemini. Few details have been finalized, but officials are reportedly discussing an executive order to create a government–industry working group. Another idea under consideration is a process for reviewing models before or around their release.

Subscribe now

As these talks move forward, they risk setting a troubling precedent for free expression.

Some AI companies such as xAI have already agreed to provide an early release of their models. One report announced that a group has already been formed within the Department of Commerce, which will review these models — sometimes testing them with fewer safety limits — to see what risks they might pose, especially for cybersecurity and national security.

This isn’t entirely new as OpenAI and Anthropic have done something similar before. But now the Trump administration is considering making this kind of review much more formal.

AI is an expressive tool — one that people use to learn, ask questions, and engage with ideas. The design of an AI system also reflects a series of choices about what information to prioritize, how to reason, what boundaries to draw, and how to respond. Those choices embed values and assumptions about knowledge, truth, and human interaction.

People who build and use AI tools do not shed their constitutional rights to freedom of expression at the prompt window. That includes the right to speak without being pressured by the government to first seek approval or give them a look under the hood. What starts as “just a review” can quickly become pressure to change what tools the public is allowed to have and what information users are allowed to see. Informal oversight has a way of turning into coercion.

Imagine an AI company being told its approval depends on how its model handles controversial issues — whether it reflects the government’s preferred stance on tariffs, energy or climate policy, or how it discusses election integrity ahead of an upcoming vote. Even without an explicit mandate, that kind of signal would pressure developers to shape outputs to satisfy officials rather than reflect independent judgment.

We’ve seen this before. When Biden administration officials approached social media platforms and repeatedly pressured them to remove or downgrade COVID-related content, it meant the public discourse about the most important issue of the day was being covertly shaped by government officials. In the current Administration, we’ve observed how the Federal Communications Commission’s authority to regulate the broadcast spectrum — which sounds fairly technical and benign in theory — has been weaponized to pressure news outlets to reshape their coverage at the government’s behest.

In those cases, pressure largely operated after content was posted. This new arrangement is more dangerous because it moves that pressure upstream, before an AI model is released to the public. Giving the government a role in reviewing speech or expressive tools at that stage creates a point of leverage it can use to ensure certain expression never sees the light of day. In First Amendment terms, this amounts to prior restraint — blocking speech before it’s communicated, rather than punishing it afterward — if the government can give thumbs-up-or-down approval. That’s why they’re presumed unconstitutional by courts.

Officials often invoke national security to justify expanding their reach. But national security isn’t a blank check. The public should be wary of arrangements that give government officials of either party a foothold over what AI systems generate.

Once the government assumes a gatekeeping role over emerging forms of expression, the line between oversight and censorship gets blurry real fast.

Artificial intelligence is rapidly becoming one of the world’s primary engines of knowledge creation. And as this meaning-making system comes online, so too does the inevitable desire for government control. How the First Amendment and other legal principles apply to AI will determine whether the future operating system for the planet acts as a tool for inquiry and expansion of human knowledge and capacity — or for compliance and top-down control. History is blunt on this point: once government acquires durable power over speech and expression, that power only ever expands — and the lure of censorship becomes virtually impossible to resist.

Subscribe now

This discussion — from Jan. 22, 2026 — brings together the individuals tracking and resisting that expansion. Dean Ball — technologist, AI governance analyst, and former White House science and technology adviser — warns that “responsible AI” rhetoric often serves as an on-ramp to permanent regulatory control. Ari Cohn, First Amendment attorney and tech policy expert, brings a free speech lawyer’s clarity to where government proposals cross constitutional lines. Matt Perault, head of AI policy at Andreessen Horowitz, examines how regulatory choices shape the incentives that drive innovation. And guiding the discussion is Kmele Foster of The Fifth Column podcast, whose work is defined by a first-principles approach and an inimitable commitment to curiosity.

Panelists:

• Dean W. Ball (X), senior fellow at the Foundation for American Innovation, former senior AI policy advisor at the White House OSTP, and principal staff drafter of “America’s AI Action Plan”

• Matt Perault (X), AI policy head at Andreessen Horowitz (a16z), and former director of the Center on Technology Policy at University of UNC Chapel Hill. Read a16z’s AI policy work on Substack

• Ari Cohn (X), a nationally recognized attorney and expert on the First Amendment, defamation, and Section 230

• Kmele Foster (X), editor-at-large at Tangle News and partner at Big Think

The encrypted messaging app Signal is back in the news — and this time, people are asking: Will using it get me arrested?

The short answer is probably not. If your speech would be protected if you were talking to a friend at a park or over the phone, your speech will still be protected when you have that conversation on Signal. And if not, you’re not any more likely to get arrested by using Signal than you were before you started using it. In other words, if you’re doing something unlawful, adding Signal to the mix probably doesn’t appreciably increase the already existing risk of getting arrested for breaking the law.

Let’s look at why that’s so.

Why are people suddenly worried about Signal?

In January, independent conservative journalist Cam Higby posted on X that he had “infiltrated organizational Signal groups all around Minneapolis with the sole intention of tracking down federal agents and impeding/assaulting/and obstructing them.” In his thread, Higby shared screenshots showing how some anti-ICE groups were organized along with the screen names of some participants, which led others to speculate on connections between organizers and Minnesota political figures.

Two days later, on the Jan. 26 episode of the Benny Johnson podcast, Johnson asked FBI Director Kash Patel about Higby’s reporting. Patel replied that the FBI had opened an investigation into Signal group chats, saying (emphasis added):

As soon as Higby put that post out, I opened an investigation on it. Just like any other case . . . We immediately opened up that investigation because that sort of Signal chat being coordinated with individuals, not just locally in Minnesota, but maybe even around the country . . . if that leads to a break in the federal statute or a violation of some law, then we are going to arrest people.

The significance a listener attributes to this quote is going to depend entirely on what they think of the actions of anti-ICE protestors in Minnesota. If the listener thinks the protestors are intending to harm the agents or interfere with federal law enforcement, then the FBI investigating an alleged coordinated effort to commit crimes is probably unremarkable. That’s what the FBI is supposed to do, at that level of abstraction.

On the other hand, if the listener thinks the protestors are engaged in protected First Amendment activity, then the FBI opening an investigation into Signal chats with the promise of arresting people if a crime is committed sounds a lot like viewpoint discrimination. Higby’s thread did not itself show evidence of illegal activity — just people sharing information about law enforcement activity, which is generally protected speech. (FIRE is suing the government on behalf of individuals who created a Facebook group and an app, respectively, that share video of ICE agents doing their jobs.) Many have used that information to protest or film ICE agents, which the First Amendment also protects. Nothing sounds more authoritarian, and less American, than the government investigating people for not liking the government.

But this piece is focused on a secondary effect of Patel’s comments: the fear that downloading Signal is going to put you in the FBI’s crosshairs.

How secure is Signal compared to other messaging apps?

What is it about Signal that has everyone’s attention? To understand why it’s popular among protesters and the Pentagon alike, let’s look at how messaging apps normally work — and how law enforcement might normally use data from those apps to investigate crimes. To do that, we need to think about two things: messages (that is, the actual content of what you’re sending) and the metadata (which is some combination of things like the time sent, the time read, location, device, phone number, network carrier, and so on).

In unsecured text messaging, like SMS or Facebook messages, your messages are sent to a service provider, where they’re stored along with the metadata. Cell phone providers store SMS data, although they generally don’t store it for very long. But in 2022, a detective subpoenaed Facebook messages to investigate crimes related to a suspected unlawful abortion by a 17-year-old. Those texts revealed the teen’s mother had been coaching her daughter on how to take abortion pills, leading to the mother being charged as well.

Next are messaging apps where the messages are end-to-end encrypted — meaning they can only be read by the devices on either end of the conversation — but the metadata is saved. One example of this is WhatsApp. This means the service provider doesn’t know what you’re saying, but it knows who you said it to, and when, and how often, and sometimes, where you were when you said it, and what phone you were using. For law enforcement, that’s often more than enough. In 2021, former Treasury official Natalie Edwards was sentenced to six months in prison for leaking documents to the press. When investigating Edwards, the FBI got a court order to access the metadata from her WhatsApp conversations. They couldn’t see what she was talking about, but they could see she was having frequent conversations with a reporter, and how those times lined up with stories containing leaked data.

(Side note: Apple’s iMessage is a special case because the messages are end-to-end encrypted, but the default behavior of iCloud is to store the encryption keys in the cloud for data recovery. That’s a setting you can change, if you want. But the default behavior is very much the digital equivalent of locking up your house and sliding the key under the mat.)

And then there’s Signal, which end-to-end encrypts its messages and the only metadata it collects is account-creation time and last-connection time. When the FBI came knocking in 2021, that’s all Signal had to turn over. That’s probably why anti-ICE protesters are using it, why the government has used it, and why tens of millions of other people have, too. It’s also open-source, so it can be (and has been) reviewed for any backdoors, which are undocumented methods of bypassing access control measures (something the government has called on companies to add).

Does that mean Signal is perfect? Not exactly, because it exists in the same imperfect world the rest of us do. For example, if you’re using your cell phone to open Signal, there will still be cell tower data about the time you were using your phone, and your cell provider might have metadata that your device communicated with Signal’s servers. Using a VPN could help with the latter, but then it’s a question of how much you trust your VPN. In theory, if you and the person you’re communicating with already have your metadata being captured by these companies, maybe just opening the app around the same time, repeatedly, could be evidence. And, of course, humans are always vulnerable to opsec failures.

But it’s fair to say that trying to reconstruct a crowd of human connections from fragments of metadata scattered across multiple companies is nontrivial. Even if the FBI could get large swaths of metadata — and maybe it can, if it buys it on the open market — it would probably need an immense neural network to try to piece together these fragments into coherent relationships. And where would the FBI get an artificial intelligence to use in that kind of mass surveillance, anyway?

FIRE is working on these issues. In an amicus brief filed last week in support of Anthropic’s lawsuit against the government, we argued:

For example, an agency could use an LLM to infer an individual’s association with a particular mosque based upon frequent visits to the mosque’s website, engagement with the mosque’s social media posts, and their cell phone’s physical proximity to the mosque during religious services. [...] It is easy to conceive how an agency, a government employee with improper intent, or a malicious third party that finds a vulnerability, could exploit these capabilities to monitor public discourse, preemptively squelch dissent, or cause myriad other harms.

A few days earlier, we joined an amicus brief in Chatrie v. U.S., arguing that geofence warrants (that is, warrants that compel service providers to hand over location data for every device near a location during a specific time window) violate First Amendment rights. Protestors, religious congregants, journalists meeting confidential sources, and innocent people who might happen to be nearby all have their rights chilled when the government engages in these digital dragnets.

Does the First Amendment protect the use of encryption like Signal?

Ultimately, Signal is encrypting your messages, and there’s good reason to suspect both the First Amendment’s protection of speech and the general right to privacy protect your messages from government intrusion. There are a few sources of law we can look to for those principles.

There should be books written about Bernstein v. U.S., and if there are, well, there should be more of them. After World War II, the U.S. government broadly regulated the export of cryptography (that is, a key that controlled the reversible transformation of a message) but did not regulate one-way hashing (a mathematical formula that would turn one string into another and could not be reversed). But are those things really different? Isn’t it all a form of speech?

In the 1990s, then-mathematics doctoral student Daniel Bernstein submitted for government review a program called Snuffle. I’ve been told I lose people when I try to explain how it worked, so I’m going to abstract this into really simple terms: Snuffle showed how it was possible to make working encryption in a way that didn’t meet the government’s technical definition for export controls. (It didn’t even actually do it. The code was missing essential pieces. It just showed how to do it.)

The government balked, and Bernstein sued. In its opinion, the Ninth Circuit panel said that Snuffle was entitled to First Amendment protection:

In this increasingly electronic age, we are all required in our everyday lives to rely on modern technology to communicate with one another . . . Government efforts to control encryption thus may well implicate not only the First Amendment rights of cryptographers intent on pushing the boundaries of their science, but also the constitutional rights of each of us as potential recipients of encryption’s bounty.  

The Ninth Circuit agreed to rehear the case before the full court, vacating the earlier panel decision. But before the case could be reheard, the government loosened the export rules and subsequently dropped the appeal. That procedural quirk rendered the panel’s decision void, but its wisdom still holds. In the 2000 case Junger v. Daley, the Sixth Circuit rejected the argument that computer source code isn’t really speech, noting that “though unintelligible to many, [it] is the preferred method of communication among computer programmers.”

Encryption aside, nothing about the use of Signal to organize groups that monitor federal activity appears to be unlawful. We have a right to organize for political action. We also have the right to monitor law enforcement with phones. And the Fourth Amendment protects us from warrantless collection of our location information or warrantless searches of our cell phones.

Can Signal keep you from getting caught if you commit a crime?

Signal isn’t a magic get-out-of-prosecution-free app for the same reason that no system is ever going to do that — because once you let humans in, the system becomes imperfect. We started down this road not because of a subpoena, but because someone just joined the groups planning the activity and then told everyone what was going on.

So that’s the hot takeaway: Don’t commit crimes, because you’ll get caught. That’s got nothing to do with Signal though. Signal’s fine.

Across the United States, teenagers freely express themselves online. But that freedom is rapidly being restricted, and make no mistake: this doesn’t just end with teens. What is often portrayed as a youth mental health issue is really a battle for everyone’s online speech rights.

Subscribe now

Worldwide, governments are moving to restrict minors’ access to social media. In Europe, France is seeking to ban children under 15 from using social media without parental consent, the United Kingdom is pursuing expanded regulatory authority over youth online access and platform design, and Spain has proposed criminal liability for tech executives who fail to remove harmful content quickly enough. In Asia, China has tightened limits on screen time and platform use for minors while Malaysia is considering a nationwide ban on social media for users under 16. And in Australia, the government has approved sweeping youth access restrictions.

The United States has shown signs of taking a similar course. Congress is advancing legislation like the Kids Online Safety Act (now combined with other bills in the House under the title “KIDS Act”) and multiple states have enacted or attempted age-based social media restrictions despite ongoing First Amendment challenges. The policy momentum is unmistakable — and troubling.

The concerns driving it are, to some extent, real. Nearly half of American teens say social media negatively affects people their age. Public health authorities have linked heavy social media use to anxiety, depression, sleep disruption, and increased exposure to cyberbullying. Research from the World Health Organization has documented declining adolescent well-being associated with excessive screen time and digital pressure. At the same time, other studies have found beneficial effects of social media use, particularly for vulnerable youth seeking community and support, and researchers continue to debate whether many of the observed harms have been causally proven.

As a 20-year-old who grew up during the rise of Instagram, Snapchat, and TikTok, I recognize those harms. I remember when social media shifted from something you checked occasionally to something that shaped your social world. Group chats determined inclusion. Posts felt permanent. Comparison was constant. I have felt the pressure of likes and the anxiety of visibility. I have watched peers struggle with online harassment and digital burnout.

Today’s youth are experiencing troubling mental health trends. But even if social media does play a role, when authorities claim a noble and urgent purpose in regulating how people can express themselves, these proposed solutions demand scrutiny.

Age-based social media laws do not simply reduce screen time. Most rely on age verification systems that require identity verification in practice. That can mean uploading government-issued identification, biometric scans, or other sensitive personal data just to create an account. What is framed as child protection can quickly become a structural shift in how speech is accessed online, and how many invasive barriers the government requires private companies to place between their users and their platforms.

From a civil liberties perspective, that shift is significant. The Supreme Court has repeatedly recognized the importance of anonymous speech in American tradition, such as in cases like Talley v. California, and McIntyre v. Ohio Elections Commission. Anonymity protects political dissidents, whistleblowers, vulnerable communities, and young people exploring their identities. If participation in digital discourse increasingly requires identity verification, anonymity weakens and the chilling effect on lawful speech grows.

Strict liability regimes create additional risk. If platforms face legal penalties for failing to remove harmful content quickly enough, which is a tool regulators in Europe especially rely on, they will inevitably err on the side of removing more speech. Automated moderation systems cannot perfectly distinguish between offensive but lawful speech, unprotected speech, and speech that is legal but nevertheless targeted by lawmakers as subjectively “harmful.” Discussions about politics, religion, gender, or social justice may be flagged and suppressed simply because platforms cannot afford the regulatory risk.

Supporters argue these laws target only minors. But the technical infrastructure required to verify age often requires verifying everyone. Platforms cannot easily build parallel systems for adults and children without expanding data collection across the board. That means more identification checks, more stored data, and greater vulnerability to breaches or misuse.

There are better ways to address youth mental health without reshaping the architecture of free expression. Platforms can voluntarily provide stronger parental control tools and resources that help families manage online experiences. Schools can invest in digital literacy education that teaches healthy online engagement. Efforts to address youth well-being should focus on empowering users and families rather than imposing government mandates that reshape how speech is accessed online.

The real question isn’t just teen safety, but this: Is the United States willing to normalize a permission-based internet?

My generation understands the downsides of social media. We have experienced comparison culture and digital pressure. But we have also used these platforms to organize protests, build communities, share stories, and participate in civic life. Social media is not merely entertainment. It is a modern public square, and when we understand it in those terms it’s clear why we cannot tolerate government intervention that censors speech and decimates anonymity.

Protecting teens is essential. Preserving free expression is essential, too. The vast majority of even “harmful” speech on social media is still protected speech that minors have a First Amendment right to access. If access to the public square increasingly depends on proving who you are before you speak, we may address one problem while creating another: a more monitored, less anonymous, and ultimately less free digital environment for everyone.

Lawmakers should approach youth social media reform with care. The goal should be to reduce harm without blocking minors from content they are entitled to, and without infringing on everyone’s rights by conditioning online speech and giving up anonymity.

In April of 2023, 14-year-old Sewell Setzer III became a user of Character.AI, a platform founded by two former employees of Google that hosts user-created interactive chatbots inspired by popular fictional properties. Going by “Ageon,” “Daenero,” and other names, Setzer began an intimate correspondence with a Game of Thrones-inspired “Daenerys Targaryen” chatbot. Less than a year later, he had killed himself. To Setzer’s family, his final exchange with Daenerys pointed to Character.AI as the culprit.

The leading subject of a New York Times feature just last week, Sewell’s parents’ wrongful death suit — filed in October 2024 — initiated what has become a growing wave of lawsuits seeking millions of dollars in damages from chatbot-based platforms. Each new plaintiff points to the last as evidence of a causal link between chatbots and the deaths of young users — and we’ve gradually seen the allegations take a turn from encouraging suicide to concocting elaborate plots.

Subscribe to Notice and Takedown

In August 2025, OpenAI found itself in the crosshairs with a lawsuit alleging 16-year-old Adam Raine’s suicide had been assisted and inspired by his interactions with ChatGPT. That same month, Stein-Erik Soelberg killed his mother and then himself, with his estate alleging that ChatGPT had convinced him he was the target of a high-level conspiracy. In November, the death of Austin Gordon by self-inflicted gunshot wound marked one of seven more high-profile lawsuits against OpenAI. His mother’s complaint, filed last month, alleged ChatGPT “created a fictional world and relationship that felt more real to Austin than anything he had ever known.”

And then somehow, things got even stranger. In the last two weeks, two lawsuits were filed:

One alleged ChatGPT essentially goaded a woman into torching a settlement agreement between her and the life insurance company that filed the suit, firing her lawyers, and engaging in a flurry of frivolous legal filings against the company. Another, filed against Google, alleges that their chatbot service Gemini had “trapped” 36-year-old Jonathan Gavalas in a “collapsing reality,” which involved coaching him through “missions” involving violence against the public and eventually his suicide.

Settling for less

The plaintiffs in these cases seem to be finding some success. Last month, Character.AI agreed to settle the Setzer-Game of Thrones case (styled Garcia v. Character Technologies) along with three other similar lawsuits filed in September.

They’ve been supported by political headwinds. Each of the complaints filed made a point to emphasize a letter by 54 state attorneys general warning of a “race against time” to “protect the children of our country from the dangers of AI,” insisting that the “walls of the city have already been breached.”

For anyone familiar with the history of civil liberties disputes, this rhetoric is instantly recognizable. Phrases like “race against time” and “protect the children” are the lingua franca of government restriction of novel expressive technology. The phrase “protect the children” gives you the license to restrict speech and the novelty of the technology provides the high stakes — if nothing is done before the technology develops past a certain point, it will presumably be too late. We have to restrict speech and we have to restrict it now.

The emotional appeal is strong. The defendant AI companies are inclined to avoid trial for a reason.

Is anyone raising First Amendment concerns?

You bet.

FIRE intervened early in Garcia after the court denied Character.AI’s motion to dismiss. In that order, the judge questioned why “words strung together by an LLM are speech.” A federal judge musing that stringing words together might not be speech because of who did the stringing together would depart from a long line of court cases holding that pure speech is protected speech agnostic to the identity of the speaker.

The order had worrying First Amendment implications for expressive technology if left to stand at the final conclusion of the case. FIRE accordingly filed a friend-of-the-court brief urging prompt appellate review of this holding, outlining all the reasons that statement — and the logic underpinning it — ran afoul of the First Amendment.

For the purposes of this blog, we’ll be evaluating the claims of these lawsuits in a bigger context. As case after case piles up, it is tempting — and quite human — to let the recurrence of tragedy take on a similar role as authoritative data in how we process the phenomenon, and importantly, assign blame. The public and the courts have confronted this temptation before.

We’ve rolled these dice before

There is a long line of entertainment-related torts and moral panics that have besieged free expression over the years, placing blame for violent acts on everything from Grand Theft Auto to the Slender Man lore. Each and every panic, taken to its logical conclusion, would have shrunk the universe of allowable expression in ways that would reverberate long past the point where clarity makes society’s past worries seem a little silly in retrospect.

No recent panic quite matches the intensity and the surreality of the current moment like the Dungeons and Dragons scare of the 1980’s. During a roughly five year period in the 1980’s there were 28 cases of adolescents who played Dungeons & Dragons and later committed murder or suicide.

There was the case of 17 year-old player James Dallas Egbert III, whose disappearance into nearby woods inspired speculation from the press that he had lost the ability to distinguish between himself and the game character he role-played. There was also 16-year-old Irving Pulling, whose death inspired his mother to start the public advocacy group “Bothered About Dungeons & Dragons” (BADD).

The media ran with it. BADD was featured in a 1985 60 Minutes segment that will help give readers a sense of just how strong the panic was, marginalizing experts with arguments from emotion. “The families who have suffered the loss of a loved one would disagree,” the narrator says, as the muted objections of a skeptical clinical expert play in the background. “If you found 12 kids in murder-suicide cases with one common factor,” he presses, “wouldn’t you question it?”

With the clarity of hindsight, the math finger-paints a pretty silly picture. “By 1984, 3 million teenagers were playing Dungeons & Dragons in the United States and the baseline suicide rate of adolescents overall would have been about 360 suicides each year,” University of Virginia professor of pathology James Zimring has pointed out. “So, when you look at the bottom of the fraction, at the denominator, Dungeons & Dragons was, if anything, protective. It had the opposite effect.”

We shouldn’t have to wait for the chatbot panic to be in the rearview mirror to do the same math with the 13-18 million teenagers and 130 million adults using ChatGPT and other AI chatbots. When you consider the small number of (emotionally potent) cases, it begins to look like maybe AI is causing psychosis — just not in the way people think.

Exploding books and dangerous ideas

It’s not just “standard” First Amendment law that these lawsuits get wrong. In an effort to get as far away from speech as possible, plaintiffs’ lawyers have gone with products liability law. After all, who could argue with the idea that a company has an obligation to design safe products, right?

But when you drill down into it, they aren’t really talking about “products” at all.

The Garcia case alleged, for example, that Character.AI designed products that caused users like Sewell to “conflate reality and fiction.” That should sound awfully familiar; it’s basically the same accusation grieving mother Sheila Watters made in 1989 against Dungeons & Dragons maker TSR.

As the court’s decision in Watters v. TSR, dismissing the suit describes, she “cast her son as a ‘devoted’ player of Dungeons & Dragons, who became totally absorbed by and consumed with the game to the point that he was incapable of separating the fantasies played out in the game from reality.” According to her suit, this made the product (i.e., the game) “unsafe” and TSR should pay.

But the Watters Court rejected this theory of liability — the same theory underlying most if not all of the chatbot lawsuits.

The Sixth Circuit, upholding the district court’s dismissal, observed that the harm originated not from the tangible properties (or even rules) of the game, but rather from the ideas expressed through its storyline — and that meant the case wasn’t really about a defective “product.” A court examining claims that violent video games caused the Columbine shooting reached the same conclusion: “There is no allegation that anyone was injured while Harris and Klebold actually played the video games . . . The actual use of the video games, then did not result in any injury . . . So, any alleged defect stems from the intangible thoughts, ideas and messages contained within.”

That’s an important distinction — product liability is generally imposed (often without requiring any fault, referred to as “strict liability”) on tangible “products” (think brakes, tires, dishwashers, etc.) with inherent and unreasonable dangers that are hidden to consumers, or for which there is a safer design — putting the manufacturer in the best position to prevent harm. In other words, the physical thing hurts you physically.

Imagine that you purchase a book. If the book’s binding explodes when you open it, you’ve got a product liability claim. The physical book, regardless of what its pages say, exploded in your hands — and there’s no harm to free expression by saying you can’t sell a book that doubles as an IED.

But suppose you were harmed because you did something stupid after reading ideas in a book. You might be able to see how imposing liability for “dangerous” ideas would set us down a dark path; every author and publisher would have to make sure that the ideas they put out in the world couldn’t possibly be interpreted or used to some harmful end. If you’ve ever met other human beings, you already know that the list of such ideas is … quite short.

And that’s exactly what drove the outcome in Watters. The district court noted that “the theories of liability sought to be imposed . . . would have a devastatingly broad chilling effect on expression of all forms . . . The First Amendment prohibits imposition of liability . . . based on the content of the game.” The appellate court saw a similar unavoidable impact of allowing for such liability: “The only practicable way of ensuring that the game could never reach a ‘mentally fragile’ individual would be to refrain from selling it at all.”

Tale as old as time, song as old as rhyme

This understanding has been applied across mediums of content and entertainment. In the cases of McCollum v. CBS, Inc. and Vance v. Judas Priest, the musical artists Ozzy Osbourne and Judas Priest were sued over the idea their music encouraged the suicide of two young men (attempted suicide in the case of Vance). Like Watters and like the recent chatbot cases, the plaintiffs were families of the young men.

Their lawsuits were unsuccessful. The court in McCollum echoed the Watters court concerns about liability chilling the expression of creators, making clear “such a burden would quickly have the effect of reducing and limiting artistic expression to only the broadest standard of taste and acceptance.” They accordingly noted that in the history of attempts to assign tort liability for electronic media inciting unlawful conduct, “all . . . have been rejected on First Amendment grounds.”

For other cases in this vein, check out Ari Cohn explaining why a law making social media platforms liable for what posts their algorithms promote is doomed to fail.

Which brings us back to Garcia and the argument FIRE made in our brief — and will inevitably have to make again.

If courts force AI developers to answer in tort every time a user has a tragic or delusional reaction to a chatbot, the incentive structure becomes obvious. They would have to “sanitize their outputs to only the most safe, anodyne, and bland ideas fit for the most sensitive members of society.” In other words, unless you want BarneyBot to be the only AI you’re allowed to use, think twice about demanding that developers anticipate the actions of every conceivable kind of user.

But it’s even worse than that. Movies and music are to a large extent statically consumed. AI helps people create and speak. It’s not only a question of what content AI can deliver to you, it’s a matter of what you will be able to say using AI. Total safety tends to come at a steep — and unacceptable — price.

When the Trump administration demanded changes to Anthropic’s AI system and backed it up with a threat to seize the system or blacklist the company, the message was clear: comply or be crushed. But cut through the rhetoric and the real question is whether Washington can bankrupt a company for saying no to the Pentagon.

Subscribe now

Though the media is busy framing this as a national security showdown, it actually poses a constitutional concern. It is a test of whether the federal government can weaponize its contracting power to force a private company to bend the knee.

AI systems are powerful expressive systems. They generate language, shape ideas, consume and interpret knowledge, and embody the values embedded in their design. The developer has the right, protected by the First Amendment, to make decisions about what capabilities to include or exclude. Anthropic isn’t willing to remove safeguards from its models for use in autonomous weapons targeting or domestic surveillance. Those limits reflect a deliberate expressive choice about what tools the company is willing to build, including what it is willing to provide to the government, and what its existing AI system is capable of achieving.

According to reports, on Feb. 24, War Secretary Pete Hegseth demanded Anthropic CEO Dario Amodei allow unrestricted use of its models “for all legal purposes” within three days or face severe consequences. Those consequences reportedly included blacklisting the company or invoking a Korean War-era law, the Defense Production Act (DPA), to take control of the company’s technology.

Anthropic refused.

In a statement, the company reiterated its commitment to responsible deployment and strict usage policies. Amodei later said the company had simply exercised its “classic First Amendment rights to speak up and disagree with the government.”

The administration’s response was swift. President Trump directed federal agencies to cease using Anthropic’s technology. Secretary Hegseth announced that “no contractor, supplier, or partner that does business with the United States military may conduct any commercial activity with Anthropic.”

Yesterday, the Department of War officially informed Anthropic’s leadership that the company and its products are deemed a supply chain risk, effective immediately.

The government’s actions, which are designed to harm Anthropic’s business, raise serious constitutional concerns, including threats of compelled speech and retaliation against a company for taking positions disfavored by government officials.

First, compelled speech. Anthropic’s decision to build specific guardrails stems from a principled disagreement about how its tools should be designed and used. The company has drawn a line against mass domestic surveillance, warning that AI can assemble commercially available data about Americans’ movements, browsing, and associations into detailed profiles at massive scale, posing serious risks to civil liberties. It has also declined, for now, to power fully autonomous weapons, arguing that today’s systems are not reliable enough to make life-and-death targeting decisions without human oversight.

Forcing Anthropic to remove those limits would compel the company to design and generate capabilities it affirmatively rejects, and has not contracted with the government to provide. And, thankfully, the First Amendment prohibits the government from forcing private speakers like Anthropic to create speech they oppose. Whether it’s a printed pamphlet or coding to enable autonomous targeting, the principle is the same.

When Hegseth threatened to invoke the Defense Production Act to take control of an AI system, he threatened the company with a clear message: The Pentagon is willing to use extraordinary powers to get its way. Enacted during the Korean War, the DPA was designed to mobilize industrial production for national defense, allowing the government to prioritize contracts and direct the manufacture of critical goods. In recent years, its use has expanded beyond traditional wartime manufacturing into domestic production and infrastructure, most notably during the pandemic, when it was invoked to accelerate the production of medical supplies.

Had it actually transpired here, applying the DPA to AI would risk giving the state control of knowledge production. Had Hegseth gotten his way, the government would have overridden the design, training, and limits that reflected Anthropic’s expressive choices about its model.

Second, retaliation. Labeling a domestic company as a security risk is an unprecedented move, and comes immediately on the heels of Anthropic’s refusal to alter its idea of what a responsible AI model should look like. When the government deploys extraordinary coercive power — particularly when justified on emergency or national security grounds — to punish a company for refusing to bow to the government’s demands, the line between legitimate procurement decisions and unconstitutional retaliation grows dangerously thin.

If the government can weaponize contracts and national security laws to coerce companies to reshape their AI systems, developers across the industry will rationally feel pressure to conform their research and design choices to official priorities. Systems guided by independent ethical guardrails risk becoming instruments of state policy via government takeover.

Of course, there are real concerns about national security and foreign adversaries outpacing the United States in AI and military development. But constitutional limits don’t evaporate in these moments. In fact, they matter more when the stakes are high and the pressure is on to centralize government power.

If the government wants AI systems without Anthropic’s restrictions, it can develop its own or contract with companies willing to provide them. (Indeed, OpenAI reportedly stepped in quickly as a replacement.) What the government cannot do is coerce a private company into abandoning its own design principles, or punish it for refusing. That violates the First Amendment.