Boiler Wrecks, Tainted Meat, and Other Red Flags: Lessons from the Industrial Revolution for AI Regulation

Introduction

California’s Senate Bill 1047 (S.B. 1047), known as the Safe and Secure Innovation for Frontier Artificial Intelligence Models Act, represents a significant early legislative effort to regulate advanced artificial intelligence (AI) models.^[1] Although ultimately returned unsigned by Governor Gavin Newsom in September 2024,^[2] the bill’s journey through the California legislature offers valuable insights into the challenges and opportunities of AI regulation. S.B. 1047 aimed to mitigate catastrophic harms from highly advanced AI models by imposing risk assessment requirements, establishing oversight mechanisms, and creating public resources like CalCompute to democratize access to computing power.^[3]

The bill sought to preemptively address growing concerns of the severe risks associated with AI advancements,^[4] concerns which were amplified by the rapid development of models like ChatGPT.^[5] It proposed ambitious oversight measures for large-scale AI models—those costing over $100 million and utilizing more than ten to the twenty-sixth power (that is, 100 septillion) operations—including mandatory risk assessments and full shutdown capability in the event that something has gone wrong.^[6] Through the involvement of the Government Operations Agency and establishment of a Board of Frontier Models, S.B. 1047 attempted to ensure compliance and guide best practices.^[7]

This Note argues that S.B. 1047 reflects several recurring design flaws common to early attempts to regulate transformative technologies. Historical experience suggests that regulatory frameworks succeed only when they incorporate technical expertise, centralized oversight, and mechanisms that adapt to rapidly evolving technologies. By examining the bill’s innovative approaches and its shortcomings and comparing them with historical attempts to regulate emerging technologies, we can better understand how to craft effective AI oversight frameworks moving forward. The analysis proceeds in three parts. First, it surveys key criticisms of S.B. 1047, focusing on concerns about its effectiveness in preventing harm and its potential to slow AI development efforts. Second, it examines historical parallels with other regulatory efforts, comparing S.B. 1047 to regulatory efforts that failed to prevent anticipated harm or that caused unintended negative consequences. Finally, it attempts to synthesize these insights to propose principles for more effective AI regulation, emphasizing the need for technical expertise, adaptive frameworks, and balanced oversight mechanisms that promote both robust development opportunity and continued safety.

Governor Newsom’s decision to return the bill highlighted several key challenges in AI regulation, including concerns about regulatory fragmentation and the need for federal coordination.^[8] However, the bill’s progression through the legislature and the robust debate it generated demonstrate California’s commitment to addressing AI safety concerns. As other states and the federal government contemplate similar legislation, the lessons learned from S.B. 1047’s journey offer valuable guidance for future regulatory efforts.

I. A Survey of S.B. 1047 Criticisms

Despite its intentions, S.B. 1047 faced substantial criticism, which can be organized into two primary categories: (1) doubts about its effectiveness in preventing harm and (2) concerns about slowing AI development and stifling competition.

A. S.B. 1047 Would Not Have Prevented Certain AI Harms

Critics argued that the bill would not have effectively prevented the harm it aimed to mitigate.^[9] The requirement for risk assessments and safeguards might be insufficient due to the unpredictable nature of AI development and the difficulty in foreseeing all potential risks.^[10] Moreover, the rapid pace of AI advancement could render the regulations obsolete shortly after implementation, failing to address emergent threats.

Central to this concern is S.B. 1047’s threshold provisions, applying the framework to models trained on a certain number of discrete computer operations and at a cost of $100 million or more, rather than applying it to models that meet either threshold alone.^[11] Commentators quickly noted that each of the prongs was roughly similar to the other in 2024 due to the compute costs at that time, but that compute costs continually fall.^[12] AI researcher Zvi Mowshowitz believed that the cost for buying one-time use of ten-to-the-twenty-sixth power (100 septillion) operations of compute was likely to fall below $100 million by the end of 2024,^[13] and he was “very confident” that by 2027 such compute would cost under $100 million.^[14] Indeed, policy analysts at the Center for a New American Security projected the cost to train a ten-to-the-twenty-sixth-power-operation model could drop from about $155 million in 2024 to only $31 million by 2029, due to projected improvements in hardware and algorithmic efficiency.^[15] Evidence bore this out: By early 2025, Anthropic’s Claude 3.7 model was trained on just under ten-to-the-twenty-sixth operations, at a price of only a few tens of millions of dollars,^[16] underscoring just how quickly frontier-level compute was becoming cheaper than the monetary threshold set by the bill. Thus, the two metrics may have initially represented the same cutting-edge scale, but rapid cost declines meant the compute threshold would soon be reached for far less money, letting developers skirt the regulation by training big models more efficiently.^[17] As hardware becomes cheaper, a model of any given algorithmic complexity becomes less and less likely to meet the $100 million threshold and therefore becomes more likely to be definitionally exempt.

Another technical critique focused on how quickly improvements in training algorithms could render a raw compute threshold obsolete. One industry letter warned in 2024 that “technology is still evolving” so a specific compute metric “may not adequately capture the capabilities or risks associated with future models.”^[18] In other words, a highly optimized model might achieve dangerous capabilities without needing ten to the twenty-sixth power operations worth of training compute and thus fall outside of S.B. 1047’s scope.^[19] This problem is an example of the political process defanging regulatory efforts, as an early version of the bill covered any model that could reasonably be expected to perform as well as a model covered by the operations threshold,^[20] but by summer of 2024, political pressure had led to that clause’s removal.^[21] Governor Newsom echoed this concern in his veto message, cautioning that “[s]maller, specialized models may emerge as equally or even more dangerous than the models targeted by S.B. 1047,” yet would not be covered.^[22]

Another critique in this category is that the reliance on developers to internally assess and self-report risks could have led to superficial compliance rather than substantive safety measures. Recent history provides compelling examples of the limitations of self-regulation in the tech industry. Meta (formerly Facebook) repeatedly assured regulators of its privacy protections while failing to prevent the Cambridge Analytica scandal, which affected 87 million users, and was criticized for responding too slowly after the scandal occurred.^[23] Similarly, Google has faced multiple fines under the GDPR for opacity in its data processing practices despite claims of compliance.^[24] In the realm of AI safety specifically, OpenAI’s internal adversarial testing failed to identify numerous capabilities that emerged after GPT-4’s release, demonstrating the challenges of comprehensive self-assessment.^[25]

Without stringent enforcement and expertise within regulatory bodies, the bill’s provisions may not have translated into tangible safety improvements. The technical complexity of frontier AI models, against a backdrop of inherent conflicts of interest implicated in self-regulation, created significant risks that companies might have prioritized development speed over thorough safety evaluation.^[26]

B. S.B. 1047 Would Slow AI Development

The second category of major criticism is a familiar refrain that regulation, here S.B. 1047, could have hindered innovation and competition. The high costs associated with compliance might have disproportionately affected startups and smaller companies, consolidating power among established tech giants who would be able to absorb the regulatory burdens. This could have led to decreased competition, reduced diversity in AI research, and the potential monopolization of the AI industry.^[27]

Furthermore, the critics said, open-source development—a cornerstone of development in AI technology—might have been jeopardized. Fears of legal liability could have deterred developers from sharing their models, slowing collaborative advancements and limiting the accessibility of AI technologies to the broader community.^[28]

II. Old Lessons for New Technologies

To contextualize the potential pitfalls of S.B. 1047, it is instructive to examine some of the historical regulatory efforts that failed to prevent the harms they were intended to protect against and some of those regulations that inadvertently caused significant harm. This Part will first examine two examples of regulation that failed to prevent harm, the succession of Steamboat Acts that the U.S. Federal Government implemented through the mid-1800s and, taken together, the Pure Food and Drug Act and Meat Inspection Act, both of 1906. After that, this Part will examine the Red Flag Laws of the United Kingdom in the late 1800s, which caused unintended harm due to how they were structured and implemented.

A. Regulatory Efforts That Failed to Prevent Harms

1. The Steamboat Acts of 1838, 1852, and 1871

Early in the nation’s history, the need for regulation was not as easy to forecast, and early attempts were understandably imperfect. The story of the regulation of steamboats demonstrates three separate early attempts, each with lessons to learn along the way.

The Steamboat Act of 1838 aimed to improve safety in steamboat operations but, as will be shown, failed largely because it deferred critical details of inspections to future determinations. This failure is much like how S.B. 1047 leaves specifics to be filled in later.^[29] The lack of immediate, detailed regulations resulted in continued accidents and limited improvements in safety standards.^[30] Just as the Steamboat Act’s deferral of inspection specifics undermined its effectiveness, S.B. 1047’s delegation of crucial technical details to the Government Operations Agency and Board of Frontier Models risks creating similar regulatory gaps.

a. The Wild West

In the early years of steam-powered navigation, American waterways were transformed into bustling highways of commerce and travel. The advent of the steamboat, a technological marvel of its time, promised unprecedented speed and efficiency. Early efforts, like those in developing many revolutionary technologies (including artificial intelligence models), were not commercially profitable. Early artificial intelligence, like early steam power, first appeared as a complex and expensive way to perform tasks humans already handled more simply. In the 1780s, early steam power pioneer John Fitch could stage repeated public demonstrations of his steamboat, yet he still could not carry passengers or freight more cheaply than a horse and turn a profit.^[31] Still, a few decades later Robert Fulton’s Clermont, famously navigating the Hudson in 1807, heralded a new era, shrinking distances and expanding economic opportunities.^[32] This more mature steamboat technology was foundational to both the industrialization of the northern United States and the great expansion of agriculture in the south.^[33] Yet, this revolutionary technology also introduced danger. In the absence of regulatory oversight, steamboat operation was hazardous and frequently ended in tragedy.^[34]

The early nineteenth century witnessed a proliferation of steamboats, each varying wildly in design, construction and operational standards.^[35] Competition was fierce, and the drive for profit often superseded any concerns about safety.^[36] Boiler explosions, collisions, and fires were common.^[37] One infamous example was the explosion of the steamboat Moselle. Built for the kind of profit that can only be obtained by being the fastest, the Moselle held the record for fastest trip between Cincinnati and St. Louis.^[38] On April 25, 1838, just moments after undocking near the Fulton neighborhood in Cincinnati, all four of the Moselle’s boilers exploded simultaneously, resulting in approximately 150 of the perhaps 260 to 280 passengers being reported as killed or missing.^[39] The disaster was covered by national papers, which captured the public’s horror and outrage with graphic descriptions of the accident and its consequences.^[40] This led to a shift in the public’s concerns from asking whether or not they could regulate steamboat activity, to asking whether steamboat technology was even worth pursuing in the first place.^[41]

b. The Wild West and the Act of 1838

In part because of the public outcry from this and other disasters like it, Congress enacted the Steamboat Act of 1838.^[42] This was the legislature’s first attempt to address the growing dissatisfaction with the frequency of incidents. Its primary focus was on requiring steamboat owners to obtain a license of inspection from a district-court-appointed inspector.^[43]

The district judge would appoint “from time to time” one or more inspectors, who had to be knowledgeable about the manufacture and use of steamboats and their components. Those who had a financial interest in the manufacture of such boats were unable to be appointed.^[44] Perhaps because of the legislature’s lack of subject-matter expertise, the 1838 Act was light on the details. It required inspectors in general terms to be sure the hulls of inspected boats were sound, and that engines and boilers were safe, but without defining any terms or specifying what level of safety was appropriate.^[45]

The 1838 Act also required steamboats to carry “suitable” lifeboats and other categories of lifesaving equipment, with the same low level of specificity.^[46] Finally, because the inspection system was executed by local district judges appointing local inspectors, there was significant variety in the standards adopted from region to region.^[47] The theory seemed to be that something needed to be done, and done fast. But just like steamboats, when legislation is designed for speed, it may not work as well as would be hoped.

c. There’s a New Sheriff in Town, But It’s Still the Wild West

While the 1838 Act represented a legislative acknowledgement of the need for safety regulations, its inherent weaknesses—vague standards, decentralized enforcement, and a lack of qualified inspectors—undermined its impact. The period from 1838 to 1852 was, unfortunately, marked by a series of horrific accidents that underscored the Act’s inadequacies and fueled public demand for more robust legislation.

Judges faced two challenges appointing inspectors. First, it turned out that district judges were not any more likely to be experts at steamboat safety than Congress was. Second, there was a heavy incentive for people who were knowledgeable about steamboat manufacturing to become financially “interested” in manufacturing steamboats, leading to a dearth of qualified, disinterested candidates and a number of inspectors being named whose credentials were questionable.^[48] This issue was compounded by the fact that each inspector was on their own to determine what they thought was “safe” and how much safety equipment was “suitable,” and what one inspector considered “safe” might be ignored by another.^[49]

One notable disaster that occurred during this period was the explosion of the Henry Clay. Steamboat operations in this era were as competitive as ever, and the Henry Clay’s operations were no exception.^[50] It navigated the Hudson River between Albany and New York City and competed with both other steamboats and the Hudson River Railroad which had been completed in 1851.^[51] On July 28, 1852, it was racing against another steamboat to attract business, and despite having about 500 passengers, it was equipped with only two lifeboats.^[52] At least seventy passengers perished just outside the Bronx in Riverside.^[53]

History repeated itself. As with the Moselle, the wreck of the Henry Clay became a national story, with the public outraged at the needless loss of life.^[54] Just like the boilers in a racing steamboat, the pressure on Congress to act must have been immense. And just like that racing steamboat, Congress made great haste in passing the Steamboat Act of 1852 on August 30, 1852—just thirty-three days after the wreck of the Henry Clay and almost eleven months before the federal manslaughter trial of its officers.^[55]

d. The Act of 1852

The Steamboat Act of 1852, officially titled “An Act to Amend an act entitled ‘An Act to provide for the better Security of the lives of Passengers on board of Vessels propelled in whole or in part by Steam,’ and for other purposes,” introduced significantly stricter standards and enforcement mechanisms than the act which it amended.^[56]

First and foremost, the 1852 Act created detailed construction and inspection standards, including limiting boiler materials to those specifically listed, requiring inspectors to perform a hydrostatic test to ensure boiler integrity, and detailing inspection steps for safety valves, fusible plugs and other safety devices.^[57] The Act also mandated licensing of pilots and engineers for the first time.^[58] Prospective pilots and engineers were required to submit to inquiry to demonstrate their competency, ensuring that only qualified individuals were entrusted with the operation of steamboats.^[59] This measure was intended to reduce the number of accidents that were attributable to human errors such as the need for speed.^[60] The Act also mandated that steamboats carrying passengers must carry a specified number of lifeboats, life preservers, and fire extinguishers regardless of locality of manufacture or operation.^[61]

The Act was aimed at unifying the safety-critical aspects of steamboat operation, and that focus demonstrates what Congress must have considered to be the most important lesson to learn from the failures of the 1838 Act. While it went a long way toward establishing a single federal system for steamboat operation, it still vested the responsibility for executing the inspection process in local officials, potentially allowing variance in adherence to the uniform standards.^[62] In addition, once a pilot and engineer were licensed, the Act of 1852 allowed them to operate their steamboats in essentially whatever way they deemed fit, having no restrictions on use other than prohibitions on tampering with the mandated safety equipment and maximum pressure limits for boilers. The Act’s prohibition on tampering with boiler safety mechanisms can plausibly be read as an attempt to suppress steamboat racing.^[63] But without an explicit prohibition on the act of racing itself, crews involved in catastrophic accidents could still argue that they complied with the statute in practice, if not in spirit. By regulating the machinery rather than the act of competition, Congress inadvertently produced the nineteenth-century equivalent of steamboat Formula One.^[64]

e. What Happens When You Overlook Oversight

While the 1852 Act undoubtedly helped the industry to become safer, accidents continued to occur.^[65] The 1852 Act mandated stricter construction standards, licensing requirements for pilots and engineers, and enhanced safety measures, but its decentralized enforcement and lack of a central regulatory body remained as its critical weaknesses. These shortcomings were tragically illustrated by the famous, horrific wreck of the Sultana.

The Sultana, a Mississippi River steamboat, was involved in the deadliest maritime disaster in U.S. history on April 27, 1865.^[66] Overloaded with more than 2,400 passengers, primarily Union soldiers recently released from Confederate prison camps, the vessel suffered a catastrophic boiler explosion near Memphis, Tennessee.^[67] The explosion, compounded by extreme overcrowding and the subsequent fire, resulted in an estimated death toll of over 1,700 people.^[68]

The Sultana disaster exposed multiple failures in the regulatory system. First, the vessel was operating with a compromised and poorly repaired boiler, which was a direct violation of the 1852 Act’s requirements for rigorous inspections and maintenance.^[69] Fragmented enforcement, corruption, and lax oversight allowed the Sultana to continue operating despite its known defects.^[70] Second, the vessel was grossly overloaded, far exceeding its design limits for passenger capacity.^[71]

This tragedy, along with numerous other incidents in the period, fueled additional public outcry and calls for more effective regulation. Because steamboat accidents at the end of this period were caused more by illegal actions than weak laws, Congress shifted from writing stricter regulations to more uniformly enforcing existing ones.

f. The Act of 1871

This new focus culminated in the Steamboat Inspection Service Act of 1871.^[72] The Act addressed the critical shortcoming of those that preceded it by establishing a centralized regulatory authority, the Steamboat Inspection Service, which was tasked with overseeing the safety of steam vessels in the United States.^[73] This Act equipped the Steamboat Inspection Service with broad authority to establish safety rules, inspect vessels, license engineers and pilots (as well as other officers), and investigate accidents.^[74] This unified approach to the execution of the regulation set it apart from the previous acts.

Crucially, the 1871 Act mandated regular inspections of steamboats’ hulls, boilers, machinery, and life-saving equipment.^[75] These inspections, conducted by federally appointed inspectors that had to meet uniform qualifications to even be considered, ensured consistent application of safety standards across all regions.^[76] The 1871 Act also prescribed updated specifications for boiler construction, testing, and maintenance, taking into account changes in the technology since the 1852 Act was passed.^[77]

The licensing and certification process for engineers and pilots was formalized and standardized under the Steamboat Inspection Service.^[78] It expanded to reach ship masters and other officers as well.^[79] Applicants were required to demonstrate their competency through rigorous examinations and practical demonstrations, ensuring a higher level of professional qualification and further reducing the risk of accidents due to human error.

On the issue of overcrowding, the 1871 Act permitted inspectors to limit the number of passengers based on a vessel’s size and capacity.^[80] This inclusion highlights how regulatory focus shifted: Rather than focus on setting standards for manufacturing the technology, it focused on administering and setting guardrails for proper use of the technology. With a centralized agency responsible for ensuring proper execution of the regulation, there wasn’t a need for massive reform of steamboat safety again. While the Steamboat Inspection Service is no longer a federal agency, the responsibilities that it had have been handed down to the U.S. Coast Guard, which traces these responsibilities back to the 1871 Act directly.^[81]

g. Steamboats to Chatbots: Lessons Learned

The history of steamboat regulation, culminating in the 1871 Act, provides a potent framework for approaching the complex challenge of AI governance. The initial 1838 Act demonstrates that government intervention is often necessary to mitigate the proven risks of inherently dangerous technologies. The evolution from vagueness in the 1838 Act to specificity in the 1852 Act underscores the critical importance of detailed, technically informed regulations in technological oversight. Likewise, effective AI governance requires legislative precision in drafting, informed by the technical intricacies of complex AI systems. The 1871 Act highlights the value of a centralized regulatory body, made up of those who are properly embedded in the field, with national authority to ensure consistent enforcement and adaptation to evolving technologies—a clear necessity for a rapidly advancing, highly technical industry, like AI.

Finally, the historic trajectory from 1838 to 1871 reveals a crucial lesson often overlooked in contemporary discussions: Regulating a public-facing technology like AI requires not only regulating its development, but also its deployment and use. Just as steamboat regulations addressed not only boiler construction, but also pilot licensing and passenger capacity, AI governance must widen its focus from regulating only design and development to also administering its real-world applications. A technological regulation approach that fails to reach use cannot effectively mitigate potential harms or maximize social benefit.

For the reasons explained above, S.B. 1047, in its final form, would have fallen short of incorporating these lessons, risking a replay of the past missteps in steamboat regulation. To be more effective, AI governance attempts must learn from these historical lessons: prioritizing specificity, centralized oversight, and a comprehensive approach to the full spectrum of AI’s development and deployment.

2. Food and Drug Regulations in 1906

The early twentieth century saw the rise of large-scale industrial food production and pharmaceuticals, creating new regulatory challenges for public health and safety. Although mass production created greater accessibility and convenience for consumers generally, it also introduced greater risks of adulteration, contamination, and deception. The public, previously unaware of the conditions in which their food and medicines were manufactured, became increasingly alarmed by investigative reports exposing the widespread dangers posed by unregulated industries. Nowhere was this alarm more vividly captured than in Upton Sinclair’s The Jungle, which revealed the squalid and unsanitary conditions of Chicago’s meatpacking plants, sparking widespread outrage and calls for reform.^[82]

Congress responded in 1906 by enacting two landmark statutes: the Pure Food and Drug Act (PFDA) and the Meat Inspection Act (MIA).^[83] The PFDA sought to combat the adulteration and misbranding of food and drugs in interstate commerce, while the MIA introduced mandatory federal inspection of slaughterhouses and meatpacking facilities. These laws represented some of the first major federal regulatory efforts aimed at ensuring consumer safety in the face of rapid technological and industrial advances. Perhaps due to their pioneering nature, the statutes contained critical shortcomings, including narrow regulatory scope, enforcement limitations, and industry influence over their implementation.^[84] These shortcomings are reflected in S.B. 1047, which seeks to regulate AI models in a manner that, like its historical counterparts, risks being underinclusive and reactive.

a. The Pure Food and Drug Act

The PFDA was primarily designed to address adulteration and misbranding of food and drugs in interstate commerce.^[85] Its passage marked a turning point in federal consumer protection, but its regulatory scope was limited in important ways.

First, the PFDA focused only on the accuracy of ingredient labels, rather than on the broader issues of whether the drugs were effective or safe.^[86] Manufacturers were required to list their ingredients truthfully, but the Act did not prevent them from making misleading therapeutic claims about a product’s effectiveness. In United States v. Johnson,^[87] the Supreme Court ruled that the PFDA did not prohibit false claims about a drug’s effectiveness, only intentionally false representations regarding its contents.^[88] This feature, which we might consider to be a loophole today, allowed patent medicine manufacturers to continue making dubious claims about their products’ ability to cure diseases, such as tuberculosis or cancer, so long as they listed their ingredients accurately.^[89]

Second, the Act did not include related consumer goods such as cosmetics and medical devices.^[90] Because it did not address these products, items such as toxic cosmetics and untested medical devices remained on the market without federal oversight for decades. This was despite the similarities between drugs and cosmetics, both in the means of exposure to the compounds and in risk of injury they present.

The PFDA had another important limitation in that its enforcement mechanism was largely reactive, relying on post-sale interventions rather than pre-market safeguards.^[91] The law did not require manufacturers to prove their products were safe before entering the market.^[92] Instead, regulators had to identify and remove dangerous products either by market inspection or by responding to reports of harm that was already done.^[93] Regulators had to prove that a product was actively harmful before taking action to remove it from the stream of commerce, and courts were understandably hesitant to substitute an ex-ante probability analysis for an actual showing in making such a determination.^[94] In United States v. Lexington Mill & Elevator Co., the Supreme Court ruled that for an adulterated product to be banned, the government needed to demonstrate that the ingredient in question “may render [food] injurious to health” rather than merely being an unnecessary additive.^[95] This strict evidentiary requirement made it difficult for regulators to act quickly against potentially hazardous products, delaying enforcement and allowing questionable goods to remain on the market.

b. The Meat Inspection Act

The Meat Inspection Act of 1906 was a more comprehensive statute than the PFDA, introducing continuous, federally mandated inspections of meat production facilities.^[96] Unlike the PFDA, which focused on labeling after products entered the market, the MIA aimed to prevent contamination at every stage of meat processing.^[97] But the Act’s effectiveness was, perhaps counterintuitively, undermined by the support it received from industry.^[98]

Initially, the nation’s largest meatpacking companies opposed federal oversight, fearing additional costs and regulatory scrutiny.^[99] However, as public trust in their products deteriorated following Sinclair’s exposé, the industry saw federal inspection as a means to restore consumer confidence.^[100] Large firms then supported the law’s passage, provided certain concessions were made like ensuring that compliance costs would be borne by taxpayers rather than by the industry itself.^[101] This effectively subsidized larger firms while imposing compliance costs and administrative barriers to entry on smaller competitors.^[102]

Furthermore, while the MIA established robust federal oversight, it did not completely eliminate industry influence over the enforcement mechanisms.^[103] Because inspectors worked closely with companies they regulated, the possibility of regulatory capture remained a persistent concern.^[104]

c. Lessons Learned

The PFDA’s emphasis on ingredient labeling rather than product effectiveness closely parallels S.B. 1047’s narrow focus on AI’s technical inputs rather than its demonstrated capabilities.^[105] S.B. 1047 defines “covered models” based on the number of mathematical operations it takes to train them and training costs, rather than assessing what the AI system can actually do.^[106] This approach leaves open the possibility of harm from AI models which are used the same way as a covered model (like harm from unregulated cosmetics, when pharmaceuticals are regulated), since models below the cost and compute threshold set by the regulation may be able to perform similarly to covered models.^[107]

While S.B. 1047 does require developers to perform a risk assessment prior to release, it does not condition deployment on a government determination that the model is in fact safe, nor does it impose an affirmative burden of proof comparable to the pre-market approval regime governing drugs and medical devices.^[108] Although the risk assessment measure is more than the PFDA required initially, it falls well short of the modern requirements for approval in releasing a medical device or pharmaceutical, which requires the manufacturer of a new drug to prove to the FDA’s satisfaction that the new drug is safe before release onto the market.^[109]

Moreover, just as the MIA’s enforcement mechanisms were shaped by the meatpacking industry to their advantage, S.B. 1047’s reliance on an industry-involved regulatory board raises concerns that AI companies may exert similar influence, weakening enforcement and shaping compliance measures in ways that prioritize corporate interest over public safety.

The PFDA and MIA illustrate both the potential and limitations of early regulatory efforts in response to new technologies. Their narrow focus, misplaced burden of proof, and susceptibility to industry influence limited their effectiveness, leaving regulatory gaps that persisted for decades.^[110] S.B. 1047, by following a similar trajectory, risks repeating these historical mistakes. A truly effective AI regulatory framework should adopt a proactive, independent, and comprehensive oversight model, ensuring that safety is demonstrated before harm occurs rather than after the fact.

B. Regulation That Caused Unintended Harm

Regulation of emerging technologies can prevent the harm it seeks to contain and still be considered a failure. This is particularly true when the regulation creates harm that may well be as bad as or worse than the harm the regulation is attempting to stop. The Red Flag Act may be an example of such a regulatory attempt.

Not to be confused with the modern American idea of “Red Flag Laws,” over 150 years ago, Britain’s Parliament responded to the newfangled horseless carriage with the Locomotive Act of 1865,^[111] now popularly known as the Red Flag Act.^[112] Enacted on July 5, 1865, the Act famously required that any self-propelled road vehicle be preceded by a person on foot carrying a red flag (and two “efficient Lights to be affixed conspicuously,” should someone undertake to drive at night) to warn others of the approach.^[113] It also introduced the world’s first numeric speed limits: no more than four miles per hour on open roads, and a much safer two miles per hour in towns.^[114] Violators faced hefty fines (up to ten pounds—a substantial sum at the time).^[115] In order to draw lessons for modern technology policy, this Section examines the rise and fall of the Red Flag Acts and seeks to distill general lessons from their history.

1. Red Flags as a Response to a Red Alert

The Victorian lawmakers who crafted the Red Flag Acts did so in response to legitimate public concerns.^[116] In the 1860s, steam-powered vehicles on roads were a novel and sometimes frightening sight, and steam-powered vehicles were not particularly well suited for operation on roads built for horse-drawn carriages and omnibuses.^[117] Large traction engines—some weighing twelve to fourteen tons^[118]—could theoretically startle horses and clog narrow carriageways. Their potential for causing accidents did not escape some legislators.^[119] Parliamentary debates from 1865 include descriptions of these formidable engines barreling down country lanes and fatal accidents that had already occurred as a result.^[120] One member of Parliament argued that while high-bred horses would not be spooked by steam cars on roads, “low bred horses would generally become timid on seeing the locomotives.”^[121] The harms that lawmakers sought to prevent were thus quite concrete: collisions, runaways, and general havoc on roads ill-prepared for motorized traffic.

Beyond safety, there were economic and social pressures at play. The horse-drawn carriage industry and even railway companies saw road locomotives as a threat to their dominance in transport. Powerful lobbying interests pushed for strict limits on road vehicles.^[122] Given this climate, Parliament’s response was to pump the brakes. The Red Flag Act of 1865 built upon an earlier 1861 law and imposed even tighter restrictions: a minimum crew of three for each vehicle,^[123] the walking flag-bearer to precede the vehicle by at least sixty yards, and the ultra-low speed caps.^[124] Lawmakers believed that forcing vehicles to crawl at a pace slower than a walking person, and giving horse traffic ample warning, would avert the most dire dangers.^[125]

But not all voices were unanimous about such onerous rules. A few parliamentarians questioned whether a two-mile-per-hour speed limit was unreasonably low.^[126] This dissent fell on deaf ears at the time, as evidenced by the passage of the 1865 Act. For the most part, Parliament was convinced that stringent regulation was necessary to prevent chaos on the roads. In their view, the Red Flag requirements were a reasonable precaution to tame a disruptive new technology before it could wreak unintended and unknown harms.

2. Waving the White Flag: The Failure of the Red Flag Acts

If the Red Flag laws were born of rational fears, they ultimately proved to be a spectacular regulatory miscalculation. As the decades passed, the provisions of the 1865 Act revealed themselves to be not only outdated but downright absurd. The core flaw was that the law’s drafters did not anticipate how rapidly automotive technology would evolve. The rules had been written with hulking steam traction engines in mind, yet by the 1880s and 1890s innovators were developing lighter, faster, and safer motor cars, culminating in vehicles powered by the internal combustion engine.^[127] The Red Flag Act made no distinction between a fourteen-ton traction engine and a (relatively) nimble, one-ton “horseless carriage”—all were legally locomotives subject to the same crawling speed and flagman requirements.^[128] As motoring innovations gathered pace (increasingly in continental Europe and America), these blanket restrictions started looking arbitrarily punitive.

By the 1870s, critics openly derided the restrictions and called for their repeal.^[129] Every aspect of the Act of 1865 received ridicule in some form or another, from interest groups as varied as agriculturalists, scientists, engineers, and sporting thrill-seekers.^[130] The speed limits in particular were discussed frequently, as the original drafters appeared to select the two-and-four-mile-per-hour limits somewhat arbitrarily—roughly matching a slow trot—without any scientific basis, and they became more indefensible by year as vehicle engineering improved.^[131]

The failure of the Red Flag laws was confirmed by their demise in Parliament. After years of lobbying by automotive pioneers, influential politicians, and other public figures,^[132] Britain finally scrapped the draconian restrictions. The Locomotives on Highways Act of 1896 repealed the red flag requirement and raised the national speed limit from four miles per hour to a blistering fourteen miles per hour.^[133] The general relief felt was on display as Harry John Lawson organized and executed the first ever London to Brighton Car Run, called in its inaugural edition the Emancipation Run, where motorists traveled sixty miles from the Charing Cross Hotel to Brighton.^[134] Although it is disputed, oral history tells of Lord Winchelsea tearing a red flag in half symbolically to start the Run.^[135] Freed from a two mile per hour speed limit, the British automotive industry suddenly had a market where automobiles would be competitive with other road-based transportation options and development of that market began to accelerate—literally and figuratively.^[136] But the damage had been done: Commentators then and since have argued that the three-decade delay imposed by the Red Flag Acts caused Britain to lose its early lead in the automobile race to more laissez-faire nations like France, Germany, and the United States.^[137] In the final analysis, the Red Flag Acts failed because their well-meaning provisions proved both arbitrary and counterproductive. The anticipated harms—harried horses and pedestrian peril—could be mitigated in less onerous ways as technology improved (for example, better brakes, lights, and horns on cars, all of which could be anticipated by lawmakers, but most importantly miniaturization of the technology, which could not be easily anticipated even by experts in 1865). Meanwhile, the costs of the regulation—a stifled market for mechanical locomotion on roads in the United Kingdom—far outweighed its swiftly diminishing benefits.

3. Flagging the Lessons Learned

Regulating emerging technology is a delicate balancing act that one must perform blindfolded with both hands tied behind the back. The saga of the Red Flag Acts offers a cautionary tale about how even well-intentioned regulations can create unintended harms. In hindsight, Britain’s attempt to govern horseless carriages with a man walking ahead carrying a flag seems comically misguided. But for Parliament in 1865, confronting an unprecedented technology, those rules felt like commonsense safeguards against chaos. The potential parallel with modern emerging technology is hard to miss. California’s S.B. 1047 was driven by legitimate fears about AI’s potential misuse and effects on society: from dangerous misinformation to harmful bias, from privacy invasions to catastrophic proliferation of weapons of mass destruction.^[138] But the risk of overshooting is worth considering carefully. S.B. 1047’s critics—including AI experts in academia and industry leaders—argued that its sweeping approach, with rigid definitions of “frontier AI” models, potential for heavy fines which led to heavy compliance burdens, and broad extraterritorial reach, could slow or offshore development and impose costs disproportionate to the speculative harm addressed.^[139] That sentiment neatly echoes the lessons of the Red Flag Acts: If regulators clamp down too hard, too early, they may end up doing more damage than the technology itself would have, and such decisions lack empirical evidence to guide the decisionmaker because of the technology’s novel capabilities.

The broader and more fundamental lesson is that, at least in the absence of empirical evidence, wise regulation of emerging technologies requires a deep understanding of both the technology and the way it will be used in the real world. Rules should not be rooted in fear of the worst outcome one can imagine or the preferences of people who benefit from the way things are. In the Victorian Era, lawmakers misjudged the arc of automobile development and the capacity of engineering solutions to mitigate dangers.^[140] Consequently, they imposed burdens that seemed reasonable to them but proved arbitrary and counterproductive as technology advanced, and which took decades to undo.^[141] If we regulate AI or any new technology based on incomplete knowledge or an entirely understandable fear of the unknown, we can expect that the solutions we enact will age poorly. The less lawmakers understand the technology and its economic and social effects, the more likely this outcome becomes.^[142] Put plainly, if absurdly, artificial intelligence regulation should avoid requiring a proverbial man with a red flag to walk in front of the algorithms. Technology, like a carriage, needs room to move, guardrails to prevent catastrophe, and rules to ensure operation is safe and accessible for everyone.

III. Lessons for AI Regulation

Historical examples highlight that regulation of emerging technologies requires a delicate balance. Effective regulation should be informed by a deep technical understanding of the technology, be adaptable to rapid advancements, and be designed to prevent regulatory capture.

Regulators must possess or consult technical expertise to craft regulations that are both effective and practical.^[143] Without understanding the nuances of AI development, legislation may miss critical aspects or impose unworkable requirements.^[144] Engaging with AI researchers and ethicists can inform more precise and impactful regulatory measures.

Given the fast-paced evolution of AI, regulations must be flexible and subject to regular review and updating.^[145] Static regulations risk becoming obsolete, failing to address new challenges or hindering beneficial innovations.^[146] Mechanisms for periodic reassessment can ensure that regulations remain relevant and effective.

To prevent industries from unduly influencing regulations to their advantage, transparency and checks and balances are essential.^[147] Independent oversight bodies and public consultation processes can help mitigate the risk of regulatory capture, promoting fairness and accountability.^[148] The experience of S.B. 1047 illustrates both the importance and complexity of this principle. While the bill proposed creating a Board of Frontier Models, questions arose about ensuring this board’s true independence from industry influence. Traditional approaches to regulatory independence—such as cooling-off periods for industry executives or financial conflict of interest rules—may prove insufficient given the concentrated expertise in AI development.^[149]

Recent examples from other technical domains suggest possible solutions. The Nuclear Regulatory Commission’s use of resident inspectors who maintain offices at nuclear facilities while reporting to independent regional administrators offers one model for maintaining both technical competence and regulatory independence.^[150] Similarly, the FDA’s external advisory committees, which combine technical expertise with strict conflict of interest requirements, demonstrate how public sector oversight can maintain independence while accessing necessary technical knowledge.^[151] Moreover, effective oversight requires robust public consultation mechanisms that can meaningfully engage with technical complexities. The European Union’s approach to AI regulation, which includes mandatory public consultations and impact assessments, provides a valuable template.^[152]

Conclusion

S.B. 1047, though ultimately unsigned, represents a significant milestone in the evolution of AI regulation. While the bill did not become law, it succeeded in advancing the dialogue around AI safety and established important precedents for future regulatory frameworks. It acknowledges the potential for catastrophic harm and takes steps toward establishing safeguards and oversight mechanisms.^[153] The creation of resources modeled in it, like CalCompute, could democratize AI development, fostering innovation among startups and researchers.^[154]

The bill’s journey through the legislative process, including its eventual return by Governor Newsom, highlights both the urgency of AI regulation and the complexity of implementing effective oversight. The concerns raised during deliberation—such as potential ineffectiveness and the risk of stifling innovation—alongside the governor’s emphasis on the need for federal coordination, underscore the importance of a more nuanced, collaborative approach.^[155]

As California and other states continue to grapple with AI regulation, the lessons learned from history and S.B. 1047 provide valuable guidance. Future regulatory efforts should focus on deep technical understanding, engagement with diverse stakeholders, and if not uniformity, at least coordination across jurisdictions. They should strive to be flexible enough to mitigate the harms they are enacted to mitigate, but be targeted enough to avoid causing new, unintended harms. By crafting regulations that are informed, adaptable, and designed to encourage safe development, we can work toward harnessing the transformative power of AI while safeguarding society against its risks.

1. . Safe and Secure Innovation for Frontier Artificial Intelligence Models Act, S.B. 1047, 2023–2024 Leg., Reg. Sess. (Cal. 2024) (vetoed Sept. 2024). ↑

2. . Letter from Gavin Newsom, Governor of Cal., to Cal. State S. (Sept. 29, 2024), https://www. gov.ca.gov/wp-content/uploads/2024/09/SB-1047-Veto-Message.pdf [https://perma.cc/ZX4M-8T8B]. ↑

3. . S.B. 1047, supra note 1. CalCompute would have been a public cloud cluster, associated with the University of California, with a mission to support startups, researchers, and community groups that lack large-scale compute resources. Id. § 5. ↑

4. . Id. § 2(d). ↑

5. . See Cade Metz, Meet GPT-3. It Has Learned to Code (and Blog and Argue)., N.Y. Times (Nov. 24, 2020), https://www.nytimes.com/2020/11/24/science/artificial-intelligence-ai-gpt3.html [https://perma.cc/26C3-BUZU]. ↑

6. . S.B. 1047, supra note 1, § 3(e)(1)(A)(i), § 22603(a)(2A). ↑

7. . Id. § 4(b)–(f). The Board of Frontier Models would have been a body within the Government Operations Agency of the California government tasked with supervising the application of the bill and providing oversight of frontier AI models, including the ability, within the constraints of the statutory text, to define what models within the ambit of the bill are frontier models and which are not. Id. ↑

8. . Letter from Gavin Newsom, supra note 2. ↑

9. . See, e.g., Andrew Ng, The Problem with California’s AI Bill, Time (Aug. 29, 2024), https:// time.com/collections/time100-voices/7016134/california-sb-1047-ai/ [https://perma.cc/2PKK-WGUL] (arguing that S.B. 1047 misidentifies the locus of AI risk by targeting model development rather than bad acts, thereby discouraging open research and safety work while failing to address downstream misuse). ↑

10. . Stuart Russell, Human Compatible: Artificial Intelligence and the Problem of Control 20 (2019) (“Once AI systems move out of the laboratory (or artificially defined environments such as the simulated chessboard) and into the real world, there is very little chance that we can specify our objectives completely and correctly in such a way that the pursuit of those objectives by more capable machines is guaranteed to result in beneficial outcomes for humans.”). In complexity science, even extremely simple rule-based programs can produce behavior that cannot be predicted without effectively running the program itself. Stephen Wolfram illustrates this with “cellular automata,” tiny programs that repeatedly apply a single rule. One such program, Rule 30, follows a rule simple enough to state in one line yet produces patterns that resist shortcut prediction. Wolfram has offered a $30,000 prize for proofs about this system’s behavior, including whether its output at a given step can be computed substantially faster than by simulation. See Stephen Wolfram, Announcing the Rule 30 Prizes, Stephen Wolfram: Writings (Oct. 1, 2019), https://writings.stephenwolfram.com/2019/10/announ cing-the-rule-30-prizes/ [https://perma.cc/3EHQ-3GBP]; Eric W. Weisstein, Elementary Cellular Automata, Wolfram Mathworld https://mathworld.wolfram.com/ElementaryCellularAutomaton.ht ml [https://perma.cc/35B2-NVBP]. A similar intuition appears in physics: the three-body problem, governed by simple equations yet capable of chaotic dynamics and extreme sensitivity to initial conditions. See Shijun Liao & Xiaoming Li, On the Periodic Solutions of the Three-Body Problem, 6 Nat’l Sci. Rev. 1070 (2019). For modern machine-learning systems the difficulty is also formal: verifying even simple properties of deep neural networks can be NP-complete—meaning there is no known efficient general method for verifying the property. See Guy Katz, Clark Barrett, David Dill, Kyle Julian & Mykel Kochenderfer, Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks, Comp. Aided Verification, July 2017, at 97, https://arxiv.org/pdf/1702.01135 [https:// perma.cc/L3K3-UJJF]. ↑

11. . S.B. 1047, supra note 1, § 3(e)(1)(A)(i). ↑

12. . Zvi Mowshowitz, Guide to S.B. 1047, Don’t Worry About the Vase (Aug. 20, 2024), https://thezvi.wordpress.com/2024/08/20/guide-to-sb-1047/ [https://perma.cc/WBD5-WWX5]. ↑

13. . Id. ↑

14. . Id. ↑

15. . Paul Scharre, Future-Proofing Frontier AI Regulation: Projecting Future Compute for Frontier AI Models, CNAS (Mar. 13, 2024), https://www. cnas.org/publications/reports/future-proofing-frontier-ai-regulation   [https://perma.cc/QDN3-8BJC] (projecting that, under current hardware and efficiency trends, the cost of training a model at a fixed capability threshold could decline by roughly an order of magnitude over five years). ↑

16. . Kyle Wiggers, Anthropic’s Latest Flagship AI Might Not Have Been Incredibly Costly to Train, TechCrunch (Feb. 25, 2025), https://techcrunch.com/2025/02/25/anthropics-latest-flagship-ai-might-not-have-been-incredibly-costly-to-train/ [https://perma.cc/3DHF-G6QU] (quoting a statement by Wharton professor Ethan Mollick, who claimed to have received that information directly from Anthropic’s PR team). Whether the claim is true is certainly debatable, but the claim has not seen credulity from the public, so it is at least considered plausible by mainstream experts. ↑

17. . Critically, while S.B. 1047 allows for the Government Operations Agency to update the compute-resource threshold after January 1, 2027 as it sees fit, the bill in its final form did not allow for a similar reduction in the monetary threshold. See S.B. 1047, supra note 1, § 3(e)(1)(B)(i)(I)–(II). This flexibility to alter the compute threshold would allow the Government Operations Agency to, at its discretion, intentionally exempt some models that cost more than $100 million from regulation, but tied the hands of the agency to extend the regulation to the ever-increasing pool of models that could be trained for less than that amount. See id. ↑

18. . Brandon Vigliarolo, Y Combinator, Startups Funnily Enough Aren’t Fans of Draft California AI Safety Law, The Register (June 24, 2024), https://www.theregister.com/2024/06/24/ai_startups_ california_bill/ [https://perma.cc/S2D5-NJZ9] (quoting an open letter from Y Combinator cosigned by more than 140 machine-learning startups). The letter in question states that the ten to the twenty-sixth power threshold is entirely arbitrary. ↑

19. . In the extreme case, one can imagine a model much smaller than GPT-3, but which is trained on nothing but the necessary materials to reverse engineer a nuclear bomb or biological weapon. This model, requiring no creative writing skill or deep contextual understanding, would be more than capable of assisting a user in bringing about one of the worst harms imagined by S.B. 1047, but would be nowhere near the threshold required for regulation since GPT-3 was trained on an estimated 3.14 times ten to the twenty-third power, or just over 3 percent of the S.B. 1047 threshold. See Chuan Li, OpenAI’s GPT-3 Language Model: A Technical Overview, Lambda (June 3, 2020), https://lambda.ai/blog/ demystifying-gpt-3 [https://perma.cc/8QBC-M82M]. ↑

20. . See Gabriel Weil, The Pros and Cons of California’s Proposed SB-1047 AI Safety Law, Lawfare (May 8, 2024), https://www.lawfaremedia.org/article/california-s-proposed-sb-1047-would-be-a-major-step-forward-for-ai-safety-but-there-s-still-room-for-improvement [https://perma.cc/WZP9 -KVZB]. ↑

21. . See Owen J. Daniels, California AI Bill Becomes a Lightning Rod—for Safety Advocates and Developers Alike, Bull. of the Atomic Scientists (June 17, 2024), https://thebulletin.org/2024/06/ california-ai-bill-becomes-a-lightning-rod-for-safety-advocates-and-developers-alike/ [https://perma.cc /B7M7-A9P7] (explaining that after significant pushback from tech firms and trade associations, amendments removed a clause that would have applied safety requirements to more efficient future models trained with less compute). ↑

22. . Letter from Gavin Newsom, supra note 2. Nathan Calvin, who was an advisor on S.B. 1047, defended the bill’s approach by taking the stance that super-specialized small models, no matter how dangerous, are essentially beyond regulatory reach regardless of the regulatory approach. See 80,000 Hours, Nathan Calvin on California’s AI Bill S.B. 1047 and Its Potential to Shape US AI Policy, at 00:46:24 (Aug. 29, 2024) (Apple Podcasts). While that position may or may not be correct, without discussing whether the dangers of regulated models are categorically different than the dangers of unregulatable models, one wonders whether any safety justification for AI model regulation is sensible. Answering that question is outside the scope of this work, unfortunately. ↑

23. . See, e.g., Julia Carrie Wong, The Cambridge Analytica Scandal Changed the World – but It Didn’t Change Facebook, The Guardian (Mar. 18, 2019), https://www.theguardian.com/technology/ 2019/mar/17/the-cambridge-analytica-scandal-changed-the-world-but-it-didnt-change-facebook [https://perma.cc/WP97-YE72]. But see Facebook, Social Media Privacy, and the Use and Abuse of Data: Joint Hearing Before the S. Comm. on the Judiciary and the S. Comm. on Com., Sci. & Transp., 115th Cong. (2018) (testimony of Mark Zuckerberg, Chairman and CEO, Facebook), https://www. judiciary.senate.gov/imo/media/doc/Zuckerberg%20Testimony.pdf [https://perma.cc/BF5M-QUHD] (characterizing the incident as a breach of trust between a third-party developer and the platform rather than a failure of internal privacy systems, and emphasizing subsequent reforms to data access policies). ↑

24. . See, e.g., Resolución PS/00140/2020, Agencia Española de Protección de Datos [AEPD] [Spanish Data Protection Agency], May 18, 2022 (Spain) (levying a €10,000,000 fine for failing to demonstrate a sufficient legal basis for certain data processing); see also Délibération SAN-2021-024, Commission Nationale de l’Informatique et des Libertés [CNIL] [National Commission on Informatics and Liberty], Dec. 31, 2021 (Fr.) (levying a €90,000,000 fine, again for not having a sufficient basis to process data); Décision 37/2020, Chambre Contentieuse [Litigation Chamber], Autorité de Protection des Données [APD] [Belgian Data Protection Authority], July 14, 2020 (Belg.) (levying a €500,000 fine for insufficient protection of data subjects’ rights). ↑

25. . Compare GPT-4 System Card, OpenAI (Mar. 2023), https://cdn.openai.com/papers/gpt-4-system-card.pdf [https://perma.cc/57CV-J9YS] (documenting targeted adversarial testing while conceding the impossibility of comprehensively evaluating all potential vulnerabilities), with Stephen Ornes, The Unpredictable Abilities Emerging from Large AI Models, Quanta Mag. (Mar. 16, 2023), https://www.quantamagazine.org/the-unpredictable-abilities-emerging-from-large-ai-models-2023031 6/ [https://perma.cc/96GD-XXZ5] (highlighting the sudden, unforeseen capabilities that emerge from LLMs at scale, confounding developers’ expectations). ↑

26. . See Gary Marcus & Ernest Davis, Rebooting AI: Building Artificial Intelligence We Can Trust 15 (2019) (arguing that the field is stuck in a “local maximum” where a “short-term obsession” with profit-driving narrow AI distracts from the “much more challenging problem” of building systems that are “safe, smart, or reliable”). ↑

27. . Sharon Goldman, California AI Bill SB-1047 Sparks Fierce Debate, Senator Likens It to ‘Jets vs. Sharks’ Feud, Fortune (July 15, 2024), https://fortune.com/2024/07/15/california-ai-bill-sb-1047-fierce-debate-regulation-safety/ [https://perma.cc/G2MH-8267] (noting industry fears of “regulatory capture,” where regulations ultimately advance the interests of major developers like OpenAI and Google “at the expense of wider competition”). ↑

28. . Id. (quoting a venture capital firm’s open letter arguing the bill would “stifle open-source AI development and have a downstream chilling effect . . . [on] small business entrepreneurship”). ↑

29. . Id. (noting critiques that the bill relies on “frustratingly vague” standards and leaves key compliance thresholds to the broad discretion of an “unelected board”). ↑

30. . See generally Louis C. Hunter, Steamboats on the Western Rivers: An Economic and Technological History 14–18 (1949) (describing the frequent loss of pioneer vessels to “disaster and misfortune,” including sinkings due to fire and boiler explosions). ↑

31. . Andrea Sutcliffe, Steam: The Untold Story of America’s First Great Invention, at xii–xiii, 48–49, 94 (2004) (describing John Fitch’s 1787 and 1790 steamboat demonstrations and noting that despite providing regular service at half the price of stagecoaches, the venture remained unprofitable and unable to compete effectively with horse-drawn transport). Loss leading to drive adoption of new technologies, it seems, is a time-honored tradition. ↑

32. . Hunter, supra note 30, at 5, 8, 16 (noting that the 1807 Clermont voyage proved the “practicability of steam navigation” and provided the “original impetus” for a mode of transport that would eventually travel one hundred miles a day, effectively “conquering” the interior rivers for commerce). ↑

33. . Id. at 3, 21–22 (characterizing the steamboat as the “principal technological agent” that facilitated the “economic emancipation” of the interior by allowing for the upstream movement of northern “merchandise and manufactures” and the downstream transport of southern “western produce”). ↑

34. . See generally Southworth Allen Howland, Steamboat Disasters and Railroad Accidents in the United States (Worcester, Mass., Warren Lazell 1846) (compiling numerous historical accounts of steamboat accidents and their causes). ↑

35. . See Hunter, supra note 30. ↑

36. . See generally William Cronon, Nature’s Metropolis: Chicago and the Great West 79 (1991) (noting briefly that early steamboat accidents “entailed horrors of a sort never before seen”). ↑

37. . See generally Howland, supra note 34. ↑

38. . James T. Lloyd, Lloyd’s Steamboat Directory, and Disasters on the Western Waters 89 (Cincinnati, Ohio, James T. Lloyd & Co. 1856). ↑

39. . Id. at 89–90; Report of the Committee Appointed by the Citizens of Cincinnati, April 26, 1838, to Enquire into the Causes of the Explosion of the Moselle, and to Suggest Such Preventative Measures as May Be Best Calculated to Guard Hereafter Against Such Occurrences (Cincinnati, Ohio, A. Flash 1838). ↑

40. . See, e.g., [untitled], U.S. Gazette (Phila., Pa.), May 5, 1838, at 4 (listing the names of those believed to be dead, those believed to be missing, and those believed to be saved); Most Awful Steamboat Accident, The Sun (Balt., Md.), Apr. 30, 1838, at 1 (“Heads, limbs, bodies and blood, were seen flying through the air in every direction, attended by the most horrible shrieks and groans from the wounded and dying.”); [untitled], Times-Picayune (New Orleans, La.), June 13, 1838, at 2 (explaining that, contrary to rumor, the Moselle was not “blown up by a box of Brandreth’s pills. The Moselle took it in the natural way”). ↑

41. . See, e.g., The Explosion of the Moselle, Evening Post (N.Y.C.), May 4, 1838, at 2 (“[H]ow much so-ever we may be shocked at the frequent occurrence of these frightful accidents, we have no reason to be surprised. The dangers incident to the navigation of the Western waters, under the most careful management, are surely sufficiently numerous without adding to them the needless risk incurred by the foolish desire of shortening a journey by a few hours.”). ↑

42. . Act of July 7, 1838, ch. 191, 5 Stat. 304 (repealed 1852). The act is officially titled “An Act to provide for the better security of the lives of passengers on board of vessels propelled in whole or in part by steam.” Id. This scope is, perhaps wisely, limited in both focusing on loss of life rather than injury and in setting the goal as merely “better” than the status quo. In that sense, it is possible that the Act was actually a success. ↑

43. . Id. §§ 1–3. ↑

44. . Id. § 3. ↑

45. . Id. ↑

46. . Id. §§ 4–9. ↑

47. . Robert Frank Bennett, A Case of Calculated Mischief, 102 U.S. Naval Inst. Proceedings 877 (1976), https://www.usni.org/magazines/proceedings/1976/march/case-calculated-mischief [https: //perma.cc/ZM86-XLRV] (explaining the 1838 regime lacked centralized supervision or technical standards, so inspector rigor could be “very friendly or very harsh”). ↑

48. . George Henry Preble, A Chronological History of the Origin and Development of Steam Navigation (Phila., L.R. Hamersly & Co., 2d ed. 1895). ↑

49. . See Bennett, supra note 47. ↑

50. . Kris A. Hansen, Death Passage on the Hudson: The Wreck of the Henry Clay 15, 18–19 (2004) (describing the competing ventures operating in the area, and their resort to “unscrupulous tactics” like “offering free fares for travel . . . but then excessively pric[ing] meals and staterooms to make up the difference”). ↑

51. . Id. at 21 (describing the Henry Clay’s regular route and listing it along with the Francis and the Armenia as “opposition day boats for the 7:00 A.M. run” (internal quotations omitted)). ↑

52. . The Henry Clay Catastrophe, N.Y. Daily Times, Aug. 2, 1852, at 1. It is notable that all officers on the Henry Clay who survived were acquitted of federal manslaughter charges. Hansen, supra note 50, at 155. Since carrying passengers without a license was a manslaughter offense under the Act of 1838, it can be reasonably inferred that the Henry Clay was in possession of a license, which means that at least one steamboat inspector believed that two lifeboats would be sufficient for 500 passengers. ↑

53. . Hansen, supra note 50, at 76. There were a number of more deadly steamboat disasters, and steamboat disasters that occurred under more questionable circumstances between 1847 and 1852. Perhaps because this occurred in such an urban location, or perhaps because many of the victims were wealthy and prominent, this accident, like the Moselle before it, became national news. ↑

54. . Id. at 77–106 (collecting examples of coverage of the public outrage). ↑

55. . Id. at 93. ↑

56. . Act of Aug. 30, 1852, ch. 106, 10 Stat. 61 (repealed Feb. 28, 1871). ↑

57. . Id. § 9. ↑

58. . Id. ↑

59. . Id. ↑

60. . Id. ↑

61. . Id. §§ 4, 9. ↑

62. . Id. §§ 1, 9. ↑

63. . Id. § 11 (prohibiting masters or engineers from “load[ing] or obstruct[ing] . . . the safety valve or valves of a boiler:” or otherwise subjecting a boiler to a pressure greater than that allowed by the inspectors’ certificate). ↑

64. . Modern Formula One similarly encourages intense competition by imposing uniform technical constraints—rules governing engines, aerodynamics, safety systems, and other vehicle components—thereby creating a level playing field within which teams compete for speed and performance. See, e.g., FIA Formula One World Championship, Fédération Internationale de l’Automobile (Feb. 27, 2026), https://www.fia.com/regulation/category/110 [https://perma.cc/C55H-SJZT] (choose “FIA 2026 FIA Regulations – Section C [Technical]” from 2026 Regulations dropdown menu). ↑

65. . See generally Howland, supra note 34 (detailing accidents in the date range between 1852 and 1871). ↑

66. . Jerry O. Potter, The Sultana Tragedy: America’s Greatest Maritime Disaster 3 (1992). ↑

67. . Id. at ix–x. ↑

68. . Id. at 131 (describing and analyzing highly varied contemporaneous death toll counts using updated passenger numbers and concluding the death toll probably exceeded 1,700). ↑

69. . Id. at 51–57, 136–57 (describing the last boilermaker to see the Sultana’s boiler as initially refusing to patch the boiler without more extensive repair, but eventually relenting under pressure from the boat’s crew); Gene Eric Salecker, Destruction of the Steamboat Sultana: The Worst Maritime Disaster in American History 148–58 (2022) (detailing the maintenance-related causes of the boiler explosion). ↑

70. . Alan Huffman, Sultana: Surviving the Civil War, Prison, and the Worst Maritime Disaster in American History 128–33, 146–49 (2009) (detailing the vessel’s known mechanical deficiencies, the intentional postponement of substantive repairs, the routine ignoring of transport regulations, and the corruption-plagued contracting system that incentivized overloading). ↑

71. . Id. at 138–43 (recounting testimony that the vessel was dangerously overloaded, that warnings were ignored, and that although officials recorded fewer than two thousand passengers, the true number approached 2,600). ↑

72. . Act of Feb. 28, 1871, ch. 100, 16 Stat. 440 (providing “[F]or the better Security of Life on board of Vessels propelled in Whole or in Part by Steam” and establishing a Steamboat Inspect Service). ↑

73. . See id. §§ 2, 4, 6–8, 11, 13–20, 63. ↑

74. . Id. §§ 1–12 (outlining the responsibilities of the newly created Steamboat Inspection Service). ↑

75. . Id. § 11. ↑

76. . Id. ↑

77. . Id. § 35. ↑

78. . Id. §§ 14, 17–18 (establishing mandatory federal examination and annual licensing requirements for engineers and pilots, with suspension and revocation procedures). ↑

79. . Id. §§ 14–16 (extending mandatory federal licensing and character examinations to captains, masters, and chief mates). ↑

80. . Id. § 9. ↑

81. . Scott Price, 225 Years of Service to Nation: Marine Safety, U.S. Coast Guard Historian’s Off. (June 8, 2015), reproduced at https://www.mypspa.org/article/more/225-years-of-service-to-nation %3A-marine-safety [https://perma.cc/KA2S-VH7L] (original no longer available) (noting that the 1871 Act established the comprehensive ‘Marine Safety Code’ that remains the basis of the Coast Guard’s modern regulatory authority following the permanent transfer of these functions in 1946). ↑

82. . See generally Upton Sinclair, The Jungle (1906). ↑

83. . Pure Food and Drug Act, ch. 3915, 24 Stat. 768 (1906) (repealed by Food, Drug, and Cosmetic Act, 21 U.S.C. §§ 301–399i) (“[P]reventing the manufacture, sale, or transportation of adulterated or misbranded or poisonous or deleterious foods, drugs, medicines, and liquors . . . .”); Meat Inspection Act of 1906, Pub. L. No. 59-382, 34 Stat. 674 (codified as amended at 21 U.S.C. §§ 601–695). ↑

84. . See generally James Harvey Young, Pure Food: Securing the Federal Food and Drugs Act of 1906 (1989) (discussing the historical weaknesses of the PFDA and its subsequent amendments). ↑

85. . Daniel Carpenter, Reputation and Power: Organizational Image and Pharmaceutical Regulation at the FDA 116 (2010). ↑

86. . See James Harvey Young, The Medical Messiahs: A Social History of Health Quackery in Twentieth-Century America 36–37 (1967). ↑

87. . 221 U.S. 488 (1911). ↑

88. . Id. at 497–98. ↑

89. . See Young, supra note 84, at 88. ↑

90. . See Pure Food and Drug Act, ch. 3915, § 6, 24 Stat. 768, 769 (1906) (repealed by Food, Drug, and Cosmetic Act of 1938, 21 U.S.C. §§ 301–399i) (setting the scope of the act). ↑

91. . See id. ↑

92. . Id. ↑

93. . See, e.g., United States v. Lexington Mill & Elevator Co., 232 U.S. 399, 411 (1914) (“The act has placed upon the Government the burden of establishing, in order to secure a verdict of condemnation under this statute, that the added poisonous or deleterious substances must be such as may render such article injurious to health.”). ↑

94. . Id. at 412 (“As to the use of the term ‘poisonous,’ let me state that everything which contains poison is not poison. It depends on the quantity and the combination.” (quoting 40 Cong. Rec. app. at 1131 (1906 (statement of Sen. Heyburn))). ↑

95. . Id. at 411. It is noteworthy that the Court in Lexington Mill made its determination, in part, on a textualist analysis of the phrase “render such article injurious to health.” While the phrase could be read in a probabilistic way, the Court chose to read it in the determinative sense. Id. It is possible, if not likely, that any court focused on textualist analysis will look for similar ways to constrain the extent of ambiguous terms to raise the evidentiary bar for a regulation based on showings of extant potential harms in the marketplace. ↑

96. . Meat Inspection Act, ch. 3913, 34 Stat. 669, 674 (1906) (codified as amended at 21 U.S.C. §§ 601–695). ↑

97. . James Harvey Young, The Pig That Fell into the Privy: Upton Sinclair’s The Jungle and the Meat Inspection Amendments of 1906, 59 Bull. Hist. Med. 467 (1985). ↑

98. . See Patrick Newman, The Big Meat: The Beef Trust, Regulatory Capture, and Government Intervention 14 (July 13, 2018) (unpublished manuscript), https://ssrn.com/abstract=3213676 [https:// perma.cc/7LFG-LARH]. ↑

99. . Id. at 9. ↑

100. . Id. at 41–42. ↑

101. . Id. at 26. Large industry players also saw the opportunity to channel popular sentiments into a concern for safety, rather than allow Upton Sinclair’s intended message (support for socialism) to take strong hold. Id. at 45. ↑

102. . See generally Gabriel Kolko, The Triumph of Conservatism: A Reinterpretation of American History, 1900–1916 (1963) (arguing that the MIA and other progressive-era legislation was ultimately favorable to big business because it standardized production costs and drove smaller competitors out of markets). ↑

103. . Newman, supra note 98, at 3. ↑

104. . Id. at 8. ↑

105. . S.B. 1047, supra note 1. ↑

106. . Id. ↑

107. . A post-S.B. 1047 model out of China, called DeepSeek R1, was trained on an estimated three septillion operations, about 3 percent of S.B. 1047’s operation threshold of 100 septillion. Ege Erdil, What Went into Training DeepSeek-R1?, Epoch AI (Jan. 31, 2025), https://epoch.ai/gradient-updates/ what-went-into-training-deepseek-r1 [https://perma.cc/L8C2-8S4E] (explaining the process of training DeepSeek R1 from a previous model and estimating a total of 3e24 calculations, where 3e24 is the scientific notation for three septillion). DeepSeek R1 was trained for a purported $6.3 million, or 6.3 percent of S.B. 1047’s $100 million threshold for inclusion in the regulatory scheme. Id. Despite falling well below S.B. 1047’s regulatory threshold, DeepSeek R1 scored higher on a total-capability benchmark than frontier models from only a few months before its release. See Mike Knoop, An Analysis of DeepSeek’s R1-Zero and R1, ARC Prize (Jan. 29, 2025), https://arcprize.org/blog/r1-zero-r1-results-analysis [https://perma.cc/7C4P-CL46]. The ARC Prize foundation is an organization whose mission is to create useful benchmarking for the capabilities of frontier AI models. Id. ↑

108. . S.B. 1047, supra note 1. ↑

109. . Food, Drug, and Cosmetics Act of 1938, 21 U.S.C. § 355. ↑

110. . See generally Newman, supra note 98. ↑

111. . Locomotive Act of 1865, 28 & 29 Vict. c. 83 (Eng.). ↑

112. . Along with the Red Flag Act of 1865, the Locomotives on Roads Acts include the Locomotive Act 1861, 24 & 25 Vict. c. 70 (Eng.) and the Highways and Locomotives (Amendment) Act 1878, 41 & 42 Vict. c. 77 (Eng.). Each of these Acts built on the foundation laid by the previous Acts. See, e.g., Phillip S. Bagwell, The Transport Revolution 1770–1985, at 187 (1988); Kenneth Richardson, The British Motor Industry 1896–1939, at 12 (1977). ↑

113. . Locomotive Act of 1865 § 3. ↑

114. . Id. § 4. ↑

115. . Id. § 3. ↑

116. . See Richardson, supra note 112. ↑

117. . See M.G. Lay, Ways of the World: A History of the World’s Roads and of the Vehicles That Used Them 139–41 (1992). A slightly more humorous, if less specific, attitude might be that captured by Thomas Hood’s poem, Conveyances, where the poet said, referencing the Gurney Steam Bus, “Instead of journeys, people now May go upon a Gurney, With steam to do the horses’ work, By powers of attorney; Tho’ with a load it may explode, And you may all be un-done! And find you’re going up to Heav’n, Instead of up to London!” Thomas Hood, Conveyances, in The Comic Poems of Thomas Hood 413, 413–14 (London, E. Moxon, Son & Co. 1876). ↑

118. . See Lay, supra note 117, at 140. ↑

119. . HC Deb (26 Apr. 1865) (178) col. 1061 (statement of Sir George Grey). ↑

120. . Id. (“At the same time it should be remembered that these engines were formidable affairs in narrow lanes. Instances of fatal accidents had occurred . . . .”); id. at col. 1067 (statement of Mr. Fellowes) (describing nine-foot-wide locomotive machines traveling down twelve-foot-wide roads). ↑

121. . Id. at col. 1063 (statement of Mr. Henley). ↑

122. . Rhodri Marsden, Rhodri Marsden’s Interesting Objects: Lord Winchelsea’s Red Flag, The Independent (July 3, 2015), https://www.the-independent.com/life-style/motoring/features/rhodri-marsden-s-interesting-objects-lord-winchelsea-s-red-flag-10358534.html   %5Bhttps://perma.cc/N72B-G5B6%5D (“The [1865] Act followed intense lobbying from horse-drawn carriage operators and the public railway industry.”). ↑

123. . A driver (“driver of the engine”), a stoker (“man on the engine”), and the flagman (who would “warn the riders and drivers of horses of the approach of such locomotive”). HC Deb (26 Apr. 1865) (178) cols. 1063, 1065. ↑

124. . Locomotive Act of 1865, 28 & 29 Vict. c. 83, §§ 3–4 (Eng.). ↑

125. . That someone should walk ahead of the vehicle was generally agreed upon. How far ahead of the vehicle they should walk was intensely discussed, with proposals including a minimum distance of one-hundred yards, a maximum distance of one-hundred yards, a minimum of sixty yards, a maximum of sixty yards, immediately in front of the vehicle, and a proposal that the distance be left to the judgement of the crews so as to account for differing road conditions. HC Deb (26 Apr. 1865) (178) cols. 1063–64. One might be inclined to wonder whether, with good-faith intuitions favoring each of these very different approaches, a prudent legislature might conclude that it was in over its head. ↑

126. . For example, one member of Parliament argued that “[t]he slower these engines travelled the greater obstruction would they offer to the ordinary traffic, and he almost believed that in the metropolis a speed of ten miles an hour would be better than two, as less obstruction would ensue.” Id. at col. 1060 (statement of Mr. Ayrton). Mr. Ayrton might therefore be considered a steam car accelerationist, not unlike artificial intelligence accelerationists in our time. ↑

127. . See Lay, supra note 117, at 153 (describing the introduction to the general public of the internal combustion engine between 1882 and 1886 by Benz and Daimler, both of whom were German). ↑

128. . Locomotive Act of 1865, 28 & 29 Vict. c. 83, § 3 (Eng.) (“Every Locomotive propelled by Steam or any other than Animal Power on any Turnpike Road or public Highway shall be worked according to the following Rules and Regulations . . . .”). ↑

129. . See generally 2 T.R. Nicholson, The Birth of the British Motorcar: 1769–1897, 252–83 (1982) (detailing the many critiques that were given voice in Parliament over the 1870s). ↑

130. . Id. ↑

131. . Id. ↑

132. . John Henry Knight, Ten Years Progress, in A History of the First Ten Years of Automobilism 30, 30 (Douglas Scott-Montagu & John Edward eds., 1906) (“Among the names of those who pushed forward the movement may be mentioned Sir David Salomons, the Hon. Evelyn Ellis, Colonel Holden, Messrs. Shaw Lefevre, C. S. Rolls, F. R. Simms, and H. Sturmey.”). ↑

133. . See Locomotives on Highways Act 1896, 59 & 60 Vict. c. 36 (Eng.). ↑

134. . See History of the Run, Veteran Car Run, https://www.veterancarrun.com/history-of-the-run [https://perma.cc/MSZ5-6HAX] (noting that the 1986 “Emancipation Run” celebrated the Locomotives on Highways Act and involved motorists driving sixty miles from London to Brighton). ↑

135. . Marsden, supra note 122. ↑

136. . See Lay, supra note 117. ↑

137. . See HC Deb (26 Apr. 1865) (178) col. 1065; see also Nicholson, supra note 129; Marsden, supra note 122. It is beyond the scope of this Note to prove that such sentiment is correct—it suits the needs of my argument that such sentiment existed prior to enactment of the Red Flag Acts, while they were in effect, and through to at least 2015. ↑

138. . S.B. 1047, supra note 1; Letter from Gavin Newsom, supra note 2. ↑

139. . See, e.g., Adam Thierer, California Rejects AI Regulatory Extremism, R Street Inst. (Sept. 30, 2024), https://www.rstreet.org/commentary/california-rejects-ai-regulatory-extremism/ [https://per ma.cc/4MYK-D9CW] (discussing the above objections from the perspective of an academic skeptic). ↑

140. . See discussion supra Section II.B.2. ↑

141. . Id. ↑

142. . Lawmakers, at least at the federal level, are reported to struggle to understand artificial intelligence technologies. See, e.g., Cecelia Kang & Adam Satariano, As A.I. Booms, Lawmakers Struggle to Understand the Technology, N.Y. Times (Mar. 3, 2023), https://www.nytimes.com/2023/ 03/03/technology/artificial-intelligence-regulation-congress.html [https://perma.cc/3QNZ-EUW2]. ↑

143. . See Erik Brynjolfsson & Andrew McAfee, The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies 189–91 (2014). ↑

144. . See Amitai Etzioni & Oren Etzioni, Incorporating Ethics into Artificial Intelligence, in Happiness Is the Wrong Metric 235 (2018). ↑

145. . Commission Proposal for a Regulation of the European Parliament and of the Council Laying Down Harmonised Rules on Artificial Intelligence (Artificial Intelligence Act) and Amending Certain Union Legislative Acts, COM (2021) 206 final (Apr. 4, 2021). ↑

146. . See Jess Whittlestone, Anna Alexandrova, Rune Nyrup & Stephen Cave, The Role and Limits of Principles in AI Ethics: Towards a Focus on Tensions, 2019 AAAI/ACM Conf. on AI Ethics & Soc’y 195, 196–98. ↑

147. . See George J. Stigler, The Theory of Economic Regulation, 2 Bell J. Econ. & Mgmt. Sci. 3, 4–6 (1971). ↑

148. . Archon Fung, Infotopia: Unleashing the Democratic Power of Transparency, 41 Pol. & Soc’y 183, 185–87 (2013). ↑

149. . See Amba Kak, Sarah Myers West & Meredith Whittaker, Make No Mistake—AI Is Owned by Big Tech, MIT Tech. Rev. (Dec. 5, 2023), https://www.technologyreview.com/2023/12/05/10843 93/make-no-mistake-ai-is-owned-by-big-tech [https://perma.cc/EU46-N8EV] (explaining that the vast majority of AI model development is performed by only five private firms). ↑

150. . U.S. Nuclear Regul. Comm’n, NUREG-1649 (Rev. 6), Reactor Oversight Process (2016). ↑

151. . Emily C. Helms Williams, Namandjé N. Bumpus & Robert M. Califf, Comment, The Role of FDA Advisory Committees, 30 Nature Med. 3050, 3052 (2024). ↑

152. . Commission White Paper on Artificial Intelligence – A European Approach to Excellence and Trust, COM (2020) 65 final (Feb. 19, 2020) (detailing public consultation requirements in EU AI regulation). ↑

153. . S.B. 1047, supra note 1. ↑

154. . Id. § 3(d). ↑

155. . See Ng, supra note 9. ↑

156. * J.D. Candidate, Washington University School of Law, 2026; B.S., Excelsior University, 2023. I am grateful to Professor Richards for encouraging my interest in drawing on the past to inform the future, and to the editors of the Washington University Law Review for their patience, care, and guidance. This Note is dedicated to my mother, who gave me and my siblings our first education, our introductions to the classics, and innumerable haircuts; to my wife, Angela, who still cuts my hair and whose support made law school possible; and to my daughters, Evie and Keira, who do not cut my hair but keep my ego neatly trimmed. All the errors in this piece are mine alone. ↑