Policy & Regulation
EU AI Act and deepfake legislation
The regulatory landscape for deepfakes and AI content
Why governments are acting now
Three forces are pushing legislators to act on synthetic media. First, deepfake generation tools have become accessible to anyone with a web browser, eliminating the technical barriers that once limited who could create convincing fakes. Second, several high-profile incidents involving election interference, financial fraud, and non-consensual intimate content have made the human cost of unregulated deepfakes visible to the public and to lawmakers. Third, the volume of AI-generated content online has reached a scale where manual identification is no longer realistic, creating pressure for systemic solutions rather than case-by-case enforcement.
The legislative response is uneven. Some jurisdictions are moving quickly with comprehensive frameworks while others are addressing specific harms through narrow, targeted laws. Understanding the full landscape matters because content provenance technology sits at the intersection of nearly all of these regulatory efforts.
The three regulatory approaches
Deepfake legislation generally follows one of three strategies, and most jurisdictions use a combination.
Disclosure
Requires creators and platforms to label AI-generated content. The EU AI Act and several US state laws take this approach. Disclosure mandates create demand for provenance technology that can attach and verify labels.
Prohibition
Bans specific harmful uses of deepfakes, such as non-consensual intimate imagery, election interference, or fraud. The US TAKE IT DOWN Act and Texas SB 751 follow this model. Prohibition creates demand for detection technology that can identify violations.
Provenance
Mandates that content carry verifiable origin information throughout its lifecycle. Article 50 of the EU AI Act moves in this direction. Provenance requirements create demand for both metadata-based systems like C2PA and forensic verification like AFIP.
Platform liability
Holds platforms responsible for hosting or distributing harmful deepfakes when they fail to act on reports. The UK Online Safety Act and proposed US legislation explore this angle. Platform liability drives investment in automated detection infrastructure.
Key legislation timeline
Texas SB 751 becomes the first US state law specifically targeting deepfakes used in elections, criminalizing the creation of deceptive videos within 30 days of an election.
California AB 730 and AB 602 address election deepfakes and non-consensual deepfake pornography respectively, establishing the two-track approach many states would follow.
China's Deep Synthesis Provisions take effect, requiring watermarking, labeling, and real-name registration for AI-generated content. The most comprehensive national regulation at the time.
EU AI Act reaches political agreement. Article 50 establishes transparency obligations for AI systems that generate synthetic content, including labeling and machine-readable marking requirements.
UK Online Safety Act comes into force, including provisions making it a criminal offense to share deepfake intimate images without consent.
US TAKE IT DOWN Act signed into law, making it a federal crime to publish non-consensual intimate imagery (including AI-generated) and requiring platforms to remove it within 48 hours of a valid takedown request.
EU AI Act full enforcement begins, with transparency obligations for AI-generated content applying to all providers operating in the EU market.
EU AI Act: provenance and AI labeling requirements
Article 50: transparency obligations for AI systems
Article 50 of the EU AI Act establishes specific transparency requirements for AI systems that generate synthetic content. The obligations apply to providers of AI systems, not end users, meaning the companies building and deploying AI tools bear responsibility for compliance.
The key requirements under Article 50 include marking AI-generated outputs in a machine-readable format that allows downstream systems to detect them, ensuring that AI-generated audio, image, video, or text content is labeled as artificially generated or manipulated when it is presented to the public, and providing this information in a way that is clear and distinguishable.
The EU AI Act requires machine-readable marking, not just visible labels. This means technical implementation through watermarking, metadata, or embedded signals, not simply adding a text disclaimer. This requirement creates a direct regulatory mandate for technologies like C2PA Content Credentials and AI watermarking systems.
Deepfake disclosure requirements
The Act specifically addresses deepfakes in Article 50(4), requiring that persons who deploy an AI system that generates or manipulates content constituting a deepfake must disclose that the content has been artificially generated or manipulated. The disclosure must be made in a clear and distinguishable manner no later than the time of first interaction or exposure.
There are limited exceptions for content that is obviously artistic, satirical, or fictional, where the labeling would be unnecessary given the context. However, the boundaries of these exceptions are not precisely defined and will likely be clarified through case law and guidance from regulators.
Enforcement and penalties
The EU AI Act operates on a tiered penalty structure. Violations of the transparency requirements in Article 50 can result in fines of up to 15 million euros or 3% of annual worldwide turnover, whichever is higher. For comparison, the highest tier of penalties (for prohibited AI practices) can reach 35 million euros or 7% of turnover.
Enforcement is handled by national market surveillance authorities in each EU member state, with coordination through the European AI Office. The practical enforcement mechanisms are still being established, and the first enforcement actions under the transparency provisions are expected to set important precedents.
Implementation timeline and compliance deadlines
The AI Act entered into force on August 1, 2024, with a phased implementation schedule. Prohibited AI practices became enforceable in February 2025. The transparency obligations under Article 50, including the deepfake labeling requirements, become fully applicable in August 2026. Organizations that deploy AI systems generating synthetic content need to have their compliance infrastructure in place by that deadline.
United States federal legislation
The TAKE IT DOWN Act
Signed into law in 2025, the TAKE IT DOWN Act (formally the Tools to Address Known Exploitation by Immobilizing Technological Deepfakes on Websites and Networks Act) represents the first comprehensive federal legislation specifically targeting deepfake content in the United States. The law makes it a federal crime to knowingly publish or threaten to publish non-consensual intimate imagery, whether real or AI-generated.
The Act also imposes obligations on online platforms, requiring them to establish procedures for receiving and processing takedown requests and to remove identified content within 48 hours. Platforms that fail to comply face enforcement action from the Federal Trade Commission.
The AI Deepfake Act and related bills
Several other federal bills have been introduced to address deepfakes more broadly. The AI Deepfake Act would require clear labeling of AI-generated content used in political advertising. The NO FAKES Act (Nurture Originals, Foster Art, and Keep Entertainment Safe) focuses on protecting individuals' digital likeness from unauthorized AI replication, creating a federal right to control the use of one's voice and image in AI-generated content.
The DEFIANCE Act targets non-consensual intimate deepfakes specifically, creating a federal civil cause of action that allows victims to sue creators and distributors. While the TAKE IT DOWN Act addresses this through criminal law, the DEFIANCE Act would add a civil remedy.
FTC and regulatory enforcement
The Federal Trade Commission has been active in addressing AI-generated content through its existing authority over unfair and deceptive practices. The FTC has issued enforcement warnings about AI-generated deceptive content and proposed rules that would address AI impersonation. The agency's approach treats deepfakes used for fraud or commercial deception as violations of existing consumer protection law rather than waiting for AI-specific legislation.
US state deepfake laws
State legislatures have been more aggressive than Congress in passing deepfake-specific laws. The state-level landscape is complex and evolving rapidly, with different states addressing different aspects of the deepfake problem.
| State | Focus area | Key provision | Year |
|---|---|---|---|
| California | Elections + intimate imagery | AB 730 prohibits deepfakes in elections within 60 days of an election; AB 602 addresses non-consensual intimate deepfakes | 2019 |
| Texas | Elections | SB 751 criminalizes deepfake videos intended to influence elections within 30 days of voting | 2019 |
| Virginia | Intimate imagery | Extended existing revenge porn law to cover AI-generated intimate images | 2019 |
| New York | Right of publicity | Protects digital likeness from unauthorized AI replication, covering both living and deceased persons | 2023 |
| Minnesota | Elections + intimate imagery | Comprehensive deepfake bill covering both election deception and non-consensual intimate content | 2023 |
| Indiana | Intimate imagery | Criminalizes distribution of AI-generated intimate images of minors and adults without consent | 2023 |
| Washington | Elections + consent | Requires disclosure for AI-generated political ads and creates civil liability for non-consensual deepfakes | 2024 |
| Tennessee | Voice and likeness | ELVIS Act protects artists' voice and likeness from AI cloning, the first state law specifically addressing AI voice replication | 2024 |
The patchwork of state laws creates compliance challenges for organizations operating across multiple jurisdictions. A deepfake that is legal in one state may violate the law in another. This inconsistency is one of the arguments driving federal legislative efforts to create a unified national standard.
Global regulatory approaches
China: Deep Synthesis Provisions
China's Deep Synthesis Provisions, administered by the Cyberspace Administration of China (CAC), represent the most comprehensive national regulation of AI-generated content. Effective January 2023, the regulations require providers of deep synthesis services to implement real-name user verification, add visible labels to synthetic content, embed technical identifiers (watermarks or metadata) that machine systems can detect, and maintain records of generated content for regulatory review.
The Chinese approach is notable for explicitly requiring both human-visible and machine-readable identification, going beyond what the EU AI Act initially mandated. Enforcement has included fines and service suspensions for platforms that failed to properly label AI-generated content.
United Kingdom: Online Safety Act
The UK addressed deepfakes primarily through the Online Safety Act 2023, which makes it a criminal offense to share intimate deepfake images without consent. The Act also places duties on platforms to proactively remove illegal content and to protect users from content that is harmful but not necessarily illegal.
The UK's approach focuses on platform responsibility rather than content labeling. Unlike the EU AI Act, it does not mandate technical provenance measures for AI-generated content broadly but does require platforms to have systems in place to detect and address the most harmful categories of synthetic media.
Canada, Australia, and other jurisdictions
Canada's proposed Artificial Intelligence and Data Act (AIDA) includes provisions for AI system transparency but does not specifically address deepfakes with the same detail as the EU AI Act. The bill has faced delays and may be revised before passage.
Australia's government launched a review of AI regulation in 2024, with recommendations expected to address synthetic media. Japan amended its Copyright Act to address AI training data but has been slower to regulate AI-generated content directly. South Korea introduced bills targeting deepfakes used in sexual exploitation following several high-profile cases.
The global trend is clear: nearly every major economy is developing or has enacted some form of AI content regulation. The differences lie in scope, enforcement mechanisms, and the balance between disclosure requirements and outright prohibitions.
Implications for content provenance
How regulation drives provenance adoption
Regulation is the single largest driver of content provenance technology adoption. When laws require AI-generated content to carry machine-readable identification, organizations need technical infrastructure to implement that requirement. This dynamic is accelerating investment in both metadata-based systems (like C2PA Content Credentials) and forensic verification approaches.
The EU AI Act's machine-readable marking requirement is particularly significant because it goes beyond simple disclosure. A visible disclaimer can be removed or ignored. Machine-readable marking requires embedding persistent, verifiable signals into the content itself, which is precisely what watermarking and provenance metadata technologies are designed to do.
Forensic verification as compliance evidence
While most current legislation focuses on what content creators and platforms must do, organizations also need ways to demonstrate compliance. Forensic verification fills this gap by providing independent, evidence-based assessment of whether content carries the required provenance signals and whether it has been generated or manipulated by AI.
Self-reported compliance
- Creator attaches provenance metadata
- Platform checks for metadata presence
- Relies on creator cooperation
- Metadata can be stripped or faked
- Cannot verify content that lacks labels
Forensic verification
- Independent analysis of content itself
- Works regardless of metadata presence
- Does not require creator cooperation
- Evidence-based, not claim-based
- Can assess unlabeled content for compliance gaps
AFIP's Forensic Integrity Protocol is designed to serve as compliance infrastructure. By analyzing content forensically, organizations can verify that AI-generated material has been properly labeled, identify unlabeled AI content that should have been disclosed, and generate audit trails documenting their compliance efforts.
What organizations need to do now
With the EU AI Act's transparency requirements becoming enforceable in August 2026 and US federal and state laws already in effect, organizations that deploy or distribute AI-generated content should be taking concrete steps to prepare.
The future of AI content regulation
Several trends are shaping where AI content regulation is heading. International harmonization efforts are underway through bodies like the G7 Hiroshima AI Process and the OECD AI Policy Observatory, though a unified global standard remains unlikely in the near term. The technical standards underlying compliance are also maturing, with C2PA releasing updated specifications and AFIP developing forensic verification standards designed to complement metadata-based approaches.
The regulatory direction is toward more comprehensive requirements, not fewer. Early laws focused on specific harms like election interference and intimate imagery. Newer frameworks like the EU AI Act address AI-generated content broadly. Future legislation is likely to expand requirements for provenance documentation across all AI output types, including text, which current regulations largely exempt.
The organizations that invest in content provenance infrastructure now will be best positioned to comply with both current and future regulations. The technical requirements will only increase from here.
Verify compliance with AFIP forensic analysis
Use AFIP's forensic tools to assess whether your AI-generated content meets regulatory transparency requirements.
Try AFIP VerifyFrequently asked questions
Does the EU AI Act apply to organizations outside the EU?
Yes. Like the GDPR, the EU AI Act has extraterritorial reach. It applies to any provider that places an AI system on the EU market or whose AI system's output is used within the EU, regardless of where the provider is based. Organizations outside Europe that serve EU users or markets must comply with the transparency requirements.
What counts as a deepfake under current legislation?
Definitions vary by jurisdiction. The EU AI Act defines deepfakes broadly as AI-generated or manipulated content that "appreciably resembles existing persons, objects, places or other entities or events and would falsely appear to a person to be authentic." US state laws tend to be narrower, typically targeting content that depicts a real, identifiable person in a misleading way. Some laws cover only video, while others include audio, images, and text.
Are AI-generated text and writing covered by deepfake laws?
Most current deepfake laws focus on visual and audio content. The EU AI Act is an exception, requiring that AI-generated text published for the purpose of informing the public on matters of public interest must be labeled. Several proposed US federal bills would also cover AI-generated text in specific contexts like political advertising. As AI text generation becomes more prevalent, expect broader coverage in future legislation.
What is the difference between the EU AI Act and Content Credentials?
The EU AI Act is a law that creates legal obligations. Content Credentials (based on the C2PA standard) is a technology that can help organizations meet those obligations. The law requires machine-readable marking of AI content; Content Credentials provide one technical mechanism for implementing that requirement. However, C2PA alone may not satisfy all compliance needs, since it relies on creator cooperation and metadata can be stripped during distribution.
How does AFIP help with regulatory compliance?
AFIP provides forensic verification that works independently of metadata or labels. This means organizations can use AFIP tools to audit their content pipeline for unlabeled AI material, verify that provenance metadata has been properly attached, detect AI-generated content in user submissions or third-party sources, and maintain evidence-based compliance documentation for regulators.