AI provenance

Lisa Braswick

AI Research & Content Strategy

A photograph has a camera sensor. A handwritten letter has ink and paper. An oil painting has brushstrokes and canvas. Each of these carries physical evidence of its origin, evidence that forensic examiners have spent decades learning to read. AI-generated content has none of these anchors. It emerges from a mathematical process, a neural network transforming noise into pixels, tokens, or audio samples with no inherent trace of its creation.

This absence is what makes AI provenance one of the defining challenges of the current decade. As generative AI produces images, text, audio, and video that are increasingly indistinguishable from human-created content, the ability to determine what was made by a machine and what was made by a person is becoming both a technical and societal imperative.

What is AI provenance

Why traditional provenance breaks down for AI content

Traditional digital provenance relies on a chain of custody that begins with a capture device. A camera embeds EXIF metadata recording the device model, GPS coordinates, timestamp, and exposure settings. A document processor records the author, creation date, and edit history. These signals provide a starting point for verification.

AI-generated content breaks this model entirely. A diffusion model producing an image does not use a camera sensor. A large language model writing text does not have a keyboard. There is no capture device, no geographic location, and no human author in the traditional sense. The content materializes from learned statistical patterns, and unless the generating system deliberately records provenance information, none exists.

The unique challenge: no camera, no human creator

This gap creates a verification problem that did not exist before 2022 at any meaningful scale. Prior to the widespread availability of tools like Stable Diffusion, Midjourney, DALL-E, and ChatGPT, the assumption that media content had a human origin was reasonable. Fakes existed, but they required effort and skill. Now, anyone can generate photorealistic images, convincing text, or cloned audio in seconds, and the output carries no inherent marker of its synthetic origin.

AI provenance vs data provenance

AI provenance and data provenance are related but distinct concepts. Data provenance tracks the origin and transformation history of data as it moves through systems, pipelines, and storage. It answers questions like "where did this dataset come from?" and "what transformations were applied?"

AI provenance focuses specifically on content produced by AI models. It asks "which model created this?", "what parameters were used?", and "has this output been modified since generation?" The two overlap in the area of training data attribution, where understanding the provenance of the data used to train an AI model becomes relevant to understanding the provenance of its outputs.

The AI provenance gap

How AI models generate content without attribution

Most AI models produce output that is, by default, indistinguishable from human-created content at the file level. A JPEG image generated by Stable Diffusion is structurally identical to a JPEG from a digital camera. An MP3 file of AI-cloned speech has the same format and encoding as a genuine voice recording. There is nothing in the file format itself that marks the content as synthetic.

Some platforms add metadata or watermarks after generation, but this is voluntary and easily circumvented. Open-source models give users full control over the output pipeline, meaning any labeling can be removed before distribution. This creates what AFIP researchers call the AI provenance gap: the difference between the volume of AI-generated content entering circulation and the fraction that carries any provenance information at all.

Model fingerprinting: identifying which AI created content

Even without deliberate labeling, AI-generated content is not perfectly clean. Different models leave different statistical fingerprints in their output, subtle patterns that result from their architecture, training data, and generation process.

GAN-generated images exhibit characteristic frequency-domain artifacts tied to the upsampling layers in the generator network. Diffusion models leave noise schedule signatures in the pixel distribution. Language models produce token distributions that deviate from natural human writing in measurable ways, particularly in the distribution of rare words and the consistency of stylistic choices.

Generation parameter tracing

Beyond identifying the model, some forensic techniques can infer generation parameters. The guidance scale used in diffusion models affects the frequency distribution of the output image. The sampling method (Euler, DDIM, DPM++) leaves identifiable patterns in the noise structure. Temperature settings in language models influence token probability distributions in ways that statistical tests can detect.

This parameter-level tracing is still an active area of research and is not yet reliable enough for evidentiary use, but it points toward a future where forensic tools can reconstruct not just that content was AI-generated, but how it was generated.

Approaches to AI provenance

Watermarking at generation (SynthID, C2PA embedding)

The most direct approach to AI provenance is embedding a signal at the moment of generation. Google's SynthID system modifies the latent space representation during the diffusion process, encoding an invisible watermark that can be detected later. The C2PA standard allows AI platforms to attach a signed manifest to generated content, recording the model, timestamp, and generation parameters in a cryptographically verifiable package.

The advantage of generation-time watermarking is precision. The signal is embedded before the content enters distribution, creating a clear provenance record from the start. The disadvantage is that it only works for cooperative platforms. Any content generated without watermarking, or with the watermark subsequently removed, falls outside this system entirely.

Post-hoc forensic detection (AFIP approach)

Forensic detection analyzes content after the fact, looking for the statistical fingerprints and artifacts that AI generation processes leave behind. This approach does not require any cooperation from the generating platform. It works on content that was never watermarked, content where watermarks were removed, and content generated by open-source models running on private hardware.

The AFIP forensic approach applies multiple detection methods in parallel across different modalities. For images, it examines frequency-domain patterns, noise distributions, and GAN fingerprints. For text, it analyzes perplexity distributions, token patterns, and stylistic consistency. For audio, it evaluates spectral characteristics, prosody, and breathing patterns. The results are combined into a confidence-scored assessment.

Training data attribution and lineage

A deeper layer of AI provenance concerns the training data itself. When an AI model generates an image that closely resembles a copyrighted photograph in its training set, the provenance question extends beyond "was this AI-generated?" to "what was it generated from?" Training data attribution attempts to trace the connection between specific outputs and the data that influenced them.

This area intersects with ongoing copyright litigation and regulatory debates. The EU AI Act requires providers of general-purpose AI models to maintain documentation of training data sources. Several ongoing lawsuits hinge on whether AI outputs constitute derivative works of their training data, a question that training data provenance tools may eventually help answer.

Blockchain-based provenance registries

Some projects use blockchain or distributed ledger technology to create immutable records of content generation events. The idea is that an AI platform registers a hash of each generated item on a public ledger at the time of creation, providing a tamper-proof timestamp and origin record.

In practice, blockchain approaches face the same voluntary adoption challenge as watermarking. They add latency and cost to the generation process, and platforms with the least interest in transparency have the least incentive to participate. Blockchain provenance also does not help with content that was generated before the registry existed or outside participating platforms.

Current industry approaches

Google SynthID

SynthID embeds imperceptible watermarks into images, audio, video, and text generated by Google's AI models. For images, the watermark is applied in the latent space during the diffusion process, making it more robust than post-processing watermarks. For text, SynthID adjusts token sampling probabilities to create a statistical signature detectable by a corresponding classifier.

SynthID is deployed across Google products including Imagen and Gemini. Google has contributed the text watermarking component to the open-source community, though the image watermarking system remains proprietary. Independent robustness testing has shown mixed results, with image watermarks surviving basic compression but degrading under more aggressive transformations.

Adobe Content Credentials

Adobe integrates C2PA-based Content Credentials into its generative AI tools, including Firefly. Each generated image receives a signed manifest containing the model used, generation timestamp, and a record of any subsequent edits made in Adobe applications. The credentials are attached as metadata and can be verified through the Content Authenticity Initiative's verification tools.

The strength of Adobe's approach is its integration across the creative workflow. The limitation is that credentials are stored as metadata and can be stripped by platforms that do not support the C2PA standard, which currently includes most social media services.

OpenAI C2PA integration

OpenAI adds C2PA metadata to images generated by DALL-E and content produced through the ChatGPT interface. This metadata identifies the content as AI-generated and records the generation timestamp. Like Adobe's approach, the metadata can be stripped during sharing, and there is no watermark embedded in the content itself to serve as a fallback.

Meta and the Stable Signature approach

Meta's research team developed Stable Signature, a method that fine-tunes the decoder of a latent diffusion model to embed a watermark during the generation process. Because the watermark is baked into the model weights rather than applied as a post-processing step, it is more robust and more difficult to bypass without retraining the model from scratch.

Meta applies AI labeling across Facebook and Instagram using a combination of self-reported labels, C2PA metadata detection, and classifier-based detection. The company has publicly stated that it labels content as "AI-generated" when any of these signals are detected, though the accuracy and consistency of this labeling has been debated.

Forensic AI provenance: the AFIP approach

Why forensic analysis succeeds when watermarks fail

Every proactive labeling system shares the same structural weakness: it requires the content creator to cooperate. Watermarks must be embedded. Metadata must be attached. Registry entries must be created. At each step, a bad actor can simply choose not to participate, and the provenance chain breaks.

Forensic analysis inverts this dynamic. Instead of relying on what the creator chose to declare, it examines what the content itself reveals. AI-generated images carry frequency-domain patterns that differ from camera-captured photographs. AI-generated text exhibits statistical regularities that differ from human writing. These signals exist whether or not the creator wanted them to, and they persist through many of the transformations that strip metadata and degrade watermarks.

The Forensic Integrity Protocol (FIP) for AI content

AFIP's Forensic Integrity Protocol provides a structured methodology for AI content verification. Rather than returning a binary "real or fake" verdict, the FIP produces a multi-dimensional assessment that considers the strength and consistency of multiple forensic signals.

Multi-modal AI detection capabilities

AI content does not stay in a single modality. A deepfake video combines AI-generated face swaps with original audio. An AI-written article might include AI-generated illustrations. A voice-cloned phone call involves synthetic audio delivered through a real telephone network.

Effective AI provenance requires the ability to analyze content across modalities simultaneously. AFIP forensic analysis examines images, audio, video, and text through modality-specific detection methods, then cross-references the findings. When forensic signals from multiple modalities agree, confidence in the assessment increases substantially. When they disagree, the discrepancy itself becomes a forensic finding worth investigating.

The future of AI provenance

Regulatory requirements (EU AI Act Article 50)

The EU AI Act, which begins phased enforcement in 2025, includes specific requirements for AI content labeling under Article 50. Providers of AI systems that generate synthetic content must ensure their outputs are marked in a machine-readable format. This applies to images, audio, video, and text that could be mistaken for human-created content.

The practical implementation of these requirements is still being defined through technical standards work. Whether watermarking, metadata, or forensic detection will satisfy the "machine-readable" requirement is an active area of discussion. AFIP participates in these standards conversations, advocating for approaches that include forensic verification alongside self-declared labels.

Toward universal AI content identification

The ultimate goal of AI provenance is a system where any piece of content can be reliably assessed for AI involvement, regardless of its source, format, or distribution history. Achieving this requires a combination of approaches working in concert.

Watermarking provides a fast, high-confidence signal when present. C2PA metadata creates a verifiable chain of custody for cooperative platforms. Forensic analysis provides the safety net that catches everything else, the content that was never labeled, the content where labels were removed, and the content from sources that refused to participate.

No single technology solves the AI provenance challenge. The solution is layered: proactive labeling where possible, forensic verification everywhere, and transparent reporting that distinguishes between what the creator claims and what the evidence shows.

Frequently asked questions

Related research