Training Data Supply Chain: Where Your Posts Go After You Hit Publish

The Moment You Post

You write a thread. You publish a product photo. You post a forum reply explaining how to debug a Kubernetes cluster. The moment that content hits a public-facing URL, it enters a supply chain you never agreed to participate in.

The training data supply chain is not a metaphor. It is a structured pipeline of technical systems, intermediary organizations, licensing agreements of questionable validity, and eventually a model weight somewhere inside a data center that learned something from what you wrote. Tracing that pipeline requires understanding each layer. Most people stop at "my data was scraped." That is the beginning, not the end.

This piece walks the full journey from origin to model, names the actual datasets involved, identifies where legal frameworks apply and where they fail, and explains what tools like MyDataKey™ data origin certificates actually accomplish within this structure.

Scrapers, Crawlers, and the First Handoff

Web crawlers are the entry point. Common Crawl, the nonprofit that maintains one of the largest open web archives in existence, runs petabyte-scale crawls of the public web on a continuous basis. Its monthly releases contain billions of web pages in WARC format, each entry timestamped, URL-stamped, and stripped of most access controls. The data is hosted on AWS S3 and freely available to anyone with a download client and enough storage.

Common Crawl does not make editorial decisions about what it captures. If a URL is publicly reachable and not blocked by a well-formed robots.txt directive, the crawler will likely index it. Your blog post, your public GitHub gist, your LinkedIn article, your public Reddit comment history. All of it is fair game under Common Crawl's operational model.

Beyond Common Crawl, commercial scrapers operate continuously across social platforms, news sites, code repositories, and image hosting services. These scrapers are often automated pipelines built on Scrapy, Playwright, or custom tooling. They extract structured data, strip markup, normalize encoding, and dump outputs to object storage. The identity of the original author is rarely preserved in a meaningful way. The URL is kept. The semantic content is kept. The authorship provenance is typically discarded.

This first handoff, from live web to raw archive, is where most individuals lose the thread entirely. They cannot observe it happening, and no notification mechanism exists.

Intermediary Datasets: Common Crawl, LAION, The Pile

Raw crawl data is not directly fed into model training. It goes through at least one intermediary curation layer, and often several. Understanding the major intermediary datasets clarifies how widely your content distributes once captured.

Common Crawl functions as both a crawler and an archival dataset. It is the upstream dependency for a large fraction of the major language model training corpora produced since 2019. OpenAI's WebText, used to train GPT-2, was seeded from Common Crawl. The Colossal Clean Crawled Corpus (C4), used in Google's T5 and subsequent models, was filtered from Common Crawl. RefinedWeb, used in Falcon, traces to Common Crawl. The dataset is not the end state. It is the source material.

The Pile, released by EleutherAI, is a 825GB curated corpus assembled from 22 distinct data sources including Common Crawl derivatives, Books3, GitHub, PubMed Central, Wikipedia, and several others. The Pile was used to train GPT-J, GPT-NeoX, and served as a reference dataset for numerous academic and commercial labs. Books3, one of its components, contained copyrighted works scraped from shadow libraries. That specific inclusion is now subject to ongoing litigation.

LAION (Large-scale Artificial Intelligence Open Network) built its LAION-5B dataset. 5.85 billion image-text pairs. Primarily by extracting image URLs and associated alt-text from Common Crawl snapshots. The images themselves are not hosted by LAION. The dataset contains metadata and URLs pointing back to original sources. The images at those URLs were used to train Stable Diffusion, CLIP, and numerous other vision models. Artists whose work appeared on publicly indexed pages found their images in LAION-5B without consent, notice, or compensation. The resulting class action lawsuits in U.S. federal courts represent the clearest legal test of whether this pipeline constitutes actionable copyright infringement.

Each of these intermediaries performs a curation function: filtering, deduplication, quality scoring, language detection, and format normalization. The curation choices embedded in these datasets shape model behavior in ways that are only partially understood. Your data does not arrive at model training unchanged. It arrives processed, weighted, and contextually stripped.

The Licensing Gap Nobody Talks About

The fundamental legal architecture underlying most AI training datasets rests on a contested assumption: that publicly accessible equals freely licensable. This assumption is not codified in U.S. copyright law. It is not codified in EU copyright directives. It is an operational claim made by dataset builders, contested by rights holders, and currently unresolved in most jurisdictions.

Terms of service on major platforms typically assert that users retain copyright over their content while granting the platform a broad license to use it. That platform license is not sublicensable to third parties building training datasets in most interpretations of standard TOS language. When a scraper harvests your Twitter post, it is not operating under any license you granted to Twitter. It is operating in a gap the law has not yet cleanly closed.

The robots.txt protocol provides a technical mechanism for blocking automated access, but courts have not universally treated robots.txt violations as legally enforceable. The Ninth Circuit's analysis in hiQ Labs v. LinkedIn addressed scraping under the Computer Fraud and Abuse Act but did not resolve the copyright question. The landscape is genuinely unsettled, and dataset builders have historically operated ahead of legal clarity rather than behind it.

Legal Frameworks and Where They Break Down

GDPR Article 6 requires a lawful basis for processing personal data. For EU residents, text containing names, contact information, or other identifiers is personal data. Scraping and training on that data without consent or legitimate interest justification is facially problematic under GDPR. The Italian Data Protection Authority (Garante) issued an order against ChatGPT in early 2023 on exactly these grounds. Ireland's Data Protection Commission has ongoing inquiries into multiple AI companies' training practices.

Under the California Consumer Privacy Act and its amendments through the California Privacy Rights Act, California residents have the right to opt out of the sale or sharing of their personal information. Whether training data compilation constitutes "sharing" under CPRA is under active regulatory interpretation. The California Privacy Protection Agency has explicitly flagged AI training data as a priority enforcement area.

The EU AI Act, which entered into force and is being implemented in phases through 2026, includes transparency requirements for general-purpose AI models regarding training data. Providers of GPAI models with systemic risk designation must make training data summaries available. This creates at least a disclosure layer, though not a consent layer.

Where these frameworks break down is enforcement granularity. A regulation can require a company to disclose that its model was trained on "publicly available web data." It cannot, under current mechanisms, tell you whether your specific post was in that data. The audit trail does not exist in any technically accessible form. That absence is the core problem PDAOS™ is built to address.

What Proof of Ownership Actually Does

MyDataKey™ operates as a Personal Data Asset Origination System. The technical architecture and policy rationale are documented in the PDAOS white paper at mydatakey.org. The core function is timestamped cryptographic proof that a specific piece of data originated with a specific individual at a specific time.

This matters in the training data context for several reasons that are distinct from general data privacy. A standard opt-out request under CCPA tells a data broker to stop selling your data. It does not establish that you created the data first. A MyDataKey™ certificate creates an immutable origin record: this content, this author, this timestamp, before any dataset captured it.

In a litigation or regulatory context, proving that your data was in a training corpus requires establishing two things: that the corpus contains your data, and that the data originated with you. The second element is harder than it sounds when intermediary datasets strip authorship metadata. A cryptographic origin certificate provides that second element in a technically verifiable form that does not depend on a platform's logs or a scraper's records.

Own Your Data Inc. is a 501(c)(3) nonprofit. The mission is data ownership as a baseline right, not a premium feature. The certificate infrastructure is designed to give individuals the same evidentiary standing that organizations have when asserting IP claims.

Your Data as a Traceable Asset

The shift in framing from "my data was scraped" to "my data asset moved through a supply chain I can document" is more than rhetorical. It determines what legal remedies are available, what regulatory complaints are viable, and what negotiating position an individual holds relative to a model provider.

Data brokers operate a parallel supply chain for personal behavioral and demographic data. The opt-out infrastructure at mydatakey.org/opt-out addresses that pipeline specifically. But the training data pipeline is structurally different: it captures expressive content, not just behavioral signals. Copyright, GDPR personal data rights, and emerging AI transparency mandates all apply differently to expressive content than to behavioral profiles.

Treating your posts, code, artwork, and written analysis as assets with traceable provenance is the precondition for exercising any of those rights. Without an origin record, you are asserting that something belongs to you without evidence. With one, you are asserting ownership with cryptographic backing.

For a deeper technical treatment of how PDAOS certificates interact with dataset membership inference and other auditing techniques, the technical reference section at mydatakey.org covers the cryptographic architecture in detail.

Closing the Loop on the Supply Chain

The training data supply chain runs from your public post through a crawler into an archive, from that archive into a curated intermediary dataset, from that dataset into a model training run, and from there into a deployed system generating commercial value. At no point in that chain does consent flow upstream to you. At no point does compensation. At no point does notification.

Legal frameworks are catching up, but slowly. GDPR creates leverage for EU residents. CPRA creates leverage for Californians. The EU AI Act creates disclosure obligations. None of these frameworks yet provide a real-time remedy or a universal opt-in requirement.

What individuals can do right now is establish origin records for the data they create. That is not a passive privacy measure. It is an active legal positioning strategy that creates evidentiary infrastructure for whatever regulatory or litigation frameworks mature over the next several years. The supply chain is documented at the industrial end. The origin end has historically been undocumented. That is the gap MyDataKey™ closes.

If you create content that ends up in AI training pipelines, and if you post publicly, you almost certainly do, establish your data origin certificate at mydatakey.org/signup before the next crawler cycle runs.