India DPDP Act for LLM and Agent Pipelines: What Engineering Teams Must Build for Consent, Purpose Limitation and Erasure

I’ve sat in a lot of compliance reviews where the legal team slides a clean consent policy across the table and the engineering lead goes quiet. The policy says a user can withdraw consent and have their data erased. The lead is thinking about the vector index, the three months of prompt logs, the evaluation set someone built from support tickets, and the fine-tuning run that finished last quarter. Nobody in the room can say which of those contains data from the user who just asked to be forgotten.

That silence is the real subject of this article. India’s Digital Personal Data Protection Act, 2023 and the Rules made under it do not care whether your system is a classic web app or a multi-step agent. They care whether you can honor a person’s choices about their data. LLM and agent pipelines make that hard in a specific way: personal data gets copied, embedded, summarized and cached in places that traditional schemas never anticipated.

My argument is simple. The hard part is plumbing, not policy. The policy is a few paragraphs. The plumbing is a metadata discipline that follows every personal data element through prompts, retrieval indexes, logs and training sets, so that when a withdrawal or erasure request arrives, you can act on it instead of apologizing for it.

Where the law stands, and what I am not claiming

Check the status yourself before you plan around any date, because secondary sources disagree on details. As I understand it from published summaries, the DPDP Rules were notified in November 2025 and the obligations are phased. The provisions setting up the Data Protection Board took effect almost immediately. Consent manager provisions were slated to follow after about twelve months. The core duties on notice, consent, security safeguards, breach reporting and the rights of data principals apply after roughly eighteen months, which points to around May 2027. One summary I read treated consent manager registration differently from the others, so read the gazette text rather than a blog post, including this one.

I am an engineer, not your counsel. Treat what follows as a design brief for the systems work, and have lawyers confirm how the Act applies to your specific processing, including exemptions, children’s data and any classification as a significant data fiduciary.

The parts of the Act that matter for pipeline design are few:

• Processing needs a lawful basis, most often consent that is free, specific, informed and tied to a stated purpose, with a notice that says what data is processed and why.
• Consent can be withdrawn, and the ease of withdrawal is meant to match the ease of giving it.
• Data should be erased when the purpose is served or consent is withdrawn, unless a law requires retention.
• The data principal has rights to access information, correct, and erase, and to have grievances handled.
• The fiduciary stays responsible for processing done on its behalf by processors, which is where hosted model APIs come in.

Every one of those is a statement about data you must be able to find. That is why lineage is the foundation.

Why LLM pipelines break the old model

A conventional application has a database, a few caches and some logs. Erasing a user means deleting rows and purging a handful of known locations. The data model is explicit, and a foreign key tells you where a person lives.

An agent pipeline scatters the same person across places that do not share a schema:

1. Prompts and context windows. User messages, retrieved documents and tool outputs get assembled into a prompt. The prompt is transient, but it is often stored.
2. Retrieval indexes. Documents are chunked and embedded. The chunk carries personal data, and the embedding is derived from it. Delete the source row and the chunk survives in the index.
3. Conversation memory. Agents summarize earlier turns into long-term memory. A summary of a conversation is still personal data about that person.
4. Logs and traces. Observability stacks capture full prompts and completions because that is how you debug. They are a second copy of everything.
5. Evaluation and fine-tuning sets. Teams build datasets from real traffic. Those datasets outlive the tickets they came from.
6. Tool calls and third parties. An agent that calls a CRM, an email service or a hosted model sends personal data outside your boundary.

I have seen teams discover the sixth category only when a customer asked where their data had gone. The agent had been quietly passing account details to a third-party enrichment tool for months, and nothing in the system recorded that it had.

The unit of tracking is the data element

Most consent systems record consent against a user: user 4821 agreed to marketing and to product analytics. That works when each purpose maps to a table. It breaks when a single support message is used to answer the user, to improve a model and to feed an analytics summary.

Track consent and purpose against the data element instead. Each piece of personal data that enters the pipeline gets a stable identifier and a small record:

• subject_id: the data principal it relates to.
• element_id: a stable ID for this item (a message, a document, an attribute).
• purposes: the purposes consented to for this element, with the notice version shown.
• consent_state: given, withdrawn, or expired, with timestamps.
• derived_from: parent element IDs, so lineage survives transformation.
• locations: where copies and derivatives live (index name, log stream, dataset version).
• retention_until: when the purpose is considered served.

The derived_from field is the one teams skip, and it is the one that makes erasure possible. A chunk derived from a document derived from a user upload inherits the subject. An embedding derived from the chunk does too. A summary derived from ten messages has ten parents.

A picture of the flow

The ledger sits in the middle. Nothing reaches a prompt, an index, a log or a dataset without a purpose check against it, and nothing leaves a withdrawal request unpropagated.

Purpose limitation as a runtime check

Purpose limitation is where policy and code meet most directly. A purpose is not a sentence in a privacy notice; it is a tag that a gateway can test.

Suppose a user consented to “answering my support question.” Using that same message to fine-tune a model is a different purpose. If your fine-tuning job pulls from the log store by date range, it will happily include the message. If it pulls through a function that filters on purposes contains model_improvement AND consent_state = given, it will not.

The pattern I recommend is a thin gateway that every pipeline stage calls:

def allowed(element, purpose):
    rec = ledger.get(element.id)
    return (
        rec.consent_state == "given"
        and purpose in rec.purposes
        and not expired(rec)
    )

# in the training data builder
rows = [r for r in candidates if allowed(r, "model_improvement")]

This is deliberately boring. The value is that the check exists in one place, is logged, and can be shown to an auditor. It also forces product teams to name purposes up front, which is where many compliance gaps start.

For agents, apply the check at tool boundaries too. Before an agent sends personal data to an external tool, the gateway asks whether that purpose and that recipient are covered. A denied call should fail closed with a clear error the agent can reason about, rather than silently dropping fields.

Erasure is a fan-out problem

When a withdrawal or erasure request lands, the ledger tells you which elements belong to the subject and, through derived_from, which derivatives inherit that link. The deletion job then fans out:

• Primary stores: delete or anonymize source records, unless a legal retention duty applies. Record the reason if you keep anything.
• Retrieval indexes: delete chunks and vectors by element ID. This only works if you stored the element ID as metadata at indexing time. If you did not, you are left rebuilding the whole index, which is the expensive version.
• Logs and traces: either redact personal fields at write time or keep per-subject keys so entries can be located. Redacting on write is cheaper than searching later.
• Memory stores: remove or regenerate summaries that include the subject.
• Datasets: remove rows and mark the dataset version as changed.
• Processors: send deletion instructions to every processor that holds the data, and keep their confirmations.

Then write an erasure receipt: what was deleted, where, when, and what was retained and why. That receipt is your evidence if a complaint reaches the Board.

The model weights question

Fine-tuning is the awkward case. Once personal data has influenced the weights, there is no clean way to subtract one person’s contribution. Machine unlearning research exists, but I would not build a compliance plan on it today.

The practical answer is prevention. Keep personal data out of training sets unless the purpose was explicitly consented to, keep dataset versions so you can retrain without a subject’s rows, and prefer retrieval over fine-tuning for knowledge that is personal or likely to be withdrawn. Retrieval keeps personal data in a store you can delete from. Weights do not offer that. Where fine-tuning on personal data is really needed, plan on periodic retraining from a refreshed, consent-filtered dataset and be honest in your notice about that cadence. Whether a given model counts as holding personal data is a legal question for your counsel, not something I will settle here.

Self-hosted or hosted: same ledger, different risks

This is a choice between legitimate approaches, and the right one depends on your workload and risk posture.

Self-hosted or open-weights models keep inference inside your boundary. Data flows are easier to inspect, you control logging, and you avoid sending personal data to a processor. That is a real advantage for sensitive workloads. But it does not touch consent bookkeeping. Running the model yourself does not tell you which purposes a user agreed to, and your own index, logs and datasets still need lineage. I have seen self-hosting teams relax because “the data never leaves,” and then fail an erasure request because their own trace store held everything. You also take on the security safeguards and operational burden yourself.

Hosted frontier APIs bring mature safety tooling, strong capability and support guarantees, and for many teams a better security posture than they could build alone. The compliance work shifts toward contracts and configuration: a data processing agreement that fits the Act’s processor obligations, clear terms on retention and on whether inputs are used for training, regional processing options, deletion and audit commitments, and sub-processor transparency. Check what the provider actually logs and for how long, and confirm in writing how deletion requests propagate. Cross-border transfer is generally permitted under the Act unless restricted by notification, but sector rules from regulators for finance or health can be stricter, so verify for your domain.

Neither route is a shortcut. A small team with strong security skills and a sensitive dataset may do well self-hosting. A team without GPU operations experience, or one that needs frontier capability, may be safer with a well-contracted hosted API. In both cases, the ledger is yours to build.

What to build, in order

If I were starting a program tomorrow, I would sequence it like this:

1. Inventory. List every place personal data enters, rests or leaves an AI pipeline. Include logs, evals and third-party tools.
2. Assign IDs and tags. Give every ingested element a stable ID and attach it as metadata in each downstream store, especially index chunks.
3. Stand up the ledger. Even a simple table with subject, element, purposes, state and parents is a big improvement on per-user flags.
4. Put the gateway in front of training and tools. Purpose checks at dataset builds and external calls first, since those are hardest to reverse.
5. Automate deletion fan-out. Make it a job with receipts, run it against test subjects monthly, and measure how long it takes.
6. Fix contracts. Review processor agreements, retention terms and regional options for every hosted service in the chain.
7. Practice. Run a game day: a withdrawal request arrives for a synthetic user seeded across all stores. Count what you miss.

The business case, in plain terms

For a business leader, the exposure is not abstract. The Act provides for substantial monetary penalties set out in its schedule, and the larger risk is operational: if you cannot honor a withdrawal, you may have to stop using a data set or pause a feature while engineers hunt for copies. A ledger turns a fire drill into a routine job.

There is an upside too. The same lineage that supports erasure improves debugging, makes dataset audits faster and gives security teams a map of where sensitive data lives. Teams that build it usually find they ship AI features faster afterward, because every launch no longer starts with a debate about what data can be used for what.

The questions to ask your engineering lead this week are concrete. Can you list every store that holds data from a given user? How long would a deletion take end to end? Which processors would you need to notify? If the answers are “not sure,” you have found the work, and it is plumbing.

Closing thought

Policy tells you what you owe people. Plumbing decides whether you can pay it. The teams that will handle this well are not the ones with the longest privacy notices; they are the ones who treated consent and purpose as data, attached it to every element, and built deletion as a first-class path. Verify the current Rules and dates with counsel, then start with the inventory.