What has to happen when a user deletes their account in an AI app?
App store and state-law deletion requirements, and where user data hides in an AI product (chat logs, embeddings, model providers, training data, logs, and backups), with a checklist to find it.
Short answer
The account and the data tied to it have to go, everywhere you keep it, not just the row in your users table. In an AI app, that includes chat history, embeddings in a vector store, logs and traces, analytics, data your model provider holds, and anything used for training. Both app stores require in-app account deletion, state privacy laws give users a right to delete, and your privacy policy's promises about deletion have to be true. Plan the deletion path before launch, because finding every copy later is much harder.
What the rules require
- Apple. If your iOS app lets people create an account, it must let them delete the account from within the app (App Store Review Guideline 5.1.1(v)). Apple's guidelines also require your privacy policy to explain how users can request deletion of their data.
- Google Play. Apps that allow account creation must offer an in-app path to delete the account and its data, and also a web page where users can request deletion without the app installed. You must delete the data associated with the account, ask your service providers to do the same, and disclose the deletion option in the Data safety form. You may keep some data for reasons like security, fraud prevention, or regulatory compliance, but you have to tell users what you keep and why.
- State privacy laws. California and most other states with comprehensive privacy laws give consumers a right to have their personal information deleted, with limited exceptions such as completing a transaction, security, and legal obligations. Covered businesses must also tell their service providers to delete it, and generally must respond within 45 days. Whether a law covers you depends on its size and data thresholds.
- Special categories. Health data under Washington's My Health My Data Act and children's data under COPPA come with their own deletion duties. See Does Washington's My Health My Data Act apply to my app? and What changed in the COPPA rule?
- Your own promises. If your privacy policy says data is deleted when an account is closed, it must be. The FTC treats a broken deletion promise as deception, and it has ordered companies to delete models built from data they shouldn't have kept, as in its 2021 settlement with the photo app Everalbum.
Where user data hides in an AI product
- Chat and conversation history. Usually the obvious copy, in your main database.
- Embeddings and vector stores. Retrieval systems store vectors and text chunks derived from users' messages and files. Deleting the source record doesn't delete these. Tag every vector with the user or account it came from so you can find and delete it.
- Caches and memory features. Summaries, "memories," and cached responses about a user often live in a separate store.
- Logs and traces. Application logs, error trackers, and LLM observability tools frequently capture full prompts and responses. Set retention limits, and avoid logging content where you can.
- Analytics. Events that carry user content or identifiers sit with your analytics vendor.
- Model providers. Your LLM provider may keep prompts and outputs for a period, for example for abuse monitoring, depending on your account settings. Know the retention settings for your account, and choose shorter or zero retention where it's offered.
- Training and evaluation data. If user data went into fine-tuning or evaluation sets, deleting it from a trained model is hard. Keep user data out of training unless you've planned for this. See Can I train or fine-tune a model on my users' data?
- Backups. Backups can keep deleted data for their retention period. Delete it as backups expire, don't restore it, and say so in your privacy policy.
- Files and integrations. Uploaded files, and data pulled from connected accounts like Gmail or Google Drive, need to be removed too, along with the access tokens.
Designing the deletion path
- One function that knows every store. A single deletion routine that removes or queues removal from every place above is easier to keep complete than deletion code scattered across services.
- Confirmation and a short grace period. A confirmation step and an optional grace period before permanent deletion protect users from mistakes, as long as your policy says so.
- Deactivation isn't deletion. If you only deactivate or hide the account, don't describe it as deletion.
- Keep only what you must, and say what. Records kept for fraud prevention, billing, or legal obligations should be listed in your privacy policy, limited to what's needed, and deleted when the reason ends. A litigation hold can require keeping data you'd otherwise delete.
- Record the deletion, not the data. Keep a minimal log that a deletion happened and when, without keeping the deleted content.
Checklist an agent can run against the codebase
- Find the in-app account deletion flow, and the web page for deletion requests if the app is on Google Play.
- List every data store that holds user data: databases, vector stores, caches, file storage, search indexes, queues, and third-party services.
- For each store, check that the deletion routine removes the user's data, or note that it doesn't.
- Check that vectors and chunks in the vector store are tagged with the user or account, and deleted with the account.
- Check logs, traces, and LLM observability tools for prompts and responses, and their retention settings.
- Check the model provider's retention settings for the account in use.
- Check whether user data has gone into fine-tuning or evaluation sets.
- Check backup retention, and that restores don't bring back deleted accounts.
- Check that connected-account tokens are revoked and pulled data is deleted.
- Compare what the code does with what the privacy policy says about deletion and retention.
Sources
- App Store Review Guidelines (Guideline 5.1.1(v))
- Google Play: understanding the account deletion requirements
If this applies to you
An app launch legal package covers deletion end to end: the privacy policy's deletion and retention terms, the app store requirements, and a build spec for the deletion flow, with a checklist to test it against the shipping app. For one change, such as adding a vector store or memory feature, a pre-build feature check covers that change before it ships.