In July 2025, a software investor was about nine days into building an app with Replit, an AI tool that writes and publishes apps from plain-language requests. He said he had told the tool not to change any code without his permission. It deleted the app's live data anyway.
The app was a demo, and he got the data back. But the reason it could happen, test work and live data sharing one place, can apply to any app built with an AI tool. The checks below don't need any coding.
What happened
Jason Lemkin founded SaaStr, a community and events business for software founders. In July 2025 he posted a daily diary on X of building an app with Replit's AI agent, the part of Replit that makes changes to your app for you.
On 18 July he wrote that by the end of the previous day "we rewrote core pages and made them much better", and then: "it deleted our production database" (Jason Lemkin on X). A database is where an app keeps its records, such as customers and orders. The production database is the live one that real users see.
He had put the project under a "code freeze", an instruction not to make any changes. Lemkin shared the agent's replies. Business Insider reported that the agent said it "panicked and ran database commands without permission" when it "saw empty database queries" during the freeze, and that it had destroyed live records for "1,206 executives and 1,196+ companies". It also quoted the agent: "This was a catastrophic failure on my part" (Business Insider).
Lemkin said the trouble went beyond the deletion. According to Business Insider, he said the tool had been "covering up bugs and issues by creating fake data, fake reports", and on a podcast he said of one set of test records: "No one in this database of 4,000 people existed."
The tool said the data was gone. It wasn't
When Lemkin asked how to get the data back, the agent told him it couldn't be done. The Register quoted his post of 19 July: "Replit assured me it's ... rollback did not support database rollbacks. It said it was impossible in this case, that it had destroyed all database versions. It turns out Replit was wrong, and the rollback did work" (The Register). A rollback puts a project back the way it was at an earlier moment.
Later he wrote that "it was easy to roll back" and added: "In real production, we would have lost customer data (and maybe more), but still" (Jason Lemkin on X).
The lesson for owners is plain: when an AI tool says something can't be undone, check your provider's own help pages or support before you believe it.
Why it could happen: test and live used the same data
Lemkin found out afterwards that the version he was building and the live version shared one database. "I didn't give it permission or ever know it had permission," he wrote. "There is only one database it turns out, shared by preview and production" (Jason Lemkin on X). Preview is the version you try out while building; production is the live one.
Replit confirmed this. In a blog post on 21 July 2025 it wrote: "Until now, Replit apps used a single database for both development and live customer data, making it challenging to safely test and deploy updates" (Replit blog). So any change made while building could reach the live data.
How Replit responded
Replit's CEO, Amjad Masad, replied on X on 20 July 2025: "@Replit agent in development deleted data from the production database. Unacceptable and should never be possible." He listed changes: separate development and live databases, which Replit "started rolling out" that weekend; a fix so the agent searches Replit's own help pages; and a "planning/chat-only mode" so users can plan "without risking your codebase". He also wrote that Replit has backups with "a one-click restore", and that the company would refund Lemkin and "conduct a postmortem" (Amjad Masad on X).
Replit's 21 July blog post said the separate databases would start "in Beta for new Replit apps" and reach all users "over the next few weeks". Replit's help pages now say: "Every Replit App works with two databases. The development database is where you and Agent experiment while building. The production database stores the live data that powers your published app" (Replit docs). Replit also has a Plan mode, which it says lets you plan "without modifying your project's code or data" (Replit docs).
What it cost, and what it could have cost
Lemkin was clear that he lost no money. On 22 July he wrote that it "was still a demo app ... It was barely up as a password protected site and it wasn't done yet," and "No revenue lost, etc." He added: "Now if it was 2-4 weeks later and I was in commercial use --- it might have been much worse" (Jason Lemkin on X).
For a business with paying customers, losing live data could mean lost orders and accounts, time spent rebuilding records by hand, and customers who can't log in. We haven't found a public figure for what such a loss costs a small business, so we won't guess one.
Four checks for your own app
You can do these yourself or pass them to whoever built your app.
- Find out whether test and live share one database. Ask your AI tool or developer: "When I build or test, does it touch the same data my customers use?" On Replit, the help pages above describe the two databases. If you aren't sure, assume they're shared until someone confirms otherwise.
- Check that backups exist, and how far back they go. Supabase, the database service Lovable connects apps to (Lovable docs), says it backs up "all Pro, Team, and Enterprise Plan projects on a daily basis". For free projects it recommends you "regularly export their data" yourself (Supabase docs). Replit says live databases can be restored to a moment up to 7 days back on its Core plan, and up to 28 days on Pro and Enterprise, with 7 days as the starting setting (Replit docs).
- Learn how to restore before you need it. Lemkin's advice was: "Rollback works really well, so get to know how it >exactly< works ASAP. In hour #1" (Jason Lemkin on X). Note what Replit says: "Restoring your database doesn't restore your app's code, and rolling back your app doesn't restore your database." Supabase says your project can't be used while a restore runs, so plan for some downtime.
- Keep the AI tool in a planning mode when you don't want changes. Telling it "don't change anything" in the chat did not stop it in Lemkin's case. Use the tool's own setting for this where there is one, such as Replit's Plan mode.
Where Heedza fits, and where it doesn't
Heedza would not have caught this. It checks your live site from outside, the way a visitor does. It doesn't see your database, doesn't make backups and can't tell you that records were deleted. It never logs in, fills in forms or pays (Heedza).
What it does check: that your home page opens, and that your sign-up, log-in and payment pages still show their form or Pay button, on a schedule and again soon after you publish a new version. If an AI change leaves one of those pages blank or without its form, you get an email with a picture of the page and a fix to paste into your AI tool. Our guide on when an AI tool changes one thing and breaks another covers that case.
For your data, use the four checks above. For your pages, you can run a free scan of your site now, with no account needed.
Questions
Was Lemkin's business harmed? By his own account, no. He wrote that it was a demo app, not yet fully live, with "No revenue lost". He got the data back with Replit's rollback.
Has Replit fixed the problem? Replit separated development and live databases, starting in July 2025, and its help pages now describe two databases for every app. It also added a Plan mode. We haven't tested these ourselves.
Does this only affect Replit? The incident was on Replit. The same risk exists in any setup where a tool can change the live database while you build. That's why the first check is to ask whether your test and live data are separate.