Back to Seatbelt

Check your vibe-coded app before you show anyone

Your agent says it is done. Run these four checks before you let another human near it: a cofounder, a beta tester, or your first customer. Showing someone is not the same as publishing a URL, but an exposed key or open database rule can leak just as fast. Veracode measured what done means in 2025: 45% of AI-generated code introduced a known vulnerability. In May 2026, security firm RedAccess scanned about 380,000 apps built on Lovable, Base44, Replit, and Netlify and found roughly 5,000 leaking sensitive data. Axios verified named cases. The pace is accelerating too: Georgia Tech's Vibe Security Radar tracked AI-attributed CVEs in public advisories at 6 in January 2026, 15 in February, and 35 in March alone (74 cumulative through March, a confirmed lower bound). Georgia Tech research news publishes the monthly counts. AI writes code that runs. It does not always write code that is safe. The four checks below catch the failures that actually hurt launches. Do each one by hand with the steps on this page, or run them all in one pass.

Reviewed July 2026

The short version

  1. No secret keys in your public build. Anything that leaked gets rotated, not just removed.
  2. Row level security on every table, with policies that actually check who is asking.
  3. Authorization enforced on the server. Every API route checks the caller, not just the UI.
  4. No admin, debug, export, or delete routes left public.

Seatbelt reads all four in one pass, plus payments and customer data. Run it in the browser, or verify each one by hand below. Searching for "audit my vibe-coded app"? See the Ship Read page.

  1. 01

    No secret keys in your public build

    When an agent needs an API to work, the fastest path is pasting the key right where the call happens. That is how live payment keys end up in browser code, readable by anyone who opens dev tools. A leaked key can mean real charges on your account and a database that answers to strangers. Exposed secrets are the most common real world failure in AI-built apps.

    Check it yourself

    • Open your live site, open the browser dev tools, and search the loaded JavaScript for key shaped strings: sk_live, service_role, AKIA, api_key.
    • Search your repo for committed .env files and for env values copied into the build output.
    • If a real key is exposed, removing it from the code is not enough. Bots scan public repos for fresh keys around the clock, so treat any key that was public even for a few seconds as compromised. Rotate it with the provider before the bots find it. Old commits and old copies of the build still hold the value.

    What Seatbelt reads: the secrets surface. Key patterns and env looking names anywhere in the build you hand it. A secret in the public build is a must fix, and with the push gate wired that push parks on a pull request instead of main until the key is out and rotated. Publishable keys, like Stripe pk_ values, are expected in front end code and stay noted.

  2. 02

    Row level security on every table

    App platforms hand the browser a public database key and trust the database's own rules to keep users apart. Row level security is that rule set. When it is off, or the policy never checks who is asking, every row in the table is one API call away from any visitor. RLS can also be toggled on while a policy still allows everything, or while INSERT and UPDATE policies omit WITH CHECK: reads look scoped in the dashboard, but a signed-in user can still write rows as someone else. Real apps have leaked every user's private messages exactly this way.

    Check it yourself

    • Supabase: check each table in the public schema. Row level security on for every one, with a policy that filters by the signed in user's id rather than allowing everything.
    • On INSERT and UPDATE policies, open the SQL. A USING clause alone is not enough: confirm the same policy has WITH CHECK so a user cannot write a row with another user's id.
    • Firebase: read your Firestore and Storage rules. allow read, write: if true means open to anyone. It is the classic paste-over for expired test mode, and it ships to production more often than anyone admits.
    • The blunt test: create a second account and use it to request the first account's data through your app's API. If it answers, the rules are not doing their job.

    What Seatbelt reads: the databases surface. Code cues only: a service key shipped to the browser, rules files that allow everything, database URLs in public code, raw SQL built by string glue, database files sitting in public folders. The careless ones are must fix. Seatbelt never connects to your live database. It reads the build you hand it.

  3. 03

    Authorization on the server, not just the screen

    Agents are good at hiding buttons from signed out users. Hiding is not checking. If the API route behind the screen never asks who is calling, anyone can call it directly and skip your UI entirely. Next.js server actions are a second gap: the page can verify the session at render while the action file deletes or updates with no ownership check. The page check runs once. The action runs on every direct POST. A common AI auth pattern decodes a JWT payload without verifying the signature first, so a forged token can pass the middleware. Login handlers that return different messages for unknown email vs wrong password leak which addresses are registered. Forgot-password flows often ship plain reset tokens in the database with no expiry or single-use guard. A separate miss is unlimited login attempts: a 2026 study of vibe-coded apps found roughly 80% lacked rate limiting on at least one auth endpoint, so brute-force and account-enumeration scripts can run as fast as the network allows. A 2026 study of 100 vibe-coded apps found roughly one in five had API endpoints with no authentication at all.

    Check it yourself

    • Sign out, then call your API routes directly with curl or the address bar. Anything that returns user data while signed out is open.
    • Sign in as a test user, find a request with an id in it, and swap the id for another user's. If their data comes back, the route checks that you are signed in but not what belongs to you.
    • Read each API handler. The auth check belongs at the top of the handler, before the database read. A check in the page component protects nothing.
    • Next.js: open each file with 'use server' that calls .delete(, .update(, or Prisma writes. The action should import your session helper and verify the caller owns the row before the mutation. Page protected does not mean action protected.
    • In JWT middleware, search for jwt.decode or decode( on the token. The same function block should call jwt.verify (or jwtVerify) with an explicit allowed-algorithms list before you trust the payload. Decode-then-use-then-verify order is still a miss.
    • Login enumeration: try a random email and a known email with a wrong password. If the API returns different messages, you are leaking which addresses are registered. Both paths should return the same generic invalid-credentials response. See auth account enumeration for why it matters.
    • Password reset: open your forgot-password handler and user schema. A plain resetToken or passwordResetToken column without a hash, an expiresAt window, and a used or invalidated flag is account takeover if the row leaks. Practitioner audits in 2026 found reset tokens with no time bound in several of ten vibe-coded codebases reviewed.
    • Rate limiting: open each login, register, and forgot-password route (or app/api/auth/ handler). Search the file and middleware.ts for rateLimit, @upstash/ratelimit, express-rate-limit, or an edge limiter. If bcrypt compare or JWT sign runs with no throttle before it, attackers can brute-force passwords or spray sign-ups. A common baseline is about 5 attempts per minute per IP on auth routes.

    What Seatbelt reads: the login surface. Seatbelt notes where sign in lives and flags identity APIs and data handlers that read a store with no auth guard in the handler body. It also soft-flags distinct login error strings in the same handler and plain reset-token storage without expiry or single-use fields. Auth rate-limit cues are on the detector roadmap: use the hand-check above until repo read ships for that class. Open looking routes are marked worth a look, with the file and line, so your agent knows exactly where the check goes.

  4. 04

    No admin, debug, or export routes left public

    Builds accumulate scaffolding: an admin page for testing, a debug route that dumps state, an export endpoint meant for one internal use, a delete helper from an early reset flow. When an agent adds "export user data as CSV" late in a long chat, it often ships a handler that streams the whole table with no session check. The feature looked scoped in the prompt. The endpoint is public. A second class hides in link-preview and image-proxy features: a URL from the query string or form flows straight into fetch() on the server with no host allowlist, so your app can be tricked into calling 169.254.169.254, localhost, or internal VPC addresses. Georgia Tech's Vibe Security Radar lists SSRF among top AI-attributed CWE classes in 2026 advisories. Agents rarely clean these up on their own, and one public delete route can end a project in an afternoon.

    Check it yourself

    • Search your routes for admin, debug, test, export, dump, backup, reset, and delete.
    • Open each export, dump, backup, or download-all route while signed out. If it returns rows, the handler needs getServerSession, getToken, or your auth helper before the query.
    • SSRF: in server actions and route handlers, search for fetch( or axios.get( where the target URL comes from searchParams, formData, or the JSON body (url, target, webhookUrl, and similar). The handler should allowlist hosts, block private IPs and link-local ranges, and set redirect: "error" on outbound fetches. A UI that never shows a URL field is not a defense: server actions are direct POST endpoints.
    • Open next.config (or next.config.mjs). images.remotePatterns with hostname: "**" turns image optimization into an open proxy. Tighten patterns to hosts you actually serve.
    • Anything that changes or exports data goes behind auth. Anything destructive should not ship at all.
    • Include the neighbors: a database admin UI deployed next to the app counts as an open admin route.

    What Seatbelt reads: the risky shortcuts surface. Open admin pages, debug leftovers, and export or delete routes get flagged, and destructive ones are a hard stop that parks the push on a pull request until they are gone. Outbound-fetch SSRF cues are on the detector roadmap: use the hand-checks above until repo read ships for that class.

Two more that bite after launch day

The four checks above cause the loudest failures. These two cost real money and trust a little later, so Seatbelt reads them in the same pass.

  • Payments and money

    Checkout that half works is worse than no checkout. Stripe shaped pieces, amounts set in the browser, and unverified webhooks get flagged worth a look, so you run a real test charge before a customer does.

  • Customer data

    The report notes where real people's data lives: orders, profiles, uploads. Loose upload rules, and customer emails or passwords printed to the browser console, get flagged worth a look.

    Check uploads yourself

    • Firebase: search your repo for getDownloadURL imported from firebase/storage (or .getDownloadURL( calls) in upload, profile, or verification handlers. A VibeWrench study of 100 Firebase apps found 52% used it. The helper mints a permanent public signed URL that bypasses Storage security rules even when rules require auth. ID photos and verification selfies are the highest-stakes case.
    • Prefer server-side signed URLs with a short expiry, or upload through a Cloud Function that returns a time-limited link. Rules that look locked down in the dashboard do not protect a link that was already minted and shared.
    • Quick grep: grep -rn getDownloadURL src/ (or your app folder). Every hit in an upload flow deserves a read before you show anyone.

    What Seatbelt reads: the customer data surface. Repo read flags getDownloadURL in upload and profile paths without a nearby server-signed-URL pattern. Distinct from open Storage rules (allow read, write: if true) and from console-log leaks.

Run the whole checklist in one pass

Seatbelt reads all six surfaces in one pass and returns a Ship Read: a plain English verdict with every flag explained. Each finding shows the file and line, why it matters, the next step, and how confident the check is. Hand it a cloned GitHub repo folder, a project path, or a ZIP export if you have one.

  • Full findings included. You get the full report, not a locked score.
  • A static read of the build you hand it. Not a pentest. It never touches your live site or your customers' data.
  • If it finds almost nothing, the report says so.
  • In your ship flow: cleared pushes land with a shareable report link, flagged pushes park on a pull request instead of main.

Questions

Do I need to check before I show a cofounder or beta tester?

Yes, if anything sensitive could leak from what you show them. Show is not publish: a screen share, a staging link, or a local demo can still expose secret keys in the build or open database rules that anyone with access can reach. Run the four checks above, or one Ship Read, before you let another human near it, whether or not you have published yet.

How do I check my vibe-coded app before launch?

Lock down four things: no secret keys in the public build, row level security on every table, authorization enforced on the server, and no admin or debug routes left public. Each section above shows how to verify one by hand. A Seatbelt Ship Read checks all of them in one pass and explains what it found in plain English.

How common are leaks in vibe-coded apps?

Common enough that a third party measured it. In May 2026, the security firm RedAccess scanned about 380,000 apps built on Lovable, Base44, Replit, and Netlify and found roughly 5,000 leaking sensitive data. About 40% of the vulnerable apps exposed medical records, financial data, corporate strategy documents, or customer service chat transcripts. Axios independently verified named cases, including a UK clinical trials tracker and a Brazilian bank's internal financial records. RedAccess traced the root cause to defaults: these platforms start new projects publicly accessible, and many builders never change that. The checks on this page are how you verify your own app by hand, or a Ship Read reads them in one pass.

How fast is AI-generated code producing new vulnerabilities?

Accelerating. Georgia Tech's Vibe Security Radar counted 74 AI-attributed CVEs through March 2026 (6 in January, 15 in February, 35 in March alone), drawn from public advisories. That is a confirmed lower bound: researcher Hanqing Zhao notes Copilot inline suggestions leave no metadata, so the real count may be 5 to 10 times higher. Seatbelt reads the six surfaces that hurt launches in your build before you share the link. It is not a CVE dependency audit. Tools like Snyk own that lane. See Georgia Tech's Vibe Security Radar for the monthly breakdown.

Row level security is on. What can still go wrong?

Three subtle misses that look safe in the dashboard: a policy that allows everything (USING (true) on Supabase or allow read, write: if true on Firebase) even though the RLS toggle is on; an INSERT or UPDATE policy with USING but no WITH CHECK, so reads look scoped but a signed-in user can write rows as another user; and auth middleware that calls jwt.decode without jwt.verify in the same handler, so a forged token can pass the check. Run check 02 and check 03 above by hand, or one Ship Read on your repo export.

I'm vibe coding a dating or safety app. What should I check first?

Apps that collect verification selfies, government IDs, or private safety reports sit in the highest-stakes Firebase bucket class. Security.org summarized the Tea App breach: "The promise was privacy and safety. Then, in July 2025, that promise collapsed." Tea left Firebase Storage open. About 72,000 images leaked, including roughly 13,000 verification IDs. Read your Firestore and Storage rules for allow read, write: if true. Chat & Ask AI (Jan 2026) was the same open-rules class on Firebase Realtime Database at chat scale: roughly 300 million private messages exposed before patch (404 Media). Seatbelt reads rules files in your repo export, a static read, not a pentest. Run check 02 above by hand or one Ship Read before you let another human near it.

Do I get the full report?

Yes. Every flag, the file and line, and the suggested fix, not a locked score. If the scan finds almost nothing, the report says so honestly.

Does Seatbelt scan my live site?

No. Seatbelt reads the build you hand it, a project folder or a ZIP export. It is a static read of your code, not a pentest, and it never touches your live site or your customers' data.

I build on Lovable, Bolt, or Replit. Does this apply to me?

Yes. Seatbelt is not tied to one platform. It reads the code itself, so it works wherever you build. On Lovable, clone the GitHub repo your project syncs to (all plans). ZIP export needs Business. On Bolt, use the Bolt.new security checklist before Netlify publish. On Replit, sync to private GitHub and scan the clone, never a Repl URL. Replit always runs a pre-publish scan, but Block publishing of critical vulnerabilities is opt-in (Deploy → Advanced, off by default), and dismissed findings unblock publish (Replit docs, July 2026). That GitHub export path is still ungated. See Get started for the day-to-day setup in Cursor or Claude Code.

My login page looks fine. Can a server action still be open?

Yes. A common Next.js miss: the page calls verifySession at render, but a 'use server' action file deletes or updates with no ownership check. The page check runs once at render. The action runs on every direct POST. Open each actions.ts file that mutates data and confirm your session helper runs inside the action before the database call. Run check 03 above by hand, or one Ship Read on your repo export.

Do different login error messages leak which emails exist?

Yes. When an unknown email gets a different message than a wrong password, attackers can script a harvest list of registered addresses before they spray passwords. The fix is one generic invalid-credentials response for both paths, same status code and JSON shape. Password-reset flows that reveal whether an email is registered vs whether a token is wrong create the same harvesting shape. Try a random email and a known email with a wrong password: if the messages differ, fix the handler. Run check 03 above by hand, read auth account enumeration, or one Ship Read on your repo export.

Does my login route need rate limiting?

Yes, if attackers can try unlimited passwords or spam sign-up on your auth endpoints. A 2026 study of vibe-coded apps found roughly 80% lacked rate limiting on at least one auth route. AI scaffolds often ship bcrypt and JWT sign-in with no throttle middleware. Open each login, register, and forgot-password handler and confirm a limiter runs before credential checks: Upstash Ratelimit, express-rate-limit, or an edge rule on Cloudflare or Vercel. A common baseline is about 5 attempts per minute per IP on auth. Run check 03 above by hand. Auth rate-limit cues are on the detector roadmap.

Can a link preview or image proxy fetch internal URLs?

Yes. AI scaffolds often ship link-preview, image-proxy, or import-from-URL features where a url query param or form field flows straight into fetch() on the server with no host allowlist. That turns your app into a confused deputy to cloud metadata endpoints, localhost, and internal VPC addresses. In Next.js server actions and route handlers, confirm user-controlled URLs pass through validateUrl, an ALLOWED_HOSTS list, private-IP blocking, and redirect: "error" before the outbound call. Also open next.config and reject images.remotePatterns with hostname: "**". Run check 04 above by hand. Repo SSRF cues are on the detector roadmap.

Firebase Storage rules look fine. Can uploaded files still leak?

Yes. A VibeWrench study of 100 Firebase apps found 52% called getDownloadURL() after upload. That helper mints a permanent public signed URL that bypasses Storage security rules even when rules require auth. ID verification photos, health records, and profile uploads are the highest-stakes case. Search your repo for getDownloadURL from firebase/storage in upload handlers. Prefer server-side signed URLs with a short expiry or a Cloud Function upload path instead. Seatbelt flags getDownloadURL in customer-data paths on repo read. Run the customer data hand-check above by hand, or one Ship Read on your repo export.

Verified by Seatbelt badge

After a cleared Ship Read, embed the badge on your site. Visitors who want their own check land on the app with ?ref=badge tracked.

Verified by Seatbelt
Copy embed HTML
<a href="https://app.withseatbelt.com/app?ref=badge" target="_blank" rel="noopener noreferrer" title="Verified by Seatbelt: independent scan report"><img src="https://www.withseatbelt.com/badge.svg" width="200" height="51" alt="Verified by Seatbelt" loading="lazy" decoding="async" /></a>

Earned badges on cleared reports link to your live proof URL with ?ref=badge. See a sample report for the full pattern.

For agent fleets

Seatbelt is an enforced human-in-the-loop layer between agents: one agent's output is evidence, not authorization. The next agent builds only after a deterministic six-surface read. You still approve blocks and accept risks. Wire it once with /seatbelt init on Get started.