Built an App with AI? Get a Security Review Before Launch

Something has changed in the last year or so. People who have never written software for a living are now building working custom applications with an AI assistant: an intake form that feeds a database, an internal dashboard, a scheduling tool for a small team, a portal for customers to upload documents. It runs, it does what they asked, and it solves a real problem. I think that’s genuinely useful, and I’m not here to tell anyone to stop.

My concern is with what “it works” means. An AI assistant is very good at producing code that does what you asked. It is much less reliable at the things you didn’t think to ask about: who is allowed to see which records, what happens when someone types something unexpected into a form, where the passwords and API keys live, and whether the whole thing is reachable from the open Internet. Veracode’s 2025 GenAI Code Security Report tested over 100 large language models on coding tasks and found that 45% of the code samples failed security tests and introduced OWASP Top 10 vulnerabilities, and that newer, larger models did not write more secure code than older ones. That’s with the models being asked to write code, by people who knew to test it.

A non-developer usually doesn’t know which questions to ask, and that’s not a criticism. It’s the same reason you hire an inspector before you buy a house you’re perfectly capable of walking through yourself.

A Few Hours at the Whiteboard

This is why we’re starting to offer expert security review of custom applications built by non-developers with AI assistance. It doesn’t need to start as a full code audit. A few hours at a whiteboard, walking through what the application does, can save a lot of grief later. The questions are the same ones I’ve taught developers for years as part of threat modeling:

1. What does it hold? Customer records, health information, payment details, documents, passwords. What would hurt if it leaked, was changed, or disappeared? Those are the five pillars of information security in practical terms.
2. Who can get in, and what can each person reach? How users sign in, whether one user can see another’s records, and what an administrator can do that others can’t. This is where I’d expect the most trouble, because broken access control is the first entry in the OWASP Top 10 for a reason.
3. Where does it run, and what does it trust? The hosting, the database, the third-party services, and the secrets that connect them.
4. What happens when something goes wrong? Backups, logs, and who notices.

Sometimes the answer from that session will be that the application is fine for what it’s doing. Sometimes it will be a short list of fixes you can hand right back to your AI assistant, and you’ll be better at asking it the right questions from now on. And sometimes it will be that the application now holds data important enough to deserve a full authentication and access control audit or application vulnerability assessment. Either way, you’ll know before your customers or an attacker find out for you.

If you’ve built something with AI and you’re starting to wonder whether it’s safe, I’d like to hear about it. Every Rietta solution starts with a free 30-minute scoping call, so contact us about a security review of your AI-built application.