Cybersecurity for AI Systems from First PrinciplesPost 6 of 19 · 19 of 19 written

6. Web and application security

Cybersecurity · 6 of 194 min readOct 2026

Nearly every web vulnerability is either an injection (data treated as code) or a missing authorization check. The browser runs code from many sources side by side, and the server receives requests it can never fully trust. AI features inherit all of these bugs and add new ways to trigger them.

The browser security model

  • Same-origin policy (SOP). An origin is scheme + host + port. A script from one origin cannot read data belonging to another. This is the browser’s core isolation boundary.
  • Cookies travel automatically with requests to their site, even when another site triggered the request. This is the root of CSRF.
  • CORS lets a server deliberately allow other origins to read its responses. Misconfigured CORS (for example, trusting any origin while allowing credentials) quietly removes the SOP’s protection.
  • Content Security Policy (CSP) restricts which scripts may run on a page. It is a strong second layer against XSS.

The classic vulnerabilities, and their AI-era versions

Vulnerability Root cause Primary fix AI-era version
SQL / command / template injection User input concatenated into a query or command Parameterized queries; no shell strings; sandboxed templates Model output placed into SQL or shell; text-to-SQL agents with write access
Cross-site scripting (XSS) Attacker’s script runs in your site’s origin Context-aware output encoding; HTML sanitization; CSP Chat UIs rendering model-generated HTML or markdown
Cross-site request forgery (CSRF) Browser sends authenticated requests another site triggered SameSite cookies; anti-CSRF tokens; Origin checks Agents with browser sessions visiting attacker pages
Server-side request forgery (SSRF) Server fetches a URL the attacker chose Destination allowlists; block internal ranges; egress proxy “Fetch this URL” tools, link summarizers, image-URL loaders
Broken access control / IDOR Missing per-object authorization Check ownership on every object, server-side RAG returning documents the user may not see
Insecure deserialization Formats that build arbitrary objects from data Safe formats; never deserialize untrusted data Loading pickled models and datasets
Security misconfiguration Debug mode, defaults, open buckets Hardened defaults; config review Unauthenticated notebooks and model servers
Vulnerable components Outdated dependencies with known bugs Dependency scanning and updates Large, fast-moving ML dependency trees
Business logic flaws Rules that can be bent (negative quantities, races) Careful design and testing Agents talked into refunds or discounts

Two of these deserve a closer look because they matter most for AI products.

XSS. If an attacker gets script running in your origin, they can do anything the logged-in user can do. Modern frameworks such as React escape output by default, and the danger comes from escape hatches (dangerouslySetInnerHTML, raw HTML in markdown renderers). The AI rule is simple: model output is untrusted input. The model may be repeating text an attacker planted in a document it read. Render it with the same care as any user-submitted content.

SSRF. When your server fetches a URL, it does so from inside your network, with your network’s access. An attacker can point it at internal admin panels, or at the cloud metadata endpoint (a special internal address where cloud machines fetch their own credentials) and steal those credentials. The 2019 Capital One breach involved this pattern. Defenses: allowlist destinations; resolve DNS first, then block private and link-local addresses (watch for DNS tricks that change the answer between check and use); require the hardened metadata service versions; and route fetches through an egress proxy. Every “browse” or “fetch” tool you give an agent is an SSRF surface.

APIs

  • Authenticate and authorize every endpoint, checking both the function (“may this user call delete?”) and the object (“may they delete this record?”).
  • Rate limits and quotas on everything. For LLM endpoints, cost is part of availability.
  • Size limits on requests, uploads and prompts.
  • Avoid mass assignment: binding request JSON straight onto database models lets users set fields like is_admin.
  • Return only what the client needs; extra fields leak data.

The OWASP API Security Top 10 is the standard checklist here.

The secure development lifecycle

Security is cheapest at design time and most expensive in production. A mature workflow layers several checks:

  1. Threat model at design (Part 1).
  2. Secure defaults from frameworks; avoid their escape hatches.
  3. Code review with security in mind.
  4. SAST (static analysis such as Semgrep, CodeQL, Bandit for Python) finds known bug patterns in code.
  5. SCA (dependency scanning such as pip-audit and Dependabot) flags vulnerable libraries.
  6. Secret scanning catches committed keys.
  7. DAST (dynamic scanning such as OWASP ZAP) probes the running application.
  8. Fuzzing feeds generated inputs to parsers and APIs to find crashes (Atheris for Python).

Vulnerabilities are tracked publicly as CVEs and rated with CVSS scores from 0 to 10. A CVSS score measures generic severity, not your risk; a critical bug in a component you never expose may matter less than a medium one on your login page.

AI-written code. If you delegate implementation to AI coding tools, treat their output like a contribution from a fast, well-read stranger. It can contain every classic bug above, use outdated patterns, disable security checks to make an error disappear, or import packages that do not exist (an attacker may register that name; Part 12). Owning the architecture includes owning a security review of what gets generated.

Questions to ask

  • Where does user-controlled or model-generated text get rendered, executed or used in a query?
  • Does the server fetch any URL a user or model can influence?
  • Is every endpoint, and every object, authorization-checked server-side?
  • Are there size, rate and cost limits on every endpoint?
  • What runs in CI to catch known bug classes, leaked secrets and vulnerable dependencies?
  • Which business rule would an attacker most like to bend?