Responsible AI

Prompt Injection & AI Security

Prompt injection is when untrusted text tries to override instructions, leak data, or manipulate tool use. Any AI system that reads external content and can act on it needs a security model.

External text is untrusted input
Authority must not come from text content
Tool calls require independent checks
Data exfiltration is a core risk
Sandbox dangerous capabilities
Security evaluation needs adversarial cases

Unsafe pattern

page = fetch_url(user_url)
prompt = system_rules + page.text + user_request
answer = model(prompt, tools=[send_email, delete_file, fetch_private_docs])

Untrusted page text now shares context with powerful tools unless the app enforces boundaries.

Separate data from authority

External text can inform answers, but authorization comes from the application.