Excessive agency
A vulnerability of an AI application given more functions, rights or autonomy than its task requires, so that an error or manipulation of the model turns into harmful actions.
An application built on a language model can be connected to tools that let it act: send messages, modify files, place orders, execute payments. The reference list of risks for large language model applications, published by the OWASP community, ranks excessive agency among the main vulnerabilities. It distinguishes three excesses. Excessive functionality: the agent has tools it does not need, for example a tool that can delete files when it only needed to read them. Excessive permissions: the tool runs with overly broad rights, an admin account rather than one limited to the user. Excessive autonomy: the agent can carry out a high-impact action without human confirmation. Since the model sometimes errs, and can be manipulated by content it reads, each of these excesses turns a reasoning flaw into real damage. Countermeasures apply an old security principle, least privilege: limit tools to what is strictly needed, execute each action with the rights of the user concerned, require human approval for irreversible operations, and log actions. For insurance, excessive agency is the criterion that separates an assistant that can be wrong from an agent that can cause a loss.
Textbook case: an AI refund agent can, to save time, approve payments up to 10,000 euros alone with an account that has access to all files. A fraudster slips instructions into an attachment that lead it to approve forty fictitious refunds; no human approval was required.
Follow the research
One letter a week. First, the week's underwriting note, when one is published: a dated, argued decision, signed by Paul Bernardski. Then what the observatory published on cyber risk, political risk and insurance. No account, no tracking.
No spam, one-click unsubscribe, no data ever sold.