Human in the loop

Tempe, Arizona, March 18, 2018: an Uber vehicle in autonomous mode kills Elaine Herzberg while the safety driver, who was supposed to take over, was looking at a phone. In 2019 the NTSB cited automation complacency among the causes. The human-in-the-loop principle keeps a human intervention (oversight, validation or the ability to take back control) inside an automated decision process. It is often presented as the ultimate guarantee of safety and responsibility as artificial intelligence systems grow more autonomous: a human watches and can correct or block an inappropriate decision. There are several degrees: the human in the loop, who validates each decision; the human on the loop, who supervises and can intervene without validating everything; and the human out of the loop, for fully autonomous systems. The difficulty is that the real value of the principle depends on the quality of the control. When the human, out of convenience, overconfidence or overload, merely rubber-stamps the system's outputs without real scrutiny, that is automation bias. Oversight becomes an empty formality that gives the illusion of control without its substance. For insurance and law, the question is central. The formal presence of a human in the loop is often invoked to establish liability or justify a deployment, even though it may no longer guarantee anything in practice. Human oversight under Article 14 of the AI Act goes further: it requires that a person can understand, monitor and stop the system, without approving each decision one by one.