Algorithmic black box

The algorithmic black box refers to a system, typically a deep neural network, whose inputs and outputs can be observed but whose internal path leading to a given decision remains largely unexplainable, even to its own designers. This opacity does not result from a trade secret but from the very nature of these models, whose decision emerges from the interaction of billions of parameters within very high-dimensional vector spaces, with no interpretable symbolic representation. For risk management and agency theory, this opacity creates a new kind of information asymmetry. The human principal, the manager or the insurer, retains control of intent but loses the power to audit the reasoning chain that produced a result. The AlgoPolis paper shows that this structural blindness replaces the classic moral hazard of the human employee with a technological opacity, and that it raises supervision costs to the point of threatening the efficiency that automation promises. Explainability research, known as XAI, attempts to make decisions more interpretable, but its progress remains partial, and the black box remains one of the main obstacles to the use of AI in regulated fields and to its insurability.

An agent declines to indemnify a claim on the basis of reasoning that neither the insured nor the insurer can reconstruct, the model's opacity preventing any auditable justification of the decision.

boîte noire, black box, opacité algorithmique