Emergent abilities

Emergent abilities refer to new behaviors or skills that appear in a large language model when it crosses a certain scale threshold, in number of parameters or volume of training, without having been explicitly programmed and without being predictable from smaller models. The term was popularized by the work of Wei and co-authors, who show that certain skills absent from small models arise relatively suddenly beyond a critical size. This phenomenon has considerable significance for risk management, because it means a model's behavior is not entirely deducible from its design, and that scaling up can give rise to abilities, but also failures, that were not anticipated. The AlgoPolis paper draws precisely on this unpredictability to explain why algorithmic risk falls under Knightian uncertainty rather than priceable risk, the execution environment evolving faster than actuaries can collect stable data on it. It should be noted that the notion is debated within the scientific community, some researchers arguing that the apparent suddenness of emergence is partly an artifact of the metrics chosen to measure it, which in no way removes the practical unpredictability of large models' behavior.

A modest-sized model fails to solve multi-step reasoning problems, but a markedly larger version suddenly succeeds, without any architectural change directly explaining it.

capacités émergentes, emergent abilities, émergence