Algorithmic monoculture (software single point of failure)

Jon Kleinberg and Manish Raghavan, PNAS, 2021: when every hiring firm adopts the same algorithm, the overall quality of decisions can fall, even if that algorithm is more accurate than each firm's own judgment. Algorithmic monoculture refers to the situation in which a large number of organizations rely on the same foundation model for their decisions, creating a perilous homogenization. The concept, highlighted by researchers at Stanford's Center for Research on Foundation Models, is the software equivalent of the Single Point of Failure. If the shared model has an analytical bias, a reasoning flaw or a vulnerability, the failure is no longer isolated but replicated simultaneously across all the actors that depend on it. Monoculture thus destroys the cognitive diversification that, within a human organization, acted as a natural buffer, the probability of thousands of individuals making exactly the same error being negligible. Conversely, agents built on a single model make perfectly correlated errors. The AlgoPolis paper shows that this mechanism turns an idiosyncratic risk, the isolated human error, into an internal systemic risk, the generalized algorithmic error, and that it makes the law of large numbers, on which insurance rests, waver. Monoculture is therefore a major source of accumulation for portfolios exposed to algorithmic risk, on a par with dependence on a shared cloud provider in cyber.