Multicollinearity
Strong correlation between a model's explanatory variables, which makes its coefficients unstable and hard to interpret without necessarily worsening its predictions, a classic trap in tariffs with many variables.
Multicollinearity exists when several of a model's explanatory variables are strongly correlated with one another: driver age and years licensed, floor area and number of rooms, a company's revenue and headcount. The model then cannot properly separate the effect of each, since they move together. The consequences are misleading. Overall predictions may remain good, but coefficients become unstable, changing sharply from one sample to another, sometimes taking signs contrary to intuition, with inflated estimation errors. In a tariff this shows up as inconsistent relativities and erratic changes from one revision to the next, hard to explain to sales networks and regulators. The remedies are known: measure collinearity with indicators such as the variance inflation factor, remove or group redundant variables, build combined variables, use ridge-type regularization to stabilize coefficients. With tree methods the problem shifts: predictions remain stable, but variable importance and individual explanations spread arbitrarily among correlated variables.
Textbook case: in a professional liability tariff, revenue and headcount are correlated at 0.9. The model gives headcount a negative coefficient, against all intuition, and the coefficient changes sign at the next revision. The team replaces the two variables with revenue alone and a revenue-per-employee ratio, which stabilizes the tariff.
Follow the research
One letter a week. First, the week's underwriting note, when one is published: a dated, argued decision, signed by Paul Bernardski. Then what the observatory published on cyber risk, political risk and insurance. No account, no tracking.
No spam, one-click unsubscribe, no data ever sold.