Feature engineering
The work of turning raw data into variables useful to the model, tenure, ratios, aggregates, geographic indicators, where much business know-how resides and a good share of discrimination risk.
A model sees only the variables it is given. Feature engineering consists in building, from raw data, the variables that actually carry the relevant information: a driver's years of experience rather than license date, the number of claims in the last three years rather than the list of claims, the ratio of insured value to floor area, the distance to the nearest watercourse rather than the address, the trend of an indicator rather than its level. That is often where most of a model's performance is decided, far more than in the choice of algorithm, and where the business know-how of the underwriter and actuary is expressed, since they know which characteristics make the risk. Deep learning models partly build their own features, but on insurance tabular data human work remains decisive. It also carries risks. A seemingly neutral variable can act as a proxy for a protected characteristic, and a variable built with information later than the decision date, data leakage, gives the model an illusory performance that collapses in production.
Textbook case: a team adds to its home model a computed variable, the building's height relative to the nearest hundred-year flood level. Discriminatory power on flood claims improves more than it did when moving from a linear model to gradient boosting on the original variables.
Follow the research
One letter a week. First, the week's underwriting note, when one is published: a dated, argued decision, signed by Paul Bernardski. Then what the observatory published on cyber risk, political risk and insurance. No account, no tracking.
No spam, one-click unsubscribe, no data ever sold.