We are told to leave around an inch of headspace (depending on the bottle), but does anyone know why?
Gas compresses under pressure. Liquid doesn't.
If you have no head space, then when CO2 is produced after bottling, there's nowhere for the pressure to go but to break the bottle.
If you have too much head space, then if enough CO2 is produced to produce the bottle, then there's quite a bit of compressed gas expanding, and that expansion is dangerous, throwing bits of glass around the place, and perhaps either hurting someone or breaking neighbouring bottles in storage.
I remember a product recall of soft-drinks in New Zealand because the head space was too large, and accidents were occurring in supermarkets.