From Water: A comprehensive Guide for Brewers by John Palmer and Colin Kaminski:
The calcium carbonate that has precipitated [from boiling] exists as micro-crystals in suspension, which will eventually grow heavy enough to settle out. According to historical brewing texts such as Syke's, the water would typically be boiled for a half hour to allow the CO2 to be well scrubbed by the steam, and would then be allowed to settle overnight, leaving a white layer of precipitate on the bottom of the kettle. The reduced-alkalinity water would then be decanted off the sediment for use as brewing liquor.
...
Note that while our goal here is to remove bicarbonate(alkalinity), we are simultaneously removing calcium, which is not necessarily desirable. Therefore brewers who decarbonate their water this way often replace the lost calcium by additions of calcium chloride and/or calcium sulfate. It helps if this is done before decarbonation as the extra calcium aids in the decarbonation process, again according to LeChatelier's Principal.
...
But what if the water is not allowed to cool or the precipitate to settle? .... So heating of brewing water to strike temperature may result in a small reduction in alkalinity and hardness. Vigorous stirring or aeration can help the CO2 to evolve and increase the precipitation. The result is that some small proportion of the initial alkalinity would likely precipitate, byt still be suspended in the hot liquor when it is added to the mash. What is the effect of the suspended precipitate in the water being added directly to the mash? That is a good question.
Reducing alkalinity with slaked lime (calcium hydroxide) is very similar to the boiling method, but slaked lime adds more calcium and raises pH, achieving lower levels of alkalinity than boiling alone. ... Water with [249 ppm CaCO3] can be reduced to [21ppm].
[pg 103-108]
So, calcium ions reacts with bicarbonate to precipitate calcium carbonate in boiling water. Not only should water be boiled for 30 minutes, but the kettle needs to remain over night to allow the precipitate to settle-- Decanting wouldn't do much of anything immediately after boiling.
I could look into CaOH, but I'm just going to buy a RO system.