Safe answer is to boil every bit of water that touches your fermenter. But that's sometimes overkill.
If you have a well like I do, follow the boil rule. Even if the well water checks out clean, they're only looking for coliform, not other things, and even though I have a UV sterilization unit to kill bacteria, there is no residual sanitizing effect in the lines, so there is the chance of clean well water but dirty lines.
If you are on public water, there is almost certainly some type of chlorine or ozone added to kill and then provide residual effects against bacteria. In the USA, they are switching almost everywhere to chloramines, which are far more stable than free chlorine, and thus they can use less of the stuff and have the water stay bacteria free for longer.
Unfortunately, chloramines' stability means you CANNOT remove it by a short boil or by leaving it out overnight. There are two common methods to remove it: Filter through carbon SLOWLY, which will remove the chloramine if done properly, but I have to admit that I wonder if carbon filters themselves can be problematic for sanitation. Adding a reverse osmosis unit after carbon filter is even better. The second method is to simply add sulfites, which react immediately with chloramine and reduce it to zero in a matter of minutes.
I prefer the sulfite method at a rate of about one campden tablet per 10 gallons of water -- you don't need much, just make sure it dissolves and mixes well. There will be some residual sulfite, but the level is so low that it will not affect your yeast, and it will gas off from there early in fermentation. Remember, this is for city water that is already bacteria-free because sulfiting at this weak level will not kill off any bugs that are already in there, and well water doesn't have chloramines anyway.
For city water with chlorine or chloramines, ALWAYS remove those first before brewing with it or adding to a fermenter. I've tasted many otherwise-well-made homebrews that were ruined by using plain chlorinated water. Chlorine plus tannins (from grain or hops) gives you chlorophenols, and these are very, very potent flavor compounds, and not in a good way: medicinal and band-aid.
Calvin Perilloux, Middletown, Maryland, USA