I have seen lots of posts where guys say they stayed up late because their wort hadn't reached pitch temp until 4am or somethinglike that.
What is the harm in just going to bed at 1 or 2, waking up and pitching 6 hours later? or even 8 hours later?
There's a couple of concerns regarding this:
That said, 8 hours isn't a terrible amount of time for wort to sit. I've been in the same circumstance you described, even to the point where I've dozed off (aka drank too much while waiting until 4AM) and pitched the yeast as soon as I woke up, realizing I let my wort sit unattended. Make sure it's covered as best as possible (not the easiest to do with immersion chillers), not in a drafty area, and you will likely be fine.
Look up some information on no-chill brewing, which is a method for doing exactly that. http://www.homebrewtalk.com/wiki/index.php/No_Chill_Method
One drawback is that if bacteria or wild yeast does make it into your wort, it will have a head start on growth, where otherwise the inoculation of brewer's yeast might overpower the bad bugs and minimize impact.
Also, if you're not doing anything to expedite the chilling process, it may affect the isomerization of hops, depending on the style you're making. Basically, late addition hops flavor and aroma will change (generally in an undesirable way) if they stay in hot wort too long. No-chill brewing incorporates some practices (like first wort hopping) to counter this.