I prefer to give my starters four or five days. Fermentation is usually done in 24 or 48 hours, but I like to cold-crash the starter so that the yeast falls to the bottom. That way, I can decant the beer off the yeast, and pitch just the yeast. If you only give your starter 24 hours, you're forced to pitch the whole starter into your wort. Not a big deal, I suppose but I prefer to pitch just the yeast.
If you can't let your starter ferment for 24 hours, I'd suggest you skip it entirely. Every time you move your yeast from one place to another, you increase the risk of contamination. If you only have a few hours for your starter, it's unlikely to affect your yeast's cell count, and there's some small risk of infection from the procedure. So you're better off to just pitch directly from the smack pack.
You'll almost certainly be under pitching, unless your beer has a low starting gravity and the yeast is very fresh. You can mitigate the effects of under pitching by oxygenating the wort.