Depending on the yeast type, and the OG of your starter wort (hopefully around the 1.040 range), it should complete fermentation within a very short time. I just did a starter using WLP090, and it completed fermentation within a day, so that can be used as a "shortest possible time", whereas others might take 48 - 72 hours at most. At that point, there isn't much need to keep it on the stir-plate. Refrigerating will help the yeast flocculate to the bottom of your starter. What this will do is take the healthy yeast and slowly cool them down to settle at the bottom of your flask/container. As long as they don't sit in your refrigerator for an extended period of time (read that as 1-2+ weeks), they're still plenty healthy and ready to go. If they are resting for that long, or longer, doing another yeast starter may be necessary before pitching. The added bonus of refrigeration vs. straight pitching the entire flask/container is that you can decant some of the wort as to not effect your OG if you have over a liter of a starter. Simply take it out of the fridge on brew day, let it set, and before you pitch, pour out down yo about 400-500 ml of the wort, swirl the container to get a nice slurry, and pitch.
Long story short, you're fine. Keep it sanitized, and you'll be ok.
EDIT: If you do go the refrigeration route, it may be best to cork the container, or put an airlock on it just to be safe. I've used everything from saran wrap to an airlock, the point is you don't want anything to creep in since it is a very low-gravity solution. It's more susceptible to infections than a moderate gravity beer would be.
