I pitched the yeast at 40°F (4.5°C). Was it way too cold and should I re-pitch?
I assume you mean 40°F, then yes way too cold. it won't harm the existing yeast organisms, but even for lager yeast that is too cold for primary fermentation.
so basically we do things at certain temps for a reason... we try to pitch yeast at a temperature that gives them the advantage over other contaminant organisms.
warm your wort to the primary temperature and fermentation should start after a short delay.
I assume you mean Fahrenheit and are brewing an ale, not lager. Yes- that is way too cold. Most ales are best pitched around 75°F (24°C), although I've yet to have a problem with temperatures up to 100°F (37°C).
For best results use a wort chiller to chill the wort as rapidly as possible. The longer the cooling time, the greater the chance of contamination. Additionally, a yeast starter is always a good idea.