This might seem dumb to most, but it's only my third batch and I just need some reassurance. So, I occasionally pop out the rubber stopper on my carboy to smell the brew and see if anything funky comes up so I know where to go with my dry hops in a couple weeks. While I was sitting around last night something hit me, when I pull the stopper out, I put it back in without sanitizing. Now it's only out for like 5 seconds tops and I hold it by the airlock, so I don't ever touch the rubber that goes back in the hole, and I don't set it down. Should I worry about contamination? I've only done this 4 or 5 times. It's just smelling AMAZING and I don't want to get through all of this and have caused an infection from my own curiosity.
As jsled says you have no worries. You are doing the right things, not touching it or putting it down. If just for a few seconds to check on the brew you'll be fine, also you will gain experience regarding how your brew evolves over time.
You should not worry as you are not setting it down for it to pick up bacterial contamination. Yes there is a tiny chance that a little critter will fall from the ceiling in the short interval whilst the stopper is out, but the risk is far outweighed by the learnings you will gain.
I have seen a lot of times that there is a layer of protection, when the ferment is going strong, the pressure of the bubbles actually push any contaminants out of the stopper hole. The layer of foam created by the ferment is also a protection from contaminants. My airlock had the ferment bubbling into the airlock, so I had to take it off and clean it, and add it back. Everyone stated that this is natural and if done correctly it would not contaminate the batch.
There is always a possibility of contamination, but if you are aware of it and protect your brew from it, you should be fine. There are natural combatants to contamination going on when the ferment is in full force.