Can't say there is much of a scientific answer to this question. If I had to guess though, I'd say I definitely wouldn't want the bung flush with the top of the carboy, maybe at least 1/4 - 1/2 an inch protruding from the top of the carboy. That way if it does get stuck, I could use something to grip onto it and pull it out. Typically it's safe to just go until it's snug, and you should be good. I've never had a bad seal from just pressing until it's snug. If the top of the bung is flush with your carboy (without applying unreasonable "hulk-smash" force on pushing the bung in), it is too small, because at that point, it could theoretically fall in with enough force. Mine never run that risk, so you may want to consider going one size bigger.
In the meantime, I'd imagine a wire coat-hanger bent to form an "L" shape would be able to fit through the hole in the bung pop it out fairly easily, so long as it isn't wedged in way too tight.
EDIT: I misread part of your question. When you say you try to push in the stopper and it pushes back out, by how much? It may still have a sufficient seal. Depending on the material, you may also try spraying the stopper with starsan before pushing it in, and the liquid may (or may not) cause it to seal without pushing out. I retract my previous suggestion of going bigger, as it sounds like if anything, your stopper may be too big, judging by how much it is pushing out. My guess is though that your seal is just fine, and the cork is the appropriate size.