'Is it simply a matter of stirring continuously?'
Pretty much, especially with DME. Once it's dissolved fully, though, there should be no further need to stir. So really 'continuously-until-it's-dissolved'.
I find it helps to add DME slowly while stirring or whisking continuously, until it has all been added, and then continue to stir until visible signs (powder, floating solid bits) are gone, then scrape the bottom and all the corners to check there, then stir until you can't fish any more noticeable chunks out with a bit spoon. It's kind of like making a sauce with a roux. You just have to stir until it's dissolved.
LME is a bit easier, especially if you warm it up before opening to decrease its viscosity, making it much easier to pour and dissolve. Just stick the can in a bowl of warm-to-hot water for 10 or so minutes (shaking the can can give you a good sense of how much it will have liquefied), open and stir well while adding. Adding cool/cold LME increases your chance of chunks that will have to dissolve, and will require more stirring.
I would also say make sure to always add extract to water, as opposed to the other way around. Water being at least warm is best, too (why not add it while you're heating your water up to either steeping temperature or to a boil; just wait until the water's warm, add extract, and stir until dissolved), though waiting until it's boiling, to me, just risks caked-up soggy DME and burnt knuckles, needlessly.