I agree with others; I think your fluctuating temperature is the likely cause. I have never tried the swamp method but I am about to convert a refrigerator into a fermentation chamber.

I have a temperature control unit I picked up on ebay from someone in Hong Kong - the unit name escapes me at the moment. I am going to by-pass the thermostat on the fridge and hook up the temperature control to activate the compressor when too warm and a low wattage light bulb, that I will fix to the bottom of the fridge, when too cool to provide heat.

I am hoping to source a second-hand fridge this week, so I might be able to get started on this at the weekend.

If you Google 'fridge fermentation chamber' you can find quite a few links to blogs and forums on this.