It is a problem. The moisture from the filler has to go somewhere. Don't waste time with a heat gun who knows what else damage you might cause. PVA in filler can't see what that does for drying times.
As you say, fill in small stages, if the filler is solid enough to turn upside down without flopping on the floor and still creamy enough to work with, thats all you can do. And have indirect heat on in the room.
Polycell Deep Hole is a 2-pack and dries very quick and is a strong filler. It is pricey but seems to go a long way. (Its that balance again, is your time being held up by slow drying filler worth more than paying top dollar for a quicker drying filler.)
I do add emulsion to filler at times but that is for making a strong backing mix to build up deep holes, not because it makes the filler dry quicker.