doesn`t sound like your computer is under powered, but you might want to check the size of the windows swap, page file, as that can help in long compiles… mines set to a min and max of 800mb, some people probably use a lot more this is fine for me.
also you can do a test with your map, where you save a copy under a new name, then use the texture menu find, replace option and change every thing to caulk, then paint a just few visible brush faces with a normal texture, like a concrete, wood or grass… if you then compile this and see a big difference in the compile time, your know you`ve got to start using caulk more often.
also try to make as many brushes detail as possible, unless they touch the outside void, where they need to be structural… as this will reduce the vis compile time.