Yeah i had the same huge compile times when i first started it was because i had barely used detail brushes. Structural brushes create portals and the more portals there are the longer the vis compile takes mine was 30000 vis portals i managed to get it down to like 6000 - 7000 and the compile times came down.
Do you know that if you select a brush and then right click in the window you can select ‘Make detail’, use the filters to block detail brushes then you can see all that is structural. I kept walls of buildings and large objects that block visibility as structural and all small eye candy things as detail.
Hope this helps you a bit.
<DJ>