I tested this once i think and im pretty sure that tings that are not permanently on the map or permanently in the same spot (think about constructibles, script_movers, and doors for example) can be left structural and will not contribute to portals… i think they are made detail by the compiler…
Think about it… make a giant constructible (a gate?), leave it structural. Assume it would block your VIS there.
Then destroy it, it would still block your VIS! (As VIS is compiled and cannot be ‘decided’ in runtime afaik…)
But no it doesn’t obviously…
(as you would get a giant HOM effect when it would do that…)
I tested this with a destroyable obj (gate) and a func_door… both werer structural and both did not block vis…
If im wrong here please correct me 
EDIT
And about above statment (only caulk will block vis) this is idd not true… All brushes can block vis, as long as they are not textures with a shader that has “surfaceparm detail” (which will always make the brush detail).
So a structural brush with a rock texture will also block vis.