Here’s what it’s doing:
Assuming you’ve not got anything else modifying the vertex alpha for your terrain (like, say, a colormap for your picoterrain), then the alpha for each of your terrain’s vertexes is straight white. All the way solid.
The q3map_alphaMod dotproduct2 ( x y z ) line of the shader tells q3map2 to look at every vertex to which this shader is applied, and modify that vertexes’ vertex alpha in a certain way. That way is sort of complicated, but not really. You define a vector in the ( x y z ) part of the shader–In my example terrain shader, the vector I defined was ( 0 0 0.85 )–and there is also a vector defining the normal direction of every vertex of your terrain. The normal vector just tells quake which way the vertex is oriented–and between the 3 vertexes needed to define a surface, the average of their vertex normals becomes the surface normal. The surface normal, just like the vertex normals do for vertexes, define how a surface is oriented. The pitch, yaw, and roll of a given triangle. The slope of a given part of your terrain. The surface normal is really what we’re interested in, but you have to work with the vertex normals in order to get at it.
Anyway, q3map2 does a dotproduct operation between the vector you define, say, ( 0 0 0.85 ), and the normal vector for whatever vertex it is working on right now. I’m not sure if the op goes shader dot vertex or vertex dot shader, but either way, this is what’s going on. It then squares the result (that’s the difference between q3map_alphaMod dotproduct and q3map_alphaMod dotproduct2) and multiplies the original vertex alpha value–in our terrain’s case, that was probably 100% white–by the final dotproduct2 value.
With my vector, which points straight up, what ends up happening is this: the vertex alpha for horizontally flat areas of the terrain ends up staying pretty white. The alpha ramps down as slope increases, all the way to the point that the vertex alpha for vertical (“cliff”) faces becomes straight black. A gentle hill sloped at about 45 degrees would have 50% grey vertex alpha.
This stuff is all dependant on how you set your vector up in your shader, though. If I had used ( 0.85 0 0 ) in my shader, the vertical faces oriented in the +X direction would be the ones that get to keep their 100% white vertex alpha, and the horizontally flat faces would be pushed to black.
Anyway, now that q3map2 has worked out the vertex alpha for your terrain, next comes the actual shader drawing.
Dirt.tga, which has no alpha channel and no blendFunc, gets drawn on every triangle of your terrain. It is everywhere.
Rock.tga, which has a bright alpha channel (~90% white), gets blended and masked off pretty normally. However, the alphaGen oneMinusVertex essentially inverts that bright alpha channel, and Rock.tga ends up only getting drawn on terrain vertexes that have very dark (as dark as or darker than Rock.tga’s alpha channel is bright) vertex alpha.
Grass.tga has a similarly bright alpha channel (~90% white), and it gets blended and masked off likewise. Its alphaGen line uses the straight vertex alpha (instead of the inverted “oneMinusVertex” version), so Grass.tga, with its almost-white alpha channel, gets drawn on all vertexes that have very bright (as bright or brighter than Grass.tga’s alpha channel is bright) alpha–the horizontally flat surfaces.
Finally, just throw a map $lightmap on there and make sure the three previous passes had rgbGen identity, and you’re good to go.
When ydnar gets back, maybe he’d correct any glaring errors I made. I’m pretty sure that this is pretty much what’s going on, though, and good luck to you in experimenting with this stuff!
