Compiling Larger Light Maps into BSPs


(richardmonette) #1

I am wondering if it is possible using q3map2 (and gtkRadiant) to compile larger than 128x128 lightmaps directly into the BSP file?

I am loading the BSPs into a custom engine I am working on as a hobby, so I have no issue engine side dealing with tBSPLightmap lumps that are larger than 128x128 - I can just modify my engine code to handle the additional data.

I have been reviewing the documentation for q3map2 and I have come up with the following shader file which nearly does what I want:


textures/firstmap/my_texture_hi_res_lightmap 
{
	qer_editorimage textures/firstmap/grey.tga
	q3map_lightmapfilterradius 2 0
	q3map_lightmapSize 1024 1024 
	q3map_lightmapsamplesize 16
	q3map_lightmapBrightness 2.0
	q3map_normalimage textures/firstmap/grey_normal.tga
	{
		tcgen lightmap
		map $lightmap
		rgbGen identity
	}
	{
		map textures/firstmap/grey.tga
		blendFunc GL_DST_COLOR GL_ZERO
	}
} 

This shader seems to create an external lightmap file lm_0000.tga in my maps folder in a folder which is automatically generated. It also appears to be auto generating a second shader - to what end I am not 100% certain.

When I run the BSP file in my engine I can see that there are some textures being included but that there is no lightmap data being added to the BSP.

Can I get those larger 1024x1024 maps into my BSP file somehow?


(VolumetricSteve) #2

http://www.splashdamage.com/forums/showthread.php?t=20334

Hopefully this will help you as it helped me.


(richardmonette) #3

VolumetricSteve,

Thanks for your post but having read through it unfortunately I don’t think it specifically addresses my problem. obsidian definitely has some good tips on how to get the best quality out of lower resolution maps and explains how bumping up the resolution doesn’t guarantee a better result but I still don’t know how to actually put higher resolution maps into the BSP file in the event that I need to do so.

Is it possible to get higher resolution lightmaps compiled into a BSP?


(Detoeni) #4

If q3map2 is generating them externaly with there own shader, they dont have to go into the bsp to work.


(richardmonette) #5

Detoeni, thank you for your response however it still doesnt exactly tell me if what I would like to do is possible. Its quite possible that there may be an alternative solution but I’d really like to know if its actually even possible to put the hi-resolution lightmaps into the BSP file.

Maybe a little more background might help to clarify my question;

Since I am loading the BSPs with my custom engine I have a load routine which handles moving through the various ‘lumps’ of data in the BSP. The faces that are drawn are defined with a structure that is pretty close to this:

int texture 	Texture index.
int effect 	Index into lump 12 (Effects), or -1.
int type 	Face type. 1=polygon, 2=patch, 3=mesh, 4=billboard
int vertex 	Index of first vertex.
int n_vertexes 	Number of vertices.
int meshvert 	Index of first meshvert.
int n_meshverts 	Number of meshverts.
int lm_index 	Lightmap index.
int[2] lm_start 	Corner of this face's lightmap image in lightmap.
int[2] lm_size 	Size of this face's lightmap image in lightmap.
float[3] lm_origin 	World space origin of lightmap.
float[2][3] lm_vecs 	World space lightmap s and t unit vectors.
float[3] normal 	Surface normal.
int[2] size 	Patch dimensions. 

As you can see there is a texture index and a lightmap index. All the textures are loaded from a list so the index tells me which one needs to be used for a face. Similarly, the lm_index (lightmap index) says which lightmaps (from a list loaded from inside the BSP) I should be rendering with.

In order to use both the lightmap and diffuse texture at the same time I have written a GLSL shader. The GLSL shader is what OpenGL actually uses on the video card to shade the surface.

This GLSL shader is different than the “shaders” which q3map2 uses to understand the surfaces it is pre-processing (generating data prior to when the engine runs)

Whats happening right now is that the lightmap index in the BSP is ending up a negative value indicating that it has not been set. What this means is that I dont have a nice way to know which external lightmap I would need to load.

If I can get q3map2 to just write the larger lightmaps into the BSP then my whole system would start working again.

One other note - a lot of people keep telling me that it is possible to increase the visual quality of a lightmap without needing to increase its size by using various filtering, etc options.
I realize that this is possible. However, in my specific case I am also trying to use the normalImage parameter in order to make q3map2 take my normal maps into account when generating lightmaps.

One of the issues with this is that my normal maps have fairly high frequency data - that is the resulting light value from texel to texel can be quite dramatic. In order to preserve this information I need higher resolution lightmaps.

Perhaps someone can clarify if I have totally missed the point here?

Is it possible to get higher resolution lightmaps compiled into a BSP?


(Detoeni) #6

http://en.wikibooks.org/wiki/Q3Map2

scroll down to “-import”

got no idea if this will work, it should work in theory…

[i]edit:

On second thoughts, that wont work.


(Detoeni) #7

[QUOTE=richardmonette;209533]… What this means is that I dont have a nice way to know which external lightmap I would need to load.
[/QUOTE]

Big light maps are a bit of a hack, the shader used does not have any light map data in the bsp. all you will have for the polys using external light maps is the shader/texture path created by q3map2

I’v only ever used large light maps for terrains, I’v got no idea of what problems you might get with the uv’ing across lots of surfaces.
When working with q3 a few years back, I had to crop and resize the external light map so the uv texture alignment was correct (quite easy to do with a single square)

Wolf ET’s bsp uses true external light maps (you can set the size as a compile switch), so it must be making a texture path that you can use, it may be a way forward with what you are trying to do.


(obsidian) #8

If you want the lightmaps to be stored internally in the .bsp at resolutions higher than 128x128, you’re going to have to modify a fair amount of code to do this. Most id Tech 3 engine games are locked at this resolution. W:ET has slightly bumped up resolutions (256 or 512, I’m not sure). Regardless, it’s a hard-coded limit unless you use external lightmaps.

It also sounds as if your custom engine isn’t loading external lightmaps.


(richardmonette) #9

Just wanted to follow up on this and report back my findings so that others hopefully can avoid all the running around I have been doing. Ill try and summarize in specific factoids (if anyone sees something wrong in these please let me know and Ill be sure to correct them).

  1. Vanilla Quake3 (that is unmodified) uses 128x128 lightmaps

  2. q3map2 cannot compile larger than 128x128 maps into a BSP (doing so would probably be a bad idea anyways)

  3. It is possible to export and re-import the 128x128 maps using q3map2 but the size limitation is still 128x128 when using the internal/compiled lightmaps

  4. It is possible to make q3map2 export external lightmaps at resolutions higher than 128x128, this is accomplished with the q3map_lightmapSize switch. The version of q3map2 that is included with gtkRadiant 1.5 will put those exported maps into a folder in your maps directory. netRadiant does a similar thing but it is important to note that inside the BSP these export lightmaps will be referenced differently. gtkRadiant’s version of q3map2 seems to make the path something like “mapname/F8shhasjaks8shddakakdh” i.e. a long random string. netRadiant is a bit better, it makes the path “mapname/lightmaps/lm_XXXX”

  5. Inside the BSP the vertex’s have a lightmap UV and a texture UV, I was unable to make either of these correctly render the external lightmaps - clearly this is possible somehow since others are doing it but I didnt get this far

  6. It is possible to have q3map2 take into account a normal map image when generating lightmaps using the q3map_normalimage switch, however having done a number of tests with and without external lightmaps it is apparent that the lightmaps would have to be ridiculously high resolution for this to even work properly. Even with 1024x1024 lightmaps the way the normals were used made the lightmaps look blurry. Near as I can tell there is no point to using this switch.

  7. There is another normals related option called deluxe mapping which does generate some useful normals data - this is possible because the normals it generates are for the relatively smooth surfaces of the faces and they dont have to encode high frequency/resolution surface data

  8. Using the lightmapscale property in the entity editor makes it possible to get very decent shadow data in the normal 128x128 maps - apart from some special situations this is probably enough for most people

Based on these findings what I would recommend to people is that they use the regular 128x128 maps with a lightmapscale of around 0.1. Dont bother trying to use the normalimage switch - just write a bit of extra code to load up the maps seperately in the engine and use GLSL to re-light the surfaces using the nearest n lights. Its a shame that it cant just be preprocessed nicely but the end visual result is pretty decent and it isnt super hard to do anyways.

Anyways - hopefully this will help someone else who is trying to figure out how to mix lightmaps, normals and the preprocessed and realtime lighting.

If anyone has any suggestions on better solutions I’d love to hear them.


(Detoeni) #10
  1. Vanilla Quake3 (that is unmodified) uses 128x128 lightmaps

Yep

  1. q3map2 cannot compile larger than 128x128 maps into a BSP (doing so would probably be a bad idea anyways)

Return to castle Wolfenstein used 256x256, but not up to what you need.

  1. It is possible to export and re-import the 128x128 maps using q3map2 but the size limitation is still 128x128 when using the internal/compiled lightmaps

Yep

  1. It is possible to make q3map2 export external lightmaps at resolutions higher than 128x128, this is accomplished with the q3map_lightmapSize switch…

Q3map2 uses “-external” in W:ET.

  1. Inside the BSP the vertex’s have a lightmap UV and a texture UV, I was unable to make either of these correctly render the external lightmaps - clearly this is possible somehow since others are doing it but I didnt get this far

Now this is where it gets intresting, this works fine in W:ET.
I did a quick hack on an old map and I now have a level running in game using two 1024x1024 external lightmaps.

Its been a few years since I found a new way to break ET, always good fun.

I need to recompile the level with some high detailed settings. I should be able to up the light map detail quite a lot, but I dont think it will get to the point where it will work for a full size level or to the detail that your after. The level I am using is no more than 512 units from origin on all axis (tiny).

If I get time over the weekend I might see if I can take this further, just for fun.


(obsidian) #11

Technically, -export (I’m not sure about -external) exports the lightmaps out of the BSP. q3map_lightmapSize changes the dimensions of the lightmap to a different size, consequently exporting them out since internal ones are limited to 128x128.

Detoeni: I’m interested in seeing your results.


(Detoeni) #12

My test level was original made for a little mapping contest held about five years ago on these forums. The rules on size were quite limiting, so to spice up my offering, I went a little mad with the lightmap detail. The level was completed with 512x512 lightmaps, with 75% brush work using a lightmapscale of 0.0625 and the rest 0.125.
In W:ET 512x512 lightmaps where normally only used for terrain while the rest of the brushwork would use 256x256.
All I did for this test was change the compile options to export to a range of lightmap sizes while upping the lightmap scale.

256x256 lightmaps – 22 textures – 4.2MB (100% @ 1.125)
512x512 lightmaps – 12 textures – 9MB (75% @ 0.0625)
1024x1024 lightmaps – 6 textures – 18MB (75% @ 0.0625)

A lightmapscale of 0.0625 seems to be the lowest value that will work.
The difference in visual quality between 512 and 1024 is very minimal. So slight that it was not worth doing screen grabs for both.
There might be some worth of using 1024 lightmaps when bump mapping*, so larger areas can be covered in more detail, also terrains should look good at this size, but other than that, its just a bit excessive.

256x256 – lightmap scale 1

256x256 – lightmap scale 0.125

1024x1024 – lightmap scale 0.0625

*Q3 style, not next gen.
Note: not sure why I’m getting odd, strong sky lights on each axis, you can see the projection from the windows behind the camera, between the two arches. This happens on all four sides of the level and with both old style sky shaders and newer q3map2 sky.

Compile options:
BSP = -meta -mv 1024 -mi 6144
VIS = -saveprt
LIGHT = -fast -samples 2 -gamma 2.2 -compensate 2 -dirty -bounce 8 -external -lightmapsize 1024
q3map_2.5.16_win32_x86


(brendar7639) #13

Hi !
I’ve just visited this forum. Happy to get acquainted with you. Thanks.