optimizing pk3 file size: use of lightgrid brush/ caulk etc.

(]UBC[ McNite) #1

From the thread on map expectations:

the final pk3 must be under 10 Mb. 3-4 is perfect.

Staying under 10 mb is possible if u don’t do complex brushwork or dont use lots of badly compressed custom texes/ tga. A map like stalingrad with 30 mb is major bug by itself tbh.
However 3-4 mb is ridiculous. Only way to do that is: no custom textures, no custom models, and no custom voice objectives. Those 3 eat the most part of a pk3. This would result in: same looks as we already know it (eg. El Kef looks like another part of the goldrush town… not really what u d call originality). And especially you ll either need objectives that are pretty much the same as in the original maps so u can use those VO, or don’t put VO into the map. The first is boring, the second is very likely to result in ppl not knowing what to do (cuz there are barely players that ever look at he CM or have a look at the limbo-menue objectives list where there are even objective cams).

So what i m wondering about right now is:

1.) Can I actually reduce the filesize by sophisticately using the lightgrid brush? Does anybody have any experiences about this?

2.) Apart from caring about a clean list of textures and as few tga’s as possible etc., what else can u do to reduce filesize when using custom textures?

3.) Does it help to caulk a lot and only have the actually visible surfaces textured? I know this is good for tris etc., but i m wondering about its impact on the bsp-size. It surely helps reducing hte .map-size (because common/caulk is less than for example theriver2reduxtextures/myfirstwalltextureever (and no, i dont use paths like that) :smiley:

(kamikazee) #2

Reducing portals can help. Caulking unseen things is not bad neither, saves at least 2 tri’s per face.

But maybe vis optimalization can help too.

(FireFly) #3

  1. well, I’m not sure about this,but I believe using a lightgrid brush will actually reduce the total filesize. What I do know for a fact is that using a lightgrid brush in your map will shorten the time it takes to compile the light-stage of the map.

  2. I convert all my tga’s into jpg’s (unless the tga contains an alpha mask). I know that many would disagree with me, but ingame I hardly notice any difference in quality using a jpg or a tga.

Currently I have 70 custom made textures in my map with a total filesize of 9MB ( uncompressed). I know that is a lot, but for me it’s the only way to make an original looking map without people saying:“oh look, another goldrush/oasis map”. and besides, creating custom textures is fun to do :smiley:

The thing that has the most impact on the final bsp size is brushwork: Having complex brushwork or small (1 or 2 units thick) brushwork will definitely increase the filesize. So I always tend to keep the brushwork as simple as possible and let custom textures do the rest.

Like you said, using custom sounds can easily increase the pk3 with a couple of mb’s. I don’t use much custom made VO’s in my map (just 2) because most of the time you can get away with the stock ones ( ‘‘command post constructed’’, ‘‘dynamite planted’’ and ‘‘objective taken’’). When I play a new map , during warm up time, I always check out the limbo menu to see what the objectives are and cycle through the obj cams. I also notice that some mappers did not include those in their map. For me it’s like playing a unfinished map, made in a hurry.

(kamikazee) #4

Custom VO’s are not really necessary, but if you add one you need to make ones for the other objectives too to avoid several different voices speaking. All these sounds add up…

Offtopic: I must be a minority where it comes to playing new maps: I play those on a local server before I play them online just to find objectives and other stuff…

(SCDS_reyalP) #5

http://www.splashdamage.com/index.php?name=pnPHPbb2&file=viewtopic&t=11854

(Loffy) #6

Using my quote here like that makes me feel sad.
File size and mapping was also discussed here:
http://www.splashdamage.com/index.php?name=pnPHPbb2&file=viewtopic&t=5070&highlight=bsp+size

(Detoeni) #7

Umm, Iv not been able to get the lightgrid.shader to work as I would expect in the last few q3map2’s, I still get lightgrid where there is no shader.

caulk faces take up a lot of space in a bsp, I did a test in AA, and by replacing the cave brushwork with an ase model and then simply clipping the map cut 2Mb of the BSP size, it also knocked up the compile time by 90mins so I went back to the brushwork version very quickly.

(.Chris.) #8

some simple signs telling you where to go could make up for any lack of voice overs and wont have such a dramatic impact on pk3 size as custom voice overs but it is a shame not many know objectives can be found in limbo with nice camera view, either that or just too lazy to look.

(SCDS_reyalP) #9

lightgrid seems to wrap around or something. if you go out of the defined range.

caulk faces take up a lot of space in a bsp, I did a test in AA, and by replacing the cave brushwork with an ase model and then simply clipping the map cut 2Mb of the BSP size, it also knocked up the compile time by 90mins so I went back to the brushwork version very quickly.

This seems a bit odd. Caulked faces shouldn’t be any more expensive than any other faces. Was the clipping using the same brushwork, or simpler ?

and what is AA ?

BTW, reading and understanding this http://graphics.stanford.edu/~kekoa/q3/ has some good information for people interested in optimizing their .bsp

(Detoeni) #10

AA,oops, sorry, I got fed up typing Alpine Assault in my shaders long ago, I’v got into the bad habit of abriviating it where ever possable.

lightgrid seems to wrap around or something. if you go out of the defined range.

Ahha! it never used to do that, nice.

Caulked faces are not any more expensive than any other faces, but its even better that there no there at all.
Clipping was very much more simple, I saved very close to 10000 brushes (40000 surfaces less in the bsp)

(carnage) #11

Caulked faces are not any more expensive than any other faces, but its even better that there no there at all.
Clipping was very much more simple, I saved very close to 10000 brushes (40000 surfaces less in the bsp)

umm will no draw faces be disregarded in the compile so not contribute to the .bsp? i’m sure in sock tutorials of something he used nodraw for terrian or rackwall of something… basicaly any detail brush use nodraw instead of caulk

and um, sorta a question. is it posible to change lightmaps to .jpg? all thoes .tga make up a few MB alone

The thing that has the most impact on the final bsp size is brushwork: Having complex brushwork or small (1 or 2 units thick) brushwork will definitely increase the filesize. So I always tend to keep the brushwork as simple as possible and let custom textures do the rest

u sure about this? i think any grid brushwork is pretty efficent but non grid vertices take more to store. so misc_models when converted to same thing as normal map stucture prob contribute to many of these

(SCDS_reyalP) #12

Non-drawing faces are discarded regardless of whether they are caulk or nodraw or whatever. the ‘common/nodraw’ shader is also non-solid, which should give better runtime performance, although from a quick look, it seems these brushes still end up in the .bsp file. So nodraw shouldn’t save you any size over detail caulk, but it may give better performance

(hrm, this brings up an interesting possible optimization for q3map2…)

If you look at the link I posted above, you will find that drawing stuff is stored as triangles, while brushes are still stored for collision detection.

Ahh, that makes much more sense.

(carnage) #13

i new the eninge draws as faces but i had no idea volume shapes were stored for collision

bit of topic but then something to consider. when you map is done filer the stuctureal brushes out then find/replace caulk with nodraw

this should get better performace? does anyone know if this would cause any side effects like sparklines for eg

(SCDS_reyalP) #14

In general, you want a large portion of your detail to be solid (terrain is one good example). Where you have detail that is exactly overlapped with structure, there might be some gain from using nodraw instead of caulk.

Also, unless you are using my custom q3map2, the actual solidness of brushes with mixed solid and non-solid shaders is effectively random. Putting nodraw on 5 of six sides just gives you 5:1 odds that it will be non-solid in any given compile.

(]UBC[ McNite) #15

In general, you want a large portion of your detail to be solid

Well… i want almost all my detail to be solid, because everything that s not absolutely necessary for VIS-blocking or splitting leafs is detail in my map. So the idea with the nodraw doesn’t really work for me.

About the user collision calculation… if I m right u can improve performance by using clip a lot, cutting areas off where the player will never be able to go anyways (or where u dont want him too).

so misc_models when converted to same thing as normal map stucture prob contribute to many of these

I tried that out with Detoenis Opelblitz which I used in my map. A complete Opelblitz (4 wheels, base, case, but no cover) gives u about 2.800 tris, and after a compile (as misc_model about 130 kb size in the .bsp. I don’t think that s really much. The real problem of custom models are the tga’s.

(SCDS_reyalP) #16

[quote=]UBC[ McNite]

In general, you want a large portion of your detail to be solid

Well… i want almost all my detail to be solid, because everything that s not absolutely necessary for VIS-blocking or splitting leafs is detail in my map. So the idea with the nodraw doesn’t really work for me.
[/quote]
I did some testing, and the difference doesn’t seem to be big in any case. Even my pathological test case (16k solid or nonsolid brushes in a single leaf) had a less than 10% difference. I’ll try to do a more detailed write up at some point, but it doesn’t look to be significant for real world cases.

About the user collision calculation… if I m right u can improve performance by using clip a lot, cutting areas off where the player will never be able to go anyways (or where u dont want him too).

It turns out that is not the case. Looking at the recently released engine code, Quake3 traces (the system used for collision detection) just do a linear search of the brushes in a given leaf. One of my test maps involved 16K nonsolid nodraw brushes completely embedded in the floor of a box map. This brings my fps down to about 180-200 on an athlon 64 @2.1ghz

Note that the above means that open space with complex brushwork should still be broken up into multiple leafs. A large number of brushes in a single leaf, even if they contribute 0 to r_speeds can still bring your CPU to its knees.

It is also worth noting that a brush which exists in multiple leafs costs more (both in terms of .bsp size and performance) than one that lives in a single leaf, although not nearly as much as 2 unique brushes.

[quote]so misc_models when converted to same thing as normal map stucture prob contribute to many of these

I tried that out with Detoenis Opelblitz which I used in my map. A complete Opelblitz (4 wheels, base, case, but no cover) gives u about 2.800 tris, and after a compile (as misc_model about 130 kb size in the .bsp. I don’t think that s really much. The real problem of custom models are the tga’s.[/quote]
All else being equal, misc_models are more efficient than equivalent brushwork (both in size and runtime performance), because the misc_model doesn’t cause any brushes to be stored in the .bsp

Of course, if you then go and put clip brushes around the model, things aren’t equal any more.

For example, in my test map, I made a 16k brush checkerboard, embedded in a caulk floor. The resulting .bsp file was about 4 megs. Converting the checkerboard to ASE, and replacing the brushes with a misc_model brings the .bsp size down to about 3 megs. It also brought the FPS up a noticeable amount (10-20%).

I have in mind a modification to q3map2 that will let you get the same result with brushwork, without going through ASE :moo:

edit:
I think it was mentioned before, but you can use q3map2 -game et -info <filename> to see what parts of the .bsp file are taking up the most space.

(carnage) #17

I have in mind a modification to q3map2 that will let you get the same result with brushwork, without going through ASE

It also brought the FPS up a noticable amount (10-20%).

a compiler that could do that would sertianly be a great tool, the more fps we can get out of a map the futher we can push the maps even for slower systems

if you are going to make the tool would you start another thread on its development so we can keep up to date

(]UBC[ McNite) #18

because the misc_model doesn’t cause any burshes to be stored in the .bsp

I thought misc_models get baked into the bsp (at least that was the answer of some of the pro’s in here in a thread i started last year)… and that s why i don’t need the md3-file in the pk3.
And, uhm, yea… the test I did was including clip_metal and clip_wood of course because leaving the clips in the map would ve been weird to say the least.

(SCDS_reyalP) #19

I thought misc_models get baked into the bsp

Yes they do, but they do not add brushes (and associated resurces like brushsides and leafbrushes). So (to grossly simplify things) if you build something out of brushes, that puts textured triangles and brushes into the .bsp. If you build it out of a misc_model, it just puts in the triangles

again: http://graphics.stanford.edu/~kekoa/q3/

carnage:
That FPS gain was a rather pathological case, so don’t get your hopes up too much. :banana:

(carnage) #20

That FPS gain was a rather pathological case, so don’t get your hopes up too much.

is their a way to simulate this tool so we can test the performance on actual maps. converting to ASE right?