Q3map2, Clustering, and you

(VolumetricSteve) #1

I’m working on cranking out a new map these days, but it’s already so complex that my new 6-core system is taking more than a day just to compute the VIS portion of the compile.

I was wondering if there were any easy (hah) ways to get q3map2 to cluster, so, say if I happened to have another 6-core system, would anyone know where to point me towards getting q3map2 to send tasks to all 12 of those cores?

What OS do I need? What software do I need? ect ect

I’m going to have to recompile q3map2, aren’t I? ugh

Thanks for your time

(Nail) #2

not exactly what you’re looking for, but may give you an idea how it can be done

http://helmer.sfe.se/

(VolumetricSteve) #3

that’s really cool, but I think that machine is possible due to :

rendering farms being one of the most popular applications of clusters, so there are a lot of tools specifically for that purpose.

and the rendering software being keyed into the message passing interface of the job distribution software he’s got going there.

Sadly, q3map2 doesn’t have these advantages

I’m starting to think it’d be easier to find some code out there that could be compiled into q3map2 that’d make it naturally cluster (way more effort than I care to put into this)

I’m surprised it doesn’t already have a cluster server/client considering what computers were like when quake 3 was in development (when they used q3map)

SO, i’m thinking I’m just going to ditch the cluster idea and try to port q3map2 to cuda, and then stuff a computer full of nvidia gpus

I saw somewhere that someone else was trying to do exactly this, but they’ve fallen on hard times (going back to school, deep in debt, living in their car, etc) and they didn’t post any of their code up…so I might have to start from scratch…I don’t know, using x86 cores for q3map2’s compiling is awful. I’m surprised no one’s ported it yet.

(Nail) #4

earlier discussion

http://www.splashdamage.com/forums/showthread.php?t=13742

(obsidian) #5

I can’t help but feel that perhaps you are doing something wrong with your construction/optimization methods. No map should take over a day to compile VIS. Light under very high settings and resolutions, maybe, but certainly not VIS. Are you using detail brushes at all? To what extent?

(VolumetricSteve) #6

Nail - I saw that thread, and it seemed to me the resolution was that he simply couldn’t do what he was trying to do with a cluster.

Obsidian - You’re more than likely right, I’m sure I have some detail brushes in there, but I haven’t looked at that part of it in a good long while (I’ve been working for 6 weeks)

Since I’m over 35,000 brushes in this map, I’m going to set them all to detail, and then go through the 1,000 or so that actually need to be structural and set those by hand.

My first concern is about walls, if you note the construction I used here:

http://www.map-factory.org/quake-3/arena/razztazzmagoria-1907

I think I made it so any vertical standing wall persisted down towards the hull of the map, through the floor tiles. This made it so that the floor tiles were all just detail brushes, inconsequential to the vis process. I tried to do the same thing with razztazzmagoria 2.

(fun note, because the power system where I live is trash, even though I had my computer on an APC battery backup, the power went out long enough to drain it all, and all vis progress was lost after 4 days of compiling; but at least now I can see what it was spending so much time doing)