Messed up letters & Segfaults;-
glibc 2.16:
This thread ==> Post #6
Pre-made stuff: http://dl.dropbox.com/u/45656686/glibc--2-16_ETQW--1-5.tar.gz
(old) glibc 2.15:
Patch: http://dl.dropbox.com/u/45656686/glibc-2.15-remove-wcslen-sse2.patch
Pre-made stuff: http://dl.dropbox.com/u/45656686/glibc--2-15_ETQW--1-5.tar.gz
The patch… “Courtesy of: JinxterX @ Arch Linux” - can you put that in a brighter font on your page? I like seeing my name in lights, lol 
Just kidding 
Here’s the bug report for those interested. Tried replicating it myself but couldn’t even get glibc 2.15 to build.
[QUOTE=JinxterX;396578]The patch… “Courtesy of: JinxterX @ Arch Linux” - can you put that in a brighter font on your page? I like seeing my name in lights, lol
Just kidding [/QUOTE]
Haha, well it kind of is in lights now, made a little change to point people in a more appropriate direction
Ah, I threw that up on the page just now!
p.s - Keep trying!
Well, it’s a compiler bug in the old version of GCC used to build ETQW for Linux which causes some localisation strings to be placed on odd (rather than even) memory addresses, thus screwing up the SSE2 optimised wide character string length function in glibc 2.15.
If you use Archlinux the easiest thing to do is build it from ABS after modfying the PKGBUILD to include:
${srcdir}/glibc/configure --prefix=/usr \
--libdir=/usr/lib --libexecdir=/usr/lib \
--with-headers=/usr/include \
--enable-add-ons=nptl,libidn \
--enable-kernel=2.6.32 \
--with-tls --with-__thread \
--enable-bind-now --without-gd \
--without-cvs --disable-profile \
[b]--disable-multi-arch[/b]
and then copy the new libc.so.6 into /opt/etqw.
I also rename libgcc_s.so.1 and libstdc++.so.6 in the game dir so only the latest (distro installed) versions are used, which stops the segfault and may help with some files not being “touched” for Pro mod maps.
The fault is not really with glibc but rather ETQW being so old. 
For Archlinux users and glibc 2.16, sync with ABS and edit the PKGBUILD :
#check() {
# cd ${srcdir}/glibc-build
# make -k check
#}
If you don’t skip these checks the compile will probably fail and you waste a lot of time.
${srcdir}/${pkgname}-${pkgver}/configure --prefix=/usr \
--libdir=/usr/lib --libexecdir=/usr/lib \
--with-headers=/usr/include \
--enable-add-ons=nptl,libidn \
--enable-obsolete-rpc \
--enable-kernel=2.6.32 \
--enable-bind-now --disable-profile \
--enable-stackguard-randomization \
[b]--disable-multi-arch[/b]
Disable the SSE optimizations to fix the string problem.
Run makepkg.
Then copy libc.so.6 and libm.so.6 from the pkg/lib into /opt/etqw to avoid:
etqw.x86: relocation error: /lib/libm.so.6: symbol __get_cpu_features, version GLIBC_PRIVATE not defined in file libc.so.6 with link time reference
Briefly glancing through this thread this seems like a bug in the current glibc too. If the current code tries to do an optimisation (SIMD code) that only works on memory buffers that are aligned in a certain way, it only should run that path when the memory is actually aligned.
Interestingly enough it hit Sublime Text too: http://www.sublimetext.com/forum/viewtopic.php?f=2&t=4473 . It is probably worth reopening that bug report. The risk is though that checking if a buffer is aligned or not will undo any performance gains from the optimisation.
Thanks for that Jinx!
I noticed the 2.16 update the other day and thought of etqw but I haven’t played the game in a while and couldn’t be bothered getting the disc, luckily RR2DO2’s posts caught my eye on the sidebar and got my arse in to gear, I needed to update my standalone game anyway so it’s all good.
Updated first post, site, etc.
Also, I checked after building whilst I was sorting stuff about commenting out the checks - ****ing hell do they make it take a long time, and it failed after about 30 minutes, so thanks for that.
The bug report with Archlinux cannot be reopened because the fault lies within ETQW and not glibc, the Sublime Text dev post confirms that (thanks for the link, good stuff) it’s up to the developer to fix these kind of problems… “Long story short, my wchar_t strings are now all properly aligned, and everyone’s happy again. The moral of the story is, of course, just use UTF-8 everywhere”
It’s conceivable glibc could be patched to work around the issue but I doubt that will happen because the cases are so rare and I can imagine glibc devs saying “it’s your problem not ours”, lol.
[QUOTE=Psykorgasm;402772]
Also, I checked after building whilst I was sorting stuff about commenting out the checks - ****ing hell do they make it take a long time, and it failed after about 30 minutes, so thanks for that.[/QUOTE]
Yep, those checks are a total waste of time in this case and they introduced even more in 2.16. My compile always fails on tst-cpuclock1, tst-cpuclock2, tst-cputimer1-3 etc which seems to be a known issue from 2.15 supposedly fixed in 2.16, but obviously not
Incorrect, the wcslen specification does not have any alignment requirements. Therefor the Archlinux/glibc implementation is broken. In the Sublime Text case they took note of this broken implementation and worked around it.
POSIX specification: http://pubs.opengroup.org/onlinepubs/9699919799/functions/wcslen.html
Now there is a slight grey area here. The malloc spec says “The pointer returned if the allocation succeeds shall be suitably aligned so that it may be assigned to a pointer to any type of object and then used to access such an object in the space allocated (until the space is explicitly freed or reallocated).” and in theory it could be argued that a ‘string object’ pointed to by a wchar pointer is ‘a type of object’ and thus your string objects could be expected to follow malloc defined alignment. However, this is not what the wcslen specification says and not what the wchar_t specification says; they are not required to have natural, or factor of two, alignment.
On the other side (and you will find various best guides here and there saying this) for performance reasons and avoidance of issues like this it’d be good to be naturally aligned. ETQW doesn’t enforce this on it’s wide strings. Still, the fact that it crashes on an unaligned string is a bug as it is behaviour against the standard.
It does appear as if the Linux kernel is written with a warning that you should be naturally aligned, or to expect exceptions on certain systems. If this also applies to glibc on Archlinux they need to document this, as well as make an announcement to developers when they implement a breaking change like this.
The normal wcslen function in glibc worked fine up until 2.15, the problem now is the automatic SSE2 optimisation of wcslen kicks in if those features are detected on the CPU. Due to the nature of the routine (written in assembler) and the size of the registers to be loaded, if a string is misaligned (i.e. at a odd memory address) then the incorrect length is returned, however, the routine is functioning 100% correctly, the fault lies with the original binary. Pretty sure there’s compiler flags to ensure alignment in C++ based programs (which ETQW is).
Take a look at wcslen-sse2.S in the glibc sources if you want, as this is the routine at the heart of the matter and it was added by a libc devs in 2.15, it’s nothing to do with Archlinux. Short of bypassing this routine altogether (by disabling multi-arch entirely or introducing a patch) you have to fix any alignment stuff at source, which I assume is what the Sublime Text people did and then recompiled.
The Linux version of ETQW however cannot be recompiled as development stopped a long time ago, therefore we have to work around the problem.
[QUOTE=JinxterX;402782]The normal wcslen function in glibc worked fine up until 2.15, the problem now is the automatic SSE2 optimisation of wcslen kicks in if those features are detected on the CPU. Due to the nature of the routine (written in assembler) and the size of the registers to be loaded, if a string is misaligned (i.e. at a odd memory address) then the incorrect length is returned, however, the routine is functioning 100% correctly, the fault lies with the original binary. Pretty sure there’s compiler flags to ensure alignment in C++ based programs (which ETQW is).
Take a look at wcslen-sse2.S in the glibc sources if you want, as this is the routine at the heart of the matter and it was added by a libc devs in 2.15, it’s nothing to do with Archlinux. Short of bypassing this routine altogether (by disabling multi-arch entirely or introducing a patch) you have to fix any alignment stuff at source, which I assume is what the Sublime Text people did and then recompiled.
The Linux version of ETQW however cannot be recompiled as development stopped a long time ago, therefore we have to work around the problem.[/QUOTE]
I disagree. Optimizations of this nature should only be enabled if 1) the hardware supports the features used and 2) the input data is valid for the optimization.
If only #2 were not true - life for cross platform devs would be just that bit easier 
Also - compilers cannot ensure alignment of dynamic data.
It’s true that wcslen in glibc doesn’t check for misaligned data before going into the SSE2 optimisation, I’m not sure why, maybe they just assume developers use memalign() to a 16 byte boundary in their code, or write their own small function to dynamically allocate aligned memory suitable for SSE2 registers? I know… these are crazy, wacky ideas huh?
NOOB question ALERT
is this what running games on linux involve? Just curious, dont kill me.
[QUOTE=Gradis;402790]NOOB question ALERT
is this what running games on linux involve? Just curious, dont kill me.[/QUOTE]
I wouldn’t worry about any bodily harm. They obviously don’t have the time for any of that.
Actually, I’m pretty sure they’re making most of this stuff up.
[QUOTE=rorgoroth;396261]Messed up letters & Segfaults;-
glibc 2.16:
This thread ==> Post #6
Pre-made stuff: http://dl.dropbox.com/u/45656686/glibc--2-16_ETQW--1-5.tar.gz
(old) glibc 2.15:
Patch: http://dl.dropbox.com/u/45656686/glibc-2.15-remove-wcslen-sse2.patch
Pre-made stuff: http://dl.dropbox.com/u/45656686/glibc--2-15_ETQW--1-5.tar.gz
[/QUOTE]
Sorry for necromancy. But glibc related crashes makes the game unbearable on Linux. Unfortunately, those Dropbox uploads are down at the moment. Can they be reuploaded please? Thank you.
Regards.