Post subject: Linux debugging options interfering with C++ exceptions?
Emulator Coder
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
A client of mine contacted me with a strange issue. Maybe some other programmer or Linux sysadmin is familiar with this bizarre behavior? The following is happening:
> ./run.sh
terminate called after throwing an instance of 'Error'
  what():  Unrecognized Parameters Used
run.sh: line 2: 25429 Aborted                 /usr/bin/app2 --confpath /etc/app2 "$@"
Two things here are strange. 1) The application is terminated as soon as an exception is thrown, the program isn't even allowed to catch its own errors. 2) The shell script running the application is noticing this and is giving verbose output as to what line it failed on, and the contents of the line. Anyone familiar with anything that would do this? Why are the exceptions not being caught? The exact same binary on a different system (both running RHEL5) runs normally, and the exception is handled internally. What is making script execution act verbose like this? I've never seen that before.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Warepire
He/Him
Editor
Joined: 3/2/2010
Posts: 2174
Location: A little to the left of nowhere (Sweden)
Unless there are any sensitive details in the bash-script, it could help greatly if it's contents was posted.
Emulator Coder
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Warepire wrote:
Unless there are any sensitive details in the bash-script, it could help greatly if it's contents was posted.
Download run.sh
Language: shell

#!/bin/sh /usr/bin/app2 --confpath /etc/app2 "$@"
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Warepire
He/Him
Editor
Joined: 3/2/2010
Posts: 2174
Location: A little to the left of nowhere (Sweden)
I was wrong :(
arflech
He/Him
Joined: 5/3/2008
Posts: 1120
What is the source code for app2?
i imgur com/QiCaaH8 png
Emulator Coder
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
arflech wrote:
What is the source code for app2?
I can't share it. But the entire main is wrapped in try/catch, rest assured that the exception is supposed to be handled internally, and on every other system, it is.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Patashu
He/Him
Joined: 10/2/2005
Posts: 4017
Have you tried running it on the same computer but from a different shell (e.g. csh instead of bash)?
My Chiptune music, made in Famitracker: http://soundcloud.com/patashu My twitch. I stream mostly shmups & rhythm games http://twitch.tv/patashu My youtube, again shmups and rhythm games and misc stuff: http://youtube.com/user/patashu
Emulator Coder
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Patashu wrote:
Have you tried running it on the same computer but from a different shell (e.g. csh instead of bash)?
No, and it's not my computer, it's a clients. The different shell may explain the changes to the shell script output which I should check, thanks. But the root problem here is why is the C++ exception not allowed to be caught? I'm starting to wonder if perhaps I should be static linking. http://www.trilithium.com/johan/2005/06/static-libstdc/
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
WST
She/Her
Active player (450)
Joined: 10/6/2011
Posts: 1690
Location: RU · ID · AM
Hmm try to compile it with -g gcc’s flag (and do not strip the executable) and run in gdb session. Then after crash you can make “bt full” and get traceback that may help you to debug the issue…
S3&A [Amy amy%] improvement (with Evil_3D & kaan55) — currently in SPZ2 my TAS channel · If I ever come into your dream, I’ll be riding an eggship :)
Emulator Coder
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
WST wrote:
Hmm try to compile it with -g gcc’s flag (and do not strip the executable) and run in gdb session. Then after crash you can make “bt full” and get traceback that may help you to debug the issue…
I can't use the client's machine. It's their machine.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
marzojr
He/Him
Experienced player (749)
Joined: 9/29/2008
Posts: 964
Location: 🇫🇷 France
Are the functions declared with "throw()" or "throw(stuff)"? If the latter, can the type of the exception being thrown be converted to any of the types specified in 'stuff'? Also, what type of exception is the program being compiled with, sjlj or dw3?
Marzo Junior
Emulator Coder
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
marzojr wrote:
Are the functions declared with "throw()" or "throw(stuff)"?
Neither.
marzojr wrote:
Also, what type of exception is the program being compiled with, sjlj or dw3?
Whatever the default is on RHEL5.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Banned User, Former player
Joined: 3/10/2004
Posts: 7698
Location: Finland
You are not giving us much to work with. However, this:
Nach wrote:
I'm starting to wonder if perhaps I should be static linking.
makes me suspect that the exception is being thrown in a dynamically linked library. Is that correct? Dynamically linked C++ libraries are often prone to not work as expected because the dynamic linker does not necessarily work properly with C++ code. (For example if there are static objects in the library, they might not get properly constructed.)
Emulator Coder
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Warp wrote:
Nach wrote:
I'm starting to wonder if perhaps I should be static linking.
makes me suspect that the exception is being thrown in a dynamically linked library. Is that correct?
They're being thrown from main(), and the entire body of main() is wrapped in try and catch. Edit, here's the output from ldd on it from my machine:
        linux-gate.so.1 =>  (0xf7775000)
        libdl.so.2 => /lib32/libdl.so.2 (0xf7735000)
        libpthread.so.0 => /lib32/libpthread.so.0 (0xf771c000)
        librt.so.1 => /lib32/librt.so.1 (0xf7713000)
        libstdc++.so.6 => /usr/lib32/libstdc++.so.6 (0xf762b000)
        libm.so.6 => /lib32/libm.so.6 (0xf7605000)
        libgcc_s.so.1 => /usr/lib32/libgcc_s.so.1 (0xf75e8000)
        libc.so.6 => /lib32/libc.so.6 (0xf748b000)
        /lib/ld-linux.so.2 (0xf7776000)
The binary does not use dlopen().
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
marzojr
He/Him
Experienced player (749)
Joined: 9/29/2008
Posts: 964
Location: 🇫🇷 France
If you are sending a binary to your client (instead of code to compile), and it works on other systems, then it is likely that it is a difference in the exception handling style between your machine and the clients (I should have paid slightly more attention to what you said). You can try running this:
gcc -dumpspecs | grep -o --color 'dwarf[1-4]'
and ask the client to do the same; if nothing is printed, then the exception style is sjlj. If there is a mismatch, this may cause the problem you describe if the exception is being thrown across one of the shared libraries (e.g., your code calls library code which calls your code which throws the exception).
Marzo Junior
Emulator Coder
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
marzojr wrote:
If you are sending a binary to your client (instead of code to compile), and it works on other systems, then it is likely that it is a difference in the exception handling style between your machine and the clients (I should have paid slightly more attention to what you said). You can try running this:
gcc -dumpspecs | grep -o --color 'dwarf[1-4]'
and ask the client to do the same; if nothing is printed, then the exception style is sjlj. If there is a mismatch, this may cause the problem you describe if the exception is being thrown across one of the shared libraries (e.g., your code calls library code which calls your code which throws the exception).
Thanks, I will do that. The only shared library which may throw an exception is libstdc++, do you think that may be the problem? Although the code is all self contained in main(), perhaps some object constructor is being passed through libstdc++ (or something similar) and causing the issue? The actual "throw" that is doing the throwning is found in main(). Edit Would static linking libstdc++ and libgcc fix these problems?
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
marzojr
He/Him
Experienced player (749)
Joined: 9/29/2008
Posts: 964
Location: 🇫🇷 France
Static linking libstdc++ and libgcc might fix the problem, but I would not recommend it. And yes, passing a constructor that throws through libstdc++ might cause the problem. Out of curiosity: would you be on Linux and cross compiling to Windows?
Marzo Junior
Emulator Coder
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
marzojr wrote:
And yes, passing a constructor that throws through libstdc++ might cause the problem.
The constructor doesn't throw. throw(Error("text")); is what is being done. Error inherits from std::exception.
marzojr wrote:
Out of curiosity: would you be on Linux and cross compiling to Windows?
Windows is not involved at all here. Things are being compiled on RHEL5 and running on RHEL5.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Banned User, Former player
Joined: 3/10/2004
Posts: 7698
Location: Finland
marzojr wrote:
Static linking libstdc++ and libgcc might fix the problem, but I would not recommend it.
(Not to talk that it would break the LGPL license unless the program in question is also (L)GPL licensed.., but that's a different issue altogether.)
Emulator Coder
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Warp wrote:
marzojr wrote:
Static linking libstdc++ and libgcc might fix the problem, but I would not recommend it.
(Not to talk that it would break the LGPL license unless the program in question is also (L)GPL licensed.., but that's a different issue altogether.)
You need to read this: http://gcc.gnu.org/ml/gcc-help/2003-07/msg00178.html
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Banned User, Former player
Joined: 3/10/2004
Posts: 7698
Location: Finland
Nach wrote:
You need to read this: http://gcc.gnu.org/ml/gcc-help/2003-07/msg00178.html
The GNU libc library is definitely LGPL (well, at least according to wikipedia; I can't for the life of me find this info on the FSF's own website), so it's easy to assume that the same holds for libstdc++. Finding the actual license for libstdc++ and libgcc seems to be particularly hard (I don't really understand why, given how adamant the FSF is about licenses). I think this quote I found somewhere in the entrails of gcc.gnu.org is rather descriptive: "Hopefully that text is self-explanatory. If it isn't, you need to speak to your lawyer, or the Free Software Foundation."
Emulator Coder
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Warp wrote:
Nach wrote:
You need to read this: http://gcc.gnu.org/ml/gcc-help/2003-07/msg00178.html
The GNU libc library is definitely LGPL (well, at least according to wikipedia; I can't for the life of me find this info on the FSF's own website), so it's easy to assume that the same holds for libstdc++.
glibc isn't being discussed here, and is unrelated to GCC, libgcc, or libstdc++. Real programmers don't assume things. http://gcc.gnu.org/onlinedocs/libstdc++/manual/bk01pt01ch01s02.html
Warp wrote:
Finding the actual license for libstdc++ and libgcc seems to be particularly hard (I don't really understand why, given how adamant the FSF is about licenses).
See the above link, and http://www.calculate-linux.org/packages/licenses/libgcc as well as http://www.gnu.org/licenses/gcc-exception.html They're GPL with a linking exception. They are not LGPL. Edit: libstdc++: http://gcc.gnu.org/onlinedocs/libstdc++/manual/license.html
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
marzojr
He/Him
Experienced player (749)
Joined: 9/29/2008
Posts: 964
Location: 🇫🇷 France
Nach wrote:
marzojr wrote:
Windows is not involved at all here. Things are being compiled on RHEL5 and running on RHEL5.
Basically the same OS. Again, having paid more attention to what you wrote would have told me that. This rules out different exception models, so this would be a false lead. Sorry for that. You said it is not a problem with functions declared with "throw(...)", and that the exception type in question is being caught in the catch block; so this leaves the constructor for the exception itself and the destructor of any objects that gets called when the exception is thrown. A difference in security policies could be responsible for the differences between the machines.
Marzo Junior
Emulator Coder
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
marzojr wrote:
Nach wrote:
marzojr wrote:
Windows is not involved at all here. Things are being compiled on RHEL5 and running on RHEL5.
Basically the same OS. Again, having paid more attention to what you wrote would have told me that. This rules out different exception models, so this would be a false lead. Sorry for that.
You said it is not a problem with functions declared with "throw(...)", and that the exception type in question is being caught in the catch block;
Um, I'm not sure where I said that. The problem is that any throw() in the entire program is not caught at all despite there being a try/catch for it. However every single throw is in the main code and not in any libraries, except for those built into libstdc++ for bad_alloc or at() range and so on. Not a single function anywhere in the entire application mentions the throw style in its declaration. In fact, if I'm not mistaken, doing so is deprecated.
marzojr wrote:
so this leaves the constructor for the exception itself and the destructor of any objects that gets called when the exception is thrown. A difference in security policies could be responsible for the differences between the machines.
Okay, and what does a developer do about it?
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Banned User, Former player
Joined: 3/10/2004
Posts: 7698
Location: Finland
Nach wrote:
glibc isn't being discussed here, and is unrelated to GCC, libgcc, or libstdc++.
I was explaining why I made the assumption.