Python threading sucks

Recently, when testing the installer, we noticed a weird issue. When ’emerge sync’ is run, rsync would complete the sync but not exit, which caused the install to hang. It took 2 days and a lot of frustration to finally track this one down. We owe the answer to d_m (on freenode).

The installer runs with 2 separate threads. The frontend is the main thread and the backend starts up a second thread to run the commands that perform the install. It turns out that when python creates a new thread, it blocks all signals for that thread. I’m not quite sure why. This has the side effect of blocking all signals for any processes that are fork()ed and/or exec()ed from that child thread.

Rsync fork()s when it’s doing it’s thing. One process waits for a USR2 signal from the other before they mutually exit. Since python had blocked all signals, the signal wasn’t getting from one process to another so they could exit. d_m’s solution was to write a small C program that resets the signal mask when running external programs. While this works, it is a huge hack. Although, this bug should be fixed in python 2.4 :-/

4 thoughts on “Python threading sucks”

  1. I have a similar program in a chat program I created in python. I am using the medium level “threading” module to create server connection accepting threads but I can’t exit them when they finish their thing, they stack up and the receiving chat program hangs. Sometimes after just one message! (I know, not the cleanest code ever).

    Do you think you could email that C program/source to me? Or at least some other info on exiting threads? My address is:


  2. Signal handling on windows .. its a big joke .. SIGALRM ..huh! i still remember my experience with it. Somewhere singals are not platform independent .. on windows use thread + win32api and for Posix use thread + std. signals .. thats how i resolved my problem.

Comments are closed.