Guido van Rossum | 48a69b7 | 1994-05-16 09:35:22 +0000 | [diff] [blame] | 1 | # The following self-contained little program usually freezes with most |
| 2 | # threads reporting |
Tim Peters | 172c40b | 2001-01-21 07:07:30 +0000 | [diff] [blame] | 3 | # |
Guido van Rossum | 48a69b7 | 1994-05-16 09:35:22 +0000 | [diff] [blame] | 4 | # Unhandled exception in thread: |
| 5 | # Traceback (innermost last): |
| 6 | # File "importbug.py", line 6 |
| 7 | # x = whrandom.randint(1,3) |
| 8 | # AttributeError: randint |
Tim Peters | 172c40b | 2001-01-21 07:07:30 +0000 | [diff] [blame] | 9 | # |
Guido van Rossum | 48a69b7 | 1994-05-16 09:35:22 +0000 | [diff] [blame] | 10 | # Here's the program; it doesn't use anything from the attached module: |
| 11 | |
| 12 | import thread |
| 13 | |
| 14 | def task(): |
| 15 | global N |
| 16 | import whrandom |
| 17 | x = whrandom.randint(1,3) |
| 18 | a.acquire() |
| 19 | N = N - 1 |
| 20 | if N == 0: done.release() |
| 21 | a.release() |
| 22 | |
| 23 | a = thread.allocate_lock() |
| 24 | done = thread.allocate_lock() |
| 25 | N = 10 |
| 26 | |
| 27 | done.acquire() |
| 28 | for i in range(N): |
| 29 | thread.start_new_thread(task, ()) |
| 30 | done.acquire() |
| 31 | print 'done' |
| 32 | |
| 33 | |
| 34 | # Sticking an acquire/release pair around the 'import' statement makes the |
| 35 | # problem go away. |
Tim Peters | 172c40b | 2001-01-21 07:07:30 +0000 | [diff] [blame] | 36 | # |
Guido van Rossum | 48a69b7 | 1994-05-16 09:35:22 +0000 | [diff] [blame] | 37 | # I believe that what happens is: |
Tim Peters | 172c40b | 2001-01-21 07:07:30 +0000 | [diff] [blame] | 38 | # |
Guido van Rossum | 48a69b7 | 1994-05-16 09:35:22 +0000 | [diff] [blame] | 39 | # 1) The first thread to hit the import atomically reaches, and executes |
| 40 | # most of, get_module. In particular, it finds Lib/whrandom.pyc, |
| 41 | # installs its name in sys.modules, and executes |
Tim Peters | 172c40b | 2001-01-21 07:07:30 +0000 | [diff] [blame] | 42 | # |
Guido van Rossum | 48a69b7 | 1994-05-16 09:35:22 +0000 | [diff] [blame] | 43 | # v = eval_code(co, d, d, d, (object *)NULL); |
Tim Peters | 172c40b | 2001-01-21 07:07:30 +0000 | [diff] [blame] | 44 | # |
Guido van Rossum | 48a69b7 | 1994-05-16 09:35:22 +0000 | [diff] [blame] | 45 | # to initialize the module. |
Tim Peters | 172c40b | 2001-01-21 07:07:30 +0000 | [diff] [blame] | 46 | # |
Guido van Rossum | 48a69b7 | 1994-05-16 09:35:22 +0000 | [diff] [blame] | 47 | # 2) eval_code "ticker"-slices the 1st thread out, and gives another thread |
| 48 | # a chance. When this 2nd thread hits the same 'import', import_module |
| 49 | # finds 'whrandom' in sys.modules, so just proceeds. |
Tim Peters | 172c40b | 2001-01-21 07:07:30 +0000 | [diff] [blame] | 50 | # |
Guido van Rossum | 48a69b7 | 1994-05-16 09:35:22 +0000 | [diff] [blame] | 51 | # 3) But the 1st thread is still "in the middle" of executing whrandom.pyc. |
| 52 | # So the 2nd thread has a good chance of trying to look up 'randint' |
| 53 | # before the 1st thread has placed it in whrandom's dict. |
Tim Peters | 172c40b | 2001-01-21 07:07:30 +0000 | [diff] [blame] | 54 | # |
Guido van Rossum | 48a69b7 | 1994-05-16 09:35:22 +0000 | [diff] [blame] | 55 | # 4) The more threads there are, the more likely that at least one of them |
| 56 | # will do this before the 1st thread finishes the import work. |
Tim Peters | 172c40b | 2001-01-21 07:07:30 +0000 | [diff] [blame] | 57 | # |
Guido van Rossum | 48a69b7 | 1994-05-16 09:35:22 +0000 | [diff] [blame] | 58 | # If that's right, a perhaps not-too-bad workaround would be to introduce a |
| 59 | # static "you can't interrupt this thread" flag in ceval.c, check it before |
| 60 | # giving up interpreter_lock, and have IMPORT_NAME set it & restore (plain |
| 61 | # clearing would not work) it around its call to import_module. To its |
| 62 | # credit, there's something wonderfully perverse about fixing a race via an |
| 63 | # unprotected static <grin>. |
Tim Peters | 172c40b | 2001-01-21 07:07:30 +0000 | [diff] [blame] | 64 | # |
Guido van Rossum | 48a69b7 | 1994-05-16 09:35:22 +0000 | [diff] [blame] | 65 | # as-with-most-other-things-(pseudo-)parallel-programming's-more-fun- |
| 66 | # in-python-too!-ly y'rs - tim |
Tim Peters | 172c40b | 2001-01-21 07:07:30 +0000 | [diff] [blame] | 67 | # |
Guido van Rossum | 48a69b7 | 1994-05-16 09:35:22 +0000 | [diff] [blame] | 68 | # Tim Peters tim@ksr.com |
| 69 | # not speaking for Kendall Square Research Corp |