首页 / 问题 / 正文

为什么在Java中嵌入CPython时会挂起？

2 投票
3 回答
877 浏览
提问于 2025-04-17 03:20

我正在使用Jepp把CPython嵌入到JVM中，但当我运行

import numpy; numpy.finfo(float)

时，程序就卡住了。用gdb调试时发现有东西在阻塞一个信号量/锁的获取，堆栈跟踪显示可能和浮点数有关：

$ LD_PRELOAD=/usr/lib/libpython2.6.so gdb --args java -jar jepp.jar -i /tmp/go.py
GNU gdb (GDB) 7.1-ubuntu
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/bin/java...(no debugging symbols found)...done.
(gdb) r
Starting program: /usr/bin/java -jar jepp.jar -i /tmp/go.py
[Thread debugging using libthread_db enabled]
process 20736 is executing new program: /usr/lib/jvm/java-6-sun-1.6.0.26/jre/bin/java
[Thread debugging using libthread_db enabled]
[New Thread 0x7ffff58fd700 (LWP 20739)]
[New Thread 0x7ffff16ca700 (LWP 20740)]
[New Thread 0x7ffff15c9700 (LWP 20741)]
[New Thread 0x7ffff14c8700 (LWP 20742)]
[New Thread 0x7ffff13c7700 (LWP 20743)]
[New Thread 0x7ffff0bf5700 (LWP 20744)]
[New Thread 0x7ffff0af4700 (LWP 20745)]
[New Thread 0x7ffff09f3700 (LWP 20746)]
[New Thread 0x7ffff08b3700 (LWP 20747)]
[New Thread 0x7ffff07b2700 (LWP 20748)]
[New Thread 0x7ffff06b1700 (LWP 20749)]
[New Thread 0x7ffff05b0700 (LWP 20750)]
[New Thread 0x7ffff04af700 (LWP 20751)]
>>> import numpy; numpy.finfo(float)
^C
Program received signal SIGINT, Interrupt.
0x00007ffff771703d in pthread_join () from /lib/libpthread.so.0
(gdb) bt
#0  0x00007ffff771703d in pthread_join () from /lib/libpthread.so.0
#1  0x000000004000591c in ContinueInNewThread ()
#2  0x0000000040001ef4 in main ()
(gdb) i threads
  14 Thread 0x7ffff04af700 (LWP 20751)  0x00007ffff771abc9 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  13 Thread 0x7ffff05b0700 (LWP 20750)  0x00007ffff771a85c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  12 Thread 0x7ffff06b1700 (LWP 20749)  0x00007ffff771a85c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  11 Thread 0x7ffff07b2700 (LWP 20748)  0x00007ffff771a85c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  10 Thread 0x7ffff08b3700 (LWP 20747)  0x00007ffff771cb50 in sem_wait ()
   from /lib/libpthread.so.0
  9 Thread 0x7ffff09f3700 (LWP 20746)  0x00007ffff771a85c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  8 Thread 0x7ffff0af4700 (LWP 20745)  0x00007ffff771a85c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  7 Thread 0x7ffff0bf5700 (LWP 20744)  0x00007ffff771abc9 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  6 Thread 0x7ffff13c7700 (LWP 20743)  0x00007ffff771a85c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  5 Thread 0x7ffff14c8700 (LWP 20742)  0x00007ffff771a85c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  4 Thread 0x7ffff15c9700 (LWP 20741)  0x00007ffff771a85c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  3 Thread 0x7ffff16ca700 (LWP 20740)  0x00007ffff771a85c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  2 Thread 0x7ffff58fd700 (LWP 20739)  0x00007ffff771cb50 in sem_wait ()
   from /lib/libpthread.so.0
* 1 Thread 0x7ffff7ed8700 (LWP 20736)  0x00007ffff771703d in pthread_join ()
   from /lib/libpthread.so.0
(gdb) thread 2
[Switching to thread 2 (Thread 0x7ffff58fd700 (LWP 20739))]#0  0x00007ffff771cb50 in sem_wait () from /lib/libpthread.so.0
(gdb) bt
#0  0x00007ffff771cb50 in sem_wait () from /lib/libpthread.so.0
#1  0x00007ffff7a4db28 in PyThread_acquire_lock ()
   from /usr/lib/libpython2.6.so
#2  0x00007ffff7a1cbb4 in PyEval_RestoreThread () from /usr/lib/libpython2.6.so
#3  0x00007ffff7a41008 in PyGILState_Ensure () from /usr/lib/libpython2.6.so
#4  0x00007fffeb1e1ff6 in _error_handler (method=-334489136, errobj=0x80,
    errtype=0x0, retstatus=-1, first=0x0)
    at numpy/core/src/umath/ufunc_object.c:95
#5  0x00007fffeb1e2379 in PyUFunc_handlefperr (errmask=521,
    errobj=0x7fffec502950, retstatus=4, first=0x7ffff58fa5fc)
    at numpy/core/src/umath/ufunc_object.c:189
#6  0x00007fffeb1e7cd1 in PyUFunc_GenericFunction (self=<value optimized out>,
    args=<value optimized out>, kwds=<value optimized out>, op=0x7ffff58faaa0)
    at numpy/core/src/umath/ufunc_object.c:2734
#7  0x00007fffeb1e7ff0 in ufunc_generic_call (self=0x7fffec1df790,
    args=<value optimized out>, kwds=<value optimized out>)
    at numpy/core/src/umath/ufunc_object.c:4180
#8  0x00007ffff797c2e3 in PyObject_Call () from /usr/lib/libpython2.6.so
#9  0x00007ffff797ed9f in _PyObject_CallFunction_SizeT ()
   from /usr/lib/libpython2.6.so
#10 0x00007ffff797c9aa in ?? () from /usr/lib/libpython2.6.so
#11 0x00007ffff797d698 in PyNumber_Multiply () from /usr/lib/libpython2.6.so
#12 0x00007ffff7a1f2bc in PyEval_EvalFrameEx () from /usr/lib/libpython2.6.so
#13 0x00007ffff7a22928 in PyEval_EvalFrameEx () from /usr/lib/libpython2.6.so
#14 0x00007ffff7a23d60 in PyEval_EvalCodeEx () from /usr/lib/libpython2.6.so
#15 0x00007ffff79a9d60 in ?? () from /usr/lib/libpython2.6.so
#16 0x00007ffff797c2e3 in PyObject_Call () from /usr/lib/libpython2.6.so
#17 0x00007ffff798dcef in ?? () from /usr/lib/libpython2.6.so
#18 0x00007ffff797c2e3 in PyObject_Call () from /usr/lib/libpython2.6.so
#19 0x00007ffff79e2cd0 in ?? () from /usr/lib/libpython2.6.so
#20 0x00007ffff79d8f98 in ?? () from /usr/lib/libpython2.6.so
#21 0x00007ffff797c2e3 in PyObject_Call () from /usr/lib/libpython2.6.so
#22 0x00007ffff7a21771 in PyEval_EvalFrameEx () from /usr/lib/libpython2.6.so
#23 0x00007ffff7a23d60 in PyEval_EvalCodeEx () from /usr/lib/libpython2.6.so
#24 0x00007ffff7a220ab in PyEval_EvalFrameEx () from /usr/lib/libpython2.6.so
#25 0x00007ffff7a23d60 in PyEval_EvalCodeEx () from /usr/lib/libpython2.6.so
#26 0x00007ffff79a9d60 in ?? () from /usr/lib/libpython2.6.so
#27 0x00007ffff797c2e3 in PyObject_Call () from /usr/lib/libpython2.6.so
#28 0x00007ffff79d928d in ?? () from /usr/lib/libpython2.6.so
#29 0x00007ffff79d8f53 in ?? () from /usr/lib/libpython2.6.so
#30 0x00007ffff797c2e3 in PyObject_Call () from /usr/lib/libpython2.6.so
#31 0x00007ffff7a21771 in PyEval_EvalFrameEx () from /usr/lib/libpython2.6.so
#32 0x00007ffff7a23d60 in PyEval_EvalCodeEx () from /usr/lib/libpython2.6.so
#33 0x00007ffff7a23e32 in PyEval_EvalCode () from /usr/lib/libpython2.6.so
#34 0x00007ffff7a41d1c in PyRun_StringFlags () from /usr/lib/libpython2.6.so
#35 0x00007ffff019d1e7 in pyembed_eval (env=0x401121d0,
    _jepThread=140737154335376,
    str=0x7fffec117390 "import numpy; numpy.finfo(float)") at pyembed.c:929
#36 0x00007ffff019c29a in Java_jep_Jep_eval (env=0x401121d0,
---Type <return> to continue, or q <return> to quit---
    obj=<value optimized out>, tstate=140737154335376, jstr=0x7ffff58fbdf8)
    at jep.c:137
#37 0x00007ffff1a67d6e in ?? ()
#38 0x0000000000000000 in ?? ()
(gdb) thr 10
[Switching to thread 10 (Thread 0x7ffff08b3700 (LWP 20747))]#0  0x00007ffff771cb50 in sem_wait () from /lib/libpthread.so.0
(gdb) bt
#0  0x00007ffff771cb50 in sem_wait () from /lib/libpthread.so.0
#1  0x00007ffff600eea5 in check_pending_signals(bool) ()
   from /usr/lib/jvm/java-6-sun-1.6.0.26/jre/lib/amd64/server/libjvm.so
#2  0x00007ffff600745c in signal_thread_entry(JavaThread*, Thread*) ()
   from /usr/lib/jvm/java-6-sun-1.6.0.26/jre/lib/amd64/server/libjvm.so
#3  0x00007ffff61170f1 in JavaThread::run() ()
   from /usr/lib/jvm/java-6-sun-1.6.0.26/jre/lib/amd64/server/libjvm.so
#4  0x00007ffff600eadf in java_start(Thread*) ()
   from /usr/lib/jvm/java-6-sun-1.6.0.26/jre/lib/amd64/server/libjvm.so
#5  0x00007ffff77159ca in start_thread () from /lib/libpthread.so.0
#6  0x00007ffff726e70d in clone () from /lib/libc.so.6
#7  0x0000000000000000 in ?? ()

通过增量调试，我发现下面这个循环运行了9次（应该运行max_iterN=10000次），然后在temp = z*t这一行停住了：

   for _ in xrange(max_iterN):
       y = z
       z = y*y
       a = z*one # Check here for underflow
       temp = z*t
       if any(a+a == zero) or any(abs(z)>=y):
           break
       temp1 = temp * betain
       if any(temp1*beta == z):
           break
       i = i + 1
       k = k + k
   else:
       print 'baz'
       raise RuntimeError, msg % (_, one.dtype)

有没有什么想法？为了完全复现这个问题：

mkdir /tmp/test/

cd /tmp/test/

wget 'http://sourceforge.net/projects/numpy/files/NumPy/1.6.1/numpy-1.6.1.tar.gz'

tar xzf numpy-1.6.1.tar.gz

cd numpy-1.6.1/

python setup.py install --prefix=/tmp/test/env

cd

git clone https://github.com/mrj0/jepp.git

cd jepp/jep/

export JAVA_HOME=/usr/lib/jvm/java-6-sun/

./configure --prefix=/tmp/jepp && make && make install

touch /tmp/go.py

PYTHONPATH=/tmp/test/env/lib/python2.6/site-packages/
LD_PRELOAD=/usr/lib/libpython2.6.so java -jar jepp.jar -i /tmp/go.py

>>> import numpy
>>> numpy.finfo(float)

我在我的机器上能复现这个问题，都是Ubuntu x86_64的系统。虽然有一段时间没碰了，但我记得在Ubuntu的numpy包上没有遇到这个问题，只有按照上面的说明从源代码构建时才会出现。

提前谢谢你。这让我很头疼。

signal numpy java CPython gdb JVM jepp semaphore

3 个回答

0

如果你有多个线程在同时使用CPython，而其中一个线程正在忙碌，那么全局锁可能会被锁住，这样CPython就只能等待这个锁被释放，期间会处于挂起状态。

你在使用Python的时候，有哪些线程在调用它呢？另外，这些线程在做什么呢？

回答于 2025-04-17 由 Python大师
分享 举报
0

你有没有特别的原因不能使用 Jython 呢？我理解它基本上可以算是一个真正的替代品。

回答于 2025-04-17 由 Python大师
分享 举报
1

看看John Wright的这篇帖子，他也遇到了同样的问题：http://mail.scipy.org/pipermail/numpy-discussion/2009-July/044046.html

简单来说，numpy在使用Python的C接口时，和Jep不兼容。Jep使用子解释器来支持多个解释器环境。如果想要修改Jep，让它使用GIL状态函数，可能是可行的，但这样会大大限制Jep在Java中的功能。

回答于 2025-04-17 由 Python大师
分享 举报

撰写回答