gc上的分段错误,使用ctypes

2024-03-28 11:34:19 发布

您现在位置:Python中文网/ 问答频道 /正文

我使用ctypes调用一个本机c库,传入一个特殊的结构,然后返回另一个结构。调用gc时出现seg错误,在gdb trace中找不到太多细节:

Thread 1 "python" received signal SIGSEGV, Segmentation fault.
0x00007ffff7a5ca44 in visit_decref (op=0xf5ee10, data=0x0) at Modules/gcmodule.c:374
374 Modules/gcmodule.c: No such file or directory.
gdb) where
#0  0x00007ffff7a5ca44 in visit_decref (op=0xf5ee10, data=0x0) at Modules/gcmodule.c:374
#1  0x00007ffff798bb0c in subtype_traverse (self=0x7fff7744fd90, visit=0x7ffff7a5ca40 <visit_decref>, arg=0x0) at Objects/typeobject.c:1011
#2  0x00007ffff7a5bd77 in subtract_refs (containers=<optimized out>) at Modules/gcmodule.c:399
#3  collect (generation=generation@entry=0, n_collected=n_collected@entry=0x7fffffffb708, n_uncollectable=n_uncollectable@entry=0x7fffffffb710, nofail=nofail@entry=0) at Modules/gcmodule.c:956
#4  0x00007ffff7a5c95d in collect_with_callback (generation=0) at Modules/gcmodule.c:1128
#5  0x00007ffff7a5d1eb in collect_generations () at Modules/gcmodule.c:1151
#6  _PyObject_GC_Alloc (basicsize=<optimized out>, use_calloc=0) at Modules/gcmodule.c:1726
#7  _PyObject_GC_Malloc (basicsize=<optimized out>) at Modules/gcmodule.c:1736

代码示例如下:

_C_DOUBLE_P = POINTER(c_double)
_C_INT_P = POINTER(c_int)


class _IN_DATA(Structure):
    _fields_ = [('in_kps', _C_DOUBLE_P),
                ('in_desc', _C_DOUBLE_P)
                ]
    def __init__(self):
        t = c_int
        self.test = ctypes.cast(t, POINTER(c_int))  

class _OUT_DATA(Structure):
    _fields_ = [
        ('num_out_kps', ctypes.c_int),
        ('out_kps', _C_DOUBLE_P),  
        ('out_desc', _C_DOUBLE_P)
    ]

class _IN_DATA_LIST(ctypes.Structure):
    _fields_ = [
        ('num_crops', c_int),
        ('crops', POINTER(_IN_DATA))
    ]


    def __init__(self, crops_raw_kps: List[np.ndarray], crops_raw_descriptors: List[np.ndarray]):
        num_crops = len(crops_raw_kps)
        self.num_crops = num_crops
        crops = (POINTER(_IN_DATA) * num_crops)()
        self.crops = ctypes.cast(crops, POINTER(_IN_DATA))
        for i in range(num_crops):
            self.crops[i].in_kps = crops_raw_kps[i].ctypes.data_as(_C_DOUBLE_P)
            self.crops[i].in_desc = crops_raw_descriptors[i].ctypes.data_as(_C_DOUBLE_P)

class _OUT_DATA_LIST(ctypes.Structure):
    _fields_ = [
        ('crops_data', ctypes.POINTER(_OUT_DATA)),
        ('num_results', c_int)
    ]  

class SPPostWrapper:

    def __init__(self):

        self._post_processor_lib = ctypes.cdll.LoadLibrary("multitracker/custom_features/build/ffme/libffme.so")
        self._post_processor_lib.py_postprocess.restype = _OUT_DATA_LIST
        self._post_processor_lib.py_postprocess.argtypes = [POINTER(_IN_DATA_LIST)]

    def post_process_multi(self, crops_raw_kps: List[np.ndarray], crops_raw_descriptors: List[np.ndarray]):
        num_crops = len(crops_raw_kps)
        adjusted_kps = [np.asarray(np.squeeze(kp), np.double) for kp in crops_raw_kps]
        adjusted_desc = [np.asarray(np.squeeze(desc), np.double) for desc in crops_raw_descriptors]
        crops_struct = _IN_DATA_LIST(adjusted_kps, adjusted_desc)
        out_result= self._post_processor_lib.py_postprocess(ctypes.byref(crops_struct))

我在python中分配IN\ u数据,在c代码中分配OUT\ u数据,我尝试在c中使用cache(假设python不会清理内存)或为每个调用分配新内存(假设python释放OUT\ u数据内存)-当调用python的gc时,这两种方法都失败。你知道吗

更新: 为了更好地隔离问题,我取消了out\数据的使用,将方法设置为void,问题仍然会发生。我还尝试将in数据作为成员保存,我认为这样可以防止数据发生,直到进程关闭。所以这一定和我分配给input/input list的内存有关。你知道吗

更新2: 我能够验证只有当我传递了in\ DATA\列表中的多个项目(超过1个裁剪)时,问题才会发生。这确实很奇怪。。。你知道吗


Tags: incropsselfmodulesdatarawnpout
1条回答
网友
1楼 · 发布于 2024-03-28 11:34:19

在验证了即使c代码什么也不做,什么也不返回的情况下问题也会发生,并且无法解决它之后,我最终使用了更浅的输入(和输出),这看起来运行正常,没有seg故障:

class _IN_DATA(Structure):
    _fields_ = [
                ('num_crops', c_int),
                ('in_kps', _C_DOUBLE_P),
                ('in_desc', _C_DOUBLE_P)
                ]
    def __init__(self, kps_flat, desc_falt, num_crops):
        self.num_crops = num_crops
        self.in_kps = kps_flat.ctypes.data_as(_C_DOUBLE_P)
        self.in_desc = desc_falt.ctypes.data_as(_C_DOUBLE_P)


class _OUT_DATA(Structure):
    _fields_ = [
        ('num_results', ctypes.c_int),
        ('results_per_crop', _C_INT_P),
        ('out_kps', _C_DOUBLE_P),  # x,y, score
        ('out_desc', _C_DOUBLE_P)
    ]

相关问题 更多 >