python 3.3 字典：如何将PyDictKeysObject结构转换为Python类？

Question

我正在尝试修改Brandon Rhodes的代码，链接在这里：检查CPython字典内部的例程，让它可以在CPython 3.3上运行。

我觉得我已经成功地翻译了这个结构。

typedef PyDictKeyEntry *(*dict_lookup_func)
    (PyDictObject *mp, PyObject *key, Py_hash_t hash, PyObject ***value_addr);

struct _dictkeysobject {
    Py_ssize_t dk_refcnt;
    Py_ssize_t dk_size;
    dict_lookup_func dk_lookup;
    Py_ssize_t dk_usable;
    PyDictKeyEntry dk_entries[1];
};

我认为下面的内容现在看起来不错：

from ctypes import Structure, c_ulong, POINTER, cast, py_object, CFUNCTYPE

LOOKUPFUNC = CFUNCTYPE(POINTER(PyDictKeyEntry), POINTER(PyDictObject), 
                       py_object, c_ulong, POINTER(POINTER(py_object)))

class PyDictKeysObject(Structure):
"""A key object"""
_fields_ = [
    ('dk_refcnt', c_ssize_t),
    ('dk_size', c_ssize_t),
    ('dk_lookup', LOOKUPFUNC),
    ('dk_usable', c_ssize_t),
    ('dk_entries', PyDictKeyEntry * 1),
]

PyDictKeysObject._dk_entries = PyDictKeysObject.dk_entries
PyDictKeysObject.dk_entries = property(lambda s: 
    cast(s._dk_entries, POINTER(PyDictKeyEntry * s.dk_size))[0])

这行代码现在可以正常工作，其中 d == {0: 0, 1: 1, 2: 2, 3: 3}：

obj = cast(id(d), POINTER(PyDictObject)).contents  # works!!`

这是我从C结构 PyDictObject的翻译：

class PyDictObject(Structure):  # an incomplete type
    """A dictionary object."""

def __len__(self):
    """Return the number of dictionary entry slots."""
    pass

def slot_of(self, key):
    """Find and return the slot at which `key` is stored."""
    pass

def slot_map(self):
    """Return a mapping of keys to their integer slot numbers."""
    pass

PyDictObject._fields_ = [
    ('ob_refcnt', c_ssize_t),
    ('ob_type', c_void_p),
    ('ma_used', c_ssize_t),
    ('ma_keys', POINTER(PyDictKeysObject)),
    ('ma_values', POINTER(py_object)),  # points to array of ptrs
]

Answer 1

我的问题是如何访问一个在Cpython 3.3中实现的Python字典背后的C结构。我从在 cpython/Objects/dictobject.c 和 Include/dictobject.h 中提供的C结构开始。定义字典时涉及到三个C结构：PyDictObject、PyDictKeysObject 和 PyDictKeyEntry。每个C结构正确翻译成Python的方式如下。注释部分说明了我需要修正的地方。感谢@eryksun一路上的指导！！

class PyDictKeyEntry(Structure):
"""An entry in a dictionary."""
    _fields_ = [
        ('me_hash', c_ulong),
        ('me_key', py_object),
        ('me_value', py_object),
    ]

class PyDictObject(Structure):
    """A dictionary object."""
    pass

LOOKUPFUNC = CFUNCTYPE(POINTER(PyDictKeyEntry), POINTER(PyDictObject), py_object, c_ulong, POINTER(POINTER(py_object)))

class PyDictKeysObject(Structure):
"""An object of key entries."""
    _fields_ = [
        ('dk_refcnt', c_ssize_t),
        ('dk_size', c_ssize_t),
        ('dk_lookup', LOOKUPFUNC),  # a function prototype per docs 
        ('dk_usable', c_ssize_t),
        ('dk_entries', PyDictKeyEntry * 1),  # an array of size 1; size grows as keys are inserted into dictionary; this variable-sized field was the trickiest part to translate into python
    ]   

PyDictObject._fields_ = [
    ('ob_refcnt', c_ssize_t),  # Py_ssize_t translates to c_ssize_t per ctypes docs
    ('ob_type', c_void_p),     # could not find this in the docs
    ('ma_used', c_ssize_t),
    ('ma_keys', POINTER(PyDictKeysObject)),
    ('ma_values', POINTER(py_object)),  # Py_Object* translates to py_object per ctypes docs
]

PyDictKeysObject._dk_entries = PyDictKeysObject.dk_entries
PyDictKeysObject.dk_entries = property(lambda s: cast(s._dk_entries, POINTER(PyDictKeyEntry * s.dk_size))[0])  # this line is called every time the attribute dk_entries is accessed by a PyDictKeyEntry instance; it returns an array of size dk_size starting at address _dk_entries. (POINTER creates a pointer to the entire array; the pointer is dereferenced (using [0]) to return the entire array); the code then accesses the ith element of the array)

下面这个函数可以访问Python字典背后的PyDictObject：

def dictobject(d):
    """Return the PyDictObject lying behind the Python dict `d`."""
    if not isinstance(d, dict):
        raise TypeError('cannot create a dictobject from %r' % (d,))
    return cast(id(d), POINTER(PyDictObject)).contents

如果d是一个包含键值对的Python字典，那么obj就是包含这些键值对的PyDictObject实例：

obj = cast(id(d), POINTER(PyDictObject)).contents

PyDictKeysObject的一个实例是：

key_obj = obj.ma_keys.contents

存储在字典槽0中的键的指针是：

key_obj.dk_entries[0].me_key

使用这些类的程序，以及探测每个插入字典的键的哈希冲突的例程，可以在这里找到。我的代码是对Brandon Rhodes为Python 2.x编写的代码的修改。他的代码在这里。