无法将对象放入队列

Question

我想把一个 scapy.layers.dhcp.BOOTP 的实例放到一个 multiprocessing.Queue 里。每次我调用 put() 时，就会出现以下错误：

Traceback (most recent call last):
  File "/usr/lib/python2.6/multiprocessing/queues.py", line 242, in _feed
    send(obj)
PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed

当然，直接用 pickle.dumps() 来尝试序列化这个实例也失败了。但是，为什么这个类不能被序列化呢？

对于那些没有安装 scapy 的人：

class BOOTP(Packet):
    name = "BOOTP"
    fields_desc = [ ByteEnumField("op",1, {1:"BOOTREQUEST", 2:"BOOTREPLY"}),
                    ByteField("htype",1),
                    ByteField("hlen",6),
                    ByteField("hops",0),
                    IntField("xid",0),
                    ShortField("secs",0),
                    FlagsField("flags", 0, 16, "???????????????B"),
                    IPField("ciaddr","0.0.0.0"),
                    IPField("yiaddr","0.0.0.0"),
                    IPField("siaddr","0.0.0.0"),
                    IPField("giaddr","0.0.0.0"),
                    Field("chaddr","", "16s"),
                    Field("sname","","64s"),
                    Field("file","","128s"),
                    StrField("options","") ]
    def guess_payload_class(self, payload):
        if self.options[:len(dhcpmagic)] == dhcpmagic:
            return DHCP
        else:
            return Packet.guess_payload_class(self, payload)
    def extract_padding(self,s):
        if self.options[:len(dhcpmagic)] == dhcpmagic:
            # set BOOTP options to DHCP magic cookie and make rest a payload of DHCP options
            payload = self.options[len(dhcpmagic):]
            self.options = self.options[:len(dhcpmagic)]
            return payload, None
        else:
            return "", None
    def hashret(self):
        return struct.pack("L", self.xid)
    def answers(self, other):
        if not isinstance(other, BOOTP):
            return 0
        return self.xid == other.xid

有没有其他方法可以把这个实例“传输”到另一个子进程呢？

Answer 1

我用的一个方法是把数据包转换成字符串，然后把它放到队列里...

Answer 2

另一个可能帮助你诊断这类问题的方法是使用pickle模块，而不是cPickle（这个可能是被queues.py隐式使用的）。

我也遇到过类似的情况，收到的错误信息完全没有帮助，

Can't pickle <type 'function'>: attribute lookup __builtin__.function failed

我进入了调试器，找到了正在被序列化的对象，然后尝试将它传递给

pickle.dump(myobj,open('outfile','w'),-1)  

结果得到了一个更有用的信息：

PicklingError: Can't pickle <function findAllRefs at 0x105809f50>: 
   it's not found as buildsys.repoclient.findAllRefs

这让问题代码的指向更加明确。

Answer 3

好吧，问题在于你不能把函数的类型进行“序列化”。这个“序列化”就像是把一个东西变成可以存储或传输的格式。你可以通过 type(some_user_function) 来查看一个函数的类型。看看下面这个：

>>> import types
>>> pickle.dumps(types.FunctionType)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'picke' is not defined
>>> pickle.dumps(types.FunctionType)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python26\lib\pickle.py", line 1366, in dumps
    Pickler(file, protocol).dump(obj)
  File "C:\Python26\lib\pickle.py", line 224, in dump
    self.save(obj)
  File "C:\Python26\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python26\lib\pickle.py", line 748, in save_global
    (obj, module, name))
pickle.PicklingError: Can't pickle <type 'function'>: it's not found as __built
n__.function

所以，这种函数类型是存储在你尝试发送的对象中的某个地方。它不在你粘贴的代码里，所以我猜它是在父类里面。

也许你可以直接发送创建 scapy.layers.dhcp.BOOTP 实例所需的所有参数，而不是发送这个实例，这样就可以避免这个问题了？