更快的python pickle替换
larch-pickle的Python项目详细描述
此模块可作为pickle的透明替换。
与原始python pickle的区别:
- no memo attribute.
- no clear_memo().
- no persistent_id interface.
- byte strings are always assumed to be utf-8 encoded.
- Pickler has an additional with_refs parameter. Setting with_refs to false, the pickler will ignore object references. This can result in an extra speed boost.
安装
落叶松泡菜需要boost库进行编译。如果boost不在 标准包含路径安装时使用:
python build_ext -I /path/to/boost install
与其他泡菜包装相比的速度
转储字典(10个循环)
Package | Seconds | Size |
---|---|---|
marshal | 0.46141 | 15975952 |
larch-pickle | 0.664333 | 9714411 |
ujson | 1.94214 | 13141354 |
msgpack | 2.04704 | 9843459 |
json | 2.14117 | 14365311 |
cPickle | 4.09599 | 12781866 |
加载字典(10个循环)
Package | Seconds |
---|---|
msgpack | 1.38378 |
marshal | 1.49512 |
larch-pickle | 1.51489 |
ujson | 2.42431 |
cPickle | 2.42524 |
json | 6.04131 |
转储对象(10个循环)
Package | Seconds | Size |
---|---|---|
larch-pickle | 1.07443 | 10094415 |
ujson | 2.55502 | 13331354 |
cPickle | 4.70805 | 12961909 |
加载对象(10个循环)
Package | Seconds |
---|---|
larch-pickle | 1.6615 |
ujson | 2.45786 |
cPickle | 2.58697 |
转储字符串(10个循环)
Package | Seconds | Size |
---|---|---|
marshal | 0.56747 | 32826787 |
msgpack | 0.975226 | 29464951 |
larch-pickle | 1.16368 | 12103729 |
json | 1.19005 | 31966498 |
ujson | 1.51648 | 30987525 |
cPickle | 4.12952 | 19871780 |
加载字符串(10个循环)
Package | Seconds |
---|---|
larch-pickle | 0.353255 |
marshal | 0.435936 |
msgpack | 0.469068 |
cPickle | 1.72308 |
ujson | 1.82656 |
json | 3.01798 |
转储列表(10个循环)
Package | Seconds | Size |
---|---|---|
marshal | 0.861886 | 42703907 |
larch-pickle | 2.33574 | 16054577 |
json | 3.11054 | 35917346 |
ujson | 4.4284 | 34938373 |
msgpack | 5.55321 | 31440375 |
cPickle | 15.5531 | 33700258 |
加载列表(10个循环)
Package | Seconds |
---|---|
larch-pickle | 2.07305 |
marshal | 2.14713 |
msgpack | 2.46054 |
ujson | 4.14067 |
json | 5.20221 |
cPickle | 8.1974 |