为什么掩蔽数组看起来比无掩蔽数组小？

Question

我想知道一个蒙面的numpy数组和一个带有nans的普通数组的大小有什么区别。你知道吗

import numpy as np
g = np.random.random((5000,5000))
indx = np.random.randint(0,4999,(500,2))
mask =  np.full((5000,5000),False,dtype=bool)
mask[indx] = True
g_mask = np.ma.array(g,mask=mask)

我使用以下answer来计算对象的大小：

import sys
from types import ModuleType, FunctionType
from gc import get_referents
​
# Custom objects know their class.
# Function objects seem to know way too much, including modules.
# Exclude modules as well.
BLACKLIST = type, ModuleType, FunctionType
​
​
def getsize(obj):
    """sum size of object & members."""
    if isinstance(obj, BLACKLIST):
        raise TypeError('getsize() does not take argument of type: '+ str(type(obj)))
    seen_ids = set()
    size = 0
    objects = [obj]
    while objects:
        need_referents = []
        for obj in objects:
            if not isinstance(obj, BLACKLIST) and id(obj) not in seen_ids:
                seen_ids.add(id(obj))
                size += sys.getsizeof(obj)
                need_referents.append(obj)
        objects = get_referents(*need_referents)
    return size

结果如下：

getsize(g)
>>>200000112
getsize(g_mask)
>>>25000924

为什么非屏蔽阵列比屏蔽阵列大？如何估计屏蔽数组与未屏蔽数组的实际大小？你知道吗

Answer 1

In [23]: g = np.random.random((5000,5000)) 
    ...: indx = np.random.randint(0,4999,(500,2)) 
    ...: mask =  np.full((5000,5000),False,dtype=bool) 
    ...: mask[indx] = True 
    ...: g_mask = np.ma.array(g,mask=mask)    

比较g数组和g_mask的_data属性，我们发现后者只是前者的view：

In [24]: g.__array_interface__                                                  
Out[24]: 
{'data': (139821997776912, False),
 'strides': None,
 'descr': [('', '<f8')],
 'typestr': '<f8',
 'shape': (5000, 5000),
 'version': 3}
In [25]: g_mask._data.__array_interface__                                       
Out[25]: 
{'data': (139821997776912, False),
 'strides': None,
 'descr': [('', '<f8')],
 'typestr': '<f8',
 'shape': (5000, 5000),
 'version': 3}

它们有相同的数据缓冲区，但它们的id不同：

In [26]: id(g)                                                                  
Out[26]: 139822758212672
In [27]: id(g_mask._data)                                                       
Out[27]: 139822386925440

面罩也一样：

In [28]: mask.__array_interface__                                               
Out[28]: 
{'data': (139822298669072, False),
 'strides': None,
 'descr': [('', '|b1')],
 'typestr': '|b1',
 'shape': (5000, 5000),
 'version': 3}
In [29]: g_mask._mask.__array_interface__                                       
Out[29]: 
{'data': (139822298669072, False),
 'strides': None,
 'descr': [('', '|b1')],
 'typestr': '|b1',
 'shape': (5000, 5000),
 'version': 3}

实际上，通过这种构造，_mask是相同的数组：

In [30]: id(mask)                                                               
Out[30]: 139822385963056
In [31]: id(g_mask._mask)                                                       
Out[31]: 139822385963056

掩蔽数组的__array_interface__是._data属性的__array_interface__：

In [32]: g_mask.__array_interface__                                             
Out[32]: 
{'data': (139821997776912, False),

nbytes是数组的数据缓冲区大小：

In [34]: g_mask.data.nbytes                                                     
Out[34]: 200000000
In [35]: g_mask.mask.nbytes                                                     
Out[35]: 25000000

布尔数组的每个元素有1个字节，浮点数为64，8个字节。你知道吗

Answer 2

numpy.ndarray没有^{}，因此它与您尝试使用的getsize函数不兼容。GC系统看不到掩码数组的ndarray部分所拥有的引用。尤其是，base的g_mask没有包含在输出中。你知道吗