内存分配numpy.代表

Question

我的脚本内存泄漏，我把它缩小到np.repeat。理论上，如果我有一个np.array有它自己的指针，叫做arr，我做一个arr_repeated = arr.repeat(10, axis = 0)，然后对它们做del，它们占用的内存应该被释放吗？你知道吗

我正在核实：

psutil.Process(os.getpid()).memory_info().rss 

更具体地说，在下面的代码中memory_before和memory_after应该是不同的吗？你知道吗

arr = np.array([[1,2,3,...], [...], [...]])
arr_repeated = arr.repeat(10, axis = 0)

memory_before = psutil.Process(os.getpid()).memory_info().rss 

del arr
del arr_repeated

memory_after = psutil.Process(os.getpid()).memory_info().rss 

我有

extra/python-numpy 1.13.1-2 [installed]

在我的安特戈斯/拱门上

Answer 1

仅仅因为使用了del变量并不意味着内存会立即被释放。它只是删除变量的名称。Pythons引用计数机制和gc负责释放内存。你知道吗

因此，为了安全起见，您需要使用^{}强制gc清除所有可能包含循环引用的内容，否则Python将“随时”清除这些内容（可能只有在内存越来越少或在特定时间之后）。你知道吗

例如the documentation about ^{}包含以下注释：

Note

del x doesn’t directly call x.__del__() — the former decrements the reference count for x by one, and the latter is only called when x’s reference count reaches zero. Some common situations that may prevent the reference count of an object from going to zero include: circular references between objects (e.g., a doubly-linked list or a tree data structure with parent and child pointers); a reference to the object on the stack frame of a function that caught an exception (the traceback stored in sys.exc_info()[2] keeps the stack frame alive); or a reference to the object on the stack frame that raised an unhandled exception in interactive mode (the traceback stored in sys.last_traceback keeps the stack frame alive). The first situation can only be remedied by explicitly breaking the cycles; the second can be resolved by freeing the reference to the traceback object when it is no longer useful, and the third can be resolved by storing None in sys.last_traceback. Circular references which are garbage are detected and cleaned up when the cyclic garbage collector is enabled (it’s on by default). Refer to the documentation for the gc module for more information about this topic.

import numpy as np
import psutil
import os
import gc

arr = np.array([list(range(10000)), list(range(10000)), list(range(10000))])
arr_repeated = arr.repeat(10, axis = 0)

gc.collect()
memory_before = psutil.Process(os.getpid()).memory_info().rss

del arr
del arr_repeated

gc.collect()
memory_after = psutil.Process(os.getpid()).memory_info().rss

---

请注意，gc还有一个用于检测泄漏的设置：gc.set_debug(gc.DEBUG_LEAK)不确定它究竟是如何工作的，但它可能值得一试，而不是使用psutil。你知道吗