MemoryError:无法分配具有形状和数据类型obj的数组

Question

我想在两个数据集之间执行一个内部连接，如下所示：

theme_ids.head()
id  Loan Theme ID   Loan Theme Type Partner ID
0   638631  a1050000000skGl General 151
1   640322  a1050000000skGl General 151
2   641006  a1050000002X1ij Higher Education    160
3   641019  a1050000002X1ij Higher Education    160
4   641594  a1050000002VbsW Subsistence Agriculture 336

以及

theme_reg.head()
Partner ID  Field Partner Name  sector  Loan Theme ID   Loan Theme Type country forkiva region  geocode_old ISO ... amount  LocationName    geocode names   geo lat lon mpi_region  mpi_geo rural_pct
0   9   KREDIT Microfinance Institution General Financial Inclusion a1050000000slfi Higher Education    Cambodia    No  Banteay Meanchey    (13.75, 103.0)  KHM ... 450 Banteay Meanchey, Cambodia  [(13.6672596, 102.8975098)] Banteay Meanchey Province; Cambodia (13.6672596, 102.8975098)   13.667259   102.897507  Banteay Mean Chey, Cambodia (13.6672596, 102.8975098)   90
1   9   KREDIT Microfinance Institution General Financial Inclusion a10500000068jPe Vulnerable Populations  Cambodia    No  Battambang Province NaN KHM ... 20275   Battambang Province, Cambodia   [(13.0286971, 102.989615)]  Battambang Province; Cambodia   (13.0286971, 102.989615)    13.028697   102.989616  Banteay Mean Chey, Cambodia (13.6672596, 102.8975098)   90
2   9   KREDIT Microfinance Institution General Financial Inclusion a1050000000slfi Higher Education    Cambodia    No  Battambang Province NaN KHM ... 9150    Battambang Province, Cambodia   [(13.0286971, 102.989615)]  Battambang Province; Cambodia   (13.0286971, 102.989615)    13.028697   102.989616  Banteay Mean Chey, Cambodia (13.6672596, 102.8975098)   90
3   9   KREDIT Microfinance Institution General Financial Inclusion a10500000068jPe Vulnerable Populations  Cambodia    No  Kampong Cham Province   (12.0, 105.5)   KHM ... 604950  Kampong Cham Province, Cambodia [(12.0982918, 105.3131185)] Kampong Cham Province; Cambodia (12.0982918, 105.3131185)   12.098291   105.313118  Kampong Cham, Cambodia  (11.9924294, 105.4645408)   90
4   9   KREDIT Microfinance Institution General Financial Inclusion a1050000002X1Uu Sanitation  Cambodia    No  Kampong Cham Province   (12.0, 105.5)   KHM ... 275 Kampong Cham Province, Cambodia [(12.0982918, 105.3131185)] Kampong Cham Province; Cambodia (12.0982918, 105.3131185)   12.098291   105.313118  Kampong Cham, Cambodia  (11.9924294, 105.4645408)   90

我试过：

data = pd.merge(theme_ids, theme_reg, on='Partner ID', how='inner') 

由此引发：

MemoryError: Unable to allocate array with shape (15, 144356281) and data type object

帮助会被告知的。谢谢！

Answer 1

正如@Iguananaut在Unable to allocate array with shape and data type中所说，如果在Linux系统上运行，则可以修改overcommit行为。为了允许overcommit，您可以在/proc/sys/vm/overcommit_memory上写1，正如他在回答中所解释的那样。