Python通过在另一个数据帧中使用不带forloop的值来实现

Question

我有一个代码正在工作，但速度很慢。 我有一个带有数据“djk”的数据帧，我想根据一个复杂的groupby函数对其进行汇总。 我需要按“交易对手”、“货币”和“到期日”对它们进行分组。“djk”和“Bucket”都是具有许多行和列（大小相同）的数据帧。我希望groupby函数在分组时使用相应的列。我已经用for循环解决了这个问题，但是对于大型数据帧来说，它的速度很慢。是否有其他方法可以更快地编写此代码，删除for循环

import pandas as pd
import numpy as np

n = 10000
m = 200
n_name = 25
data_1 = pd.DataFrame(np.random.randint(1, 4, size=(n, m))).astype(int)
data_2 = pd.DataFrame(np.random.randint(100, 200, size=(n, m)))
data_1['COUNTERPARTY'] = (np.random.randint(10, n_name, n)).astype(str)
data_1['COUNTERPARTY'] = 'COUNTERPARTY_' + data_1['COUNTERPARTY']
data_1['CURRENCY'] = (np.random.randint(0, 3, n)).astype(str)
data_1['CURRENCY'] = 'CURRENCY_' + data_1['CURRENCY']


result_pd = pd.DataFrame(0, index=data_1['COUNTERPARTY'].unique(), columns=range(m))


def f_2_support(srs):
    cnt = {k: v for k, v in zip(srs.index.get_level_values(2), srs)}
    a = cnt.get(1, 0)
    b = cnt.get(2, 0)
    c = cnt.get(3, 0)
    return np.sqrt(a ** 2 + b ** 2 + c ** 2 + 1.4 * a * b + 1.4 * b * c + 0.6 * a * c)


for i in range(m):
    df = pd.DataFrame()
    df['COUNTERPARTY'] = data_1['COUNTERPARTY']
    df['CURRENCY'] = data_1['CURRENCY']
    df['djk'] = data_2.loc[:, i]
    df['Maturity_Bucket'] = data_1.loc[:, i]
    result_pd.loc[:, i] = df.groupby(['COUNTERPARTY', 'CURRENCY', 'Maturity_Bucket']).agg({'djk': 'sum'}).groupby(
            ['COUNTERPARTY', 'CURRENCY']).agg({'djk': lambda x: f_2_support(x)}).groupby('COUNTERPARTY').agg(
            {'djk': 'sum'})

我正在尝试下面的代码，但没有成功。它只返回一个空序列。 怎么了

df_result = pd.DataFrame({i: f_2_new_column(data_2 , data_1, i) for i in range(m)})

def f_2_new_column(data_2 , data_1, n):
    return data_2 .iloc[:, n].groupby([data_1['COUNTERPARTY'], data_1['CURRENCY'], data_1.iloc[:, n]]).agg('sum').groupby(
            [data_1['COUNTERPARTY'], data_1['CURRENCY']]).agg(lambda x: f_2_support(x)).groupby(data_1['COUNTERPARTY']).agg(
            'sum')

Answer 1

你试过这样做吗

df = pd.DataFrame()
df['COUNTERPARTY'] = data_1['COUNTERPARTY']
df['CURRENCY'] = data_1['CURRENCY']
df['Maturity_Bucket'] = data_1.loc[:, 1]

for i in range(m):
df['djk'] = data_2.loc[:, i]
result_pd.loc[:, i] = df.groupby(['COUNTERPARTY', 'CURRENCY', 'Maturity_Bucket']).agg({'djk': 'sum'}).groupby(
        ['COUNTERPARTY', 'CURRENCY']).agg({'djk': lambda x: f_2_support(x)}).groupby('COUNTERPARTY').agg(
        {'djk': 'sum'})

同样的代码只是稍微修改了一下。每次运行for循环时，您都在创建一个数据帧，并一次又一次地对列进行切片