与pandas.ewm（）类似的排序反向权重

Question

我正在寻找一种方法来获得每一行的加权平均值，计算出的权重根据所有以前的值减半（并且对于每一行，继续增加以前的值）

所以像Out[1]这样的输出应该像Out[2]：

权重应为1/1至1/2至1/3至1/4至1/5…等，具体取决于之前的观察次数

>> Out[1]:
   B
0  1
1  2
2  5
3  3

Out[2]:
   B
0  1
1  1.66
2  3.455
3  3.080


# Where row 1 is calculated as: (2*1/1 + 1*1/2)/(1/1 + 1/2) = 1.66,
# and row 2 as: (5*1/1 + 2*1/2 + 1*1/3)/(1/1 + 1/2 + 1/3) = 3.455,
# row 3 as: (3*1/1 + 5*1/2 + 2*1/3 + 1*1/4)/(1/1 + 1/2 + 1/3 + 1/4) = 3.080,
# ... and that way keeping forward depending on how much observations I have.

有没有可能修改pandas.ewm（）函数以获得一个解决方案，因为它有类似的方法？我只是不知道如何根据前面的所有行为每一行设置权重

也许有人能够将其转换为使用Python的代码

Answer 1

您不能使用pandasEWM函数执行此操作

这是因为无法表示几何级数

S_1 = 1 + 1/2 + 1/3 + ...

在形式上

S_2 = 1 + (1 - alpha) + (1 - alpha)**2 + ...

这会让你得到你想要的

import numpy as np


def rolling_mean_harmonic(x: np.ndarray) -> np.ndarray:
    """
    Calculates the rolling mean using harmonic series weightings
    along the first dimension of ``x``

    Example:
        >>> rolling_mean_harmonic(np.array([1, 2, 5, 5]))
        array([1.        , 1.66666667, 3.45454545, 3.08      ])
    """
    out = np.empty(x.shape, dtype=np.float64)
    h = (1/np.arange(1, len(x) + 1))
    h_cumsum = h.cumsum()
    for i in range(x.shape[0]):
        out[i] = (x[:i + 1] * h[:i + 1][::-1]).sum() / h_cumsum[i]
    return out

如果速度是一个问题，上面的代码可以很容易地改进，并与Numba njit+prange并行

编辑…

我试着用numba，发现并行实际上没有加速：

import numpy as np
import numba as nb


nb.njit('float64[:](float64[:])', nogil=True, fastmath=True)
def rolling_mean_harmonic_nb_noparr(x: np.ndarray) -> np.ndarray:
    """
    Calculates the rolling mean using harmonic series weightings

    Example:
        >>> rolling_mean_harmonic(np.array([1, 2, 5, 5], dtype=float))
        array([1.        , 1.66666667, 3.45454545, 3.08      ])
    """
    out = np.empty_like(x)
    h = (1/np.arange(1, len(x) + 1))
    h_cumsum = h.cumsum()
    for i in range(x.shape[0]):
        out[i] = (x[:i + 1] * h[:i + 1][::-1]).sum() / h_cumsum[i]
    return out


nb.njit('float64[:](float64[:])', parallel=True, nogil=True, fastmath=True)
def rolling_mean_harmonic_nb(x: np.ndarray) -> np.ndarray:
    """
    Calculates the rolling mean using harmonic series weightings

    Example:
        >>> rolling_mean_harmonic(np.array([1, 2, 5, 5], dtype=float))
        array([1.        , 1.66666667, 3.45454545, 3.08      ])
    """
    out = np.empty_like(x)
    h = (1/np.arange(1, len(x) + 1))
    h_cumsum = h.cumsum()
    for i in nb.prange(x.shape[0]):
        out[i] = (x[:i + 1] * h[:i + 1][::-1]).sum() / h_cumsum[i]
    return out