Pandas,数据帧,groupby,性病

2024-03-29 08:44:03 发布

您现在位置:Python中文网/ 问答频道 /正文

这里的熊猫是新来的。一个(小)问题:主机、操作、执行时间。我想按主机分组,然后按主机+操作分组,计算每个主机的执行时间的std偏差,然后按主机+操作对分组。看起来很简单?

它适用于按单个列分组:

df
Out[360]:
<class 'pandas.core.frame.DataFrame'>
Int64Index: 132564 entries, 0 to 132563
Data columns (total 9 columns):
datespecial    132564  non-null values
host           132564  non-null values
idnum          132564  non-null values
operation      132564  non-null values
time           132564  non-null values
...
dtypes: float32(1), int64(2), object(6)



byhost = df.groupby('host')


byhost.std()
Out[362]:
                 datespecial         idnum      time
host
ahost1.test  11946.961952  40367.033852  0.003699
host1.test   15484.975077  38206.578115  0.008800
host10.test           NaN  37644.137631  0.018001
...

很好。现在:

byhostandop = df.groupby(['host', 'operation'])

byhostandop.std()
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-364-2c2566b866c4> in <module>()
----> 1 byhostandop.std()

/home/username/anaconda/lib/python2.7/site-packages/pandas/core/groupby.pyc in std(self, ddof)
    386         # todo, implement at cython level?
    387         if ddof == 1:
--> 388             return self._cython_agg_general('std')
    389         else:
    390             f = lambda x: x.std(ddof=ddof)

/home/username/anaconda/lib/python2.7/site-packages/pandas/core/groupby.pyc in _cython_agg_general(self, how, numeric_only)
   1615
   1616     def _cython_agg_general(self, how, numeric_only=True):
-> 1617         new_blocks = self._cython_agg_blocks(how, numeric_only=numeric_only)
   1618         return self._wrap_agged_blocks(new_blocks)
   1619

/home/username/anaconda/lib/python2.7/site-packages/pandas/core/groupby.pyc in _cython_agg_blocks(self, how, numeric_only)
   1653                 values = com.ensure_float(values)
   1654
-> 1655             result, _ = self.grouper.aggregate(values, how, axis=agg_axis)
   1656
   1657             # see if we can cast the block back to the original dtype

/home/username/anaconda/lib/python2.7/site-packages/pandas/core/groupby.pyc in aggregate(self, values, how, axis)
    838                 if is_numeric:
    839                     result = lib.row_bool_subset(result,
--> 840                                                  (counts > 0).view(np.uint8))
    841                 else:
    842                     result = lib.row_bool_subset_object(result,

/home/username/anaconda/lib/python2.7/site-packages/pandas/lib.so in pandas.lib.row_bool_subset (pandas/lib.c:16540)()

ValueError: Buffer dtype mismatch, expected 'float64_t' but got 'float'

嗯??为什么我会有这个例外?

更多问题:

  • 如何计算dataframe.groupby([several columns])上的std偏差?

  • 如何将计算限制为选定列?E、 显然,在这里计算日期/时间戳上的std dev是没有意义的。


Tags: incoreselfpandashomelibnullcython
1条回答
网友
1楼 · 发布于 2024-03-29 08:44:03

了解Pandas/Python的版本很重要。看起来在Pandas版本<;0.10中可能会出现此异常(请参见ValueError: Buffer dtype mismatch, expected 'float64_t' but got 'float')。为了避免这种情况,可以将float列强制转换为float64

df.astype('float64')

要计算选定列上的std(),只需选择列:)

>>> df = pd.DataFrame({'a':range(10), 'b':range(10,20), 'c':list('abcdefghij'), 'g':[1]*3 + [2]*3 + [3]*4})
>>> df
   a   b  c  g
0  0  10  a  1
1  1  11  b  1
2  2  12  c  1
3  3  13  d  2
4  4  14  e  2
5  5  15  f  2
6  6  16  g  3
7  7  17  h  3
8  8  18  i  3
9  9  19  j  3
>>> df.groupby('g')[['a', 'b']].std()
          a         b
g                    
1  1.000000  1.000000
2  1.000000  1.000000
3  1.290994  1.290994

更新

就目前而言,似乎std()正在对groupby结果调用aggregation(),这是一个微妙的错误(请参见-Python Pandas: Using Aggregate vs Apply to define new columns)。为了避免这种情况,可以使用apply()

byhostandop['time'].apply(lambda x: x.std())

相关问题 更多 >