datetime multindex上的IndexSlice不起作用，但与properlyworking toy等价物似乎没有什么不同

Question

我习惯于在datetime索引上使用IndexSlice。这是一个玩具相当于我的多指标数据帧，你可以看到切片工程

#slicing works on a simple DateTime index
qf = pd.DataFrame(index=pd.date_range(start="1Jan2019",freq="d",periods=30))
qf.loc[idx['2019-1-15':None]]   #works


#the same slicing works on a multindex
qf.reset_index(inplace=True)
qf['foo']="bar"
qf['other']=range(len(qf))
qf['filler']="egbdf"
qf.set_index(['index','foo', 'other'], inplace=True)

qf.loc[idx['2019-1-15':'2019-1-20',:,:],:] #wrks    
qf.loc[idx['2019-1-15':None,'bar',:],:]   #works

但我的真实数据帧出了问题。我看不出有什么区别。你知道吗

xf.loc[idx['2019-5-1':'2019-6-1',"squat",:],:]     # This works ok
xf.loc[idx['2019-5-1':None,"squat",:],:]           # This fails

当我用'2019-5-1':None切片时得到的错误是

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-280-b0dce8e9e337> in <module>
      1 xf.loc[idx['2019-5-1':'2019-6-1',"squat",:],:]     # This works ok
----> 2 xf.loc[idx['2019-5-1':None,"squat",:],:]           # This fails
      3 #xf

C:\ProgramData\Anaconda3\envs\nambu\lib\site-packages\pandas\core\indexing.py in __getitem__(self, key)
   1492             except (KeyError, IndexError, AttributeError):
   1493                 pass
-> 1494             return self._getitem_tuple(key)
   1495         else:
   1496             # we by definition only have the 0th axis

C:\ProgramData\Anaconda3\envs\nambu\lib\site-packages\pandas\core\indexing.py in _getitem_tuple(self, tup)
    866     def _getitem_tuple(self, tup):
    867         try:
--> 868             return self._getitem_lowerdim(tup)
    869         except IndexingError:
    870             pass

C:\ProgramData\Anaconda3\envs\nambu\lib\site-packages\pandas\core\indexing.py in _getitem_lowerdim(self, tup)
    967         # we may have a nested tuples indexer here
    968         if self._is_nested_tuple_indexer(tup):
--> 969             return self._getitem_nested_tuple(tup)
    970 
    971         # we maybe be using a tuple to represent multiple dimensions here

C:\ProgramData\Anaconda3\envs\nambu\lib\site-packages\pandas\core\indexing.py in _getitem_nested_tuple(self, tup)
   1046 
   1047             current_ndim = obj.ndim
-> 1048             obj = getattr(obj, self.name)._getitem_axis(key, axis=axis)
   1049             axis += 1
   1050 

C:\ProgramData\Anaconda3\envs\nambu\lib\site-packages\pandas\core\indexing.py in _getitem_axis(self, key, axis)
   1904             # nested tuple slicing
   1905             if is_nested_tuple(key, labels):
-> 1906                 locs = labels.get_locs(key)
   1907                 indexer = [slice(None)] * self.ndim
   1908                 indexer[axis] = locs

C:\ProgramData\Anaconda3\envs\nambu\lib\site-packages\pandas\core\indexes\multi.py in get_locs(self, seq)
   2774                 # a slice, include BOTH of the labels
   2775                 indexer = _update_indexer(_convert_to_indexer(
-> 2776                     self._get_level_indexer(k, level=i, indexer=indexer)),
   2777                     indexer=indexer)
   2778             else:

C:\ProgramData\Anaconda3\envs\nambu\lib\site-packages\pandas\core\indexes\multi.py in _get_level_indexer(self, key, level, indexer)
   2635                 # note that the stop ALREADY includes the stopped point (if
   2636                 # it was a string sliced)
-> 2637                 return convert_indexer(start.start, stop.stop, step)
   2638 
   2639             elif level > 0 or self.lexsort_depth == 0 or step is not None:

AttributeError: 'int' object has no attribute 'stop'

我看不到toy索引和real索引之间有任何实质性的区别，也看不到错误消息是如何将None传递到切片器的。你知道吗

=====================================================================

我弄明白了为什么它在不同的例子中有效/无效。你知道吗

当索引完全为dates时，代码工作正常。但是如果索引中有datetimes，它就会失败。你知道吗

#this index is solely dates,  not dateTimes, and everything works
dt_index = pd.date_range(start="1jan2019",periods=100,freq="d")
zf = pd.DataFrame(index=dt_index)
zf['foo']=10
zf['bar']="squat"
zf['zaa']=range(len(dt_index))
zf.index.name="date"
zf = zf.reset_index().set_index(["date", "bar", "zaa"])


zf.loc[idx['2019-1-1':'2019-1-3',"squat",:],:]     # This works ok
zf.loc[idx['2019-1-1':,"squat",:],:]     # This works
zf.loc[idx['2019-1-1':None,'squat',:,:],:]           # This works

失败的例子：

dt_index = pd.date_range(start="1jan2019 00:15:33",periods=100,freq="h")
zf = pd.DataFrame(index=dt_index)
zf['foo']=10
zf['bar']="squat"
zf['zaa']=range(len(dt_index))
zf.index.name="date"
zf = zf.reset_index().set_index(["date", "bar", "zaa"])


zf.loc[idx['2019-1-1':'2019-1-3',"squat",:],:]     # This works ok
#zf.loc[idx['2019-1-1':,"squat",:],:]     # This fails  AttributeError: 'int' object has no attribute 'stop'
#zf.loc[idx['2019-1-1':None,'squat',:,:],:]           # AttributeError: 'int' object has no attribute 'stop'

Answer 1

这好像是个虫子。根据this discussion，检查pandas包multi.py的第2614-2637行：

            try:
                if key.start is not None:
                    start = level_index.get_loc(key.start)
                else:
                    start = 0
                if key.stop is not None:
                    stop = level_index.get_loc(key.stop)
                else:
                    stop = len(level_index) - 1
                step = key.step
            except KeyError:

                # we have a partial slice (like looking up a partial date
                # string)
                start = stop = level_index.slice_indexer(key.start, key.stop,
                                                         key.step, kind='loc')
                step = start.step

            if isinstance(start, slice) or isinstance(stop, slice):
                # we have a slice for start and/or stop
                # a partial date slicer on a DatetimeIndex generates a slice
                # note that the stop ALREADY includes the stopped point (if
                # it was a string sliced)
                return convert_indexer(start.start, stop.stop, step)

停止始终是int，因为端点是None。但是qf和xf的开始时间是不同的。qf的datetime\u索引的分辨率为1day，qf.index.levels[0].get_loc('2019-01-17')将是一个“int”。但是xf的分辨率是0.001S，xf.index.levels[0].get_loc('2019-01-17')将是slice，这导致了stop.stop的调用，stop是int。你知道吗

作为解决方法，您可以使用非常大的日期而不是None：

xf.loc[idx['2019-5-1':'2222',"squat",:],:]