Python 可重置的实例方法缓存装饰器

Question

我正在尝试为一个类的实例方法创建一个装饰器，这个装饰器可以记住方法的结果。（这个事情以前做过很多次了）不过，我希望能在任何时候重置这个记住的缓存，比如说，当实例的状态发生变化时，这可能会影响方法的结果，而这些变化和方法的参数没有关系。所以，我尝试把装饰器做成一个类，而不是一个函数，这样我就可以把缓存作为类的成员来访问。这让我开始学习描述符，特别是 __get__ 方法，但我在这里遇到了困难。我的代码是这样的：

import time

class memoized(object):

    def __init__(self, func):
        self.func = func
        self.cache = {}

    def __call__(self, *args, **kwargs):

        key = (self.func, args, frozenset(kwargs.iteritems()))

        try:
            return self.cache[key]
        except KeyError:
            self.cache[key] = self.func(*args, **kwargs)
            return self.cache[key]
        except TypeError:
            # uncacheable, so just return calculated value without caching
            return self.func(*args, **kwargs)

    # self == instance of memoized
    # obj == instance of my_class
    # objtype == class object of __main__.my_class
    def __get__(self, obj, objtype=None):
        """Support instance methods"""
        if obj is None:
            return self

        # new_func is the bound method my_func of my_class instance
        new_func = self.func.__get__(obj, objtype)

        # instantiates a brand new class...this is not helping us, because it's a 
        # new class each time, which starts with a fresh cache
        return self.__class__(new_func)

    # new method that will allow me to reset the memoized cache
    def reset(self):
        print "IN RESET"
        self.cache = {}

class my_class:
    @memoized
    def my_func(self, val):
        print "in my_func"
        time.sleep(2)
        return val


c = my_class()

print "should take time"
print c.my_func(55)
print

print "should be instant"
print c.my_func(55)
print

c.my_func.reset()

print "should take time"
print c.my_func(55)

这样做清楚吗？或者说可能吗？每次调用 __get__ 时，我都会得到一个全新的记忆类的实例，这样我就失去了里面实际数据的缓存。我一直在努力使用 __get__，但进展不大。

有没有其他完全不同的方法可以解决这个问题，而我却没有想到？任何建议或意见都非常欢迎和感激。谢谢。

Answer 1

我想指出你代码中的两个性能问题。这不是对你问题的回答，但我无法把它当作评论。感谢@delnan指出has_key已经不推荐使用了。你可以用以下方式替代：

    try:
        return self.cache[key]
    except KeyError:
        self.cache[key] = self.func(*args, **kwargs)
        return self.cache[key]
    except TypeError:
        # uncacheable, so just return calculated value without caching
        return self.func(*args, **kwargs)

我建议这样写：

resultDone = False
result = None
try:
  if key in self.cache: return self.cache[key]
  else:
    result = self.func(*args, **kwargs)
    resultDone = True
    self.cache[key] = result
except TypeError: # unhashable key
  pass
if resultDone: return result
else: return self.func(*args, **kwargs)

这样做可以避免：a) 处理KeyError的错误；b) 在返回时调用cache[key]；c) 对无法哈希的键再次调用函数。

Answer 2

在@aix给出的原始问题的回答基础上，我创建了一个类，我觉得这个类可以让它更好。这个类的主要特点是，缓存的值被存储为正在被装饰的方法所在实例的一个属性，因此重置这些值非常简单。

class memoize(object):
  def __init__(self, func):
    #print "Init"
    self.func = func

  def __call__(self, *args):
    #print "Call"
    if not self.func in self.cache:
        self.cache[self.func] = {}
    try:
        return self.cache[self.func][args]
    except KeyError:
        value = self.func(*args)
        self.cache[self.func][args] = value
        return value
    except TypeError:
        # uncachable -- for instance, passing a list as an argument.
        # Better to not cache than to blow up entirely.
        return self.func(*args)

  def __repr__(self):
    """Return the function's docstring."""
    return self.func.__doc__

  def __get__(self, obj, objtype):
    """Support instance methods."""
    #print "Get", obj, objtype
    fn = functools.partial(self.__call__, obj)
    try:
        self.cache = obj.cache
    except:
        obj.cache = {}
        self.cache = obj.cache
    #print self.cache
    return fn

使用示例：

class MyClass(object):
    def __init__(self,data):
        self.data = data

    def update(self,data):
        self.data = data
        self.cache = {}

    @memoize
    def func1(self,x):
        print "Computing func1"
        return "I am func1 of %s. Data is %s. x is %s\n" % (self, self.data, x)

    @memoize
    def func2(self,x):
        print "Computing func2"
        return "I am func2 of %s. Data is %s. x is %s\n" % (self, self.data, x)

    def func3(self,x):
        print "Computing func3"
        return "I am func3 of %s. Data is %s. x is %s\n" % (self, self.data, x)

mc1 = MyClass("data1")
mc2 = MyClass("data2")
mc3 = MyClass("data3")

print mc1.func1(1) 
print mc1.func1(1) 
print mc1.func2(1) 
print mc1.func2(1) 
print mc1.func3(1) 
print mc1.func3(1) 

print mc2.func1(1) 
print mc2.func1(1) 
print mc2.func2(1) 
print mc2.func2(1) 
print mc2.func3(1) 
print mc2.func3(1) 

print "Update mc1\n"
mc1.update("data1new")

print mc1.func1(1) 
print mc1.func2(1) 
print mc1.func3(1) 
print mc2.func1(1) 
print mc2.func2(1) 
print mc2.func3(1) 

输出结果是：

Computing func1
I am func1 of <__main__.MyClass object at 0x100470fd0>. Data is data1. x is 1

I am func1 of <__main__.MyClass object at 0x100470fd0>. Data is data1. x is 1

Computing func2
I am func2 of <__main__.MyClass object at 0x100470fd0>. Data is data1. x is 1

I am func2 of <__main__.MyClass object at 0x100470fd0>. Data is data1. x is 1

Computing func3
I am func3 of <__main__.MyClass object at 0x100470fd0>. Data is data1. x is 1

Computing func3
I am func3 of <__main__.MyClass object at 0x100470fd0>. Data is data1. x is 1

Computing func1
I am func1 of <__main__.MyClass object at 0x100476050>. Data is data2. x is 1

I am func1 of <__main__.MyClass object at 0x100476050>. Data is data2. x is 1

Computing func2
I am func2 of <__main__.MyClass object at 0x100476050>. Data is data2. x is 1

I am func2 of <__main__.MyClass object at 0x100476050>. Data is data2. x is 1

Computing func3
I am func3 of <__main__.MyClass object at 0x100476050>. Data is data2. x is 1

Computing func3
I am func3 of <__main__.MyClass object at 0x100476050>. Data is data2. x is 1

Update mc1

Computing func1
I am func1 of <__main__.MyClass object at 0x100470fd0>. Data is data1new. x is 1

Computing func2
I am func2 of <__main__.MyClass object at 0x100470fd0>. Data is data1new. x is 1

Computing func3
I am func3 of <__main__.MyClass object at 0x100470fd0>. Data is data1new. x is 1

I am func1 of <__main__.MyClass object at 0x100476050>. Data is data2. x is 1

I am func2 of <__main__.MyClass object at 0x100476050>. Data is data2. x is 1

Computing func3
I am func3 of <__main__.MyClass object at 0x100476050>. Data is data2. x is 1

Answer 3

与其费力去弄清楚你实现的具体细节，我从PythonDecoratorLibrary中拿到了一个叫做memoized的装饰器类，并对它进行了修改，增加了reset功能。下面是修改后的结果；我用的技巧是给被装饰的函数本身添加一个可以调用的reset属性。

    class memoized2(object):
       """Decorator that caches a function's return value each time it is called.
       If called later with the same arguments, the cached value is returned, and
       not re-evaluated.
       """
       def __init__(self, func):
          self.func = func
          self.cache = {}
       def __call__(self, *args):
          try:
             return self.cache[args]
          except KeyError:
             value = self.func(*args)
             self.cache[args] = value
             return value
          except TypeError:
             # uncachable -- for instance, passing a list as an argument.
             # Better to not cache than to blow up entirely.
             return self.func(*args)
       def __repr__(self):
          """Return the function's docstring."""
          return self.func.__doc__
       def __get__(self, obj, objtype):
          """Support instance methods."""
          fn = functools.partial(self.__call__, obj)
          fn.reset = self._reset
          return fn
       def _reset(self):
          self.cache = {}


    class my_class:
        @memoized2
        def my_func(self, val):
            print "in my_func"
            time.sleep(2)
            return val


    c = my_class()

    print "should take time"
    print c.my_func(55)
    print

    print "should be instant"
    print c.my_func(55)
    print

    c.my_func.reset()

    print "should take time"
    print c.my_func(55)