列表索引超出范围，但该项存在

Question

我得到这个错误，我不明白为什么。列表索引超出范围，但该项存在。你知道吗

1 [u'http://(ip1):(port1)', u'http://(ip2):(port2)']
Exception in thread Thread-11:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "(path).py", line 57, in match_fetcher
    self.fetch_match(match)
  File "(path).py", line 65, in fetch_match
    response = self.http_get(url)
  File "(path).py", line 75, in http_get
    proxy = self.proxies.get_proxy()
  File "(path).py", line 51, in get_proxy
    proxy = self.proxies[self.index]
IndexError: list index out of range

代码：

def get_proxy(self):

    if self.index >= len(self.proxies):
        self.index = 0
    print self.index, self.proxies
    proxy = self.proxies[self.index]
    self.index += 1
    return proxy

我很困惑。有什么问题吗？你知道吗

编辑：

You are using threads, is some other one manipulating the same data? – Thierry Lathuille

输出cat log | grep (proxy1)

0 [u'(proxy1):(port1)', u'(proxy2):(port2)']
1 [u'(proxy1):(port1)', u'(proxy2):(port2)']
0 [u'(proxy1):(port1)', u'(proxy2):(port2)']
1 [u'(proxy1):(port1)', u'(proxy2):(port2)']
0 [u'(proxy1):(port1)', u'(proxy2):(port2)']
1 [u'(proxy1):(port1)', u'(proxy2):(port2)']
0 [u'(proxy1):(port1)', u'(proxy2):(port2)']
1 [u'(proxy1):(port1)', u'(proxy2):(port2)']
0 [u'(proxy1):(port1)', u'(proxy2):(port2)']
1 [u'(proxy1):(port1)', u'(proxy2):(port2)']
0 [u'(proxy1):(port1)', u'(proxy2):(port2)']
1 [u'(proxy1):(port1)', u'(proxy2):(port2)']
0 [u'(proxy1):(port1)', u'(proxy2):(port2)']
1 [u'(proxy1):(port1)', u'(proxy2):(port2)']
0 [u'(proxy1):(port1)', u'(proxy2):(port2)']
1 [u'(proxy1):(port1)', u'(proxy2):(port2)']
0 [u'(proxy1):(port1)', u'(proxy2):(port2)']
(...)

Answer 1

看起来你有一个典型的线程同步问题，20个线程访问一个共享资源(自我代理以及自索引). 那些线程正在增加自索引一个，在if之后自索引>。。。选中，这将导致它超过列表的大小（索引>2）。你知道吗

你需要一些同步机制来“保护”你的共享资源。一个非常简单的是锁：

from threading import Lock

# at your init method
self.lock = threading.Lock()

def get_proxy(self):
    self.lock.acquire() # will block if lock is already held
     ... access shared resource
    # basically the entire method in your case
    self.lock.release()

我建议您阅读更多关于线程和同步的内容，这里有一个不错的教程： https://hackernoon.com/synchronization-primitives-in-python-564f89fee732