如何在Flask微框架中使用Neo4j-embedded的Python(多线程)?
我正在跟着Flask教程(Flaskr)来尝试在Python中使用Neo4j-embedded。这是在一个虚拟环境中进行的。以下是我的“主”应用代码:
import os
import jpype
from neo4j import GraphDatabase
from flask import Flask, request, session, g, redirect, url_for, abort, render_template, flash
app = Flask(__name__)
app.config.from_pyfile(os.environ['APP_SETTINGS'])
def connectDB():
return GraphDatabase(app.config['DATABASE'])
def initDB():
db = connectDB()
with db.transaction:
users = db.node()
roles = db.node()
db.reference_node.USERS(users)
db.reference_node.ROLES(roles)
userIndex = db.node.indexes.create('users')
user = db.node(name=app.config['ADMIN'])
user.INSTANCE_OF(users)
userIndex['name'][app.config['ADMIN']] = user
role = db.node(type='superadmin')
role.INSTANCE_OF(roles)
role.ASSIGN_TO(user)
db.shutdown()
print "Database initialized."
def testDB():
db = connectDB()
with db.transaction:
userIndex = db.node.indexes.get('users')
user = userIndex['name'][app.config['ADMIN']].single
username = user['name']
db.shutdown()
print "Admin username is '%s'. Database exists." % username
@app.before_request
def before_request():
jpype.attachThreadToJVM()
g.db = connectDB()
@app.teardown_request
def teardown_request(exception):
g.db.shutdown()
@app.route('/')
def index():
with g.db.transaction:
userIndex = g.db.node.indexes.get('users')
user = userIndex['name'][app.config['ADMIN']].single
username = user['name']
fields = dict(username=username)
return render_template('index.html', fields=fields)
if os.path.exists(app.config['DATABASE']) == False:
initDB()
else:
testDB()
initDB()和testDB()运行得很好——不需要Gremlin、PyLucene等,只用jpype和neo4j-embedded。最开始,当我请求index()时,JVM会失败,应用程序会终止。我在网上查了很多资料,了解到我需要在before_request()中添加“jpype.attachThreadToJVM()”这一行代码,这样可以解决Python线程和JVM之间的问题,应用程序就不会终止了。然而,这又引出了另一个问题:
Traceback (most recent call last):
File "/ht/dev/envFlask/lib/python2.7/site-packages/flask/app.py", line 1518, in __call__
return self.wsgi_app(environ, start_response)
File "/ht/dev/envFlask/lib/python2.7/site-packages/flask/app.py", line 1506, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/ht/dev/envFlask/lib/python2.7/site-packages/flask/app.py", line 1504, in wsgi_app
response = self.full_dispatch_request()
File "/ht/dev/envFlask/lib/python2.7/site-packages/flask/app.py", line 1264, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/ht/dev/envFlask/lib/python2.7/site-packages/flask/app.py", line 1262, in full_dispatch_request
rv = self.dispatch_request()
File "/ht/dev/envFlask/lib/python2.7/site-packages/flask/app.py", line 1248, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/ht/dev/apps/evobox/evobox/__init__.py", line 68, in index
userIndex = g.db.node.indexes.get('users')
File "/ht/dev/envFlask/lib/python2.7/site-packages/neo4j/index.py", line 36, in get
return self._index.forNodes(name)
java.lang.RuntimeExceptionPyRaisable: java.lang.IllegalArgumentException: No index provider 'lucene' found. Maybe the intended provider (or one more of its dependencies) aren't on the classpath or it failed to load.
我在谷歌上搜索了最后一行的内容,但没有找到任何有用的信息。仅仅搜索“java.lang.IllegalArgumentException: No index provider 'lucene' found.”在Python的上下文中也没有找到相关内容。
neo4j的messages.log似乎显示数据库被打开了三次(initDB()、testDB()和index())。每个实例的classpath都是一样的:
Class Path: /ht/dev/envFlask/local/lib/python2.7/site-packages/neo4j/javalib/neo4j-jmx-1.5.M02.jar:/ht/dev/envFlask/local/lib/python2.7/site-packages/neo4j/javalib/neo4j-lucene-index-1.5.M02.jar:/ht/dev/envFlask/local/lib/python2.7/site-packages/neo4j/javalib/neo4j-graph-matching-1.5.M02.jar:/ht/dev/envFlask/local/lib/python2.7/site-packages/neo4j/javalib/neo4j-1.5.M02.jar:/ht/dev/envFlask/local/lib/python2.7/site-packages/neo4j/javalib/neo4j-kernel-1.5.M02.jar:/ht/dev/envFlask/local/lib/python2.7/site-packages/neo4j/javalib/geronimo-jta_1.1_spec-1.1.1.jar:/ht/dev/envFlask/local/lib/python2.7/site-packages/neo4j/javalib/lucene-core-3.1.0.jar:/ht/dev/envFlask/local/lib/python2.7/site-packages/neo4j/javalib/neo4j-graph-algo-1.5.M02.jar:/ht/dev/envFlask/local/lib/python2.7/site-packages/neo4j/javalib/neo4j-udc-1.5.M02.jar:/ht/dev/envFlask/local/lib/python2.7/site-packages/neo4j/javalib/neo4j-cypher-1.5.M02.jar:/ht/dev/envFlask/local/lib/python2.7/site-packages/neo4j/javalib/scala-library-2.9.0-1.jar
我还修改了index(),让它像initDB()和testDB()那样直接连接数据库并附加线程到JVM,而不使用全局变量'g'——结果还是出现了完全相同的错误。
我可能遗漏了什么,或者有什么地方没有注意到,以便让neo4j-embedded和jpype在多线程请求中正常工作,而不仅仅是在“主”应用中?
注意:我知道有使用py2neo或Rexster/Bulbs的RESTful Web Service解决方案,但我现在想避免使用这些。
编辑:使用的是JPype-0.5.4.2,Neo4j-embedded-1.5.b2,Java-6-openjdk
2 个回答
我尝试重现这个错误,使用了全新安装的neo4j、flask和jpipe,并且用了标准的Java默认路径。但是我没有找到那个日志。
不过我在使用 initDB
的时候遇到了一个错误:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "teststack.py", line 27, in initDB
userIndex = db.node.indexes.create('users')
File "/usr/local/lib/python2.7/dist-packages/neo4j/index.py", line 32, in create
return self._index.forNodes(name, to_java(config))
jpype._jexception.RuntimeExceptionPyRaisable: java.lang.IllegalArgumentException: Supplied index configuration:
{}
doesn't match stored config in a valid way:
{provider=lucene, type=exact}
for 'users'
这个错误是:
我用下面的方式解决了这个问题:
userIndex = db.node.indexes.create(users)
这个模式有一个问题,就是你可能会不小心启动多个数据库指向同一个位置,这样会导致一些麻烦。你其实想要的是一个数据库实例,它能够跟随你应用的整个生命周期。
为什么找不到lucene提供者这个问题就复杂了。索引提供者是通过Java的服务加载器来加载的,这意味着JPype应该不会影响它。只要JVM启动正常,并且lucene-index的实现jar文件在类路径上,它就应该能正常工作。
这可能和线程有关。我现在正在写一个修复程序,自动处理“attachThreadToJVM()”的调用。我还会添加一个测试用例,以确保从一个单独的线程读取索引能够按预期工作。
关于线程的工作目前在这个邮件列表的讨论中保持更新: