如何在Flask微框架中使用Neo4j-embedded的Python(多线程)?

4 投票
2 回答
1611 浏览
提问于 2025-04-17 05:35

我正在跟着Flask教程(Flaskr)来尝试在Python中使用Neo4j-embedded。这是在一个虚拟环境中进行的。以下是我的“主”应用代码:

import os
import jpype
from neo4j import GraphDatabase
from flask import Flask, request, session, g, redirect, url_for, abort, render_template, flash

app = Flask(__name__)
app.config.from_pyfile(os.environ['APP_SETTINGS'])


def connectDB(): 
    return GraphDatabase(app.config['DATABASE'])


def initDB():
    db = connectDB()

    with db.transaction:
        users = db.node()
        roles = db.node()

        db.reference_node.USERS(users)
        db.reference_node.ROLES(roles)

        userIndex = db.node.indexes.create('users')

        user = db.node(name=app.config['ADMIN'])
        user.INSTANCE_OF(users)
        userIndex['name'][app.config['ADMIN']] = user

        role = db.node(type='superadmin')
        role.INSTANCE_OF(roles)

        role.ASSIGN_TO(user)

    db.shutdown()

    print "Database initialized."


def testDB():
    db = connectDB()

    with db.transaction:
        userIndex = db.node.indexes.get('users')
        user = userIndex['name'][app.config['ADMIN']].single
        username = user['name']

    db.shutdown()

    print "Admin username is '%s'. Database exists." % username


@app.before_request
def before_request():
    jpype.attachThreadToJVM()
    g.db = connectDB()


@app.teardown_request
def teardown_request(exception):
    g.db.shutdown()


@app.route('/')
def index():

    with g.db.transaction:
        userIndex = g.db.node.indexes.get('users')
        user = userIndex['name'][app.config['ADMIN']].single
        username = user['name']

    fields = dict(username=username)
    return render_template('index.html', fields=fields)


if os.path.exists(app.config['DATABASE']) == False:
    initDB()
else:
    testDB()

initDB()和testDB()运行得很好——不需要Gremlin、PyLucene等,只用jpype和neo4j-embedded。最开始,当我请求index()时,JVM会失败,应用程序会终止。我在网上查了很多资料,了解到我需要在before_request()中添加“jpype.attachThreadToJVM()”这一行代码,这样可以解决Python线程和JVM之间的问题,应用程序就不会终止了。然而,这又引出了另一个问题:

Traceback (most recent call last):
  File "/ht/dev/envFlask/lib/python2.7/site-packages/flask/app.py", line 1518, in __call__
    return self.wsgi_app(environ, start_response)
  File "/ht/dev/envFlask/lib/python2.7/site-packages/flask/app.py", line 1506, in wsgi_app
    response = self.make_response(self.handle_exception(e))
  File "/ht/dev/envFlask/lib/python2.7/site-packages/flask/app.py", line 1504, in wsgi_app
    response = self.full_dispatch_request()
  File "/ht/dev/envFlask/lib/python2.7/site-packages/flask/app.py", line 1264, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/ht/dev/envFlask/lib/python2.7/site-packages/flask/app.py", line 1262, in full_dispatch_request
    rv = self.dispatch_request()
  File "/ht/dev/envFlask/lib/python2.7/site-packages/flask/app.py", line 1248, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/ht/dev/apps/evobox/evobox/__init__.py", line 68, in index
    userIndex = g.db.node.indexes.get('users')
  File "/ht/dev/envFlask/lib/python2.7/site-packages/neo4j/index.py", line 36, in get
    return self._index.forNodes(name)
java.lang.RuntimeExceptionPyRaisable: java.lang.IllegalArgumentException: No index provider 'lucene' found. Maybe the intended provider (or one more of its dependencies) aren't on the classpath or it failed to load.

我在谷歌上搜索了最后一行的内容,但没有找到任何有用的信息。仅仅搜索“java.lang.IllegalArgumentException: No index provider 'lucene' found.”在Python的上下文中也没有找到相关内容。

neo4j的messages.log似乎显示数据库被打开了三次(initDB()、testDB()和index())。每个实例的classpath都是一样的:

Class Path: /ht/dev/envFlask/local/lib/python2.7/site-packages/neo4j/javalib/neo4j-jmx-1.5.M02.jar:/ht/dev/envFlask/local/lib/python2.7/site-packages/neo4j/javalib/neo4j-lucene-index-1.5.M02.jar:/ht/dev/envFlask/local/lib/python2.7/site-packages/neo4j/javalib/neo4j-graph-matching-1.5.M02.jar:/ht/dev/envFlask/local/lib/python2.7/site-packages/neo4j/javalib/neo4j-1.5.M02.jar:/ht/dev/envFlask/local/lib/python2.7/site-packages/neo4j/javalib/neo4j-kernel-1.5.M02.jar:/ht/dev/envFlask/local/lib/python2.7/site-packages/neo4j/javalib/geronimo-jta_1.1_spec-1.1.1.jar:/ht/dev/envFlask/local/lib/python2.7/site-packages/neo4j/javalib/lucene-core-3.1.0.jar:/ht/dev/envFlask/local/lib/python2.7/site-packages/neo4j/javalib/neo4j-graph-algo-1.5.M02.jar:/ht/dev/envFlask/local/lib/python2.7/site-packages/neo4j/javalib/neo4j-udc-1.5.M02.jar:/ht/dev/envFlask/local/lib/python2.7/site-packages/neo4j/javalib/neo4j-cypher-1.5.M02.jar:/ht/dev/envFlask/local/lib/python2.7/site-packages/neo4j/javalib/scala-library-2.9.0-1.jar

我还修改了index(),让它像initDB()和testDB()那样直接连接数据库并附加线程到JVM,而不使用全局变量'g'——结果还是出现了完全相同的错误。

我可能遗漏了什么,或者有什么地方没有注意到,以便让neo4j-embedded和jpype在多线程请求中正常工作,而不仅仅是在“主”应用中?

注意:我知道有使用py2neo或Rexster/Bulbs的RESTful Web Service解决方案,但我现在想避免使用这些。

编辑:使用的是JPype-0.5.4.2,Neo4j-embedded-1.5.b2,Java-6-openjdk

2 个回答

3

我尝试重现这个错误,使用了全新安装的neo4j、flask和jpipe,并且用了标准的Java默认路径。但是我没有找到那个日志。

不过我在使用 initDB 的时候遇到了一个错误:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "teststack.py", line 27, in initDB
    userIndex = db.node.indexes.create('users')
 File "/usr/local/lib/python2.7/dist-packages/neo4j/index.py", line 32, in create
   return self._index.forNodes(name, to_java(config))
 jpype._jexception.RuntimeExceptionPyRaisable: java.lang.IllegalArgumentException:      Supplied index configuration:
{}
doesn't match stored config in a valid way:
{provider=lucene, type=exact}
for 'users'

这个错误是:

我用下面的方式解决了这个问题:

userIndex = db.node.indexes.create(users)
6

这个模式有一个问题,就是你可能会不小心启动多个数据库指向同一个位置,这样会导致一些麻烦。你其实想要的是一个数据库实例,它能够跟随你应用的整个生命周期。

为什么找不到lucene提供者这个问题就复杂了。索引提供者是通过Java的服务加载器来加载的,这意味着JPype应该不会影响它。只要JVM启动正常,并且lucene-index的实现jar文件在类路径上,它就应该能正常工作。

这可能和线程有关。我现在正在写一个修复程序,自动处理“attachThreadToJVM()”的调用。我还会添加一个测试用例,以确保从一个单独的线程读取索引能够按预期工作。

关于线程的工作目前在这个邮件列表的讨论中保持更新:

http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-Python-embedding-problems-with-shutdown-and-threads-td3476163.html

撰写回答