使用ScyllaDB进行Python添加数据是否可能更高效?

2024-04-27 09:45:58 发布

您现在位置:Python中文网/ 问答频道 /正文

我尝试在python中使用scyllab,但是速度很慢。当我运行下面显示的示例代码时,我得到:

26:23:109998
26:23:112695

我关心的是最好的性能,不幸的是,这次向数据库添加数据的时间肯定太长了。有没有办法加快这个过程?在

^{pr2}$

更新

在这个主题中,我决定根据官方文件使用准备好的报表和批处理来提高向“锡拉”添加数据的性能。我的代码目前看起来像下面所示的那样,但是效率并没有显著变化。还有别的主意吗?在

print("time 0: " + str(datetime.now()))
query = "INSERT INTO message (id, message) VALUES (uuid(), ?)"
prepared = session.prepare(query)

for key in range(100):

    print(key)

    try:

        batch = BatchStatement(consistency_level=ConsistencyLevel.QUORUM)
        for key in range(100):

            batch.add(prepared, ("example message",))

        session.execute(batch)

    except Exception as e:
        print("An error occured : " + str(e))
        pass

print("time 1: " + str(datetime.now()))

运行此源代码后,结果如下所示:

test 0: 2018-06-19 11:10:13.990691
0
1
...
41
cAn error occured : Error from server: code=1100 [Coordinator node timed out waiting for replica nodes' responses] message="Operation timed out for messages.message - received only 1 responses from 2 CL=QUORUM." info={'write_type': 'BATCH', 'required_responses': 2, 'consistency': 'QUORUM', 'received_responses': 1}
42
...
52                                                                                                                                                                             An error occured : errors={'....0.3': 'Client request timeout. See Session.execute[_async](timeout)'}, last_host=.....0.3
53
An error occured : Error from server: code=1100 [Coordinator node timed out waiting for replica nodes' responses] message="Operation timed out for messages.message - received only 1 responses from 2 CL=QUORUM." info={'write_type': 'BATCH', 'required_responses': 2, 'consistency': 'QUORUM', 'received_responses': 1}
54
...
59
An error occured : Error from server: code=1100 [Coordinator node timed out waiting for replica nodes' responses] message="Operation timed out for messages.message - received only 1 responses from 2 CL=QUORUM." info={'write_type': 'BATCH', 'required_responses': 2, 'consistency': 'QUORUM', 'received_responses': 1}
60
61
62
...
69
70
71
An error occured : errors={'.....0.2': 'Client request timeout. See Session.execute[_async](timeout)'}, last_host=.....0.2
72
An error occured : errors={'....0.1': 'Client request timeout. See Session.execute[_async](timeout)'}, last_host=....0.1
73
74
...
98
99
test 1: 2018-06-19 11:11:03.494957

Tags: fromanmessageforexecutetimeouterrorresponses
2条回答

从使用准备好的语句开始,然后并行执行多个语句。在

有几个因素会限制你的表现。从锡拉服务器配置开始。例如,如果您创建了一个具有非常小的慢速网络实例的集群。继续,在实例本身上使用客户端HW和工作负载,同时考虑每个主机的连接数、每个连接的线程数以及驱动程序/连接器端的其他可调参数。最后,使用更有效的方法,使用事先准备好的语句将信息写入锡拉。在

更多地了解您正在使用的环境和工作负载的用途,以便建议更具体的操作过程,这将很有帮助。在

相关问题 更多 >