“OSError:[Errno 127]密钥已过期”df.to_csv方法

2024-04-20 12:59:21 发布

您现在位置:Python中文网/ 问答频道 /正文

作为multiprocessing进程的一部分运行以下df.to_csv()行时:

filtered_chunk.to_csv(os.path.join(output_dir, "filtered_{}".format(csv_filename)), mode='a+')

我得到以下错误。我不知道问题出在哪里(下面的代码片段是针对一个multiprocessing进程的):

Process ForkPoolWorker-13:
Traceback (most recent call last):
  File "/home/User1/miniconda3/envs/part_ii_dev-conda/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
  File "/home/User1/miniconda3/envs/part_ii_dev-conda/lib/python3.8/multiprocessing/process.py", line 108, in run
  File "/home/User1/miniconda3/envs/part_ii_dev-conda/lib/python3.8/multiprocessing/pool.py", line 109, in worker
  <insert stacktrace for code I wrote...>
    filtered_chunk.to_csv(os.path.join(output_dir, "filtered_{}".format(csv_filename)), mode='a+')
  File "/home/User1/miniconda3/envs/part_ii_dev-conda/lib/python3.8/site-packages/pandas/core/generic.py", line 3170, in to_csv
  File "/home/User1/miniconda3/envs/part_ii_dev-conda/lib/python3.8/site-packages/pandas/io/formats/csvs.py", line 185, in save
  File "/home/User1/miniconda3/envs/part_ii_dev-conda/lib/python3.8/site-packages/pandas/io/common.py", line 418, in get_handle
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 914, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1342, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1314, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1443, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1483, in _fill_cache
OSError: [Errno 127] Key has expired: '/home/User1/miniconda3/envs/part_ii_dev-conda/lib/python3.8'

在上面的stacktrace的每个进程都失败之后,我会得到一个Bus error (core dumped)打印输出

“密钥已过期”让我想到一些需要验证或提供有时间限制的访问(文件指针?访问服务器的密钥/票据?)。我在服务器上的tmux会话中运行了脚本,因此我认为访问服务器不会是一个问题(一些错误127的谷歌搜索指出了装载问题,但我不认为这是一个问题)。由于脚本在get_handle方法中失败,基本上返回一个文件句柄,所以文件指针似乎是问题所在,但我不确定原因

如果有帮助的话,我正在使用multiprocessing来并行化一个函数,但是不同的进程写入不同的csv文件,所以这在这里不应该是一个问题


Tags: csvindevhomeliblinebootstrapconda