使用Python子进程记录和追踪获取简洁错误信息

Question

我正在写一个自动化程序，希望它能处理错误并收集信息。在我现在的设置下，我得到了一次循环实例的以下日志记录：

2024-03-21 10:14:44,843 - ERROR - error processing shot '20221201.033':
Traceback (most recent call last):
 File "/home/Desktop/Code/git/populate_database.py", line 76, in <module>
    raise Exception(f"n{stderr.decode()}")
Exception: n
 Fetching parlog information for shot 20221201.033 from archive: 0%|    | 0/5 [00:00<?, ?it/s]
 Fetching parlog information for shot 20221201.033 from archive: 0%|    | 0/5 [00:00<?, ?it/s]

     Mapping rates : 0%|    | 0/10 [00:00<?, ?it/s]
     Mapping rates : 100%|██████████| 10/10 [00:00<00:00, 3360.28it/s]

     Calculating impurity densities : 0%|   | 0/10 [00:00<?, ?it/s]
     Calculating impurity densities : 80%|████████ | 8/10 [00:00<00:00, 73.92it/s]
     Calculating impurity densities : 100%|██████████| 10/10 [00:00<00:00, 73.91it/s]

     Updating profile: 0%|      | 0/10 [00:00<?, ?it/s]
     Updating profile: 20%|██   | 2/10 [00:00<00:00, 8.86it/s]
     Updating profile: 100%|██████████| 10/10 [00:00<00:00, 34.81it/s]
     Updating profile: 100%|██████████| 10/10 [00:00<00:00, 29.59it/s]

         Assessing convergence: 0%|     | 0/10 [00:00<?, ?it/s]
         Assessing convergence: 100%|██████████| 10/10 [00:00<00:00, 84054.19it/s]
Traceback (most recent call last):
 File "/home/Desktop/Code/git/Examples/Program_runfiles/20221201.033.py", line 37, in <module>
    data.run(
 File "/home/Desktop/Code/git/eval.py", line 348, in run
    self.run_post_methods()
 File "/home/Desktop/Code/git/eval.py", line 469, in run_post_methods
    _store_grad_length(
 File "/home/Desktop/Code/git/utilities/write_locally.py", line 103, in _store_grad_length
    dictionary_added["tau_e"] = _get_energy_confinement_time(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/home/Desktop/Code/git/utilities/fetch_data.py", line 293, in _get_energy_confinement_time
    w_dia_spline = UnivariateSpline(t_w_dia, w_dia, k=4, s=0)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/home/anaconda3/lib/python3.11/site-packages/scipy/interpolate/_fitpack2.py", line 236, in __init__
    data = dfitpack.fpcurf0(x, y, k, w=w, xb=bbox[0],
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dfitpack.error: (m>k) failed for hidden m: fpcurf0:m=0

在上面的完整控制台日志中，我只对包含错误的以下部分感兴趣：

Traceback (most recent call last):
 File "/home/Desktop/Code/git/Examples/Program_runfiles/20221201.033.py", line 37, in <module>
    data.run(
 File "/home/Desktop/Code/git/eval.py", line 348, in run
    self.run_post_methods()
 File "/home/Desktop/Code/git/eval.py", line 469, in run_post_methods
    _store_grad_length(
 File "/home/Desktop/Code/git/utilities/write_locally.py", line 103, in _store_grad_length
    dictionary_added["tau_e"] = _get_energy_confinement_time(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/home/Desktop/Code/git/utilities/fetch_data.py", line 293, in _get_energy_confinement_time
    w_dia_spline = UnivariateSpline(t_w_dia, w_dia, k=4, s=0)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/home/anaconda3/lib/python3.11/site-packages/scipy/interpolate/_fitpack2.py", line 236, in __init__
    data = dfitpack.fpcurf0(x, y, k, w=w, xb=bbox[0],
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dfitpack.error: (m>k) failed for hidden m: fpcurf0:m=0

我现在的代码使用了以下的 try except 方法：

from subprocess import Popen, PIPE, STDOUT, DEVNULL

logging.basicConfig(filename='error.log', level=logging.ERROR, format='%(asctime)s - %(levelname)s - %(message)s')

 try:
    # run the file for the shot with error handling
    process = Popen(["python", dest_file], stdout=DEVNULL, stderr=PIPE)

    # wait while file is running
    process.wait()
     
    # get the console error of the subprocess
    stderr = process.communicate()[1]

    # close all figures and clear console
    plt.close()
    os.system('clear')
     
    # if we encountered an error raise it
    if stderr:
    raise Exception(f"n{stderr.decode()}")
 
    # except the raised error and log the error
    except Exception as e:
    # Log the shot name along with the traceback info
    logging.error(f"error processing shot '{shot_name}':", exc_info=True)

我该如何筛选出相关部分并记录下来呢？

Answer 1

看起来你只对Python的错误信息感兴趣，而这个信息通常是最后一段打印出来的。你可以使用 rpartition 这个方法，把文本在最后一次出现的 Traceback (most recent call last): 这个标题处分开。

if stderr:
    error = stderr.decode()
    error = error.rpartition("Traceback (most recent call last):\n")[-1]
    raise Exception(error)