如何用Python防止H2O集群无故关闭

Question

我的代码是用来加载h2o MOJO模型，以便对一个小数据集进行预测。然而，h2o却自己突然关闭了。相同的代码在一台机器上运行得很好，输入也一样，但在另一台机器上却出现了异常关闭的情况。

self.test = h2o.import_file(dataset_file)
preds = imported_model.predict(self.test)

我是在一台有1TB内存和72个核心的机器上运行这个程序。我不相信这是内存的问题。最让人困惑的是，相同的代码在另一台配置不同的机器上却能正常工作。我不知道具体有哪些不同之处。我之前是用Python的冻结版本运行的，没法详细看到错误信息。现在我直接运行Python代码，可以看到更详细的错误信息。

 File "h2o_model_eval.py", line 160, in ModelEval
    preds = imported_model.predict(self.test)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.11/site-packages/h2o/model/model_base.py", line 334, in predict
    j.poll()
  File ".venv/lib/python3.11/site-packages/h2o/job.py", line 71, in poll
    pb.execute(self._refresh_job_status)
  File ".venv/lib/python3.11/site-packages/h2o/utils/progressbar.py", line 187, in execute
    res = progress_fn()  # may raise StopIteration
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.11/site-packages/h2o/job.py", line 136, in _refresh_job_status
    jobs = self._query_job_status_safe()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.11/site-packages/h2o/job.py", line 132, in _query_job_status_safe
    raise last_err
  File ".venv/lib/python3.11/site-packages/h2o/job.py", line 114, in _query_job_status_safe
    result = h2o.api("GET /3/Jobs/%s" % self.job_key)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.11/site-packages/h2o/h2o.py", line 123, in api
    return h2oconn.request(endpoint, data=data, json=json, filename=filename, save_to=save_to)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.11/site-packages/h2o/backend/connection.py", line 507, in request
    raise H2OConnectionError("Unexpected HTTP error: %s" % e)
h2o.exceptions.H2OConnectionError: Unexpected HTTP error: HTTPConnectionPool(host='localhost', port=54321): Max retries exceeded with url: /3/Jobs/$03017f00000132d4ffffffff$_acab67512114e05db6ec9865ea9849d3 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x2aab3f59ea90>: Failed to establish a new connection: [Errno 111] Connection refused'))
~

我该如何调试这个问题呢？

内存管理代码调试错误信息 h2o集群 mojos模型异常关闭机器配置

如何用Python防止H2O集群无故关闭

1 个回答

撰写回答