Beam mongoio DisplayDataItem值{}的类型不受支持

Question

当我试图通过

         | "Write User Doc to Mongo" >> beam.io.WriteToMongoDB(uri=MONGO_URI,
                                                               db="dbname",
                                                               coll="col_name"
                                                               ))

错误ValueError: Invalid DisplayDataItem. Value {} is of an unsupported type.

ERROR:root:Error while visiting Write User Doc to Mongo/ParDo(_WriteMongoFn)
Traceback (most recent call last):
  File "beam_home.py", line 317, in <module>
    run()
  File "beam_home.py", line 312, in run
    p.run().wait_until_finish()
  File "/usr/local/lib/python2.7/site-packages/apache_beam/pipeline.py", line 406, in run
    self._options).run(False)
  File "/usr/local/lib/python2.7/site-packages/apache_beam/pipeline.py", line 419, in run
    return self.runner.run_pipeline(self, self._options)
  File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/dataflow/dataflow_runner.py", line 469, in run_pipeline
    super(DataflowRunner, self).run_pipeline(pipeline, options)
  File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/runner.py", line 158, in run_pipeline
    pipeline.visit(RunVisitor(self))
  File "/usr/local/lib/python2.7/site-packages/apache_beam/pipeline.py", line 447, in visit
    self._root_transform().visit(visitor, self, visited)
  File "/usr/local/lib/python2.7/site-packages/apache_beam/pipeline.py", line 824, in visit
    part.visit(visitor, pipeline, visited)
  File "/usr/local/lib/python2.7/site-packages/apache_beam/pipeline.py", line 824, in visit
    part.visit(visitor, pipeline, visited)
  File "/usr/local/lib/python2.7/site-packages/apache_beam/pipeline.py", line 827, in visit
    visitor.visit_transform(self)
  File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/runner.py", line 153, in visit_transform
    self.runner.run_transform(transform_node, options)
  File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/runner.py", line 196, in run_transform
    return m(transform_node, options)
  File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/dataflow/dataflow_runner.py", line 807, in run_ParDo
    transform_node.transform.output_tags)
  File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/dataflow/dataflow_runner.py", line 590, in _add_step
    DisplayData.create_from(transform_node.transform).items])
  File "/usr/local/lib/python2.7/site-packages/apache_beam/transforms/display.py", line 274, in get_dict
    self.is_valid()
  File "/usr/local/lib/python2.7/site-packages/apache_beam/transforms/display.py", line 246, in is_valid
    .format(self.value))
ValueError: Invalid DisplayDataItem. Value {} is of an unsupported type.

它似乎是由这个codes引起的

    if extra_client_params is None:
      extra_client_params = {}
    if not isinstance(db, str):

似乎我必须设置extra_client_params的WriteToMongoDB。这是默认行为还是我遗漏了什么？你知道吗

Answer 1

不幸的是，这是一个错误。在2.16.0中有一个补丁，应该在接下来的几周内发布。你知道吗

主要问题就在这里：https://github.com/apache/beam/blob/v2.15.0/sdks/python/apache_beam/io/mongodbio.py#L472

显示数据不支持字典类型的参数，我们正在传递规范。若要解决此错误，您必须在本地Beam代码中重写此行为（很遗憾：（-或等待几周以获取2.16.0:）