<p>如果<code>filesystem</code>是<code>None</code>,则可以将内存中的文件对象传递给<code>write_to_dataset</code>。你知道吗</p>
<p>所以你的电话可能会变成:</p>
<pre><code>from io import BytesIO
import pyarrow.parquet as pq
with BytesIO() as f:
pq.write_to_dataset(
score_table,
root_path=f,
filesystem=None,
partition_cols=[
EQF_SNAPSHOT_YEAR_PARTITION,
EQF_SNAPSHOT_MONTH_PARTITION,
EQF_SNAPSHOT_DAY_PARTITION,
ZEBRA_COMPUTATION_TIMESTAMP
]
)
</code></pre>
<p>pyarrow来源的相关行:</p>
<pre><code>def resolve_filesystem_and_path(where, filesystem=None):
"""
Return filesystem from path which could be an HDFS URI, a local URI,
or a plain filesystem path.
"""
if not _is_path_like(where):
if filesystem is not None:
raise ValueError("filesystem passed but where is file-like, so"
" there is nothing to open with filesystem.")
return filesystem, where
</code></pre>
<p><a href="https://github.com/apache/arrow/blob/207b3507be82e92ebf29ec7d6d3b0bb86091c09a/python/pyarrow/filesystem.py#L402-L411" rel="nofollow noreferrer">https://github.com/apache/arrow/blob/207b3507be82e92ebf29ec7d6d3b0bb86091c09a/python/pyarrow/filesystem.py#L402-L411</a></p>