2024-05-20 00:54:46 发布
网友
如何使用python将文件从azure data lake Gen 1移动到临时目录? 我需要找到ADL中的每个文件,将其移动到temp目录中,对其进行解密并读取
我正在使用
azure.datalake.store import core core.AzureDLFileSystem(token, 'store_name') #create a temp directory with tempfile.TemporaryDirectory() as tmpdirname:
关于该问题,请参考以下代码
from azure.datalake.store import core, lib, multithreaded ##auth adlCreds = lib.auth(tenant_id = tenant, client_secret = client_secret, client_id = client_id, resource = RESOURCE) subscriptionId = 'FILL-IN-HERE' adlsAccountName = 'FILL-IN-HERE' ## Create a filesystem client object adlsFileSystemClient = core.AzureDLFileSystem(adlCreds, store_name=adlsAccountName) ## Download a file multithread.ADLDownloader(adlsFileSystemClient, lpath='<local path>', rpath='<path on adls>', nthreads=64, overwrite=True, chunksize=4194304,buffersize=4194304, blocksize=4194304) ##Move file between locations on ADL adlsFileSystemClient.mv('<Source Path>','Destination path')
有关更多详情,请参阅here和here
关于该问题,请参考以下代码
有关更多详情,请参阅here和here
相关问题 更多 >
编程相关推荐