使用Python下载Kaggle数据集

2024-06-07 15:37:59 发布

您现在位置：Python中文网/ 问答频道 /正文
614 3

我试图使用python下载kaggle dataset。但是，我使用request方法时遇到了问题，下载的output.csv文件是损坏的html文件。

import requests

# The direct link to the Kaggle data set
data_url = 'https://www.kaggle.com/crawford/gene-expression/downloads/actual.csv'

# The local path where the data set is saved.
local_filename = "actsual.csv"

# Kaggle Username and Password
kaggle_info = {'UserName': "myUsername", 'Password': "myPassword"}

# Attempts to download the CSV file. Gets rejected because we are not logged in.
r = requests.get(data_url)

# Login to Kaggle and retrieve the data.
r = requests.post(r.url, data = kaggle_info)

# Writes the data to a local file one chunk at a time.
f = open(local_filename, 'wb')
for chunk in r.iter_content(chunk_size = 512 * 1024): # Reads 512KB at a time into memory

    if chunk: # filter out keep-alive new chunks
        f.write(chunk)
f.close()

输出文件

<!DOCTYPE html>
<html>
<head>
    <title>Gene expression dataset (Golub et al.) | Kaggle</title>
    <meta charset="utf-8" />
    <meta name="robots" content="index, follow"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, minimum-scale=1.0">    <meta name="theme-color" content="#008ABC" />
    <link rel="dns-prefetch" href="https://www.google-analytics.com" /><link rel="dns-prefetch" href="https://stats.g.doubleclick.net" /><link rel="dns-prefetch" href="https://js.intercomcdn.com" /><link rel="preload" href="https://az416426.vo.msecnd.net/scripts/a/ai.0.js" as=script /><link rel="dns-prefetch" href="https://kaggle2.blob.core.windows.net" />
    <link href="/content/v/d420a040e581/kaggle/favicon.ico" rel="shortcut icon" type="image/x-icon" />
    <link rel="manifest" href="/static/json/manifest.json">
    <link href="//fonts.googleapis.com/css?family=Open+Sans:400,300,300italic,400italic,600,600italic,700,700italic" rel='stylesheet' type='text/css'>
                    <link rel="stylesheet" type="text/css" href="/static/assets/vendor.css?v=72f4ef2ebe4f"/>
        <link rel="stylesheet" type="text/css" href="/static/assets/app.css?v=d997fa977b65"/>
        <script>

            (function () {
                var originalError = window.onerror;

                window.onerror = function (message, url, lineNumber, columnNumber, error) {
                    var handled = originalError && originalError(message, url, lineNumber, columnNumber, error);
                    var blockedByCors = message && message.toLowerCase().indexOf("script error") >= 0;
                    return handled || blockedByCors;
                };
            })();
        </script>
    <script>
        var appInsights=window.appInsights||function(config){
        function i(config){t[config]=function(){var i=arguments;t.queue.push(function(){t[config].apply(t,i)})}}var t={config:config},u=document,e=window,o="script",s="AuthenticatedUserContext",h="start",c="stop",l="Track",a=l+"Event",v=l+"Page",y=u.createElement(o),r,f;y.src=config.url||"https://az416426.vo.msecnd.net/scripts/a/ai.0.js";u.getElementsByTagName(o)[0].parentNode.appendChild(y);try{t.cookie=u.cookie}catch(p){}for(t.queue=[],t.version="1.0",r=["Event","Exception","Metric","PageView","Trace","Dependency"];r.length;)i("track"+r.pop());return i("set"+s),i("clear"+s),i(h+a),i(c+a),i(h+v),i(c+v),i("flush"),config.disableExceptionTracking||(r="onerror",i("_"+r),f=e[r],e[r]=function(config,i,u,e,o){var s=f&&f(config,i,u,e,o);return s!==!0&&t["_"+r](config,i,u,e,o),s}),t
        }({
            instrumentationKey:"5b3d6014-f021-4304-8366-3cf961d5b90f",
            disableAjaxTracking: true
        });
        window.appInsights=appInsights;
        appInsights.trackPageView();
    </script>

Tags： thehttpsconfigurldatavarlinkscript
3条回答
网友
1楼 · 编辑于 2024-06-07 15:37:59

基本上，如果要使用KagglepythonAPI（由@minh triet提供的解决方案是针对命令行的，而不是针对python的），则必须执行以下操作：

import kaggle

kaggle.api.authenticate()

kaggle.api.dataset_download_files('The_name_of_the_dataset', path='the_path_you_want_to_download_the_files_to', unzip=True)

我希望这能有帮助。

网友
2楼 · 编辑于 2024-06-07 15:37:59

kaggle api key和usersame在kaggle profile页面和数据集上可用 下载链接可以在kaggle的dataset details页面上找到

#Set the enviroment variables
import os
os.environ['KAGGLE_USERNAME'] = "xxxx"
os.environ['KAGGLE_KEY'] = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
!kaggle competitions download -c dogs-vs-cats-redux-kernels-edition
网友
3楼 · 编辑于 2024-06-07 15:37:59

我建议签出Kaggle API而不是使用您自己的代码。根据最新版本，下载数据集的命令示例如下 kaggle datasets download -d zillow/zecon

相关问题 更多 >

    编程相关推荐

    热门问题