用scraped填充MySQL表

2024-04-24 05:47:04 发布

您现在位置:Python中文网/ 问答频道 /正文

我使用的是python3、MySQL、Sequel Pro和beauthoulsoup。在

简单地说,我想创建一个SQL表,然后将下载的数据插入到该数据中。在

我用这个答案作为模板来构建SQL部分Beautiful soup webscrape into mysql,但它不起作用。在

引发的错误:

line 86 finally:SyntaxError: invalid syntax

当我注释掉最后一个finally:(看看其余代码是否有效)时,我得到:

^{pr2}$

另一个常见的错误是:

ProgrammingError: (1146, "Table 'simple_scrape.simple3' doesn't exist", 虽然我不记得我做了什么样的改变最终导致了这个错误。在

最后,不到四个星期前,我开始学习编程(不仅仅是Python,而是‘编程’)——如果你想知道为什么我做了一些愚蠢或低效的事情,那几乎可以肯定,因为那是我让它工作的第一个方法! 请帮忙!在

代码:

from selenium import webdriver #Guess BER Number for i in range(108053983,108053985): try: # ber_try = 100000000 ber_try =+i #Open page & insert BER Number browser = webdriver.Firefox() type(browser) browser.get('https://ndber.seai.ie/pass/ber/search.aspx') ber_send = browser.find_element_by_id('ctl00_DefaultContent_BERSearch_dfSearch_txtBERNumber') ber_send.send_keys(ber_try) #click search form = browser.find_element_by_id('ctl00_DefaultContent_BERSearch_dfSearch_Bottomsearch') form.click() #click intermediate page form = browser.find_element_by_id('ctl00_DefaultContent_BERSearch_gridRatings_gridview_ctl02_ViewDetails') form.click() #scrape the page import bs4 soup = bs4.BeautifulSoup(browser.page_source) # First Section ber_dec = soup.find('fieldset', {'id':'ctl00_DefaultContent_BERSearch_fsBER'}) address = ber_dec.find('div', {'id':'ctl00_DefaultContent_BERSearch_dfBER_div_PublishingAddress'}) address = (address.get_text(', ').strip()) print(address) date_issue = ber_dec.find('span', {'id':'ctl00_DefaultContent_BERSearch_dfBER_container_DateOfIssue'}) date_issue = date_issue.get_text().strip() print(date_issue) except: print('Invalid BER Number:', ber_try) browser.quit() #connecting to mysql finally: import pymysql.cursors from pymysql import connect, err, sys, cursors #Making the connection connection = pymysql.connect(host = '127.0.0.1', port = 3306, user = 'root', passwd = 'root11', db = 'simple_scrape', cursorclass=pymysql.cursors.DictCursor); with connection.cursor() as cursor: sql= """CREATE TABLE `simple3`( ( `ID` INT AUTO_INCREMENT NOT NULL, `address` VARCHAR( 200 ) NOT NULL, `date_issue` VARCHAR( 200 ) NOT NULL, PRIMARY KEY ( `ID` ) )Engine = MyISAM)""" sql = "INSERT INTO `simple3` (`address`, `date_issue`) VALUES (%s, %s)" cursor.execute(sql, (address, date_issue)) connection.commit() finally: connection.close() browser.quit()


Tags: importbrowseriddateaddresspageissuefind
1条回答
网友
1楼 · 发布于 2024-04-24 05:47:04

问题: 并实际创建表

            sql= """CREATE TABLE simple3(
            (
            ID INT AUTO_INCREMENT NOT NULL,
            address VARCHAR( 200 ) NOT NULL,
            date_issue VARCHAR( 200 ) NOT NULL,

            PRIMARY KEY ( ID )
        )Engine = MyISAM)"""
// Added this line since your table was not being created.
            cursor.execute(sql)

            sql = "INSERT INTO simple3 (address, date_issue) VALUES (%s, %s)"
            cursor.execute(sql, (address, date_issue))

相关问题 更多 >