如何将计数器连接到我的Flask网页

Question

我正在制作一个网站，这个网站有一个抓取工具，可以从另一个网站抓取信息。在我的网站上，我想要一个计数器（显示当前正在抓取哪个页面，这样用户就知道一切都在顺利进行）

#---------------------------------------------- scarper.py

# ... rest of the code

counter = 0

while True:
    counter += 1
    print(f'Extracting infromation from page: {counter}')

#... rest of the code

这个计数器要显示在我的网站上（而不是在控制台里），放在这两个按钮之间。我该怎么做呢？

#---------------------------------------------- scrape.html

#... rest of the code

<button type="submit" class="btn btn-primary">Start Scraping</button>
<a href="{{ url_for('view_database') }}" class="btn btn-info">View Database</a>

#... rest of the code

这个路由及其功能的代码如下：

@app.route('/scrape', methods=['GET', 'POST'])
def scrape():
    form = ScrapingForm()
    if form.validate_on_submit():
        city = request.form.get('city')
        subregion = request.form.get('subregion')
        apart_or_house = request.form.get('apart_or_house')
        words_to_check = request.form.get('words_to_check')  # Retrieve words to check input

        if city and apart_or_house:
            g.scraping_finished = False
            threading.Thread(target=run_scraper, args=(city, subregion, apart_or_house, words_to_check)).start() 
            flash('Scraping started!', 'success')
        else:
            flash('Please fill all required fields.', 'error')

    if g.get('scraping_finished', False):
        flash('Scraper has finished!', 'info')

    return render_template('scrape.html', form=form)


@app.route('/scraping-finished')
def scraping_finished():
    return render_template('scraping_finished.html')


def run_scraper(city, subregion, apart_or_house, words_to_check):
    # Ask the user for input
    base_url = ""
    while True:
        if city != "":
            break
        else:
            flash("City name cannot be empty. Please enter a valid city name.", "error")
            return

    while True:
        if apart_or_house == "wohnung" or apart_or_house == 'haus':
            break
        elif apart_or_house == "":
            flash("This field cannot be empty. Please enter what are you buying.", "error")
        else:
            flash("Please enter either 'wohnung' or 'haus'.", "error")
            return

    if subregion:
        base_url = f"https://www.immobilienscout24.de/Suche/de/{city}/{city}/{subregion}/{apart_or_house}-kaufen"
    else:
        base_url = f"https://www.immobilienscout24.de/Suche/de/{city}/{city}/{apart_or_house}-kaufen"

    # Run the scraper script with the provided inputs and base_url
    subprocess.run(['python', 'scraper.py', city, subregion, apart_or_house, base_url, words_to_check])

    with current_app.test_request_context():
        return redirect(url_for('scraping_finished'))

Answer 1

在你的HTML代码中：

计数器值: <div id="counter"></div>

在你的JavaScript文件中：

function updateCounter() {
            fetch('/get_counter').then(response => response.json())
                .then(data => {
                    document.getElementById('counter').innerText = data.count;
                });
        }
        setInterval(updateCounter, 1000);#time the updates as needed

在你的Flask后台代码中：

@app.route('/get_counter')
def get_count():
    counter = #your logic for getting the counter count running on backend
    return jsonify({'count': counter})

你可以使用ajax来实现这个功能，但这样会不断发送数据包，没必要。更好的方法是在客户端用JavaScript来处理，或者使用一个按钮在需要的时候从服务器获取数据。

Answer 2

一种解决办法是使用服务器端渲染（SSR），并在客户端创建一个事件。

在你的 scrape.html 文件中，添加一个元素来显示计数器的值：

<button type="submit" class="btn btn-primary">Start Scraping</button>
    <span id="counter">0</span>
    <a href="/" class="btn btn-info">View Database</a>

    <script>
        var source = new EventSource("{{ url_for('scrape') }}");
        source.onmessage = function(event) {
            document.getElementById('counter').innerHTML = event.data;
        };
    </script>

把你的 scrape.py 文件改成：

app = Flask(__name__)
app.config['SECRET_KEY'] = 'your-secret-key-for-csrf'


class ScrapingForm(FlaskForm):
    city = StringField('City', validators=[DataRequired()])
    subregion = StringField('Subregion')
    apart_or_house = SelectField('Apartment or House', choices=[('apartment', 'Apartment'), ('house', 'House')],
                                 validators=[DataRequired()])
    words_to_check = StringField('Words to Check')
    submit = SubmitField('Start Scraping')


def run_scraper(city, subregion, apart_or_house, words_to_check):
    counter = 0
    while True:
        counter += 1
        print(f'Extracting information from page: {counter}')
        yield f'data: {counter}\n\n'


@app.route('/', methods=['GET', 'POST'])
def scrape():
    form = ScrapingForm()
    if form.validate_on_submit():
        city = request.form.get('city')
        subregion = request.form.get('subregion')
        apart_or_house = request.form.get('apart_or_house')
        words_to_check = request.form.get('words_to_check')

        if city and apart_or_house:
            return Response(stream_with_context(run_scraper(city, subregion, apart_or_house, words_to_check)),
                            mimetype='text/event-stream')
        else:
            flash('Please fill all required fields.', 'error')

    return render_template('scrape.html', form=form)


if __name__ == '__main__':
    app.run(debug=True)

我还把线程替换成了 stream_with_context，并且去掉了 g。计数器的值是通过 document.getElementById('counter').innerHTML = event.data 来更新的。