Fan welfare (with Python implementation code) crawler simple analysis code

Fan welfare (with Python implementation code) crawler simple analysis code

Fans of Jianshu have broken 1,000 followers. In order to thank you all for your support, we will send you a wave of small benefits. Like and comment on this article, 5 judges will be randomly selected from the comments, and each person will give a red envelope of 10 yuan. The draw time is 7pm on August 6th~

Although there are not many 1,000 fans, it is also affirmation of my study for more than half a year. I am currently studying in graduate school and have no extra income. Don't think it is too small for the judges. When I work in the future, I will give the judges more and better. Welfare~

The random lottery draw is of course using our python crawler. Take a previous article as an example to write the code.

Simple analysis of crawlers

  • Find a bag
  • Get total_page, this is the number of comment pages
  • json gets the user name and saves it in the list
  • Set conversion to remove duplicates, randomly select 5 users

Code

import requests
import json
import random

def get_user(url):
    html = requests.get(url)
    json_data = json.loads(html.text)
    comments = json_data['comments']
    for comment in comments:
        nickname = comment['user']['nickname']
        user_list.append(nickname)

if __name__ =='__main__':
    user_list = []
    url ='http://www.jianshu.com/notes/9315244/comments?page=1'
    html = requests.get(url)
    json_data = json.loads(html.text)
    total_pages = json_data['total_pages']
    urls = ['http://www.jianshu.com/notes/9315244/comments?page={}'.format(str(i)) for i in range(1,int(total_pages)+1)]
    for url in urls:
        get_user(url)
    user_list = set(user_list)
    print(random.sample(user_list,5))

Note: This is a random sampling, the result of each run is different, here only run the code once to get lucky audience

I will publish the results in this article

The result is released:

I will contact all the winning fans~ Thank you for your support

Reference: https://cloud.tencent.com/developer/article/1155626 Fan Welfare (with Python implementation code) Crawler Simple Analysis Code-Cloud + Community-Tencent Cloud