基于 python 的网络爬虫设计The Web Crawler Design Based on Python 内容摘要哔哩哔哩视频弹幕网，从一开始的无名小站发展到现在月活跃用户超一亿的巨大规模，不仅引起了行业浪潮，也吸引越来越多人选择成为 UP 主（Uploader）即视频上传者。Bilibili 网为 UP 主提供了平台和各种功能支持，但还是无法面面俱到。本软件设计旨在提供便捷的刚需功能，为新兴 UP 主提供功能支持。本毕业设计采用标准的软件开发流程，包含了需求分析、整体设计、模块实现、编码、测试、打包软件等等步骤。该软件算法和轮廓基于python 设计，调用了包含 requests、random、pickle 等等成熟的集成库环境。软件主要针对用户需求颇多的两个主要分支：评论和抽奖进行设计实现。评论区功能包括爬取完整的表单数据，解决复数层评论的获取和存储，实现快速浏览、存储和多种查找功能；抽奖区功能包括实现爬取多种抽奖材料，根据需求分析样本并最终使用随机算法进行随机数获取，实现抽奖功能。关键词：爬虫 bilibili 用户 pythonAbstractBilibili has started as an unfamous acg web at the beginning and now have a huge scale of over 100 million active users each mouth, which is causing a wave and attracted more and more people to choose to become a UPloader.Bilibili provides Uploaders sport and various functions for the users, but it still cannot cover everything. This software is designed to provide convenient and needed functions to users. This graduation design uses a standard software development process, including requirements analysis, overall design, module implementation, coding, testing, packaging and other steps. The software algorithm and outline are designed based on python, and call the mature integrated library environment including requests, random, pickle and so on. The software is mainly designed and implemented to slove two main functions with many user needs: reviews and lotteries.The review function incuding crawling complete form datas, solving the acquisition and storage of multiple l...
