Python网络爬虫及数据可视化

北京理工大学珠海学院 2020 届本科生毕业设计Python 网络爬虫及数据可视化Python 网络爬虫及数据可视化摘 要随着 4G 的全面普及，以及 5G 基站建设开展，带来互联网相关产业的高速发展，电子数据的爆发式增长。对于互联网公司来说，海量用户带来的巨量数据就是待挖掘的金矿，同时数据也是互联网公司的生命线。获取海量的数据如果通过人工一一进行获取，速度缓慢，耗费时间长，效率低下，对于此痛点，网络爬虫是最好的选择，通过计算机的协助，获取海量具有价值的数据。bilibili 弹幕网站（以下简称 B 站）是近年来在年轻人中比较受欢迎的网站，用户趋向 90 后乃至 00 后，对于该网站的用户对象比较感兴趣，网站官方对爬虫也比较友好，因此选择该网站为爬取对象。本课题以 Python 为开发语言，借助 requests 库编写爬虫，MySQL 为数据库，用于存储数据，在后续数据可视化中，还是使用 Python 进行分析，通过这个程序，可以实现高效获取数据，筛选数据，存储数据，分析数据，数据可视化。关键词：网络爬虫；Python；bilibili 弹幕网站；数据可视化北京理工大学珠海学院 2020 届本科生毕业设计Python Web Crawler And Data VisualizationAbstractWith the full popularity of 4G and the construction of 5G base stations, the rapid development of Internet-related industries has brought about explosive growth of electronic data. For Internet companies, the huge amount of data brought by massive users is the gold mine to be mined, and the data is also the lifeline of Internet companies. Obtaining huge amounts of data manually is slow, time-consuming, and inefficient. For this pain point, web crawlers are the best choice. With the assistance of a computer, you can obtain massive amounts of valuable data. The bilibili barrage website(Referred to as B Station) is a popular website among young people in recent years. Users tend to be post-90s or even 00s. They are more interested in the user objects of the site and friendly to crawlers.This topic uses Python as the development language, write craw...