搜索引擎爬虫数据抓取VIP专享VIP免费

简单搜索引擎模型ASimpleScratchofSearchEngine作者史春奇，搜索工程师，中科院计算所毕业，chunqi.shi@hotmail.comhttp://hi.baidu.com/shichunqi计划：1，需求迫切07/06完成2，搜索引擎简单模型07/08完成3，信息导航模型07/16完成1/3数据抓取07/30预处理4，商家推广模型5，未来本文是学习搜索引擎的涂鸦草稿，高深读者请拐弯到：http://sewm.pku.edu.cn/IR-Guide.txt（北大搜索引擎小组--信息检索指南）简单搜索引擎模型.................................................................................................................................1ASimpleScratchofSearchEngine...................................................................................................1第一章需求迫切................................................................................................................................2一）泛信息化.................................................................................................................................2二）泛商品化.................................................................................................................................2第二章导航模型--草根需求信息......................................................................................................3第一节最直观简单模型............................................................................................................3第二节互联网简单模型............................................................................................................51．发展历史...........................................................................................................................62．大陆互联网现状...............................................................................................................73．草根需求.........................................................................................................................10第三节网页抓取简单模型.........................................................................................................101.最简单Spider抓取模型..............................................................................................112.最简单Spider调度模型..............................................................................................123.最简单Spider调度质量模型......................................................................................154.最简单Spider调度策略模型......................................................................................185.Spider的常见问题.......................................................................................................23第四节网页预处理简单模型.....................................................................................................231.质量筛选（QualitySelection）...................................................................................242.相似滤重（De-duplicate）..........................................................................................353.反垃圾（Anti-spam）..................................................................................................43第五节索引存储简单模型.........................................................................................................48第六节检索框架简单模型......................................................................................................48信息检索评价指...