Archive

Posts Tagged ‘spiders’

Lecture 3 - CSCI494 - Anatomy of Search Engine (Coding a Basic Crawler)

January 26th, 2012 admin No comments

CSCI494 Lect. 3 Jan. 25 2012 (Slides)

Last lecture (1/25/12) we discussed the elements of a search engine including crawlers, spiders, indexers, repositories, lexicons, ranking modules, and query processors. We also talked about fetching URLs and reviewed initial code for a crawler for assignment 2.

Here are the slides:
Anatomy of A Search Engine. Assignment 2 Building a Basic Crawler.

Next week we will derive “The Google Matrix” formula.