Daily we use Internet services and search tools in particular when looking for data. The search results are normally called hits and are provided in the form of a list. The information may include web pages, images, information and other types of files. Some search engines also gather data available in databases or open directories. If compared to Web directories that are maintained by human editors, search tools function algorithmically or are a mixture of algorithmic and human input.
Internet search tools operate by storing information about a huge number of web pages which they retrieve from the INTERNET. These pages are retrieved by a web crawler, also known as a spider. It is an automatically-controlled Web browser that follows every link it sees. The content of each page is then analyzed to determine how it should be indexed. Words, for example, are extracted from titles, headings or special fields called meta tags. Data about web pages are stored in an index catalogue for further use in queries. Some search engines, such as Google, store the entire or part of the source page (known as a cache) as well as information about web pages, whereas others, such as AltaVista, save and store every word of every page they have found. This cached page always holds the actual search text, as it is the one that was actually indexed. Therefore, it can be very helpful because it contains information that may no longer be found anywhere else on the Internet.
As soon as a web user has typed key words in the search field, the engine checks its index and provides a listing of the most suitable web pages in accordance with its parameters, commonly with a short summary coupled with the document’s title and sometimes excerpts from the text. Some search tools have installed an advanced option called proximity search which allows users to define the distance between search words.
The usefulness of a search engine depends on the relevancy of the results it provides. Since there can be millions of web pages containing a particular search term or word combination, some pages may appear to be more relevant and popular than others. The results can be ranked to display the “best” ones first.
How a search engine determines which pages are the best matches, and in what arrangement the results should be displayed, is specific to a search engine. The techniques also change with time, because the use of the Internet undergoes alterations and new techniques appear.