Analyzing Search Engines

<div style="sub-title">1. Formulate five criteria for the evaluation of search engines

To effectively evaluate three different search engines from the perspective of an advanced web user, the following criteria were established:

1. Relevance and accuracy of search results

2. Search speed

3. Advanced search options

4. Other services

5. Site design and layout

More information about these criteria and how they were used is available below.

2. Test three search engines against your criteria

For this evaluation, the following three search engines were tested:

1. AltaVista - http://www.altavista.com/

2. Google - http://www.google.com/

3. DirectHit - http://www.directhit.com/

<div style="sub-title">3. Explain your methodology

The criteria outlined in Part 1 were reached by considering the things that would be important to an advanced user. Of course, factors like Relevance and Speed are critical to such a user (particularly in a work environment). The search options criteria is particularly important to advanced users who are after a specific page and do not want to waste time 'wading' through results from a 'standard' search. The other criteria are important to most users and mean that the search engines are more usable and useful.

The search results from the three test search engines (Part 2) were evaluate according the criteria, explained it more detail below.

Criteria 1 - Relevance and Accuracy of Results

This criterion was to evaluate the usefulness of a particular search engine according to the accuracy of its results. I defined the relevance, according to personal standards, after browsing the pages in the results, for three separate search terms. The Relevance Percentage is calculated by counting the number of relevant pages that appear in the first 20 search 'hits' and then multiplying by 20 (for a percentage).

Criteria 2 - Search Speed

This criterion was to evaluate which of the test search engines returns results quicker.

However, it is important to note that the speed test is both a test of the Internet connection to the remote machine, as well as the time taken by the search engine to perform the search. Furthermore, tests of Internet speeds are very variable based on time of day, other internet traffic etc and are therefore of only limited accuracy. For this reason, no 'hard numbers' were calculated (ie timed searches). Instead, I rated each search engine based on 5 searches per engine at the same time, same day. The possible Speed Ratings are as follows: Excellent, Good, Poor

Criteria 3 - Search Options

This criteria was to evaluate the search engines advanced search options.