Why we do this: Although many evaluations have been done on different search engines in the past, they mainly used fixed sets of queries and judged the relevance of each returned page from a search engine by a panel of human judges. The queries are usually sampled from all queries issued by users to a search engine. However, we believe that this evaluation method is by no means ideal because the human judges are not the users who issued the queries and thus do not know what the users really want from the queries, and furthermore relevance does not mean user satisfaction. User satisfaction can only be measured using queries from the user’s daily information needs and based on his/her personal assessment of utility of the returned results to the queries. A more appropriate evaluation is a personal evaluation. That is what we want to do using SearchRater. Two small scale evaluations were done at the University of Illinois at Chicago. If you are interested in the results, please click this link: http://www.cs.uic.edu/~liub/searchEval/SearchEngineEvaluation.htm
Ask your friends to come! We would like to have more people and people from all walks of life to participate in the evaluation to obtain more reliable results.
No spam please! Please do not run automatic robots on the system. Do not participate in the evaluation if you have conflict of interests. We welcome people from Google, Yahoo! and Microsoft to view the evaluation results when they are available, but not to participate in the evaluation.
Results and privacy: When we have enough results, we will make them available on the Web. Note that only the aggregated results will be released. No query logs will be made public.
Who we are: We are a research group interested in search engine evaluation from the Department of Computer Science, University of Illinois at Chicago. The group consists of:
William N. Underwood (Graduate student)
Nitin Jindal (Graduate student)
Bing Liu (Advisor)
Home | Discussion Forum
|