The number within () in each cell is the percent of queries that falls into the cell.
Likewise, row 2 and row 3 give the corresponding results for partially satisfied and completely satisfied queries. The students were only unsatisfied with 4% of the queries for Google, which is truly remarkable!
They were also asked to record two pieces of simple information for each query, type of query and level of personal satisfaction.
No precision or rank positions of the search results were measured.
However, there is still no general consensus on the classification.
I only used the two most frequent ones in order to make the evaluation simple and less painful to the students (not to confuse them too).
The evaluation of usefulness can only be done based on queries derived from the user's personal information needs and his/her personal perception of the returned results to the queries. Such an evaluation can truly tell us why users choose one search engine over another. The students were asked to use only one search engine in each week.
The results also show the weaknesses of each search engine and hence tell the search engine company where they should focus their efforts on in order to improve their search effectiveness. Since they were all graduate students, thus the results given below only reflect this segment of the general population. Only if the designated search engine was unable to give a satisfactory result, might other search engines be employed.
The students were asked to perform their daily searches as usual based on their daily information needs without any change.
The only requirement was that they needed to stick to the same search engine for the week and only to use another search engine if the first search engine did not give good results.
In order not to favor any search engine and to mitigate the impact of positive or negative sentiments on one search engine affecting the evaluation of the next search engine, each group is assigned to use a different search engine in each week with the following schedule: Week 1: Group 3: Google We did not have a user interface system that could hide the identities of search engines such that students did not know which search engines they were using.
Thus, the evaluation could be slightly affected by their pre-conceptions about each search engine.
is by no means ideal because relevance does not mean user satisfaction.