Monday, September 24, 2007

Search Quality is Brand Quality

Search Quality is Brand Quality: Greg Linden points to 'The Effect of Brand Awareness on the Evaluation of Search Engine Results', a paper by Bernard J. Jansen et al:

[...] Based on average relevance ratings, there was a 25% difference between the most highly rated search engine and the lowest, even though search engine results were identical in both content and presentation. We discuss implications for search engine marketing and the design of empirical studies measuring search engine quality.[...]

Greg summarizes some of the implications of branding with respect to competition in the search space. However, one might also consider branding in the context of a single provider. [...] Consider the challenge ahead of companies like Powerset and Hakia - which are attempting to bring another fundamental shift to search. Much of the criticism has been leveled at assumed issues with the technology. However, this is not the only battle ground. Establishing brand where there is none is a huge barrier to entry.

The cited study implies nothing of the kind. It ignores the well-known effect that when everything important (in this case, actual differences among search results) is removed from a stimulus, other variables that are still present dominate the response, even though those variables might be masked in a realistic situation. That is, the experimental design assumed but did not test additivity. A good study would have presented all combinations of brand and objective (double blind) search quality to the subjects. Then we would really know how much brand biases assessment of search quality.

No comments: