Online Evaluation Devices' Disruptive and Revealing Effects: A Lexicometric Analysis of Online Consumer Reviews in the Hotel Industry.

Which car should I buy? Which candidate should I hire? In which hotel will I sleep tonight? Those common puzzles are not self-resolving because on markets information is not perfect. Economic agents are regularly exposed to adverse selection (Akerlof, 1970), a market failure they ward off using signals (Spence, 1973). Those signals reduce the uncertainty on quality but also help build a consensus on what makes a good or a service valuable (Beckert, 2009). Markets cannot exist without actors, standards, and valuation devices defining the quality and qualities of goods and services (Eymard-Duvernay, 1989). A traveler seeking a hotel or, more accurately, seeking a hotel satisfying his wants, can rely on different signals, produced by numerous entities (directories, travel guides, specialized magazines, commercials, hotel standards symbolized by stars etc.), most of them claiming and expertise, diversely certified (technical qualification, sharp knowledge of the market etc.). The Internet gave the opportunity to individuals without certified expertise to express themselves on any kind of topics, in particular to say what they think of the goods and services they purchased (Cardon, 2010; Flichy, 2010). Amazon popularized online consumer reviews (OCR). Based on a rating and a comment they are now pervasive (Beauvisage et al., 2013) and can be considered as an algorithmically shaped word-of-mouth.

Very often, when a new valuation device bursts in a market, not only is the coordination between supply and demand impacted but also the very definition of the prevailing quality criteria (Karpik, 2000). Do OCRs have such an influence on the way hotels are evaluated and valued? We show that this innovation induced some elements of disruption. But the prominent point of our talk will be to demonstrate that it reveals how individuals express themselves when talking about their consumption experiences. Signaling theory has addressed the issues of the relevance and reliability of signals but has not or little examined their content. We propose to do so, through the lexicometric analysis of 680000 reviews on French hotels left by Internet users on TripAdvisor. It was performed using Cortext Manager, a platform developped by the French institute Ifris. This text-mining tool enables the mapping of heterogeneous (lexical and non lexical) networks. The dual format of OCRs makes it possible to link ratings on the one hand and semantic networks and enunciation schemes (objective versus narrative account for instance) used on the other hand. Analyzing OCRs then gives a better understanding of the judgment categories and rhetoric mobilized by individuals when they talk about what they liked, disliked or found average. 

Discussing Scott and Orlikowski’s works on the effects of social media in the travel industry (Orlikowski and Scott, 2014; Scott and Orlikowski, 2012), our study is two-fold. As most commensuration devices (Espeland and Sauder, 2007), OCRs have triggered many reactions of the existing economic actors. We carried out an ethnographic and quantitative (online survey, 895 respondents) investigation to examine the reactions of the hotel sector to the introduction of OCRs, in particular their attempt to restore a control on this valuation device, and to ensure its reliability. But we also performed the lexical analysis of the very content of the reviews, with the following questions in mind: Does the Internet involve an extension of the domain of what Karpik (Karpik, 2010) describes as singularities, i.e. multi-dimensional incommensurable goods marked by radical uncertainty on quality? Do Internet users objectivize or give subjective accounts of their evaluation of hotels? What words do they use to express the quality and qualities of hotels? Do they innovate in terms of judgment categories or use established ones? Do they use the same words and rhetoric to write about palaces and low-budget hotels, about hotels they enjoyed staying in and hotels they disliked? Lexical analysis reveals writing practices and judgment categories that vary greatly according to the rating, the travel context but also the language used. For instance, the more Internet users write about hotels they appreciate, the more they use a narrative style to relate their subjective experience of consumption. Market and evaluative hierarchies also have their textual expressions. 

References

Akerlof, G.A., 1970. The Market for “Lemons”: Quality Uncertainty and the Market Mechanism. Quarterly Journal of Economics 84, 488–500.

Beauvisage, T., Beuscart, J.-S., Cardon, V., Mellet, K., Trespeuch, M., 2013. Notes et avis des consommateurs sur le web: Les marchés à l’épreuve de l’évaluation profane. Réseaux 177, 131–161.

Beckert, J., 2009. The social order of markets. Theory and Society 38, 245–269.

Cardon, D., 2010. La démocratie Internet: promesses et limites. Seuil, Paris.

Espeland, W.N., Sauder, M., 2007. Rankings and Reactivity: How Public Measures Recreate Social Worlds. American Journal of Sociology 113, 1–40.

Eymard-Duvernay, F., 1989. Convention de qualité et formes de coordination. Revue Economique 2, 329–359.

Flichy, P., 2010. Le sacre de l’amateur: sociologie des passions ordinaires à l’ère numérique, La république des idées. Seuil, Paris.

Karpik, L., 2010. Valuing the unique: the economics of singularities. Princeton University Press, Princeton.

Karpik, L., 2000. Le Guide rouge Michelin. Revue Française de Sociologie 3, 369–389.

Orlikowski, W.J., Scott, S.V., 2014. What Happens When Evaluation Goes Online? Exploring Apparatuses of Valuation in the Travel Sector. Organization Science 25, 868–891.

Scott, S.V., Orlikowski, W.J., 2012. Reconfiguring relations of accountability: Materialization of social media in the travel sector. Accounting, Organizations and Society 37, 26–40.

Spence, M., 1973. Job Market Signaling. The Quarterly Journal of Economics 87, 355–374.