When and Why Do Buyers Rate in Online Markets
May 12, 2022 Xiang Hui

Follow Us


Xiang Hui is an Assistant Professor in the Marketing Area at the Olin Business School at Washington University in St. Louis, as well as a digital fellow at the Stanford University Digital Economy Lab. Prof. Hui studies the design of information mechanisms and economics of digitization, drawing theory-driven insights from large-scale datasets from industry collaborations. More specifically, his latest research focuses on designing trust systems on e-commerce platforms and on quantifying the economic impact of nascent technology in different fields. Prof. Hui holds a Ph.D. in Economics from Ohio State University.


We see reviews and ratings everywhere these days. People rely on reviews and ratings on platforms when ordering food, renting hotels, buying clothes, etc. The underlying rationale behind this is that reviews and ratings help people guess the true quality of the seller. However, previous literature points out that the information content of ratings, especially aggregate ratings, is subject to selection bias. For example, only a fraction of all transactions is rated, the seller may adjust its quality to the ratings; or the population of buyers may change over time. Is the rating still informative even with such selection bias?


On May 12, 2022, Professor Xiang Hui from Washington University in St. Louis joined us in the Luohan Webinar to present his new paper and to answer this important question. First, he proposed a simple model of consumer behavior of rating. Then, two testable hypotheses were derived based on the established model. To test the hypotheses, he conducted comprehensive empirical analysis using detailed transaction-level data from eBay. After establishing that the data supports the predictions derived from the model, he executed a set of simulation studies to estimate how many transactions are needed before a consumer learns the true quality of a seller.


The model features a consumer who likes to share her experience as she learns more from using the good she ordered. More specifically, the utility of leaving a rating is positively related to the difference between the consumer’s prior and posterior belief of the seller’s quality. From this model, Prof. Hui derived two testable predictions on the behavior of aggregate rating scores.


The first is that a seller is less and less likely to receive a rating as the number of transactions grows. Suppose a seller has already made many transactions. A consumer seeing such large number of ratings should be confident of the true quality of the seller. The more confident she is, the smaller the impact of additional information from her own experience will be on the original estimate of the quality of the seller. As a result, the difference between the prior and posterior should be smaller when a consumer buys from a seller with more transaction history, which leads to a lower probability of leaving a rating.


The second prediction is that, when a consumer believes that a seller is high of quality with high probability, an additional negative rating makes a consumer more likely to leave a rating, and the effect is stronger for negative experiences than for positive experiences of a consumer. If a consumer updates her prior with an additional negative review, then she becomes less confident of the seller being high quality. To the extent that she becomes less confident, the impact of her own experience would be able to affect the prior of the seller being high quality. Moreover, such an effect is stronger for negative experience since negative experience contains more information than positive experience in the model. This setup captures common features of consumer rating behavior.


Next, Prof. Hui tests those two predictions using detailed transaction-level data from eBay. To test the first prediction, he constructed a panel of transactions-sellers. Then, he ran a set of regressions to see how the probability of leaving a rating changes as the number of transactions grows. Comprehensive robustness checks support the first prediction of his model. To test the second prediction, he made use of negative ratings given “by mistake.” Using those negative ratings given by mistake as an exogenous shock to the prior of a consumer, Prof. Hui estimates the impact of a negative rating on the probability of a consumer leaving a rating. Interestingly, his result shows that a consumer is more likely to leave a rating after receiving a negative shock to her prior, which confirms the second prediction of the model.


Lastly, Prof. Hui runs a set of simulation studies to estimate how fast a consumer can learn the true quality of a seller when she knows the previous ratings are subject to behavioral bias described in the model. The simulations suggest that the rating records reveal the seller’s true quality after about 100 transactions.

This paper helps us understand an important question in the literature on consumer rating and review: the relation between the distribution of ratings and the distribution of consumer experiences. Moreover, it tells us that we need to understand the fundamentals of consumer rating behavior to understand the relation between ratings and consumer experiences.


If you would like to give a presentation in a future webinar, contact our Senior Economist Dr. Wen Chen (wen.chen@luohanacademy.com). For other inquiries, please contact: events@luohanacademy.com.



Subscribe to our news
SUBMIT

    Alibaba Digital Ecosystem Innovation Park, No. 1 Ai Cheng Street, Yuhang District, Hangzhou, China.


    events@luohanacademy.com


Luohan Academy    ©  2024《Luohan Academy Service Agreement》 浙ICP备2024120373号-1