Comparing how reddit finds the “best” comments with how Google ranks pages
Reddit is an anonymous social media website that ranks posts and comments in aggregate. Unlike how Google crawls websites for links and then finds PageRank values to choose what websites show first, reddit relies on user up-votes and down-votes to choose what links get shown first. While reddit gets direct feedback about which comments should be listed first, there are many flaws with simply using the most up votes. Early comments tend to stay on top because they are in a vicious cycle of getting more and more votes. This is a problem on many websites. For example, Amazon sorting its products by reviews. Should a product with one 5-star review come before a product with 200 review and a 4.5-star average? I’d argue the product with more reviews should be rated higher and this is how reddit implements their system.
To prevent a heavy bias towards early comments (the ones that get into a vicious cycle of upvotes) reddit uses the 95th percentile lower bound of a Wilson confidence interface as the method to determine which comments are first to be shown. Coming back to the Amazon example, the first product with only one perfect review would be below the second product with 200 good reviews. In this system the confidence interval of the first product is extremely large because the sample size is small, and the second product’s confidence interval is small because it has a large sample size. Therefore, by using the lower bound of the confidence interval the products are ordered in a sensible manner.
I found this example an interesting comparison to how search engines order results. While reddit uses a different system to satisfy their different needs, they both try to find the “best” links. Many other websites could benefit from implementing a similar system.
reddit’s new comment sorting system
http://www.evanmiller.org/how-not-to-sort-by-average-rating.html