Content Focused Review Strategy

Why Content, Not Scores, Should Drive Your Review Strategy

If you work for a software vendor, you probably already know that your buyers are using TrustRadius.com and other review sites to compare products. According to our latest survey, that’s how 82% of buyers on TrustRadius.com are using our site. As they build their long and short lists, reviews and ratings give them a helpful view of the “competitive environment,” as one buyer put it.

Here’s a breakdown of the various reasons your buyers are using TrustRadius. On average, respondents selected 2.2 use cases, meaning the vast majority of buyers compare products on TrustRadius and have one additional use or goal.

why buyers use reviewsSource: March 2017 survey of 235 buyers who used TrustRadius.com

Knowing this, you may think that overall score is the primary way buyers compare products. Indeed, when starting out with reviews, many vendors focus squarely — even exclusively — on their star rating. But before you get too obsessed with your score, I’ll share another important insight: buyers care more about in-depth, qualitative feedback than your product’s overall score.

We know that buyers consider multiple factors when they use reviews. These factors work together to determine not only the buyer’s perception of your company and your product, but also how helpful and trustworthy they find your reviews overall. But, when push comes to shove, what is the top priority for buyers? Which piece of information is really going to help them in their selection process? Where should you, the vendor, and we, as a review site, be focusing our energy to have the biggest impact on buyers?

We asked survey respondents to rank the following five factors in order of importance. Review content (qualitative feedback) was the most important factor when evaluating a product on TrustRadius; 36% of buyers ranked it number one. This sentiment was also consistently reflected in dozens of in-depth interviews we’ve conducted with individual B2B software buyers.

buyers important factorsSource: March 2017 survey of 235 buyers who used TrustRadius.com

Buyers Know Ratings Don’t Tell the Whole Story

Of course, we had to gather some qualitative feedback of our own about why buyers care more about review content than a score. While buyers want to see scores, the number alone is less useful than the context behind it. (To illustrate this point: you might be skeptical of the survey results above, until we unpack the buyer mindset behind them!)

Here are some themes that emerged from buyers’ qualitative feedback about reviews and ratings.

1. Star ratings could be based on skewed, old or incomplete data

Without digging into the underlying user feedback, buyers feel an overall score could be misleading. When it was collected, how it was solicited and numbers of data points were all mentioned as critical pieces of context.

“If a software rates well but doesn’t have a large number of reviews that equates to an overall score then it doesn’t feel like it’s relevant (could be pushed for paid reviews, advocacy, etc).”

“The score/rating is a critical metric, but it has less validity if there a few reviews or the reviews are not recent.”

“Recent reviews are more relevant, as previous issues may have been corrected, or a product previously a leader may have lost its focus.”

(Note that at TrustRadius, because we’ve heard the feedback from buyers that overall scores can be misleading if they aren’t factoring things like recency and selection bias, we introduced our trScore algorithm to address these concerns.)

2. Buyers want to know the person behind the feedback

Buyers also want to view an individual’s rating in context. First, buyers know that no product is the perfect fit for everyone. They want to dig into end-user feedback to understand whether a product will work for their use case, company size, maturity level, industry, and level of complexity. (In this vein, filtered scores, such as those on our segmented TrustMaps, are often more useful to buyers than overall scores.)

“It is important that I be able to look at the score of the product based on the types of companies that primarily use it and the number of reviews, particularly those that are recent. This helps me to better determine if the product is the right fit for a company based on their size and needs.”

“In order to properly assess the reviewer’s rating and its applicability to our circumstances it is important to have context such as understanding the business problems they were trying to solve, their industry and size. Without this background you cannot properly understand the score, compare the results across reviewers and determine applicability to your situation.”

“The product overall score/rating [ranks] last as what is important to one company may be lower on the list. Our needs aren’t all the same.”

“Although I looked at overall score as the most important, I read each review to really understand what drove the reviewer’s to assign it the score they did. Some of those concerns did not apply to my business, so could look at the scores where users were trying to solve similar issues as myself.”

Additionally, buyers use the qualitative feedback to gauge whether the reviewer knows what they’re talking about, and whether their perspective is relevant or trustworthy.

“A superficial review may reflect a lack of insight, perspective, expertise, and/or experience on the part of the reviewer. As such, substantive reviews are most important to me. A close second, assuming such reviews are indeed substantive, is the overall score.”

“Products don’t exist in vacuum, so it’s really valuable to get perspectives from reviewers with experience with multiple products that can be specific in their comparisons.”

3. Buyers use the score as a jumping off point

Even buyers who indicated that the overall score was important gave caveats as to how they used it. Generally it was viewed as a way to quickly identify products with unusually low satisfaction, or as an indicator of what to expect in the reviews. 

“I like the overall ranking as that helps eliminate options that have poor overall [satisfaction].”

“I am not worried about the overall score/rating of the product unless it’s terrible. This is mainly because people’s needs are different and that may skew the numbers.”

“Quantitative rankings are good, but people tend to vote similarly. I find the differences in people’s opinions more helpful.”

“Rankings whilst useful are often not granular enough to be able to give more than an indicator.”

“The overall rating helped confirm or reset my thoughts on particular companies I was leaning toward.”

4. Content is king, as one buyer put it

The substance of the reviews is where buyers say they find immense value. The content helps them understand product strengths and shortcomings, suss out best-fit scenarios for different products they’re considering, identify questions to ask during a sales process, and know what to expect post-purchase — all of which are common goals among buyers using review sites, and none of which can be inferred from an overall score.

“I really wanted to know who was using the product, when was the last time they used it, and what was their experience when they did use the product.”

“I wanted to know what people were saying about the products and compare/contrast opinions and how people were using each.”

“The reviews helped me understand about the product and its shortfalls. It made me more aware of what to expect from each product when I start using them”

“Wanted to determine what others were saying about the product. [TrustRadius] was helpful especially the pros and cons of the software. I was able to form my questions related to the cons of the product.”

In some cases, the dynamic between content and score varied depending on the buyer’s stage.

“As I went through the process of selecting a solution, I returned to review sites multiple times with a different goal each time… For example, the overall score/rating was critical at the beginning to help set a direction in my research. The review content was important as I dug into the details of the final 2-4 options.”

So if you are confronted with a negative rating or even a less-than-desired overall score, remember that what buyers really want to see is the qualitative feedback behind the numbers, especially as they get closer to a purchase decision. When you put all your energy into micromanaging your score, you’re not focused on enabling the most influential thing for buyers: in-depth, candid insights from people like them, not just the carefully cultivated advocates and heroes who will give your product a five-star rating.

How You Can Influence Buyers

If you want to influence buyers researching your product on TrustRadius, the best way to do so is to ensure there is plenty of high-quality feedback — positive and negative — that gives them a sense of where your product is a good fit, as well as where it is not. Buyers are especially interested in direct comparisons made by users who have past experience with other similar products, and stories about the difficulties they might want to anticipate themselves.

“People’s experiences, what they found good or bad and why are a very good gauge of the product. Good reviews with both positive and negative feedback, [as] it’s rare for a product to [be] perfect in all regards to any person.”

“Products do not exist on their own, they exist in a context, a market place. Understanding the competitors/alternatives define a product as much as the product’s abilities to perform in any given task. Often within the reviews you can find certain problems for your use case of the product that might be a deal breaker even if the product has the highest score”

You have the power to help them find what they’re looking for by inviting a broad, representative sample of your user base to write balanced, in-depth reviews, and by keeping the dialogue open so that feedback stays current.

So take a deep breath, and look beyond your score. If you do, you’ll be on the path toward a buyer- and customer-centric sales and marketing strategy. Having high-quality feedback from a diverse group of customers will benefit your buyers — and your company — in the long run.

For more insights on what is important to buyers, and what you can do to influence them, download our study on the B2B Buying Disconnect.

Emily Sue Tomac

Emily Sue Tomac is Research Analyst at TrustRadius, where she studies the business software landscape, trends, and user feedback. She writes objective, user-focused reports that help buyers navigate crowded markets. She thinks of herself as a translator: she can help you understand marketing-speak, technical jargon, and crowd-sourced opinion, in plain English. Emily Sue covers the Sales technology landscape, Marketing Automation, eCommerce, Help Desk, SMMS, and Project Management software. Prior to joining TrustRadius, Emily Sue worked on research in linguistics and the digital humanities.