thumbnail

Ratings AI Summary

Summarising dealer ratings on a car marketplace, using AI

Year


2025

Role


Product Designer

Platform


Web, Mobile

Team


PM, data scientists, engineers, designer

Work done

Designed an AI-powered feature that analyzes and summarizes hundreds of dealer reviews, providing potential buyers with quick, digestible insights about dealer performance and customer satisfaction.

Context

Dealerships often have hundreds of reviews. Most consumers read the first 5 reviews or filter for worst reviews to get a measure of a dealership. This user behaviour however doesn't accurately capture the whole picture.

Dealers POV

Reviews matter to dealerships. The perception consumers get from reviews makes a difference in if they contact a dealer or not. Therefore, creating a summary that accurately depicts the nuances of customers experiences at their dealership, is a delicate balancing act. One that must be done while keeping up dealer satisfaction.

AI ratings summary interface

Ratings dashboard redesigned to accomodate the AI summary

AI ratings summary interface

Up close. Summary truncated by default

AI ratings summary interface

CSAT to get a sense of satisfaction with dealers

  • When a dealer clicks to "Read more", we ask how satisfied they are below the summary
  • This was necessary because we had a threshold of satisfaction score that we had to get to with dealers, before releasing to consumers
AI ratings summary interface

If satisfaction is less than 4, tell us why

AI ratings summary interface

Disclosure that it's generated by AI, and more details of how it works.

AI ratings summary interface

Control mechanism implemented, for dealerships that might wish to turn this feature off on their pages.

Consumers POV

While reviews provide 'wisdom of the crowd' to discerning consumers, it can be a hassle to read through when there are hundreds of them. Hence, a summary that accurately captures the sentiments of the crowd, saves time and helps consumers make quick decisions.

AI ratings summary interface

AI-generated summary feature integrated into the ratings page

AI ratings summary interface

Feature anatomy

  • Theme extraction for common praise and complaint categories
  • Links to supporting review examples for verification
  • AI disclosure that this summary is AI generated
  • Last updated date, to show summary is a continual generation based on new reviews
  • Rating of summary's helpfulness
AI ratings summary interface

Summary's evaluation

  • Rating of summary's helpfulness
  • Tracking and asking for feedback helps us to continously improve the summary

Design Challenges

  • Ensuring AI summaries accurately represent the review corpus
  • Balancing comprehensiveness with brevity and readability
  • Maintaining user trust in AI-generated content
  • Handling edge cases like very few or biased reviews
  • Creating summaries that work across different dealer types and sizes

Outcome

  • Increased engagement with dealer profiles and contact rates
  • Improved buyer confidence in dealer selection process
  • Higher user satisfaction scores for dealer experience
  • Established foundation for AI-powered features across the marketplace