My concern is that reading star ratings and satisfaction scores as a sign that an AI product works can hide what users are actually grading. This article walks through a new study of 6,000 app-store reviews, what it found about how people judge a companion AI and a task AI, what it did not show, and a few checks you can use.
Two apps, two kinds of review
The study is “Divergent user expectations of conversational AI: a comparative sentiment and thematic analysis of ChatGPT and Replika app store reviews” by N. Kausalya and Gemini V. Joy of VIT Business School, Vellore Institute of Technology. Scientific Reports published it on October 28, 2026 as an open-access Article in Press, which the journal describes as peer-reviewed, accepted research shared early. The final edited version will replace it.
The authors compared two apps they chose as clear examples of two uses. ChatGPT stands for the task assistant. Replika stands for the companion, which the authors describe as designed to remember its user, express affection and keep up a long-term emotional connection. In the authors’ words, the two kinds of app, task tools and companions, are “built on the same underlying language technology.”
The data were public reviews from the Google Play Store, collected in 2026. From 38,241 reviews retrieved, the authors kept English-language reviews and drew a random sample of 3,000 per app, 6,000 in total. They then ran computer text analysis on the words: sentiment scoring, emotion word counts and topic modeling, which groups reviews by the words they share. Each review came with its one-to-five star rating.
This is not a survey or an experiment. Nobody was recruited, asked questions or assigned to a condition. The paper says the work “involved no human participants,” and the data are what reviewers chose to post.
What people wrote about
The clearest result is about topics. Topic modeling found six themes. Conversation and companionship made up 53.6% of Replika reviews against 10.9% of ChatGPT reviews. Task assistance and usefulness made up 30.9% of ChatGPT reviews against 1.2% of Replika reviews.
Other themes also split by app. Reviews about output errors and malfunction were more common for ChatGPT (24.1% versus 7.7%).
The authors add a caution here. The topic names are their own short labels for groups of words, and they say the labels “have not yet been validated by independent coding.” The split in the numbers does not depend on the names, but the names are a reading of the word lists.
What barely differed
Several things you might expect to differ did not, or only a little.
Overall positivity was not a stable difference. With one scoring method Replika reviews came out slightly more positive. With another, ChatGPT came out ahead. The authors conclude that “aggregate polarity was therefore not a robust point of difference.”
Emotion words differed only modestly. After adjusting for review length, Replika reviews had more words linked to joy, anger, surprise and anticipation, “with all effect sizes small.” Words linked to fear, sadness and trust “did not differ significantly between the applications.”
Star ratings did differ. ChatGPT ratings were more spread to the extremes: 36.8% versus 21.7% gave five stars, and 27.4% versus 23.8% gave one star. Replika ratings sat more often in the middle.
Ratings and words, a small and provisional gap
The finding closest to governance is also the weakest. For Replika, the link between what reviewers wrote and the star they gave was a little looser than for ChatGPT. The authors call the difference small (partial eta squared of 0.016, about 1.6% of variance, against 21% for the rating itself).
They also report it as provisional. The two samples cover different years. ChatGPT reviews run from September 2024 to January 2026. Replika reviews run from September 2018 to January 2026, and 73.8% of them came before March 2023. When the authors repeated the test on the period both apps share, the difference was not significant, though that window held only 525 Replika reviews.
So this result points in a direction. It does not settle anything, and it does not show that companion app ratings are meaningless.
An expert’s argument, labeled as opinion
Two days before the study appeared, Nature Human Behaviour published a Comment by Daniel Västfjäll of Linköping University and Decision Research, titled “AI governance needs to consider users’ emotional responses.” Its public summary makes an argument and a proposal; it does not report study results. Only its short summary is public; the full text is behind a paywall and is not used here. The summary says:
“People will often trust or distrust an artificial intelligence (AI) output because of how it makes them feel, rather than how reliable it is. AI governance frameworks should require that platforms disclose design choices that target user emotions, allow users control over these, and include features aimed to calibrate trust.”
That is a proposal. The public summary does not say that any regulator has adopted it.
My interpretation
What follows is my reading of the two pieces together. The link is mine; neither source cites the other.
A star rating tells you how a product landed with the people who chose to review it. The study suggests that what those people are grading can differ by product. Reviews of the task app talk about getting things done and things going wrong. Reviews of the companion app talk mostly about conversation and companionship. Neither kind of review is a test of whether the AI gave correct answers or behaved safely.
That matters most for products designed to feel warm. If a product is built for connection, its satisfaction scores may mostly reflect the connection. Whether people should rely on what it says is a separate question, and it needs its own measure. The Comment argues that feelings often drive trust in AI. The study does not test that claim, but it is a reason not to treat a happy rating as a reliability check.
What the study did not show
It is easy to stretch this paper, so here is where it stops.
- It did not measure harm, addiction, mental health, loneliness, wellbeing, reliance or how much users trust either app. It analyzed the words in reviews, not users’ experiences.
- It did not find that people trust one app more. Trust-related words did not differ significantly.
- It did not find that companion app users are happier or more negative overall. That result flipped with the method.
- It did not show that star ratings are meaningless for companion apps. That gap was small and provisional.
- It covered two apps. The authors present them as “two representative cases rather than establishing distinct categories of conversational AI systems.”
- Reviewers chose to review. The authors say the estimates “apply only to the users who reviewed the app on the store, not to all users.” The reviews came only from Google Play, in English.
- It is cross-sectional, so it cannot say what caused what.
Checks you can use
These are my suggestions, not the authors’.
Keep satisfaction and reliability on separate lines of any AI report. A star rating or a thumbs-up rate is feedback on experience. Accuracy, error rates and safe behavior need their own tests.
Read what reviews are about, not just the average. The authors make a similar point for companion apps: “monitor for specific emotions, not just the polarity of emotions.” They also suggest that anger-related language “may serve as an early-warning indicator of potential user concerns following updates.”
Watch changes to warm products closely. The authors argue that products which draw people into relationship-like engagement carry a responsibility to consider how users describe them when the product changes.
Ask which design choices aim at feelings. Whether or not any rule ever requires it, the Comment’s proposal gives you a fair question for a vendor or your own product team: which features are there to make people feel something, and can users control them?
Before you cite a satisfaction score as evidence that an AI works, write down what that score actually measured.
This article reflects the author’s personal views, not those of any employer, and is based on a publicly available peer-reviewed study by Kausalya and Joy in Scientific Reports and the public summary of an opinion Comment by Västfjäll in Nature Human Behaviour, as available on October 30, 2026. It is not affiliated with or endorsed by the authors, Scientific Reports or Nature Human Behaviour, and quoted findings and opinions are theirs. It is general information, not legal or professional advice, and reading it does not create any advisory relationship. Get qualified advice before acting on how it applies to your organization.