personalaiagents.io

Home / How we score

How we score

An independent project that tracks personal AI agents every day. No company pays for a place here, and no one at the companies listed sees a page before you do.

Where the numbers come from

  • Store ratings and installs. The US App Store (Apple's public lookup API) and US Google Play, pulled daily for every agent that has an app: 10 apps behind the 13 agents (Poke, Lindy and Hermes Agent have none). When an agent lives inside a bigger app (dots in ChatGPT), we show that app's numbers in gray and say so: they rate the whole app.
  • Store reviews. The newest reviews of the agent's own app: Apple's public review feed (it returns only the latest ~500, which for a big app is two or three days) and Google Play (up to 37 days back, at most 15,000 reviews a day's run).
  • Reddit. Every post from the agent's own subreddits, plus posts in big shared subreddits that mention it by name, through the public Arctic Shift archive (Reddit refuses requests from servers). Over-18 subreddits are skipped. The archive's word search sometimes times out; a week it could not answer keeps what an earlier run found.
  • Prices, plans and capabilities. The makers' own pages: product pages, pricing pages, help centers, store listings, and the Wayback Machine for older prices. Each row links to the page and quotes it, with the date we checked. If we could not find a source, the cell says “no data yet”.
  • What changed. App Store version notes, dated announcements, and complaint spikes we count ourselves.
  • Videos. YouTube search for the agent's name, uploaded this year. Reviews and hands-on tests (titles with words like review, tested, vs, tutorial) come first, then the rest, each group by views. The maker's own channels (for Muse: Muse, Meta, AI at Meta; for dots: OpenAI, ChatGPT) are left out. YouTube's first results page only, so counts are a floor.

The mood index

Each Reddit post is read once by a language model and labeled −1, 0 or +1 toward the agent; posts that turn out not to be about it are dropped. Store reviews take their sentiment from the stars, not from the model: 1–2 stars count −1, 3 stars 0, 4–5 stars +1; the model only drops the few reviews that turn out not to be about the agent (106 of 12,380 for Muse this month). Then it is arithmetic:

net   = Σ(w·s) / Σw
w     = log(1 + upvotes + comments)
        a store review: w = 1
raw   = 50 · (1 + net)    0 to 100
score = (n·raw + 25·50) / (n + 25)
        pulls small samples toward 50
        under 50 items: source left out
index = Reddit 0.35 · stores 0.25
        reweighted over sources present

The window is the last 30 days; “7 days” is today's index minus the index of the 30 days that ended a week ago. The chart on each page is the same formula over a rolling 7-day window, with at least 30 items per source, from Reddit and Google Play only; a week with fewer posts leaves a gap in the line, marked “too few posts”.

Complaints, praise and quotes

The same pass tags each post or review with up to two complaint and two praise themes from one fixed list, the same for every agent. A share is plain counting: posts and reviews that raise the theme, divided by all posts and reviews about the agent in 30 days. Reviews of a few words carry no themes and are not counted there.

Complaint themes

  • Privacy and data access: asks for or shares too much, unsafe permissions
  • Misses or repeats tasks: misses, fails, forgets or repeats tasks; does not finish what it was asked
  • Confidently wrong: wrong facts, wrong actions, made-up results
  • Limits and paywalls: usage limits, caps, quota, features locked behind a paid plan
  • Price and billing: too expensive, billing problems, unwanted charges
  • Too pushy or noisy: too many notifications or messages, acts without asking
  • Bugs, crashes and login: bugs, crashes, glitches, login or account problems, outages
  • Setup and connectors: hard to set up, connectors or integrations not working
  • Slow: slow responses or tasks take too long
  • Got worse after a change: a feature was removed, changed for the worse, or the product got worse
  • Missing a feature or platform: missing a feature, platform or region the writer wants

Praise themes

  • Gets things done: gets real tasks done, saves time, handles errands
  • Easy to start: easy to set up and use
  • Helpful nudges: useful reminders, nudges, follow-ups
  • Remembers context: remembers context and personal details well
  • Works with my apps: works well with my apps, email, calendar or accounts
  • Voice and personality: natural voice, conversation or personality
  • Free or good value: free or good value for the money
  • Smart, good answers: smart, accurate, high-quality answers or work
  • Fun to use: fun, delightful, cute

Quotes are copied word for word; our code drops any quote that is not an exact piece of the original text, and the model is told to skip text with personal details. Each quote links to its thread or review; App Store reviews have no public link of their own, so they link to the app's reviews.

A spike in “What changed” means a complaint theme's share in the last 7 days is at least double its share in the 23 days before and at least 2 points higher, on at least 8 posts and reviews. We only look for spikes when both periods hold 100 or more posts and reviews and nine in ten of them have been read.

Read the numbers with care

  • Reddit leans negative (people come to complain); stores lean positive (apps ask happy users for ratings). Compare agents to each other and to their own past, not to an ideal 100.
  • An agent without its own app gets no store source. Its index is Reddit only: a smaller sample, not a penalty.
  • A fresh Reddit post can show a score of 1 until the archive reads it again, and the archive keeps the counts of its last read, which may be hours or days old; until that read, a post's weight comes from its comments.
  • YouTube rounds view counts and ages (“1 month ago”), so “about 3 months” is approximate.
  • On comparison pages, “What it does” and “Who it’s for” are our own one-liners, written from the sourced facts on the same page, not the makers’ words.
  • We do not test the agents ourselves yet. That comes next, with the same tasks for every agent.

Freshness

Everything is collected and recomputed once a day. Each page shows the date of its data; a source that fails keeps yesterday's numbers and its old date rather than a guess. Last run: Nov 22, 2026.