Accurate listings? It's a lot of work!

Voting and Reviews by Users

There are more than 115,000 registered users voting in the Joomla Extensions Directory, and most of them do it the right way.  In the past I've talked about reasons we reject reviews from users, but we've never really opened up the process we use to make sure the votes are valid, honest and accurate.

Because the JED is the most complete and accurate listing of Joomla extensions available, its traffic is high and as such listings are valuable.  The temptation to influence a JED listing is clearly great, and we spend quite a bit of our time monitoring voting patterns and preventing abuse. Because of the value of that traffic, we believe it is our duty to make sure the listings are as accurate and honest as we possibly can make them.

Voting and Reviews by Extension Owners

The vast majority of the 1600+ extension owners in the JED use the system properly, and for that we're grateful.

When an extension developer decides to influence their listing with fraudulent votes, they are hurting not only the other developers but the users that trust the ratings in the JED.  They take our time from working on extension and review approval, and slow the process down for everyone.

In the past six months we've had 24 different incidents of voting abuse discovered.  Of those 24 infractions, 12 resulted in a warning, 10 resulted in a 1 month suspension, one in a 2 month suspension and one repeat offender racked up 6 months of JED-free living.

Why not just remove voting for owners? The Joomla community has folks who write extensions and use extensions.  We'd prefer not to remove the voting ability for someone who codes a module and submits it to the JED.  It's just not fair to lose all those privileges just because you want to contribute.  So we've written some simple rules about when and how extension owners can vote.

The Rules are Short and Simple

The voting rules in the JED are available in our Terms of Service. An excerpt of our TOS:

Voting

1. A user account can only vote once per extension. Please do not vote until you've used an extension thoroughly, making sure you have configured it correctly.
2. It is not allowed for any person to solely rate an extension with the purpose of artificially inflating the rating or vote count.
3. It is not allowed for the same person to have more than one account on the Joomla! Extensions Directory.
4. It is not allowed for an extension developer to give negative votes to other extensions who are in competition for rating ranks, or to submit votes to extensions which offer similar functionality.
5. An extension developer, his family and\or colleagues are not allowed to vote on extensions from that developer.

There are varying degrees of damage a developer can do with fraudulent votes.  They might decide to give their own extensions good (5 star) votes.  Against the rules, but not the worst thing that could happen.  Getting a little more bold, some extension owners will create other user accounts using their personal, work, and freebie email addresses to give their extensions good votes. This is clearly fraudulent and a conscious attempt to sway the ratings.

The ones that really bother me are those who jump right to using those additional accounts to also give competitors 1-star votes as well.  Not just inflating their own extensions anymore, these folks are actually deliberately bringing down another developer's rating.  They do the most damage and the intent is abundantly clear.

We See Everything

We have a tremendous amount of data at our fingertips to help us investigate voting patterns.  I won't detail that data here, as publishing our methods would just give violators a road map to circumvent our investigations.  I'll be vague instead.

What do we do when we suspect fraudulent voting has occurred?  We investigate and compare the accounts, votes, time frames and other information we have available to us and a pattern will emerge.  We then discuss among the team the possible reasons for the voting pattern and come to conclusion as to the validity of the votes.

When we've confirmed the fraudulent voting privately, we unpublish the user's extensions and we email the developer to let them know what we've found and ask them to explain themselves.  Sometimes, we find mitigating circumstances.  Sometimes, the developer is remorseful.  Sometimes, they deny it vehemently.  We take whatever response we get, discuss it, and determine the appropriate suspension length. In the past we have asked for an explanation before unpublishing a developers extensions, but unfortunately too many developers ignored our emails while their extensions remained published. Importantly, this is the process for clear-cut cases of fraud. If the data is not conclusive, we start by contacting the dev and then proceed as appropriate.

The first infraction may warrant a warning or a one month suspension.  A few times the severity has caused us to jump straight to the three month suspension for a first offense. Typically, we settle on one month.  If abuse continues, the suspension can be three months, six months or permanent (with yearly review).

Most Developers Do the Right Thing

Most of the time, abuse does not continue.  Developers see that each listing is treated equally.  Occasionally, the "excuse" for the voting fraud was that "we thought everyone was doing it".  When they discovered that listings are as accurate as we could keep them, they have apologized for mucking it up. We've been known to lighten a term for folks who are honest and remorseful.

On the other side of the coin are those who will vehemently deny culpability when we have indisputable evidence.  They complain the loudest that they are being treated unfairly.  This does not make us feel all warm and fuzzy.

We never discuss individual cases in public unless the extension owner reveals the reason their extension has been unpublished in public. It is not necessary, and we do not think that public shaming is helpful even in the most serious cases.

It's a shame we had to post this to begin with, but we'd like the public to know that we don't take voting abuse lightly, nor do we jump to conclusions and deal with a heavy hand.  We hope it will prevent misunderstandings and preempt new issues and keep the JED the most reliable source of information on Joomla extensions.