I am writing an objective review of things. This is the first section, which needs comment, criticism and concensus before I can really continue. Hopefully it will lead to a correct decision being makable and so made.
Okay, but first I'd like to request you both cut the crap on the public forum. How TGM can promote his system while not responding to a post filled with the system's issues is beyond me. I feel like I've wasted a whole load of time going over the last Elo design spec because now it looks like it's being redesigned completely, or that something new is coming along.
So that everyone knows what a replacement system has to handle let's take a look at what the points system was designed to do and why:
detail-graph.png [ 6.69 KiB | Viewed 15830 times ]
The red line is the period after 0.75 was released; a massive number of users joined, an order of magnitude more than the software had dealt with before. After everyone initially joined the 0.72 community was totally swamped, and went from an everyone-knows-everyone atmosphere to a sea of random players. With no easy way to know who was a dedicated player and who was likely to quit there was no way to have a public game which you could expect to be CD free. People would join a game, hope to get a good start, and leave if it didn't go quite to plan.
There were also players who would join dozens of games at a time, then have to be somewhere and have to leave them all (and play terribly in all those games anyway). Also players without an objective or goal would lose interest; after playing a game players wanted to feel like they had earned something for their success. Finally there was no real community, no way to tell who were the community members and who were the newcomers, and this means there's no loyalty/desire to stick with the site.
The end result is no-one has fun playing on phpdiplomacy.net, and the initial burst of activity starts to deflate over 6 months because the active users don't translate into a community of players, but just a bunch of anonymous people bumping into each other in an online game.
The points system was designed to fix the above problems, and it was entered in towards the end of August where the blue arrow is. Nothing else changed at the time; the sudden increase in activity and formation of a lasting community happened as players realized they now had to stick with games to progress, that it was worth fighting for every SC you could grab, and that if you joined loads of games you'd play bad and end up losing overall.
Since then the new adjudicator helped push the rate of increase up, the improved search engine optimization helped, as did the Facebook Diplomacy hookup, but it's the points system which first started building the site that we have today
There are a few points I'd like to take from this:
- It's not a bad system, it does what I designed it to do and then some, probably in less code than the characters which make up this post so far. Any contending systems have their work cut out for them
- If this was just a ranking system I wouldn't care; however the majority want to rank themselves would be fine with me, I'm not going to be #1 in any ranking scheme. The problem is the ranking system is just a part of it, and it's closely tied up with things I do care about which keep the site manageable.
- The functionality the current system provides is important, as you can see from before/after points system change
- It's dangerous to change the points system, since it's what ties the community together, so I'm going to be very thorough when vetting new systems. I won't bet the project on "math says so"
Here are a list of the things the points system rewards and punishes.
- Improving at all from a CD player's bad position (taking over CD players costs as much as their SCs are worth)
- Mingling with lots of players (Points originate from the low players with few points, you have to be part of a chain that reaches new players, players can't organize themselves into isolated groups
- Playing regularly (Because the total amount of points going into the system only goes up there is "inflation" of points, i.e. points become worth less as time goes on, this means everyone is getting "rewarded" on average, no-one is just staying in the same spot. This results in a sort of "race" instead of someone going up at the expense of someone getting dragged down)
- Sticking to a single account (Your starting position is the worst possible position; the only incentive to create a new account is to multi-account, no-one will be scoring negatively in their account and simply start a new one. The incentive to multi-account is also diminished because points scores increase exponentially, so to get a multi-account higher up by sacrificing other accounts you'd need a whole pyramid of multi-accounters beneath the one you want to win, it gets impractical fast and the negative effects are diminished)
- Surviving (PPSC encourages newcomers to stick to the end, and form alliances. Strong alliances and draws are rewarded in low skilled games, but incentives for newcomers to learn to talk is a positive thing)
- Winning (WTA fits on nicely to the points system, so that both PPSC and WTA can exist using the same points)
- Joining the right number of games (Players are encouraged to join as many games as they can play well in, but discouraged from joining any more than that)
- Civil disorder (The points they invested in the game are also passed on to the other players in their game, which means dedicated players get pushed up at the expense of civil disorder players even faster than they otherwise would)
- Resting on your laurels (If someone is at the top and by playing can only go down, the system is rewarding people who leave. Because of inflation no-one can reach the top and then leave. However it doesn't get out of hand either; if Rait hadn't played at all since the points system started he'd still be in the top 15 after over a year, hardly out of control or forcing players to be tied to phpDiplomacy 24/7 like a slave. Also remember that a player will always be able to catch up to their skill level with relative ease however inflated it gets. This is why so many have now made it to Rait's old #1 3000 points position with fewer games won)
Some design principles:
- It's simple (Not only the core idea of points/pots/bets is simple, but crucially the implementation is simple too. An idea is worthless if it'd be too difficult/time-consuming to implement, or would be demanding on the server)
- No magic numbers (These just open room for more tiresome debate, and probably need to be maintained/tweaked
- All-in-one (It's one integrated, easy to understand parameter, not a whole load of small systems)
- It needs to encourage activity; I like seeing the code used by lots of people, and having database optimization, security, concurrency, and user interface issues truly tested. A lifeless list of people either trying to get themselves into the most favorable games or having left after made it to the top doesn't sound like an improvment
- It needs to be open and inviting even to newcomers. This was one of the main design goals of phpDiplomacy which affected loads of decisions, and a pro-oriented ranking scheme wouldn't fit in with phpDiplomacy's attempt to be able to host a wide range of skill together as well as possible
- It should be enjoyable to compete in, phpDiplomacy is about hosting an active community and fun diplomacy games (preferably for a wide definition of "fun diplomacy game", from easy to hard etc), it isn't about hosting a ranking list
- All games shouldn't be equal; players should be able to play an "important" game which they play seriously and try hard in, and at the same time be able to play practice/off-beat games where they try a variant ruleset (e.g. gunboat), a different tactic, teach someone to play, etc
- A design spec is a doc containing a summary of all the info needed to turn the system into code, leaving no ambiguity. It has to be something a developer can take and fully analyze and code. Think of each of the places where the rating system comes into play, and figure out the inputs, outputs, formulas and/or algorithms which would be used in that place
- Even if I end up approving a design spec I won't be implementing it. The points system does everything I want it to do and has since Aug 07
- I won't look at and figure out problems with a design spec just to be presented with another until it works
Hope that helps