Recently I had to choose an issue tracking system for a small, web application developing operation (small meaning 4 developers and a dozen projects). Main requirements were
- ease of use
- an intuitive and attractive GUI (yeah, you heard that right, after all it would be used daily by developers and managers alike)
- open source project, and possibly free to use
- time tracking feature
- automatic roadmap and changelog
- and a lot more smaller requirements, but you’ll see those from the comparison matrix
I’ve used mantis in the past, and trac during the last year and a half, and was not entirely happy with either of those. Nonetheless, I included them in the comparison, just because both are popular and they were the “standards”, other systems would be measured against. I did my homework, and after a lot of research, I added Jira and Redmine to my shortlist.
Now that I had 4 issue trackers shortlisted, the next step was to gain hands on experience with each. I installed all of them on a development server, went throught the whole setup, created projects, issues, went through the steps of a usual issue lifecycle a couple of times. I also set up svn repositories for a fake project to test VCS integration capabilities.
All this evaluation went into a comparison matrix, available here. Disclaimer: the comparison reflects my personal opinion and impression of the test subjects, and nothing else.
Here is a summary for you on each test subject.
Score: 41 out of 70
The good: trac is wildy popular, has an active development, and a lot of goodies via 3rd party plugins.
The bad: the setup. Setting up trac will make you cry like a child. The user interface is also a bit clumsy, it is just acceptable but will never make you say “wow, nicely done!”
Score: 44 out of 70
The good: I really have a hard time to say anything positive about good ole Mantis. Maybe that it’s very easy to set up, and the default settings are quite ok?
The bad: the user interface. I’m sorry to say but using Mantis is like talking to Zuzu Petals, it’s a rather painful experience.
Score: 62 out of 70
The good: practically everything. It has all the features you can dream of, and then some. Robust, actively developed, used by Fortune 500 companies, huge repository of both free and commercial plugins.
The bad: it’s not free. Though it has a very attractive looking licencing starting at 10 USD for small operations, that can quickly get into the range of thousands, if you need to add more developers or you’re planning to use extra features. It has also a rather steep learning curve.
Score: 57 out of 70
The good: Redmine has the “wow” experience in terms of GUI and core functionality that trac was unable to deliver. Everything is at the perfect place, it’s very intuitive to use. Kind of makes you wonder why other issue tracking systems haven’t thought of this and that.
The bad: it is closed development, supervised and maintained by a single person. Moreover, as of late 2010, half of the core developers stood up and left Redmine, which will probably affect the roadmap, and many planned features might stall.
The folks who walked out on Redmine started a new fork called Chili, available here. It looks very promising, and it is aimed at using the best from Redmine but curing the part that is most troubling, namely the closed development. These guys mean business, they are commited to monthly releases, and have a decent looking roadmap. However, as of writing this, the project is still in its early childhood, just celebrating its first public release. I’d say Chili worths keeping an eye on, but it will need some time until it can show significant advantages over Redmine.
The results: we choose Redmine.
The feedback: after two weeks of usage, we love it.