The trouble with SUMO

First a bit of a disclaimer – this post is in no way meant to belittle the hard work people have put into SUMO, still put into SUMO, or the successes it has had. However, as indicated from the title, this post is going to focus pretty heavily on the negative.

SUMO – SUpport dot Mozilla dot Org [Thanks to Tanner for the catch. SUMO became a dot com site, but the O can still come from Mozilla.]- has a pretty interesting history. Sought after for years among the community, Mozilla finally decided to take user support seriously. Kinda. At the time, the commitment from Moco was pretty small and there were rumblings that SUMO still had to prove user support was actually helpful before it was sure to stick around. – which it certainly has by now.

The shape SUMO was meant to take changed quite a bit. There was talk about a web-based help viewer for Firefox on a separate thread from the discussions for forming a real support offering. In fact, Mozillazine, already running a user forum and a knowledge base was vetted as a possible solution – simply endorse the site that already exists. For several reasons, some of which I was in on, some of which were discussed privately, it was decided that a Mozilla run solution would be best.

At first, SUMO was meant to be a quiet, community run project. People already active in the support community, including myself, were brought in to grow our areas of expertise and form, well, a community. A full time team-lead was hired and we had a manager to guide us all. However, the nature of the project changed really quickly. Our manager left for a start up and wasn’t replaced. Our team lead became the manager and for the most part the team didn’t interact with Mozilla outside of itself. – Excluding the team lead/manager, I can’t speak for him.

Pretty early on we adopted the ambitious goal to be the full support system for Firefox 3, which meant a full set of KB docs in a minimum number of locales. The point of SUMO as communicated to me at this point was to be the BEST Firefox support there is. Along the lines of ambitious goals not only was the KB to be up and running, but the forums and live chat were targeted to go live by the end of the year, and they did. Now I have no idea where the goals came from. Did our manager believe in us? Was he being pushed by his manager to make SUMO relevant as quickly as possible? So I’m not going to comment on whether or not I think they were good goals. We certainly achieved them.

But that’s the problem. SUMO is constantly being judged on what it does do, but as this is Mozilla’s first venture into user support there’s no other internal baseline. What could SUMO be? How does SUMO measure up, especially to the passed over Mozillazine which, as far as I can tell, is still thriving? Most importantly to me, is SUMO avoiding the mistakes that made us pass over Mozillazine in the first place?

Now again, SUMO has been doing great stuff. We did great things with tikiwiki and the new custom software looks pretty cool, too. Except is SUMO innovating? The new forums end up working a lot like Get Satisfaction. There’s been talk of a web based Live Chat client (on the helper’s end) for literally years now. Meanwhile the Spark client was the only part of the live chat software that still had active maintainers and yet no changes were pursued in favor of the in house client that’s been around the corner this whole time. The Army of Awesome is pretty cool, I’m not familiar with what already exists for this kind of thing so I have no idea of it’s innovative, but it’s definitely progress so I won’t knock it.

There’s also no real innovation in document writing. Obviously the hire of Michael Verdi is meant to change that, but for now it hasn’t happened yet. The guidelines for how to write an article haven’t changed much since SUMO was launched, and there’s no template on how an article should be layed out. SUMO has been collecting survey data on its articles from the beginning – a handy feature of tikiwiki – to find out which articles are working and which need help. By now, having chosen a consistent format would be a reasonable expectation. –I checked out SUMO’s Best Practices guide just to make sure my bold claim was accurate, well you can see when that article was updated.

There’s also no contributor training outside of the old docs, more specifically the one I wrote for live chat helpers on how to troubleshoot an unfamiliar issue. The top issues and how to solve them are not clearly communicated, and certainly not outside of the team. Support Firefox Days haven’t happened in a long time. There was apparently a SUMO newsletter, but I can’t find any editions to prove it existed, just the old sign-up form. The SUMO blog basically gets news, but nothing practical that will help someone provide better user support.

Openness has been an issue, as well. Some things are better than they were, some things are worse. The team goals appear to be public now, but as far as I can tell are not published to the community (I found them linked in one of the meeting minutes). Meeting minutes used to be published to the newsgroup (no longer used), the blog and the contributor forum (stopped in both due to undesired “noise”) which isn’t such a big deal as the bulk of the planning discussion moved to the sumodev meeting which was not promoted to the rest of the community. Now there’s a private moderator’s only forum, which I only found because I looked at the forum lists (and I’m still in the moderator group, apparently). The discussion about why it’s private is quite lively, I’d suggest reading it but I’d have to leak it for you to do so, which, in spite of this posting, I’m not prepared to do.

Is SUMO doing the best it can do? I don’t know. I think people are certainly working hard and deserve credit. I think, though, that not enough people are involved. Not enough people are asking tough questions (though I can understand why, the Flash vs Ogg bug didn’t go very well). I also think that there’s been too much focus on sending data inward rather than out. There’s also been way too much focus on cultivating the software in relation to cultivating the community. With the new hires, and with Kitsune almost done, it looks like that will change, however if people aren’t asking the right questions, using the right measures, then in the end it won’t be any different from where it started.

3 Responses to “The trouble with SUMO”