Reviewing an online collaboration initiative – the IBM Partner Jam

An interesting experiment in what NOT to do.

Normally I would not write about something which my company might just about be involved in In this case I am prompted to share my thoughts based on visiting the web site for the first time in a couple of months, and seeing some pretty damning statistics. My background is PhD research using server logs to see how people actually use computers (Cass Business School London – school of Information Management – and sorry, uncompleted).

The Jam invite went out to easily more than a thousand people, call it 1,500. We call this the ‘invited audience’.

–> we know that not everyone will go in, so we always would have an audience wider than our target.

The likely ‘target audience’ would be some proportion of this, the real number of people an event planner would want to go into an online system in order to ‘guarantee’ a critical mass of visitors and contributors to get the content one needs to have a successful end event.

– in this case the likely audience, I would guess, would be 500

The actual number of people who visited the web site, and took the time to register is: 74

–> this is considered to be the ‘Visitor’ or ‘Participant’ rate. Imaginatik originally called ours the Participant rate, with the view that anyone taking the time to go into a system and register is likely going to at least passively take on board content. Some sites prefer the term visitor – and often this is more appropriate.

–> the Participant Rate is therefore 74 / 1500 (5%) from the overall audience OR 74 / 500 (15%) from the Target Audience (more reasonable)

In terms of contributions, the site only appears to have ‘Discussion Forum’. This is based on the view that people will spontaneously contribute to topics, create topics, and start discussing. Unfortunately one needs a lot more volume of participants before this can happen in an emergent method, and a way of seeing one’s contributions in the context of a wider volume. Why? If it hard to find the thread of a discussion, one feels as a visitor that one is alone in the system. There might be content – but you have to hunt and peck to find it, which is unlikely when the average session time in a system such as this is 5 – 10 minutes.

I counted around 100 content entries – main documents created by the session owners, and comments on the more popular themes. This gives us a Contribution Rate – 100 / 74 people = 1.35 documents – not bad, bit not fantastic. In addition, a visual skim through the submissions finds that there is a decent volume coming from the most frequent submitters – a tendancy for density of contributors. Ideally an event is spread out, and indeed research by Dr Rob Spencer at Pfizer, presented at the Imaginatik European User Forum in November 2008 highlighted that most of the contributions by volume – and by value – come from the Long Tail (first proposed by Chris Anderson) of users.

Now, if my view on audience is out, this would be a very poor result indeed. One can argue, as even I do, that one should focus on the end result, and if even one of these contributions is meaningful then the initiative was a success. But at the same time, one really must question whether sub-optimal projects should be celebrated. So what if this year you get a 2 / 10, compared to last year’s 1 / 10… when everyone else is getting 8 / 10 and above. Higher standards, please!

One Response to “Reviewing an online collaboration initiative – the IBM Partner Jam”

  1. Another point to note is that a significant number of posts is from the IBM moderators. The actual contribution rate from the partners is therefore even lower.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: