Successful BI -- You'll Know It When You Find It

I came across an excellent article in the most recent DM Review magazine about indicators of successful Business Intelligence (BI) solutions.  Many of the points therein apply to Knowledge Management (KM) initiatives as well.

The author encourages people responsible for BI initiatives to look beyond metrics such as the number of users with access to the system (e.g., "we rolled out our new dashboard to over 500 users").  We should be focused instead on:

• who is accessing the system
• the information that users are able to find
• how the information is used to make business decisions
• the quality of decisions made based on the information
• the process for improving the BI solution based on user feedback

There is also a simple questionnaire for collecting these data included in the article.

Clearly, some of these are easier to measure than others (transaction logs, lines of code, etc. are always easier to measure than subjective "good/bad decision" measures), but the author does a good job of reminding us to focus on outcomes, rather than the most easily measured pure metrics.

A few measures I have seen used to indicate success of a BI or KM initiative:

• the percent of searches for info that are abandoned
• the latency between an adverse event, a business decision, and a corrective action
• the number of repositories users need to search separately to find data (e.g., local hard drive, removable media, Outlook archives, shared folders, SharePoint document libraries, off-site backups, etc.)
• the number of company-wide e-mails sent out looking for information about a client, a project, a proposal, etc.
• the number of documents e-mailed around for review, with no integrated version control (i.e., the number of times the question "Is this the most current version?" is asked)
• when an experienced resource leaves the firm, the amounf of information that leaves with him/her, never to be recovered

I'm sure you have some of your own -- feel free to comment below...

[ps  Thanks for reading in 2006, and have a happy and safe holiday.  See you in 2007! -- mg]

Comments

[sadalit]

The number of calls made to the help desk / support center on the subject. These are usually pretty well tracked and categorized, so the metrics are easy, but you also get to see a wide range of questions and issues - everything from feedback to training needs to serious bugs in the system.