Also, this isn’t meant to be a condescending piece where we definitively declare ourselves to be perfect.  We recognize that we’ve made mistakes in reporting just like everyone else.  The only difference is that we do it all day long within the same set of systems.  Many internal BI teams cover multiple applications which makes data quality more of an issue.  An IT department with over 100 enterprise applications can’t possibly expect to hire 100 BI specialists.  That’s why we believe that the BI application space is the place to be.

And, truth be told, it’s common for reporting in many different disciplines to go wrong for the reasons we’re about to discuss.  So, we’d like to focus on a real-life story (while protecting the names of the guilty… and innocent).

On to the story!  Several months back we were visiting with a customer who was using the packaged analytics product (BO Universe) from one of the major ITSM manufacturers.  Basically, here’s how it went.

Manager:  I’d like a report to show me Incident count for the month, grouped by category… the group that resolved it… the individual that resolved it… and the resolve time (in hours).  And could you sort it to show the highest resolve time at the top for both the group and the individual within the group?

ITSM Administrator/Report Developer: Sure!  No problem, that’s simple.

Report created using Business Objects WEBI in a couple of hours because the calculation is already available out of the box… done.

ITSM Administrator/Report Developer: Here boss.  It’s done.

Manager: Thanks!  This looks great.  That was pretty quick.  You’re my hero again.  Did I ever let you know how much I care about you?  I shall show you in the next performance review by giving you a 2.13% raise which is the maximum for your band (ok, I ad-libbed this part).

Report is used for about 3 months… no worries…

Us:  Doing some other things for the customer… but then out of curiosity… can we have a look at your some of your reports?  We love getting new ideas for our application to stay ahead of the game.

ITSM Admin/Report Developer: Sure man, here’s one… it’s a resolve time report, pretty straightforward.

Us: Ok, let’s see here, can you bring up the measure for Resolution time?  Ok, let’s see here it says… Closed Date/Time – Open Date/Time =… Hey wait a minute.  Shouldn’t this be using the Resolve Date/Time field instead?  Do you have an auto-close workflow set?  Tell me how this thing works…

Rather than go on and on, let me just say this.  Even the simplest resolution time calculation was wrong out of the box from the ITSM software manufacturer.  It’s no big deal because that’s not what they’re in business to create.  But, What if you want to calculate something more complicated such as Average time for 2^nd level to respond?  Who is over there sharpening their Algebra skills on a daily basis to figure that one out?  Can an IT manager or director inspect every metric for accuracy? Well sure, but it just takes time and expertise in the data model of the underlying application.

In the end, manufacturers of ITSM software don’t spend their time all day long in business intelligence.  And, the BI team for most large companies is focused on core competencies and mission critical applications, but even so, many times one needs to spend all day every day with the application AND discipline.

So, this is simply a plea for recognition of the concept of specialization.  You don’t grow your own corn, you don’t write your own Service Desk application, so why write your own BI application?

Today, we are at the other end of the spectrum. Not only can we purchase applications to run our company’s business, we can purchase just about any tool we need to support any aspect of IT. And there are literally thousands of choices. So, it begs the question, should we even be asking ourselves this when we need a software solution?

One area that still seems to lean toward the build side is analytic reporting. And, as with any build decision in IT, there are enormous cost and risk factors that need to be considered.

From a cost aspect, a build decision requires the following:

• Time it takes to re-build reports & metrics when you switch ITSM platform (Avg. is 5 years)

• Time it takes to re-build reports & metrics when you upgrade ITSM platform (major upgrades)

• Subject matter expertise your organization has in the service management platform

• Project management expense

• Technical development, either internal resources or external resources

• Business intelligence development tools

• Infrastructure cost, for development and production

• Support cost (you build it, you support it)

• Other software cost such as reporting tools and dashboard tools

From a risk aspect, a build decision inherently has these considerations:

• Can a solution be built within the time a solution is required?

• Inadequate/incomplete communication between project participants

• Less than complete requirements definition

• Changing scope and objectives, “need more than I thought I did”

• Project staff turnover

• Project delayed or put on hold due to higher priorities in other projects

• Less than adequate skill sets on the project team

• Inadequate funding

• For a longer list, see http://www.dummies.com/how-to/content/the-essentials-of-managing-risk-in-your-project.html

IBM puts the development project success rate at under 62%. Furthermore, the factor contributing the highest percentage of failure, at 50%, is failures due to requirements gathering errors. And it hasn’t improved for the last 20 years. An off-the-shelf application has most, if not all the requirements established, as vendors have the experience of numerous customer implementations, and the resulting feedback, to get it right.

So if you need to report and monitor key metrics and KPIs in your service management system, does it really make sense to build what you can buy?

In a recent Bain trends publication (Mitt Romney’s old management consulting stomping grounds, http://www.bain.com), we noticed how much momentum the balanced scorecard is beginning to make after remaining in the theoretical stage for so many years. Also, there’s been a good bit of momentum with our customer base (particularly those for whom IT is a profit center like MSPs, but overall as well). So, we thought it might be worth spending a bit of time on the concepts in summary fashion.

A balanced scorecard is a worthy goal, especially for:

- Justifying the IT investments that have been made (and will be made) to acheive business goals

- Improving communication between IT & the Business, with increased visibility of actual performance vs. objectives

- Improving the transparency of IT activity

And, with Key Performance Indicators, Critical Success Factors and metrics, we have a framework for something very achievable. However, to date, only a handful of organizations can claim victory in using a balanced scorecard to align IT around the business mission. Let’s sum up the key elements needed at a high-level:

- Understanding of the company’s mission, goals and priorities – Are they revenue, cost reduction, risk reduction or social in nature? In what order? In what priority?

- What Frameworks have been adopted at the company (at every level) that are beneficial to the effort? Sure, Six Sigma might be helpful for detailing availability numbers, but is it common knowledge? Most in IT know ITIL to some extent, but what about your C-level executives? Will this information be meaningful to them in their common terminology?

- Know the Company – Who is going to drive this effort? Do they really want what they are saying they want? Is it achievable?

- Metric Composition – The typical balanced scorecard is composed of at least 150 metric calculations at the bottom level. It actually takes quite a bit of complexity to simplify things! BI applications like Northcraft obviously help dramatically reduce the need for IT BI development.

- Audience – This is not a mid-level manager dashboard. This is meant for Executives, both in business and IT. Does it need to be meaningful and accessible to an external customer or other 3rd party?

- Availability of the Data – Which systems will the data need to be collected from? Monitoring? IT Service Management? Project & Portfolio Management?

While Northcraft doesn’t provide a canned solution for the IT balanced scorecard, we can greatly reduce the effort associated with getting you there. We’d be happy to provide detailed balanced scorecard examples, metrics lists, consulting and our products to assist. A simple first step is to engage with us in our 5-day IT Power Score assessment.

02-27-2012

State of the IT Service Management Market, 2012, R4

There have been some major events in the market since Release 3 of the ITSM Market research released last August, such that we felt it was necessary to make an announcement. Let’s begin with the major events:

* Service-Now (Negative events for customers since last review) – Brings in new CEO Frank Slootman to transform Service-Now from an $130 M ITSM Application SaaS provider to a $1B PaaS powerhouse. We’re going to go down on the record as being skeptical about the possibility of SNDC competing with salesforce.com. Also, Service-Now won many of their early deals with heavy discounting in order to secure a customer base. Now that new policies and procedures are put into place that are necessary requisites for going public, customers are going to feel the sting in negotiations as SNDC needs to adhere to a more rigid pricing structure.

* BMC Acquires Numara (Positive and Negative) – and now owns roughly 40% of the ITSM Market. The current rumor is that BMC Service Desk Express will hit EOL in 2014. On-site SMBs will be encouraged to move to footprints. Smallest IT shops will be encouraged to go investigate Remedyforce, rather than Track.IT. The reality is that BMC can’t maintain Remedy On Demand, Remedy on-site, Service Desk Express, Numara Footprints, Track.IT and Remedyforce (and probably shouldn’t attempt to anyway). So, one of these technologies is going to be going away (if not multiple).

* HP (Positive) – Gets serious about SaaS for ITSM, gains flexibility in pricing model and establishes the most partner-friendly model on the market. HP has also addressed on-going interface issues and continues to have the most robust CMDB offering on the market.

* CA (Strong Negative) – Languishing in ITSM Market. Nimsoft Service Desk is comparable to Remedyforce functionality but without the power of the Force.com platform. Even though the most recent Gartner numbers show growth, the revenue numbers contained a massive outlier deal with the US Federal government that hides a major decline in ITSM.

BMC is #1 in the market with @40% market share, HP Service Manager is #2 at a little under 17%, Service-Now is #3 @10% CA is #4 at 8% and Frontrange is #5 at 5% (but most of this market share is heat and not Frontrange ITSM) In terms of ranking, we would rank the vendors using these criteria: Interface, Architecture, Functionality and Price (list pricing, because most vendors offer special discounts based on a wide range of criteria), Platform Integration, Innovation, Administration, Operations (for SaaS solutions only).

** One recommendation (and disclaimer) before we list the rankings. We highly recommend that as a customer you go into depth on each module (Incident, Change, CMDB, Release, etc…, because the devil is definitely in the details). For example, you could argue that Incident Management is a commodity application at this point, but CMDB definitely is not (in terms of functionality and architecture).

Ratings (1 is Lowest, 5 is Highest) – We’ve changed the rating system recently to account for similarities between the ITSM platforms.

- BMC Remedy ITSM 7.6.4 – Functionality 4, Architecture 4, Pricing (Blue/SaaS) 4, Interface 4, Sales Team 4, Implementation Community 4, Platform Integration 4, Innovation 3, Administration 4, Operations, 3

- Service-Now Aspen – Functionality 3, Architecture 3, Pricing 2, Interface 4, Sales Team 3, Implementation Community 2, Platform Integration 5, Innovation 4, Administration 3, Operations, 4

- HP Service Manager 9.3 – Functionality 5, Architecture 3, Pricing 4, Interface 3, Sales Team 5, Implementation Community 5, Platform Integration 2, Innovation 3, Administration, 2, Operations, 4

- CA Service Desk Manager 12.5 – Functionality 2, Architecture 1, Pricing 4, Interface 2, Sales Team 4, Implementation Community 2, Platform Integration 2, Innovation 2, Administration, 2 (No operations rating for Nimsoft Service Desk due to limited market presence).

- Frontrange ITSM 7 (not Heat) – Functionality 3, Architecture 3, Pricing 5, Interface 3, Sales Team 3, Implementation Community 2, Platform Integration 3, Innovation 3, Administration, 3

Additional remarks, thoughts and comments:

* We expect CA to move away for the Service Desk Manager platform towards the Nimsoft architecture. Service Desk Manager will probably remain supported in perpetuity, but continue to lag in on-going development as Nimsoft takes the top spot due to it’s native SaaS architecture.

* IBM TSRM remains largely a footnote in the ITSM market… in 6th place.

* Numara had annual revenues of about $75 Million before being acquired by BMC.

* HP has made some surprisingly positive moves of late after a very tough 2010… and a decent 2011.

* Caution: We believe Service-Now has hit it’s first growth obstacle (after posting remarkable 125% YoY growth in 2010) which will make it difficult to go public. Furthermore, we believe this increases the chance of an acquisition due to the investments made by Sequoia capital because VC firms need to make a substantial ROI.

* Remedyforce is built on Force.com, therefore it is built on arguably the best SaaS application architecture available. So, it is a native SaaS architecture, opposed to popular opinion. Many customers have confused this offering with BMC Remedy on Demand which still leverages the AR System architecture. Remedyforce is probably one of the most promising upcoming solutions in this market.

* ICCM is an ITSM platform that leverages a native Business Process Management engine (Metastorm). While ICCM’s presence is limited, BPM engines do give customers unprecedented flexibility in automating unique processes outside of IT.

* ITRP is a new ITSM SaaS offering that is positioned as serious and packaged ITSM. They have purposefully taken the opposite approach as ICCM as an application that isn’t designed to be customized, but leverages best practices. The interface is truly excellent. So, for those customers looking for a packaged ITSM application with robust functionality, a superb user interface and a simple SaaS licensing model, ITRP is worth considering.

For more information, or ITSM vendor consulting, please contact us at our website: http://www.northcraftanalytics.com

In the past, I worked for a large software company in a pre-sales consulting organization. Well, we had a Key Performance Indicator for the consultants to spend 240 hours of our time in customer-facing situations. The top people in each region were awarded points which could be exchanged for pretty nice gifts from the company store. If you won, you could expect to receive @$2,500/year in addition to your normal pay. In general, this was a great concept which incentivizes the consultant to get out of his/her shell and help the people that pay the bills! The KPI was even set at the right amount of  hours (about 50% of total time) because it was quite a stretch to spend that much time in customer activities as we had plenty of paperwork, training and technical work to accomplish in that same period. Personally, I felt like I had a real advantage in this situation because of the relationships I had built in the region over the years.

Reality… here’s how it set in:

- There were definitely 1 or 2 consultants who didn’t have a problem fudging their number.

- Every consultant had a different definition of “customer-facing” time – Did prep time count? Time thinking about the meeting? Travel time to and from the meeting?

- Also, the manager was awarded points for total time in the field for all of his/her consultants, so he/she didn’t really have a reason to argue against people who were optimistic about their time spent!

- This information was entered into an SFA system by the consultant. We were kind of biased – right?

- The manager had favorites – Now, let’s be honest. We all have favorites… Of course we do. You can’t just turn off your human nature and stop liking/respecting people that you actually like and respect. This favoritism ensured that tough questions weren’t asked to certain individuals.

So what happened? I busted my hump to make it into the yellow (KPI could be Red, Green Yellow… gotta love the traffic lights). Then, I was raked over the coals for being yellow. To this day, in my opinion, I believe the 2 consultants who won the 1st quarterly contest were absolutely, positively not in front of customers as much as I was. This was a pretty small group, so I had a pretty good idea. Was this demoralizing? Definitely. Did I stop trying as hard? No, but I kept on bringing up the topic on conference calls to the annoyance of everyone (including myself).

So what? What are my takeaways from this situation given my chosen career in analytics?

- Having a good metric and goal remain valuable, but the implementation really matters most.

- Beware of bias in a metric at multiple levels in the organization.

- Beware of carrots and sticks based on key metrics. Don’t avoid them, but be certain that proper controls are in place to ensure compliance. By controls, I mean rules, the interpretation of rules and the authority structure. Don’t be fooled, even in seemingly flat organizations, there is a final authority.

- Ensure that the systems in place to capture the metric have a chance of getting the data without being subject to influence – External/Objective (this is covered in our first blog in greater detail.) Northcraft’s products give you an advantage in this area.

- Transparency – This metric provided cover for the department and gave the appearance of transparency, but the truth was that 2 people were rewarded with incentives that shouldn’t have been given to them.

How does this apply to IT?

Remember these concepts when calculating Availability SLAs… and feel free to call us for help.

If you’re interested…

We’re been noticing through our own analytics that the most popular Northcraft blogs of 2011 have been:

#1: State of the ITSM Market, Release 3 – Google “ITSM Market Share”

#2: Service Desk Manager Job Description – Google “Service Desk Manager Job Description”

#3: Change Management Metric of the Month – Google “Failed vs. Unsuccessful Changes”

which has helped us come to the realization that… the “Metric of the Month” needs to be end-of-lifed and replaced with an article/story or “small tidbits that provide quick value” format (like the above). While we think you’ll agree that Metric of the Month is a very catchy title… it doesn’t appear to be resonating. And, since our major advantage in business is responsiveness to change… we must bid thee farewell MoM.

Finally…

Since this is our first article of the year, we’d like to end by saying THANK YOU to our customers who have helped Northcraft Analytics:

- Grow 900+% last fiscal year (CY2011) – Amazing! Again, Thank you!

- Establish industry analyst recognition – EMA (and we didn’t even have to pay for it! Yet.)

- Grow our metrics library by more than 100% – Good practices are getting a little better.

- Extend our platform into Service-Now – We’ll be happy to talk to customers about what’s next. The product direction is exciting.

In general, our theory that BI platforms are consolidating (with Microsoft in the lead at 45+%), BI applications are rising and enterprise IT departments are open to BI applications that provide a quicker route to value for non-core business functions appears to be validated by the market. But, who knows what 2012 will bring?

If there’s one thing we can all agree on it’s the fact that metrics alone don’t improve a business, but the interpretation of the metrics, the quality of the metrics and the interpretER matter the most.

http://blogs.enterprisemanagement.com/charlesbetz/2012/01/26/cmdb-data-warehouse/

Hello again all! I’m sure it’s been quite a busy time of year for us here in the IT world and the vendor world. So, it took me until the last day of the month to get that Metric of the Month out.  In any case, Knowledge Management is the process that has captured my fancy this month due to an interesting company & story that has grabbed my attention for a number of years.

Back in 2001, when I was on an engagement with Genuine Parts here in Atlanta, I learned what I still believe to be a “Best Practice” in Knowledge Management. There was a gentleman named Dave Cockerill (VP of Accounting and Information Systems at the time) who described to me a little bit about how they handled knowledge management at GPC. We were replacing their exiting mainframe-based solution with Remedy Knowledge Management and Dave wanted to import all of their existing knowledge management articles into the new system. My position was… “What? You want to take a bunch of EBCDIC mainframe text and pump it into our fancy new HTML-based Natural Language Knowledge System? “Uh… NO, BAD IDEA.” I was still thinking a bit like a technologist in those days (I still do sometimes!). Anyway, Dave proceeded to explain to me that they paid their IT analysts .50 every time an article was used. This was possible because the stores were paying for support. But, essentially, these articles were VERY well-written and valuable because of this incentive that GPC provided, even if the articles weren’t in the nicest format. We did improve their system a little bit since we gave them the capability to see which articles has been viewed as well as used, but that’s a story for another day…

As a result of this story, maybe I should have chosen # of articles used, but… this month, the metric is:

Average # of Days to Create Published Solution Article

The time in days from the ‘Create Date’ of the Article to the ‘Published Date’ for Articles with a Status of ‘Published’

Purpose: Measure the productivity of your end to end process for creating Knowledge Articles.

External: Not really, since this is the measurement of the full process, there are many fingers in the cookie jar potentially. You have the knowledge author, approver and editor roles at a minimum.

Context: This metric is most effective when focused on content initiated due to the resolution of problems (from the problem management process that is) around critical business services.

Objective?: Yes, this metric can be used objectively for comparison purposes against publish times for other articles (helping to prioritize time based on importance of solution).

To manage the performance of Level 1 and Level 2 services & support to clients (internal and external) and ensure that service levels are achieved. To ensure that customer expectations are met or exceeded. Responsible for ensuring the staff are meeting and exceeding expectations in regards to performance, meeting defined metrics/benchmarks, and that standards and processes are followed to provide effective customer service and meet requirements.

Key Responsibilities (% of Time)

1.  Oversee 100% of the requests, incidents and problems. Manages and coordinates urgent and complicated support issues. Act as escalation point for all requests and incidents. Develop and mature phone/ticket escalation processes to ensure free flowing escalation and information within the organization. Determine root cause of issues and communicate appropriately to internal and external customers. (25%)

2.  Train, coach and mentor Service Desk Specialists (Level 1 / 2) including career development. Oversee staff activities. Builds/obtains (from other departments) training material for support staff. As needed, schedule employees working times and provide backup support. Interact with internal and external customers. (20%)

3.  Provide data and reporting of KPI’s and trends to IT department and others in ad-hoc, weekly, monthly and as needed. Will drive Ticket Deep Dive and develop strategies for improvement. Work to make Service Desk the single source of truth and service delivery channel for IT. Monitor and manage phone queue (participating in escalated calls as needed). (20%)

4.  Oversee Solutions repository and ensure top quality solutions are available to the staff. Develops Service and Business Level Agreements to set expectations and measure performance. Develops an effective and workable framework for managing and improving customer IT support in the organization. Advise management on situations that may require additional client support or escalation. (20%)

5.  Manage process for communicating outage/emergency activities to the organization. Manage vendor relationships as it depends on daily operational needs. PO review and approval/budgeting responsibility. Review survey feedback to improve services, tools and support experience. Keep confidential all applicant, client, and verification and company proprietary information. (10%)

While this month presents another simple metric, we figured that gives us time to focus on an important principle of successful business intelligence roll-outs in general. Nomenclature.

We can all agree that metrics bring a new level of visibility to the performance of an organization. With the increased visibility comes increased communication about the effectiveness (or lack thereof) associated with the new information at your fingertips. But… what do we mean when we say emergency change, failed change or expedited change? Are we speaking the same language?

This might remind us all of the famous Clintonism around the definition of “is”. Seriously, there is an issue that needs to addressed in terms of communication around the newly established metrics. Establishing proper nomenclature becomes a primary objective. Because what happens when you start sharing your unsuccessful change metric? You can get a bunch of backseat drivers telling you how you need to improve performance in IT! What’s worse, they may not understand what the metric means or how it was calculated. This could be compounded by the fact that you might have bonuses or MBOs tied to performance around these metrics. So, the stakes are high.

Still, that’s not to say the increased communication and visibility is bad. In fact, the flip side of the newly empowered performance-based organization is that people really do care about the performance of the company and might help you think of ideas that benefit your organization that you hadn’t considered.

Bottom Line:

Understand what you are calculating (data model), what it means (nomenclature) and why you are calculating it (methodology)… with a communication plan before you share. Of course, we’d love to help you with that.

Metric(s) of the Month for September and October, Change Management

Failed Change Request - “Change Requests with a Status Reason of ‘Unsuccessful’ or ‘Backed Out’, a Status of ‘Completed’ or ‘Closed’” and a Change Type of ‘Change’”

Unsuccessful Changes – “Change Requests with a Status Reason of ‘Successful with Issues’, ‘Backed Out’, a Status of ‘Completed’, ‘Closed’, or ‘Cancelled’, and has a Change Type of ‘Change’”

Again, keep in mind that the metrics below are Remedy ITSM 7.X specific metrics. In our metrics catalog, we define over 1000 metrics and KPIs across 8 process areas. Of course, they’re nicely organized so as not to overwhelm you with metric nausea. Our Service-Now platform will be released in February. We currently support Remedy ITSM, custom Remedy and any relational back-end with services.

Of course, it always depends on your processes, people and ITSM configuration!

We’re certainly not giving away the farm with this one.  Many of you are probably calculating this, but sometimes that can make for the most interesting discussion, so we thought we’d spend a moment discussing a pretty common metric.  This month’s metric is time to build, which is simply measured (Time To Build/Releases Built).
Of course, that’s where the simplicity stops because:

• Organizations use different methodologies for their release process (SDLC, Agile, Etc.)
• Different applications have inherently different build times
• Developers have differing levels of productivity inherently
• Certain applications are more critical than others.
• Release management isn’t for applications only – It could be used for any CI that’s new to the environment which needs to go through a testing process (QA, Performance, Etc…).  This is an over-simplification, but hopefully you get the point.

So, we thought it might be a good time to introduce the Northcraft concept of perspectives.  A perspective is different for each process area, gives you a unique and specific look at items that are critical to the process area and helps to account for what is particular and unique to the area.
For example – Incident Management might have perspectives of Location, Categorization and Assignee.  Whereas Release Management might have the perspective of CI Class, Prioritization, Phase and Assignee.  So, there can be overlap in the perspectives, but there are also unique perspectives for each process discipline.

• Purpose:  Measure the productivity of your releases in relation to priorities of the business, capabilities of the implementation team and release methodology.
• External:  Fully external with Northcraft date management.
• Context: This metric is most fully understood using CI classes and the Definitive Media Library to ensure proper productivity potential and prioritization.
• Objective? Yes.

Finally, in our opinion, this is a metric that doesn’t lend itself well to benchmarking due to the vastly different methods companies use to implement releases.  Have a nice labor day weekend!  Hopefully we’ll see you at Fusion!

Northcraft