In the past, I worked for a large software company (actually several, that’s why I can tell this story anonymously!) in a pre-sales organization. Well, we had a Key Performance Indicator for the consultants to spend 240 hours of our time in customer-facing situations. The top people in each region were awarded points which could be exchanged for pretty nice gifts from the company store. If you won, you could expect to receive @$2,500/year in addition to your normal pay. In general, this was a great concept which incentivizes the consultant to get out of his/her shell and help the people that pay the bills! The KPI was even set at the right amount of  hours (about 50% of total time) because it was quite a stretch to spend that much time in customer activities as we had plenty of paperwork, training and technical work to accomplish in that same period. Personally, I felt like I had a real advantage in this situation because of the relationships I had built in the region over the years.

Reality… here’s how it set in:

- There were definitely 1 or 2 consultants who didn’t have a problem fudging their number.

- Every consultant had a different definition of “customer-facing” time – Did prep time count? Time thinking about the meeting? Travel time to and from the meeting?

- Also, the manager was awarded points for total time in the field for all of his/her consultants, so he/she didn’t really have a reason to argue against people who were optimistic about their time spent!

- This information was entered into an SFA system by the consultant. We were kind of biased – right?

- The manager had favorites – Now, let’s be honest. We all have favorites… Of course we do. You can’t just turn off your human nature and stop liking/respecting people that you actually like and respect. This favoritism ensured that tough questions weren’t asked to certain individuals.

So what happened? I busted my hump to make it into the yellow (KPI could be Red, Green Yellow… gotta love the traffic lights). Then, I was raked over the coals for being yellow. To this day, in my opinion, I believe the 2 consultants who won the 1st quarterly contest were absolutely, positively not in front of customers as much as I was. This was a pretty small group, so I had a pretty good idea. Was this demoralizing? Definitely. Did I stop trying as hard? No, but I kept on bringing up the topic on conference calls to the annoyance of everyone (including myself).

So what? What are my takeaways from this situation given my chosen career in analytics?

- Having a good metric and goal remain valuable, but the implementation really matters most.

- Beware of bias in a metric at multiple levels in the organization.

- Beware of carrots and sticks based on key metrics. Don’t avoid them, but be certain that proper controls are in place to ensure compliance. By controls, I mean rules, the interpretation of rules and the authority structure. Don’t be fooled, even in seemingly flat organizations, there is a final authority.

- Ensure that the systems in place to capture the metric have a chance of getting the data without being subject to influence – External/Objective (this is covered in our first blog in greater detail.) Northcraft’s products give you an advantage in this area.

- Transparency – This metric provided cover for the department and gave the appearance of transparency, but the truth was that 2 people were rewarded with incentives that shouldn’t have been given to them.

How does this apply to IT?

Remember these concepts when calculating Availability SLAs… and feel free to call us for help.

If you’re interested…

We’re been noticing through our own analytics that the most popular Northcraft blogs of 2011 have been:

#1: State of the ITSM Market, Release 3 – Google “ITSM Market Share”

#2: Service Desk Manager Job Description – Google “Service Desk Manager Job Description”

#3: Change Management Metric of the Month – Google “Failed vs. Unsuccessful Changes”

which has helped us come to the realization that… the “Metric of the Month” needs to be end-of-lifed and replaced with an article/story or “small tidbits that provide quick value” format (like the above). While we think you’ll agree that Metric of the Month is a very catchy title… it doesn’t appear to be resonating. And, since our major advantage in business is responsiveness to change… we must bid thee farewell MoM.

Finally…

Since this is our first article of the year, we’d like to end by saying THANK YOU to our customers who have helped Northcraft Analytics:

- Grow 900+% last fiscal year (CY2011) – Amazing! Again, Thank you!

- Establish industry analyst recognition – EMA (and we didn’t even have to pay for it! Yet.)

- Grow our metrics library by more than 100% – Good practices are getting a little better.

- Extend our platform into Service-Now – We’ll be happy to talk to customers about what’s next. The product direction is exciting.

In general, our theory that BI platforms are consolidating (with Microsoft in the lead at 45+%), BI applications are rising and enterprise IT departments are open to BI applications that provide a quicker route to value for non-core business functions appears to be validated by the market. But, who knows what 2012 will bring?

If there’s one thing we can all agree on it’s the fact that metrics alone don’t improve a business, but the interpretation of the metrics, the quality of the metrics and the interpretER matter the most.

http://blogs.enterprisemanagement.com/charlesbetz/2012/01/26/cmdb-data-warehouse/

Hello again all! I’m sure it’s been quite a busy time of year for us here in the IT world and the vendor world. So, it took me until the last day of the month to get that Metric of the Month out.  In any case, Knowledge Management is the process that has captured my fancy this month due to an interesting company & story that has grabbed my attention for a number of years.

Back in 2001, when I was on an engagement with Genuine Parts here in Atlanta, I learned what I still believe to be a “Best Practice” in Knowledge Management. There was a gentleman named Dave Cockerill (VP of Accounting and Information Systems at the time) who described to me a little bit about how they handled knowledge management at GPC. We were replacing their exiting mainframe-based solution with Remedy Knowledge Management and Dave wanted to import all of their existing knowledge management articles into the new system. My position was… “What? You want to take a bunch of EBCDIC mainframe text and pump it into our fancy new HTML-based Natural Language Knowledge System? “Uh… NO, BAD IDEA.” I was still thinking a bit like a technologist in those days (I still do sometimes!). Anyway, Dave proceeded to explain to me that they paid their IT analysts .50 every time an article was used. This was possible because the stores were paying for support. But, essentially, these articles were VERY well-written and valuable because of this incentive that GPC provided, even if the articles weren’t in the nicest format. We did improve their system a little bit since we gave them the capability to see which articles has been viewed as well as used, but that’s a story for another day…

As a result of this story, maybe I should have chosen # of articles used, but… this month, the metric is:

Average # of Days to Create Published Solution Article

The time in days from the ‘Create Date’ of the Article to the ‘Published Date’ for Articles with a Status of ‘Published’

Purpose: Measure the productivity of your end to end process for creating Knowledge Articles.

External: Not really, since this is the measurement of the full process, there are many fingers in the cookie jar potentially. You have the knowledge author, approver and editor roles at a minimum.

Context: This metric is most effective when focused on content initiated due to the resolution of problems (from the problem management process that is) around critical business services.

Objective?: Yes, this metric can be used objectively for comparison purposes against publish times for other articles (helping to prioritize time based on importance of solution).

To manage the performance of Level 1 and Level 2 services & support to clients (internal and external) and ensure that service levels are achieved. To ensure that customer expectations are met or exceeded. Responsible for ensuring the staff are meeting and exceeding expectations in regards to performance, meeting defined metrics/benchmarks, and that standards and processes are followed to provide effective customer service and meet requirements.

Key Responsibilities (% of Time)

1.  Oversee 100% of the requests, incidents and problems. Manages and coordinates urgent and complicated support issues. Act as escalation point for all requests and incidents. Develop and mature phone/ticket escalation processes to ensure free flowing escalation and information within the organization. Determine root cause of issues and communicate appropriately to internal and external customers. (25%)

2.  Train, coach and mentor Service Desk Specialists (Level 1 / 2) including career development. Oversee staff activities. Builds/obtains (from other departments) training material for support staff. As needed, schedule employees working times and provide backup support. Interact with internal and external customers. (20%)

3.  Provide data and reporting of KPI’s and trends to IT department and others in ad-hoc, weekly, monthly and as needed. Will drive Ticket Deep Dive and develop strategies for improvement. Work to make Service Desk the single source of truth and service delivery channel for IT. Monitor and manage phone queue (participating in escalated calls as needed). (20%)

4.  Oversee Solutions repository and ensure top quality solutions are available to the staff. Develops Service and Business Level Agreements to set expectations and measure performance. Develops an effective and workable framework for managing and improving customer IT support in the organization. Advise management on situations that may require additional client support or escalation. (20%)

5.  Manage process for communicating outage/emergency activities to the organization. Manage vendor relationships as it depends on daily operational needs. PO review and approval/budgeting responsibility. Review survey feedback to improve services, tools and support experience. Keep confidential all applicant, client, and verification and company proprietary information. (10%)

While this month presents another simple metric, we figured that gives us time to focus on an important principle of successful business intelligence roll-outs in general. Nomenclature.

We can all agree that metrics bring a new level of visibility to the performance of an organization. With the increased visibility comes increased communication about the effectiveness (or lack thereof) associated with the new information at your fingertips. But… what do we mean when we say emergency change, failed change or expedited change? Are we speaking the same language?

This might remind us all of the famous Clintonism around the definition of “is”. Seriously, there is an issue that needs to addressed in terms of communication around the newly established metrics. Establishing proper nomenclature becomes a primary objective. Because what happens when you start sharing your unsuccessful change metric? You can get a bunch of backseat drivers telling you how you need to improve performance in IT! What’s worse, they may not understand what the metric means or how it was calculated. This could be compounded by the fact that you might have bonuses or MBOs tied to performance around these metrics. So, the stakes are high.

Still, that’s not to say the increased communication and visibility is bad. In fact, the flip side of the newly empowered performance-based organization is that people really do care about the performance of the company and might help you think of ideas that benefit your organization that you hadn’t considered.

Bottom Line:

Understand what you are calculating (data model), what it means (nomenclature) and why you are calculating it (methodology)… with a communication plan before you share. Of course, we’d love to help you with that.

Metric(s) of the Month for September and October, Change Management

Failed Change Request - “Change Requests with a Status Reason of ‘Unsuccessful’ or ‘Backed Out’, a Status of ‘Completed’ or ‘Closed’” and a Change Type of ‘Change’”

Unsuccessful Changes – “Change Requests with a Status Reason of ‘Successful with Issues’, ‘Backed Out’, a Status of ‘Completed’, ‘Closed’, or ‘Cancelled’, and has a Change Type of ‘Change’”

Again, keep in mind that the metrics below are Remedy ITSM 7.X specific metrics. In our metrics catalog, we define over 1000 metrics and KPIs across 8 process areas. Of course, they’re nicely organized so as not to overwhelm you with metric nausea. Our Service-Now platform will be released in February. We currently support Remedy ITSM, custom Remedy and any relational back-end with services.

Of course, it always depends on your processes, people and ITSM configuration!

We’re certainly not giving away the farm with this one.  Many of you are probably calculating this, but sometimes that can make for the most interesting discussion, so we thought we’d spend a moment discussing a pretty common metric.  This month’s metric is time to build, which is simply measured (Time To Build/Releases Built).
Of course, that’s where the simplicity stops because:

• Organizations use different methodologies for their release process (SDLC, Agile, Etc.)
• Different applications have inherently different build times
• Developers have differing levels of productivity inherently
• Certain applications are more critical than others.
• Release management isn’t for applications only – It could be used for any CI that’s new to the environment which needs to go through a testing process (QA, Performance, Etc…).  This is an over-simplification, but hopefully you get the point.

So, we thought it might be a good time to introduce the Northcraft concept of perspectives.  A perspective is different for each process area, gives you a unique and specific look at items that are critical to the process area and helps to account for what is particular and unique to the area.
For example – Incident Management might have perspectives of Location, Categorization and Assignee.  Whereas Release Management might have the perspective of CI Class, Prioritization, Phase and Assignee.  So, there can be overlap in the perspectives, but there are also unique perspectives for each process discipline.

• Purpose:  Measure the productivity of your releases in relation to priorities of the business, capabilities of the implementation team and release methodology.
• External:  Fully external with Northcraft date management.
• Context: This metric is most fully understood using CI classes and the Definitive Media Library to ensure proper productivity potential and prioritization.
• Objective? Yes.

Finally, in our opinion, this is a metric that doesn’t lend itself well to benchmarking due to the vastly different methods companies use to implement releases.  Have a nice labor day weekend!  Hopefully we’ll see you at Fusion!

Northcraft

State of the IT Service Management Market, 2011, R3

Thank you once again for your patronage into our blogosphere.  Stop by anytime.  We’re tackling a tough topic this week, but one that’s necessary to have a tactical understanding of continuous improvement.  Most of us in IT operations and support recognize how necessary continuous improvement is (at the strategic level), but the front lines can be rough sometimes.

So while we know much of this might sound like common sense…   Many times common sense requires a thoughtful description… and we hope this article is.

At times, we walk into companies that want to shy away from reporting the poor/negative results that are almost certain to show up when you start implementing external metrics.  Of course all companies have issues, but it’s tough to hear specifically what they are!

As an example… many times, it can be shocking to learn that your first call resolution is actually 48% when you have been reporting 80% to management for several years.  Or, that your changes take longer to implement (and fail more often) then you actually knew.

But, it’s important to know where you’re falling short and ensure that your IT department doesn’t develop a culture of fear in reporting bad news.  It’s best to know your weaknesses, reflect on them, report them and consider how to improve.  What are some of the ramifications of permanently delaying communication?

- Lower Customer Satisfaction

- Function outsourcing

- Termination

While there may be short-term benefits, or even good reasons for holding off immediate communication to a broader audience, there are also major benefits in responsibly communicating results such as:

- Hearing a unique perspective on the causes of failure

- Learning what is inside your control and outside

- Finding unique solutions from other individuals coming from an unbiased perspective

Given that the stakes are high… be proactive!  IT is a tough place to be these days.  Companies are still looking to cut costs.  If management feels that you are doing all you can to stay on top of performance, you’ll be in better shape!

So what do we recommend to soften the blow?  Proactively communicate principles on what is acceptable and unacceptable for the IT department.  The specific definition might be different for your organization, but the principles outlined by examples below can be a simple guide:

Here’s what should be acceptable (for the Change Process):

- An unintentional technical mistake during a scheduled change window.

- If you’re an IT Analyst, admitting that you don’t have all of the skills necessary to perform the change (when you realize it).

- Mistakes due to extended work hours.

What’s not acceptable?

- Lack of communication about a mistake during a scheduled change window.

- Misrepresentation of actual change implementation timeframe post completion.

- Failure to ask for help the moment you realize you don’t know something.

So, encourage your teams to be humble in reporting their mistakes.  Truly, the biggest mistake is intentional mis-communication… and not asking for help!

State of the IT Service Management Market, 2011, Redux

Since this has been such a popular topic, we thought we’d re-issue our research with some clarifications, updates and additional thoughts. If you’ve participated in the blog on our website… we really appreciate it. This has spurred some excellent and needed discussion on this topic! Of course, there is a great deal of bias around this topic due to the product-oriented nature of the research.

BMC is #1 in the market with 38.6% market share (Includes BMC Remedy ITSM and Custom applications, Service Desk Express, RemedyForce and BMC Remedy on Demand), HP Service Manager is #2 at a little over 20%, CA Service Desk Manager is #3 at 7.2% (Source is Gartner MQ 2011 on this market share information), IBM Tivoli Service Request Manager is @4% and Service-Now (first major ITSM Software as a Service-only offering – SaaS) is @3%, but growing the most quickly.

In terms of ranking, we would rank the vendors using these criteria:Interface, Architecture, Functionality and Price (list pricing, because any vendor offers special discounts based on a wide range of criteria). Also, you should really go into depth on each module (Incident, Change, CMDB, Release, etc…, because the devil is definitely in the details). For example, you could argue that Incident Management is a commodity application at this point, but CMDB definitely is not (in terms of functionality and architecture). So, even though we’re about to generalize, it’s a generalization across all of the ITSM modules. So, this would be more of an enterprise ITSM tool ranking (in our opinion)… in order of market share:

- BMC Remedy ITSM 7.6.4 – Functionality #2, Architecture #1, Pricing #4, Interface #2, Sales Team #4, Implementation Community #1

- HP Service Manager 9.2 – Functionality #1, Architecture #4, Pricing #5, Interface #4, Sales Team #2, Implementation Community #2

- CA Service Desk Manager 12.5 – Functionality #3, Architecture #5, Pricing #2, Interface #5, Sales Team #3, Implementation Community #5

- BMC Service Desk Express 10.1 – Functionality #5, Architecture #3, Pricing #1, Interface #3, Sales Team #4, Implementation Community #4

- Service-Now 11.5 – Functionality #4, Architecture, #2, Pricing #3, Interface #1, Sales Team #1, Implementation Community #3

Additional remarks, thoughts and comments based on blog discussion:

- IBM Tivoli Service Request Manager has now moved from 4th – 5th in the market due to Service-Now’s momentum.

- Numara remains a key player in the SMB space, but the SMB space isn’t being covered with this research. This is more geared towards Fortune 500 ITSM.

- Service-Now has now surpassed CA to become #3 in this market as they have recently hit $120 million in recurring revenue.

- Remedyforce is built on Force.com, therefore it is built on arguably the best SaaS application architecture available. So, it is a native SaaS architecture, opposed to popular opinion. Many customers have confused this offering with BMC Remedy on Demand which still leverages the AR System architecture. Remedyforce currently has limited functionality with Incident Management, Self-Service, limited Knowledge Management (no NLS) and limited configuration management. BMC Remedy on Demand has the characteristics rated above, but is offered in a SaaS model.

- ICCM is a promising new platform that leverages a native Business Process Management engine (Metastorm). While ICCM’s presence is limited, BPM engines do give customers unprecedented flexibility in automating unique processes outside of IT.

- ITRP is a new native SaaS offering that is positioned as serious and packaged ITSM. They have purposefully taken the opposite approach as ICCM as an application that isn’t designed to be customized, but leverages best practices. The interface is truly excellent. So, for those customers looking for a packaged ITSM application with robust functionality, a superb user interface and a simple SaaS licensing model, ITRP is worth considering.

For more information, or ITSM vendor consulting, please contact us at our website: http://www.northcraftanalytics.com