See also:
CPM: What it is and how it is different from traditional approaches? -- Part One
CPM: Systems and Steps to CPM -- Part Two
CPM: 12 Best Practices in Implementing a Solution -- Part Three
CPM: Selecting the Right Technologies -- Part Four (A)
CPM: Selecting the Right Technologies -- Part Four (B)
CPM: Emerging Technology Systems -- Part Six

          In the last article [Part Four (B)], we took a look at how to evaluate CPM software. In this article we will look at how to evaluate CPM software vendors.

Vendor Types

          CPM vendors fall into three basic categories:

   Business Intelligence (BI) vendors who specialize in tools for holding, reporting and viewing any kinds of data

   BI Application Vendors who have traditionally focused on providing planning, budgeting and consolidation solutions

   ERP/General Ledger vendors

          Each of these organizations offers various levels of expertise and are usually influenced by their background. But perhaps what is most important when researching vendors is to make sure that they are committed to CPM, have a vision for its development and have suitable applications that meet the needs of CPM. Do they have expertise in this area or are they just jumping on the latest bandwagon? Check out whether they are reselling someone else's solution or if it is there own. If it is someone else's then it can get confusing trying to work out who is responsible for support and fixing problems that may arise.

Create a Shortlist

          The internet is a good place to start when researching vendors. Most vendors will provide overviews of their applications on their web sites and usually give web-based seminars and even 'personal' demonstrations - all from the comfort and convenience of your office. The objective when preparing a shortlist is to see which vendors are best able to meet your requirements who will then pass onto the next stage of the evaluation process.

          Beware of the 'canned' demonstration. Vendors will typically only show you what they want you to see. A better way is to inform the vendor about your basic need and ask for a demonstration on how they would meet those specific needs. A good vendor will respect your time and get straight to the point saving both your and their time.

Detailed Product Evaluation

          Once a short list has been created, the next step is to conduct a detailed evaluation. Make sure you that you actually see the capabilities that you need to meet your requirements -- do not rely on a simple 'yes' from the vendor. It's the only way you can be assured that the product does what you want.

          Evaluating vendor products is best done in a pre-planned systematic way that ensures you evaluate all vendors in the same way. In particular:

   Carry out the evaluation on a full-size application, i.e., the maximum number of dimensions and members. Don't be fooled by someone giving a slick demonstration of building a single user, simple application in front of your eyes. The real system will have multiple users and will need to meet the differing demands of those users.

   Make sure the measures in the demonstration model are realistic for your purpose. For example, have some measures analyzed in more detail by a different business dimension such as contribution by product.

   Ensure that the demonstration system is set up so the maximum number of people involved in the process can use it. A multi-user system quite often looks substantially different from a single-user system. The system should also be able to handle the number of people that may be involved in the next 12-18 months as the system develops.

   Does the system contain all the functionality for CPM? Does it support scenario planning, top-down target setting, bottom-up budgeting, management reporting, financial consolidation, end-user analysis? And are these provided in a single application? See Part Four (B) for an overview of the attributes of a CPM system and compare the products to this.

   Look at the capabilities provided for reporting exceptions. Do users have to look through detail reports or can exceptions be reported as they are generated?

   Can the production and delivery of reports be automated? This can save substantial amounts of time and effort.

   Investigate what capabilities are provided for end-user analysis. Can users drill down into a variance and then calculate new analyses? Ask the vendor to create a variance on the fly, without giving them prior warning. For example, calculate actual/budget variance percentage for all companies. Sort the result to show the top 10 performers by revenue. Then show this for total costs. Can this variance be color-coded or presented as a chart? Systems that cater for end-user analysis will be able to do this easily.

   Does the system require you to duplicate data e.g., is reporting performed from a separate data store. Duplicating data means that data transfers need to be set-up, maintained and run, with the chance that someone may not be seeing the latest version of the data.

   What technologies have to be learned to maintain and support the system? If a new technology is involved, new skills will be required that will extend the learning curve.

   Make a change (modify one of the measures) and see what is involved in getting the change reflected at the user site. Add a new budget center and perform a re-organization. Check that these changes are automatically reflected in data entry screens, reports and analyses. If they are not, then what effort is required to do this?

   Check the effort required to roll out the application to the new user.

Other Considerations

   Calculate the True Cost of Ownership -- the software cost is often not the largest cost you will incur. You need to take into account implementation costs, user training, software maintenance costs, ongoing maintenance, and the life of the product. If the system won't scale to meet a future need, the chances are that the whole system will have to be thrown away and you will need to start again -- which is very expensive in both time and money.

   Research References -- make sure the vendor has done this type of application before. Most vendors have impressive client lists, but are the clients using the application in the same way you would? Check that the reference application is of the same size and complexity as the one you want to implement. Find out the real cost of implementation and the kind of support they received.

   Find Out Product Direction -- systems for CPM are currently undergoing a transformation as organizations seek to implement enterprise wide solutions. A number of vendors are developing new products to meet this need, so is the product being evaluated one of the new ones or is it one that is going to be replaced? If it is going to be replaced, what will happen to your application?

   Evaluate Support -- what kind of support is available? Can you get help 24 hours a day? What happens if you find a problem late one night -- what kind of support will the vendor be able to give you? And if you have users around the world, what kind of support will they get and will it be in a local language.

   Accurately assess implementation effort. Take time to plan out the implementation with each vendor and understand the efforts that will be required on both sides.

Create a scorecard

          Make sure that the same detailed evaluation is carried out for each vendor. Use a 'scorecard' to record results and comments. Never rely on a function that will be available in 'the next version'. Software vendors are notorious at missing deadlines and releasing products that are not quite as functional as described in the pre-release plan.

Agree Terms, Conditions and Deliverables

          Before a decision can be taken, understand exactly what a vendor is proposing that will then be used as a binding contract. Your requirement is for a solution -- whereas the vendor is primarily interested in making a software sale.

          With consultancy proposals, a lot of time will be required in thoroughly investigating exactly what you want. If you cannot specify this, then you may need to enter an agreement where the first stage is to specify your actual requirements.

Make the Selection

          This final stage can only happen when you have all completed a common evaluation of all steps for all short listed vendors.

          In the final article (Part Six) in this series, we will take a look at some of the new emerging technologies designed specifically to support the formulation, communication and monitoring of strategy.

About the Author:

Profile of Michael Coveney

Email: mcoveney@comshare.com

See also:
The Strategy Gap: Leveraging Technology to Execute Winning Strategies
Michael Coveney, Brian Hartlen, Dennis Ganster and David King
John Wiley & Sons, 224 pages, US$27.97 / £22.98 (Amazon.com)

Thomas A. Faulhaber, Editor Email: editor@businessforum.com Telephone: 617.232.6596 -- FAX: 617.232.6674 227 Fuller Street Brookline, Massachusetts 02446.5757

consult@businessforum.com

URL: http://www.businessforum.com/Comshare05.html
Revised: May 5, 2003 TAF

© Copyright 2003 Michael Coveney / Comshare, All Rights Reserved