Author: Colin Mayers
The information contained in this section is based around what I have found, the observations and conclusions are based primarily around the operational requirements in order to get things done, to achieve a good working solution. To that end I have produced both thoughts and checklists. As with ITIL itself the end need will always be to use what is appropriate, 'horses for courses' is a very old saying, it has survived because of the elements of truth in it.
The original thinking behind the article is based on the talk given to the Wales and South West regional itSMF meeting at the Land Registry in Plymouth in March 2003.
It came about because Ray Paice the regional organiser was looking for contributors and it occurred to me that I had a somewhat rounded perspective on the reasonably 'hot topic' of 'outsourcing'.
Within the previous 10 years I had been a permanent employee faced with the spectre of outsourcing costing me my job. I have also, in my contracting life, worked for the biggest IT company in the world to deliver a managed service, (commercial world) to their customer, in addition I have worked for another company providing managed services to the UK Government (Home Office) and am currently working for a government department that are looking to outsource all their services, with the exception of their Service Desk, to a number of strategic suppliers.
The aim of the current government agency IT department being to 'manage' rather than do - an aim that a lot of us may well have!
1. Acme1 - This and the subsequent case studies were real projects
The time was mid 2000, almost everyone in IT was happy to have got past the Y2K nightmare that never truly materialised - so was it good planning or actually something that was never really a problem? Answers are not required, not even on a postcard.
Many organisations that had chosen not to go with the first rush of 'e' everything were now deciding to do so, perhaps in a more considered fashion.
Whilst working on a change to a very large out sourced project I came across an example of how a badly written contract can actually distort the way an organisation's IT operates or in this case how the services are delivered as part of it.
The organisation that operates in the international financial sector is, for the purposes of this article, called Acme 1.
Their infrastructure and its management had been outsourced to an organisation that for the purposes of this article is called Emca (yes it is Acme backwards - so no points for original thought then!!).
My involvement in the project took place when Acme 1 were planning to add 'e-banking' to their considerable presence in the finance world.
Emca had bought in a third party to design and deliver the infrastructure and also to supply the operational model for their proposed e-services.
This section illustrates how the contractual model for the infrastructure out source had built in effects that were detrimental to efficiency, effectiveness and financial management
C) The story
As good ITIL consultants we first investigated what was currently been done at Acme 1 in terms of Service Management so that we could ensure that our operation model could integrate with the existing processes and contractual arrangements. The plan being to model what we considered the operational optimum and then, if required do a gap analysis.
What follows is what we found.
Acme1 had entered into a contract with Emca for them to charge on what was described as a 'cost plus' basis. This meant that if Emca installed an item of infrastructure, Acme 1 would pay for the item itself plus a percentage for Emca 'doing it'
The effects of this part of the contract were:
- The customer, Acme 1, was always right - this laudable principle however was taken to the extreme, if a customer manager wanted a new server they got one, whether or not there was spare capacity on existing servers.
The driver for the outsourcer being that there was no financial incentive for Acme1 to use hardware attached to the infrastructure efficiently, in fact if they did so it adversely affected their revenue stream. The converse of which was that the more hardware was purchased and installed the greater the revenue stream.
During investigations for base lining the infrastructure the following details were discovered. Acme 1 had 25,000 Microsoft NT Workstations supporting their follow-the-sun business. Supporting these workstations were a staggering 13,000 Servers. During investigations on capacity it came to light that the average server utilisation was 13%.
It certainly was not all bad within the organisation; we found that Acme 1 did some things stunningly well. They had developed an Intranet based ordering system that worked beautifully.
Their system had an on-line catalogue from which managers could order, if something was requested from outside the catalogue the request went automatically to the technical evaluation committee where the request was checked - if it fitted into the strategy and architecture for Acme 1's IT the item was put onto the catalogue.
Authorisation of orders was managed via the Intranet, built in rules drove authorisation pages through the Intranet with agreement being confirmed by the click of a radio type button
Complete order tracking was maintained and available using Acme 1's intranet system until delivery confirmed - with all data being held by Acme 1.
With Acme 1 owning both process and data it was looking very slick indeed until the point of equipment delivery, at that point the ordered item was handed over to Emca. Unfortunately the quality configuration management data was not, Emca had the role and responsibility for installing the item onto the infrastructure.
It was so disappointing to see that the Configuration Management information was simply wasted. Emca, as outsourcers, had their own systems and Acme 1 apparently saw no reason to pass them all the information they had. I refer to the collection of CMDB data again in Section D - Lessons Learnt in both subsections a) and b) below.
Emca had no Configuration Management Database (CMDB) at the time order manage the infrastructure, they had recognised the fact but at that time had no immediate plans to rectify the issue.
This led to some interesting operational practises being required, especially when there was a requirement for a roll-out, usually software, out to a number of workstations.
Emca used a discovery software that went out and checked the status of every machine, the problem was that with the numbers of installed servers and workstations there was insufficient bandwidth to cope with an investigation of the whole network and attached devices at one hit - it took four days as a background task!
Following the enquiry of the entire estate the rollout would then be planned and take place over 5 days - any workstations not upgraded by that time were passed to the service desk for individual catch up.
The Change Management Process was well developed and mature and again rule driven through Acme 1's Intranet. It was extremely business focussed, it's major plus. Changes were identified in terms of the services that would be affected and service owners then had the opportunity to 'yes', 'no' or 'more information required' using the intranet pages and radio buttons, similar in look and feel to the ordering system mentioned above.
Credit where credit is due the consistency of look and feel was well done and helped greatly in its usage.
It worked well for their current set-up but the thoughts of an emergency process and the kind of speed required for putting up a holding page or for changing financial detail on web pages had so far eluded them
D) Lessons Learnt
a) At the time
This section deals with the way things seemed at the time in terms of lessons learnt, the next section has all the joys of hindsight, and individual topics are also fed into a consolidated
The contractual payment arrangements had the effect of distorting the efficiency of how the infrastructure was run. It just seemed almost designed to lead to over capacity and inefficiencies. Whilst speaking to Emca it was clear that they were aware of the issue and said that they had plans to approach Acme 1 with a different cost charging model.
The CMDB data not being passed between the organisations, it should be pointed out at this juncture that I was not aware of the history of the contract between Acme 1 and Emca, so I am completely unaware of how the contract had evolved, there may have been historic reasons for the clear separation, it just seemed like an awful shame and a waste of good data.
The use of the intranet, driven by rules, set up to look and feel the same over a number of different functions was really good, perhaps the most important thing was that it worked for them, and that it was used by both the business and IT.
Change Management based entirely on the business values is absolutely correct, but with a CMDB they could have made it so much slicker.
b) With Hindsight
As ever, the passage of time makes things clearer and perspectives alter, the following are the way I see things now and probably have a more considered view as to some of the reasons why things were the way they were.
Contractual arrangements evolve, without the specific knowledge of why they were the way they were it is easy to stand back and be glib. However, as a consultant, one is usually faced with things as they are, with the great advantage over the people currently involved in the day to day management and running of the services being the time and ability time to stand back and view the effects without having been involved in the fire fighting.
The financial arrangements of all contracts are crucial, the contract in this case appeared to be equitable in as much as the supplier was paid for work done. However the real service improvement issues such as the tuning of capacity and supplier based initiatives to help the customer were more difficult to see.
There were not clear drivers for cost savings and efficiencies. Things like economies of scale in purchasing did not show benefit to the supplier owing to the 'cost plus' model. Capacity Management was driven to ensure that there was never a lack of it - which is of course is a prime aim, but the efficient use of capacity had no direct pay back.
With all large organisations the rate of change is enormous and the financial market place is a very competitive business. As well as making life within IT challenging in support of the business it does also have an advantage in the realms of Configuration Management. The higher the rate of change the easier it is to instigate a rolling build of a CMDB.
The use of discovery tools again and again and the bandwidth concerns could have been avoided, although at that time it showed itself to be a useful tool that could easily have sat in the Configuration Managers cupboard for audits and verification purposes.
A good CMDB would have add speed and efficiency to the Change Management Process by ensuring that all related services and hence their owners were speedily identified.
I have always believed that one of the prime and best sources of accurate CMDB data is the ordering process. In any normal organisation they will know why they are buying something, usually what they intend to do with it and who will be its initial owner. My advice on building a rolling CMDB is to always build a good working interface with procurement and as well as Change Management.
As the data within the CMDB builds and it will have credibility because it is correct and being maintained. There will come the point where you reach critical mass, the point where you have sufficient good CMDB data that it is worth auditing to collect the remainder will usually become self evident. The higher the rate of change the quicker the point of critical mass will be reached.
About the Author - Colin Mayers is the Managing Director of Mayers Consulting. Colin operates across all market sectors and provides a breath of experience with regards Service Management. As for formal qualifications he holds the ITIL Managers Certificate, ISO20000 Internal Auditor and Consultant Certificate and also the Prince 2 Practitioner.