The DeLone and McLean Information Systems (IS) Success Model as a framework and model for measuring the complex-dependent variable in IS research. In this paper, we discuss many of the important IS success research contributions of the last decade, focusing especially on research efforts that apply, validate, challenge, and propose enhancements to our original model. Based on our evaluation of those contributions, we propose minor refinements to the model and propose an updated DeLone and McLean IS Success Model. We discuss the utility of the updated model for measuring e-commerce system success. Finally, we make a series of recommendations regarding current and future measurement of IS success.
Three levels of information management
Written by James Robertson, published March 1st, 2005
When developing an information management strategy within an organisation, it is useful to consider information needs on three levels:
corporate
team, division, business unit, etc
individual
The needs of each of these three levels must be met if a coordinated and effective solution is to be maintained in the long-term.
Failure to address any one of the levels will lead to areas of the business or individuals finding their own solution, which may not fit well within the strategic goals of the organisation.
These are not new ideas, but they will be explored in the context of intranets and other corporate information systems.
Corporate
At the top is the corporate information that is useful for the whole organisation. This 'global' information is generally fairly well addressed by the corporate intranet (even if the intranet itself needs improvement).
Examples of corporate information include policies and procedures, HR information, online forms, phone directory, etc.
Interestingly, there may be a limited amount of truly global information, and it may not deliver the greatest (measurable) business benefits.
Team, division, business unit
The middle level is perhaps the most interesting, as it covers all the information shared within teams, divisions, business units, etc. This information may be critical to the day-to-day activities of the group, but of little interest to the rest of the organisation.
Examples include project documentation, business unit specific content, meeting minutes, etc.
This level is generally poorly-served within organisations, although collaboration tools are increasingly being used to address team information needs. It is also being recognised that it is this 'local' information that may be the most valuable, in terms of driving the day-to-day activity of the organisation.
Individual
At the lowest level is the personal information needs of staff throughout the organisation. Examples include correspondence (both internal and external), reports and spreadsheets.
In most organisations, staff must struggle with using e-mail to meet their information management needs. While staff generally recognise the inadequacy of e-mail, they have few other approaches or technologies at their disposal.
Note that some organisations (such as consulting firms) are heavily dependent on personal information management amongst their staff.
Managing the levels
When managing the information within each of the three levels, consider the following:
An information management solution must be provided for staff at each of the three levels.
If corporate solutions aren't provided, then staff will find their own solutions. This is the source of poor-quality intranet sub-sites, and other undesirable approaches.
A clear policy must be developed, outlining when each of the three levels applies, and how information should be managed within each level.
Processes must be put in place to 'bubble up' or 'promote' information from lower levels up to higher levels. For example, some team-generated information will be critical for the whole organization.
As much as possible, a seamless information management environment should be delivered that covers all three levels.
Part B: The Call centre how they using the information systems.
Today's successful companies are the ones who continuously keep their finger on the pulse of what their customers think, no where this is truer than in a contact centre.
In contact centres the quality of customer interaction has traditionally been measured using quantitative measures of call answering and subjective checks on the quality of the interaction. That is beginning to change with the appearance of applications specifically designed to give call centre managers good quality objective information about how customers feel their interaction was handled.
The Traditional Approach
Most call centres are familiar with operating to service levels. Metrics such as time to answer, longest queue time, longest wait to abandon are the familiar measures by which call centre and customer contact staff manage their effectiveness. This type of information is easy to produce and monitor using established contact centre MIS systems. These give call centre managers the information they need to manage their staffing and call handling plans.
But what happens when the call is answered? How do managers know if the caller has had a quality interaction with a contact centre?
Most contact centres already try to monitor the quality of the interaction. Traditional techniques include
• Test Calling: Calls are made to the call centre by staff pretending to be customers.
• Silent Monitoring of calls: A manager or supervisor listens in on certain calls.
• Call Recording: Recording the entire call.
• Making Follow-up outbound calls: Calls are made after the initial interaction and the customer is questioned about their experience of the original interaction.
Resource intensive
Test calling, silent monitoring and call recording all help check that calls are being handled well, but these techniques are often time consuming and inconsistent.
Test calling and silent monitoring require someone to make or monitor the call. They require someone to take some real time action: this is costly, and often inconsistent. The staff time required to make this kind of assessment is put under pressure when the call centre is busy. This creates a tension between monitoring the quality of interaction and answering the calls. Consequently when quality monitoring is most needed, during busy times, it is least likely to be undertaken.
Poor Management Information
The above quality control mechanisms are hardly ever used in conjunction with any quantitative analysis, so it is hard to build up an objective picture of the quality of the service. And even if good call monitoring practices are coupled with quantitative analysis you still do not know whether or not your customers are benefiting from these call monitoring practices unless the impact of any changes made can be objectively assessed.
Though listening to recordings can be an "off-line" process, sampling of call recordings is rarely done. Recording is most often only used as an audit trail to provide "evidence" in the event of a complaint or dispute. Customer satisfaction monitoring is therefore often perceived by agents as a negative, punitive process, rather than as a positive, proactive measure.
Subjective assessment
The fundamental flaw in traditional systems however is that however call monitoring is performed, having a third party assessing the interaction misses the essential component; namely the actual opinion of the customer. It is what the customer thinks about how they were dealt with that matters, and a third party interpretation of the interaction may be biased. The only way of having a real insight into customer perceptions is to find out directly. Making outbound calls gets the customer's opinion but is costly and time consuming, as well as often just too late. By the time a customer is contacted their opinion of an interaction may be coloured by a follow up or lack-of a follow up event such as a letter or email.
Non-voice media
An important point is that contact centres are increasingly handling interactions from media apart from telephone voice calls, e.g. through email and text. These interactions need to be monitored in the same way and preferably using the same criteria.
Part C: The Internet Technology giving opportunities.
Finally, traditional methods of customer satisfaction monitoring in call centres have played a large part in contributing to the poor press of the sector in terms of work conditions for employees. These systems are open to abuse, because judgements of performance rely on supervisors' subjective judgement of agents' call handling, and because they are seen as punitive mechanisms. However well it is done however, agents can experience such surveillance as disempowering and overly controlling. Abandoning these traditional methods may have significant pay-offs for call centres in terms of PR, recruitment and retention of staff. The high turnover of staff is a major draw on resources in call centres and improving work conditions for employees through less intrusive quality control is likely to pay for itself many-fold.
Getting Better Information
"If you don't measure it, you cannot manage it".
This seems a statement of the obvious but how many companies have continuous and meaningful measurement of customer satisfaction within the contact centre.
A customer's satisfaction with a contact centre interaction needs to be obtained in a way that is:
• Objective - The actual opinions of your customers are what you require, not a third party's interpretation of them.
• Immediate - The customer's opinions should be obtained immediately after the interaction, not as part of a completely unrelated interaction by mail, email or phone.
• Continuous - The monitoring of satisfaction needs to proceed regardless of the prevailing levels of interactions. Busy call centre's have even more reason to monitor their quality: being too busy should not be an excuse.
• Significant - Enough transactions need to be monitored to enable the production of statistically significant and representative data.
• Measurable - The data needs to be analyzed in a way that gives quick and easy insights into changes in customer satisfaction. Ideally cross-analysis with "hard" data about the original call such as agent-id, queue-time etc should be done. If you don't measure it, you cannot manage it.
• Actionable - Information needs to be made available quickly. If results are falling outside acceptable limits alerts need to be raised so management can take action.
• Visible - You need a process that is visible to your customers, demonstrating to them that you are proactively asking for their feedback with the objective of improving the service you provide.
New Solutions
Solutions that provide answers to these problems have started to appear. One such is a system called Opinion-8 from Square Systems. Opinion-8 offers, in the form of a managed service, the ability to obtain real-time customer feedback on satisfaction, perceptions and preferences.
It is automated, objective, requires little or no ongoing staff input, gives good quantitative as well as qualitative management information, and eliminates the need for subjective surveillance methods. This may improve employee work conditions and hence staff retention rates, as well as improving the quality of service for the customer at a reduced cost to the organisation.
How Opinion-8 Works in a Contact Centre
Opinion-8 surveys are very simple to implement. Customers are asked if they would like to give feedback on the quality of the service they received. If they say yes, their call is transferred to the Opinion-8 voice server, along with relevant information about the call (agent ID, caller telephone number, etc). The caller is then presented with a set of automated survey questions and asked to respond using the telephone keypad. Callers can be invited to leave spoken feedback. This can be triggered by particularly high or low ratings for questions. Data and any recordings of comments are saved and compiled on an ongoing basis. This information is immediately available for review. A range of analysis tools can be used to create graphs or tabular reports for different sets of data, over any specified time period. An optional SMS alerting system alerts staff to responses outside of pre-defined thresholds, letting managers take action wherever they are.
Survey scripts (ie. the questions asked of customers) can be created on-line over the internet. The whole process from set-up to implementation of survey can be done in less than an hour. This means that customer satisfaction can be measured almost immediately, on any change in service. Once created the same survey can be deployed using both telephone and web / email.
The Opinion-8 service runs 24 hours a day, seven days a week. Because it automates the customer satisfaction survey, it takes up a minimal amount of contact centre management and agent time. This frees up time for contact centre agents to handle more customer calls. Another advantage of an automated survey is that customers do not feel pressured to state anything but the truth about their experience. Feedback is also more likely to be accurate as their opinions are captured straight after their contact centre interaction.
The captured data is available immediately after the survey has ended, providing an ongoing, real time picture of customers' experiences of the quality of the interaction with the organisation.
All this can is done without installing any software, and without any large capital costs. Analysis and monitoring can be done on any PC with a browser and an Internet connection. You just need to enter a user name and password. Opinion-8 is provided as a service and is paid for on a per use basis. This puts this type of quality management tool within reach of all organisations regardless size and budget.
Summary
Reporting on standard contact centre metrics is established practice in call centres but this does not monitor what the customer actually thinks of the service. However there are significant problems with these methods. Traditional quality monitoring tools do not provide objective, continuous and measurable data, and surveillance methods are linked to the high staff turnover in the call centre industry. However, tools are beginning to appear that give organisations real insight into customer satisfaction with interactions without the need for costly surveillance. These tools are easy and quick to use and cheap to deploy, and have a number of wider benefits to the organisation.
For more information about using IVR automated telephone surveys for market research, visit www.opinion-8.co.uk or contact the author.
{Ref: Author of Paul Murphy(Managing Director of Square Systems Ltd )}
