Prepared by Information Victoria, Department of Innovation, Industry and Regional Development, July 2010
A customer focussed culture is a given in private sector organisations where knowing what the customer wants is driven by the need for greater market share. The equivalent driver does not often apply to the delivery of public sector services.
Even though customer insight is not of itself a new concept, embedding the practice of regular or on-going satisfaction measurement, especially as it relates to government service provision, is still either an ad hoc, add-on or non-existent process in many public sector environments. And, when it is measured, it is sometimes confounded by an emphasis on outputs rather than outcomes – outputs being the measurement of the process or product'deliverable' produced by a service, for example, the number of customer calls handled.
An outcome is what happens as a result of that service, for example how satisfied was the customer with the quality of the experience and did they get what they wanted?
Although notions like citizen-centric services have and are continuing to gain traction in the rhetoric of public sector innovation, to become truly citizen focussed, public sector agencies need to embed both the principal and regular practise of satisfaction measurement into all aspects of the culture.
The measurement of outputs and outcomes together, needs to occur to as many aspects of government service delivery as possible, so that the needs of citizens are front and centre in planning and priority setting, in policy deliberation and in the delivery of front-line services.
Knowing what citizens want or expect from a government service is an essential part of designing better services to meet those needs.
Customer satisfaction measurement is therefore a key focus and should not be relegated to an occasional one-off research task but incorporated in an ongoing journey for continuous service improvement – in effect, the road to satisfaction.
Measurement tools and techniques can only work if there is a prerequisite organisational cultural commitment in place. It has to be championed from the top, with chief executives openly committed to regular customer satisfaction measurement.
However, it would be both idealistic and simplistic to overlook the fact that measuring satisfaction with government services can be more complicated in the public sector. This is because it is inextricably tied to notions of public value which encompasses harder to measure concepts like social outcomes.
Customers for public goods are also citizens in a democracy in which desired social outcomes, as public value, are contestable – for example – public services like policing, teaching or other welfare services, where citizens collectively authorise defined social goals.
And, to add a further layer of complexity when thinking of the customer of public services, we need to remember that some interactions with government services are an obligation imposed by government, so the 'customer', has no choice in the transaction - for example in licensing or other regulatory requirements.
Even where there is no legal obligation, government is in many instances, a monopoly provider of services with the 'customer' unable to choose where they direct their custom.
Despite these conceptual challenges in determining if we're measuring the needs of a citizen/customer who may or may not have a real choice when interacting with government services, there is still a strong argument for the regular use of satisfaction measurements – not least of which is the expectation by citizens that public sector services be as good as if not better than any comparison with the private sector.
Apart from any expectation that all services should be as good as they can be, measuring the satisfaction of the consumer of government services is both important and valuable because it:
A key thinker in this space puts it as follows: 'Performance measurement is fundamental to the ability to manage organisations. If you do not have such a tool, then I would argue that you are merely presiding over the organisation's activities. You are not really managing it and you are not creating conditions under which the organisation's performance could be expected to improve over a period of time. Left to their own devices, organisations tend to accumulate slack and the only way to move that slack around and get it performing is to embrace a strong form of external accountability in order to animate and drive performance within the organisation.' (1)
In effect, a positive outcome from satisfaction measurement becomes the government equivalent of shareholder value.
Measuring outcomes supports the concept of public service value based on results that government agencies produce for the citizens that they serve.
Promoting a customer focused culture is the best way to maintain this focus and the Canadian government has lead the way in formalising public sector performance measurement.
In 1998, the Canadian government conducted the first in the series of Citizens First surveys aimed at discovering what citizens thought about the delivery of public services.
That initial Citizens First survey determined the level of citizens' satisfaction with public sector service delivery by analysing over three thousand recent service experiences across a range of programs and channels.
It found that there were five “drivers” of satisfaction which accounted for over seventy percent of the service satisfaction outcome in the public sector.
These drivers were:
Perhaps not surprisingly, timeliness and a positive outcome delivered by staff that is competent, knowledgeable and willing to make an extra effort to meet the needs of citizens, emerged as the standard findings of this research.

Figure 1/ICCS (2)
More specifically, the first survey discovered that:
This groundbreaking research about public sector services provided managers with practical guidance to meet expectations for service delivery through a service-standards approach.
Based on these initial findings, Canadian governments went on to introduce other measures of agency performance such as 'mystery shopper' methods.
As part of its vision for a citizen focussed public sector, the Canadian government subsequently established the Institute for Citizen-Centred Service (ICCS) to promote the principal and the practice of measuring citizen satisfaction with public-sector service delivery.
The Institute is funded by all 3 levels of government across Canada, and is governed by a public sector Board of Directors with a focus on a citizen-centred public sector aiming for higher levels of citizen satisfaction with government service delivery.
Dedicated to the purpose of improving government services and identifying service expectations of citizens, the ICCS assists the public sector with the tools to support service quality. Its research continues to maintain a focus on the actual needs of the citizen, what they term - the 'outside- in' view - as opposed to an in-house managerial assumption about those needs.

Figure 2/ICCS (4)
The ICCS is now an integral part of Canada's research based service improvement strategy and has evolved into a centre of excellence for electronic service delivery within a single-window service integration framework.
The ICCS is the custodian of the survey instrument, the Common Measurements Tool (CMT
Since its initial use in 1998, the biennial Citizens First survey means Canada is uniquely placed in being able to measure and demonstrate a consistent improvement in citizen satisfaction with public sector service delivery over a ten-year period
As Ralph Heintzman Adjunct Research Professor at the University of Ottawa, noted, “The Citizens First series…has established the gold-standard for research on public sector service delivery, not only in Canada, but around the world. The ICCS methodology and approach have equipped public sector managers with the tools they need to identify action priorities for service improvement in the public sector.” (5)
The most recent Citizens First survey in 2008 has again reinforced that the drivers of satisfaction with government services continue to be:
The initial Citizens First study in 1998 did not ask about internet use because government services were barely available online. Today, the use of the web is practically on a par with visits to government offices and use of the telephone.
Surprisingly, the recent survey showed that the web is not displacing traditional government contact channels. Citizens see the internet as a complement to, rather than as a replacement for, the more traditional channels.
The survey also revealed that the telephone continued to need improvement because of problems relating to access (busy lines, frustration over waiting while on hold) and difficulty finding the right number and trouble with automated phone systems.
The Canadian Citizens First surveys and subsequent research have lead to the design of the Common Measurements Tool (CMT).
The Common Measurements Tool (CMT) is a citizen/client feedback survey created by public servants for public servants. It's designed to provide accurate and reliable customer feedback on key dimensions of service delivery. The CMT is basically a question bank, which uses a standard five point scale of measurement with 1 being strongly dissatisfied and 5 being very satisfied.
The CMT Question Bank contains more than 150 questions as well as the set of core questions based on the main drivers of satisfaction identified by initial and subsequent Citizens First surveys. The Table of Core Questions as shown below identifies the appropriateness of each question to the channel being used.
Because it is based on empirical research on the drivers of citizen satisfaction with public services, the CMT ensures that agencies measure the things that matter to citizens. The core questions relate to those aspects of service which have been identified as key to customer satisfaction.
Drivers and outcome measures |
Question |
Service Delivery Channel |
||||
|---|---|---|---|---|---|---|
Tele- |
In- |
Web |
||||
SATISFACTION |
||||||
Timeliness |
Overall, how satisfied were you with the amount of time it took to get the service? |
√ |
√ |
√ |
√ |
√ |
Accessibility |
Overall, how satisfied were you with the accessibility of the service/product? |
√ |
√ |
√ |
√ |
√ |
Overall satisfaction |
How satisfied were you with the overall quality of service delivery? |
√ |
√ |
√ |
√ |
√ |
PERFORMANCE |
||||||
Fairness |
I was treated fairly |
√ |
√ |
√ |
√ |
√ |
Information |
I was informed of everything I had to do to get the service/product |
√ |
√ |
√ |
√ |
√ |
Extra mile |
Staff went the extra mile to make sure I got what I needed |
√ |
√ |
* |
|
√ |
Competence |
Staff were knowledgeable and competent |
√ |
√ |
* |
|
√ |
Accessibility |
I was able to get through to an agent without difficulty |
√ |
|
* |
|
√ |
Waiting time |
I waited a reasonable amount of time at the service location |
|
√ |
|
|
|
Navigation |
It was easy to find what I was looking for |
|
|
√ |
|
|
Appeal |
The site is visually appealing |
|
|
√ |
|
|
Information |
The site had the information I needed |
|
|
√ |
|
|
Privacy |
I feel confident that my privacy is fully protected on this site (OR: when I communicate by email with [agency].) |
|
|
√ |
|
√ |
OUTCOME |
||||||
Outcome |
In the end, did you get what you needed? |
√ |
√ |
√ |
√ |
√ |
As a survey tool, the CMT can be adapted for use for in-person, over the telephone, or for web-based services and agencies can customise by adding organisation-specific questions. Agencies are strongly advised to use the set core questions if they want to use the CMT to benchmark with others.
Using the CMT to collect citizen feedback assists public sector managers to improve practices and attain higher levels of satisfaction with services.
In addition to purchasing the CMT licence, a benchmarking service is available through the ICCS that allows organisations to anonymously compare results against peers.
The CMT is being used across government agencies in Canada and in several jurisdictions internationally and is gaining a reputation as an established multi-channel instrument for designing client satisfaction surveys of government services.
Other ICCS research continues to support a focus on internal service alignment so that citizen satisfaction with service delivery is maintained and resources allocated to target priority areas. The Service Improvement Planning and Implementation model below displays this process.

Figure 3/ICCS (6)
A continuous service improvement culture starts from having a baseline satisfaction measure for services to the public and this is where the CMT is of value. The methodology of the CMT survey design allows for different levels of measurement.
At its most sophisticated, the CMT offers a performance/importance matrix measurement of the relative importance attributed to any of the drivers of satisfaction. This double-barrelled approach helps to identify:

Figure 4/ICCS (7)
This more thorough approach helps to reveal service areas affecting the respondent's satisfaction, to reveal the service delivery areas requiring the highest level of attention as well as where to set priorities for service improvements
Even if agencies do not use the double-barrelled performance/ importance matrix to questions, there is still value in opting to apply the core questions of the CMT as a way of documenting a baseline satisfaction measure for services. This aids with organisational performance reporting.
More recent Canadian research on the original 5 drivers of citizen satisfaction (as shown in figure 1) continues to show that:
Timeliness - is the single most important driver across all services and all governments;
Positive Outcome - I got what I needed - is the desired end result; and
Staff Performance - which includes fair treatment, knowledgeable, competent and proactive staff prepared to 'go the extra mile' stands out as the primary drivers in government service delivery.
Comparative Canadian research over a seven year period (1998/2005) documents the rank order of importance as follows:
Drivers in 1998 (Citizens First By rank order of importance |
Drivers in 2005 (Citizens First 4) By rank order of importance |
|---|---|
| Timeliness | Timeliness |
| Fairness | Outcome |
| Knowledge and competence | Extra mile |
| Courtesy, comfort | Fairness |
| Outcome | Knowledge |
Figure 5/1CCS (8)
Not surprisingly, timeliness continues to rank at the top with a shift to outcomes rather than fairness as the next most important expectation over this period.
Use of the CMT is widespread in Canadian federal, provincial and local governments and internationally, jurisdictions include New Zealand, Singapore, Kenya, Namibia, the U.A.E and Australia.
The Institute for Citizen-Centred Service is currently conducting a review of the use of the Common Measurements Tool (CMT). The review is designed to examine the current version of the instrument in the context of the evolving service delivery environment and leading research methods.
Recommendations issuing from the review are expected to result in a new and updated edition of the instrument that will provide CMT users with a tool that preserves the main features of the current version while enhancing usage of the instrument and its overall value.
For more detailed information on the Canadian approach or the CMT, visit http://www.iccs-isac.org/en/ and for a recent summary of CMT activity see the attached document link (9).
Prior to any consideration of the specific use of the CMT by Australian government agencies, it is worth noting the findings of research on satisfaction ratings with State Governments more generally.
In 2010, Quantum's social research survey - AustraliaSCAN – asked a question aimed at measuring satisfaction ratings against a list of 15 variables, across the three tiers of government. The chart below shows the net total satisfied/total dissatisfied results for those who reside in each of the states measured. (10)

Although none of the State governments scored particularly highly, the Victorian government appears to be doing reasonably well in a general satisfaction measurement.
However, as would be expected, the specific methods at any given time to the measurement of customer satisfaction with specific government programs, varies enormously across the array of government agencies.
Similarly, the setting of targets and the methodology used to measure these is very varied. There is no one common approach in use.
In Australia, government agencies in Victoria, South Australia and the ACT are actively applying versions of the CMT and Queensland and Tasmania are also in the process of either incorporating or assessing its potential use in existing customer satisfaction surveys.
In 2007 the South Australian Government mandated a new target to increase public satisfaction with government services by 10% by 2010 and maintaining or exceeding that level of satisfaction thereafter.
The target reflected the need for a skilled and responsive public sector to actively understand and address community needs.
With a view to ensure that individual departments used a consistent measurement instrument that would assist with any subsequent international benchmarking, the South Australian Government purchased a licence to use the Canadian Common Measurements Tool (CMT) to measure customer satisfaction.
Agencies were asked to assess customer satisfaction against the eight core questions/statements and encouraged to also use any other additional and relevant CMT questions.
All SA government departments were expected to report against this target and the majority of agencies implemented the use of the eight core CMT questions.
Because the approach for the implementation of its use was not also mandated, variation across the agencies in the methodology used precluded comparative results.
For example some agencies chose to use an external consultancy to apply the CMT who opted to use a 7 point scale instead of the recommended 5 point scale. Using a different scale creates problems when attempting to compare or benchmark.
In addition, the SA Cabinet Office conducts a Household Survey via the telephone to randomly selected householders which incorporates the eight core questions from the CMT. The result of this survey is used to determine progress against the target of 10% improvement in customer satisfaction.
The most recent survey revealed that while there was no significant change between 2008 and 2009 in the proportion of respondents who rate their satisfaction as 3 or above in the CMT 5 point scale, there was an increase in the proportion of rating 4 and a corresponding decrease in the proportion of rating 3 responses.
This contributed to an increase in the calculated mean score of overall satisfaction ratings from 3.31 in 2008 to 3.37 in 2009 which is trending towards greater satisfaction with government services.
The CMT is being used within the Service SA unit located within the South Australian Department of Transport, Energy and Infrastructure.
In order to determine a baseline overall satisfaction with Service SA, a statistically valid externally managed assessment using over 1300 respondents via the telephone and hundreds of exit interviews at the service centres was undertaken recently which established a 92% overall satisfaction.
Across their three service delivery channels of Service SA, the application of CMT questions is as follows:
The results of these surveys are used as indicators for areas of improvement.
Service SA uses the 8 core CMT questions and does not use the 'importance' vs. 'satisfaction' matrix but rather asks for a rating on a scale from strongly agree to strongly disagree.
Service SA uses these results to report quarterly with survey results analysed in-house.
The following questions and scores from the Contact Centre represent the last quarter's results:
Although not deemed to be statistically valid due to low sample numbers, Service SA reports that their overall satisfaction rating of 91 % for the year to date is used as a base from which to continue to fine tune their business model.
Service SA intends to go to market for an externally run and statistically valid survey in the new financial year which will probably incorporate the CMT questions.
Also in SA, the Department for Families and Communities covering operational divisions including Housing SA, Disability SA, Families SA, and Concessions Services, has been using the CMT for service improvement for a number of years.
The 8 Core Questions of the CMT in the Department for Families and Communities were first used during February and October 2008. These results were used to establish the baseline measurement for customer satisfaction.
The methodology involved face to face interviews with customers immediately following an interaction at a customer service centre, or via the phone.In some cases, the research was conducted via survey sent through a mail out to known clients.
Responses were recorded on a hard copy (for face to face interviews and mail out format). Staff subsequently entered raw data onto web-based survey software.
For phone surveys, staff entered data directly into a survey tool whilst conducting the survey with a script linked to the questions to ensure consistency and to minimise any bias.
As with any change process, DFC reported some initial organisational resistance to the introduction of the CMT because of the characteristics of a direct service organisation serving the needs of socially vulnerable cohort of clients.
In addition, other reported barriers related to using the double-barrelled Importance-to-Satisfaction ratio weighting the questions.
The Department for Families and Communities is currently at round 5 in its application of the CMT.
Each organisational division is intending to use their individual results for internal purposes as a continuous improvement tool.
Canberra Connect is a customer service delivery arm of the ACT Government providing information, payments, services and emergency support to the ACT community on behalf of ACT Government agencies.
Canberra Connect is known through its service operations of five Canberra Connect Shopfronts, a Canberra Connect Contact Centre (13 22 81), and Canberra Connect online at:
http://www.canberraconnect.act.gov.au/CAP/accesspoint/cc?action=menuHome
Canberra Connect undertakes different benchmarking activities each year and has set a target of 90% for their customer satisfaction reporting.
One of these is a Customer Service Benchmarking Survey conducted by an external market research company. The survey samples 600 Canberra citizens and asks approximately 40 questions to assess brand awareness, customer satisfaction with service and outcomes, and seeks feedback on specific initiatives. This information is internally analysed to improve the scope and quality of service delivery.
In addition, in 2010, Canberra Connect also participated in a cross jurisdictional benchmarking activity conducted by Customer Service Benchmarking Australia on behalf of the jurisdictions as a collective.
This activity benchmarked the client experience by utilising a "Mystery Shopper" methodology and used similar questions across jurisdictions.
It is intended that the results of this recent research will provide further insight into the public sector customer experience in Australia. (11)
These benchmarking activities are intended to contribute to Canberra Connect governance and form the basis of annual reporting and key performance management.
In 2010, Canberra Connect included the CMT questions in the Customer Service Benchmarking Survey. In previous years, Canberra Connect had included some questions very similar to the CMT but 2010 was the first year that the exact wording was used to ensure cross-jurisdictional compatibility.
It is anticipated that as more Australians agencies start to use the CMT, comparisons between the jurisdictions on key service delivery areas will highlight specific elements of customer service delivery and promote a sharing of knowledge to improve service delivery.
Service Tasmania measures customer satisfaction though the use of biennial surveys with a target of 98% customer satisfaction. Past survey results have been as follows:
The final report for the 2008 Service Tasmania Customer Satisfaction Survey, indicated a continued high level of customer satisfaction with Service Tasmania overall with 96 per cent of respondents indicating that they were either “very satisfied” or “quite satisfied”. The next survey is due to be completed later in 2010 and it will be the first time that Service Tasmania will have incorporated CMT questions.
Service Tasmania is intending to develop a better understanding of their customers and their expectations for accessing government services and will introduce a Community Engagement Program.
Information Victoria, a business unit of the Victorian state government Department of Innovation Industry and Regional Development introduced the CMT in mid 2009.
Information Victoria is a whole of government information delivery agency providing State, Local and Federal Government information about services via three channels:
Prior to the introduction of the CMT, measurement of customer satisfaction in Information Victoria for Annual Budget reporting purposeshas been based on the output performance targets as shown in the table below.
Performance Measure |
Unit of Measure | Q3 Target | Q3 Actual | % +/- Variance |
|---|---|---|---|---|
Average monthly number of visits to Victoria Online |
Number |
150,000 |
346,623 |
131.08 |
Information Victoria public contact per contact officer per day |
Number |
41 |
43.69 |
2.69 |
Customer Satisfaction with information services from Information Victoria |
Percent |
90 |
86.06 |
-4.38 |
Timely Delivery of Public Information |
Percent |
95 |
95.71 |
0.71 |
The figures above are for the 2009/10 cycle and did not include the results from the CMT. They show a target of 90% customer satisfaction with an actual attainment of 86.06 % and a timeliness of delivery that exceeded target.
It's expected that use of the CMT across business channels will help to drive a better alignment of existing, internal Information Victoria feedback processes and enhance understanding of customer drivers of satisfaction across the channels.
Ultimately, the process is intended to inform service improvement in order to strengthen Information Victoria's business strategy and to better meet the needs ofstakeholders who have been identified as follows:
Each service channel in Information Victoria is using a channel-appropriate set of the core CMT questions while retaining the right to add additional questions.
Use of the CMT by Information Victoria has been an iterative and staged process whereby initially the survey relied on a paper version of the questionnaire (for the in-person and telephone channels) with a subsequent web – enabled version of each survey being created.
Unlike other agencies that have outsourced application of the surveying process to research companies, or that conduct the survey at specific periodic intervals, Information Victoria has opted to apply the CMT in-house by customer support staff on a continuous and random basis.
The telephone and in-person channels proactively request customers to undertake the survey at the completion of a service or transaction.
Active consideration is also being given to the idea of either automating the CMT survey within the call-centre channel to overcome sensitivities associated with Customer Service Operators being required to survey their own service levels or transferring the CMT component to staff of a co-located call centre.
In addition, the web-enabled version of the in-person survey used by the Information Victoria Bookshop has been uploaded on to a stand alone, touch-screen installed in the shopfront to better entice customers to respond privately to the service provided by counter staff.
The web channel (Victoria Online) offers an option for visitors to the site to voluntarily complete the survey located on the home page of the website.
Arguably, this more 'passive' aspect of surveying customers, characteristic of web sites generally, has contributed to a lower satisfaction rating with the web channel.
This has prompted consideration of ways of overcoming this built in disadvantage by directly surveying known user cohorts of the web site or conducting one-off focus group research to establish a comparative benchmark.
The experience in using the CMT across the channels in Information Victoria has facilitated a comparative analysis. In particular, three questions common to all the channels are now readily compared graphically. The three questions are:
It should be noted that question 3 is not part of the CMT but has been added as an additional question to all the surveys.
An internal analysis of the surveys on a monthly basis (as shown below for March 2010) is providing Information Victoria with the type of comparative data that may subsequently be used to populate a visual dashboard of metrics relating to the overall business of Information Victoria.



The staged introduction of the 8 core questions of the CMT to the business of Information Victoria has been a learning process.
Information Victoria is mindful of the fact that the key metric for government agencies is not profit but services and that the measurement of satisfaction with the quality of those services needs to embedded in the culture as a continuous commitment.
Continuing to trial the application of the CMT across the three service channels of Information Victoria will maintain a systematic focus on customer satisfaction.
It is anticipated that this type of systematic focus will continue to provide service insights, enhance the skills of staff and provide additional valuable data for on-going service improvement initiatives.
Increasingly, citizens expect the same quality of service from government as they experience in their dealings with the private sector.
As this brief overview indicates, the measurement of customer satisfaction with government services is potentially both complex and demanding.
However, although there is no one perfect model or ideal methodology for measuring satisfaction with government services, the starting premise is based on the assumption that government agencies want to close any gaps between citizen expectations and their actual experiences of public services.
To do that, government agencies must be prepared to understand what these expectations might be. And this is where the Canadian government's research and experiences over a ten year period has contributed to our knowledge of the five key drivers of citizen satisfaction with public sector programs.
And, because these citizen drivers were based on extensive empirical research relating to public services across a wide range of Canadian government agencies, they remain both relevant and valuable as a starting focus for governments everywhere.
Irrespective of the service channel chosen, citizens expect that government will seek to understand their needs and that they continue to do so as part of the on-going journey on the road to customer satisfaction.
Information Victoria, Department of Innovation, Industry and Regional Development.
Email: administration@egov.vic.gov.au
1. Prof. Mark. Moore, Recognising Public Value: The Challenge of Measuring Performance in Government, Nov. 2006
2. http://www.iccs-isac.org/en/about/building.htm
3. ICCS research – from Research to Results: A Decade of Results-Based Service Improvement in Canada by B. Marsden and R.Heintzman, March 2009
4. http://www.iccs-isac.org/en/about/building.htm
5. From Press Release for Citizens First Number 5, September 5, 2008
6. http://www.iccs-isac.org/en/about/building.htm
7. http://www.iccs-isac.org/en/about/building.htm
8. http://www.iccs-isac.org/en/about/building.htm
9. http://www.iccs-isac.org/en/cmt/docs/CMT_Case_Studies.pdf (This document requires the use of Adobe Acrobat Reader). .
10. The exact question is: “For each item listed below please tell me whether you are very satisfied, somewhat satisfied, somewhat dissatisfied or very dissatisfied? (Please circle one number for each item)”. The items for government are “Your national government”, “Your state government” and “Your local government”, from Quantum AustraliaScan, 2010 research.
11. To be shared with other Australian jurisdictions at the annual 'Serve You Right' conference in August 2010.
Last updated: 29 September 2010Enter your email address to subscribe or unsubscribe from the eGov What's New mailing list.
Please enter email address of the person you wish to send this page to.