Too often, performance measurement is placed at the end of any
process. It should, of course, be positioned close to the
beginning. As one business participant at ITC's 1999 Executive
Forum (Anthony Guiseppi, General Manager, Trinidad and Tobago
Manufacturers Association, Port of Spain) remarked: "If you can't
measure it, you are not doing it." Some form of assessment
mechanism, therefore, needs to be built into the export strategy
process at the start.
When performance measurement does take place, it is generally
the result of external pressure from a Ministry of Finance and
Planning or from an international funding agency in response to a
request for funds to implement a national export strategy. A major
reason should be to increase effectiveness and efficiency. To quote
Raymond Yip of Hong Kong's Trade Development Council (TDC): "To
stay relevant and vibrant as trade promotion organizations, we must
continue to review, renovate and reinvent ourselves. And to do this
we need to establish benchmarks to measure how far we are achieving
the objectives we have agreed."
More competitive world
In an increasingly competitive world, neither client nor service
provider can rest on their laurels if they wish to remain in
business. Performance measurement is a powerful learning tool: it
helps both parties to understand better what works, what doesn't
and why. As a result, they can change strategy or their way of
doing something, in order to increase impact and possibly to reduce
costs. Cambodia's Secretary of State for Commerce, Sok Siphana, has
even suggested that for developing countries: "Maybe learning
should be more important than measurement."
The questions then arise for strategy-makers:
- What should be measured?
- How to measure performance?
- What are the 'best practices' and tools?
What to measure and how?
There are three recognized levels of performance measurement of
increasing difficulty:
- The first focuses on inputs
(manage-ment, strategies and resources such as staff and funds)
and throughputs (timeliness, reach and
the same resources). All of these are entirely within the
capabilities of the trade support institution (TSI) to
measure.
- The second focuses on outputs (i.e.
services rendered, information provided, docu-mentation issued).
These are often measured fairly easily through customer
satisfaction surveys.
- The third measurement focuses on
outcomes, or impact. But this is very difficult to
measure. It concerns effects such as the increase in clients'
wealth, on trade flows and on national income. Nevertheless, the
ultimate aim should be to measure impact, because that is what
really counts.
Strategists must also recognize that 'one size doesn't fit all'.
Best practice suggests that one should be monitoring and reviewing
all stages of the process on a regular basis. As Mr. Yip observes,
strategy-makers and TSI executives must get used to:
"Asking the 'Tough Questions':
- Should we still be doing this?
- Is this the best way to do this?
- Could these resources be put to better use?"
These questions need to be posed at every stage.
Reviewing strategy
Many countries seem to reassess their national export strategy
only as part of a review of the national trade promotion
organization (TPO). In recent years, Sweden and Switzerland have
carried out major reviews of their TPOs, which implicitly involved
a review of their national trade strategy. In both of these
countries, the TPO structure remained essentially the same. In
other countries, however, the review has led to a merger of trade
and investment promotion agencies, as is currently under way in
Malta, or a separation of agencies, as occurred in Mauritius and
was recently being considered in Portugal. New Zealand has
restructured its trade promotion network several times as its trade
strategy has developed since 1980.
Reviewing programmes
The evaluation of individual programmes in a national strategy
is relatively straightforward and frequently undertaken.
Participants in conferences, training and promotional events -
trade fairs, missions, etc. - routinely receive an evaluation form
at the end.
Administering a questionnaire immediately after a trade fair
gives an indication of the output - number of visitors, the type of
visitors, the number and value of orders taken, etc. However,
unless there is another follow-up six months or a year later, TSIs
still do not know what the real impact of that event has been, i.e.
the actual export improvement. Ecuador's trade and investment
promotion organization, CORPEI, makes an effort to measure the
service given to people who seek information. "Sometimes we try to
make a cost-benefit analysis of participation in trade fairs,"
CORPEI's Executive President, Ricardo Estrada, reports.
Many organizations engaged in training are now using
participants' evaluations of those events to change their
employees' behaviour. In some cases, pay increases and bonuses are
being linked directly to performance measurement, based on client
feedback from the delivery of specific services (e.g. Canada's
foreign trade representation service).
Common tools
A few of the most commonly used performance measurement tools
are:
Statistical indicators
Among the statistics which countries use as a guide to quantify
export strategy performance are changes in:
- import and export figures;
- foreign exchange earnings;
- number of exporters;
- level of employment;
- number and geographical spread of export markets;
- increase in market share;
- number and range of export products and services; and
- level of payment for trade support services.
The difficulty is to relate these changes to specific elements
of the strategy, or indeed the strategy at all, since other factors
are frequently influential. These might include economic policy (at
home or abroad), lack of government or private sector commitment,
foreign direct investment, or even natural disasters, terrorism or
war.
Consumer satisfaction surveys or other client
feedback
Possibly the most widely used measure of impact assessment is
client feedback. It is arguably the most accurate tool for judging
specific programmes. However, it needs to be carefully designed,
independently applied and provide for a sufficient lapse of time
for the activity to have achieved a measurable impact. Mr. Yip
reports of the Hong Kong TDC: "We measure some things three months
down the line, and for some projects one year on. We also benchmark
ourselves against other private bodies and against other similar
organizations."
New tools
Balanced scorecard
The balanced scorecard is one of the newer tools to be tried in
performance measurement of trade strategies and TSIs. The
methodology examines performance in four areas:
- financial analysis looks at issues
such as operating costs, return-on-investment;
- customer analysis looks at customer
satisfaction and retention;
- internal analysis looks at production
and innovation, measuring performance in terms of maximizing profit
from current products and following indicators for future
productivity; and finally,
- learning and growth analysis explores
the effectiveness of management in terms of measures of employee
satisfaction and retention and information system
performance.
The Portuguese trade promotion organization, ICEP, introduced
the system in 2001 in a phased approach. Finpro, Finland's trade
development institution, has used it for some years, focusing on
client relationship, quality of service, staff competence,
efficiency and productivity, as its four areas. Seppo Laine, Senior
Vice President of Finpro, notes that it can be quite difficult to
judge "national impact" or "competence development". Over the
years, Finpro has built on its experience with one numeric approach
to develop a better system, but admits that a final version is
still a long way off.
Contingent valuation
Developed by the Helsinki School of Economics and Business
Administration in 2001, the contingent valuation method looks at
the value that the beneficiary of a trade support service would be
willing to pay for that service in retrospect (i.e. after a
reasonable interval of time needed to assess impact).
It then analyses the difference between the actual cost of the
service and the ex-post facto value the client attributes to the
service. If positive, it can be used to justify increased funding.
Alternatively, if marginal or negative, it clearly indicates the
need for some remedial action, or a dropping of the activity
altogether.
In Finpro's pilot study, assessment is carried out through
one-hour interviews with representative samples from its 3,000
company clients. "Don't confuse the questions we are asking when we
seek to assess impact with the questions we ask about client
satisfaction," Mr. Laine stresses. He also notes that once research
is completed, it still needs to be translated into a political
language whose policy implications can be understood.
Benchmarking
Benchmarking is a process used to assist with performance
measurement. It is used extensively to measure corporate sector
performance. For several years now, it has been used as a tool for
measuring countries' competitiveness in the World Economic Forum's
Global Competitiveness Report.
Benchmarking is less widely used for trade strategy performance
measurement, but there is an increasing interest at the trade
promotion organization level in looking at the performance of other
TPOs in selected areas. Canada's Department of Foreign Affairs and
International Trade carried out an extensive study of the pricing
policy of selected TPOs in 1998. More recently OSEC, Switzerland's
trade support organization, has been benchmarking other European
TPOs.
What should TSIs do?
TSIs need to engage in performance measurement, not only to
satisfy funding sources, but also to satisfy increasing demands of
clients and to help improve the effectiveness, efficiency and
relevance of services offered. There are a number of tools
available - trade statistics, client surveys, balanced scorecards,
etc. But each TSI must choose the methods that best suit its
current situation and that will satisfy its stakeholders -
financial backers, clients, staff and the general public.
Performance measurement and other forms of assessment are not
popular. They are regarded as an additional chore, with sometimes
unexpected and unpleasant results. There is, therefore, a sizable
job of advocacy and education to be done by development agencies
like ITC and the other institutional members of the global trade
support network, particularly in developing and transition economy
countries, as well as by strategy-makers.
But this is only half the job. As Stephen Sultana of Malta's
export promotion organization notes: "Change is not fundamentally
about structure, it is about people." There is no one way to manage
change - "All change processes are imperfect," Mr. Sultana said -
and it can take two years for stability to re-establish itself
after reorganization. "Transparency is crucial. People in the
organization have to know what you are doing. You have to make
changes incrementally, through a consultative process."
Finally, though effective performance measurement starts with
clear objectives, the trading environment is ever-different and
creating constantly changing needs for information. Tools must
therefore also change over time.
Views on measuring performance
"If you don't measure it, you had better be sure that it is
done. We have found that if you don't measure it, it won't get done
or not entirely."
Raymond Yip, Regional Manager (Western Europe), Hong Kong
Trade Development Council, at Executive Forum 2002
"National strategy-makers have to analyse and benchmark national
comparative advantage in terms of country resources and level of
development. But the main comparisons must be at the sector,
business and company level."
Nurgul Erdogan of the Export Promotion Centre, Turkey, in
the Executive Forum 2002 e-discussion
"The days of trade promotion are over. Today it's about
supporting companies to make them more successful in exporting. We
are moving from promoters to advisors, from services to solutions,
from clients as customers to clients as partners. […]
If you are seeking more money from the government, it's not
enough to tell the minister that you have 80% customer satisfaction
(METCO's rating). We were able to get almost a doubling of our
budget a few years ago by showing the minister that in two markets
we had been able to increase exports by a significant percentage by
promotion over the previous two years."
Stephen Sultana, Malta External Trade Corporation (METCO),
at Executive Forum 2002
"Performance measurement, and its corollary - results-based
management - are, in my view, no longer an option for national
export strategists. […] Another very important indicator is
satisfaction of employees. Employee satisfaction has been
documented to have a strong correlation with customer
satisfaction."
Norman Baillie-David, Managing Partner, The Antima Group,
Canada, in the Executive Forum 2002 e-discussion
"What should managers do to take advantage of benchmarking?
First, they should understand the company that is being benchmarked
- its environment, management, governmental policies, current laws,
etc. Only then can a benchmark model have the potential to take the
company further. Managers should benchmark other companies with a
degree of flexibility.
Performance measures are crucial, not to determine whether
strategy is effective, but as a key ingredient for continuous
improvement.
Macroeconomic statistics are useful but only part of the
equation. […] There has to be a compromise between considering
customer feedback and what the manager wants to do. Managers cannot
be 'just floating ships' navigating by the customer's sails."
Arturo Corrales, Economist from CC&R Pacific Business,
Peru, in the Executive Forum 2002 e-discussion.
For more information, contact: Philip Williams, ITC Senior
Adviser on Institutional Aspects of Trade Promotion, at williams@intracen.org
A detailed study of performance measurement and a report on
Finpro's experience can be found at the Executive Forum web site
(http://www.intracen.org/execforum).