Measuring Performance

For a country, maintaining competitive advantage means managing the system which supports export development - and measuring performance. The tools are available. Nevertheless, they are rarely used, particularly by developing countries. Why is this so?

Too often, performance measurement is placed at the end of any process. It should, of course, be positioned close to the beginning. As one business participant at ITC's 1999 Executive Forum (Anthony Guiseppi, General Manager, Trinidad and Tobago Manufacturers Association, Port of Spain) remarked: "If you can't measure it, you are not doing it." Some form of assessment mechanism, therefore, needs to be built into the export strategy process at the start.

When performance measurement does take place, it is generally the result of external pressure from a Ministry of Finance and Planning or from an international funding agency in response to a request for funds to implement a national export strategy. A major reason should be to increase effectiveness and efficiency. To quote Raymond Yip of Hong Kong's Trade Development Council (TDC): "To stay relevant and vibrant as trade promotion organizations, we must continue to review, renovate and reinvent ourselves. And to do this we need to establish benchmarks to measure how far we are achieving the objectives we have agreed."

More competitive world

In an increasingly competitive world, neither client nor service provider can rest on their laurels if they wish to remain in business. Performance measurement is a powerful learning tool: it helps both parties to understand better what works, what doesn't and why. As a result, they can change strategy or their way of doing something, in order to increase impact and possibly to reduce costs. Cambodia's Secretary of State for Commerce, Sok Siphana, has even suggested that for developing countries: "Maybe learning should be more important than measurement."

The questions then arise for strategy-makers:

• What should be measured?
  
  
• How to measure performance?
  
  
• What are the 'best practices' and tools?
  
  

What to measure and how?

There are three recognized levels of performance measurement of increasing difficulty:

• The first focuses on inputs (manage-ment, strategies and resources such as staff and funds) and throughputs (timeliness, reach and the same resources). All of these are entirely within the capabilities of the trade support institution (TSI) to measure.
  
  
• The second focuses on outputs (i.e. services rendered, information provided, docu-mentation issued). These are often measured fairly easily through customer satisfaction surveys.
  
  
• The third measurement focuses on outcomes, or impact. But this is very difficult to measure. It concerns effects such as the increase in clients' wealth, on trade flows and on national income. Nevertheless, the ultimate aim should be to measure impact, because that is what really counts.
  
  

Strategists must also recognize that 'one size doesn't fit all'. Best practice suggests that one should be monitoring and reviewing all stages of the process on a regular basis. As Mr. Yip observes, strategy-makers and TSI executives must get used to:

"Asking the 'Tough Questions':

• Should we still be doing this?
  
  
• Is this the best way to do this?
  
  
• Could these resources be put to better use?"
  
  

These questions need to be posed at every stage.

Reviewing strategy

Many countries seem to reassess their national export strategy only as part of a review of the national trade promotion organization (TPO). In recent years, Sweden and Switzerland have carried out major reviews of their TPOs, which implicitly involved a review of their national trade strategy. In both of these countries, the TPO structure remained essentially the same. In other countries, however, the review has led to a merger of trade and investment promotion agencies, as is currently under way in Malta, or a separation of agencies, as occurred in Mauritius and was recently being considered in Portugal. New Zealand has restructured its trade promotion network several times as its trade strategy has developed since 1980.

Reviewing programmes

The evaluation of individual programmes in a national strategy is relatively straightforward and frequently undertaken. Participants in conferences, training and promotional events - trade fairs, missions, etc. - routinely receive an evaluation form at the end.

Administering a questionnaire immediately after a trade fair gives an indication of the output - number of visitors, the type of visitors, the number and value of orders taken, etc. However, unless there is another follow-up six months or a year later, TSIs still do not know what the real impact of that event has been, i.e. the actual export improvement. Ecuador's trade and investment promotion organization, CORPEI, makes an effort to measure the service given to people who seek information. "Sometimes we try to make a cost-benefit analysis of participation in trade fairs," CORPEI's Executive President, Ricardo Estrada, reports.

Many organizations engaged in training are now using participants' evaluations of those events to change their employees' behaviour. In some cases, pay increases and bonuses are being linked directly to performance measurement, based on client feedback from the delivery of specific services (e.g. Canada's foreign trade representation service).

Common tools

A few of the most commonly used performance measurement tools are:

Statistical indicators

Among the statistics which countries use as a guide to quantify export strategy performance are changes in:

• import and export figures;
  
  
• foreign exchange earnings;
  
  
• number of exporters;
  
  
• level of employment;
  
  
• number and geographical spread of export markets;
  
  
• increase in market share;
  
  
• number and range of export products and services; and
  
  
• level of payment for trade support services.
  
  

The difficulty is to relate these changes to specific elements of the strategy, or indeed the strategy at all, since other factors are frequently influential. These might include economic policy (at home or abroad), lack of government or private sector commitment, foreign direct investment, or even natural disasters, terrorism or war.

Consumer satisfaction surveys or other client feedback

Possibly the most widely used measure of impact assessment is client feedback. It is arguably the most accurate tool for judging specific programmes. However, it needs to be carefully designed, independently applied and provide for a sufficient lapse of time for the activity to have achieved a measurable impact. Mr. Yip reports of the Hong Kong TDC: "We measure some things three months down the line, and for some projects one year on. We also benchmark ourselves against other private bodies and against other similar organizations."

New tools

Balanced scorecard

The balanced scorecard is one of the newer tools to be tried in performance measurement of trade strategies and TSIs. The methodology examines performance in four areas:

• financial analysis looks at issues such as operating costs, return-on-investment;
  
  
• customer analysis looks at customer satisfaction and retention;
  
  
• internal analysis looks at production and innovation, measuring performance in terms of maximizing profit from current products and following indicators for future productivity; and finally,
  
  
• learning and growth analysis explores the effectiveness of management in terms of measures of employee satisfaction and retention and information system performance.
  
  

The Portuguese trade promotion organization, ICEP, introduced the system in 2001 in a phased approach. Finpro, Finland's trade development institution, has used it for some years, focusing on client relationship, quality of service, staff competence, efficiency and productivity, as its four areas. Seppo Laine, Senior Vice President of Finpro, notes that it can be quite difficult to judge "national impact" or "competence development". Over the years, Finpro has built on its experience with one numeric approach to develop a better system, but admits that a final version is still a long way off.

Contingent valuation

Developed by the Helsinki School of Economics and Business Administration in 2001, the contingent valuation method looks at the value that the beneficiary of a trade support service would be willing to pay for that service in retrospect (i.e. after a reasonable interval of time needed to assess impact).

It then analyses the difference between the actual cost of the service and the ex-post facto value the client attributes to the service. If positive, it can be used to justify increased funding. Alternatively, if marginal or negative, it clearly indicates the need for some remedial action, or a dropping of the activity altogether.

In Finpro's pilot study, assessment is carried out through one-hour interviews with representative samples from its 3,000 company clients. "Don't confuse the questions we are asking when we seek to assess impact with the questions we ask about client satisfaction," Mr. Laine stresses. He also notes that once research is completed, it still needs to be translated into a political language whose policy implications can be understood.

Benchmarking

Benchmarking is a process used to assist with performance measurement. It is used extensively to measure corporate sector performance. For several years now, it has been used as a tool for measuring countries' competitiveness in the World Economic Forum's Global Competitiveness Report.

Benchmarking is less widely used for trade strategy performance measurement, but there is an increasing interest at the trade promotion organization level in looking at the performance of other TPOs in selected areas. Canada's Department of Foreign Affairs and International Trade carried out an extensive study of the pricing policy of selected TPOs in 1998. More recently OSEC, Switzerland's trade support organization, has been benchmarking other European TPOs.

What should TSIs do?

TSIs need to engage in performance measurement, not only to satisfy funding sources, but also to satisfy increasing demands of clients and to help improve the effectiveness, efficiency and relevance of services offered. There are a number of tools available - trade statistics, client surveys, balanced scorecards, etc. But each TSI must choose the methods that best suit its current situation and that will satisfy its stakeholders - financial backers, clients, staff and the general public.

Performance measurement and other forms of assessment are not popular. They are regarded as an additional chore, with sometimes unexpected and unpleasant results. There is, therefore, a sizable job of advocacy and education to be done by development agencies like ITC and the other institutional members of the global trade support network, particularly in developing and transition economy countries, as well as by strategy-makers.

But this is only half the job. As Stephen Sultana of Malta's export promotion organization notes: "Change is not fundamentally about structure, it is about people." There is no one way to manage change - "All change processes are imperfect," Mr. Sultana said - and it can take two years for stability to re-establish itself after reorganization. "Transparency is crucial. People in the organization have to know what you are doing. You have to make changes incrementally, through a consultative process."

Finally, though effective performance measurement starts with clear objectives, the trading environment is ever-different and creating constantly changing needs for information. Tools must therefore also change over time.

Views on measuring performance

"If you don't measure it, you had better be sure that it is done. We have found that if you don't measure it, it won't get done or not entirely."

Raymond Yip, Regional Manager (Western Europe), Hong Kong Trade Development Council, at Executive Forum 2002

"National strategy-makers have to analyse and benchmark national comparative advantage in terms of country resources and level of development. But the main comparisons must be at the sector, business and company level."

Nurgul Erdogan of the Export Promotion Centre, Turkey, in the Executive Forum 2002 e-discussion

"The days of trade promotion are over. Today it's about supporting companies to make them more successful in exporting. We are moving from promoters to advisors, from services to solutions, from clients as customers to clients as partners. […]

If you are seeking more money from the government, it's not enough to tell the minister that you have 80% customer satisfaction (METCO's rating). We were able to get almost a doubling of our budget a few years ago by showing the minister that in two markets we had been able to increase exports by a significant percentage by promotion over the previous two years."

Stephen Sultana, Malta External Trade Corporation (METCO), at Executive Forum 2002

"Performance measurement, and its corollary - results-based management - are, in my view, no longer an option for national export strategists. […] Another very important indicator is satisfaction of employees. Employee satisfaction has been documented to have a strong correlation with customer satisfaction."

Norman Baillie-David, Managing Partner, The Antima Group, Canada, in the Executive Forum 2002 e-discussion

"What should managers do to take advantage of benchmarking? First, they should understand the company that is being benchmarked - its environment, management, governmental policies, current laws, etc. Only then can a benchmark model have the potential to take the company further. Managers should benchmark other companies with a degree of flexibility.

Performance measures are crucial, not to determine whether strategy is effective, but as a key ingredient for continuous improvement.

Macroeconomic statistics are useful but only part of the equation. […] There has to be a compromise between considering customer feedback and what the manager wants to do. Managers cannot be 'just floating ships' navigating by the customer's sails."

Arturo Corrales, Economist from CC&R Pacific Business, Peru, in the Executive Forum 2002 e-discussion.

For more information, contact: Philip Williams, ITC Senior Adviser on Institutional Aspects of Trade Promotion, at williams@intracen.org
A detailed study of performance measurement and a report on Finpro's experience can be found at the Executive Forum web site (http://www.intracen.org/execforum).