Performance Indicators
Contact persons for Performance Indicators: Victoria Henson-Apollonio.
Contributors to this page: CGIAR CAS-IP, Rome, Italy (Victoria Henson-Apollonio); Global Crop Diversity Trust, Rome, Italy (Mellissa Wood, Jane Toll, Hang Nguyen); Bioversity International, Rome, Italy (Brigitte Laliberte, Jonathan Robinson, Jamie Watts); IRRI, Los Baños, Philippines (Ruaraidh Sackville Hamilton); IITA, Nigeria (Dominique Dumet); CIMMYT, Mexico (Thomas Payne).
Introduction
Performance Indicators (PIs) are a way to attempt to succinctly measure progress made (over time) towards achieving a goal. Ideally, PIs are quantifiable rather than qualitative, allow an analysis of trends, and are agreed upon by the organization or group that is using them. Obviously, goals change and PIs need to be reviewed on a periodic basis. PIs should help an organization improve its performance in achieving an agreed set of goals.
The activities that have led to a set of PIs for the CGIAR genebanks included:
- A background paper that describes the current state of discussion regarding the usefulness of indicators in general, the potential effects of using PIs to measure the performance of genebanks, and the indicators that had been developed by other genebanks.
- A meeting of the activity coordinators and interested parties for several of the Global Public Goods (GPG) activities related to PIs such as the “Best Practices”, “Decision Support Tools for Financing”, “Risk Management Guidelines”, and “Identification of User Groups”.
- A workshop that developed the initial set of GPG2-PIs, that included participation by the CGIAR genebank managers, managers of other public genebanks that had developed PIs and performance measurement systems, the Crop Diversity Trust, and professionals that develop and maintain quality management systems (QMS). Recognising the importance of not placing an unnecessary measurement and reporting burden onto already strapped genebanks, it seemed logical and efficient for the Trust to contribute to the development of these indicators with genebank managers, rather than develop an additional set in isolation, to do essentially the same thing.
- Further development and utilization of the initial set of PIs by the Trust for their long-term grant technical reporting.
- Incorporation of goals that were described in the GPG2-developed sustainability plan and human resources plans.
- Adjustment review of the PIs by the Trust based on feedback from reports from genebanks working with the Trust.
- Adjustment of the PIs, based on feedback from three CGIAR test genebanks.
- Final activity discussions with the Trust to produce a harmonized set of PIs.
In essence, two parallel streams of PI development activity took place over a three-year period. The Trust, building on the work resulting from the Lunteren workshop, further developed a set of PIs for assessing performance (primarily requiring quantitative data) that were utilized by partner genebanks over a two-year period. There was some review after the first year. In parallel, GPG2 further developed a set of PIs (more focused on processes) that were tested by three genebanks. A number of times these PI activities came together for alignment where possible (recommending that the same genebanks be involved in the reporting). Finally the genebanks involved requested only one set of PIs be used, recommending the Trust set as the base as they had all actively been involved in its development and testing. As a result, the two streams were completely harmonized in January 2010.
Recommended joint indicators
INDICATOR # |
PERFORMANCE INDICATORS FOR IN-TRUST COLLECTIONS |
Category A |
Conserving and making available the collection |
1. Management and planning |
|
1a |
Active progress towards implementing a Quality Management System - Yes/No/In Progress |
A Quality Management System (QMS) can be defined as a set of policies, processes and procedures required to plan and execute the activities of any genebank. Implementing a QMS within any organization needs to be a top management decision. QMS integrates the various internal processes within the organization to provide a process approach for execution. QMS enables organizations to identify, measure, control and improve the various core business processes that will ultimately lead to improved business performance. PLAN-DO-CHECK-ACT. See the quality management pages for more information. |
|
1b |
Development of a multi-year budget, subject to annual financial audits - Yes/No |
A multi-year budget plan is essential for long-term security of the collection. An external financial audit may be undertaken annually and at least every five years as part of external evaluation of projects. |
|
1c.1 |
Human Resources management plan - Yes/No/In Progress |
1c.2 |
Staff succession plan - Yes/No/In Progress |
1c.3 |
Staff performance appraisal plans - Yes/No/In Progress (see also http://sgrp.cgiar.org) |
A HR management plan would likely include a review and assessment of the organization's needs for employees, followed by the development of job descriptions, recruiting employees and hiring them. |
|
1d.1 |
Risk management plan - Yes/No |
1d.2 |
If yes, how often is the plan reviewed/updated? |
A risk management plan is concerned with how the organization assures itself, across the range of activities it is engaged in, that the opportunities and risks associated with existing operations and with potential new activities are identified, evaluated and acted upon in a timely manner. See the risk management pages for more information. |
Technical indicators
Category A |
Conserving and making available the collection |
2. Long-term storage and management of the collection according to agreed scientific and technical best practices. |
|
2a.1 |
Facilities/equipment meet long-term storage standards - Yes/No/In Progress |
Long-term storage requirements involve specific temperature, light, moisture control, physical space and use of packaging appropriate for individual crops. For vegetative accessions it involves cryopreservation. |
|
2a.2 |
A capital depreciation plan is in place - Yes/No |
Capital depreciation is a term used to describe the process of attributing the purchase cost of an asset over its useful life. A capital depreciation plan should provide details of the categories of capital assets held and the time period over which those assets are depreciated. |
|
2b.1 |
% of seed accessions stored under long-term conditions |
The target for this indicator is 100%. |
|
2b.2 |
% of vegetative accessions stored in cryopreservation |
The targets are still to be determined for individual crops. |
|
2c |
% of seed accessions requiring seed viability testing |
Assuming you have a plan for required monitoring of seed viability, state what % of seed accessions still require viability testing. The target is to have no backlogs in seed viability testing. |
|
2d.1 |
% of seed accessions requiring regeneration to restore viability |
Assuming you have a plan for required regeneration, state what % of seed accessions still require regeneration. The target is zero. |
|
2d.2 |
Are there accessions held in vitro that are in need of being planted out and checking for trueness-to-type? If yes, what % of the in vitro collection? |
2e |
% of accessions free of quarantine high risk pathogens |
3. Safety duplication of the collection |
|
3a.1 |
% of seed accessions safety duplicated in an off-site storage facility |
3a.2 |
% of in vitro collection safety duplicated in an off-site storage facility |
3b.1 |
% of seed accessions stored in Svalbard Global Seed Vault |
3b.2 |
% of cryopreserved collection safety duplicated in an off-site storage facility |
4. Characterization of the collection |
|
4a.1 |
% of accessions for which at least half of the standard minimum descriptor set has been recorded |
|
For 2008, the standard minimum descriptor set has been taken to be a key subset of the old descriptors, comprising 37 descriptors for sativa and glaberrima and 46 descriptors for wild relatives. A full list of the descriptors is available on request. |
4a.2a |
% for which at least six isozymes have been recorded |
4a.2b |
% for which at least 20 isozymes have been recorded |
4a.2c |
% characterized using current, modern techniques and specify the technique used |
5. Documentation of the collection and provision of data in publicly available information systems |
|
5 |
% of accessions in your collection with data available (from any institutional source) on the web |
For definitions see: http://www.bioversityinternational.org/nc/publications/publications/publication/ |
|
5a |
passport data |
5b |
characterization data |
5c |
evaluation data |
6. Distribution of germplasm in accordance with the International Treaty on PGRFA |
|
6a.1 |
Response rate (%) from customer satisfaction surveys |
6a.2 |
Average rating of customer satisfaction (from surveys) |
Use the scale of response from 1 - 5 |
|
6a.3 |
What % of requests for germplasm were you able to meet? |
6a.4 |
For those not able to be met, what were the main constraints for the unavailability of germplasm? |
Categories for unavailability include: low viability; low stock; diseases; difficult to regenerate; other. |
|
6b |
Number of samples distributed outside the host institute |
Do not include samples distributed for virus indexing purposes or for providing other conservation-related services. |
|
6c |
Number of distinct accessions distributed outside the host institute |
6d |
Number of new/different accessions distributed outside the host institute (compared to baseline – past four years) |
6e |
Number of countries distributed to |
Global system genebank indicators
Category B |
Promoting global collaboration in crop conservation |
7. Collaboration in crop conservation |
|
7a.1 |
Number of coordination and/or networking mechanisms the genebank is involved in with regard to PGR conservation and use |
7a.2 |
Please list them |
7b.1 |
Number of leadership actions the genebank has taken that furthers the implementation of the relevant global crop strategy |
7b.2 |
Please list them |
7c.1 |
How many institutes requested your institute to provide safety duplication services? |
7c.2 |
How many requests have been made for any other conservation services from external institutes (e.g. tissue culture, multiplication, disease indexing or therapy, etc)? |
8. Improving the coverage of the genepool ex situ (including analyzing and filling gaps in collections) in partnership with other genebanks |
|
8a.1 |
Geographic and/or taxonomic coverage of the collection assessed?- Yes/No/In Progress |
8a.2 |
If yes, what % of the total cultivated genepool does the collection represent? |
8a.3 |
If yes, what % of the genepool of primary wild relatives does the collection represent? |
8b |
Number of new accessions acquired |
8c |
Number of collaborative collecting activities for gap filling |
9. Providing training and capacity building to others (external to the organization) |
|
9a |
Number of people trained and description of key activities |
Please categorize trainees, for example: Undergraduate students; postgraduate students; technicians; researcher; farmers; others. |
|
9b |
Number of institutes to whom you have provided technical backstopping activities and describe key activities |
Technical backstopping activities include: providing specific technical or scientific advice in informal or formal interactions. |
|
10. Developing links to users and promoting use |
|
10a |
Number of germplasm recipients in each of the following categories (please only use one category per recipient): |
10a.1 |
ARIs |
10a.2 |
CGIAR Centre |
10a.3 |
Commercial company (private) |
10a.4 |
Farmer or farmer organization |
10a.5 |
Genebank |
10a.6 |
Germplasm network |
10a.7 |
Individual |
10a.8 |
NARS |
10a.9 |
NGO |
10a.10 |
University |
10a.11 |
Unknown |
10a.12 |
Other |
10b |
Number of ‘new’ germplasm recipients (compared to baseline – last four years) |
10c |
Are your publications that include information on Centre germplasm accessions available to patent examination offices via databases, such as the European Patent Office's (EPO) non-patent literature (NPL) database? - Yes/No |
References and further reading
Batterham R. 1995. How can we argue with performance indicators? In: Problems with performance indicators. Centre for Health Program Evaluation. Working Paper 54.
Campbell DT. 1979. Assessing the impact of planned social change. Evaluation and Program Planning, 2:67-90
CGIAR. 2006. Results 2005 for the CGIAR Performance Measurement Excercise. [online] Available from: http://www.cgiar.org/pdf/agm06_pm_results_2005.pdf. Date accessed 27 April 2010.
Department for Environment, Food and Rural Affairs. 2006. Environmental key performance indicators. Reporting guidelines for UK businesses, 76pp.
de Vincente MC. 2004. The evolving role of genebanks in the fast-developing field of molecular genetics. Issues in Genetic Resources. No. 11, International Plant Genetic Resources Institute, Rome, Italy.
European Academies' Science Advisory Council. 2004. A user's guide to biodiversity indicators. 60pp.
FAO/IPGRI. 1994. Genebank standards. Food and Agriculture Organization of the United Nations, Rome and International Plant Genetic Resources Institute, Rome. Available in English, Spanish, French and Arabic.
Gawande A. 2006. The Score, The New Yorker. [online] Available from: http://www.newyorker.com/printables/fact/061009a_fact. Date accessed 27 April 2010.
Grol R. 2000. Between evidence-based practice and total quality management: The implementation of cost-effective care. International Journal for Quality in Health Care, 12:297-304.
Guidelines for the reporting of performance indictors for CGIAR Centers. 2006 data. Science Council and CGIAR Secrtariat, January 11, 2007.
Houghton G, Rouse A. 2004. Are NHS primary care performance indicators scores acceptable as markers of general practitioner quality? British Journal of General Practice, May 2004:341-344.
Ibrahim JE. 2001. Performance indicators from all perspecives. International Journal for Quality in Health Care, 13:431-432.
Jones PD, James N, Mellalieu SD. 2004. Possession as a performance indicator in soccer. International Journal of Performance Analysis in Sport, 4:98-102.
Lindal M. 2000. Forest sector indicators, An approach for Central America. CIAT, World Bank, UNEP, 48 pp.
Millennium development holes. 2007. Nature, 446:347.
Perrin B. 2002. How to - and how not to - evaluate innovation. Evaluation, 8:13-28.
Rao NK. 2006. Managing seed genebanks - Challenges and options for improving efficiency. IPGRI. Newsletter, January 2006:5-6.
Rao NK, Hanson J, Dulloo ME, Ghosh K, Nowel D, Larinde M. 2006. Manual of seed handling in genebanks. Handbooks for Genebanks No. 8. Bioversity International, Rome, Italy. Available in English (1.5 MB), Spanish (1.4 MB) and French (1.9 MB).
SINGER. (Homepage of SINGER) [online] Available from: http://www.singer.cgiar.org. Date accessed 27 April 2010.
Van Koppen B. 2002. A gender performance indicator for irrigation: Concepts, tools and applications. IWMI Research Report no. 59, 52pp.
Van Thiel S, Leeuw FL. 2002. The performance paradox. Public Performance and Management Review, 25:267-281.
Comments
- No comments found
Leave your comments
Post comment as a guest