About the toolkit

In November 1993, the International Development Research Centre (IDRC) and several partner organizations sponsored an international conference entitled Needs-Based Technology Assessment: Exploring Global Interfaces, in Ottawa, Canada. The conference was attended by 103 delegates, including government representatives from 15 developing countries, international non-governmental organizations, donor agencies and several universities. The major recommendation of this conference addressed the urgent need for the international technology assessment community to develop a means by which developing countries could acquire the expertise to implement a needs-based technology approach, linking the needs and priorities of the population to policy development and implementation.

The favoured approach to meet this need was the development of a toolkit which would contain a variety of tools and methods appropriate for conducting needs-based health technology assessments at different levels of sophistication, as well as case studies which would test applications. The development of a toolkit for needs-based technology assessment was considered to have a number of important points. Ideally, it should:

• facilitate knowledge brokering to communities and countries that do not have ready access to the existing range of researched options;
• conserve scarce resources by not re-inventing the wheel;
• promote a sound decision-making and policy process; and
• be useful to donor agencies and countries in their discussions regarding community health development and resource allocation.

In response to this recommendation, the World Health Organization (WHO) Collaborating Centre for Health Technology Assessment at the University of Ottawa brought together a team of international experts in health technology assessment to develop such a toolkit. A draft conceptual framework was prepared and presented to a small group of international experts at a half-day workshop held in Ottawa. The draft concept and overall framework with draft sub-components was presented at the International Society for Technology Assessment in Health Care (ISTAHC) conference in June 1995. This forum afforded the researchers leading this initiative, the opportunity to debate and receive comments on the overall direction and approach for the toolkit from a range of experts in the field. Provisional consensus was reached on the framework and a project proposal was submitted to IDRC who then provided core funding.

This section gives an overview of the basic steps in the technology assessment process based on a health needs perspective as described with this toolkit. It also incorporates the method of technology assessment as developed at the Canadian Coordinating Office for Health Technology Assessment (CCOHTA).

Technology Assessment Process

Needs Assessment and Priority Setting

1. Perform a health needs assessment of the population group.
   - What type of needs will be addressed? (e.g. health vs healthcare, absolute vs relative, perceived vs expressed or measured)
   - How will the needs be measured (e.g. morbidity, mortality, risk factors, equity, indicators)
   
2. Prioritize the health needs, and decide which need(s) you will concentrate on first.
   - What criteria, process and participants will be used to prioritize these needs?
   
3. Identify the technologies that could be used or are in use to treat, prevent or manage the priority health need(s).
   
4. Prioritize these technologies to decide which ones will be assessed.
   - A preliminary literature review and/or a consultation with experts on this technology will help to identify the issues.
   - Determine the criteria to be used to prioritize which technology will be assessed.
   Examples of criteria:
   • Is there sufficient quality information available to conduct an assessment?
   • Is this technology likely to have a significant impact on the quality of patient care, or on the health care system?
   • Is there a variation in practice patterns?
   • Is there a question regarding over- or under-utilization?
   • Does this technology call into question the existing "gold standard" of therapy or practice?
   • Has this technology been replaced by another technology?
   • Is there no existing therapy or treatment?
   • Technology Assessment CCOHTA's Method

1. Identify external specialists with expertise in the area who are willing to participate in the assessment as advisors and reviewers.
   
2. Define the assessment objectives and define the target audience in consultation with the advisory group and with those who have requested the assessment.
   - What does your audience need to know?
   - How will the results of the assessment be used?
   - What format will the results take? Eg. full written report, summaries, online.
   - Identify primary HTA issues of interest including safety and adverse effects, efficacy / effectiveness, cost and cost-effectiveness, budget impact of health system, appropriate utilization, significant social implications, and ethical / distributional / equity issues, etc.
   - Narrow the scope as much as possible. (Use SMART criteria: specific, measurable, action-oriented, realistic and time-bound).
   
3. Define specific questions to be asked. Consider clinical, social and economic impacts and identify relevant populations (e.g. Is the intervention effective in high risk groups? Are there frequent adverse effects? Is it cost-effective for healthy adults?).
   
4. Identify and collect relevant evidence, including:
   - a literature search with a defined strategy
   - information from key contacts, sources, other HTA agencies in this area and the owners or holders of the technology
   - studies, reports suggested by the advisory group
   
5. Analyze and synthesize the evidence:
   • Clinical review - this may include a systematic review of studies with or without a statistical analysis or meta-analysis. If a systematic review is undertaken, then define the criteria for study inclusion and exclusion, quality assessment and data extraction (reference can be made to specific criteria e.g. Cochrane). Specify the main health outcomes to be reported.
   • Economic evaluation - design an economic evaluation. Plan and develop an economic model, if required. Consider items 1-25 of CCOHTA's "Guidelines for Economic Evaluation of Pharmaceuticals: Canada", and indicate any deviations.

1. Initiate review of the drafts of the assessment to verify content and methods. This step is important to ensure scientific rigor. (A consistent review process should be developed that includes internal and external reviewers.)
   
2. Write the final report, edit, format and print for distribution.
   
3. Dissemination
   

CCOHTA Standardized Report Format

1. Reviewers - a list of external reviewers
   
2. Table of Contents
   
3. Executive Summary
   
4. Introduction
   - clinical background
   - main issues surrounding the technology
   - product description
   
5. Objectives of the assessment
   
6. Clinical Review
   - methods
   - results
   
7. Meta-analysis results
   
8. Cost-effectiveness analysis/ economic evaluation Method:
   - type of analyses
   - decision model
   - target audience
   - viewpoint
   - treatment comparator(s)
   - time horizon
   - outcome measurement
   - cost measurement and valuation
   - sub-group analyses
   -assumptionsResults - analysis and results
   Sensitivity analyses
9. Discussion
   
10. Limitations
    
11. Conclusions
    
12. References
    
13. Appendices