Table 6

Table 6. List of outcome indicators
Domain	Name of indicator
Difusion	Adoption of the new technology
Difusion	Coverage of the technology
Accessibility	Accessibility
Adequacy of use	Adequacy of patient selection criteria
Effectiveness	Effectiveness
Effectiveness	Effectiveness by subgroup
Safety	Severe adverse effects
	Mild adverse effects
	Adverse effects by subgroup
Economic impact	Adequacy of costs

Indicator	Adoption of the new technology
	In what proportion of centres has the new technology been adopted?
Justification	The approval/funding of a new technology does not assure that it will be efficaciously adopted within the health system. Frequently, the implementation of a new technology requires, not only financial investment, but also structural and organisational changes, availability of trained staff, and time. Ascertaining the centres that have adopted the new technology can be essential in order to detect problems of implementation.
Formula	Total number of centres that have adopted the new technology within a given period of time Total number of centres at which adoption of the new technology is deemed desirable within a given period of time X 100
Explanation of terms	Adoption of a new technology: implementation and use of the new technology in clinical practice in the centres considered within the target area. Centres at which the adoption of the new technology is deemed desirable: centres at which the adoption of the technology is deemed desirable/foreseeable must be defined a priori. Depending on the health system and/or type of technology, these could be all the centres within the target area, all centres with the relevant specialisation or solely reference centres or centres that have been authorised to use the new technology.
Scope of study	The scope of study (geographical area, province, region, country) must be defined in order to delimit the centres that are to be targeted by the study and are to form the numerators and denominators of the formula.
Guideline standard	80-100%.
Time frame	A mean follow-up of 1-year after approval/funding of the new technology is taken as reference. This period may be extended to 2-3 years in the case of technologies that may require major financial, organisational or structural changes.
Data-sources	Suppliers or manufactures of the new technology (technologies that require a specific apparatus, device or fungible goods for their use). Health-centre service catalogue (if existing and updated). Direct contact with each centre and/or department (innovative procedures or techniques based on the learning of skills). Contact with administrative centres tasked with approval of centres and allocation of resources for public health care.

Indicator	Coverage of the technology
	How many patients of those eligible to use the new technology, actually use it?
Justification	Ascertaining the relationship between patients eligible to use the new technology and those that use it in clinical practice is important when it comes to detecting problems of coverage, detecting whether there may be an over-use of the new technology with respect to what was envisaged, or detecting possible resistance to its use (under-use)
Formula	Total number of patients that have used the new technology within a given period of time Total number of patients eligible to use the new technology within a given period of time X 100
Explanation of terms	Patients that have used the new technology: consideration will only be given to subjects for whom results of applying the technology are available. Scheduled interventions that were not performed must be excluded. Patients eligible to use the new technology: patients in the target area who meet the selection criteria set a priori by the multidisciplinary group
Guideline standard	80-100%. This must be set a priori.
Scope of study	The scope of the study (hospital, health area, geographical area, province, region, country) must be defined in order to delimit the units and services to be targeted by the study.
Time frame	A mean follow-up of 1-year after approval/funding of the new technology is taken as reference, though this may be extended in the case of technologies indicated in a small number of patients. This must be set a priori.
Data sources	Patients that have used the new technology: clinical registries, medical records and/or MBDS registries Patients eligible to use the new technology: Disease frequency registries (number of cases, prevalence..) Studies published in the scientific literature Registries on the number of procedures using alternative techniques to those that this would replace.

Indicator	Accesibilidad
	What is the degree of use of the new technology in the various geographical areas?
Justification	When a new technology is introduced, an appropriate patient-reference circuit must be established to ensure that the degree of use is independent of this technology’s availability in the health area. Calculating and comparing the use of a new technology in different geographical areas is fundamental for identifying problems of geographical accessibility
Formula	Number of patients residing in a predefined geographical area who have used the new technology within a given period of time Patients residing in a geographical area eligible for using the new technology within this period of time X 100
Explanation of terms	Subjects that have used the new technology: consideration will be given only to subjects for whom results of applying the technology are available. Scheduled interventions that were not performed must be excluded. Subjects susceptible to using the new technology: patients in the target area who meet the selection criteria set a priori by the multidisciplinary group.
Guideline standard	This must be set a priori
Scope of study	The scope of the study (health area, province, region, country) must be indicated, in order to delimit the health centres and areas to be targeted by the study.
Time frame	A mean follow-up of 1-year after approval/funding of the new technology is taken as reference, though this may be extended in the case of technologies indicated in a small number of patients. This must be set a priori.
Data sources	Patients that have used the new technology: clinical registries, medical records and/or MBDS registries Patients eligible to use the new technology: Disease frequency registries (number of cases, prevalence) Studies published in the scientific literature Registries on the number of procedures using previous alternative techniques for the same indication.

Indicator	Adecuación de criterios de selección de pacientes
	Are the patient-selection criteria used within the various centres, departments or units in line with what is deemed appropriate and/or desirable for this technology?
Justification	It is important to ascertain whether patients on whom the new technology is used meet the selection criteria. Inappropriate use could cause the loss of important resources without any proven benefit (financial, personal, time) or even impact negatively on the health of the population.
Formula de cálculo	Number of patients among whom the technology has been used, who meet the inclusion criteria Number of patients that have used the technology X 100
Descripción de términos	Inclusion criteria: consideration will be given to the clinical indications and to the different characteristics of the patients (age, degree of pain, ineligibility for other treatments) which are used to define patients to be treated/diagnosed with the new technology. The inclusion criteria will be established a priori, taking into account the authorised selection criteria for use of the technology within the health system (if these exist), scientific evidence and/or opinion of experts. Subjects that have used the new technology: consideration will only be given to subjects for whom results of applying the technology are available. Scheduled interventions that were not performed must be excluded.
Guideline standard	100%
Scope of study	The scope of the study (hospital, health area, geographical area, province, region, country) must be indicated, in order to delimit the departments and units that are going to be deemed the study target.
Time frame	A mean follow-up of 1-year after approval/funding of the new technology is taken as reference, though this may be extended in order to attain a minimum number of 25-30 interventions (patients).This must be defined a priori.
Data sources	clinical registries, medical records and/or MBDS registries.

Indicator	Efectividad
	Is the new technology’s degree of success in line with expectations?
Justification	The expectations foreseen in preliminary studies may not be fulfilled when the technology is applied in daily clinical practice (different level of experience, patient profiles, etc.). Should this be so, it may well be necessary for the use of such technology to be reconsidered.
Formula	Number of subjects among whom a beneficial effect has been observed Total number of subjects that have used the new technology X 100
Descripción de términos	Beneficial effect: effect deemed acceptable or desirable for this technology. According to the type of technology, the outcome variables to be used to define the new technology’s success and the values deemed acceptable or desirable for this variable must be defined a priori. Total number of subjects that have used the new technology: deemed to be all subjects in the target population who have been treated or diagnosed with the new technology.
Outcome variables	They must be set a priori. As far as possible, these variables must be objective or capable of being rendered objective.
Guideline standard	90-100%. This must be set a priori..
Scope of study	The scope of the study (hospital, health area, geographical area, province, AR, country) must be indicated, in order to delimit the units and departments that are going to be considered.
Tiempo de seguimiento	SA mean follow-up of 1-year after approval/funding is taken as reference, though this may be extended in order to attain a minimum number of 25-30 interventions (patients).
Data sources	Specific clinical registries Medical records Surveys/questionnaires

	Subgroup effectiveness
	Is the new technology’s percentage of success in line with what was expected for certain subgroups of patients?
Justification	Preliminary studies often fail to envisage assessment of effectiveness in specific subgroups (children, the elderly, subjects with serious co-morbidities) and this may differ significantly from overall effectiveness. Similarly, where similar though not identical procedures or procedures that require a learning curve are involved, effectiveness may also differ significantly as between the various subgroups (team, degree of specialisation, etc.).
Formula	Number of patients in a study subgroup among whom a beneficial effect has been observed Total number of subjects in the subgroup that have used the new technology X 100
Descripción de términos	Study subgroup: subgroups susceptible to registering differences in terms of effectiveness. These must be established a priori. Beneficial effect: effect deemed acceptable or desirable for this technology. According to the type of technology, the outcome variables to be used to define the new technology’s success and the values deemed acceptable or desirable for this variable must be defined a priori. Total number of subgroup subjects that have used the new technology: deemed to be all subjects in the target study subgroup who have been treated or diagnosed with the new technology.
Outcome variables	They must be set a priori. As far as possible, these must be objective or capable of being rendered objective.
Guideline standard	90-100%. This must be set a priori.
Scope of study	The scope of the study (hospital, health area, geographical area, province, region, country) must be indicated, in order to delimit the departments and units that are going to be considered.
Tiempo de seguimiento	A mean follow-up of 1-year after approval/funding is taken as reference, though this may be extended in order to attain a minimum number of 25-30 interventions in each subgroup.
Data sources	Specific clinical registries Medical records Surveys/questionnaires

	Severe adverse effects
	What proportion of patients present with severe adverse effects?
Justification	In daily clinical practice, it frequently happens that severe adverse effects not envisaged in preliminary studies appear, or that these appear in a greater proportion of subjects. Quantifying these variations is important for ensuring patient safety.
Formula	Number of patients among whom a severe adverse effect has been observed Total number of patients on whom the new technology has been used X 100
Descripción de términos	Severe adverse effects: for practical purposes severe adverse effects are deemed to be any injury, damage or complication deriving from the use of the technology, including diagnostic errors (FP, FN) which can lead to death or major deterioration of patient’s state of health. The type of adverse effects to be considered for study purposes must be established a priori. Total number of subjects that have used the new technology: deemed to be all subjects in the target population who have been treated or diagnosed with the new technology.
Guideline standardes	This must be defined a priori
Scope of study	The scope of the study (hospital, health area, geographical area, province, AR, country) must be indicated, in order to delimit the departments and units that are going to be considered objective of the study.
Tiempo de seguimiento	A mean follow-up of 1-year after approval/funding is taken as reference, though this may be extended in order to attain a minimum number of 25-30 interventions (patients).This must be defined a priori.
Data sources	Specific clinical registries Medical records Surveys/questionnaires

	Mild to moderate adverse effects
	What proportion of patients present with moderate and/or mild adverse effects?
Justification	Analysis of adverse effects is the basis for identifying genuine opportunities for improvement. Adverse effects, albeit not severe, may affect patients’ quality of life and, where frequent, can give rise to considerable resource use.
Formula	Number of subjects among whom moderate and/or mild adverse effects have been observed Total number of subjects on whom the new technology has been used X 100
Descripción de términos	Moderate and/or mild adverse effects: for practical purposes these are deemed to be any non-severe injury, damage or complication deriving from the use of the technology, as well as any diagnostic error (FP, FN) that has led to an alteration in the patients’ state of health. The type of adverse effects to be considered for study purposes must be established a priori. Total number of subjects that have used the new technology: deemed to be all subjects in the target population who have been treated or diagnosed with the new technology.
Guideline standardes	These must be defined a priori
Scope of study	The scope of the study (hospital, health area, geographical area, province, region, country) must be indicated, in order to delimit the units and departments that are going to be considered objective of the study.
Tiempo de seguimiento	A mean follow-up of 1-year after approval/financing is taken as reference, though this may be extended in order to attain a minimum number of 25-30 interventions (patients).This must be defined a priori.
Data sources	Specific clinical registries Medical records Surveys/questionnaires

	Adverse effects in subgroups
	Are the adverse effects in line with what was expected in especially susceptible subgroups?
Justification	Especially sensitive subgroups (children, the elderly, pregnant women, subjects with serious co-morbidities), subgroups with different degrees of specialisation or different ways of performing the intervention (apparatus, protocols, teams, etc.), could register previously unexpected adverse effects.
Formula	Number of subjects in a subgroup among whom an adverse effect has been observed Total number of subjects in the subgroup that have used the new technology X 100
Descripción de términos	Study subgroup: subgroups susceptible to present a higher proportion of adverse effects. These must be established a priori. Adverse effects: for practical purposes these are deemed to be any injury, damage or complication deriving from the use of the technology, as well as any diagnostic error (FP, FN) which has led to an alteration in the patient’s state of health. The type of adverse effects to be considered for study purposes must be established a priori. Total number of subjects that have used the new technology: deemed to be all subjects in the target population who have been treated or diagnosed with the new technology.
Guideline standardes	These must be defined a priori.
Scope of study	The scope of the study (hospital, health area, geographical area, province, AR, country) must be indicated, in order to delimit the units and departments considered for study purposes.
Follow-up time	A mean follow-up of 1-year after approval/financing is taken as reference, though this may be extended in order to attain a minimum number of 25-30 interventions in each subgroup.
Data sources	Specific clinical registries Medical records Surveys/questionnaires

	Adequacy of costs
	Are the observed costs in line with what was expected?
Justification	For adequate planning, it is essential to ascertain whether the estimated costs are in line with the real costs of the technology.
Formula	Observed costs under specific chapters of major financial impact Estimated costs under specific chapters of major impact X 100
Description of terms	Costs: includes costs needed for implementing the new technology but also costs derived from its utilisation in clinical practice. Includes costs of apparatus, support material, fungible costs, maintenance costs, waste management, health service transport, etc.). The costs to be considered for study purposes must be established a priori. Costs must be measured in monetary units. Chapter of major impact: according to the type of technology, the cost chapters responsible for major resource use must be established a priori.
Guideline standardes aceptables	These must be defined a priori.
Scope of study	The health centres to be targeted by the study must be defined a priori.
Data sourcess	Specific registries Pharmaceutical companies, producers and distributers of the technology Ad hoc calculation