Foundations of Economic Evaluation

iHEA Congress, July 2023

Learning Objectives and Outline

Learning Objectives

  • Identify theoretical and methodological differences between different economic evaluation techniques.

  • Understand the concepts of summary measures of health, including quality-adjusted life years (QALYs) and disability-adjusted life years (DALYs).

  • Be familiar with the steps of valuing costs in economic evaluations.

Outline

  • Introduction to Economic Evaluations

    • Types of Economic Evaluations

    • Who Uses Economic Evaluations

  • Valuing Health Outcomes

    • QALYs (briefly)

    • DALYs (focus of lecture)

  • Valuing Costs

Introduction to Economic Evaluations

Economic Evaluation

  • Relevant when decision alternatives have different costs and health consequences.

  • We want to measure the relative value of one strategy in comparison to others.

  • This can help us make resource allocation decisions in the face of constraints (e.g., budget).

Features of Economic Evaluation

  • Systematic quantification of costs and consequences.
  • Comparative analysis of alternative courses of action.

Techniques for Economic Evaluation

Type of study Measurement/Valuation of costs both alternative Identification of consequences Measurement / valuation of consequences
Cost analysis Monetary units None None

Cost analysis

  • Only looks at healthcare costs

  • Relevant when alternative options are equally effective (provide equal benefits)

    • Rarely the case in reality!
  • Costs are valued in monetary terms (e.g., U.S. dollars)

  • Decision criterion: often to minimize cost

Techniques for Economic Evaluation

Type of study Measurement/Valuation of costs both alternative Identification of consequences Measurement / valuation of consequences
Cost analysis Monetary units None None
Cost-effectiveness analysis Monetary units Single effect of interest, common to both alternatives, but achieved to different degrees. Natural units (e.g., life-years gained, disability days saved, points of blood pressure reduction, etc.)

Cost-Effectiveness Analysis (CEA)

Most useful when decision makers consider multiple options within a budget, and the relevant outcome is common across strategies

  • Costs are valued in monetary terms ($)
  • Benefits are valued in terms of clinical outcomes (e.g., cases prevented or cured, lives saved, years of life gained, quality-adjusted life years gained)
  • Results often reported as a cost-effectiveness ratio

Cost-Effectiveness Analysis

  • Suppose we are interested in the prolongation of life after an intervention.

  • Outcome of interest: life-years gained.

  • The outcome is common to alternative strategies; they differ only in the magnitude of life-years gained.

  • We can report results in terms of $/Life-years gained

Techniques for Economic Evaluation

Type of study Measurement/Valuation of costs both alternative Identification of consequences Measurement / valuation of consequences
Cost analysis Monetary units None None
Cost-effectiveness analysis Monetary units Single effect of interest, common to both alternatives, but achieved to different degrees. Natural units (e.g., life-years gained, disability days saved, points of blood pressure reduction, etc.)
Cost-utility analysis Monetary units Single or multiple effects, not necessarily common to both alternatives. Healthy years (typically measured as quality-adjusted life-years)

Cost-Utility Analysis

  • Essentially a variant of cost-effectiveness analysis.
  • Major feature: use of generic measure of health.
  • Quality-Adjusted Life Year (QALY): A metric that reflects both changes in life expectancy and quality of life (pain, function, or both).
  • By far the most widely published form of economic evaluation.

We will focus mostly on CEA (especially CUA) throughout the workshop

Techniques for Economic Evaluation

Type of study Measurement/Valuation of costs both alternative Identification of consequences Measurement / valuation of consequences
Cost analysis Monetary units None None
Cost-effectiveness analysis Monetary units Single effect of interest, common to both alternatives, but achieved to different degrees. Natural units (e.g., life-years gained, disability days saved, points of blood pressure reduction, etc.)
Cost-utility analysis Monetary units Single or multiple effects, not necessarily common to both alternatives. Healthy years (typically measured as quality-adjusted life-years)
Cost-benefit analysis Monetary units Single or multiple effects, not necessarily common to both alternatives Monetary units

Cost-Benefit Analysis

  • Also known as Benefit-Cost Analysis
  • Relevant for resource allocation between health care and other areas (e.g., education)
  • Costs and health consequences are valued in monetary terms (e.g., U.S. dollars)
  • Valuation of health consequences in monetary terms ($) is obtained by estimating individuals willingness to pay for life saving or health improving interventions.
    • e.g.: US estimate of value per statistical life ~$9 million
  • Cost-benefit criterion: the benefits of a program > its costs
    • Notice that we’re not making comparisons across strategies–only comparisons of costs and benefits for the same strategy
  • To read more: Robinson et al, 2019

Cost-Benefit Analysis

https://pubmed.ncbi.nlm.nih.gov/28183740/

Cost-Benefit Analysis

https://www.cambridge.org/core/product/identifier/S2194588818000271/type/journal_article

Who uses economic evaluations?

  • Health Technology Advisory Committees

    • PBAC (Pharmaceutical Benefits Advisory Committee in Australia)

    • Canada’s Drug and Health Technology Agency

    • NICE (The National Institute for Health and Care Excellence, UK)

    • Brazil’s health technology assessment institute

  • Groups developing clinical guidelines

    • WHO

    • CDC

    • Disease-specific organizations: American Cancer Society; American Heart Association; European Stroke Organisation

  • Regulatory agencies:

    • FDA (U.S. Food and Drug Administration)

    • EPA (U.S. Environmental Protection Agency)

CEAs: Identifying Alternatives

Identifying Alternatives

  • Decision modeling / economic evaluation requires identifying strategies or alternative courses of action.

  • These alternatives could include different therapies / policies / technologies.

  • Or, our alternatives could capture different combinations or sequences of treatment (e.g., what dose? what age to start?)

Once we have identified the alternatives, we’ll want to quantify their associated consequences in terms of:

  • Health outcomes

  • Costs

CEA components


\[ \frac{\text{(Cost Intervention A - Cost Intervention B)}}{\text{(Benefit A - Benefit B)}}\]

Valuing Health Outcomes

Why summary measures of health?

  • QALYs and DALYs both provide a summary measure of health

  • Allows comparison of health attainment / burden across diseases

    • Across diseases

    • Across populations

    • Across interventions

    • Over time etc.

QALYs

  • Origin story: welfare economics

    • Utility = holistic measure of satisfaction or wellbeing
  • With QALYs, two dimensions of interest:

    • length of life (measured in life-years)

    • quality of life (measured by utility weight, usually between 0 and 1)

QALYs


QALY: A metric that reflects both changes in life expectancy and quality of life (pain, function, or both)


1 = perfect health; 0= death;
Sum of weight*duration of life = quality-adjusted life expectancy

Example: Patient with coronary heart disease (with surgery)

Example: Patient with coronary heart disease (with surgery)

Example: Patient with coronary heart disease (without surgery)

Example: Patient with coronary heart disease

  • With surgery: 7.875 QALYs
  • Without surgery: 6.625 QALYs
  • Benefit from surgery intervention:
    • In QALYs: 7.875 – 6.625 QALYs = 1.25 QALYs

    • In life years: 10 years – 10 years = 0 LYs

Utility weights – How are they obtained?

  • Utility weights for most health states are between 0 (death) and 1 (perfect health)

  • Direct methods

    • Standard gamble

    • Time trade-Off

    • Rating scales

  • Indirect methods:

    • EQ-5D

    • Other utility instrument: SF-36; Health Utilities Index (HUI)

Off-the-shelf numbers for your own CEAs?


  • Large scale utility studies
  • Balancing trade offs between different measures
    • Sensitivity analyses!
  • Tufts CEA registry
  • Alongside Randomized Clinical Trials

DALYs


QALY: “0” = death; “1” = perfect health

DALY: “0” = perfect health; “1” = death

  • QALY measures the number of healthy years gained (uses preference-based utility weights)
  • DALY measures the number of healthy years lost (uses standardized disability weights)

Source: ghcearegistry.org

DALYs

A measure of population ill-health based on “years of life lost” due to premature mortality (Anand & Reddy LSE 2019).

  • Origin story: Global Burden of Disease Study

  • Deliberately a measure of health, not welfare/utility

  • Similar to QALYs, two dimensions of interest:

    • length of life (differences in life expectancy)

    • quality of life or morbidity (measured by disability weight)

DALYs

DALYs = YLL + YLD

  • YLL (Years of Life Lost): The # of life years a person could have expected to live had they not died
  • YLD (Years Lived with Disability)

DALYs = YLL + YLD


  • Years of Life Lost (YLL): changes in life expectancy; time lost due to premature mortality

  • Different approaches to identifying the time lost due to premature mortality:

    • Exogenous: Maximum length of life observed in modern world, i.e., “synthetic life table”; irrespective of country and socioeconomic characterstics/etc. where death occurs

    • Endogenous: Dependent on a person’s country of residence & other factors in which the death occurs

    • Simulation-based: Apply 1-disability weight to each health state; based on life expectancy from model for a specific disease cohort (e.g., CVD; life expectancy might be different than whole population)

DALYs = YLL + YLD

  • Approach depends on purpose of study

    • Synthetic life table: Quantify disease burden across countries in relation to normative benchmark and/or in light of global justice and resource re-allocation to low-income countries
    • Country-specific life table: Assessing alternative disease interventions in Uganda; & want to quantify how many expected years of life are lost due to a disease in conditions specific to Uganda.

Source: Anand & Reddy LSE 2019

DALYs Example: Burden of HIV

  • DALYs = YLL + YLD

  • Example: The overall disease burden of someone with HIV and not on treatment

    • An individual dies at age 50 due to untreated HIV
    • Calculating YLL from a synthetic life table
Age Life Expectancy Age Life Expectancy
0 88.9 50 39.6
1 88.0 55 34.9
5 84.0 60 30.3
10 79.0 65 25.7
15 74.1 70 21.3
20 69.1 75 17.1
25 64.1 80 13.2
30 59.2 85 10.0
35 54.3 90 7.6
40 49.3 95 5.9
45 44.4

Source: http://ghdx.healthdata.org/record/ihme-data/global-burden-disease-study-2019-gbd-2019-reference-life-table

DALYs Example: Burden of HIV

  • DALYs = YLL + YLD

    • YLL, HIV no treatment burden: 39.6 YLL
      • Compared to life expectancy of someone without HIV & lives in “ideal” environment with maximum life expectancy
    • This is the mortality part of the DALY
  • Now suppose that untreated HIV would render someone 50% disabled for the final 10 years of their life

    • YLD, HIV no treatment: 0.5*10 = 5 more DALYs incurred
  • Overall disease burden due to untreated HIV can be represented as: 39.6 + 5 = 44.6 total DALYs

DALYs = YLL + YLD

  • Now let’s look at the DALY effect of treatment

  • YLL: Providing HIV treatment extends life by 20 years compared to no treatment (an individual with HIV now lives until 70 years on treatment)

    • YLL?

Synthetic, Reference Life Table

Age Life Expectancy Age Life Expectancy
0 88.9 50 39.6
1 88.0 55 34.9
5 84.0 60 30.3
10 79.0 65 25.7
15 74.1 70 21.3
20 69.1 75 17.1
25 64.1 80 13.2
30 59.2 85 10.0
35 54.3 90 7.6
40 49.3 95 5.9
45 44.4

Source: http://ghdx.healthdata.org/record/ihme-data/global-burden-disease-study-2019-gbd-2019-reference-life-table

Synthetic, Reference Life Table

Age Life Expectancy Age Life Expectancy
0 88.9 50 39.6
1 88.0 55 34.9
5 84.0 60 30.3
10 79.0 65 25.7
15 74.1 70 21.3
20 69.1 75 17.1
25 64.1 80 13.2
30 59.2 85 10.0
35 54.3 90 7.6
40 49.3 95 5.9
45 44.4

Source: http://ghdx.healthdata.org/record/ihme-data/global-burden-disease-study-2019-gbd-2019-reference-life-table

DALYs = YLL + YLD

  • Years of Life Lost (YLL): changes in life expectancy, calculated from comparison to synthetic life table

    • YLL, HIV on treatment: 21.3
      • Compared to life expectancy of someone without HIV & lives in “ideal” environment with maximum life expectancy
    • This is the mortality part of the DALY

DALYs = YLL + YLD

Note

YLL (measured as DALYs averted) \(\neq\) LYs gained!

DALYs = YLL + YLD

  • Now let’s say that HIV treatment reduces the duration of disability of HIV from 10 years to 2.

    • YLD, HIV on treatment: = 0.5 * 2 = 1
    • Now we have the following effects of treatment: YLL = 21.3 + YLD = 1 = 22.3 DALYs

YLL, HIV on treatment compared to HIV NOT on treatment: LE(50)-LE(70) = 21.3-39.6 = -18.3 YLLs averted

YLD, HIV on treatment compared to HIV NOT on treatment: 5 - 1 = 4 YLDs averted

TOTAL DALYs averted from treatment = 18.3 + 4 = 22.3 total DALYs averted from treatment

DALYs: Another Example

  • Years Lived with Disability (YLD): calculated similar to QALYs, utility weight ≈ 1 - disability weight

  • YLD example: Effective asthma control for 10 years

    • Disability weight (uncontrolled asthma) = ?

    • Disability weight (controlled asthma) = ?

Disability Weights

  • Common values for small set of named health conditions (e.g. early/late HIV, HIV/ART)
  • First iteration: expert opinion
  • Second iteration: Pop-based HH surveys in several world regions (13,902 respondents)
    • Paired comparison of two health state descriptions which worse

    • Probit regression to calculate disability weights

    • 235 unique health states

Source: Salomon, Joshua A., et al. “Disability weights for the Global Burden of Disease 2013 study.” The Lancet Global Health 3.11 (2015): e712-e723.

DALYs = YLL + YLD

  • Years Lived with Disability (YLD): calculated similar to QALYs, utility weight ≈ 1 - disability weight

  • YLD example: Effective asthma control for 10 years

    • Disability weight (uncontrolled asthma) = 0.133

    • Disability weight (controlled asthma) = 0.015

    • YLD = 10 * 0.015 - 10 * 0.133 = -1.18 DALYs = 1.18 DALYs averted

DALYs for CEA

  • Recommended calculation approach has changed over time (age weighting, discounting, now both out)
  • Some will calculate a “QALY-like” DALY, using utility weight = 1 - disability weight

Important

Common practice

  • High-income setting: QALYs
  • Low- and middle- income setting = DALYs***Since disability weights are freely & publicly available (these weights are required for DALY calculations), it can reduce costs/time/resources compared to collecting QALY estimates

Back to CEA components


\[ \frac{\text{(Cost Intervention A - Cost Intervention B)}}{\text{(Benefit A - Benefit B)}}\]

  • A common outcome is: Cost per DALY averted

  • Next up: Costs; though just briefly & then we will move on to CEA thresholds

Valuing Costs

Valuing Costs: Steps

Source: Gold 1996, Drummond 2015, Gray 2012)

  1. Identify – Estimate the different categories of resources likely to be required (e.g., surgical staff, medical equipment, surgical complications, re-admissions)

  2. Measure – Estimate how much of each resource category is required (e.g. type of staff performing the surgery and time involved, post-surgery length of stay, re-admission rates)

  3. Value – Apply unit costs to each resource category (e.g., salary scales from the relevant hospital or national wage rates for staff inputs, cost per inpatient day for the post-surgery hospital stay)

We can identify different types of healthcare costs

  • Direct Health Care Costs

    • Hospital, office, home, facilities

    • Medications, procedures, tests, professional fees

  • Direct Non-Health Care Costs

    • Childcare, transportation costs
  • Time Costs

    • Patient time receiving care, opportunity cost of time
  • Productivity costs (‘indirect costs’)

    • impaired ability to work due to morbidity?

    • lost economic productivity due to death?

  • Unrelated healthcare costs

    • Cumulative trajectory of total healthcare costs over time (unrelated to medical interventions; more on this later)

Identifying costs (continued)


  • In practice, we count what is likely to matter

    • Exclude what is likely to have little effect or equal effects across alternatives
  • Any exclusion must be noted & possible bias examined

  • We are constrained by what data are available

We can measure costs using different approaches

  • Micro-costing (bottom-up)

    • Measure all resources used by individual patients, then assign the unit cost for each type of resource consumed to calculate the total cost
  • Gross-costing (top-down)

    • Estimate cost for a given volume of patients by dividing the total cost by the volume of service use
    • Example: Downstream costs (e.g., hospitalization due to opioid overdose)
  • Ingredients-based approach (P x Q x C)

    • Probability of occurrence (P)

    • Quantity (Q)

    • Unit costs (C)

Whose perspective?


PERSPECTIVE MATTERS –

Formal Healthcare Sector: Medical costs borne by third-party payers & paid for out-of-pocket by patients. Should include current + future costs, related & unrelated to the condition under consideration

Societal perspective: Represents the wider “public interest” & inter-sectoral distribution of resources that are important to consider - reflects costs on all affected parties

Where to draw the line? CEA thresholds

CEA Thresholds

  • What CEA cannot tell us is HOW TO SPEND
  • Let’s say we now have our cost per DALY averted estimate, how do we make a decision?
  • We must define a threshold – a value that determines whether or not we implement a given strategy.
  • What are common thresholds and how are they determined?

CEA Thresholds

  • Different ways thresholds have been estimated: - “supply-side” (UK & Europe) - “demand-side” (US) - per capita consumption (US/LMICs)

  • LMICs have often defined thresholds in terms of per capita consumption - consistent with the idea that people living in countries with higher incomes are able and willing to pay more for health - which makes intuitive sense

CEA Thresholds in LMICs

  • In the past, countries have used thresholds based on per capita GDP
  • Mainly the threshold of 1-3X per capita GDP
  • An intervention is cost-effective if cost/DALY averted is less than 1-3X per capita GDP of country

CEA Thresholds in LMICs

CEA Thresholds in LMICs

  • Although this threshold range roughly corresponds to what has become convention for high-income countries…

  • Per capita consumption in wealthier countries exceeds the per capita consumption in low & middle-income countries by one-to-two orders of magnitude & therefore, some analysts have argued that healthcare spending should represent a smaller portion of per capita GDP in low to middle income countries.

Some have argued the WHO’s guidelines may be too high and result in adoption of interventions that displace existing services that provide greater health benefit.

CEA Thresholds in LMICs

  • 0.5 GDPpc may be a more appropriate benchmark for low-income countries and 0.71 GDPpc for middle-income countries (see Woods et al 2016)

  • GDP: Measure of actual economic output of a country

  • Purchasing Power Parity (PPP): Used to compare purchasing power & living standards across countries

  • Purchasing Power Parity (PPP) adjusted GDP; but often alongside unadjusted

  • Adjusting GDP using PPP may provide a more accurate measure of economic performance and living standards, allowing comparisons between countries

CEA Thresholds in LMICs


  • Until we come up with a better measure, the DSP3 (Disease Control Priorities, LSHTM) & other academics have estimated more conservative thresholds (previous slide) that I would use for LMICs & would NOT use the per capita assumptions

Useful Resources

Useful Resources

Useful Resources

Useful Resources

Drummond, Michael F., Mark J. Sculpher, Karl Claxton, Greg L. Stoddart, and George W. Torrance. 2015. Methods for the Economic Evaluation of Health Care Programmes. Oxford university press.