Introduction to a process that aids value chain decisions.
- Rapid health technology assessments (R-HTA) are focused systematic reviews of evidence for using biologicals, drugs, devices, and procedures to assist healthcare decision makers.
- R-HTAs are of most interest to advanced practice nurses and nurses in value chain positions.
- For a biological, drug, device, or procedure to be considered to have a positive impact, a statistically significant treatment effect must be accompanied by at least a minimally clinically important difference.
The World Health Organization defines a health technology assessment (HTA) as the systematic evaluation of biologicals, drugs, devices, and interventions. Rapid HTAs (R-HTAs), the focus of this article, help healthcare decision-makers use evidence-based medicine to come to quick conclusions about the properties, effects, and impacts of health technologies. R-HTAs may be written under time and resource constraints, such as reimbursement decisions, purchasing decisions, capital funding and formulary decisions, treatment referrals, and professional guideline development. R-HTAs typically encompass safety, efficacy, and cost data but don’t include the socioeconomic, legal, and ethical considerations intended to inform high-level policy-making decisions.
Nurses in value-analysis positions and advanced practice RNs (APRNs) should have an understanding of R-HTAs. For example, a nurse in the value chain receiving an R-HTA from a third-party vendor evaluating cardiac stents will need to make an informed decision about the assessment, and an APRN evaluating a guideline on adult immunizations should understand the strength of the evidence. Nurses also may be involved in creating an R-HTA.
This article provides an overview of the R-HTA process and development. (See R-HTA
This diagram illustrates the process for authoring a rapid health technology assessment (R-HTA). Other variations are possible.
Consider regulatory status
Your technology evaluation must be performed in the context of its regulatory status. For example, drugs and biologics are subject to Food and Drug Administration (FDA) New Drug Application and Biologics License Application approvals with requirements for clinical trial data. Medical devices are sorted into Class I, II, or III, with increasing regulatory control based on the intended use and perceived risk of patient harm.
Most Class II devices (for example, infusion pumps) require 510(k) clearance, for which the standard of review is substantial equivalence to legally marketed devices, so clinical trial data aren’t required and available literature may be limited. Class III devices (such as pacemakers) have greater potential for harm and typically require clinical study data for premarket approval. However, Class III devices receive less scrutiny than pharmaceuticals. Procedures aren’t subject to FDA regulation, but any drug or device used in the procedure is.
In Europe, the European Medicines Agency regulates drugs and the Conformité Européene regulates devices.
Focus the analysis
An R-HTA usually will focus on one intervention and a limited number of indications. For example, “Caplyta for the treatment of schizophrenia,” not “antipsychotics as a class for the treatment of schizophrenia.” Results for one specific indication shouldn’t be applied to another. For instance, tenotomy procedure results for treating plantar fasciitis shouldn’t be extended to treating elbow pain.
The PICOT framework can help focus your research question.
Population: What’s the disease or health status? (age, gender, and less often race)
Intervention: What’s being tested?
Comparison: What’s the intervention being compared to? (no control, pretest/posttest values, placebo, or active control that is or isn’t the standard of care)
Outcomes: What will be measured?
Time: How long is follow-up?
Guidelines and position papers are good sources for an indication’s standard of care.
Conduct a search
The literature search aims to capture the best available evidence, not necessarily all of the evidence. Start with a broad search string using terms that capture all the literature and simultaneously focus the search. For example, “(Caplyta OR lumateperone OR ITI) AND schizophrenia” would capture generic drug names, the indication of interest, and the trade name. Procedure terminology can be challenging because a procedure may be known by more than one name. For example, “percutaneous ultrasound-guided needle tenotomy” and “percutaneous ultrasound-guided needle fenestration.” Check the keyword field of a few available references to ensure that your search is comprehensive. Limiting the literature search to 5 years before the technology’s approval or clearance date is reasonable.
End users of R-HTAs, especially value chain users, want to know if a technology is reimbursable. Select up to six of the major payers most relevant to your organization, and check their commercial coverage determination for the technology you’re investigating. If you can’t find a specific technology, reference one that’s closely related.
Abstracts in medical literature databases are indexed by study type, patient age, gender, human vs. preclinical animal studies, and other characteristics. Use a minimum of filters, such as abstract and English language, in case the indexing is incorrect.
After concluding your search, determine if you’ve found sufficient evidence for review. A minimum standard for an R-HTA is two to five studies, depending on the number of patients and the number of controlled studies. Two randomized controlled trials (RCTs) usually are sufficient. Studies about rare debilitating disorders, clinical situations in which conducting controlled trials is difficult (such as emergency and critical care medicine), and indications with an unmet need (such as when currently available treatment options are unsatisfactory) deserve special consideration with respect to sufficient evidence for review.
If randomized or well-controlled trials are available, exclude less-informative studies (case reports, studies with small numbers of patients [<20], indications with inadequate follow-up, reports with no statistical analysis, noncomparative studies, studies too old to be of relevance) and note why.
In determining the number of available studies, discount those in which patients or investigators overlap. This includes post-hoc analyses, which are after-the-fact evaluations of study results that should be interpreted with caution and clearly identified as such.
Assess individual study quality
Base your R-HTA on a review of full texts, not just the abstracts. Organize studies by design quality, with RCTs first, followed by nonrandomized studies (pretest/posttest studies in which patients’ baseline and posttreatment measurements serve as their own controls), case-control studies, and database analyses with comparison groups. Prospective studies take precedence over retrospective. The lowest levels of evidence include descriptive, nonanalytical, and uncontrolled studies, such as case reports and case series without treatment analysis.
Outcomes must be statistically as well as clinically significant. For example, a statistically significant reduction in blood pressure of just a few points may not be clinically significant. Also look for a minimally clinically important difference (MCID), which is the minimal improvement in the outcome that a patient or clinician would find important. These should be validated in a relevant indication, well-accepted, and ideally predefined in the study protocol. For example, a reduction of 30% is a typical MCID for a pain outcome. An MCID may be challenging to define when there’s an unmet need and no known effective treatment (for example, ocrelizumab for treating primary progressive multiple sclerosis). (See Study quality assessment.)
Study quality assessment
Ask these questions when assessing the quality of studies included in a health technology assessment.
· Does it represent the patients the technology is intended to treat?
|· Pediatric and elderly patients may be excluded from clinical trials of technologies these groups may be exposed to.
· Patients with milder and more severe forms of a disease frequently are excluded from controlled studies.
Ocrelizumab was approved to treat relapsing remitting multiple sclerosis (RRMS) but also, on the basis of a single randomized clinical trial, primary progressive MS (PPMS). PPMS is more debilitating than RRMS and also is an unmet need. It was controversial that the results for RRMS were extrapolated to patients with PPMS.
· Was a power calculation to determine the number of patients necessary to show a treatment effect performed?
· Was the number of patients in the sample size reasonable for the indication?
|· Despite the demonstration of an adequate power calculation, review studies by indication for sample size.
Many cardiovascular indications are so common that hundreds to even thousands of patients would be a reasonable expectation of sample size. Conversely, studies of rare diseases with about 20 patients may be appropriate.
· If a benefit was shown numerically but wasn’t statistically significant, the sample size may have been too small.
|Choice of comparator
· Is the comparator the most appropriate for the indication?
|· Know the standard of care for the indication.
· When treatment alternatives are available, question whether a placebo is less appropriate than an active control. The choice of a placebo comparator also may limit the treatment period due to ethical considerations.
· Historical controls count as comparators but are of lower quality than contemporaneous controls, especially when markedly remote.
· Who was blinded, and was there potential for unblinding?
|· Blinding is difficult-to-impossible for some interventions, and the investigators shouldn’t be criticized unfairly. Look for any attempts to mitigate the issue.
Patients may need to be unblinded to manage adverse events.
· Are the outcome measures patient-centered, standard, well-accepted, and validated?
|· Patient-centered outcomes include mortality, morbidity, pain measures, function, and quality-of-life. Downgrade study quality for nonvalidated outcome measures.
· Any surrogate endpoint should be highly correlated with the clinical endpoint of interest (for example, hypertension for stroke risk).
Biomarkers and imaging results are surrogate endpoints that may be considered of limited relevance alone.
· Are the outcomes statistically and clinically significant?
|· If an outcome is statistically significant, it also must be clinically significant.|
· Is the follow-up period appropriate for the indication and technology?
|· The follow-up period should be sufficient to study durability of the effect and detect any late morbidity.
· A 6-month to 1-year follow-up wouldn’t be adequate for most orthopedic implants and could miss revision surgeries.
· The carcinogenic potential of a drug requires years of surveillance.
· Is the dropout rate acceptable?
|· A dropout rate of ≤20% with ≤10% difference between groups is acceptable.|
|Conflicts of interest
· Do any conflicts of interest exist?
|· Studies performed before regulatory approval or clearance of a drug or device are expected to be sponsored by the manufacturer.
· Downgrade study quality for post-approval studies without independent funding or with investigator conflicts of interest.
Assess results for each study and then overall. This is the most challenging part of an R-HTA. Form your own conclusions, not those of the study authors, a regulatory agency, guidelines, or position statements.
Look for consistency and reproducibility in efficacy outcomes across the included studies. Focus on the primary endpoint with consideration of secondary outcomes and look for any concerning safety signals. Calculations of how many patients are needed in a study are based on efficacy outcomes, so a reported statistical significance for safety isn’t necessarily valid.
Keep your summary of efficacy and safety at a high level. For example: “Overall, the review of published, peer-reviewed literature located a moderate-sized body of evidence evaluating the use of X (biologic/drug/device/procedure) for the treatment of Y (disease/disorder). Ten studies were located, including three RCTs, one noncomparative prospective study, and one noncomparative retrospective study. An MCID of 30% improvement in the Visual Analogue Scale for pain was prespecified. The primary endpoint of pain reduction was statistically significant, and MCID improvements across studies ranged from 20% to 35%. There were no concerning safety signals. No cost or cost effectiveness data were located.”
Your report should be candid about study limitations. For example: “Overall, the body of evidence for the use of X (drug/device/procedure) for the treatment of Y (disease/disorder) was limited by a relatively small number of studies, a relatively small number of patients given the prevalence of X (indication), inadequate follow-up that was inappropriate for the indication, and manufacturer involvement.”
Potential conclusions for an R-HTA include:
- Results evaluating X for the treatment of Y in Z population are conflicting. It’s not possible to make a recommendation without further studies.
- Results evaluating X for the treatment of Y in Z population suggest a potential benefit without overly concerning safety signals. However, limitations of the body of evidence suggest that further study is needed.
Various schema, such as Cochrane, are available for grading the level of evidence.
Clinical studies in progress may indicate if gaps in the evidence are under investigation. They should be included in your analysis. You can find U.S. studies in progress at clinicaltrials.gov and European studies at clinicaltrialsregister.eu.
R-HTAs—focused evidence-based systematic reviews of biologicals, drugs, devices, and procedures—are the basis for value-chain decisions, payer determinations, and guidelines. When nurses, especially those in value chain positions, and APRNs understand the R-HTA process they can make informed decisions about the technologies used in their organizations. (See Resources and databases.)
Resources and databases
Writing rapid health technology assessments (R-HTAs) can be a rewarding career path. Strong writing and analytical skills, as well as knowledge of the fundamentals of statistics, are required. The following resources and databases can aid your work in this field and support your analysis of R-HTAs*.
- Centers for Disease Control and Prevention Guidelines and Guidance Library: gov/infectioncontrol/guidelines/index.html
- Cochrane Database of Systematic Reviews: cochranelibrary.com/cdsr/about-cdsr
- Health Quality Ontario Health Technology Assessment Series: ca/Evidence-to-Improve-Care/Health-Technology-Assessment/Journal-Ontario-Health-Technology-Assessment-Series
- National Information Center on Health Services Research and Health Care Technology: nlm.nih.gov/hsrph.html
- National Institute for Health and Care Excellence (NICE) guidance: org.uk/guidance
Open access and subscription-based databases are available to aid your search and analysis.
- Centers for Medicare & Medicaid Services Medicare Database: gov/medicare-coverage-database/search/advanced-search.aspx
- Cumulative Index of Nursing and Allied Health Literature (with an American Nurses Association membership): ebsco.com/products/research-databases/cinahl-database
- Food & Drug Administration premarket approval database: www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfPMA/pma.cfm
- Google Scholar: scholar.google.com
- National Library of Medicine clinical trials database: clincaltrials.gov
- Open Access: sparcopen.org/open-access
- PubMed Central: ncbi.nlm.nih.gov/pmc
- S. Food & Drug Administration 510(k) Premarket Notification database: https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfPMN/pmn.cfm
- S. Food & Drug Administration new drug application database: www.accessdata.fda.gov/scripts/cder/daf/
- Clinical Key: clinicalkey.com/#!
Kathleen K. Brewer is a contract medical research analyst at Hayes, Inc. (a symplr Company) in Dallas, Texas.
Hatswell AJ, Baio G, Berlin JA, Irs A, Freematle N. Regulatory approval of pharmaceuticals without a randomised controlled study: Analysis of EMA and FDA approvals 1999–2014. BMJ Open. 2016;6:e011666. doi: 10.1136/bmjopen-2016-011666
Lieberman JA, Davis RE, Correll CU, et al. ITI-007 for the treatment of schizophrenia: A 4-week randomized, double-blind, controlled trial. Biol Psychiatry. 2016;79(12):952-61. doi: 10.1016/j.biopsych.2015.08.026
Montalban X, Hauser SL, Kappos L, et al. Ocrelizumab versus placebo in primary progressive multiple sclerosis. New Engl J Med. 2017;376:209-20. doi: 10.1056/NEJMoa1606468
National Information Center on Health Services Research and Health Care Technology (NICHSR). HTA 101: Introduction to health technology assessment. July 22, 2019. nlm.nih.gov/nichsr/hta101/ta10103.html
World Health Organization. Institutionalizing health technology assessment mechanisms: A how to guide. March 7, 2021. who.int/publications/i/item/9789240020665