Population-adjusted indirect comparisons for HTA: MAIC, STC and ML-NMR

The problem population adjustment tries to solve

When two treatments have not been compared head to head, the standard approach is an anchored indirect comparison through a common comparator. Bucher and colleagues set out the basic method in 1997: if trial AB compares A with B and trial AC compares A with C, the relative effect of C versus B is estimated from the difference between the two relative effects against A, preserving randomisation within each trial.^[1] Network meta-analysis generalises the same logic to larger networks.

These methods assume that relative effects would be the same across the trial populations. That assumption fails when the trials differ in the distribution of effect modifiers, characteristics that change the relative effect of one treatment versus another. NICE's methods manual notes that indirect comparisons in small networks are highly prone to systematic bias, and that population adjustment can be considered where effect modifiers are imbalanced, provided individual patient data (IPD) are available from at least one trial and the limitations and likely size of any bias are acknowledged.^[2]

Population-adjusted indirect comparisons use IPD from one or more trials to adjust for differences in the covariates of trials for which only aggregate data are available. The three methods most often seen in submissions are matching-adjusted indirect comparison (MAIC), simulated treatment comparison (STC) and multilevel network meta-regression (ML-NMR).

Anchored and unanchored comparisons

The single most important distinction is whether a comparison is anchored. An anchored comparison uses a common comparator arm and compares relative effects, so randomisation within each trial is respected. An unanchored comparison has no common comparator, typically because the evidence for one or both treatments comes from single-arm studies, and compares absolute outcomes across trials.

The HTA Coordination Group's methodological guideline on direct and indirect comparisons, adopted on 8 March 2024, is unambiguous. Anchored indirect comparisons preserve randomisation and should always be used in preference to unanchored methods. Unanchored MAIC and STC rely on the assumption of conditional constancy of absolute effects, meaning that absolute outcomes are assumed constant at any given level of the prognostic variables and effect modifiers, and the guideline states that in almost all practical applications this assumption is not justifiable.^[3] NICE DSU Technical Support Document (TSD) 18 takes the same starting point: where a connected network with a common comparator exists, only anchored forms of population adjustment may be used, and unanchored adjustment is reserved for disconnected networks or single-arm evidence.^[4]

The companion practical guideline for JCA assessors goes further, stating that by describing these methods it is not endorsing them, and that estimates from population adjustment in unanchored comparisons are unreliable.^[5] Where unanchored comparisons are needed to answer a PICO question, it states that methods for non-randomised data using full IPD are generally preferable to unanchored MAIC or STC.^[5]

Why the stakes differ

In an anchored comparison, adjustment needs to account only for covariates that change the relative effect. In an unanchored comparison, it must account for everything that affects the absolute outcome, observed or not. That is a much larger and usually unverifiable set.

Effect modifiers, prognostic factors and the scale of comparison

The HTA Coordination Group defines prognostic variables as patient characteristics that affect the outcome irrespective of which treatment is received, and effect modifiers as characteristics that alter the relative effectiveness between two treatments. It stresses that effect modifiers are specific to the pair of treatments being compared and to the scale on which the relative effect is measured.^[3]

Scale dependence is easy to state and easy to forget. The practical guideline gives a hypothetical vaccine example: risk falls from 10% to 5% in women and from 6% to 3% in men.^[5] The arithmetic shows why the choice of scale matters:

The risk ratio and risk difference rows reproduce the guideline example; the odds ratio row is our own calculation. TSD 18 recommends that indirect comparisons be carried out on the usual linear predictor scale for the outcome, for example the logit scale for binary outcomes, rather than the natural outcome scale.^[4] The practical guideline adds that an outcome regression model must estimate effects on the same scale as the indirect comparison, and that it is not appropriate to use logistic regression to adjust absolute risks and then compare on the risk difference or log risk ratio scale.^[5]

Which covariates to adjust for

TSD 18 gives clear rules. In anchored comparisons using propensity-score-type weighting such as MAIC, adjust for all effect modifiers but not for purely prognostic variables, which inflate the standard error without reducing bias. In anchored outcome regression such as STC, adjust for effect modifiers that are in imbalance, adding other variables only if they improve model fit. Unanchored comparisons should adjust for all effect modifiers and prognostic variables.^[4] The practical guideline adds that covariates already balanced at baseline should not be dropped, because adjustment could create imbalance where none existed, and that significance-based selection is of limited use when IPD are scarce.^[5]

How the candidate list is built now matters as much as the list itself. The HTA Coordination Group has stated that a systematic literature review is required to identify covariates such as prognostic factors, and that an existing review of prognostic factors and effect modifiers is acceptable only if it is of sufficient quality and timeliness.^[6]

How MAIC, STC and ML-NMR differ

Matching-adjusted indirect comparison

MAIC, introduced by Signorovitch and colleagues, reweights patients in the IPD trial so that their baseline characteristics match the averages reported for the aggregate-data trial, using weights based on an estimated propensity of trial membership.^[7] TSD 18 describes the weights as log-linear in the covariates and estimated by a method of moments, because a standard logistic regression cannot be fitted without IPD from both trials.^[4] The result is a marginal, population-average effect in the aggregate-data trial population. The HTA Coordination Group notes that MAIC is valid only where the populations overlap sufficiently, and that it can only produce a comparison in the population of the aggregate-data study, which may not match the decision population.^[3]

Simulated treatment comparison

STC fits an outcome regression model to the IPD and uses it to predict outcomes in the aggregate-data population. Its validity depends on correct specification of that model. The HTA Coordination Group points out that when the model uses a non-identity link, the common practice of plugging in mean covariate values mixes conditional and marginal effects and biases both estimands; approaches targeting marginal effects exist but are not yet widely adopted.^[3] The practical guideline asks that the JCA report state which STC approach was used and which estimand it targets.^[5]

Multilevel network meta-regression

ML-NMR, developed by Phillippo and colleagues and published in the Journal of the Royal Statistical Society Series A in 2020, extends network meta-analysis. It defines an individual-level regression model, fits the IPD directly, and fits the aggregate data by integrating that model over each study's covariate distribution, using quasi-Monte Carlo integration and copulae to handle correlated covariates. This avoids the aggregation bias of conventional meta-regression, works in networks of any size, and can produce estimates in any target population with a known covariate distribution.^[8]

The HTA Coordination Group describes ML-NMR as a more general framework of which full IPD meta-analysis, STC and aggregate NMA are special cases. It also sets out its data demands: either full IPD for at least one study of each treatment, enough aggregate studies to estimate the interactions, a shared effect modifier assumption, or informative priors on treatment-covariate interactions.^[3] The 2024 guideline stated that ML-NMR as then proposed could not be applied to time-to-event outcomes.^{[3, 5]} Phillippo and colleagues have since published an extension to general likelihoods, including survival outcomes with flexible baseline hazards, implemented in the multinma R package.^[9] How quickly assessors will accept the extension in practice remains to be seen.

Effective sample size: the overlap diagnostic

Weighting in MAIC has a price. The approximate effective sample size (ESS), given in TSD 18, is:^[4]

ESS = (sum of weights)^2 / (sum of squared weights)

A worked example shows how quickly it falls. Suppose five IPD patients receive weights of 0.2, 0.5, 1.0, 1.5 and 2.8. The sum of weights is 6.0, so the numerator is 36.0. The sum of squared weights is 0.04 + 0.25 + 1.00 + 2.25 + 7.84 = 11.38. The ESS is 36.0 / 11.38 = 3.2, a 37% reduction from five patients, driven largely by one patient with a weight of 2.8. Real analyses behave the same way at larger scale: a few heavily weighted patients dominate the estimate.

TSD 18 notes that small ESS values indicate highly variable weights, and that the approximation is accurate only if the weights are treated as fixed and known; the distribution of weights should be examined directly as well.^[4] A review of NICE technology appraisals published between 2010 and April 2018 found population adjustment in 18 of 268 appraisals, of which 16 were unanchored. Only 9 of the 16 MAICs reported an ESS; among those, the median reduction from the original sample size was 74.2%, which the authors interpreted as a sign of poor overlap in many cases.^[10]

The JCA practical guideline asks for the distribution of weights and the ESS to be reported, notes that large ESS reductions mean the target population of the MAIC differs considerably from the IPD trial population, and requires confidence intervals that reflect the reweighting, for example through robust standard errors or bootstrapping.^[5]

What assessors look for

Agency guidance and published assessments point to a consistent set of questions. In our experience, most critiques of population-adjusted comparisons fall into one of the following.

• Is adjustment justified? TSD 18 asks for evidence that a covariate is plausibly an effect modifier and that its imbalance could cause material bias.^[4] The practical guideline asks for the same in anchored comparisons.^[5]
• Was the covariate list built transparently? In the published JCA report on tovorafenib, which assessed an unanchored MAIC, the assessors concluded that because the identification process was not transparently reported, they could not establish whether the list of prognostic and effect-modifying variables was complete.^[11]
• Could all relevant covariates actually be adjusted for? In the same report, several variables identified as likely prognostic or effect-modifying could not be included because they were not reported for the comparator study, and the assessors concluded this led to bias of unknown magnitude and direction.^[11]
• How good is the overlap? The tovorafenib assessors noted an ESS reduction from 12 to 5.81 in one analysis set and judged that a single highly weighted patient was likely to be very influential.^[11]
• Was the analysis prespecified? The practical guideline highlights the multiplicity created by choices of method and covariates, calls these methods often more suitable as exploratory than primary analyses, and expects model selection to be prespecified in the protocol and statistical analysis plan.^[5]
• Whose population is the estimate for? MAIC and STC estimate effects in the aggregate-data trial population. The practical guideline notes that in a JCA the IPD trial population is likely to be of greater interest.^[5] TSD 18 requires the target population for the decision to be stated explicitly.^[4]
• Is the effect large enough to survive residual bias? The HTA Coordination Group describes testing against a shifted null hypothesis, so that a conclusion is drawn only if the confidence interval excludes a threshold set away from zero.^[3] The practical guideline leaves the decision threshold to individual Member States.^[5]

National appraisal committees have reached similar conclusions. The review of NICE appraisals found that committees received population-adjusted results with caution and typically looked for greater cost effectiveness to offset the decision risk.^[10]

A decision framework for choosing a method

The table below summarises how we would approach the choice. It reflects the guidance cited above combined with our interpretation; the right answer always depends on the network, the data and the scope.

Two principles sit behind the table. First, the practical guideline states that MAIC and STC should not be used when full IPD are available for all studies.^[5] Second, TSD 18 requires unanchored analyses to provide evidence on the likely extent of error from unaccounted-for covariates, relative to the size of the observed effect.^[4] An unanchored MAIC submitted without that evidence is unlikely to carry weight.

Our view

Population adjustment is most persuasive as a check on a well-built anchored analysis, not as a substitute for one. Where the only option is unanchored, the submission should say so plainly and invest in quantifying the residual bias rather than in additional model variants.

What this means for evidence planning

Most problems with population-adjusted comparisons are created well before the analysis is run. In our experience, the following steps make the largest difference.

1. Decide early whether an anchored network is achievable: Trial design choices, including the control arm, determine whether later comparisons can be anchored. Joint scientific advice is the place to test this.
2. Run a systematic review of effect modifiers and prognostic factors before the statistical analysis plan is written, combining literature, trial subgroup data and documented clinical input. The JCA expects a systematic review for covariate identification.^[6]
3. Prespecify methods, covariates and the target population in a statistical analysis plan, including the rule for choosing between MAIC and STC and any shifted null hypothesis.
4. Check comparator reporting before committing to a method: If key covariates are not reported for the comparator trial, adjustment for them is impossible, and abstract-only comparator data may not be accepted at all.^[11]
5. Plan IPD access: ML-NMR becomes more feasible with IPD from more than one trial, which may require data-sharing agreements made well before submission.
6. Report the diagnostics in full: Weight distributions, ESS, covariate balance after weighting, model fit and overlap plots, with confidence intervals that reflect the adjustment.
7. Define the target population for each market: The population a JCA cares about may differ from those of NICE or other national agencies, and the choice of method should follow from that.

Phillippo and colleagues warned in 2018 that MAIC and STC remained largely untested and that assumptions varied widely between submissions for the same indication.^[12] Methods and guidance have advanced considerably since, but the central point stands: the credibility of a population-adjusted comparison depends far more on the evidence plan behind it than on the choice of software.

Evidaxia designs evidence synthesis strategies for HTA, from systematic reviews of effect modifiers to anchored network meta-analyses and population-adjusted comparisons, and supports companies in presenting them to NICE and in EU JCA dossiers. See systematic literature reviews, EU JCA dossier support and health economics and evidence.

References

1. Bucher HC, Guyatt GH, Griffith LE, Walter SD. The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. J Clin Epidemiol. 1997;50(6):683-91. Available from: https://experts.mcmaster.ca/scholarly-works/1442611
2. National Institute for Health and Care Excellence. NICE health technology evaluations: the manual (PMG36), chapter 3: Evidence [Internet]. London: National Institute for Health and Care Excellence; [date unknown] [cited 2026 Oct]. Available from: https://www.nice.org.uk/process/pmg36/chapter/evidence
3. Member State Coordination Group on HTA. Methodological Guideline for Quantitative Evidence Synthesis: Direct and Indirect Comparisons [Internet]. Brussels: European Commission; 2024 Mar 8 [cited 2026 Oct 30]. Available from: https://health.ec.europa.eu/document/download/4ec8288e-6d15-49c5-a490-d8ad7748578f_en?filename=hta_methodological-guideline_direct-indirect-comparisons_en.pdf
4. Phillippo DM, Ades AE, Dias S, Palmer S, Abrams KR, Welton NJ. NICE DSU Technical Support Document 18: Methods for population-adjusted indirect comparisons in submissions to NICE [Internet]. Sheffield: NICE Decision Support Unit; 2016 [cited 2026 Oct 30]. Available from: https://sheffield.ac.uk/nice-dsu/tsds/population-adjusted
5. Member State Coordination Group on HTA. Practical Guideline for Quantitative Evidence Synthesis: Direct and Indirect Comparisons [Internet]. Brussels: European Commission; 2024 Mar 8 [cited 2026 Oct 30]. Available from: https://health.ec.europa.eu/document/download/1f6b8a70-5ce0-404e-9066-120dc9a8df75_en?filename=hta_practical-guideline_direct-and-indirect-comparisons_en.pdf
6. Member State Coordination Group on HTA. Questions & Answers on general methodological and procedural issues for joint clinical assessments [Internet]. Brussels: European Commission; 2026 May 15 [cited 2026 Oct 30]. Available from: https://health.ec.europa.eu/document/download/b89dc548-10d7-4ef7-9c3b-deea90b639ca_en?filename=hta_20260518_QA_methodological_procedural_questions_HTACG_en.pdf
7. Signorovitch JE, Wu EQ, Yu AP, et al. Comparative effectiveness without head-to-head trials: a method for matching-adjusted indirect comparisons applied to psoriasis treatment with adalimumab or etanercept. PharmacoEconomics. 2010;28(10):935-45. Available from: https://econpapers.repec.org/RePEc:spr:pharme:v:28:y:2010:i:10:p:935-945
8. Phillippo DM, Dias S, Ades AE, et al. Multilevel network meta-regression for population-adjusted treatment comparisons. J R Stat Soc Ser A. 2020;183(3):1189-210. Available from: https://research-information.bris.ac.uk/en/publications/multilevel-network-meta-regression-for-population-adjusted-treatm/
9. Phillippo DM, Dias S, Ades AE, Welton NJ. Multilevel network meta-regression for general likelihoods: synthesis of individual and aggregate data with applications to survival analysis. J R Stat Soc Ser A. 2026;189(3):1856-75. Available from: https://research-information.bris.ac.uk/en/publications/multilevel-network-meta-regression-for-general-likelihoods-synthe/
10. Phillippo DM, Dias S, Elsada A, Ades AE, Welton NJ. Population adjustment methods for indirect comparisons: a review of National Institute for Health and Care Excellence technology appraisals. Int J Technol Assess Health Care. 2019;35(3):221-8. Available from: https://eprints.whiterose.ac.uk/id/eprint/145561/
11. Member State Coordination Group on HTA. Joint Clinical Assessment Report: Tovorafenib, version 1.0 [Internet]. Brussels: European Commission; 2026 [cited 2026 Oct 30]. Available from: https://health.ec.europa.eu/publications/joint-clinical-assessment-report-tovorafenib-ojemda_en
12. Phillippo DM, Ades AE, Dias S, Palmer S, Abrams KR, Welton NJ. Methods for population-adjusted indirect comparisons in health technology appraisal. Med Decis Making. 2018;38(2):200-11. Available from: https://ideas.repec.org/a/sae/medema/v38y2018i2p200-211.html

Sources accessed October 2026. This article is analysis for information and is not legal, regulatory or tax advice.