CatContinuous < Main < Vanderbilt Biostatistics Wiki

---++ Problems Caused by Categorizing Continuous Variables | [[http://hbiostat.org/bib/dichotomania.html][References]] | [[https://onlinelibrary.wiley.com/doi/abs/10.1002/sim.2331][Key Reference]] | [[https://onlinelibrary.wiley.com/doi/abs/10.1002/pst.331][Key Reference for Response Variables]] | [[https://twitter.com/GSCollins/status/1096401368162684929][More References]] | [[https://bmcmedicine.biomedcentral.com/articles/10.1186/s12916-019-1425-3][Myths About Risk Thresholds]] | [[https://www.methodsappraisal.com/education-dichotomania Dichotomania]]

*A newer version of this page may be found at https://discourse.datamethods.org/t/categorizing-continuous-variables*

   1 Optimum decisions are made by applying a utility function to a predicted value (e.g., predicted risk).  At the decision point, one can solve for the personalized cutpoint for predicted risk that optimizes the decision.  Dichotomization on independent variables is completely at odds with making optimal decisions.  To make an optimal decision, the cutpoint for a predictor would necessarily be a function of the continuous values of all the other predictors, as shown [[http://fharrell.com/doc/bbr.pdf][here]] in Section 18.3.1.
   1 Loss of power and loss of precision of estimated means, odds, hazards, etc.  Dichotomization of a predictor requires the researcher to add a new predictor to the mix to make up for the lost information.
   1 Categorization assumes that the relationship between the predictor and the response is flat within intervals; this assumption is far less reasonable than a linearity assumption in most cases
   1 Researchers seldom agree on the choice of cutpoint, thus there is a severe interpretation problem.  One study may provide an odds ratio for comparing BMI > 30 with BMI <= 30, another for comparing BMI > 28 with BMI <= 28.  Neither of these has a good definition and they have different meanings.
   1 Categorization of continuous variables using percentiles is particularly hazardous.  The percentiles are usually estimated from the data at hand, are estimated with sampling error, and do not relate to percentiles of the same variable in a population.  Percentiling a variable is declaring to readers that how similar a person is to other persons is as important as how the physical characteristics of the measurement predict outcomes.  For example, it is common to group the  continuous variable BMI into quantile intervals.  BMI has a smooth relationship with every outcome studied, and relates to outcome according to anatomy and physiology and not according to how many subjects have a similar BMI.
   1 To make a continuous predictor be more accurately modeled when categorization is used, multiple intervals are required.  The needed dummy variables will spend more degrees of freedom than will fitting a smooth relationship, hence power and precision will suffer.  And because of sample size limitations in the very low and very high range of the variable, the outer intervals (e.g., outer quintiles) will be wide, resulting in significant heterogeneity of subjects within those intervals, and residual confounding.
   1 Categorization assumes that there is a discontinuity in response as interval boundaries are crossed
   1 Categorization only seems to yield interpretable estimates such as odds ratios.  For example, suppose one computes the odds ratio for stroke for persons with a systolic blood pressure > 160 mmHg compared to persons with a blood pressure <= 160 mmHg.  The interpretation of the resulting odds ratio will depend on the exact distribution of blood pressures in the sample (the proportion of subjects > 170, > 180, etc.).  On the other hand, if blood pressure is modeled as a continuous variable (e.g., using a regression spline, quadratic, or linear effect) one can estimate the ratio of odds for *exact* settings of the predictor, e.g., the odds ratio for 200 mmHg compared to 120 mmHg.
   1 When the risk of stroke is being assessed for a new subject with a known blood pressure (say 162), the subject does not report to her physician "my blood pressure exceeds 160" but rather reports 162 mmHg.  The risk for this subject will be much lower than that of a subject with a blood pressure of 200 mmHg.
   1 If cutpoints are determined in a way that is not blinded to the response variable, calculation of _P_ -values and confidence intervals requires special simulation techniques; ordinary inferential methods are completely invalid.  For example, if cutpoints are chosen by trial and error in a way that utilizes the response, even informally, ordinary _P_ -values will be too small and confidence intervals will not have the claimed coverage probabilities.  The correct Monte-Carlo simulations must take into account both multiplicities and uncertainty in the choice of cutpoints.  For example, if a cutpoint is chosen that minimizes the _P_ -value and the resulting _P_ -value is 0.05, the true type I error can easily be above 0.5; see [[https://onlinelibrary.wiley.com/doi/abs/10.1002/sim.1611][here]]
   1 Likewise, categorization that is not blinded to the response variable results in biased effect estimates (see [[https://academic.oup.com/jnci/article-abstract/86/11/829/955201?redirectedFrom=fulltext][this]] and [[http://repository.essex.ac.uk/2513/][this]])
   1 "Optimal" cutpoints do not replicate over studies.  Hollander, Sauerbrei, and Schumacher (see [[https://onlinelibrary.wiley.com/doi/abs/10.1002/sim.1611][here]]) state that "... the optimal cutpoint approach has disadvantages.  One of these is that in almost every study where this method is applied, another cutpoint will emerge. This makes comparisons across studies extremely difficult or even impossible.  Altman et al. point out this problem for studies of the prognostic relevance of the S-phase fraction in breast cancer published in the literature.  They identified 19 different cutpoints used in the literature; some of them were solely used because they emerged as the `optimal' cutpoint in a specific data set. In a meta-analysis on the relationship between cathepsin-D content and disease-free survival in node-negative breast cancer patients, 12 studies were in included with 12 different cutpoints ... Interestingly, neither cathepsin-D nor the S-phase fraction are recommended to be used as prognostic markers in breast cancer in the recent update of the American Society of Clinical Oncology."
   1 Cutpoints are arbitrary and manipulatable; cutpoints can be found that can result in both positive and negative associations (see [[https://www.tandfonline.com/doi/abs/10.1080/09332480.2006.10722771][this]])
   1 If a confounder is adjusted for by categorization, there will be residual confounding that can be explained away by inclusion of the continuous form of the predictor in the model in addition to the categories.
   1 A better approach that maximizes power and that only assumes a smooth relationship is to use a restricted cubic spline (regression spline; piecewise cubic polynomial) function for predictors that are not known to predict linearly.  Use of flexible parametric approaches such as this allows standard inference techniques (_P_ -values, confidence limits) to be used

---+++ Papers and Links
   * [[https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/1471-2288-12-21][Bad use of quantiles in epidemiology]]
   * [[http://methodsappraisal.com/education-dichotomania][Methods appraisal: dichotomania]]

---+++ Interactive Demonstrations
   * Run under RStudio: =require(Hmisc); getRs('catgNoise.r')= or watch [[https://youtu.be/Q6snkhhTPnQ this]] video
   * [[http://psych.colorado.edu/~mcclella/MedianSplit Java applet]] demonstrating the information loss from dichotomizing a continuous variable
   * [[http://www.bolderstats.com/jmsl/doc/medianSplit.html Median split]] demo
   * [[https://youtu.be/-GEgR71KtwI Hazards of categorizing continuous predictors]] video
--------
-- Main.FrankHarrell - 27 Jun 2004, 21 Jun 2006, 25 Sep 2007, 7 Jan 2008, 15 Aug 2010, 16 Dec 2010, 13 Mar 2014, 7 Jul 2015, 18 Mar 2017, 8 Apr 2018, 10 Apr 2018, 15 Feb 2019, 1 Dec 2019, 16 Dec 2019, 4 June 2020
This topic: Main > WebHome > Education > HandoutsBioRes > BioMod > CatContinuous
Topic revision: 04 Jun 2020, FrankHarrell (raw view)
Copyright © 2013-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Vanderbilt Biostatistics Wiki? Send feedback