[Skip to Content]
[Skip to Content Landing]
Figure 1.  Receiver Operating Characteristic Curve for Safe Discharge
Receiver Operating Characteristic Curve for Safe Discharge

ROC indicates receiver operating characteristic.

Figure 2.  Proportion of Patients Meeting Criteria for Safe Discharge by Total Oakland Score
Proportion of Patients Meeting Criteria for Safe Discharge by Total Oakland Score
Table 1.  Oakland Score Variables
Oakland Score Variables
Table 2.  Demographic Characteristics and Presenting Features of Patients Admitted to Hospital With Acute LGIB
Demographic Characteristics and Presenting Features of Patients Admitted to Hospital With Acute LGIB
Table 3.  Adverse Outcomes Among Patients With Low-Risk Oakland Scores
Adverse Outcomes Among Patients With Low-Risk Oakland Scores
1.
Oakland  K, Guy  R, Uberoi  R,  et al.  Acute lower GI bleeding in the UK: patient characteristics, interventions and outcomes in the first nationwide audit.   Gut. 2018;67(4):654-662. PubMedGoogle Scholar
2.
Rockall  TA, Logan  RF, Devlin  HB, Northfield  TC.  Risk assessment after acute upper gastrointestinal haemorrhage.   Gut. 1996;38(3):316-321. doi:10.1136/gut.38.3.316 PubMedGoogle ScholarCrossref
3.
Blatchford  O, Murray  WR, Blatchford  M.  A risk score to predict need for treatment for upper-gastrointestinal haemorrhage.   Lancet. 2000;356(9238):1318-1321. doi:10.1016/S0140-6736(00)02816-6 PubMedGoogle ScholarCrossref
4.
Strate  LL, Gralnek  IM.  ACG clinical guideline: management of patients with acute lower gastrointestinal bleeding.   Am J Gastroenterol. 2016;111(5):755. doi:10.1038/ajg.2016.155 PubMedGoogle ScholarCrossref
5.
Oakland  K, Jairath  V, Uberoi  R,  et al.  Derivation and validation of a novel risk score for safe discharge after acute lower gastrointestinal bleeding: a modelling study.   Lancet Gastroenterol Hepatol. 2017;2(9):635-643. doi:10.1016/S2468-1253(17)30150-4 PubMedGoogle ScholarCrossref
6.
Tapaskar  N, Jones  B, Mei  S, Sengupta  N.  Comparison of clinical prediction tools and identification of risk factors for adverse outcomes in acute lower GI bleeding.   Gastrointest Endosc. 2019;89(5):1005-1013. doi:10.1016/j.gie.2018.12.011 PubMedGoogle Scholar
7.
Oakland  K, Chadwick  G, East  JE,  et al.  Diagnosis and management of acute lower gastrointestinal bleeding: guidelines from the British Society of Gastroenterology.   Gut. 2019;68(5):776-789. doi:10.1136/gutjnl-2018-317807 PubMedGoogle ScholarCrossref
8.
Collins  GS, Reitsma  JB, Altman  DG, Moons  KGM.  Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement.   Ann Intern Med. 2015;162(1):55-63. doi:10.7326/M14-0697 PubMedGoogle ScholarCrossref
9.
World Health Organization.  International Statistical Classification of Diseases and Related Health Problems, Tenth Revision. World Health Organization; 1992.
10.
Strate  LL, Orav  EJ, Syngal  S.  Early predictors of severity in acute lower intestinal tract bleeding.   Arch Intern Med. 2003;163(7):838-843. doi:10.1001/archinte.163.7.838 PubMedGoogle ScholarCrossref
11.
Barkun  AN, Almadi  M, Kuipers  EJ,  et al.  Management of nonvariceal upper gastrointestinal bleeding: guideline recommendations from the International Consensus Group.   Ann Intern Med. 2019;171(11):805-822. doi:10.7326/M19-1795PubMedGoogle ScholarCrossref
12.
Stanley  AJ, Laine  L, Dalton  HR,  et al; International Gastrointestinal Bleeding Consortium.  Comparison of risk scoring systems for patients presenting with upper gastrointestinal bleeding: international multicentre prospective study.   BMJ. 2017;356:i6432. doi:10.1136/bmj.i6432 PubMedGoogle ScholarCrossref
13.
Lanas  A, Garcia-Rodriguez  LA, Polo-Tomas  M,  et al.  Time trends and impact of upper and lower gastrointestinal bleeding and perforation in clinical practice.   Am J Gastroenterol. 2009;104(7):1633-1641. doi:10.1038/ajg.2009.164 PubMedGoogle ScholarCrossref
14.
Ron-Tal Fisher  O, Gralnek  IM, Eisen  GM, Williams  JL, Holub  JL.  Endoscopic hemostasis is rarely used for hematochezia: a population-based study from the Clinical Outcomes Research Initiative National Endoscopic Database.   Gastrointest Endosc. 2014;79(2):317-325. doi:10.1016/j.gie.2013.09.004 PubMedGoogle ScholarCrossref
15.
Navaneethan  U, Njei  B, Venkatesh  PG, Sanaka  MR.  Timing of colonoscopy and outcomes in patients with lower GI bleeding: a nationwide population-based study.   Gastrointest Endosc. 2014;79(2):297-306. doi:10.1016/j.gie.2013.08.001 PubMedGoogle ScholarCrossref
16.
Strate  LL, Ayanian  JZ, Kotler  G, Syngal  S.  Risk factors for mortality in lower intestinal bleeding.   Clin Gastroenterol Hepatol. 2008;6(9):1004-1010. doi:10.1016/j.cgh.2008.03.021PubMedGoogle ScholarCrossref
17.
Ng  K-S, Nassar  N, Soares  D, Stewart  P, Gladman  MA.  Acute lower gastrointestinal haemorrhage: outcomes and risk factors for intervention in 949 emergency cases.   Int J Colorectal Dis. 2017;32(9):1327-1335. doi:10.1007/s00384-017-2844-2 PubMedGoogle ScholarCrossref
18.
Kollef  MH, O’Brien  JD, Zuckerman  GR, Shannon  W.  BLEED: a classification tool to predict outcomes in patients with acute upper and lower gastrointestinal hemorrhage.   Crit Care Med. 1997;25(7):1125-1132. doi:10.1097/00003246-199707000-00011 PubMedGoogle ScholarCrossref
19.
Aoki  T, Nagata  N, Shimbo  T,  et al.  Development and validation of a risk scoring system for severe acute lower gastrointestinal bleeding.   Clin Gastroenterol Hepatol. 2016;14(11):1562-1570.e2. doi:10.1016/j.cgh.2016.05.042PubMedGoogle ScholarCrossref
20.
Sengupta  N, Tapper  EB.  Derivation and internal validation of a clinical prediction tool for 30-day mortality in lower gastrointestinal bleeding.   Am J Med. 2017;130(5):601.e1-601.e8. doi:10.1016/j.amjmed.2016.12.009 PubMedGoogle ScholarCrossref
21.
Saltzman  JR, Tabak  YP, Hyett  BH, Sun  X, Travis  AC, Johannes  RS.  A simple risk score accurately predicts in-hospital mortality, length of stay, and cost in acute upper GI bleeding.   Gastrointest Endosc. 2011;74(6):1215-1224. doi:10.1016/j.gie.2011.06.024 PubMedGoogle ScholarCrossref
22.
Hreinsson  JP, Sigurdardottir  R, Lund  SH, Bjornsson  ES.  The SHA2PE score: a new score for lower gastrointestinal bleeding that predicts low-risk of hospital-based intervention.   Scand J Gastroenterol. 2018;53(12):1484-1489. doi:10.1080/00365521.2018.1532019 PubMedGoogle ScholarCrossref
23.
Collins  GS, Ogundimu  EO, Altman  DG.  Sample size considerations for the external validation of a multivariable prognostic model: a resampling study.   Stat Med. 2016;35(2):214-226. doi:10.1002/sim.6787PubMedGoogle ScholarCrossref
24.
Oakland  K, Kahan  BC, Guizzetti  L,  et al.  Development, validation, and comparative assessment of an international scoring system to determine risk of upper gastrointestinal bleeding.   Clin Gastroenterol Hepatol. 2019;17(6):1121-1129.e2. doi:10.1016/j.cgh.2018.09.039PubMedGoogle ScholarCrossref
25.
Cryer  BL, Wilcox  CM, Henk  HJ, Zlateva  G, Chen  L, Zarotsky  V.  The economics of upper gastrointestinal bleeding in a US managed-care setting: a retrospective, claims-based analysis.   J Med Econ. 2010;13(1):70-77. doi:10.3111/13696990903526676 PubMedGoogle ScholarCrossref
26.
Nigam  N, Ham  SA, Sengupta  N.  Early colonoscopy for diverticular bleeding does not reduce risk of postdischarge recurrent bleeding: a propensity score-matching analysis.   Clin Gastroenterol Hepatol. 2019;17(6):1105-1111.e1. doi:10.1016/j.cgh.2018.09.050PubMedGoogle ScholarCrossref
27.
Jun  M, James  MT, Manns  BJ,  et al; Alberta Kidney Disease Network.  The association between kidney function and major bleeding in older adults with atrial fibrillation starting warfarin treatment: population based observational study.   BMJ. 2015;350:h246. doi:10.1136/bmj.h246 PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Views 2,071
    Citations 0
    Original Investigation
    Gastroenterology and Hepatology
    July 7, 2020

    使用奥克兰评分评估美国急性下消化道出血成人患者能否安全出院的外部验证

    Author Affiliations
    • 1Department of Digestive Diseases, HCA Healthcare UK, London, United Kingdom
    • 2Faculty of Medicine, Imperial College London, London, United Kingdom
    • 3Department of Data Science, HCA Healthcare, Nashville, Tennessee
    • 4Division of Gastroenterology, Western University, London, Ontario, Canada
    • 5Division of Gastroenterology and Biomedical Informatics, University of California, San Diego, San Diego
    JAMA Netw Open. 2020;3(7):e209630. doi:10.1001/jamanetworkopen.2020.9630
    关键点 español English

    问题  奥克兰评分是否是用于评估美国大量急性下消化道出血成人患者不良后果风险的有效工具?

    结果  在这项对 38,067 名因急性下消化道出血住院的成年患者进行的预后研究中,奥克兰评分系统一致确定了不良后果风险较低的患者。将奥克兰得分阈值从 8 分或更低扩展到 10 分或更低,以评估患者是否可以安全出院,可以发现更多具有较低不良后果风险的患者,并有可能避免 17.8% 的患者住院治疗,这些患者的病情可以在门诊基础上得到安全控制。

    意义  这项研究的调查结果表明,在美国医院就诊患者的分流过程中,采用奥克兰评分可以降低急性下消化道出血患者的住院率。

    Abstract

    Importance  Lower gastrointestinal bleeding (LGIB), which manifests as blood in the colon or anorectum, is a common reason for hospitalization. In most patients, LGIB stops spontaneously with no in-hospital intervention. A risk score that could identify patients at low risk of experiencing adverse outcomes could help improve the triage process and allow greater numbers of patients to receive outpatient management of LGIB.

    Objective  To externally validate the Oakland Score, which was previously developed using a score threshold of 8 points to identify patients with LGIB who are at low risk of adverse outcomes.

    Design, Setting, and Participants  This multicenter prognostic study was conducted in 140 US hospitals in the Hospital Corporation of America network. A total of 46 179 adult patients (aged ≥16 years) admitted to the hospital with a primary diagnosis of LGIB between June 1, 2016, and October 15, 2018, were initially identified using diagnostic codes. Of those, 51 patients were excluded because they were more likely to have upper gastrointestinal bleeding, leaving a study population of 46 128 patients with LGIB. For the statistical analysis of the Oakland Score, an additional 8061 patients were excluded because they were missing data on Oakland Score components or clinical outcomes, resulting in 38 067 patients included in the analysis. The study used area under the receiver operating characteristic curves with 95% CIs for external validation of the model. Sensitivity and specificity were calculated for each score threshold (≤8 points, ≤9 points, and ≤10 points). Data were analyzed from October 16, 2018, to September 4, 2019.

    Main Outcomes and Measures  Identification of patients who met the criteria for safe discharge from the hospital and comparison of the performance of 2 score thresholds (≤8 points vs ≤10 points). Safe discharge was defined as the absence of blood transfusion, rebleeding, hemostatic intervention, hospital readmission, and death.

    Results  Among 46 128 adult patients with LGIB, the mean (SD) age was 70.1 (16.5) years; 23 091 patients (50.1%) were female. Of those, 22 074 patients (47.9%) met the criteria for safe discharge from the hospital. In this group, the mean (SD) age was 67.9 (18.1) years, and 11 056 patients (50.1%) were female. In the statistical analysis of the Oakland Score, which included only the 38 067 patients with complete data, the area under the receiver operating characteristic curve for safe discharge was 0.87 (95% CI, 0.87-0.87). An Oakland Score threshold of 8 points or lower identified 3305 patients (8.7%), with a sensitivity and specificity for safe discharge of 98.4% and 16.0%, respectively. Extension of the Oakland Score threshold to 10 points or lower identified 6770 patients (17.8%), with a sensitivity and specificity for safe discharge of 96.0% and 31.9%, respectively.

    Conclusions and Relevance  In this study, the Oakland Score consistently identified patients with acute LGIB who were at low risk of experiencing adverse outcomes and whose conditions could safely be managed without hospitalization. The score threshold to identify low-risk patients could be extended from 8 points or lower to 10 points or lower to allow identification of a greater proportion of low-risk patients.

    Introduction

    Lower gastrointestinal bleeding (LGIB) is a common presentation in the emergency departments of hospitals worldwide. In comparison with upper gastrointestinal bleeding (UGIB), LGIB is likely to have a less severe course. Compared with patients with UGIB, patients with LGIB are less likely to present with hemorrhagic shock or require red blood cell (RBC) transfusions or interventions to treat bleeding, and in-hospital mortality rates among patients with LGIB are lower.1 Acute LGIB typically presents with bright red rectal bleeding or blood clots from the rectum,1 whereas the presenting features of UGIB include hematemesis, coffee-ground emesis, and melena. Unlike patients with UGIB, for which risk stratification scores, such as the Rockall2 and Glasgow-Blatchford3 scores, are used, patients with LGIB have no equivalent risk score tool available.

    In 2016, the American College of Gastroenterology recommended that risk assessment be performed but did not endorse the use of any single tool.4 Since the publication of this guideline, the Oakland Score5 was developed within a nationally representative sample of patients in the United Kingdom. Rather than calculating a risk score for death or inpatient intervention, the Oakland Score was designed to identify patients who were at low risk of experiencing adverse outcomes and whose conditions could safely be managed without hospitalization. Despite limited external validation of the Oakland Score,5,6 national guidelines in the United Kingdom have recently recommended use of the tool for the triage of patients with acute LGIB.7 Therefore, the aim of this study was to externally validate the Oakland Score in a large population of patients with acute LGIB from the United States and compare the performance of the Oakland Score at 2 score thresholds (≤8 points vs ≤10 points).

    Methods

    Patients with acute LGIB were identified from 140 hospitals in the Hospital Corporation of America (HCA) network across the United States. The study was approved by the HCA Research Review Council. Written informed consent to use unidentified patient data for clinical improvement purposes was obtained from all participants, as written informed consent is built into the overall informed consent process when a patient agrees to receive treatment or diagnostic testing at any HCA hospital. The study used data that are routinely collected for quality improvement purposes, collected no patient identifiers, and involved no new clinical intervention. This study followed the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) reporting guideline8 for prognostic studies.

    Adult patients (aged ≥16 years) who were admitted to the hospital with acute LGIB between June 1, 2016, and October 15, 2018, were identified retrospectively using codes from the International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM)9 that were consistent with a primary diagnosis of LGIB (eMethods in the Supplement). In 2018, the HCA network of US hospitals comprised 179 general and acute care hospitals in 20 states that delivered care to patients who paid for care directly or through Medicare and Medicaid programs, managed care plans, or private insurance. Clinical data from HCA facilities have been stored in a single data warehouse since 2011. For each patient encounter, these data are automatically collected in real time from electronic health records (EHRs) and consolidated in the data warehouse, from which longitudinal EHRs for each patient can be extracted. These EHRs contain data on current hospitalizations as well as comorbidities reported during previous health care events. Hospitals in the HCA network were eligible to participate in the present study if they had an emergency department, routinely admitted patients with emergency conditions, and had comprehensive EHRs to allow record linkage of clinical outcomes.

    Patients provisionally identified as having LGIB who received endoscopic hemostasis during esophagogastoduodenoscopy (using Current Procedural Terminology [CPT] code 43255) were excluded because they were more likely to have UGIB. Data on demographic characteristics, comorbidities, medications, vital signs, blood test results, treatments, and outcomes were collected for each patient. Heart rate (measured in beats per minute), systolic blood pressure (measured in mm Hg), hemoglobin concentration (measured in g/L), platelet count (measured in 109/L), white blood cell count (measured in 109/L), blood urea nitrogen level (measured in mg/dL), creatinine level (measured in mg/dL), albumin level (measured in g/dL), and international normalized ratio were extracted from the results of each patient’s first recorded set of vital signs and blood tests. Previous hospital admission with LGIB was identified through ICD-10-CM codes for LGIB that were recorded at any point in the patient’s longitudinal EHR.

    Comorbidities were classified using codes from the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) that were consistent with cancer and cardiovascular, renal, and liver diseases (eMethods in the Supplement). Patient receipt of oral antiplatelet or anticoagulant medications was identified using the Standard Industrial Classification codes M9L and M9P, and these data were extracted only if receipt of the medications appeared in the patient’s regular medication records. To identify medical procedures received, the following CPT codes were used: codes 45378 to 45398 for inpatient colonoscopy, code 36430 for RBC transfusion, code 37242 for mesenteric embolization, and codes specific to small bowel, colon, or rectum resection for abdominal surgery for bleeding (full list of CPT codes available in eMethods in the Supplement).

    The primary outcome was the composite outcome of safe discharge from the hospital,5 with safe discharge defined as the absence of all of the following after hospital presentation: in-hospital rebleeding (defined as a decrease in hematocrit concentrations of 20% or more after 24 hours of clinical stability10); RBC transfusion; therapeutic colonoscopy, mesenteric embolization, or laparotomy for bleeding; in-hospital death (all causes); and readmission with subsequent LGIB within 28 days. We were able to identify patients who were readmitted to the index hospital or another hospital in the HCA network using ICD-10-CM codes (eMethods in the Supplement) but were not able to detect patients who were readmitted to hospitals outside of the HCA network.

    Oakland Score

    The Oakland Score was originally derived from prospective data obtained from 2336 patients with LGIB from 143 hospitals in the United Kingdom in 2015 (eMethods in the Supplement), with the aim of identifying patients at low risk of experiencing adverse outcomes.5 The total score contains 7 variables (age, sex, previous hospital admission with LGIB, digital rectal examination results, heart rate, systolic blood pressure, and hemoglobin concentration) and ranges from 0 to 35 points, with higher scores indicating greater risk of experiencing an adverse outcome (Table 1).

    Data on the Oakland Score variables were extracted from patient EHRs. Digital rectal examination findings were not available, so this variable was omitted from the score calculations. When calculating a total Oakland Score, hemoglobin concentration and systolic blood pressure are assigned the highest point weightings (0-22 points and 0-5 points, respectively). Points ascribed to digital rectal examination results are either 1 point if blood is present or 0 points if blood is absent. The remaining variables were used to calculate an Oakland Score for each patient with LGIB.

    Statistical Analysis

    Any patients with missing data on score component variables or clinical outcomes were excluded from the statistical analysis of the Oakland Score. The ability of the Oakland Score to predict in-hospital rebleeding, RBC transfusion, therapeutic intervention to control bleeding, in-hospital mortality, subsequent LGIB within 28 days, and the composite outcome of safe discharge was assessed using area under the receiver operating characteristic (AUROC) curves and 95% CIs.

    When assessing the performance of a risk score that is designed to identify low-risk patients, sensitivity is the most important outcome.11 When clinicians make decisions about early discharge, it is important that patients at high risk are not misclassified as low risk. Previous studies of patients with UGIB have reported that a sensitivity of 95% or more can be used to identify optimal score thresholds for low-risk patients whose conditions would be potentially suitable for outpatient management.12 The sensitivity for safe discharge was calculated for point scores of 8 or lower, 9 or lower, and 10 or lower, and the performance of these score thresholds was compared to identify thresholds that would maintain a sensitivity of 95%. Statistical analysis was performed with Python software (Python Software Foundation) using scikit, SciPy, and NumPy arrays. Data were analyzed from October 16, 2018, to September 4, 2019.

    Results

    A total of 46 179 patients admitted to the hospital with a primary diagnosis of LGIB were initially identified. Of those, 51 patients received endoscopic hemostasis at esophagogastoduodenoscopy and were excluded because they were more likely to have UGIB, leaving a study population of 46 128 patients (mean [SD] age, 70.1 [16.5] years; 23 091 women [50.1%]) (eFigure in the Supplement). Of those, 3251 patients (7.0%) were receiving oral anticoagulant medications at the time of hospital admission (Table 2). Overall, 17 896 patients (38.8%) received inpatient colonoscopy, 21 629 patients (68.1%; missing data in 14 387 patients) received RBC transfusion, 3097 patients (6.7%) experienced rebleeding during admission, 79 patients (0.2%) received endoscopic hemostasis during colonoscopy, 15 patients (0.03%) received mesenteric embolization, and 1 patient underwent laparotomy for refractory bleeding. The median length of stay was 3 days (range, 1-175 days), 2048 patients (4.4%) died during hospitalization, and 1166 patients (2.5%) were readmitted with subsequent LGIB. Across the study population, the most common diagnoses were diverticular bleeding (10 657 patients [23.1%]), hemorrhoids (5339 patients [11.6%]), and angiodysplasia (2227 patients [4.8%]). An additional 17 957 patients (38.9%) were classified as having an unspecified gastrointestinal hemorrhage, and 3933 patients (8.5%) were classified as having a hemorrhage of the anus and rectum.

    Overall, 22 074 (47.9%; 95% CI, 47.4%-48.3%) experienced none of the adverse outcomes specified and could be classified as meeting the criteria for safe discharge. A total of 11 056 patients (50.1%) in this group were female. Compared with patients who did not meet the safe discharge criteria, patients who met the criteria were younger (mean [SD] age, 72.2 [14.7] years vs 67.9 [18.1] years, respectively) with fewer comorbidities (26 229 comorbidities vs 15 857 comorbidities) and fewer patients had previous hospital admissions with LGIB (880 admissions vs 282 admissions), and fewer patients were receiving oral anticoagulant or antiplatelet medications (5000 patients vs 3314 patients) at hospital admission.

    In total, 38 067 patients (82.5%) had complete data on all components of the Oakland Score and clinical outcomes. The AUROC of the composite outcome of safe discharge was 0.87 (95% CI, 0.87-0.87), suggesting good discriminative performance (Figure 1). In the models predicting adverse outcomes, the AUROCs were as follows: for RBC transfusion, 0.90 (95% CI, 0.90-0.90); for in-hospital rebleeding, 0.46 (95% CI, 0.45-0.47); for death, 0.63 (95% CI, 0.62-0.64); and for hospital readmission with subsequent bleeding, 0.60 (95% CI, 0.59-0.62). Because only 190 patients or fewer (≤0.5%) received therapeutic interventions to control bleeding, AUROCs were not calculated for these outcomes. The median Oakland Score was 18 points (range, 2-33 points) (Figure 2). In total, 3305 of 38 067 patients (8.7%) scored 8 points or lower, with a sensitivity of 98.4% and a specificity of 16.0% for safe discharge (Table 3). A sensitivity of 96.0% for safe discharge was maintained to a score threshold of 10 points or lower, with a specificity of 31.9%. A total of 4888 patients (12.8%) had a score of 9 points or lower, and 6770 patients (17.8%) had a score of 10 points or lower.

    The most common adverse outcomes in patients with Oakland Scores of 10 points or lower were RBC transfusion and in-hospital rebleeding. No patients received mesenteric embolization or surgery, and the percentage of patients who received endoscopic hemostasis was constant between those scoring 8 points or lower (11 of 3305 patients [0.3%]), 9 points or lower (16 of 4888 patients [0.3%]), and 10 points or lower (21 of 6770 patients [0.3%]). Death occurred in 37 of 3305 patients (1.1%) with a score of 8 points or lower, 60 of 4888 patients (1.2%) with a score of 9 points or lower, and 96 of 6770 patients (1.4%) with a score of 10 points or lower (Table 3).

    Discussion

    This study of 140 US hospitals in the HCA network found that the Oakland Score could identify patients at low risk of experiencing adverse outcomes who may be safe for hospital discharge. The Oakland Score was able to discriminate patients at low risk of adverse outcomes despite having data on only 6 of the 7 previously validated variables, suggesting that a modification of the total score that does not include the variable of digital rectal examination could be safely used. We also found that the previously recommended threshold of 8 points or lower7 to identify low-risk patients could be extended to 10 points or lower while maintaining a sensitivity of 96% for safe discharge.

    Consistent with other large observational studies, the most common intervention observed in the present study was RBC transfusion, which was identified in 68.1% of patients. The percentage of deaths (4.4%) is similar to those reported elsewhere (3.4% and 8.8%).1,13 Studies from the United States and the United Kingdom indicated that endoscopic hemostasis is performed in only 2.1% to 4.6% of patients with LGIB,1,14 but the frequency of less than 1% of patients who received endoscopic hemostasis in the present study is comparatively low. Only 1 CPT code is specific for endoscopic hemostasis, but other codes corresponding with nonspecific band ligation or submucosal injection exist. These CPT codes may have been used in place of the bleeding-specific code, which could have produced underestimation of the frequency of endoscopic hemostasis in our study. The proportion of patients receiving inpatient colonoscopy was also low, at only 38.8%. Reports on the rates of colonoscopy in a US population of patients with LGIB are rare, as most observational studies use colonoscopy to identify patients. The largest study, performed by Navaneethan et al,15 identified hospitalizations of patients with LGIB using ICD-9-CM codes. Of the 58 296 patients discharged from the hospital who were identified in that study, only 2270 patients (38.9%) received an inpatient colonoscopy. This percentage is similar to those reported in a smaller US study (34.7%)16 and in studies from the United Kingdom1 and Australia.17

    Other scores used to assess risk among patients with LGIB have been developed. The BLEED (ongoing bleeding, low systolic blood pressure, elevated prothrombin time, erratic mental status, and unstable comorbid disease) score18 was designed to assess the risk of in-hospital complications, the NOBLADS (nonsteroidal anti-inflammatory drug use, no diarrhea or abdominal tenderness, blood pressure ≤100 mm Hg, antiplatelet drug use, albumin level <3.0 g/dL, disease score ≥2 points, and syncope) score19 and the Strate score10 were developed to assess the risk of severe bleeding, and the Sengupta score20 was designed to assess the 30-day mortality risk.

    Risk scores developed for patients with UGIB, such as the AIMS-65 (albumin level <3.0 g/dL, international normalized ratio >1.5, altered mental status, systolic blood pressure ≤90 mm Hg, and age >65 years) score21 and the Glasgow-Blatchford score,3 have also been evaluated in patients with LGIB. A study by Oakland et al5 reported that the AIMS-65 score was the best predictor of death (AUROC, 0.78), the Oakland Score and Glasgow-Blatchford score were the best predictors of rebleeding (AUROC, 0.74), and the Oakland Score was the best predictor of RBC transfusion (AUROC, 0.92). Tapaskar et al6 reported that the Oakland Score was the best predictor of severe bleeding (AUROC, 0.74), the Strate score was the best predictor of rebleeding (AUROC, 0.66), and the Glasgow-Blatchford score was the best predictor of RBC transfusion (AUROC, 0.87). Since publication of the Oakland Score, other scores aimed at identifying low-risk patients have been developed. The SHA2PE (systolic blood pressure ≥100 mm Hg, hemoglobin level >12 g/dL, hemoglobin level 10.5-12.0 g/dL, no antiplatelet medication, no anticoagulant medication, pulse ≤100 beats/min, and visible bleeding in the emergency department) score22 was derived from a study of 580 patients but has not been externally validated.

    To assess the generalizability of a prognostic model, the TRIPOD guidelines state that “it is preferable to use a slightly different case-mix in external validation to judge model transportability. Successful external validation studies in diverse settings (with different case-mix) indicate that it is more likely that the model will be generalizable to plausibly related, but untested settings.”23(p214) The Oakland Score was originally derived using prospective data from patients in the United Kingdom; however, in the present study, we found that the Oakland Score also has prognostic value in a US population. A strength of the Oakland Score is the simplicity of its components, which include demographic factors, vital signs, and a single blood test, allowing the score to be fully calculated at initial assessment without an observation period or endoscopic findings. A key criticism of the Oakland Score is that because it was designed to be highly sensitive, some patients who might safely be discharged may instead be identified as requiring hospitalization. This possibility is less important than the potential of misclassifying high-risk patients as low risk. The specificities reported in the present article are low (16.0% for an Oakland Score of ≤8 points and 31.9% for an Oakland Score of ≤10 points). These low specificities are consistent with those reported for other risk scores; for example, the specificity of the Glasgow-Blatchford score is 8% to 22% for a score of 0 points and 34% to 39% for a score of 1 point.11

    Although most patients identified as low risk will not experience adverse outcomes, some patients at each Oakland Score threshold experienced rebleeding, required hospital-based intervention, or died. The same outcomes have been observed in studies of other risk scores that have been adopted into clinical practice.12,24 Determining the safe threshold for identification of patients who have a low risk of experiencing adverse outcomes and whose conditions can safely be managed without hospitalization requires balancing the risk of misclassification with the need to identify an adequate real-world population for the score to be clinically useful. In the Oakland Score development study, a score of 8 points or lower was found to be the safe threshold for identifying low-risk patients despite the threshold being applicable to only 8% of patients.5 In the present study, an Oakland Score of 8 points or lower also identified only 8.7% of patients. When the score threshold was extended to 10 points or lower, 17.8% of patients were identified as low risk, with a sensitivity for safe discharge of 96%. Nonetheless, because the study population comprised patients admitted to the hospital with LGIB, the proportion of patients scoring 8 points or lower or 10 points or lower would likely be greater if patients who were discharged from the emergency department were included. To determine this proportion, a prospective cohort study is needed, in which all patients presenting to the emergency department are included, regardless of their admission status.

    When comparing patient outcomes using a score threshold of 8 points or lower vs 10 points or lower, the number of patients who received hospital-based interventions to treat bleeding remained constant, the percentage of patients who received RBC transfusion increased, and the percentage of patients who died increased. The percentage of deaths at these score thresholds (1.1% at ≤8 points and 1.4% at ≤10 points) is concerning and is similar to the percentage of deaths among low-risk patients reported for the Glasgow-Blatchford score24 and other risk scores.11 The use of clinician judgment in combination with risk scores is important. The need for RBC transfusion does not necessarily require patient hospitalization because transfusions or iron can be administered in an ambulatory setting. If patients have bled sufficiently to develop symptomatic anemia, they are unlikely to meet the Oakland Score threshold for safe discharge, which allocates at least 13 points to hemoglobin concentrations less than 110 g/L.5

    Reducing the number of hospitalizations of patients with LGIB has important benefits. Few data are available describing the economic burden of LGIB, but data from a national US database suggested that costs are between $22 142 and $28 749 per hospitalization, or $4492 per bed-day.15 Although LGIB is likely to have a more benign course than UGIB, hospitalizations for patients with LGIB are more expensive than those for patients with UGIB, primarily owing to the longer length of stay and higher resource use.13 If an Oakland Score threshold of 8 points or lower was used to identify low-risk patients, hospital admission could potentially be avoided in 8.7% of patients. Given that the median length of stay in the present study was 3 days, this reduction in hospital admissions could produce a savings of $44.5 million within the study population alone. If the score threshold was extended to 10 points or lower, this savings could be $91.2 million. A retrospective analysis performed in the United States found that 40% of costs associated with UGIB were incurred after hospital discharge.25 Similar findings are likely to apply to LGIB. In addition, because patients with LGIB are likely to be older than 65 years and to have a high comorbidity burden, other reasons for hospital admission may be present.

    Limitations

    This study has several limitations. First, patient identification relied on the use of administrative codes; however, previous large studies of patients with LGIB have successfully used ICD and CPT codes for this purpose.26,27 Patients with hospital codes consistent with UGIB were excluded, as were patients who received endotherapy during esophagogastoduodenoscopy. However, the frequency of patients with the discharge code corresponding with an unspecified gastrointestinal hemorrhage suggests that some patients may have had bleeding that originated in the upper rather than the lower gastrointestinal tract, and some may have had small-bowel bleeding. This uncertainty reflects that of clinicians during clinical assessment, in which it is often difficult to distinguish the site of bleeding based on the patient’s medical history and examination alone.

    Second, because only 38.8% of patients received inpatient colonoscopy, it is unclear how some of the definitive diagnoses were made. Third, the digital rectal examination variable needed to calculate the Oakland Score was missing; despite this limitation, the score performed well, which provides the option of using this modification in patients who cannot tolerate digital rectal examination, such as those with bleeding from an anal fissure. Fourth, the present study is limited to patents who were hospitalized with LGIB. Fifth, data on RBC transfusion were frequently missing. It is likely that these cases represent an absence of transfusion rather than missing data; however, the missing data may have produced an overestimation of the proportion of patients who received transfusions.

    Conclusions

    This large multicenter prognostic study found that the Oakland Score was externally valid for use in assessing the risk of adverse outcomes in patients with LGIB. The Oakland Score threshold of 8 points or lower, which is currently used to identify patients at low risk of experiencing adverse outcomes, could be extended to 10 points or lower to allow identification of a greater proportion of low-risk patients while maintaining sensitivity; however, an increase in adverse events may occur with use of the higher score threshold.

    Back to top
    Article Information

    Accepted for Publication: April 23, 2020.

    Published: July 7, 2020. doi:10.1001/jamanetworkopen.2020.9630

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Oakland K et al. JAMA Network Open.

    Corresponding Author: Kathryn Oakland, MD, Department of Digestive Diseases, HCA Healthcare UK, 242 Marylebone Rd, Marylebone, London, United Kingdom (kathryn.oakland@hcahealthcare.co.uk).

    Author Contributions: Drs Oakland and Kothiwale had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Oakland, Kothiwale, Perlin.

    Acquisition, analysis, or interpretation of data: All authors.

    Drafting of the manuscript: Oakland, Kothiwale, Bucknall, Jairath.

    Critical revision of the manuscript for important intellectual content: Oakland, Kothiwale, Forehand, Jackson, Sey, Singh, Jairath, Perlin.

    Statistical analysis: Kothiwale, Sey.

    Administrative, technical, or material support: Oakland, Forehand, Bucknall, Perlin.

    Supervision: Oakland, Jackson, Jairath, Perlin.

    Conflict of Interest Disclosures: Drs Oakland and Jairath developed the original Oakland Score. Dr Singh reported receiving grants from AbbVie and Janssen Pharmaceuticals and personal fees from AbbVie, Pfizer, and Takeda Pharmaceutical Company outside the submitted work. No other disclosures were reported.

    Funding/Support: HCA Healthcare sponsored this study.

    Role of the Funder/Sponsor: HCA Healthcare sponsored the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, and approval of the manuscript; and decision to submit the manuscript for publication.

    Meeting Presentation: An abstract of this paper was published on the Digestive Disease Week website as part of the 2020 Digestive Disease Week; May 2, 2020; https://ddw.apprisor.org/epsAbstractDDW.cfm?id=1.

    References
    1.
    Oakland  K, Guy  R, Uberoi  R,  et al.  Acute lower GI bleeding in the UK: patient characteristics, interventions and outcomes in the first nationwide audit.   Gut. 2018;67(4):654-662. PubMedGoogle Scholar
    2.
    Rockall  TA, Logan  RF, Devlin  HB, Northfield  TC.  Risk assessment after acute upper gastrointestinal haemorrhage.   Gut. 1996;38(3):316-321. doi:10.1136/gut.38.3.316 PubMedGoogle ScholarCrossref
    3.
    Blatchford  O, Murray  WR, Blatchford  M.  A risk score to predict need for treatment for upper-gastrointestinal haemorrhage.   Lancet. 2000;356(9238):1318-1321. doi:10.1016/S0140-6736(00)02816-6 PubMedGoogle ScholarCrossref
    4.
    Strate  LL, Gralnek  IM.  ACG clinical guideline: management of patients with acute lower gastrointestinal bleeding.   Am J Gastroenterol. 2016;111(5):755. doi:10.1038/ajg.2016.155 PubMedGoogle ScholarCrossref
    5.
    Oakland  K, Jairath  V, Uberoi  R,  et al.  Derivation and validation of a novel risk score for safe discharge after acute lower gastrointestinal bleeding: a modelling study.   Lancet Gastroenterol Hepatol. 2017;2(9):635-643. doi:10.1016/S2468-1253(17)30150-4 PubMedGoogle ScholarCrossref
    6.
    Tapaskar  N, Jones  B, Mei  S, Sengupta  N.  Comparison of clinical prediction tools and identification of risk factors for adverse outcomes in acute lower GI bleeding.   Gastrointest Endosc. 2019;89(5):1005-1013. doi:10.1016/j.gie.2018.12.011 PubMedGoogle Scholar
    7.
    Oakland  K, Chadwick  G, East  JE,  et al.  Diagnosis and management of acute lower gastrointestinal bleeding: guidelines from the British Society of Gastroenterology.   Gut. 2019;68(5):776-789. doi:10.1136/gutjnl-2018-317807 PubMedGoogle ScholarCrossref
    8.
    Collins  GS, Reitsma  JB, Altman  DG, Moons  KGM.  Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement.   Ann Intern Med. 2015;162(1):55-63. doi:10.7326/M14-0697 PubMedGoogle ScholarCrossref
    9.
    World Health Organization.  International Statistical Classification of Diseases and Related Health Problems, Tenth Revision. World Health Organization; 1992.
    10.
    Strate  LL, Orav  EJ, Syngal  S.  Early predictors of severity in acute lower intestinal tract bleeding.   Arch Intern Med. 2003;163(7):838-843. doi:10.1001/archinte.163.7.838 PubMedGoogle ScholarCrossref
    11.
    Barkun  AN, Almadi  M, Kuipers  EJ,  et al.  Management of nonvariceal upper gastrointestinal bleeding: guideline recommendations from the International Consensus Group.   Ann Intern Med. 2019;171(11):805-822. doi:10.7326/M19-1795PubMedGoogle ScholarCrossref
    12.
    Stanley  AJ, Laine  L, Dalton  HR,  et al; International Gastrointestinal Bleeding Consortium.  Comparison of risk scoring systems for patients presenting with upper gastrointestinal bleeding: international multicentre prospective study.   BMJ. 2017;356:i6432. doi:10.1136/bmj.i6432 PubMedGoogle ScholarCrossref
    13.
    Lanas  A, Garcia-Rodriguez  LA, Polo-Tomas  M,  et al.  Time trends and impact of upper and lower gastrointestinal bleeding and perforation in clinical practice.   Am J Gastroenterol. 2009;104(7):1633-1641. doi:10.1038/ajg.2009.164 PubMedGoogle ScholarCrossref
    14.
    Ron-Tal Fisher  O, Gralnek  IM, Eisen  GM, Williams  JL, Holub  JL.  Endoscopic hemostasis is rarely used for hematochezia: a population-based study from the Clinical Outcomes Research Initiative National Endoscopic Database.   Gastrointest Endosc. 2014;79(2):317-325. doi:10.1016/j.gie.2013.09.004 PubMedGoogle ScholarCrossref
    15.
    Navaneethan  U, Njei  B, Venkatesh  PG, Sanaka  MR.  Timing of colonoscopy and outcomes in patients with lower GI bleeding: a nationwide population-based study.   Gastrointest Endosc. 2014;79(2):297-306. doi:10.1016/j.gie.2013.08.001 PubMedGoogle ScholarCrossref
    16.
    Strate  LL, Ayanian  JZ, Kotler  G, Syngal  S.  Risk factors for mortality in lower intestinal bleeding.   Clin Gastroenterol Hepatol. 2008;6(9):1004-1010. doi:10.1016/j.cgh.2008.03.021PubMedGoogle ScholarCrossref
    17.
    Ng  K-S, Nassar  N, Soares  D, Stewart  P, Gladman  MA.  Acute lower gastrointestinal haemorrhage: outcomes and risk factors for intervention in 949 emergency cases.   Int J Colorectal Dis. 2017;32(9):1327-1335. doi:10.1007/s00384-017-2844-2 PubMedGoogle ScholarCrossref
    18.
    Kollef  MH, O’Brien  JD, Zuckerman  GR, Shannon  W.  BLEED: a classification tool to predict outcomes in patients with acute upper and lower gastrointestinal hemorrhage.   Crit Care Med. 1997;25(7):1125-1132. doi:10.1097/00003246-199707000-00011 PubMedGoogle ScholarCrossref
    19.
    Aoki  T, Nagata  N, Shimbo  T,  et al.  Development and validation of a risk scoring system for severe acute lower gastrointestinal bleeding.   Clin Gastroenterol Hepatol. 2016;14(11):1562-1570.e2. doi:10.1016/j.cgh.2016.05.042PubMedGoogle ScholarCrossref
    20.
    Sengupta  N, Tapper  EB.  Derivation and internal validation of a clinical prediction tool for 30-day mortality in lower gastrointestinal bleeding.   Am J Med. 2017;130(5):601.e1-601.e8. doi:10.1016/j.amjmed.2016.12.009 PubMedGoogle ScholarCrossref
    21.
    Saltzman  JR, Tabak  YP, Hyett  BH, Sun  X, Travis  AC, Johannes  RS.  A simple risk score accurately predicts in-hospital mortality, length of stay, and cost in acute upper GI bleeding.   Gastrointest Endosc. 2011;74(6):1215-1224. doi:10.1016/j.gie.2011.06.024 PubMedGoogle ScholarCrossref
    22.
    Hreinsson  JP, Sigurdardottir  R, Lund  SH, Bjornsson  ES.  The SHA2PE score: a new score for lower gastrointestinal bleeding that predicts low-risk of hospital-based intervention.   Scand J Gastroenterol. 2018;53(12):1484-1489. doi:10.1080/00365521.2018.1532019 PubMedGoogle ScholarCrossref
    23.
    Collins  GS, Ogundimu  EO, Altman  DG.  Sample size considerations for the external validation of a multivariable prognostic model: a resampling study.   Stat Med. 2016;35(2):214-226. doi:10.1002/sim.6787PubMedGoogle ScholarCrossref
    24.
    Oakland  K, Kahan  BC, Guizzetti  L,  et al.  Development, validation, and comparative assessment of an international scoring system to determine risk of upper gastrointestinal bleeding.   Clin Gastroenterol Hepatol. 2019;17(6):1121-1129.e2. doi:10.1016/j.cgh.2018.09.039PubMedGoogle ScholarCrossref
    25.
    Cryer  BL, Wilcox  CM, Henk  HJ, Zlateva  G, Chen  L, Zarotsky  V.  The economics of upper gastrointestinal bleeding in a US managed-care setting: a retrospective, claims-based analysis.   J Med Econ. 2010;13(1):70-77. doi:10.3111/13696990903526676 PubMedGoogle ScholarCrossref
    26.
    Nigam  N, Ham  SA, Sengupta  N.  Early colonoscopy for diverticular bleeding does not reduce risk of postdischarge recurrent bleeding: a propensity score-matching analysis.   Clin Gastroenterol Hepatol. 2019;17(6):1105-1111.e1. doi:10.1016/j.cgh.2018.09.050PubMedGoogle ScholarCrossref
    27.
    Jun  M, James  MT, Manns  BJ,  et al; Alberta Kidney Disease Network.  The association between kidney function and major bleeding in older adults with atrial fibrillation starting warfarin treatment: population based observational study.   BMJ. 2015;350:h246. doi:10.1136/bmj.h246 PubMedGoogle ScholarCrossref
    ×