[Skip to Content]
[Skip to Content Landing]
Table 1.  Intensivist Demographic Characteristics Stratified by Follow-up Study Participation
Intensivist Demographic Characteristics Stratified by Follow-up Study Participation
Table 2.  Categorized Choices Identified in Text Marked by 32 Intensivists Reviewing Their Own Transcripts Who Responded Yes to the Question, “During the Simulation, Did You Highlight That There Is a Choice?”
Categorized Choices Identified in Text Marked by 32 Intensivists Reviewing Their Own Transcripts Who Responded Yes to the Question, “During the Simulation, Did You Highlight That There Is a Choice?”
Table 3.  Categorized Recommendations Identified in Text Marked by 33 Intensivists Reviewing Their Own Transcripts Who Responded Yes to the Question, “During the Simulation, Did You Provide a Recommendation?”
Categorized Recommendations Identified in Text Marked by 33 Intensivists Reviewing Their Own Transcripts Who Responded Yes to the Question, “During the Simulation, Did You Provide a Recommendation?”
Table 4.  Clarity of Comfort Care Offer as Rated by Blinded Intensivists
Clarity of Comfort Care Offer as Rated by Blinded Intensivists
1.
Teno  JM, Gozalo  P, Trivedi  AN,  et al.  Site of death, place of care, and health care transitions among US Medicare beneficiaries, 2000-2015.   JAMA. 2018;320(3):264-271. doi:10.1001/jama.2018.8981PubMedGoogle ScholarCrossref
2.
Teno  JM, Gozalo  P, Khandelwal  N,  et al.  Association of increasing use of mechanical ventilation among nursing home residents with advanced dementia and intensive care unit beds.   JAMA Intern Med. 2016;176(12):1809-1816. doi:10.1001/jamainternmed.2016.5964PubMedGoogle ScholarCrossref
3.
Kon  AA, Davidson  JE, Morrison  W, Danis  M, White  DB; American College of Critical Care Medicine; American Thoracic Society.  Shared decision making in ICUs: An American College of Critical Care Medicine and American Thoracic Society policy statement.   Crit Care Med. 2016;44(1):188-201. doi:10.1097/CCM.0000000000001396PubMedGoogle ScholarCrossref
4.
Heyland  DK, Heyland  R, Dodek  P,  et al; ACCEPT Study Team and the Canadian Researchers at the End of Life Network (CARENET).  Discordance between patients’ stated values and treatment preferences for end-of-life care: results of a multicentre survey.   BMJ Support Palliat Care. 2017;7(3):292-299. doi:10.1136/bmjspcare-2015-001056PubMedGoogle Scholar
5.
Krawczyk  M, Gallagher  R.  Communicating prognostic uncertainty in potential end-of-life contexts: experiences of family members.   BMC Palliat Care. 2016;15(1):59. doi:10.1186/s12904-016-0133-4PubMedGoogle ScholarCrossref
6.
Scheunemann  LP, Ernecoff  NC, Buddadhumaruk  P,  et al.  Clinician-family communication about patients’ values and preferences in intensive care units.   JAMA Intern Med. 2019;179(5):676-684. doi:10.1001/jamainternmed.2019.0027PubMedGoogle ScholarCrossref
7.
Scheunemann  LP, Cunningham  TV, Arnold  RM, Buddadhumaruk  P, White  DB.  How clinicians discuss critically ill patients’ preferences and values with surrogates: an empirical analysis.   Crit Care Med. 2015;43(4):757-764. doi:10.1097/CCM.0000000000000772PubMedGoogle ScholarCrossref
8.
Smith  MA, Clayman  ML, Frader  J,  et al.  A descriptive study of decision-making conversations during pediatric intensive care unit family conferences.   J Palliat Med. 2018;21(9):1290-1299. doi:10.1089/jpm.2017.0528PubMedGoogle ScholarCrossref
9.
Teno  JM, Fisher  E, Hamel  MB,  et al.  Decision-making and outcomes of prolonged ICU stays in seriously ill patients.   J Am Geriatr Soc. 2000;48(S1):S70-S74. doi:10.1111/j.1532-5415.2000.tb03144.xPubMedGoogle ScholarCrossref
10.
White  DB, Braddock  CH  III, Bereknyei  S, Curtis  JR.  Toward shared decision making at the end of life in intensive care units: opportunities for improvement.   Arch Intern Med. 2007;167(5):461-467. doi:10.1001/archinte.167.5.461PubMedGoogle ScholarCrossref
11.
Roter  D, Larson  S.  The Roter Interaction Analysis System (RIAS): utility and flexibility for analysis of medical interactions.   Patient Educ Couns. 2002;46(4):243-251. doi:10.1016/S0738-3991(02)00012-5PubMedGoogle ScholarCrossref
12.
Elwyn  G, Tsulukidze  M, Edwards  A, Légaré  F, Newcombe  R.  Using a ‘talk’ model of shared decision making to propose an observation-based measure: Observer OPTION 5 Item.   Patient Educ Couns. 2013;93(2):265-271. doi:10.1016/j.pec.2013.08.005PubMedGoogle ScholarCrossref
13.
Cabana  MD, Rand  CS, Powe  NR,  et al.  Why don’t physicians follow clinical practice guidelines? a framework for improvement.   JAMA. 1999;282(15):1458-1465. doi:10.1001/jama.282.15.1458PubMedGoogle ScholarCrossref
14.
Turnbull  AE, Hayes  MM, Brower  RG,  et al.  Effect of documenting prognosis on the information provided to ICU proxies: a randomized trial.   Crit Care Med. 2019;47(6):757-764. doi:10.1097/CCM.0000000000003731PubMedGoogle ScholarCrossref
15.
Lemeshow  S, Teres  D, Klar  J, Avrunin  JS, Gehlbach  SH, Rapoport  J.  Mortality Probability Models (MPM II) based on an international cohort of intensive care unit patients.   JAMA. 1993;270(20):2478-2486. doi:10.1001/jama.1993.03510200084037PubMedGoogle ScholarCrossref
16.
Lemeshow  S, Klar  J, Teres  D,  et al.  Mortality probability models for patients in the intensive care unit for 48 or 72 hours: a prospective, multicenter study.   Crit Care Med. 1994;22(9):1351-1358. doi:10.1097/00003246-199409000-00003PubMedGoogle ScholarCrossref
17.
Halpern  SD, Becker  D, Curtis  JR,  et al; Choosing Wisely Taskforce; American Thoracic Society; American Association of Critical-Care Nurses; Society of Critical Care Medicine.  An official American Thoracic Society/American Association of Critical-Care Nurses/American College of Chest Physicians/Society of Critical Care Medicine policy statement: the Choosing Wisely Top 5 list in critical care medicine.   Am J Respir Crit Care Med. 2014;190(7):818-826. doi:10.1164/rccm.201407-1317STPubMedGoogle ScholarCrossref
18.
Fay  MP.  Confidence intervals that match Fisher’s exact or Blaker’s exact tests.   Biostatistics. 2010;11(2):373-374. doi:10.1093/biostatistics/kxp050PubMedGoogle ScholarCrossref
19.
Fay  MP.  Two-sided exact tests and matching confidence intervals for discrete data.   R J. 2010;2(1):53. doi:10.32614/RJ-2010-008Google ScholarCrossref
20.
Vasher  ST, Zaeh  SE, Eakin  MN, Turnbull  AE.  Responses to a daughter’s question about prognosis when the patient is expected to die: a qualitative analysis.   Ann Am Thorac Soc. 2019;16(12):1595-1598. doi:10.1513/AnnalsATS.201906-435RLPubMedGoogle ScholarCrossref
21.
White  DB, Engelberg  RA, Wenrich  MD, Lo  B, Curtis  JR.  The language of prognostication in intensive care units.   Med Decis Making. 2010;30(1):76-83. doi:10.1177/0272989X08317012PubMedGoogle ScholarCrossref
22.
Ernecoff  NC, Curlin  FA, Buddadhumaruk  P, White  DB.  Health care professionals’ responses to religious or spiritual statements by surrogate decision makers during goals-of-care discussions.   JAMA Intern Med. 2015;175(10):1662-1669. doi:10.1001/jamainternmed.2015.4124PubMedGoogle ScholarCrossref
23.
October  TW, Dizon  ZB, Arnold  RM, Rosenberg  AR.  Characteristics of physician empathetic statements during pediatric intensive care conferences with family members: a qualitative study.   JAMA Netw Open. 2018;1(3):e180351. doi:10.1001/jamanetworkopen.2018.0351PubMedGoogle Scholar
24.
Barr  PJ, Elwyn  G.  Measurement challenges in shared decision making: putting the ‘patient’ in patient-reported measures.   Health Expect. 2016;19(5):993-1001. doi:10.1111/hex.12380PubMedGoogle ScholarCrossref
25.
Kruger  J, Dunning  D.  Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessments.   J Pers Soc Psychol. 1999;77(6):1121-1134. doi:10.1037/0022-3514.77.6.1121PubMedGoogle ScholarCrossref
26.
Kruger  J, Dunning  D.  Unskilled and unaware--but why? a reply to Krueger and Mueller (2002).   J Pers Soc Psychol. 2002;82(2):189-192. doi:10.1037/0022-3514.82.2.189PubMedGoogle ScholarCrossref
27.
Ehrlinger  J, Johnson  K, Banner  M, Dunning  D, Kruger  J.  Why the unskilled are unaware: further explorations of (absent) self-insight among the incompetent.   Organ Behav Hum Decis Process. 2008;105(1):98-121. doi:10.1016/j.obhdp.2007.05.002PubMedGoogle ScholarCrossref
28.
Barnato  AE.  Challenges in understanding and respecting patients’ preferences.   Health Aff (Millwood). 2017;36(7):1252-1257. doi:10.1377/hlthaff.2017.0177PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Original Investigation
    Critical Care Medicine
    May 19, 2020

    在模拟的重症监护病房家庭会议中,医生对共同决策的自我评估

    Author Affiliations
    • 1Department of Internal Medicine, Division of Hospital Medicine at Johns Hopkins Bayview Medical Center, Johns Hopkins University School of Medicine, Baltimore, Maryland
    • 2Division of Pulmonary and Critical Care Medicine, Johns Hopkins University School of Medicine, Baltimore, Maryland
    • 3Division of Geriatric Medicine and Gerontology, Johns Hopkins University School of Medicine, Baltimore, Maryland
    • 4Outcomes After Critical Illness and Surgery (OACIS) Group, Johns Hopkins University School of Medicine, Baltimore, Maryland
    • 5Division of Pulmonary, Critical Care, and Sleep Medicine, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts
    • 6Shapiro Institute for Education and Research at Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts
    • 7Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland
    JAMA Netw Open. 2020;3(5):e205188. doi:10.1001/jamanetworkopen.2020.5188
    关键点 español English

    问题  重症监护医生对专业指南所认可的沟通技能的自我评估是否与盲选的专家同事对相同技能的评估相一致?

    结果  在这项调查中,有 76 位美国重症监护医生阅读了来自重症监护室家庭会议且去除身份标识的文字稿,并对自己在沟通预后、强调选择、提供建议以及提供以舒适为重点的护理方案方面的表现进行评价。在 76 位重症监护医生中,有 61 位表示其沟通了预后,而盲选的同事认同,在这 61 位医生中,有 42 位沟通了患者的死亡风险。

    意义  缺乏专业指南所认可沟通技能的临床医生可能也缺乏识别其缺陷所需的元认知技能,这意味着在所有执业层级,有关沟通和共同决策的日常反馈和继续教育至关重要。

    Abstract

    Importance  Professional guidelines have identified key communication skills for shared decision-making for critically ill patients, but it is unclear how intensivists interpret and implement them.

    Objective  To compare the self-evaluations of intensivists reviewing transcripts of their own simulated intensive care unit family meetings with the evaluations of trained expert colleagues.

    Design, Setting, and Participants  A posttrial web-based survey of intensivists was conducted between January and March 2019. Intensivists reviewed transcripts of simulated intensive care unit family meetings in which they participated in a previous trial from October 2016 to November 2017. In the follow-up survey, participants identified if and how they performed key elements of shared decision-making for an intensive care unit patient at high risk of death. Transcript texts that intensivists self-identified as examples of key communication skills recommended by their professional society’s policy on shared decision-making were categorized.

    Main Outcomes and Measures  Comparison of the evaluations of 2 blinded nonparticipant intensivist colleagues with the self-reported responses of the intensivists.

    Results  Of 116 eligible intensivists, 76 (66%) completed the follow-up survey (mean [SD] respondent age was 43.1 [8.1] years; 72% were male). Sixty-one of 76 intensivists reported conveying prognosis; however, blinded colleagues who reviewed the deidentified transcripts were less likely to report that prognosis had been conveyed than intensivists reviewing their own transcripts (42 of 61; odds ratio, 0.10; 95% CI, 0.01-0.44; P < .001). When reviewing their own transcript, intensivists reported presenting many choices, with the most common choice being code status. They also provided a variety of recommendations, with the most common being to continue the current treatment plan. Thirty-three participants (43%) reported that they offered care focused on comfort, but blinded colleagues rated only 1 (4%) as explaining this option in a clear manner.

    Conclusions and Relevance  In this study, guidelines for shared decision-making and end of life care were interpreted by intensivists in disparate ways. In the absence of training or personalized feedback, self-assessment of communication skills may not be interpreted consistently.

    Introduction

    For 2 decades, treatment in intensive care units (ICUs) during the last month of life has increased among Americans age 65 years and older,1 including among people with advanced dementia who are unlikely to live longer as a consequence of being in an ICU.2 Although intensive care at the end of life is common, many people would prefer to prioritize comfort. To help ensure that critically ill patients and their families can choose care that aligns with their values and goals, critical care societies encourage ICU physicians (intensivists) to practice shared decision-making regarding preference-sensitive interventions.3

    Determining how doctors are interpreting and implementing recommendations about shared decision-making in the ICU presents a challenge. When family meetings are recorded, analyses are usually performed by clinical experts who review written transcripts, audio recordings, or video of family meetings for research purposes.4-12 This method of studying communication is efficient and practical but may not reflect how intensivists experience or interpret a conversation. For a recommendation or guideline to change practice, clinicians must either be able to independently determine whether they are following it or receive feedback on adherence.13

    To understand how intensivists self-evaluate their shared decision-making skills, we asked 116 attending intensivists from US hospitals to review the transcript of a simulated ICU family meeting they participated in as part of a randomized clinical trial.14 Intensivists who agreed identified both if and how they performed key elements of shared decision-making for an ICU patient at high risk of death. We then characterized the text that intensivists self-identified as examples of key communication skills recommended by their professional society’s policy on shared decision-making.3 We also compared intensivist self-evaluation with the evaluations of blinded colleagues reviewing the same transcripts for research purposes.

    Methods
    Study Design and Participants

    We conducted a survey study that followed up intensivists who had participated in the Simulated Communication with ICU Proxies (SCIP) trial (NCT02721810).14 Briefly, SCIP was a randomized clinical trial that enrolled 116 attending intensivists from 17 states to participate in a medical simulation between October 2016 and November 2017. Participants provided written informed consent and received $500 in compensation. Enrolled intensivists individually reviewed the medical record of a hypothetical 81-year-old patient on ICU day 3, with an estimated risk of mortality of 88% based on the Mortality Probability Model II-72 hours15,16 despite appropriate medical management. Each enrolled intensivist then participated in an audio- and video-recorded simulated family meeting with an actor portraying the hypothetical patient’s daughter and surrogate decision maker. Intensivists were instructed to interact with the daughter however they felt was appropriate given the clinical scenario. The actor was instructed to portray a daughter who was passive, had low health literacy, and would accept physician recommendations. Encounter recordings were transcribed verbatim and deidentified. This study was approved by the Johns Hopkins Medicine Institutional Review Board. Completion of the survey served as informed consent. This study followed the American Association for Public Opinion Research (AAPOR) reporting guideline for survey studies.

    Survey Design and Distribution

    In January 2019, at least 1 year after they had participated in the original trial, all SCIP trial participants were recruited via email to complete a follow-up survey. The survey asked the trial participants to identify key elements of shared decision-making in their own transcribed family meeting from the SCIP trial. The elements were drawn from a joint policy statement issued by The American College of Critical Care Medicine and American Thoracic Society in 2016 on shared decision-making in ICUs.3 The policy statement endorsed key communication skills including disclosing prognosis, highlighting choice, and providing a recommendation. Participants’ self-evaluated performance of these guidelines was assessed via the following 3 survey questions: “During the simulation, did you (1) convey prognosis for risk of death? (2) highlight that there is a choice? and (3) provide a recommendation?”

    Additionally, the Critical Care Choosing Wisely Task Force recommends offering the option of care focused entirely on comfort for patients at high risk for death or severely impaired functional recovery.17 To assess how intensivists implemented this recommendation, study participants were asked a fourth question: During the simulation, did you offer the alternative of care focused entirely on comfort? All 4 survey questions had the following response options: done, not done, or not applicable. If an intensivist marked a communication skill as done, they were instructed to mark all text in their transcript corresponding to the performed skill. Intensivists received $50 for returning their questionnaire and annotated transcript.

    We excerpted the text marked in simulation transcripts in response to 2 of the survey questions: (1) During the simulation did you highlight that there is a choice? and (2) During the simulation did you provide a recommendation? Two of us (I.M.O. and E.M.L.) reviewed these excerpts and inductively created categories to describe common response patterns. Category definitions were discussed with all authors and further refined. Differences were resolved through iterative discussion until all authors agreed on definitions for each category. Two of us (S.T.V. and M.M.H.) then reread and categorized all excerpts with some excerpts fitting multiple categories.

    As part of the original SCIP trial analysis the deidentified transcripts from each simulated encounter were reviewed by 2 blinded nonparticipant intensivist colleagues. These colleagues evaluated with a yes or no response whether intensivists conveyed prognosis for risk of death during the simulated family meeting. The colleagues also evaluated the clarity with which intensivists offered the option of care focused on comfort with responses of not offered, not understandable, vague, understandable, or clear. Interreviewer differences were reconciled via discussion.14 We compared these evaluations with intensivists’ self-reported responses to the same questions collected via survey more than a year later.

    Statistical Analysis

    We used descriptive statistics to characterize enrolled participants and summarized deductively coded response categories using counts and proportions. To test whether self-assessment differed from assessments by blinded colleagues when deciding whether prognosis had been conveyed, we estimated an odds ratio using an exact version of the McNemar test18,19 for paired binary response data because of small count size in some cells. A 2-sided P<.05 was considered statistically significant. Data were analyzed using R, version 3.6.0 (R Project for Statistical Computing).

    Results

    The SCIP trial included 116 participants from 17 states.14 Follow-up surveys were completed and returned by 76 (66%) intensivists. The mean (SD) respondent age was 43.1 (8.1), 72% were male, 76% identified as white (non-Hispanic), and first residency was completed a mean (SD) of 13.7 (8.5) years prior to the simulation. There were no statistically significant differences in age, sex, race/ethnicity, or years since training when comparing intensivists who participated in vs those who declined to participate in the follow-up study (Table 1).

    Qualitative Analysis of Highlighting Choice

    Thirty-two intensivists (42%) responded yes to the question, “During the simulation, did you highlight that there is a choice?” and no to the question, “During the simulation, did you offer care focused on comfort?” Representative text marked as highlighting choice by these 32 intensivists is presented in Table 2. The most common choice intensivists offered involved code status (34%). Intensivists commonly presented this choice by contrasting cardiopulmonary resuscitation (CPR) and natural death: “There's um… [a] couple of things we could do if his heart would stop beating. One is we could do CPR. And try to bring him back. The other option is to allow him to… allow him to go naturally if his heart would stop on its own.”

    Less common choices included continuing or adding forms of life support (16%) such as dialysis: “…would he want dialysis, do you think?” Three intensivists ( 9%) marked unclear or ambiguous language as highlighting choice. For example, this excerpt, which is a question to the surrogate, was marked as highlighting the existence of a choice: “Has he ever talked to you about what to do if say, his heart were to stop and he was to need somebody to push on his chest to bring his heart back, has he ever talked to you about what he would want in such a situation?”

    One intensivist (3%) offered the choice of a palliative care consultation. No discernable choice was offered in the text marked by 14 intensivists (44%). However, most of these statements alluded that the daughter might be offered choices in the future if her father’s clinical condition deteriorated further.

    Qualitative Analysis of Providing a Recommendation

    The 33 intensivists (43%) who responded yes to the question, “During the simulation, did you provide a recommendation?” identified a wide range of recommendations in their transcripts (Table 3). The most common recommendation was to continue the current treatment plan (33%). Intensivists either stated they would continue the current plan or recommended a treatment plan to maximize the patient’s chances of survival: “I'd like to try to lighten the sedation and other things that I think we can do that are basically noninvasive and not really big deals. I'd like to do an ultrasound of the heart to try to get an idea about what his fluid status is and we can make a decision about giving fluid, talking with the kidney docs about doing dialysis and trying to take it away.”

    Nine intensivists (27%) recommended withholding CPR: “but, if he's as sick as this and his heart stops, it just wouldn't make sense to do CPR.” Eight intensivists (24%) recommended meeting again in the near future to reassess or make further plans: “Again, we have 24 hours. You and I will [then] sit down again and discuss.” Less common recommendations included specific therapies such as a thoracentesis or dialysis (12%) and a recommendation for palliative care consultation (6%). One intensivist (3%) recommended comfort care and 1 (3%) recommended self-care for the surrogate. In 12 transcripts (36%) reviewed, the recommendation was unclear. Marked text from these manuscripts often used indirect language about end of life care, such as: “Sort of in the midst of thinking about those bigger picture things is whether maybe we should say well, if something sudden happens like that, we just let him pass comfortably, which is really hard to think about.”

    Blinded Colleague Binary Assessment of Conveying Risk of Death (Prognosis)

    After reviewing their own transcripts, 61 intensivists (80%) reported conveying the patient’s risk of death. Blinded colleagues reviewing the same transcripts agreed that 42 (69%) of those 61 had conveyed the patient’s risk of death. Examples of language from transcripts in which there was agreement include, “My concern is that this infection is going to take his life,” and “[there’s a] high chance that he’s not going to make it out of the hospital.” Nineteen intensivists (25%) reported conveying the patient's risk of death but blinded colleagues disagreed, for example: "and because of that and because of the fact that he’s on two different forms of life support, I’m worried about how he’s going to do during this hospitalization. So he’s very, very sick. He’s one of the sicker patients that we have in our ICU.” Two intensivists (3%) who reported that they had not conveyed prognosis for risk of death were rated as having done so by their blinded colleagues. One example of that is, “but, I think it would be important not to ignore that possibility. I'm not sure if he's going to make it through... pull through this one. I'm very concerned about him not being able to pull through this one.” In addition, there was agreement between intensivists and the blinded colleagues that risk of death was not conveyed in 13 transcripts (17%). Blinded colleagues reviewing transcripts were significantly less likely to code that prognosis was conveyed compared with intensivists reviewing their own transcript (odds ratio, 0.10; 95% CI,0.01-0.44; P < .001).

    Blinded Colleague Ordinal Assessment of Offering Care Focused on Comfort

    Twenty-five intensivists (33%) reported offering comfort care during the simulated family meeting after reviewing their own transcript (Table 4). Blinded colleagues reading the same transcripts rated 1 transcript (4%) as containing a clear offer of care focused on comfort. Colleagues rated offers in 4 transcripts (16%) as understandable, 3 (12%) as vague, and 2 (8%) as offering care focused on comfort in a way they did not believe would be understandable to a family surrogate. The remaining 15 transcripts (60%) were rated as not including an offer of care focused on comfort although some of these transcripts included the words comfort or comfortable. For example, 1 intensivist stated: “And if [he gets worse], the only thing I can promise you… we can keep him comfortable, but I cannot promise we can save his life.”

    Discussion

    Attending intensivists in this study reviewed the transcript of a simulated ICU family meeting in which they participated at least 13 months earlier. The simulation scenario involved a patient at high risk of death on ICU day 3. Intensivists were instructed to identify key communication skills for shared decision-making about the care of critically ill patients endorsed by national professional societies. Fewer than half of the intensivists in this study indicated offering a choice or providing a recommendation, and both the nature and framing of choices and recommendations varied widely. Intensivists were also significantly more likely to self-report they conveyed risk of death when compared with blinded colleagues reviewing the same transcripts. In addition, one-third of intensivists reviewing their own transcript reported that they offered care focused exclusively on comfort, but blinded colleagues reading these transcripts disagreed with this assessment more than half the time.

    Measuring shared decision-making is difficult and evaluating decision-making at the end of life is particularly challenging. Landmark studies assessing shared decision-making in the ICU10 as well as how ICU clinicians discuss patient preferences and values6,7 have used trained study personnel to deductively code transcripts of recorded conversations for the presence of specific topics and skills. Instruments and methods for evaluating clinician-patient interactions such as the Observer OPTION 5 scale12 and the Roter Interaction Analysis System,11 are similar in that trained experts with high interrater agreement review audio- or video-recorded encounters and decide whether predefined acts occurred. Qualitative analysis using inductive coding by experts has also been used to develop theories about how intensivists respond to questions about prognosis,20,21 statements about spirituality,22 and empathize23 during family meetings.

    The discrepancies between the communication skills self-identified by intensivists and identified by blinded colleagues in reviewing the same transcripts are notable. It is practical to treat coding by study team members as a reference standard, particularly when they are experts in the field or exhibit high interrater agreement. However, this approach may lack content validity for the physicians enrolled in the study. Moreover, if the goal of evaluating transcripts of patient-clinician interactions is to quantify what was successfully communicated or understood by patients and their proxies, whether research staff and clinicians are capable of assessing this outcome accurately given their familiarity with the topic area and comparative high health literacy is unclear. This question means that expert coding of a transcribed conversation may not reflect what intensivists believe they communicated, or what a patient or their family actually understood.

    Our results suggest that methods that ask doctors to self-evaluate shared decision-making face the same potential compromises to validity as patient-reported measures of shared decision-making, namely that physicians may not recognize the encounter as a decision point or agree that a decision exists and that encounters between clinicians and families of ICU patients often include more than one complex decision.24 Moreover, methods of evaluating physician communication skills that rely on self-report are vulnerable to the Dunning-Kruger effect, in which people who lack a skill also lack the ability to evaluate competence—including their own competence—in that skill.25-27 This effect may also prevent a clinician with underdeveloped communication skills from recognizing when they are not adherent with shared decision-making recommendations.

    Almost half of intensivists in this study reported that they highlighted a choice during the simulation but did not offer the option of care focused on comfort. In about one-third of these transcripts, the intensivists explained that the surrogate might have a choice in the future if her father’s condition deteriorated further. For example: “If he's not getting better or certainly if he's getting worse over the next few days or a week or so, we're going to certainly have to sit down again and decide how to deal with things.” Similarly, one-third of intensivists who reported that they made a recommendation marked text in which they recommended continuing with current treatment, with another 24% recommending that the conversation be reinitiated in the future. This finding suggests that while critical care guidelines endorse providing the option of care focused on comfort,17 intensivists may view these guidelines as applying only after patients have failed lengthy trials of intensive care.28

    Limitations

    This study has limitations. First, 34% of invited intensivists chose not to participate in this follow-up survey. Second, disagreements between self-assessment and blinded colleague assessment could be attributable to the colleagues harboring unusual views and that interrater reliability was not assessed. Third, nonverbal communication is not reflected in a transcript. Fourth, intensivists may behave differently in simulations than in clinical settings. In addition, we asked intensivists rather than patient surrogates to assess what was communicated to a simulated surrogate with low health literacy. Patient surrogates are better qualified to report this outcome and would ideally be used as a reference standard in this scenario. However, using surrogates with varying degrees of health literacy to interpret lengthy transcripts of simulated encounters is expensive and logistically complicated.

    Conclusions

    A year after participating in simulated shared decision-making, intensivists who read a transcript of their simulation were significantly more likely than colleagues reading the same transcript to report they had performed key communication skills endorsed by their professional societies. This finding suggests that evaluations of transcribed clinical interactions for research purposes may not reflect how intensivists experienced or remembered them. More importantly, clinicians who lack communication skills endorsed in professional guidelines may also lack the metacognitive skills required to recognize their deficiencies. In essence, self-assessment may not be possible. This finding makes routine feedback and continuing education on communication and shared decision-making essential at all levels of practice. Without feedback and continuing education, those who would benefit most from supplemental coaching may believe they are already performing recommended skills.

    Back to top
    Article Information

    Accepted for Publication: March 13, 2020.

    Published: May 19, 2020. doi:10.1001/jamanetworkopen.2020.5188

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Vasher ST et al. JAMA Network Open.

    Corresponding Author: Scott T. Vasher, MD, Johns Hopkins Bayview Medical Center, 5200 Eastern Ave, Mason F. Lord East Tower, 2nd Floor, Baltimore, MD 21224 (svasher1@jhmi.edu).

    Author Contributions: Drs Vasher and Turnbull had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Vasher, Hayes, Turnbull.

    Acquisition, analysis, or interpretation of data: All authors.

    Drafting of the manuscript: Vasher.

    Critical revision of the manuscript for important intellectual content: All authors.

    Statistical analysis: Oppenheim, Hayes, Turnbull.

    Obtained funding: Turnbull.

    Administrative, technical, or material support: Oppenheim, Sharma Basyal, Hayes.

    Supervision: Turnbull.

    Conflict of Interest Disclosures: Drs Sharma Basyal and Turnbull reported receiving grants from The Gordon and Betty Moore Foundation during the conduct of the study. Dr Turnbull reported receiving grants from the National Institutes of Health–National Heart, Lung, and Blood Institute during the conduct of the study. No other disclosures were reported.

    Funding/Support: This study was funded by The Gordon and Betty Moore Foundation, supported by the National Heart, Lung, and Blood Institute grant T32HL007534-37 (Dr Oppenheim), by the National Heart, Lung, and Blood Institute grant K01HL141637 (Dr Turnbull), and by The Gordon and Betty Moore Foundation grant 4626 (Dr Turnbull).

    Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Additional Contributions: We thank the intensivists who participated in the original study. We also thank Roy Brower, MD (Johns Hopkins Medicine), and the Johns Hopkins Simulation Center for their assistance.

    References
    1.
    Teno  JM, Gozalo  P, Trivedi  AN,  et al.  Site of death, place of care, and health care transitions among US Medicare beneficiaries, 2000-2015.   JAMA. 2018;320(3):264-271. doi:10.1001/jama.2018.8981PubMedGoogle ScholarCrossref
    2.
    Teno  JM, Gozalo  P, Khandelwal  N,  et al.  Association of increasing use of mechanical ventilation among nursing home residents with advanced dementia and intensive care unit beds.   JAMA Intern Med. 2016;176(12):1809-1816. doi:10.1001/jamainternmed.2016.5964PubMedGoogle ScholarCrossref
    3.
    Kon  AA, Davidson  JE, Morrison  W, Danis  M, White  DB; American College of Critical Care Medicine; American Thoracic Society.  Shared decision making in ICUs: An American College of Critical Care Medicine and American Thoracic Society policy statement.   Crit Care Med. 2016;44(1):188-201. doi:10.1097/CCM.0000000000001396PubMedGoogle ScholarCrossref
    4.
    Heyland  DK, Heyland  R, Dodek  P,  et al; ACCEPT Study Team and the Canadian Researchers at the End of Life Network (CARENET).  Discordance between patients’ stated values and treatment preferences for end-of-life care: results of a multicentre survey.   BMJ Support Palliat Care. 2017;7(3):292-299. doi:10.1136/bmjspcare-2015-001056PubMedGoogle Scholar
    5.
    Krawczyk  M, Gallagher  R.  Communicating prognostic uncertainty in potential end-of-life contexts: experiences of family members.   BMC Palliat Care. 2016;15(1):59. doi:10.1186/s12904-016-0133-4PubMedGoogle ScholarCrossref
    6.
    Scheunemann  LP, Ernecoff  NC, Buddadhumaruk  P,  et al.  Clinician-family communication about patients’ values and preferences in intensive care units.   JAMA Intern Med. 2019;179(5):676-684. doi:10.1001/jamainternmed.2019.0027PubMedGoogle ScholarCrossref
    7.
    Scheunemann  LP, Cunningham  TV, Arnold  RM, Buddadhumaruk  P, White  DB.  How clinicians discuss critically ill patients’ preferences and values with surrogates: an empirical analysis.   Crit Care Med. 2015;43(4):757-764. doi:10.1097/CCM.0000000000000772PubMedGoogle ScholarCrossref
    8.
    Smith  MA, Clayman  ML, Frader  J,  et al.  A descriptive study of decision-making conversations during pediatric intensive care unit family conferences.   J Palliat Med. 2018;21(9):1290-1299. doi:10.1089/jpm.2017.0528PubMedGoogle ScholarCrossref
    9.
    Teno  JM, Fisher  E, Hamel  MB,  et al.  Decision-making and outcomes of prolonged ICU stays in seriously ill patients.   J Am Geriatr Soc. 2000;48(S1):S70-S74. doi:10.1111/j.1532-5415.2000.tb03144.xPubMedGoogle ScholarCrossref
    10.
    White  DB, Braddock  CH  III, Bereknyei  S, Curtis  JR.  Toward shared decision making at the end of life in intensive care units: opportunities for improvement.   Arch Intern Med. 2007;167(5):461-467. doi:10.1001/archinte.167.5.461PubMedGoogle ScholarCrossref
    11.
    Roter  D, Larson  S.  The Roter Interaction Analysis System (RIAS): utility and flexibility for analysis of medical interactions.   Patient Educ Couns. 2002;46(4):243-251. doi:10.1016/S0738-3991(02)00012-5PubMedGoogle ScholarCrossref
    12.
    Elwyn  G, Tsulukidze  M, Edwards  A, Légaré  F, Newcombe  R.  Using a ‘talk’ model of shared decision making to propose an observation-based measure: Observer OPTION 5 Item.   Patient Educ Couns. 2013;93(2):265-271. doi:10.1016/j.pec.2013.08.005PubMedGoogle ScholarCrossref
    13.
    Cabana  MD, Rand  CS, Powe  NR,  et al.  Why don’t physicians follow clinical practice guidelines? a framework for improvement.   JAMA. 1999;282(15):1458-1465. doi:10.1001/jama.282.15.1458PubMedGoogle ScholarCrossref
    14.
    Turnbull  AE, Hayes  MM, Brower  RG,  et al.  Effect of documenting prognosis on the information provided to ICU proxies: a randomized trial.   Crit Care Med. 2019;47(6):757-764. doi:10.1097/CCM.0000000000003731PubMedGoogle ScholarCrossref
    15.
    Lemeshow  S, Teres  D, Klar  J, Avrunin  JS, Gehlbach  SH, Rapoport  J.  Mortality Probability Models (MPM II) based on an international cohort of intensive care unit patients.   JAMA. 1993;270(20):2478-2486. doi:10.1001/jama.1993.03510200084037PubMedGoogle ScholarCrossref
    16.
    Lemeshow  S, Klar  J, Teres  D,  et al.  Mortality probability models for patients in the intensive care unit for 48 or 72 hours: a prospective, multicenter study.   Crit Care Med. 1994;22(9):1351-1358. doi:10.1097/00003246-199409000-00003PubMedGoogle ScholarCrossref
    17.
    Halpern  SD, Becker  D, Curtis  JR,  et al; Choosing Wisely Taskforce; American Thoracic Society; American Association of Critical-Care Nurses; Society of Critical Care Medicine.  An official American Thoracic Society/American Association of Critical-Care Nurses/American College of Chest Physicians/Society of Critical Care Medicine policy statement: the Choosing Wisely Top 5 list in critical care medicine.   Am J Respir Crit Care Med. 2014;190(7):818-826. doi:10.1164/rccm.201407-1317STPubMedGoogle ScholarCrossref
    18.
    Fay  MP.  Confidence intervals that match Fisher’s exact or Blaker’s exact tests.   Biostatistics. 2010;11(2):373-374. doi:10.1093/biostatistics/kxp050PubMedGoogle ScholarCrossref
    19.
    Fay  MP.  Two-sided exact tests and matching confidence intervals for discrete data.   R J. 2010;2(1):53. doi:10.32614/RJ-2010-008Google ScholarCrossref
    20.
    Vasher  ST, Zaeh  SE, Eakin  MN, Turnbull  AE.  Responses to a daughter’s question about prognosis when the patient is expected to die: a qualitative analysis.   Ann Am Thorac Soc. 2019;16(12):1595-1598. doi:10.1513/AnnalsATS.201906-435RLPubMedGoogle ScholarCrossref
    21.
    White  DB, Engelberg  RA, Wenrich  MD, Lo  B, Curtis  JR.  The language of prognostication in intensive care units.   Med Decis Making. 2010;30(1):76-83. doi:10.1177/0272989X08317012PubMedGoogle ScholarCrossref
    22.
    Ernecoff  NC, Curlin  FA, Buddadhumaruk  P, White  DB.  Health care professionals’ responses to religious or spiritual statements by surrogate decision makers during goals-of-care discussions.   JAMA Intern Med. 2015;175(10):1662-1669. doi:10.1001/jamainternmed.2015.4124PubMedGoogle ScholarCrossref
    23.
    October  TW, Dizon  ZB, Arnold  RM, Rosenberg  AR.  Characteristics of physician empathetic statements during pediatric intensive care conferences with family members: a qualitative study.   JAMA Netw Open. 2018;1(3):e180351. doi:10.1001/jamanetworkopen.2018.0351PubMedGoogle Scholar
    24.
    Barr  PJ, Elwyn  G.  Measurement challenges in shared decision making: putting the ‘patient’ in patient-reported measures.   Health Expect. 2016;19(5):993-1001. doi:10.1111/hex.12380PubMedGoogle ScholarCrossref
    25.
    Kruger  J, Dunning  D.  Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessments.   J Pers Soc Psychol. 1999;77(6):1121-1134. doi:10.1037/0022-3514.77.6.1121PubMedGoogle ScholarCrossref
    26.
    Kruger  J, Dunning  D.  Unskilled and unaware--but why? a reply to Krueger and Mueller (2002).   J Pers Soc Psychol. 2002;82(2):189-192. doi:10.1037/0022-3514.82.2.189PubMedGoogle ScholarCrossref
    27.
    Ehrlinger  J, Johnson  K, Banner  M, Dunning  D, Kruger  J.  Why the unskilled are unaware: further explorations of (absent) self-insight among the incompetent.   Organ Behav Hum Decis Process. 2008;105(1):98-121. doi:10.1016/j.obhdp.2007.05.002PubMedGoogle ScholarCrossref
    28.
    Barnato  AE.  Challenges in understanding and respecting patients’ preferences.   Health Aff (Millwood). 2017;36(7):1252-1257. doi:10.1377/hlthaff.2017.0177PubMedGoogle ScholarCrossref
    ×