[Skip to Content]
[Skip to Content Landing]
Figure.  Participant Flow Diagram of Purposively Sampled Sites
Participant Flow Diagram of Purposively Sampled Sites
Table 1.  Purposively Sampled Sites and Host Demographic Characteristics
Purposively Sampled Sites and Host Demographic Characteristics
Table 2.  Characteristics of 380 Participants From Purposively Sampled Sitesa,b
Characteristics of 380 Participants From Purposively Sampled Sitesa,b
Table 3.  Rates of ACP Behavior Among 220 Participants
Rates of ACP Behavior Among 220 Participants
Table 4.  ACP Engagement Survey (34-Item) Results From 220 Participantsa
ACP Engagement Survey (34-Item) Results From 220 Participantsa
1.
Dying in America.  Improving Quality and Honoring Individual Preferences Near the End of Life. The Institute of Medicine; 2014.
2.
LoPresti  MA, Dement  F, Gold  HT.  End-of-life care for people with cancer from ethnic minority groups: a systematic review.   Am J Hosp Palliat Care. 2016;33(3):291-305. doi:10.1177/1049909114565658 PubMedGoogle ScholarCrossref
3.
Mack  JW, Paulk  ME, Viswanath  K, Prigerson  HG.  Racial disparities in the outcomes of communication on medical care received near death.   Arch Intern Med. 2010;170(17):1533-1540. doi:10.1001/archinternmed.2010.322 PubMedGoogle ScholarCrossref
4.
Degenholtz  HB, Arnold  RA, Meisel  A, Lave  JR.  Persistence of racial disparities in advance care plan documents among nursing home residents.   J Am Geriatr Soc. 2002;50(2):378-381. doi:10.1046/j.1532-5415.2002.50073.x PubMedGoogle ScholarCrossref
5.
Connor  SR, Elwert  F, Spence  C, Christakis  NA.  Racial disparity in hospice use in the United States in 2002.   Palliat Med. 2008;22(3):205-213. doi:10.1177/0269216308089305 PubMedGoogle ScholarCrossref
6.
Cohen  LL.  Racial/ethnic disparities in hospice care: a systematic review.   J Palliat Med. 2008;11(5):763-768. doi:10.1089/jpm.2007.0216 PubMedGoogle ScholarCrossref
7.
Sudore  RL, Lum  HD, You  JJ,  et al.  Defining advance care planning for adults: a consensus definition from a multidisciplinary Delphi panel.   J Pain Symptom Manage. 2017;53(5):821-832.e1. doi:10.1016/j.jpainsymman.2016.12.331PubMedGoogle ScholarCrossref
8.
Teno  JM, Gozalo  PL, Bynum  JPW,  et al.  Change in end-of-life care for Medicare beneficiaries: site of death, place of care, and health care transitions in 2000, 2005, and 2009.   JAMA. 2013;309(5):470-477. doi:10.1001/jama.2012.207624 PubMedGoogle ScholarCrossref
9.
Detering  KM, Hancock  AD, Reade  MC, Silvester  W.  The impact of advance care planning on end of life care in elderly patients: randomised controlled trial.   BMJ. 2010;340:c1345. doi:10.1136/bmj.c1345 PubMedGoogle ScholarCrossref
10.
Morrison  RS, Chichin  E, Carter  J, Burack  O, Lantz  M, Meier  DE.  The effect of a social work intervention to enhance advance care planning documentation in the nursing home.   J Am Geriatr Soc. 2005;53(2):290-294. doi:10.1111/j.1532-5415.2005.53116.x PubMedGoogle ScholarCrossref
11.
Casarett  D, Karlawish  J, Morales  K, Crowley  R, Mirsch  T, Asch  DA.  Improving the use of hospice services in nursing homes: a randomized controlled trial.   JAMA. 2005;294(2):211-217. doi:10.1001/jama.294.2.211 PubMedGoogle ScholarCrossref
12.
Kramer  BJ, Kavanaugh  M, Trentham-Dietz  A, Walsh  M, Yonker  JA.  Predictors of family conflict at the end of life: the experience of spouses and adult children of persons with lung cancer.   Gerontologist. 2010;50(2):215-225. doi:10.1093/geront/gnp121 PubMedGoogle ScholarCrossref
13.
Nicholas  LH, Langa  KM, Iwashyna  TJ, Weir  DR.  Regional variation in the association between advance directives and end-of-life Medicare expenditures.   JAMA. 2011;306(13):1447-1453. doi:10.1001/jama.2011.1410 PubMedGoogle ScholarCrossref
14.
Zhang  B, Wright  AA, Huskamp  HA,  et al.  Health care costs in the last week of life: associations with end-of-life conversations.   Arch Intern Med. 2009;169(5):480-488. doi:10.1001/archinternmed.2008.587 PubMedGoogle ScholarCrossref
15.
Angus  DC, Barnato  AE, Linde-Zwirble  WT,  et al; Robert Wood Johnson Foundation ICU End-Of-Life Peer Group.  Use of intensive care at the end of life in the United States: an epidemiologic study.   Crit Care Med. 2004;32(3):638-643. doi:10.1097/01.CCM.0000114816.62331.08 PubMedGoogle ScholarCrossref
16.
Harrison  KL, Adrion  ER, Ritchie  CS, Sudore  RL, Smith  AK.  Low completion and disparities in advance care planning activities among older Medicare beneficiaries.   JAMA Intern Med. 2016;176(12):1872-1875. doi:10.1001/jamainternmed.2016.6751 PubMedGoogle ScholarCrossref
17.
Rao  JK, Anderson  LA, Lin  FC, Laux  JP.  Completion of advance directives among U.S. consumers.   Am J Prev Med. 2014;46(1):65-70. doi:10.1016/j.amepre.2013.09.008 PubMedGoogle ScholarCrossref
18.
Sanders  JJ, Robinson  MT, Block  SD.  Factors impacting advance care planning among African Americans: results of a systematic integrated review.   J Palliat Med. 2016;19(2):202-227. doi:10.1089/jpm.2015.0325 PubMedGoogle ScholarCrossref
19.
Silveira  MJ, Wiitala  W, Piette  J.  Advance directive completion by elderly Americans: a decade of change.   J Am Geriatr Soc. 2014;62(4):706-710. doi:10.1111/jgs.12736 PubMedGoogle ScholarCrossref
20.
Jimenez  G, Tan  WS, Virk  AK, Low  CK, Car  J, Ho  AHY.  Overview of systematic reviews of advance care planning: summary of evidence and global lessons.   J Pain Symptom Manage. 2018;56(3):436-459. doi:10.1016/j.jpainsymman.2018.05.016 PubMedGoogle ScholarCrossref
21.
Ramsaroop  SD, Reid  MC, Adelman  RD.  Completing an advance directive in the primary care setting: what do we need for success?   J Am Geriatr Soc. 2007;55(2):277-283. doi:10.1111/j.1532-5415.2007.01065.x PubMedGoogle ScholarCrossref
22.
Johnson  KS, Kuchibhatla  M, Tulsky  JA.  What explains racial differences in the use of advance directives and attitudes toward hospice care?   J Am Geriatr Soc. 2008;56(10):1953-1958. doi:10.1111/j.1532-5415.2008.01919.x PubMedGoogle ScholarCrossref
23.
Wicher  CP, Meeker  MA.  What influences African American end-of-life preferences?   J Health Care Poor Underserved. 2012;23(1):28-58. doi:10.1353/hpu.2012.0027 PubMedGoogle ScholarCrossref
24.
Ladd  SC.  Systematic review of research literature on African Americans’ end-of-life healthcare preferences.   J Afr Am Stud (New Brunsw). 2014;18(4):373-397. doi:10.1007/s12111-013-9276-z Google ScholarCrossref
25.
Van Scoy  LJ, Reading  JM, Hopkins  M,  et al.  Community game day: using an end-of-life conversation game to encourage advance care planning.   J Pain Symptom Manage. 2017;54(5):680-691. doi:10.1016/j.jpainsymman.2017.07.034 PubMedGoogle ScholarCrossref
26.
Van Scoy  LJ, Green  MJ, Reading  JM, Scott  AM, Chuang  CH, Levi  BH.  Can playing an end-of-life conversation game motivate people to engage in advance care planning?   Am J Hosp Palliat Care. 2017;34(8):754-761. doi:10.1177/1049909116656353 PubMedGoogle ScholarCrossref
27.
Van Scoy  LJ, Reading  JM, Scott  AM, Green  MJ, Levi  BH.  Conversation game effectively engages groups of individuals in discussions about death and dying.   J Palliat Med. 2016;19(6):661-667. doi:10.1089/jpm.2015.0390 PubMedGoogle ScholarCrossref
28.
Radhakrishnan  K, Van Scoy  LJ, Jillapalli  R, Saxena  S, Kim  MT.  Community-based game intervention to improve South Asian Indian Americans’ engagement with advanced care planning.   Ethn Health. 2019;24(6):705-723. doi:10.1080/13557858.2017.1357068PubMedGoogle ScholarCrossref
29.
Van Scoy  LJ, Reading  JM, Scott  AM, Chuang  C, Levi  BH, Green  MJ.  Exploring the topics discussed during a conversation card game about death and dying: a content analysis.   J Pain Symptom Manage. 2016;52(5):655-662. doi:10.1016/j.jpainsymman.2016.03.021 PubMedGoogle ScholarCrossref
30.
National Institute on Minority Health and Health Disparities. Overview: health disparity populations. Accessed March 27, 2020. https://www.nimhd.nih.gov/about/overview/
31.
Van Scoy  LJ, Watson-Martin  E, Bohr  TA, Levi  BH, Green  MJ.  End-of-life conversation game increases confidence for having end-of-life conversations for chaplains-in-training.   Am J Hosp Palliat Care. 2018;35(4):592-600. doi:10.1177/1049909117723619PubMedGoogle ScholarCrossref
32.
Hecht  ML.  The conceptualization and measurement of interpersonal communication satisfaction.   Hum Commun Res. 1978;4(3):253-264. doi:10.1111/j.1468-2958.1978.tb00614.x Google ScholarCrossref
33.
Scott  AM, Caughlin  JP.  Enacted goal attention in family conversations about end-of-life health decisions.   Commun Monogr. 2014;81(3):261-284. doi:10.1080/03637751.2014.925568 Google ScholarCrossref
34.
Marsden  PVSA, Upton  N.  Advocacy Drives Growth: Customer Advocacy Drives UK Business Growth. London School of Economics; 2005.
35.
Reichheld  FF.  The one number you need to grow.   Harv Bus Rev. 2003;81(12):46-54, 124.PubMedGoogle Scholar
36.
Sudore  RL, Fried  TR.  Redefining the “planning” in advance care planning: preparing for end-of-life decision making.   Ann Intern Med. 2010;153(4):256-261. doi:10.7326/0003-4819-153-4-201008170-00008 PubMedGoogle ScholarCrossref
37.
Sudore  RL, Heyland  DK, Barnes  DE,  et al.  Measuring advance care planning: optimizing the advance care planning engagement survey.   J Pain and Symptom Manage. 2017;53(4):669-681. doi:10.1016/j.jpainsymman.2016.10.367PubMedGoogle ScholarCrossref
38.
Sudore  RL, Boscardin  J, Feuz  MA, McMahan  RD, Katen  MT, Barnes  DE.  Effect of the PREPARE website vs an easy-to-read advance directive on advance care planning documentation and engagement among veterans: a randomized clinical trial.   JAMA Intern Med. 2017;177(8):1102-1109. doi:10.1001/jamainternmed.2017.1607 PubMedGoogle ScholarCrossref
39.
Cohen  J.  Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Lawrence Erlbaum Associates; 1988.
40.
Sudore  RL, Stewart  AL, Knight  SJ,  et al.  Development and validation of a questionnaire to detect behavior change in multiple advance care planning behaviors.   PLoS One. 2013;8(9):e72465. doi:10.1371/journal.pone.0072465 PubMedGoogle Scholar
41.
NICE Satmetrix. What is Net Promoter? Accessed October 31, 2019. https://www.netpromoter.com/know/
42.
Glaser  BG.  The constant comparative method of qualitative analysis.   Soc Probl. 1965;12(4):436-445. doi:10.2307/798843 Google ScholarCrossref
43.
Guetterman  TC, Fetters  MD, Creswell  JW.  Integrating quantitative and qualitative results in health science mixed methods research through joint displays.   Ann Fam Med. 2015;13(6):554-561. doi:10.1370/afm.1865 PubMedGoogle ScholarCrossref
44.
Victor  RG, Lynch  K, Li  N,  et al.  A Cluster-randomized trial of blood-pressure reduction in black barbershops.   N Engl J Med. 2018;378(14):1291-1301. doi:10.1056/NEJMoa1717250 PubMedGoogle ScholarCrossref
45.
Fried  TR, Bullock  K, Iannone  L, O’Leary  JR.  Understanding advance care planning as a process of health behavior change.   J Am Geriatr Soc. 2009;57(9):1547-1555. doi:10.1111/j.1532-5415.2009.02396.x PubMedGoogle ScholarCrossref
46.
Schickedanz  AD, Schillinger  D, Landefeld  CS, Knight  SJ, Williams  BA, Sudore  RL.  A clinical framework for improving the advance care planning process: start with patients’ self-identified barriers.   J Am Geriatr Soc. 2009;57(1):31-39. doi:10.1111/j.1532-5415.2008.02093.x PubMedGoogle ScholarCrossref
47.
Edmondson  A.  Psychological safety and learning behavior in work teams.   Adm Sci Q. 1999;44(2):350-383. doi:10.2307/2666999 Google ScholarCrossref
48.
Bischoff  KE, Sudore  R, Miao  Y, Boscardin  WJ, Smith  AK.  Advance care planning and the quality of end-of-life care in older adults.   J Am Geriatr Soc. 2013;61(2):209-214. doi:10.1111/jgs.12105 PubMedGoogle ScholarCrossref
49.
Sudore  RL, Schickedanz  AD, Landefeld  CS,  et al.  Engagement in multiple steps of the advance care planning process: a descriptive study of diverse older adults.   J Am Geriatr Soc. 2008;56(6):1006-1013. doi:10.1111/j.1532-5415.2008.01701.x PubMedGoogle ScholarCrossref
50.
Resnick  HE, Hickman  S, Foster  GL.  Documentation of advance directives among home health and hospice patients: United States, 2007.   Am J Hosp Palliat Care. 2012;29(1):26-35. doi:10.1177/1049909111407627 PubMedGoogle ScholarCrossref
51.
Greenberg  MR, Weiner  M, Greenberg  GB.  Risk-reducing legal documents: controlling personal health and financial resources.   Risk Anal. 2009;29(11):1578-1587. doi:10.1111/j.1539-6924.2009.01282.x PubMedGoogle ScholarCrossref
52.
Koss  CS, Baker  TA.  Race differences in advance directive completion.   J Aging Health. 2017;29(2):324-342. doi:10.1177/0898264316635568 PubMedGoogle ScholarCrossref
53.
Creswell  JW.  Qualitative Inquiry and Research Design: Choosing Among Five Traditions. Sage Publications, Inc; 1998.
54.
Guba  EG, Lincoln  YS.  Effective Evaluation: Improving the Usefulness of Evaluation Results Through Responsive and Naturalistic Approaches. Jossey-Bass; 1981.
55.
Morse  JM.  Critical analysis of strategies for determining rigor in qualitative inquiry.   Qual Health Res. 2015;25(9):1212-1222. doi:10.1177/1049732315588501 PubMedGoogle ScholarCrossref
56.
National Institutes of Health Office of Behavioral and Social Sciences Research. Best practices for mixed methods research in the health sciences. Accessed March 27, 2020. https://obssr.od.nih.gov/wp-content/uploads/2016/02/Best_Practices_for_Mixed_Methods_Research.pdf
57.
Plano Clark  VL.  Meaningful integration within mixed methods studies: identifying why, what, when, and how.   Contemp Educ Psychol. 2019;57:106-111. doi:10.1016/j.cedpsych.2019.01.007Google ScholarCrossref
58.
Fetters  MD, Curry  LA, Creswell  JW.  Achieving integration in mixed methods designs-principles and practices.   Health Serv Res. 2013;48(6 Pt 2):2134-2156. doi:10.1111/1475-6773.12117 PubMedGoogle ScholarCrossref
59.
Creswell  JW, Creswell  JD.  Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Sage publications; 2017.
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Original Investigation
    Ethics
    May 8, 2020

    在非裔美国人中,参与临终对话游戏与预先护理计划行为和看法的关联性

    Author Affiliations
    • 1Department of Medicine, Penn State Milton S. Hershey Medical Center, Penn State College of Medicine, Hershey, Pennsylvania
    • 2Department of Humanities, Penn State Milton S. Hershey Medical Center, Penn State College of Medicine, Hershey, Pennsylvania
    • 3Department of Public Health Sciences, Penn State Milton S. Hershey Medical Center, Penn State College of Medicine, Hershey, Pennsylvania
    • 4Department of Pediatrics, Penn State Milton S. Hershey Medical Center, Penn State College of Medicine, Hershey, Pennsylvania
    • 5Hospice Foundation of America, Washington, DC
    JAMA Netw Open. 2020;3(5):e204315. doi:10.1001/jamanetworkopen.2020.4315
    关键点 español English

    问题  低成本、易于扩展的临终对话游戏是否能激励无法得到充分服务的非裔美国人参与预先护理计划?

    结果  这项全国性的混合方法队列研究涉及 384 名无法得到充分服务的非裔美国人。研究发现,高比例的预先护理计划行为与参与社区场所的游戏活动有关。

    意义  临终对话游戏可能是让无法得到充分服务的非裔美国人社区参与预先护理计划的有用工具,而这个步骤有助于减少与临终护理相关的健康差异。

    Abstract

    Importance  Less than 25% of African American individuals have completed advance directives and are thus vulnerable to poor end-of-life care. Low-cost interventions are needed to increase engagement in advance care planning (ACP).

    Objectives  To investigate whether an end-of-life conversation game motivates African American attendees to engage in ACP and to assess whether the game is well received and endorsed.

    Exposures  Attendance at an end-of-life conversation game (Hello) played in groups of 4 to 6 participants for 60 minutes.

    Design, Setting, and Participants  Prospective, mixed-methods cohort study conducted from 2018 to 2019 with a 3- to 11-month follow-up interview. Game events were held in 53 community venues across the US; 15 were purposively sampled for onsite research procedures. Of 428 attendees at purposively sampled sites, 386 (90%) consented to research procedures (6 attendees were removed from analysis for protocol deviation). Of 367 attendees who provided accurate contact information, 232 (63%) were contacted, and 220 were included in follow-up analyses.

    Main Outcomes and Measures  The primary outcome was advance directive completion rates after the intervention. Secondary outcomes included rates of other ACP behaviors, ACP engagement, conversation satisfaction and realism, and participants’ Net Promoter Score (a measure of endorsement). Follow-up telephone interviews explored the game experience and relevant ACP behaviors of attendees.

    Results  Of 380 individuals who participated (mean [SD] age, 62.2 [13.8] years; 304 were female [80%], and 348 were [92%] African American), none withdrew because of an adverse event. After the intervention, 91 of 220 attendees (41%) completed a new advance directive; 176 of 220 attendees (80%) discussed end-of-life wishes with loved ones, and 214 of 219 attendees (98%) completed at least 1 ACP behavior. There was a moderate increase in the self-efficacy domain on the ACP Engagement Survey (mean [SD] change from before to after the game, 0.54 [0.98]; P < .001). The mean (SD) conversation satisfaction score was 6.21 (0.93) (range, 1-7, with 7 being highest satisfaction), and the overall Net Promoter Score was 57.89 (range, −100 to 100, with 100 being highest endorsement). Interviews revealed 5 themes about the game: (1) it was a useful forum for ACP; (2) it provided new information and perspective; (3) it was emotionally beneficial; (4) it increased appreciation for ACP; and (5) it empowered and motivated participants to perform ACP. Mixed-methods integration showed convergence across data sets.

    Conclusions and Relevance  Among a nationwide sample of African American individuals, the end-of-life conversation game appeared to be well received and was associated with high rates of ACP behavior. This low-cost and scalable tool may help reduce health disparities associated with end-of-life care.

    Introduction

    Underserved populations, particularly African American communities, are vulnerable to low-quality end-of-life care.1 Compared with white individuals living in the United States, African American individuals are less likely to receive end-of-life care aligned with their preferences2-4 and are less likely to receive hospice services.5,6 Such disparities can be addressed in part by advance care planning (ACP)—a process involving conversations about values and preferences for end-of-life care, documentation in advance directives (ADs), and periodic reviews or updates.7 The completion of an AD is associated with reduced unwanted end-of-life medical interventions,8-10 increased hospice use,11 and decreased psychological distress9,12 and may reduce end-of-life costs.13-15

    While the percentage of individuals in the US engaging in ACP has nearly doubled to approximately 60% in the last decade, among African American individuals, it remains stagnant at less than 25%.1,16-19 Most strategies for increasing ACP involve resource-intensive 1-to-1 encounters with clinicians, an approach not easily scaled.20,21 Furthermore, traditional approaches to ACP neglect the 2 most well-documented barriers among underserved populations: mistrust of the health care system22-24 and reluctance to discuss dying.22,24 Our team sought to address these issues by evaluating an inexpensive and easily disseminated intervention—a serious game that promotes ACP conversations by combining an important topic with an enjoyable activity to help overcome reluctance to discuss death and dying.25-27

    In prior research, participants have reported that the game’s open-ended questions prompted in-depth discussions of values and preferences about end-of-life care,25-29 with 98% subsequently performing at least 1 ACP behavior (eg, AD completion or discussing end-of-life issues with loved ones).25,26,28 However, these studies were conducted primarily in white and South Asian communities. The present study examined the feasibility and acceptability of the game in underserved African American populations and explored whether the game empowered them to complete ACP.

    To overcome barriers associated with skepticism about ACP and distrust of the health care system, we developed a pragmatic, community-based delivery model leveraging social networks. We hypothesized that the game would be highly endorsed and engaging for underserved African American communities.

    Methods
    Study Design

    This was a nationally scaled, prospective, mixed-methods cohort study. The primary outcome was completion of a new AD or review or an update of an existing AD within 3 to 11 months after finishing the game. Although a randomized clinical trial may permit conclusions regarding causation, project organizers expressed preferences for giving all participating communities access to the intervention and thus adopted the present mixed-methods cohort design. This study follows the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline for cohort studies and COREQ guidelines for qualitative data and was registered at ClinicalTrials.gov (NCT03456921). The Penn State Hershey Institutional Review Board approved the protocol. Participants provided verbal informed consent, which fulfilled the consent criteria set forth by the institutional review board for this minimal risk study, after reviewing a Summary Explanation of Research. Participants received a $20 gift card for completing the study activities after finishing the game.

    Community-Based Delivery Model and Sampling

    The Hello Project was a national initiative that engaged geographically diverse individuals from underserved communities. Partnering with the nonprofit Hospice Foundation of America, we used a community-based delivery model to host game events. Influential community organizations (eg, places of worship and community centers) were recruited as hosts if they had experience engaging underserved communities. Using press releases and email listservs, we interviewed 63 applicants via telephone; 53 were selected based on community connections and demographic considerations. All hosts represented underserved communities defined by the National Institutes of Health as “including black/African Americans, socioeconomically disadvantaged, and rural communities.”30 On the basis of funder priority and the desire to address unmet ACP needs in these communities, 17 sites in African American communities were purposively sampled for onsite research based on geographic region and hosts’ prior outreach success. Hosts underwent training on running game events and managing logistics (eg, inviting participants, arranging venues, and introducing the game). Two sites were unable to schedule events within the project timeline. Research staff traveled to 15 sites and obtained informed consent and collected data. Taking a conservative approach, we excluded data from nonpurposively sampled sites because potential procedural variations could not be rigorously ruled out.

    Data Management

    Onsite data were collected via paper forms and entered into a secure, electronic REDCap database. Telephone interviews were audiorecorded, and responses were entered into REDCap.

    Setting and Host Sites

    The full scope of the project involved 15 purposively sampled sites and 38 nonpurposively sampled sites in 27 states. In total, 1122 individuals participated from 4 US regions: the Northeastern (n = 8), Southern (n = 24), Midwestern (n = 11), and Western (n = 10) regions. Urban (n = 32) and rural (n = 21) sites were included. Recruitment occurred from May to November 2018; follow-up calls were completed by September 2019. Differences between purposively and nonpurposively sampled sites were procedural in nature because purposively sampled sites had an onsite research assistant (eAppenidx 1 in the Supplement). Only data and procedures from purposively sampled sites are reported hereafter.

    Participant Recruitment

    Hosts advertised events using institutional review board–approved fliers and newsletters. Participants were research eligible if they attended a game event and self-reported being 18 years of age or older. Those who did not speak English or self-reported difficulties with hearing or speaking were excluded from the analysis but were invited to play the game absent research questionnaires.

    Event Procedures

    Before the intervention, the onsite research team administered questionnaires on demographic characteristics, health status, experience with medical decision-making, and ACP engagement. Hosts opened the event with a scripted greeting providing background about ACP and explaining the event agenda and game rules (eTable 1 in the Supplement). The game was then played for 60 minutes in groups of 4 to 6 participants, using a booklet of 32 open-ended questions (published previously).27,29,31 A player read aloud a question (eg, “In order to provide you with the best care possible, what 3 nonmedical facts should your doctor know about you?”). Players wrote answers and took turns sharing with the group (or were allowed to pass). Answers typically prompted free-flowing conversation. Players controlled how long they shared, what they shared, and when they were ready to proceed to the next question. Players could give others a game chip to acknowledge a particularly thoughtful comment. To promote lighthearted competition, a “winner” was named at the end, with a pregame coin flip determining whether the “winner” would be the player with the most or fewest chips revealed at the end.

    Postintervention Measures

    Immediately after the game, participants completed a questionnaire assessing game conversation satisfaction (8-item mean score ranging from 1 to 7, with 7 representing the highest satisfaction).32 Participants also completed a 5-item, validated questionnaire measuring conversation realism (5-item mean score ranging from 1 to 7, with 7 indicating most realistic).33 Intervention endorsement was assessed with the Net Promoter Score (NPS), a validated measure used widely in marketing research to estimate product uptake and recommendation.34,35 The single-item question asked “How likely is it that you would recommend the game to friends or family?” on a scale from 1 (not likely at all) to 10 (extremely likely).

    Follow-up Measures

    Follow-up telephone calls were made 3 to 11 months after each game event to administer questionnaires and conduct an audiorecorded interview about perceptions of the game and relevant actions taken after participating in the game (eAppendix 2 in the Supplement). Participants were contacted starting at 3 months after the intervention. Telephone calls were made until participants were interviewed or declined to be interviewed or 10 calls had been made without contact or 11 months had elapsed since the event. Participants not reached after 10 attempts or 11 months were considered lost to follow-up. The primary outcome was self-reported completion of an AD (defined as any legal document that provides guidance on medical decision-making). Self-report was necessary because review of medical records for AD documentation was not feasible for a study of this scope. Participants who completed ADs prior to the event were asked if they had reviewed or updated their existing AD because periodic review is considered an essential ACP behavior.36 Secondary outcomes included (1) completion of other ACP behaviors (eg, discussing end-of-life wishes with loved ones or clinicians and reviewing ACP resources) and (2) change in score on the 34-item validated ACP Engagement Survey from before the intervention to 3 to 11 months after the intervention.37 The ACP Engagement Survey can detect change in response to ACP interventions.37,38 We defined a moderate, clinically meaningful increase to be a score change of 0.50 to 0.79.38,39

    Statistical Analysis
    Sample Size

    The sample size was chosen in collaboration with the sponsor to reach a diverse, national sample of underserved individuals in the US, with emphasis on African American communities. The target sample size was 50 communities (20-50 participants per site event, anticipating 10% attrition). Owing to resource and staff limitations, 15 of the 53 sites were purposively selected to deploy research staff onsite for data collection. Site enrollment was stratified according to urban or rural area and US region (Table 1).

    Data and Statistical Methods

    The AD completion rates and completion of other ACP behaviors were calculated. The 34-item ACP Engagement Survey consists of 34 items measured on 5-point Likert scale, with an overall mean and 4 domain scores37: knowledge (2 items), contemplation (3 items), self-efficacy (12 items), and readiness (17 items). Scores have strong internal consistency, test-retest reliability, and construct validity and have shown the ability to detect change in ACP behavior.37,40 The Wilcoxon signed rank test was used to assess changes in ACP Engagement Survey scores as the difference between the time of the follow-up call and immediately prior to the event, analyzing only respondents with scores at both time points. The NPS uses a 10-point Likert scale and classifies detractors (1-6), passives (7-8), or promoters (9-10).41 The NPS is calculated by taking the difference between the percentage of promoters and detractors (scores range from −100 to 100). Positive scores greater than 0 indicate positive endorsement. Conversation satisfaction scores were calculated by averaging 8 items from a 7-point Likert scale.32 Conversation realism is a 5-item mean score on a 7-point Likert scale.33

    For all calculated scores, missing items resulted in a missing composite score. Follow-up time was calculated as the number of days between the event and the telephone interview (eAppendix 3 in the Supplement). All analyses were conducted using SAS, version 9.4 (SAS Institute Inc), and 2-sided tests with α = .05 were considered statistically significant.

    Qualitative Analysis

    Thematic analysis was applied to transcribed interviews. Two analysts (L.J.V.S. and A.R.L.) independently reviewed 20% of responses and created categories from the data to form a preliminary codebook. Codes within each category were defined and used to analyze another 20% of responses via the constant comparison method.42 After responses were independently coded, conflicts were reconciled through discussion, and the codebook was finalized. Two analysts (L.J.V.S. and A.R.L.) coded the remaining data and then organized the codes into themes.

    Results

    Table 1 gives host site locations, venues, demographic characteristics, and consent rates at each purposively sampled site. One site was excluded from analysis due to low turnout (6 attendees) and resultant protocol deviation. Of the 1122 event attendees, 428 participated at purposively sampled sites. Of those, 386 attendees (90%) consented to participate in research (minus the 6 removed from analysis; Figure), and 232 of 367 attendees (63%) who provided accurate contact information completed follow-up telephone interviews. The mean (SD) follow-up for the 220 participant interviews (Figure) was 5.4 (1.8) months (median, 4.8 months; interquartile range, 4.0-6.5 months).

    Characteristics of Participants

    Participants’ mean (SD) age was 62.2 (13.8) years, with 304 of 380 participants (80%) being female and 348 of 380 (92%) being African American (Table 2). The characteristics of the participants are also shown by site (eTable 2 in the Supplement), demographic characteristics (urban vs rural), and region (eTable 3 in the Supplement).

    Acceptability Outcomes

    The calculated NPS was positive for all sites, ranging from 5.88 to 90.91, with an overall score of 57.89 (Table 3). The overall mean (SD) raw NPS score was 8.76 (2.02), with the mean score by sites ranging from 7.21 to 9.75. The mean (SD) conversation satisfaction score was 6.21 (0.93), and mean (SD) site scores ranged from 5.58 (0.98) to 6.88 (0.32). The mean (SD) conversation realism score was 5.20 (1.01), and the mean (SD) site scores ranged from 4.86 (1.27) to 5.40 (1.54) (eTable 4 in the Supplement).

    AD Completion Rates and ACP Behaviors

    Table 3 gives rates of ACP behaviors reported at the follow-up telephone call. Of 220 participants, 68 (31%) reported having had an AD prior to the game, 91 (41%) completed a new AD, and 106 (48%) completed a new AD or revised an existing AD. Furthermore, 176 (80%) discussed end-of-life issues with loved ones, 214 of 219 (98%) completed at least 1 ACP behavior, and 145 of 215 (67%) completed 3 or more ACP behaviors. Scores on all domains of the ACP Engagement Survey increased (Table 4 and eTable 5 in the Supplement). There was a moderate and significant increase in the self-efficacy domain (mean [SD] difference before and after the game, 0.54 [0.98]; P < .001) and a small but significant increase in the knowledge (difference, 0.38 [1.24]; P < .001) and readiness (difference, 0.33 [0.98]; P < .001) domains as well as in the total score (difference, 0.40 [0.74]; P < .001). Behavioral rates and scores on the ACP Engagement Survey had no consistent patterns by site, demographic characteristics, or region (eTable 6 and eTable 7 in the Supplement).

    Themes From Telephone Interviews

    Five major themes emerged from participants playing the game (eTable 5 in the Supplement): (1) it was a safe, fun, and enjoyable context for engaging in ACP conversations; (2) it offered new information and perspectives; (3) it was emotionally beneficial; (4) it increased appreciation for both the value and the need for ACP; and (5) it empowered and motivated performance of ACP behaviors. Additional subthemes and representative quotes are given in eTable 5 and eTable 8 in the Supplement.

    A joint display that aligns the quantitative and qualitative results in accordance with this convergent, mixed-methods study is given in eTable 5 in the Supplement.43 We found that the data consistently converged in all 3 constructs of interest: satisfaction with the game, acceptability and endorsement of the experience, and self-efficacy and motivation for behavioral change.

    Discussion

    This national study showed that a low-cost ($2.50/participant) and scalable game intervention may offer a feasible and acceptable approach for engaging underserved African American populations in ACP. Finding new and innovative ways to engage this hard-to-reach community in ACP is a critical first step toward reducing health disparities associated with end-of-life care for underserved populations.1 To our knowledge, this is the largest community-based dissemination of an ACP intervention among underserved African American communities. Our data indicated that the game events were well attended and highly endorsed. These data suggest that the game intervention was not only feasible to implement but also acceptable in African American communities, in which reticence for discussing end-of-life issues has been well documented.7,18,22

    Unlike traditional approaches involving in-person interactions with health care professionals,20 we used a pragmatic delivery model that leveraged community networks. Such an approach is particularly appropriate in communities that may distrust or are less likely to use the health care system. Our qualitative data suggest that participants appreciated having the activity hosted within their social and faith-based communities, which in turn provided opportunities to share and learn from the experiences of trusted peers. Both of these findings highlight the value of addressing ACP within community networks via a trusted community venue (eg, hosting events in places of worship and community centers). This may be particularly salient within African American communities with strong reliance on social networks for information dissemination. The high levels of community engagement may be explained by our community-based delivery model because it sidestepped the need to interact with a distrusted health care system, an approach that has been successfully modeled in other health care initiatives using barbershops.44 Thus, our model using trusted community organizations might be used for health care initiatives beyond ACP whose goal is to engage underserved communities in important health behaviors.

    Our qualitative data also suggested that the game itself may be associated with the success observed in this project. Numerous studies have reported that discussions about death and dying are perceived as unpleasant, uncomfortable, or intimidating.21,45,46 The game overcomes this barrier by reframing these discussions as an enjoyable activity in which players share stories, laugh, and learn from one anothers’ experiences. Using a social, conversation game helps establish psychological safety—the shared belief that individuals in a group can bring up risky topics or ideas.47 Players consistently report that the game creates a safe, nonthreatening environment that supports sensitive conversations.27-29 Furthermore, because the game is engaging and enjoyable and promotes positive reinforcement from the group, it serves to motivate players that may facilitate follow-through with subsequent ACP behaviors.

    It is notable that 80% of participants had end-of-life conversations with loved ones because, even in populations where ACP is more prevalent, rates of end-of-life discussions are only 40% to 60%.16,19,48,49 Although our study was not designed to assess the effectiveness of the intervention with regard to behavior, our finding that 41% of participants completed a new AD is encouraging given the less than 25% baseline rate of AD completion among African American individuals1,16-19,48,50-52 and the much lower rates (13%) reported in other studies with underserved populations.49 That said, the secondary outcome of change in score on the ACP intervention was low in some domains, and the effect sizes on this instrument were small to moderate. Furthermore, in mixed-methods data reported separately, participants reported a low level skepticism and positive attitudes about ACP in general (unpublished data, 2020).

    Although the study was not designed to compare findings across sites, similar rates of behavioral performance and levels of satisfaction and endorsement were observed regardless of site, demographic characteristics, and region. This suggests that use of a serious game may translate well across varied geographic settings.

    Limitations and Strengths

    Given the national scope and the community-based nature of data collection, outcomes relied on self-report, leaving open the potential for social desirability bias and overreporting of ACP behaviors. Visual verification of AD completion would be useful in future studies. Furthermore, in the absence of a randomized clinical trial, it was not possible to infer or speak directly to any causation between the game and the ACP behaviors that followed. To manage potential researcher bias, research assistants with no relationship with the game’s producer collected the data. Finally, our study included predominantly female participants and African American participants, thus findings may not be generalizable to other populations.

    Despite these limitations, this study has several strengths. First, this was a national sample with high rate of recruitment in a population that is traditionally hard to reach. Second, to our knowledge, this project is among the largest to evaluate an ACP intervention in so many communities and regions of the US. Third, the study protocol and analyses closely followed the National Institutes of Health best practices for mixed-methods health research, and our qualitative procedures adhered to published guidelines of methodologic rigor.53-56 Fourth, the consistent and highly convergent quantitative and qualitative data integration increased the validity and reliability of findings.57-59

    Conclusions

    This project successfully engaged a nationwide audience of underserved communities in ACP. The present findings suggest that a serious game may be a feasible and well-received intervention in African American communities. As a low-cost and pragmatic intervention for increasing ACP engagement in underserved African American communities, such a game may help reduce health disparities associated with end-of-life care. Randomized clinical trials are needed to assess its effect on ACP behavioral performance and actual end-of-life care.

    Back to top
    Article Information

    Accepted for Publication: February 20, 2020.

    Published: May 8, 2020. doi:10.1001/jamanetworkopen.2020.4315

    Correction: This article was corrected on June 4, 2020, to change the degree of the sixth author, Irene Putzig, from BA to BS.

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Van Scoy LJ et al. JAMA Network Open.

    Corresponding Author: Lauren Jodi Van Scoy, MD, Department of Medicine, Penn State Milton S. Hershey Medical Center, Penn State College of Medicine, 500 University Dr, H-041, Hershey, PA 17033 (lvanscoy@pennstatehealth.psu.edu).

    Author Contributions: Dr Van Scoy had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Van Scoy, B. H. Levi, Witt, Bramble, Richardson, Putzig, Chinchilli, Tucci.

    Acquisition, analysis, or interpretation of data: Van Scoy, B. H. Levi, Witt, A. R. Levi, Wasserman, Chinchilli, Tucci, Green.

    Drafting of the manuscript: Van Scoy, B. H. Levi, Witt, A. R. Levi, Wasserman, Chinchilli.

    Critical revision of the manuscript for important intellectual content: Van Scoy, B. H. Levi, Witt, Bramble, Richardson, Putzig, A. R. Levi, Tucci, Green.

    Statistical analysis: Van Scoy, Witt, A. R. Levi, Wasserman, Chinchilli.

    Obtained funding: Van Scoy, Tucci.

    Administrative, technical, or material support: Van Scoy, B. H. Levi, Witt, Bramble, Richardson, Putzig, A. R. Levi, Tucci.

    Supervision: Van Scoy, B. H. Levi, Tucci, Green.

    Conflict of Interest Disclosures: Dr Van Scoy reported serving as an unpaid scientific advisor to Common Practice, LLC, which is the company that produces and sells the game used in this study; receiving funding from the National Institutes of Health (NIH), the Canadian Institute of Health Research, the Society for Critical Care Medicine, the Francis Family Foundation, and the Association for Clinical Pastoral Education. Dr B. H. Levi reported receiving funding from the National Institute of Nursing Research and the Children’s Miracle Network during the conduct of the study. Drs B. H. Levi and Green reported receiving personal fees as consultants for Vital Decisions, the parent company that owns the electronic version of the advance directive used in this study (My Living Voice). Drs B. H. Levi and Green are the co-creators of the decision aid, Making Your Wishes Known, which was developed for research purposes and continues to be available free of charge. Any research involving Making Your Wishes Known or My Living Voice is monitored by the Institutional Review Board and Conflict of Interest Review Committee of Penn State. Drs Bramble and Richardson and Ms Putzig reported receiving grants from the Hospice Foundation of America during the conduct of the study. Dr Chinchilli reported receiving funding from the NIH and the Patient Centered Outcomes Research Institute. Dr Green reported receiving funding from the NIH. No other disclosures were reported.

    Funding/Support: Research reported in this project was funded by the John and Wauna Harman Foundation, a private family foundation focused on improving end-of-life care in the US. The REDCap database used in this project was funded by the Penn State Clinical and Translational Science Institute, Pennsylvania State University Clinical and Translational Science Award Network (UL1 TR002014), and the National Center for Advancing Translational Sciences grant UL1 TR000127.

    Role of the Funder/Sponsor: The John and Wauna Harman Foundation was involved in the design of the study, but was not involved in collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. No other funder had any role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Additional Contributions: The authors thank The Hospice Foundation of America, the Hello Project Advisory Board, all the host organizations who participated in the Hello Project nationwide, and Common Practice, LLC, for permission to use their game Hello, which is available online. Data collection, entry, and management assistance were provided by Katherine Callahan, PhD candidate, Kayla Confer, MEd, James Harness, BA, Margaret Hopkins, DEd, Yining Ma, MS, Sara Marlin, MS, Timothy Sheehan, MS, and Xingran Wren, Dr Ph candidate, all affiliated with Penn State University, Penn State College of Medicine; and Lindsay Currin, BA, Meghan Lee, MA, and Nicole Matluck, MS, all affiliated with Hospice Foundation of America. Andrew Foy, MD, Penn State Milton S. Hershey Medical Center, provided critical review of the manuscript. All assistants received financial compensation except Drs Foy and Hopkins, Katherine Callahan, Yining Ma, and Xingran Wren. Vital Decisions provided permission to reproduce paper copies of My Living Voice.

    References
    1.
    Dying in America.  Improving Quality and Honoring Individual Preferences Near the End of Life. The Institute of Medicine; 2014.
    2.
    LoPresti  MA, Dement  F, Gold  HT.  End-of-life care for people with cancer from ethnic minority groups: a systematic review.   Am J Hosp Palliat Care. 2016;33(3):291-305. doi:10.1177/1049909114565658 PubMedGoogle ScholarCrossref
    3.
    Mack  JW, Paulk  ME, Viswanath  K, Prigerson  HG.  Racial disparities in the outcomes of communication on medical care received near death.   Arch Intern Med. 2010;170(17):1533-1540. doi:10.1001/archinternmed.2010.322 PubMedGoogle ScholarCrossref
    4.
    Degenholtz  HB, Arnold  RA, Meisel  A, Lave  JR.  Persistence of racial disparities in advance care plan documents among nursing home residents.   J Am Geriatr Soc. 2002;50(2):378-381. doi:10.1046/j.1532-5415.2002.50073.x PubMedGoogle ScholarCrossref
    5.
    Connor  SR, Elwert  F, Spence  C, Christakis  NA.  Racial disparity in hospice use in the United States in 2002.   Palliat Med. 2008;22(3):205-213. doi:10.1177/0269216308089305 PubMedGoogle ScholarCrossref
    6.
    Cohen  LL.  Racial/ethnic disparities in hospice care: a systematic review.   J Palliat Med. 2008;11(5):763-768. doi:10.1089/jpm.2007.0216 PubMedGoogle ScholarCrossref
    7.
    Sudore  RL, Lum  HD, You  JJ,  et al.  Defining advance care planning for adults: a consensus definition from a multidisciplinary Delphi panel.   J Pain Symptom Manage. 2017;53(5):821-832.e1. doi:10.1016/j.jpainsymman.2016.12.331PubMedGoogle ScholarCrossref
    8.
    Teno  JM, Gozalo  PL, Bynum  JPW,  et al.  Change in end-of-life care for Medicare beneficiaries: site of death, place of care, and health care transitions in 2000, 2005, and 2009.   JAMA. 2013;309(5):470-477. doi:10.1001/jama.2012.207624 PubMedGoogle ScholarCrossref
    9.
    Detering  KM, Hancock  AD, Reade  MC, Silvester  W.  The impact of advance care planning on end of life care in elderly patients: randomised controlled trial.   BMJ. 2010;340:c1345. doi:10.1136/bmj.c1345 PubMedGoogle ScholarCrossref
    10.
    Morrison  RS, Chichin  E, Carter  J, Burack  O, Lantz  M, Meier  DE.  The effect of a social work intervention to enhance advance care planning documentation in the nursing home.   J Am Geriatr Soc. 2005;53(2):290-294. doi:10.1111/j.1532-5415.2005.53116.x PubMedGoogle ScholarCrossref
    11.
    Casarett  D, Karlawish  J, Morales  K, Crowley  R, Mirsch  T, Asch  DA.  Improving the use of hospice services in nursing homes: a randomized controlled trial.   JAMA. 2005;294(2):211-217. doi:10.1001/jama.294.2.211 PubMedGoogle ScholarCrossref
    12.
    Kramer  BJ, Kavanaugh  M, Trentham-Dietz  A, Walsh  M, Yonker  JA.  Predictors of family conflict at the end of life: the experience of spouses and adult children of persons with lung cancer.   Gerontologist. 2010;50(2):215-225. doi:10.1093/geront/gnp121 PubMedGoogle ScholarCrossref
    13.
    Nicholas  LH, Langa  KM, Iwashyna  TJ, Weir  DR.  Regional variation in the association between advance directives and end-of-life Medicare expenditures.   JAMA. 2011;306(13):1447-1453. doi:10.1001/jama.2011.1410 PubMedGoogle ScholarCrossref
    14.
    Zhang  B, Wright  AA, Huskamp  HA,  et al.  Health care costs in the last week of life: associations with end-of-life conversations.   Arch Intern Med. 2009;169(5):480-488. doi:10.1001/archinternmed.2008.587 PubMedGoogle ScholarCrossref
    15.
    Angus  DC, Barnato  AE, Linde-Zwirble  WT,  et al; Robert Wood Johnson Foundation ICU End-Of-Life Peer Group.  Use of intensive care at the end of life in the United States: an epidemiologic study.   Crit Care Med. 2004;32(3):638-643. doi:10.1097/01.CCM.0000114816.62331.08 PubMedGoogle ScholarCrossref
    16.
    Harrison  KL, Adrion  ER, Ritchie  CS, Sudore  RL, Smith  AK.  Low completion and disparities in advance care planning activities among older Medicare beneficiaries.   JAMA Intern Med. 2016;176(12):1872-1875. doi:10.1001/jamainternmed.2016.6751 PubMedGoogle ScholarCrossref
    17.
    Rao  JK, Anderson  LA, Lin  FC, Laux  JP.  Completion of advance directives among U.S. consumers.   Am J Prev Med. 2014;46(1):65-70. doi:10.1016/j.amepre.2013.09.008 PubMedGoogle ScholarCrossref
    18.
    Sanders  JJ, Robinson  MT, Block  SD.  Factors impacting advance care planning among African Americans: results of a systematic integrated review.   J Palliat Med. 2016;19(2):202-227. doi:10.1089/jpm.2015.0325 PubMedGoogle ScholarCrossref
    19.
    Silveira  MJ, Wiitala  W, Piette  J.  Advance directive completion by elderly Americans: a decade of change.   J Am Geriatr Soc. 2014;62(4):706-710. doi:10.1111/jgs.12736 PubMedGoogle ScholarCrossref
    20.
    Jimenez  G, Tan  WS, Virk  AK, Low  CK, Car  J, Ho  AHY.  Overview of systematic reviews of advance care planning: summary of evidence and global lessons.   J Pain Symptom Manage. 2018;56(3):436-459. doi:10.1016/j.jpainsymman.2018.05.016 PubMedGoogle ScholarCrossref
    21.
    Ramsaroop  SD, Reid  MC, Adelman  RD.  Completing an advance directive in the primary care setting: what do we need for success?   J Am Geriatr Soc. 2007;55(2):277-283. doi:10.1111/j.1532-5415.2007.01065.x PubMedGoogle ScholarCrossref
    22.
    Johnson  KS, Kuchibhatla  M, Tulsky  JA.  What explains racial differences in the use of advance directives and attitudes toward hospice care?   J Am Geriatr Soc. 2008;56(10):1953-1958. doi:10.1111/j.1532-5415.2008.01919.x PubMedGoogle ScholarCrossref
    23.
    Wicher  CP, Meeker  MA.  What influences African American end-of-life preferences?   J Health Care Poor Underserved. 2012;23(1):28-58. doi:10.1353/hpu.2012.0027 PubMedGoogle ScholarCrossref
    24.
    Ladd  SC.  Systematic review of research literature on African Americans’ end-of-life healthcare preferences.   J Afr Am Stud (New Brunsw). 2014;18(4):373-397. doi:10.1007/s12111-013-9276-z Google ScholarCrossref
    25.
    Van Scoy  LJ, Reading  JM, Hopkins  M,  et al.  Community game day: using an end-of-life conversation game to encourage advance care planning.   J Pain Symptom Manage. 2017;54(5):680-691. doi:10.1016/j.jpainsymman.2017.07.034 PubMedGoogle ScholarCrossref
    26.
    Van Scoy  LJ, Green  MJ, Reading  JM, Scott  AM, Chuang  CH, Levi  BH.  Can playing an end-of-life conversation game motivate people to engage in advance care planning?   Am J Hosp Palliat Care. 2017;34(8):754-761. doi:10.1177/1049909116656353 PubMedGoogle ScholarCrossref
    27.
    Van Scoy  LJ, Reading  JM, Scott  AM, Green  MJ, Levi  BH.  Conversation game effectively engages groups of individuals in discussions about death and dying.   J Palliat Med. 2016;19(6):661-667. doi:10.1089/jpm.2015.0390 PubMedGoogle ScholarCrossref
    28.
    Radhakrishnan  K, Van Scoy  LJ, Jillapalli  R, Saxena  S, Kim  MT.  Community-based game intervention to improve South Asian Indian Americans’ engagement with advanced care planning.   Ethn Health. 2019;24(6):705-723. doi:10.1080/13557858.2017.1357068PubMedGoogle ScholarCrossref
    29.
    Van Scoy  LJ, Reading  JM, Scott  AM, Chuang  C, Levi  BH, Green  MJ.  Exploring the topics discussed during a conversation card game about death and dying: a content analysis.   J Pain Symptom Manage. 2016;52(5):655-662. doi:10.1016/j.jpainsymman.2016.03.021 PubMedGoogle ScholarCrossref
    30.
    National Institute on Minority Health and Health Disparities. Overview: health disparity populations. Accessed March 27, 2020. https://www.nimhd.nih.gov/about/overview/
    31.
    Van Scoy  LJ, Watson-Martin  E, Bohr  TA, Levi  BH, Green  MJ.  End-of-life conversation game increases confidence for having end-of-life conversations for chaplains-in-training.   Am J Hosp Palliat Care. 2018;35(4):592-600. doi:10.1177/1049909117723619PubMedGoogle ScholarCrossref
    32.
    Hecht  ML.  The conceptualization and measurement of interpersonal communication satisfaction.   Hum Commun Res. 1978;4(3):253-264. doi:10.1111/j.1468-2958.1978.tb00614.x Google ScholarCrossref
    33.
    Scott  AM, Caughlin  JP.  Enacted goal attention in family conversations about end-of-life health decisions.   Commun Monogr. 2014;81(3):261-284. doi:10.1080/03637751.2014.925568 Google ScholarCrossref
    34.
    Marsden  PVSA, Upton  N.  Advocacy Drives Growth: Customer Advocacy Drives UK Business Growth. London School of Economics; 2005.
    35.
    Reichheld  FF.  The one number you need to grow.   Harv Bus Rev. 2003;81(12):46-54, 124.PubMedGoogle Scholar
    36.
    Sudore  RL, Fried  TR.  Redefining the “planning” in advance care planning: preparing for end-of-life decision making.   Ann Intern Med. 2010;153(4):256-261. doi:10.7326/0003-4819-153-4-201008170-00008 PubMedGoogle ScholarCrossref
    37.
    Sudore  RL, Heyland  DK, Barnes  DE,  et al.  Measuring advance care planning: optimizing the advance care planning engagement survey.   J Pain and Symptom Manage. 2017;53(4):669-681. doi:10.1016/j.jpainsymman.2016.10.367PubMedGoogle ScholarCrossref
    38.
    Sudore  RL, Boscardin  J, Feuz  MA, McMahan  RD, Katen  MT, Barnes  DE.  Effect of the PREPARE website vs an easy-to-read advance directive on advance care planning documentation and engagement among veterans: a randomized clinical trial.   JAMA Intern Med. 2017;177(8):1102-1109. doi:10.1001/jamainternmed.2017.1607 PubMedGoogle ScholarCrossref
    39.
    Cohen  J.  Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Lawrence Erlbaum Associates; 1988.
    40.
    Sudore  RL, Stewart  AL, Knight  SJ,  et al.  Development and validation of a questionnaire to detect behavior change in multiple advance care planning behaviors.   PLoS One. 2013;8(9):e72465. doi:10.1371/journal.pone.0072465 PubMedGoogle Scholar
    41.
    NICE Satmetrix. What is Net Promoter? Accessed October 31, 2019. https://www.netpromoter.com/know/
    42.
    Glaser  BG.  The constant comparative method of qualitative analysis.   Soc Probl. 1965;12(4):436-445. doi:10.2307/798843 Google ScholarCrossref
    43.
    Guetterman  TC, Fetters  MD, Creswell  JW.  Integrating quantitative and qualitative results in health science mixed methods research through joint displays.   Ann Fam Med. 2015;13(6):554-561. doi:10.1370/afm.1865 PubMedGoogle ScholarCrossref
    44.
    Victor  RG, Lynch  K, Li  N,  et al.  A Cluster-randomized trial of blood-pressure reduction in black barbershops.   N Engl J Med. 2018;378(14):1291-1301. doi:10.1056/NEJMoa1717250 PubMedGoogle ScholarCrossref
    45.
    Fried  TR, Bullock  K, Iannone  L, O’Leary  JR.  Understanding advance care planning as a process of health behavior change.   J Am Geriatr Soc. 2009;57(9):1547-1555. doi:10.1111/j.1532-5415.2009.02396.x PubMedGoogle ScholarCrossref
    46.
    Schickedanz  AD, Schillinger  D, Landefeld  CS, Knight  SJ, Williams  BA, Sudore  RL.  A clinical framework for improving the advance care planning process: start with patients’ self-identified barriers.   J Am Geriatr Soc. 2009;57(1):31-39. doi:10.1111/j.1532-5415.2008.02093.x PubMedGoogle ScholarCrossref
    47.
    Edmondson  A.  Psychological safety and learning behavior in work teams.   Adm Sci Q. 1999;44(2):350-383. doi:10.2307/2666999 Google ScholarCrossref
    48.
    Bischoff  KE, Sudore  R, Miao  Y, Boscardin  WJ, Smith  AK.  Advance care planning and the quality of end-of-life care in older adults.   J Am Geriatr Soc. 2013;61(2):209-214. doi:10.1111/jgs.12105 PubMedGoogle ScholarCrossref
    49.
    Sudore  RL, Schickedanz  AD, Landefeld  CS,  et al.  Engagement in multiple steps of the advance care planning process: a descriptive study of diverse older adults.   J Am Geriatr Soc. 2008;56(6):1006-1013. doi:10.1111/j.1532-5415.2008.01701.x PubMedGoogle ScholarCrossref
    50.
    Resnick  HE, Hickman  S, Foster  GL.  Documentation of advance directives among home health and hospice patients: United States, 2007.   Am J Hosp Palliat Care. 2012;29(1):26-35. doi:10.1177/1049909111407627 PubMedGoogle ScholarCrossref
    51.
    Greenberg  MR, Weiner  M, Greenberg  GB.  Risk-reducing legal documents: controlling personal health and financial resources.   Risk Anal. 2009;29(11):1578-1587. doi:10.1111/j.1539-6924.2009.01282.x PubMedGoogle ScholarCrossref
    52.
    Koss  CS, Baker  TA.  Race differences in advance directive completion.   J Aging Health. 2017;29(2):324-342. doi:10.1177/0898264316635568 PubMedGoogle ScholarCrossref
    53.
    Creswell  JW.  Qualitative Inquiry and Research Design: Choosing Among Five Traditions. Sage Publications, Inc; 1998.
    54.
    Guba  EG, Lincoln  YS.  Effective Evaluation: Improving the Usefulness of Evaluation Results Through Responsive and Naturalistic Approaches. Jossey-Bass; 1981.
    55.
    Morse  JM.  Critical analysis of strategies for determining rigor in qualitative inquiry.   Qual Health Res. 2015;25(9):1212-1222. doi:10.1177/1049732315588501 PubMedGoogle ScholarCrossref
    56.
    National Institutes of Health Office of Behavioral and Social Sciences Research. Best practices for mixed methods research in the health sciences. Accessed March 27, 2020. https://obssr.od.nih.gov/wp-content/uploads/2016/02/Best_Practices_for_Mixed_Methods_Research.pdf
    57.
    Plano Clark  VL.  Meaningful integration within mixed methods studies: identifying why, what, when, and how.   Contemp Educ Psychol. 2019;57:106-111. doi:10.1016/j.cedpsych.2019.01.007Google ScholarCrossref
    58.
    Fetters  MD, Curry  LA, Creswell  JW.  Achieving integration in mixed methods designs-principles and practices.   Health Serv Res. 2013;48(6 Pt 2):2134-2156. doi:10.1111/1475-6773.12117 PubMedGoogle ScholarCrossref
    59.
    Creswell  JW, Creswell  JD.  Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Sage publications; 2017.
    ×