Developing a Patient Classification System for a

JONA Volume 46, Number 12, pp 636-641 Copyright B 2016 Wolters Kluwer Health, Inc. All rights reserved.

THE JOURNAL OF NURSING ADMINISTRATION

Developing a Patient Classification System for a Neonatal ICU Nancy M. Daraiseh, PhD William P. Vidonish, MBA

Pam Kiessling, MSN, RN Li Lin, MS

OBJECTIVE: The purpose of this study was to develop a valid and reliable patient classification system (PCS) for a neonatal ICU (NICU). BACKGROUND: PCSs have been widely used to determine required care hours, budgeting, and staffing. There is a lack of and a vital need for a valid and reliable pediatric PCS because of differences in needs and treatment from adults. METHODS: Data were collected in a NICU using work sampling, chart reviews, and expert opinion. The resulting PCS was assessed for validity and reliability, ease of use, effectiveness, and satisfaction. RESULTS: The PCS showed significantly high reliability and validity. Survey scores revealed nurses perceived the tool to be easy to use and effective. CONCLUSIONS: Using subjective and objective methods, a NICU PCS was shown to be a valid and reliable measure to determine the hours per patient day required to provide care.

staff in relation to the provision of needed care, treatment, and services[1(p439) and requires organizations to Buse data to assess and continuously improve staffing effectiveness.[1(p322) The development of an accurate and reliable patient classification system (PCS) will ensure hospitals meet this standard by providing a process by which needed care, treatment, and services can be quantified. Without such a measure, needed is a subjective estimate, which can be dramatically influenced by the experiences, expertise, interpretation, and even intuition of the person making the estimate. A calculation using nursing hours per patient day (NHPPDs) or the amount of nursing time that a patient needs for a 24-hour period, provides an objective statement of patient care needs on which to base the budget for RNs. Assessing NHPPDs at the patient level allows actual or anticipated changes in the patient mix to be appropriately reflected in staff planning. Thirteen states and the District of Columbia have enacted legislation and/or adopted regulations to address nurse staffing.2 Seven states require hospitals to have staffing committees responsible for plans and staffing policy. Hospitals seeking Magnet 3 recognition will benefit from utilizing a tool that accurately reflects their patients_ needs and staffing requirements, includes nurses in the determination of both, and focuses on the quality of care. Developing a PCS addresses 3 components of the Magnet Model : structural empowerment, exemplary professional practice, and new knowledge.3 PCSs for nursing have been used to determine required patient care hours, budgeting, productivity measurement, and staffing efficiency.4-8 In recent years, the nursing shortage and a focus on patient safety have led TJC to focus standards on staffing efficiency, thus increasing the need for a valid measurement.1 A common notion is that by increasing the number

Providing adequate nursing staff is critical to providing quality care, which must include the personal and evidence-based needs of hospitalized patients. The Joint Commission (TJC), an accrediting agency for healthcare organizations, defines staffing effectiveness as Bthe number, competency, and skill mix of Author Affiliations: Assistant Professor (Dr Daraiseh), Research in Patient Services and James M. Anderson Center for Health Systems Excellence; Project Manager (Mr Vidonish and Ms Keissling), Center for Professional Excellence; Statistician, Research in Patient Services (Ms Lin), Cincinnati Children_s Hospital Medical Center, Ohio. The authors declare no conflicts of interest. Correspondence: Dr Daraiseh, Cincinnati Children_s Hospital Medical Center, MLC 7014, 3333 Burnet Ave, Cincinnati, OH 45229 ([email protected]). Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal_s Web site (www.jonajournal.com). DOI: 10.1097/NNA.0000000000000419

636

A

A

JONA Vol. 46, No. 12 December 2016

Copyright © 2016 Wolters Kluwer Health, Inc. All rights reserved.

of RNs, staffing and efficiency will improve. Research suggests that the focus should be placed on providing accurate staffing and allocation to achieve any clinically significant association.9-11 Studies have investigated the impact of staffing levels on patient outcomes, for example, pneumonia, pressure ulcers, and infections.10,12-18 An increase in the number of full-time staff has shown to improve patient outcomes,16,19 but Kane et al14 found no consistent evidence that increasing full-time RNs could improve patient safety. Blegen et al9 found that negative outcomes were lower with an increase in RN care, but as care increased beyond a particular level (87.5%), negative outcomes also increased. Inadequate staffing impacts more than the quality of patient care,20 as revealed by the American Nurses Association21 that 87.9% of nurses indicated their own health and safety concerns have an impact on their performance and continued practice. Understaffing is a major contributor to job satisfaction and burnout,12,20 causing nurses to leave their jobs and reducing their time to comfort, assist, and educate patients and families.20 More recently, Leigh et al22 found that California-mandated minimum staffing ratios were associated with 55.57% fewer occupational injuries and illnesses. PCSs have been used extensively,4-8 but there is no current standard for their development or design. In a 2005 survey of children_s hospitals, only 4 of 45 hospitals responding used a single commercially available system.23 While efforts have been made to incorporate pediatric concerns into commercially developed systems, they are designed primarily for adult patients. Consequently, most children_s hospitals use in-house systems to describe the unique needs of children and patient mixes. We report here on the development of a PCS to replace the outdated model used in a neonatal ICU (NICU). Because multiple data collection modalities are required to sufficiently capture the full range of RN work,24,25 investigators used work sampling, chart reviews, and expert opinion. Data were validated with RN experts throughout the development process and used to create a PCS that accurately and reliably classifies patients into categories representing the HPPD required for care. At the time of this study, institutional policy required that charge nurses (CNs) complete patient classification. However, because bedside RNs have immediate knowledge of patients, there was interest in allowing them to complete the assessment to increase accuracy and improve safety. Therefore, in addition to effectiveness and ease of use of the new PCS, the research team assessed interrater reliability between RNs and CNs; high estimates would support allowing bedside RNs to complete the classification.

This project improves upon previous studies by incorporating multimodal methodologies that include both objective (work sampling, chart review) and subjective measures (clinician expertise) to measure workload.26 Work sampling is a statistical technique for determining the proportion of time spent in various activities27 and is effective for measuring highvariability jobs more accurately than self-reports.25,28,29 This method has been used extensively to measure the work of care providers.28-31 This study uses continuous real-time work sampling where previous studies used only intermittent sampling32 or relied only on expert opinion.33-35 Continuous sampling is more suitable to capture the diverse and unpredictable activities of bedside RNs, allowing observers to record natural fluctuations in patient flow and shift activity.

Methods Setting This study was conducted at a Midwestern pediatric hospital with a 59-bed NICU. Approval was obtained by the hospital_s institutional review board. Workload Study To develop a patient-specific PCS based on weighted work indicators that represent RN care, measuring workload centered on developing standard times for the various RN activities. A database of work activities was developed by an interdisciplinary team of experts in nursing, PCSs, work measurement, safety, and project management consisting of 2 scholars, an industrial engineer, and 3 expert RNs. To cover all care activities, direct care, indirect care, and hospital duties were included. Direct care was defined as activities where the patient had to be involved (eg, laboratory tests, medication administration). Indirect care was performed for a patient, but patient involvement was not necessary (eg, documentation, chart review), and hospital tasks did not affect a patient directly (eg, training). RN experts were then asked to rank each activity by importance and estimate duration and frequency to establish face and content validity. The resulting database included more than 15 000 nursing activities, reduced to 300 after categorization. Work Sampling Nursing students with at least 2 years of clinical experience observed 48 randomly chosen RNs for 4 hours. A 90% interrater agreement on observation procedures was achieved prior to and during the study using videos of nursing activities. Four-hour observations continued to cumulate and represent a continuous 24-hour day. The observers used a PDA program to record activity time and frequency and



637

manually documented demographics and any activities not in the PDA. Thirty-seven randomly selected patients (assigned to RN participants) were observed for 24 hours to obtain the per-patient frequency of each nursing activity. Medical records were reviewed to validate the data. Tool Development Activity time and frequency were combined to set a time standard of care for each patient over a 24-hour period, that is, NHPPDs, which were validated by 2 expert RNs. If a value was questioned, the activity was reobserved for validation. Each activity was then ranked on its criticality to nursing care. Higherranking activities of care priority were assigned a higher weight. Work that might not be time consuming, but represented a higher level of acuity (eg, critical tubes) would increase NHPPDs. The team worked to cluster the weighted activities into tool indicators that would separate patients into NHPPD groups. RNs were observed during all activities related to patient care, including documentation, plan-of-care meetings and development, collaborations, and rounds. Although these activities consume large portions of time, they were carried out for all NICU patients and considered part of baseline care. Therefore, indicators in the final tool significantly contributed to variability by increasing the time required for care. The final PCS segmented each patient into 5 predefined NHPPD classifications. Psychometrics NICU RNs received training on the PCS, and testing of interrater reliability and validity was conducted over a 2-week period. Forty-nine patients were randomized prior to shift change, and their assigned RN pairs (oncoming and outgoing) independently classified the same patient. CNs also participated for comparison. The resulting acuity and NHPPDs were compared with expert opinion for face validity. An online survey assessed ease of use, effectiveness, and satisfaction.

Analysis Mean time, frequency, and NHPPD standards were calculated, and intraclass correlation (ICC) was used to compute reliability coefficients. The cutoff for the ICC was 0.75. Pearson correlation coefficients compared patient acuity based on expert RN judgment using a 1- to 5-point scale (1 = lowest) and the overall score from the PCS.

Results The final PCS included 62 items in 9 categories (Table 1 and Document, Supplemental Digital Content 1, http://links.lww.com/JONA/A495). Bedside RNs complete the PCS at 1:00 AM and 1:00 PM and is embedded in the electronic medical record, taking 1 to 2 minutes to complete. The final score results in 1 of 5 levels of care corresponding to a total NHPPDs and associated RN-to-patient ratio. NHPPDs are aggregated to the unit level and are used as a key metric to assess nursing demand. This demand is compared with staff availability to deploy necessary resources. High-acuity patients with a critical tube/airway or requiring extracorporeal membrane oxygenation are assigned a 1:1 and 2:1 RN-to-patient ratio, respectively, regardless of other required care. Interrater Reliability Table 2 shows reliability estimates for 98 observations (n = 48 RN pairs) on 49 patients. Validity Scoring included 42 patients (1 exclusion due to incomplete data); 38% were female with an average gestational age of 34.7 (SD, 4.4) weeks (range, 2-40 weeks). Results are in Table 3. Ease of Use, Effectiveness, and Satisfaction On a scale from 0 (very difficult) to 10 (very easy), 50 respondents (79.4%, n = 63) rated their ability to complete the tool 7 to 10. On a scale from 0 (very ineffective) to 10 (very effective), 71.4% rated the

Table 1. PCS Indicators Category/Indicator High acuity Diet Access (IV port/line access) Medications Respiratory Procedures Interventions Dressings Education/family support

638

No. of Items

Range of Scores

Example

2 8 2 8 5 10 9 5 10

0-72 0-25 0-8 0-84 0-54 0-200 0-86 0-54 0-108

Critical tube(s)/air Tube feeding Line change Oral Ventilator-acquired respiratory infection Laboratory tests Intra-abdominal pressure monitoring Dressings Family care conference



Table 2. Reliability Estimates Agreement Type Per item RNs vs CNs (AM) RNs vs CNs (PM) ICC: RNs vs CNs (AM) ICC: RNs vs CNs (PM) ICC for sums scores

Agreement

Range

95% Confidence Interval

93.2% (SD, 4.4%) 85.8% (SD, 5.6%) 90% (SD, 5.2%) 0.89 0.80 0.90

83.9%-100% 73%-95% 75%-100% V V V

V V V 0.808-0.940 0.604-0.894 0.855-0.943

ability of the tool to measure a child_s acuity from 7 to 10. On a scale from 0 (very dissatisfied) to 10 (very satisfied), 77.4% rated their satisfaction 7 to 10. Nearly half (10/24) of all RNs providing comments proposed completing a PCS assessment twice per day, and 3 suggested putting a warning in the electronic health record as a reminder to complete the PCS; both suggestions were implemented.

Discussion Patient classification tools for pediatrics are rare.36 Past acuity tools have relied primarily on expert opinion,33-35,37 do not detail how they were developed,38 or do not provide reliability and validity measures.39 In this study, we integrate direct observations with RN expertise of the NICU population, provide details on tool development, and assess not only reliability and validity but also ease of use, user satisfaction, and perceived effectiveness. Continuous observation ensured that all types of nursing activities (direct, indirect, hospital) were measured and addressed the concern that time studies measure only discrete nursing tasks.39 Categorization allowed for the measurement of all care activities including communication, education, and emotional/psychosocial activities. This is particularly important in the complex NICU environment where patients cannot verbally provide information or feedback regarding their care, and new parents have had little to no time to gain a sufficient understanding or knowledge of their baby_s behavior and needs.26 Therefore, the tool developed in this study classifies patients by the care required by the

patient, regardless of medical diagnosis. For example, a preterm baby with a less severe condition (eg, neonatal abstinence syndrome) may be fitful or temperamental and thus difficult to treat, requiring additional effort from nurses despite low medical acuity. Significantly high reliability and strong validity provide assurance that the tool is scientifically sound. This is important as healthcare leadership continues to be unconvinced of PCSs_ objectivity and scientific vigor.40 However, usability, user satisfaction, and perceived effectiveness are also important elements of implementing a new tool and sustaining its use. Positive outcomes in these areas and the knowledge that direct care RNs were involved throughout tool development may improve nurse Bbuy-in[ and reduce the need to adjust scores to more accurately assess patient acuity.41 Although reliability and validity assessments of acuity tools have been conducted,36,37,42,43 the authors have not identified studies that take the end user into consideration. High interrater agreement between bedside RNs and CNs illustrated that bedside RNs are knowledgeable of the patient_s condition and capable of completing these assessments. Results of this study captured the attention of nurse leadership, and a policy change was made to allow bedside RNs to complete the assessment. This outcome is a testament to the impact nurses may have on policy and system change within their institutions. The authors note study limitations. Because of inaccuracies in the institution_s prior PCS, comparisons with pre- and post-NHPPDs could not be carried out. In addition, comparisons with existing tools

Table 3. RN Ranking Versus PCS Variable

Mean (SD)

Range

r

P

RN ranking (n = 182) PCS sum scorea RN ranking (n = 41) Average PCS sum scorea per patient

2.61 (1.05) 55.8 (41.3) 2.66 (.92) 54.0 (36.2)

1-5 14-254 1.2-4.8 19.0-196.4

0.7125

G.0001

0.8035

G.0001

a

Level points: 1 = not included in assessment (represents baseline care), 2 = 0 to 3, 3 = 36 to 55, 4 = 56 to 90, 5 = 91 to 99, 6 = 600 (high acuity).



639

were not completed because the authors were unable to find NICU tools with established reliability and validity. Finally, generalizability of this PCS is limited to the NICU population; however, study methodology can be used for developing PCSs for adolescent and adults care units. Future studies should examine the impact of the PCS on patient outcomes (eg, length of stay), nurse outcomes (eg, scheduling, quality of care), and accuracy over time as treatment and populations change. In addition, populating items in the PCS from electronic medical records would further streamline work-

flow.44 This methodology could also be implemented for other care providers (eg, allied health) to accurately allocate care.

Acknowledgments The authors acknowledge the following individuals for their participation and efforts in all aspects of the study: Angela Barnett, BSN, RNC-NIC; Theresa, Taylor BSN, RN; Eric Hall, PhD; and Sue Davis, PhD, RN. The authors also acknowledge Lauren Summerville, MS, for her assistance on the manuscript of this study.

References 1. Joint Commission on Accreditation of Healthcare Organizations and Joint Commission Accreditation Hospital. Comprehensive Accreditation Manual for Hospitals: The Official Handbook: CAMH. Oakbrook Terrace, IL: Joint Commission Resources; 2007. 2. American Nursing Association. Nurse staffing. 2015. http://www. nursingworld.org/MainMenuCategories/Policy-Advocacy/State/ Legislative-Agenda-Reports/State-StaffingPlansRatios. Accessed November 2015. 3. American Nurses Credentialing Center. Forces of Magnetism. 2014. http://www.nursecredentialing.org/Magnet/ProgramOverview/ New-Magnet-Model. Accessed March 2016. 4. Adomat R, Hewison A. Assessing patient category/dependence systems for determining the nurse/patient ratio in ICU and HDU: a review of approaches. J Nurs Manag. 2004;12(5): 299-308. 5. Botter ML. The use of information generated by a patient classification system. J Nurs Adm. 2000;30(11):544-551. 6. Fagerstrom L, Rainio AK. Professional assessment of optimal nursing care intensity level: a new method of assessing personnel resources for nursing care. J Clin Nurs. 1999;8(4): 369-379. 7. Levenstam AK, Engberg IB. The Zebra systemVa new patient classification system. J Nurs Manag. 1993;1(5):229-237. 8. Yamase H. Development of a comprehensive scoring system to measure multifaceted nursing workloads in ICU. Nurs Health Sci. 2003;5(4):299-308. 9. Blegen MA, Goode CJ, Reed L. Nurse staffing and patient outcomes. Nurs Res. 1998;47(1):43-50. 10. Halm M, Peterso M, Kandels M, et al. Hospital nurse staffing and patient mortality, emotional exhaustion, and job dissatisfaction. Clin Nurse Spec. 2005;19(5):241-251. 11. Mark BA, Harless DW, Berman WF. Nurse staffing and adverse events in hospitalized children. Policy Polit Nurs Pract. 2007;8(2):83-92. 12. Aiken LH, Clarke SP, Sloane DM, Sochalski J, Silber JH. Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction. JAMA. 2002;288(16):1987-1993. 13. Hamilton KE, Redshaw ME, Tarnow-Mordi W. Nurse staffing in relation to risk-adjusted mortality in neonatal care. Arch Dis Child Fetal Neonatal Ed. 2007;92(2):F99-F103. 14. Kane RL, Shamliyan T, Mueller C, Duval S, Wilt TJ. Nurse staffing and quality of patient care. Evid Rep Technol Assess (Full Rep). 2007a;(151):1-115. 15. Kovner C, Jones C, Zhan C, Gergen PJ, Basu J. Nurse staffing and postsurgical adverse events: an analysis of administrative

640

16.

17.

18.

19.

20.

21.

22.

23.

24. 25. 26.

27.

28.

29.

data from a sample of U.S. hospitals, 1990