Critical Appraisal #1 of Scott, Hofmeister, Rogness, and Rogers Article
The University of Texas at Arlington College of Nursing

Shirley McKenzie-Pace
In partial fulfillment of the requirements for
N5301 Research in Nursing
Susan K. Grove, PhD, RN, ANP-BC, GNP-BC
July 12, 2014

Critical Appraisal #1
Theoretical Framework
The study framework is not clearly identified; however, a tentative theory is proposed since a tentative theory “has had minimal exposure to critique by the discipline, and has had little testing, (Burns and Grove, 2009). The conceptual framework is limited to the model impaired sleep. The linking of the concept impaired sleep to independent variables of the fatigue countermeasures program for nurses (FCMPN). The FCMPN in this study was “modeled after the National Aeronautics and Space Administration Ames Research Center’s Fatigue Countermeasures Program and the Sleep, Alertness, and Fatigue Education in Residency Program” (Scott et al., 2010, p. 253). The major study concepts include: “sleep deprivation (inadequate sleep), sleep disruption (fragmented sleep), lifestyle situation, and health related issues” (p. 251). There is no map or model of the impaired sleep model purposed by Lee et al., 2004 (as cited in Scott et al., 2010) in the article; however, there is a diagram of the conceptual framework for the FCMPN.
Variable Identification and Definitions
The research variables that Scott et al. (2010) observed and measured were the FCMPN, sleep duration, sleep quality, daytime sleepiness, drowsiness episodes, drowsy driving and motor vehicle crashes, and potential or actual errors. Research variables are used when a study occurs in an uncontrolled or field setting, without application of treatment to the subjects, to observe or measure variables (Burns & Grove, 2011).
Independent Variable
Fatigue countermeasures program for nurses (FCMPN). The operational definition of the independent variable was “the intervention for evaluating the feasibility of an FCMPN for improving sleep duration and quality while reducing daytime sleepiness and patient care errors was guided by the Lee et al. (2004) model of impaired sleep” (as cited in Scott et al., 2010).… The conceptual definition was a comprehensive program for nurses to manage fatigue in the acute care hospital setting using six elements: “education and training, compliance with hours of service regulations, appropriate scheduling practices, countermeasures, and design” (p.252). The FCMPN was modeled after the “National Aeronautics and Space Administration Ames Research Center’s Fatigue Countermeasures Program and the Sleep, Alertness, and Fatigue Education in Residency Program” (p. 252-253).
Dependent Variables
Sleep duration. Sleep duration is one of the dependent study variables. Sleep duration was conceptually defined as an indication of the amount of sleep loss either through sleep deprivation or sleep disruption. It is operationally defined through the self-report completion of a logbook regarding sleep patterns. Sleep duration were measured with the total sleep duration, work day sleep duration, non-work day sleep duration, and night shift sleep duration.
Sleep quality. Sleep quality was conceptually defined as a subjective response to persistently interrupted or uninterrupted sleep with or without difficulty falling asleep. It was operationally defined through measurement using the Pittsburgh Sleep Quality Index (PSQI) scores.
Daytime sleepiness. Daytime sleepiness was conceptually defined as a cognitive-behavioral response of insufficient sleep during the normal sleep cycle that leads to daytime sleepiness. It is operationally defined through measurement using the Epworth Sleepiness Scale (ESS) of daytime sleepiness was before the intervention, 4 weeks post intervention, and 12 weeks post interventions using the logbooks.
Drowsiness episodes. Drowsiness episodes were conceptually defined as a cognitive-behavioral response of fluctuations of alertness and feelings drowsiness at work due to sleep quality and duration. It was operationally defined as drowsiness at work or while driving and was measured in a Logbook through self-reports.
Drowsy driving and motor vehicle crashes. Drowsy driving was not conceptually defined; however, a possible conceptual definition is serious cognitive-behavioral action that results from driving while sleepy due to sleep deprivation, shift work, or untreated sleep conditions. Motor vehicle crashes (MVC) are the serious outcomes of driving while drowsy. Drowsy driving and MVC were operationally defined through measurement by logbooks during the initial data collection, 4 weeks post intervention, and 12 weeks post intervention.
Errors, near errors, and interrupted errors. Errors, near errors, and interrupted errors were conceptually defined as mainly “involving medication administration, patient care procedures, physician order processing, and transcription issues” (Scott, et al. 2010, p. 256). They were operationally defined through measurement using logbooks pre intervention, 4 weeks post intervention, and 12 weeks post intervention periods.
Sampling and Setting
The nurses were obtained through a mailing list for three Michigan acute care hospitals. The participants were paid $5 for each logbook page completed. The researchers were qualified to conduct the study based on their level of education and fields of study. All of the researchers in this study had attained either a PhD or MSN. The participants were readily available at either their home or hospital to enter logbook data. The division of monies for advertising, data collection, data coding, and data analysis were not explained.
Sample Inclusion and Exclusion Criteria
The participants are acute care hospital nurses with fatigue at work. The inclusion criteria were “full-time staff nurses, working at least 36 hours per week, practicing on the selected units” The exclusion criteria excluded those nurses who were ”advanced practice nurses, nurse managers, or nurses in specialized roles such as discharge planning ” (Scott, et al. 2010, p. 252).
Sampling Method
The sampling method was a nonprobability sample of convenience since all potential participants in the three hospitals were sent a cover letter explaining the study and asked to participate (Burns & Grove, 2009).
Sample Size
The sample size was identified as “62 full-time hospital staff nurses (43%) enrolled in the study” (Scott et al., 2010, p. 252). A power analysis was done to determine the sample size needed for the study. The power analysis “established methods and definitions and a sample size of 30 participants would be needed to provide a Cronbach alpha of 0.80 and to detect a medium effect size with a Type I error of 0.50 in the study” (Scott et al., 2010, p. 254). All 147 subjects in the sampling frame met the criteria to participate and the original sample was determined by random selection via a mailing list. No subjects refused to participate. A total of 62 staff nurses enrolled in the intervention, but only 47 nurses responded and completed the interventions across the entire study, a 24% attrition rate. Thus, the refusal number and percentage was
147-62=85 (57.8%).
Institutional Review Board (IRB) and Informed Consent
Informed consent was not identified in this study. “All potential participants received a cover letter explaining the study and a request to complete a demographic questionnaire. . . Nurses who indicated interest in enrolling in the study were provided additional information using a scripted approach” (Scott et al., 2010, p. 252). The article provides no indication that the nurses signed consent forms but their completion of the study forms was probably designed as informed consent. “The institutional review board at Grand Valle State University and each participating data collection site approved this study protocol” Scott et al. (2010, p. 254).
Setting
The settings took place on acute care hospital nursing units and at the participant’s home. “Nurse Managers in each unit provided sleep recliners and a 20-minute timer for scheduled naps during breaks” (Scott et al., 2010, p. 251).
Measurement Methods
Table of Study Measurement Methods Study Variables | Author and Name of MeasurementMethod | Type of MeasurementMethod | Reliability orPrecision | Validity orAccuracy | Sleep duration | Sleep duration logbook-Self-report sleep/wake patterns | Logbooks | No specific reliability or precision, ”Logbooks have been used to collect data…of alertness (in other industries for more than 10 years” (Scott et. al., 2010) | No validity or accuracy | Sleep quality | Sleep quality Buysse et. al., (1989)/Pittsburgh Sleep Quality Index | Psychometric scale | Reliability: Internal consistency of 0.69 to 0.81 in various studies. Cronbach alpha coefficient of 0.70 in this study. | “The test-retest reliability for the short interval, 2 days, was high for the global score and all sub-scores. For the longer intervals, the test-retest reliability was low for the sub-scores, sleep quality and sleep disturbance” (Backhausa et al. 2002) | Daytime sleepiness | Daytime sleepiness Johns (1991)/Epworth Sleepiness Scale (ESS) | Likert scale | Reliability: Cronback alpha of 0.73-0.88, stability (r=0.82) with reliability of 0.71 in this study. | “The ESS has been identified as a valid measure of sleep propensity in adults” (Scott et al., 2010) | Drowsiness episodes | Log used to collect data | Logbooks | No reliability information | No validity or accuracy | Drowsy driving andMotor vehicle crashes | Log used to collect data | Logbooks | No reliability information | No validity or accuracy | Error, near errors, interrupted or discovered errors | Log used to collect data | Logbooks | No reliability information | No validity or accuracy |

Statistical Analysis and Results
Analysis Techniques
The sample was described with frequencies, means, percentages and standard deviation as depicted in Table 1 Demographic Summary of Study Participants (Scott, et al. 2010, p. 252).
The PSQI scale was described using internal consistency coefficients. The ESS was described using internal consistency, reliability, and stability. A one group pretest-posttest repeated measure was used overtime at 2 weeks pre-intervention, 4 weeks post intervention, and 12 weeks post intervention. The data analyses were linked to the study purpose, objectives, and hypotheses. The purpose of the study “was to evaluate the feasibility of fatigue countermeasures program for nurses (FCMPN) for reducing fatigue and patient care errors” (Scott et al., 2010,
p.251). It was hypothesized that adoption of a standard “fatigue intervention program used in many other industries would improve nurses’ alertness and therefore decrease the number of near errors or actual patient care errors” (p. 257).
Findings
The findings were not linked to the study framework. The concepts of sleep loss and poor sleep quality were discussed but not cognitive-behavioral outcomes. “Initial findings support the feasibility of using fatigue countermeasures program for nurses mitigating fatigue, improving sleep, and reducing errors among hospital staff nurses” (Scott et al. 2010, p. 257). The expected findings were “significant improvements were noted immediately after the intervention.
Participants averaged an increase in sleep time by 50 minutes. Compared with the minimum amount of sleep obtained at baseline and at the 4 and 12 week post intervention periods” (Scott, et al. 2010, p. 254). The unexpected findings were “although significant improvements were not found in daytime sleepiness scores, severity of daytime sleepiness appeared to decrease” (Scott et al. 2010, p 256). The nurses continued to report poor sleep quality despite the improvement in sleep quality. Consistency of the study findings with other research was noted with the use of the Logbooks. Studies have been done with pilots’ alertness at the cockpit over 10 years, including physicians, flight and traffic controllers during space shuttles.
Study Limitations

Although the sample size was appropriate based on the power analysis a larger more geographically diverse study population would have been beneficial. A generalizations in the study was “the use of convenient sampling and a pre-experimental research design limits the generalizability of this study… there were sufficient statistical power to examine the variables of interest in a one-group repeated measure design” (Scott et al. 2010, p. 257)
Nursing Implications
“Fatigue management should be a shared responsibility. Human resource policies should be established that prohibit moonlighting among full-time employees. In addition, sufficient remunerations should be in place that would limit the need for additional employment” (Scott et al., 2010, pp 256-257).
The significance of the study by Scott et al. (2010) was to examine symptoms of fatigue reported by nurses, the impact it had on patient outcomes, and the effectiveness of the fatigue countermeasures to improve work-time drowsiness in the participants. Nursing is the driving force of healthcare and essential to patient care. However, with a shortage of nurses, they are often lacking in self-care and often do not allow for adequate rest between shifts to heal themselves. It is every nurse’s responsibility to have an understanding of the profound impact that fatigue has on work performance and the potential for medical errors. Nurses have to drive the initiative in implementing fatigue management and mandatory rest between shifts for the problem to be improved. The findings from this research has added valuable insight to the nursing profession using the nurse’s report of fatigue as an indication that countermeasures should occur to improve alertness, sleep duration, sleep quality, and reduce the risk of errors.

Recommendations for Further Research
There is a reasonable goal for future study to include a larger and more geographically diverse nursing population. Additionally further study is needed to determine the efficacy and nursing social acceptance of fatigue countermeasures for nurses. Recent studies have documented that risk for error is significantly higher when nurses work more than 12 consecutive hours, work beyond their scheduled shift time, work more than 40 hours in a week, and obtain insufficient sleep” (Scott, et al. 2010, p. 256).
Conclusion
The study conducted by Scott et al. (2010) had a feasible sample size and data collection process. The study was funded by Blue Cross Blue Shield of Michigan Foundation. The only equipment used for the study was a logbook to collect data about work hours, breaks, and drowsiness on duty, and sleep and wake cycles. The logbooks had been used in previous industry studies to collect data related to alertness. The study showed correlation between improvements in sleep duration, quality, and alertness with the use of a FCMPN intervention. Research needs to continue to address the work cultures issues that continue to encourage nurses to work longer hours with adequate periods of rest between shifts to meet today’s health care demands.

References
Backhausa, J., Junghannsa, K., Broocksa, A, Riemannh, D. & Hohagena, F. (2002). Test–retest reliability and validity of the Pittsburgh Sleep Quality Index in primary insomnia.
Journal of Psychosomatic Research, 53(3), 737-740. Retrieved from http://dx.doi.org.ezproxy.uta.edu/10.1016/S0022-3999(02)00330-6 Burns, N. & Groves, S. K. (2011). Understanding nursing research: Building an evidence based practice (5th ed.). Maryland Heights, MO: Elsevier.
Groves, S.K., Burns, N., & Gray, J. R. (2013). The practice of nursing research: Appraisal, synthesis, and generation of evidence (7th Ed.). St. Louis, MO: Saunders Elsevier.
Scott, L. D., Hofmeister, N., Rogness, N., Rogers, A. E. (2010). An interventional approach for patient and nurse safety: A fatigue countermeasures feasibility study. Nursing Research,
59(4), 250-258.