Checking Your HCAHPS Data? How Often... is the Question!
By Gwen Faust, RN, MS, Consultant (HCAHPS & Other Surveys), HealthStream
As the first HCAHPS performance period (discharges July 1, 2011 through March 31, 2012) draws to a close and Value Based Purchasing is becoming a reality, hospital leaders across America are looking at their patient experience data (HCAHPS) with a deeper interest and greater scrutiny. With this higher level of concern, clients are looking to their vendors for timely reporting of data and want the ability to view data easily and often. How often to view HCAHPS data is the subject of this blog. I asked my HealthStream colleagues for their recommendations.
CMS GUIDELINES FOR HCAHPS SAMPLING AND SURVEYING MAY HAVE AN IMPACT ON HOW OFTEN NEW DATA IS AVAILABLE
CMS mandates that calls begin within 42 days post discharge. After receipt of client files, HealthStream validates and processes all sample files. HCAHPS studies are then placed on a rotating call schedule. This rotation is necessary to make sure each record is called at different times of the day, on different days of the week, and in different weeks in order to be compliant with CMS rules. Each time HealthStream administers your HCAHPS study, we will attempt to call all available patient records for the study. Any patient records not reached during a session will be set for callbacks the next time the project is scheduled to be worked.
Under CMS rules, only the patient may be interviewed. HealthStream interviewers must follow the CMS script word for word when contacting patients. This approach protects patients’ confidentiality and ensures the questions are asked the same way each time, thereby minimizing the variability in results. For hospitals that elect to include pediatric patients (results are not reported to CMS), surveys are conducted with the parent or guardian of the patient.
HOW OFTEN SHOULD I LOOK AT OUR HCAHPS DATA?
Cyndi Tierney, HealthStream Consultant, advises “Patient satisfaction data (i.e., your HCAHPS results) fall within the realm of social sciences. Using social science guidelines, an N size of 30 generally represents the smallest data sample by which to gauge progress or take action. In a study of HCAHPS data, HealthStream reviewed results across clients, as well as a more granular study of a larger healthcare system’s HCAHPS outcomes. As predicted, we found the greatest stability in viewing data by quarter and by year, where the n size was large enough to stabilize the results. Monthly results showed more variation in scores, while daily reporting, based on a few number of responses, reflected large swings and less confidence that the result accurately reflected the total population. One or two exceptionally happy or unhappy patients will skew the results, creating a false sense of urgency or complacency, depending on the survey.
We recommend using quarterly and annual data for reporting purposes, monthly data for quick pulse checks and course alignment, and weekly review on comments, to provide timely feedback to staff, physicians and leadership. This gives you optimum use of qualitative and quantitative data in ways that reliably reflect your larger patient population.”
Another HealthStream Consultant, Bo Hansen...”generally tells clients that being in the data weekly is fine if they incorporate data for the past month, or better yet for the past quarter. I have been explaining the HealthStream analysis of volatility in the daily and weekly results versus the long term trend.
I have found that asking staff to respond to a small increase in weekly data representing ONE or a few clients who are very unhappy about “the courtesy of the staff” for example, will take their focus away from the long term trend that shows that the low scoring, high correlating item over time is keeping the patient informed about delays. Checking and responding to data weekly dilutes the message from the manager to staff and makes staff confused about where their concentration should be and the reason for it.”
Bob Ogden, HealthStream Senior Consultant recommends “that clients check data monthly or every two weeks. Check verbatim comments/Voice Of The Patient (VOTP) 2-3 times per week. Twelve months of data should be used to identify opportunities and to set goals.”
John Merritt, HealthStream Regional Representative shared that “in working with hospitals, I often am asked about how often to look at the data. As with many things, the answer is always, “It depends!” One of the strengths of our online tool, Insights Online, is the ability to run reports on a daily basis. Because of this, many clients schedule frequent reports, such as weekly or monthly. While admirable, if they do not have at least 30 respondents during this time frame, then the data could be unstable. A lot of people tend to confuse statistical significance with statistical stability. The reason that a sample size of 30 is the “magical” number is due to this being a large enough sample for the Central Limit Theorem to take effect. In simple terms, the Central Limit Theorem is when data normalizes. Perhaps better stated, this is when the sample takes on the same distribution as the overall population. If the overall population looks like a normal bell curve, then once you reach a sample size of 30, then the sample will look like a normal bell curve.”
Merritt adds, “therefore, hospitals have two choices when it comes to how often to look at their data: (1.) Increase their sample size, or (2.) Use a rolling total. If the data is being used for pay-for-performance, then increasing the sample size is the appropriate answer. When increasing the sample size is not a viable option, I typically recommend using a rolling total. Many hospitals will set their sampling plan on a quarterly basis, so if they want to look at data monthly, then I recommend doing a three-month look back. Each month, they would drop the oldest month and add the current month. In theory, a hospital could look at data less frequently, but with VBP and public reporting, they may not want to wait. Our clients haveaccess to timely data, especially because we utilize a telephone methodology. Therefore, I feel it is our obligation to help them best use this timely data.”
Finally, I spoke with representatives of Baptist Leadership Group (BLG) and Studer Group regarding this topic and both take stances similar to our HealthStream Consultants and Regional Representatives about how often to examine HCAHPS data. “BLG recommends leaders review HCAHPS with their staff every month, staff and the leader agree on 3 behavior changes and leader follows up for accountability. We recommend looking at that [particular] month's performance relative to current quarter or fiscal year.” Studer Group recommends looking at data frequently as a peek into what is happening and as a tool for identifying coaching opportunities. They recommend that reported data be trended over longer period of time.
USE DATA TO SET AND STAY THE COURSE
In conclusion:
- Trended data over 12 months or so should be used to set goals and for determining improvement strategies necessary to affect those goals both at the organizational and unit levels. HealthStream consultants and analysts can assist your organization with setting goals.
- Use Voice Of the Patient (VOTP) for coaching and recognition of individuals
- Hold staff accountable for the improvement strategies....verify that these are being done consistently by rounding on patients, huddling, and observing staff