1. IT’S ACTUALLY MEANINGLESS
Well, perhaps not all of it. Although “real-time data” has been a buzz-phrase for years now, does it really carry as much clout as the business world has given it? Its use has become synonymous with boardroom meetings and trendy “agile” business models. So what is it about real-time data that you should be concerned with before making any snap decisions?
In a word, VOLUME.
Without sufficient volume, the statistical significance of real-time data is pretty meaningless. The less data you have available to analyze, the less reliable it is from a managerial standpoint. Consider the following scenario:
Our hypothetical Administrator, Frank, has introduced a new program to improve the tracking of patient-provider interactions. To assist this effort, he’s worked with his vendor to add a question on the CAHPS survey to measure its effectiveness. As the first round of survey data is uploaded to Frank’s dashboard, things don’t look promising. The initial data suggests that patient satisfaction is actually negatively impacted by the change. The second batch of data is uploaded one week later and there’s more of the same. Concerned with the trend, he sends a memo to his team, canceling the effort until further review. The rest of the data is uploaded and the end result is surprising – the overall rating scores for the entire survey period are not significantly different from the previous month.
So, what happened?
Instead of focusing on the entire sample, Frank allowed himself to get caught up in the early results. His “real-time” data was suggesting that change was needed, despite the absence of sufficient volume to support his theory. This leads us to the next item –
2. RESULTS CAN BE MISLEADING
If you’ve spent any time in or around analytics in the last decade, chances are you’ve heard the expression “correlation does not equal causation” or something similar. Although there are certainly instances where the correlation between two variables can point to the cause, a good analyst never stops at the first step.
You see, the patient experience is a vastly more complicated measurement than it may seem. Whenever you are trying to predict human behavior, there is a low “signal to noise” ratio that impacts the final result. Here’s one way we can think of it:
- Signal – these are the variables that, once identified, can be used to build a predictive model for your patient scores. in the case of CAHPS surveys, they represent single questions on the survey.
- Noise – these are all the remaining variables that, while they add to the predictive model and may even have a high correlation with your target variable, they fail to show statistical significance when tested against the other variables.
But what does this look like in practice? Well, let’s say that when your patient data was tested, we identify 5 questions that have a high correlation with the variable you are concerned with.
(More often than not, this is the Overall Rating question)
We want to be sure that the recommendations we provide are going to drive real results so we then test these variables against each other to find out which ones contribute the most to your predictive model.
3. IT REMOVES THE HUMAN TOUCH
You’ve worked with your vendor on points of improvement and have a plan in place for driving up scores.
Your employees and medical staff have all received training on the specific areas of improvement and everyone knows the expectations and goals.
The next iteration of CAHPS scores arrive and the scores still haven’t budged.
It may be worthwhile to consider the delivery and impact of the training on your staff in order to pinpoint what went wrong. We did mention that human behavior is difficult to predict, right? One thing that happens when you focus on real-time data is that your organization becomes laser-focused on micro events instead of keeping the bigger picture in mind. It changes the thought process from
“That patient seems to be having a really rough day, maybe I should check with them on external factors that we can help with.”
“That patient seems to be having a rough day, I better ask about improvement factor A to make sure that’s not the problem.”
Now you’ve drawn your provider’s attention away from simply delivering the best care possible and made the interaction less natural. This in itself can lead a patient to perceive the care to be of lesser quality than they would have otherwise.
4. IT TURNS YOUR STAFF INTO FIREFIGHTERS
There’s something to be said about the differences between proactive management styles and reactive ones. Of course, we would all like to think of ourselves as the former, demonstrating that we are forward thinking and able to perceive and address issues before they arise.
But is this really the best use of your staff and time?
As one client recently shared with us, they really like the ability to view any negative patient comments and follow up on them as soon as possible. Unfortunately, there’s a twofold problem with this approach to managing the patient experience:
1. You are committing resources to scores that have already been recorded.
Sure, there are always opportunities to learn from your mistakes as an organization. Apart from the “sour grapes” (i.e. those patients who gave a bad score simply because they had a bad day in general), many patients who gave a low score can offer one or two items that add qualitative value to your analysis. But don’t lose sight of the fact that this patient cannot go back and change their answer. View these as learning opportunities for the future, not corrective opportunities for the past.
2. Your staff develops a paranoia to receiving unfavorable scores
This goes back to our third reason, suggesting that it distracts your team from a more simple focus of just providing the best care possible. It changes their mentality to that of a “firefighter”, or an employee who feels compelled to go around putting out fires whenever there are signs of trouble. If your organization has the means to devote this much staff time to chasing down every single unfavorable score or comment, we invite you to do so! Also consider how that time could be spent more efficiently in improving future patient scores.
Want to talk with someone about the proper use and application of data? Contact us today and see how we can help.