Validating Clinical AI from the Perspective of a Neuroradiologist
The candid viewpoint of a radiologist on the challenge of implementing AI in healthcare
Alan Pitt, M.D., Neuroradiologist at the Barrow Neurological Institute – Phoenix, AZ
This excerpt from the webinar examines:
- How clinicians define transformative healthcare
- The need for AI technology within the health space
- Possible barriers to entry and implementation
Note: This conversation has been edited for length and clarity.
Florent: If there is any kind of AI that’s gonna be transformative enough to be of interest to you as a physician in your job today at Dignity Health, what would a technology company consider to be low-hanging fruit that really needs to be addressed ASAP, that AI can solve?
Dr. Alan Pitt: In my core expertise as a neuroradiologist, the low-hanging fruit, are the big public health problems that I see that currently imaging is not well addressing. The first thing I would choose would be a rapid volumetric assessment of the brain that would allow me to correlate treatment with changes in the brain morphology. Number two would be the ability to look at multiple sclerosis plaques. An AI algorithm that could quickly assess those plaques and show me changes over time would probably be a relatively easy, but a very useful thing to do. A third thing would be to look at the cortical surface of the brain because I don’t think we do a very good assessment of many mental disorders. if we could get a better assessment of the cortical surface of the brain, which is something I cannot do as an imager, but an AI algorithm could possibly do, that would be probably a third place that I think AI can do something about that I cannot do for myself.
Florent: There are some very real use cases that are not really being tackled by AI or machine learning. To be specific, the impetus for the development of those algorithms today generally comes from the technology side, right?
Dr. Alan Pitt: That’s right. As a general rule, if you want clinicians to adopt whatever technology you have, I tell people you need to be a ripple, not a wave. If I have to get out of my chair to go use your technology, I’m pretty much not gonna do it. I have to have everything within my workflow for me to really start using that at scale, to use it very frequently. This is really true of any AI algorithm. It really has to be just something in the background that’s super easy for me to use. If you don’t do that, it doesn’t matter really how much value that AI algorithm brings to me, I’m pretty much not gonna do it at scale. I’m not gonna use it frequently. And so, the biggest barrier for me as a radiologist is not the AI algorithm itself, it’s how you’re gonna integrate that into my workflow if you really want that to be a successful offering.
Florent: Today, in your clinical environment, how many algorithms are actually being used clinically? Considering the amount of noise in the industry coming from all the large and small companies, why is that today?
Dr. Pitt: Zero. Let me give you another story and it may give you some context. My friends developed a company, Montage Health, that allowed me to do data analytics on my own reports. Think of it as a Google crawler for radiology reports, where it would consume my reports, and allow me to see them by referring, by type of imaging, by age of patient, all sorts of things. It would allow me to see all sorts of dashboards. And I knew that my EMR could not give me these reports. I wanted this application in my hospital environment for two purposes. I wanted it for quality assessment, which is a big driver for the healthcare system, but I also wanted it for research. I wanted to be able to figure out do I have enough patience to even do a research study on a particular problem? Can I identify those patients? I came back home to Dignity and I started going everywhere I could, asking people, “Will you help me get this thing?” It literally took me two and a half years to get through all the red tape within budgeting, but also the privacy and security within my hospital to bring this application to the institution. You cannot expect clinicians, even if they’re super passionate about your algorithm, to spend a couple of years knocking on doors, trying to break down all the silos to get the necessary budgeting, privacy, and security pieces that are gonna be required.
Florent: How to operationalize this technology is one of the key barriers to entry, right?
Dr. Pitt: It’s a major barrier. The production of the algorithm is the easy part. But even if you have the algorithm, you gonna have to figure out a way to get it deployed. And it’s really no different than a start-up company going to a venture capital firm to present a new enterprise. The things that a VC will look for in the technology is the secret sauce, the process, and the experience of the people building the technology. All of those things are ingredients that go into the decision making, and so, before we even get to the operationalization of an algorithm, there are some real questions that need to be asked.
Florent: So when you get beyond all the noise in our industry, what is it that you could be doing as a physician to educate yourself on what’s possible in AI and be a contributor to the input the technology side really needs from physicians?
Dr. Pitt: Physicians make a lot of assumptions while the AI industry is really interested in promoting itself. AI companies really kind of over-promise and under-deliver a little bit. I’d really like to get a good assessment of what is possible, and I think many clinicians would be happy to say, “Look, this is my problem. I’ve gotta count up the number of dots on a field and it’s just way too many dots to count and characterize. Can you accurately do this for me?” Can you measure the amount of volume loss in the brain? Can you come up with an accurate and rapid way to quantitate fluid and fat inside the brain? I can show you by imaging how to differentiate those, but I don’t really have a sense of how strong the algorithms are to correct for that. So I think that getting a better dialogue would help clinicians frame their ask of the tech community I think we’re starting to have that dialogue, but it’s still pretty limited. I’m usually presented with an algorithm as opposed to having a conversation.
Florent: How many of those algorithms do you run into, Alan, that you go, “Okay, well, that’s nice, but I can’t use this today?”
Dr. Pitt: If I’m gonna invest in technology, there’s the nice-to-have things and the have-to-have things. So there are certain things that make my world better and makes it easier for me. There are certain things that I can’t do for myself, but the algorithms can provide. I want to emphasize that I believe it’s a true partnership between health and tech. Now, you may not be able to think like a doctor, but you can think like a patient and you can start to identify problems that you may have seen for you or a loved one that you think technology might be able to solve. And you can begin that conversation by bringing a clinician in early, to ask him, “My loved one has this problem. I was noticing that XYZ, they’re unable to do XYZ. How do you think about that?” And if I were able to help you better understand this tumor or better evaluate this multiple sclerosis and follow it over time. If I was able to do that for you, would that be of value for you in caring for this patient?” If you can get the clinician to say, “Yes, that would definitely be of value to me.” Then I think you begin a conversation around, should this be developed? How much value does it bring?
What does Frankenstein have to do with health IT consolidation? Are IT professionals more like Indiana Jones or McGyver? Is there an end in sight to the consolidation trend?
Is AI a threat to physicians? Is it possible for AI algorithms to make physicians obsolete? Will the robots take over? In the second part of this series, Florent Saint-Clair kiboshes some of the many fears associated with AI. And highlights the many advancements the merging of AI and healthcare has benefitted the industry.
Florent Saint-Clair challenges the healthcare community to let go of lingering fear or doubts and accept that in 2021 AI is here to stay.