June 30, 2020
Understanding how AI algorithms work gives trust
Fabien Scalzo is an Assistant Professor at UCLA and Director of the AI in Imaging and Neuroscience Lab there.
He develops Machine Learning and Computer Vision algorithms to better our understanding of neurological disorders, particularly stroke and traumatic brain injuries (TBI).
He does this using brain mapping of imaging and biosignals.
So Fabien, how did you come to do what you do today?
I was trained in computer vision and machine learning at the University of Liege in Belgium and carried out research to build new algorithms for visual learning.
I became fascinated by how we perform these visual tasks as humans and interested to learn about the learning mechanisms in the brain.
This was a big inspiration for my PhD work and one of the reasons I later joined the department of neurology and neurosurgery at UCLA. This was 10 years ago when I realized that machine learning, although had its limitations, could really make a difference in clinical care.
Why do you think AI has proven so transformative to medical imaging and why is this particularly the case in something like stroke?
With stroke, it wasn’t even clear 10 years ago that imaging would be helpful in guiding decisions, and there was a lot of controversy regarding the value of it.
With machine learning, we let the computer figure out the patterns that really matter by retrospectively analyzing the outcome of hundreds of patients.
It reduces the manual information and the prior assumptions that we need to input and – because we make a better assessment of the patient, thanks to the algorithm – the patient may have access to the best treatment option, the one with the highest likelihood of good outcome.
Machine learning streamlines automation and has an ability to extract information that we would not be able to comprehend or even visualize, making it accessible to clinicians.
What do you think about the hesitation to AI in the medical field? Where do you think it comes from?
One part of this is tradition. The medical field has operated in a certain way for so many years, very successfully in many cases.
There is also a hierarchy amongst doctors, and bringing on board a machine that is part of the decision process can be met with skepticism and, in some cases, it is valid.
There is a need for understanding what that machine is doing; I don’t mean being able to code it, but getting a grasp of the structure and the rationale behind its diagnostic output.
What do you think needs to be done to reduce this?
The clinicians need to be able to give trust to the machine learning model, which comes with having a good general understanding of it. Also, regulation through the FDA is becoming even more important because they are the only organization that the clinician can trust to adopt and regulate these tools. I think we need to customize special tracks for approving machine learning tools.
These are very different from anything we’ve worked with before so even though it may fall within the category of a medical device there is another dimension to it. I see far too often clinicians taking for granted the ability of machine learning to always give the perfect answer, and then if it offers a wrong prediction they immediately lose their trust in it.
There is always some uncertainty, and that’s ok as long as we are aware of it. I try to tell people it’s not perfect; it has its own weaknesses but as long as you know when that happens and what to look for, it is still useful.
10 Questions with Dr. Ludo Beenen
Dr. Ludo Beenen feels the stroke workflow will be different following COVID-19.
Do you think there are issues with transferability from different technical perspectives?
Yes totally. There was a very big hurdle because we have two very different worlds trying to come together. On one side, we have the clinical world, which measures the value of the technology by how much it improves the care of the patient and the workflow. On the other side, machine learning scientists normally measure usefulness by how well the model performs with certain types of accuracy metrics.
Even though these metrics may work well in the experimental setting, they may have zero impact when transferred into clinical care. This is something I’m working on, trying to evaluate machine learning not only with respect to the accuracy of the models but with respect to the clinical value in terms of decisions and workflow.
Do you think in the next 5 years we are going to see big changes in the merge of these two worlds?
Yes, machine learning is very young and novel and will continually improve. At some point, machine learning will be at least on par with the best neurologists in the world for diagnosis, so in that sense, it’s going to be very interesting to see how it develops.
At the moment, all the models we have are extremely specialized for specific tasks, but I think, eventually, models will develop to be more versatile and general, so a neurologist will be able to see things that previously would have been missed. Some of that is not part of machine learning, it is common sense, so, eventually, we want to bring common sense to the machine!
Is the best of AI currently in medical imaging or is it to any extent behind because of the difficulty with patient data and accessing it?
Today if you compare the magnitude of data in medical imaging to marketing/advertisement, we have extremely small data sets and they are for very heterogeneous diseases, where every patient is different. There are a lot of issues regarding privacy that need to be addressed. We are behind because we rely on data coming from a medical provider, such as a hospital, but across the world, they do not have the infrastructure to deal with large amounts of data in an efficient way so which slows things down.
It is great that there are so many new companies like Nicolab that build around existing infrastructure in the hospital, which makes it easily accessible to all physicians and renders the process of using data in a transparent and secure way which is what we need right now.
If patients were more comfortable with sharing their data for research would this have a positive impact on AI?
Yes, millions of people will benefit from sharing your data, so it’s increasing awareness of that and working out an infrastructure that will make people feel comfortable about sharing it. But I think it will be a gradual cultural shift.
Fifteen years ago, Google was using our data, and there was a big backlash, but now most people don’t mind about that to some degree. It is part of the world we live in. As long as it is recognized and we know how it is used in a positive way toward the betterment of society, I think it is acceptable. But some would disagree, and that is also ok! One should have the right not to share their data.
On a personal level, what is the most exciting aspect of your work?
Well, two things. You can use machine learning to do finance and make a lot of money, but you can also use it to save lives, and that is what excites me the most. To be more precise, I am very interested in stroke because it is a disease that is extremely hard to understand – some patients may return home with almost no side effects, and others may not survive.
It is phenomenal that soon we will be able to accurately predict a patient’s outcome for a given treatment course. This works because machine learning can incorporate data from so many patients with different treatment options so, eventually, the model will be able to make simulations of the likely outcomes for different treatments, including the different benefits versus risks. It will be truly personalized medicine!
As well as prediction, all the different elements of AI streamlining and speeding up care will improve healthcare systems as a whole but do you think that AI will also save money?
Yes, AI will make the whole process from reading images to treatment much more efficient therefore reducing time and making work easier and more pleasurable for neuroradiologists which will of course reduce costs. It seems obvious to me but I guess it depends on how versatile the model is. If it is only for stroke, you need another one for cancer and screening brain injuries, then you will be surrounded by AI.
Versatility and some AI are a good direction in the future where we will have a model not just for stroke. But we are not there yet, we are in different compartments, but eventually, I hope we can bring it together.
Discover how Nicolab’s solutions can streamline your stroke workflow