Natural Language Processing in Healthcare

Status

Published

Video

Ready to Publish

Publish Date

May 26, 2021

Target

Long Form

Blog Ideas

Design

Drafted

Graphics

Type

YouTube

Video Recording

Natural language processing and its applications are significantly important as we move forward with digitizing healthcare in general. It is a subfield of artificial intelligence, linguistics, computer science, in which it teaches computers how to read, write, listen and speak. It is an amazing breakthrough that has been going on for the past 10 years and has significantly accelerated in the last few years.

This is part of the larger series — Digital Voice Assistant in Telehealth and it includes four major topics:

True Impact of Telehealth

Natural Language Processing in Healthcare and Overview

Ambient intelligence in Healthcare

Cloud vs Edge Computing in Healthcare

So, in this article, we will be discussing Natural Language Processing in Healthcare and its Overview.

NLP a Subfield of AI:

Artificial Intelligence has multiple subfields and natural language processing is one of them. Its applications in healthcare are three:

Intelligent EMR — Electronic Medical Records are a bane of Physicians’ existence. We need to make them smarter and that can only be done through NLP.

Intelligent Chatbots — Providing wellness to patients & answers to physicians quickly.

Voice Assistants — Assistants that help patients manage their health much better.

There’s one more area called Sentiment Analysis and that will be discussed separately.

Evolution of NLP:

Natural Language Processing went through an evolution before the deep learning era. Before the deep learning era, natural language processing was mainly developed on statistical models (1). They were good but at the end of the day were not that great. Those were initial attempts towards creating translation services, rather than a true intelligence system.

The second evolution that came in, was after the deep learning era. And during this era, neural network models were used. NLP Neural Nets, Sequence to sequence learning, and Attention led to the pre-trained models (2). These pre-train models can transfer that learning to different kinds of applications, including healthcare.

Machine & Deep Learning:

The difference between machine learning and deep learning is significant (3). Just for a quick refresher, in machine learning, you need feature extraction done by humans, which is the key limiting factor. Because there’s a limit in the feature extraction humans can do, especially when the language is so complicated and convoluted. What deep learning did was that the feature extraction and classification were done by the computer itself. The problem with deep learning is that it requires massive amounts of data. For NLP, different models use the whole internet as their data model.

The key difference came with ‘Attention’ in 2015. It was the key breakthrough because what attention was able to do was that it provided context. Please consider reading this amazing article from Towards Data Science (4). Here is an example:

“I poured water from the bottle into the cup until it was full.”

“I poured water from the bottle into the cup until it was empty.”

The word ‘it’, what is ‘it’ here? Is it the cup or is it the ball? Just changing the difference of the context makes a huge difference. What self-attention was able to identify for the machine was the context itself. And when the context was able to be identified, it produces a significant amount of breakthroughs as far as the processing of natural language is concerned.

Generative Pre-Trained Transformer 3:

GPT-3, which is an open AI project was formed as a result of the breakthroughs. GPT-3 was able to significantly improve natural language processing. It can answer questions, write essays, summarize long texts, therefore it is the key innovation in which intelligent EHR can be born. Because what we need is real-time current EHR summarization. And with this technology, it is now possible. What it did was that it took 175 billion parameters and put them into the natural language processing model and pre-trained models on it (5). What the breakthrough was that it was able to have increasingly efficient because it was trained on such a high level. And then it was able to have in-context information, which is key for EHR because we don’t want to have information that is not contextual to our patients. And as we know, context is for kings.

Context is for Kings — with New NLP tools the Major breakthrough is Context awareness. This will lead to even more data turned into Knowledge!

Google’s Trillion Parameter Model:

Google came out with its model with a trillion parameters (6). So we went from 175 billion to 1 trillion. So clearly, there’s a race towards natural language processing between Google and Microsoft, open AI, etc. China is also developing its own model from the Beijing Academy of Artificial Intelligence (7). And they also have 2.6 billion parameters working within that system.

Switch Transformers:

These transformer methods (8) in which there’s tension and context built-in, are bringing more and more efficiency as well. This paves the pathway to have these be real-time. Real-time analysis is a holy grail and it’s not that easy. You need to build efficiencies into the system to have some real-time analysis, and these models not only have the breadth and the depth to analyze it, but also the efficiency to produce it in real-time.

Intelligent EMR:

The below image is a screenshot from a study (9) in which they reviewed critical care charts and wanted to know their outcomes. NLP Derived Terms yielded excellent predictive performance. There are now real-world examples in which NLP is applied on EHR to get real-world results for the patients. So this is extremely important.

Intelligent Chatbots:

Intelligent chatbots have been there for a long time and there is nothing new. They are mainly fueled by artificial intelligence, and become a viable option for human and machine interaction. And this is extremely important because Chatbots like Siri, Alexa, Google, etc can turn on & off lights as virtual assistants. However, these chatbots have an extremely important role in managing our health. And if they’re intelligent, if they have the proper contextual data about the patient itself, then they can remind medications, make appointments, and start a telehealth visit. So this is an extremely important breakthrough that we need in healthcare (10).

And talking about these, the question really is that, unfortunately, with the interoperability issues and data silos, we don’t really have truly open-source models. This is one example (11), when the BERT model was made open-source and publicly available on clinical terms.

Voice Assistants:

Voice assistants are rapidly being deployed into hospitals (12). They have now Alexa at the bedside. And they can say, ‘Can you please bring me water?’ They don’t have to click the nurse button and then the nurse comes in. And then the nurse has to ask, ‘What do you need?’, that was the water and the water can be brought in by the secretary or someone else. But to bring a nurse into it was actually kind of stupid. Now with these kinds of technologies, you can improve the efficiency of the whole network and then be able to produce better care for the patients the same way. These can monitor for real-time signs as well. Like for example, in the case of Obstructive sleep apnea, you can actually listen for gaps in the breathing while you’re actually sleeping next to it. So there is a multitude of possibilities that digital voice assistants can bring in. And this is going to be a fantastic application for healthcare in the future.

Query Focused EHR Summarization to Aid Imaging Diagnosis:

As we discussed, patient’s EHR summarization can be done and can aid image diagnosis as well (13,14). And query-focused Image diagnosis is also extremely important. Here is an example:

If a patient comes in with an acute stroke. Then, all that the doctor has to say is, “Hey, x, give me acute stroke summary.” They can pop up on a screen or even say to the doctor that the patient was previously on aspirin, the patient is not on warfarin or eliquis. What is the last known well? Well, that is something that the nurse or the triage person already puts in the system the minute they come in. CT brain, what is the status of it? Is the patient already in the scanner? Or is it done? What was the last glucose level? That is done by the EMS generally, it is not even in the hospital electronic medical record. And what is the last blood pressure? Again done by the EMS, completely in a different system. When they’re connected when there are no data silos, then we can truly empower physicians to take care of patients faster and better. This will lead to increased and better outcomes.

Role of AI in TeleStroke:

I did a whole paper on the role of Artificial Intelligence in Telestroke. It’s a fantastic read, it’s a review paper, it’s a mini-review. And you should definitely go ahead and review this (15). It discusses the role of stroke imaging, explains continuous advanced outcome prediction, intelligent EMR, how documentation imaging workflow enhancement can be done, and contextually aware information. It is the key thing here and can be made possible to improve telestroke care with artificial intelligence which becomes from the natural language processing. I created a clinical case around it and made an animation explainer (16) for this particular way of providing telestroke care with enhanced artificial intelligence.

“If you save a life it is as if you save the life of all mankind.”

References:

https://medium.com/@antoine.louis/a-brief-history-of-natural-language-processing-part-1-ffbcb937ebce

https://medium.com/@antoine.louis/a-brief-history-of-natural-language-processing-part-2-f5e575e8e37

https://semiengineering.com/deep-learning-spreads/

https://towardsdatascience.com/transformers-89034557de14

Gao, T., Fisch, A. & Chen, D. Making Pre-trained Language Models Better Few-shot Learners. arXiv [cs.CL] (2020)

https://analyticsindiamag.com/google-trains-a-trillion-parameter-model-largest-of-its-kind/

https://medium.com/syncedreview/chinas-gpt-3-baai-introduces-superscale-intelligence-model-wu-dao-1-0-98a573fc4d70

Fedus, W., Zoph, B., & Shazeer, N. (2021). Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity. In arXiv [cs.LG]. arXiv. http://arxiv.org/abs/2101.03961

https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2719128?utm_source=linkedin&utm_campaign=content-shareicons&utm_content=article_engagement&utm_medium=social&utm_term=123119#.XguEU1l3PN8.linkedin

Pati, B., Panigrahi, C. R., Misra, S., Pujari, A. K., & Bakshi, S. (Eds.). (2019). Progress in Advanced Computing and Intelligent Engineering: Proceedings of ICACIE 2017, Volume 1. Springer, Singapore.

https://www.aclweb.org/anthology/W19-1909/

https://hitconsultant.net/2019/12/10/digital-voice-assistants-hospital-room-near-future/

McInerney, D. J. et al. Query-Focused EHR Summarization to Aid Imaging Diagnosis. arXiv [cs.LG] (2020)

https://arxiv.org/abs/2004.04645

https://www.frontiersin.org/articles/10.3389/fneur.2020.559322/full#:~:text=AI%20in%20telestroke%20will%20pay,potential%20for%20improving%20patient%20care.

https://www.youtube.com/watch?v=seX58zp8I3o