Harvey: Welcome to the GPT Podcast.com I'm Harvey along with my co-host Brooks, and this is our AI in Healthcare Series.
Brooks: Sounds like we're in for quite a discussion today. What's our topic, Harvey?
Harvey: We have a rather impactful and somewhat controversial topic today, Brooks. The question is, Who pays the bills when AI kills?
Brooks: Wow, that sounds pretty intense. Where do we start?
Harvey: Well, AI is revolutionizing many aspects of our life, including critical areas like health care. It's providing more accurate diagnoses, personalized treatments, and making quality care more accessible to rural patients. 
Brooks: That's quite remarkable, but I gather there must be some serious challenges as well?
Harvey: Exactly, As AI becomes increasingly pervasive, it's posing unique challenges in determining who bears the responsibility when AI systems cause harm. The concept of the “wrong” here could range from direct physical harm, privacy violations to biases that can be more easily perpetuated by AI throughout certain mediums.
Brooks: So, the crux of the debate is about who pays when something goes wrong. Can you explain a bit more about how this usually pans out?
Harvey: That's a great question, Modern AI systems are based on statistical methods, and due to their nature, it's guaranteed that certain systems will not perform as intended on some occasions.  Before the widespread use of AI, most legal systems held humans responsible for their actions, based on negligence or even intentional wrongdoing. But with AI systems becoming more autonomous, it's hard to assign blame to a human actor in many cases. 
Brooks: And this system isn't just built by one person or one company, is it? There would be multiple parties involved.
Harvey: Absolutely, We have the data provider, designer, manufacturer, developer, user, and the AI system itself. Thus, if damage occurs, establishing liability becomes difficult. Did the damage happen because instructions weren't followed? Or was it caused while the AI system was still learning and collecting data? 
Brooks: Could you perhaps provide an example to illustrate this?
Harvey: Sure, take the 2017 class action suit against Tesla as an example. Here, the manufacturer was blamed, with allegations stating that the automated vehicle’s autopilot system contained inoperative safety features and fault enhancements. The implications could lead to fines upwards of $115 million, not to mention the hundreds of millions lost in reputational and shareholder fallout.
Brooks: That does put things into perspective. But what about our health care system, ? How does this impact an area as sensitive as health care?
Harvey: It's indeed a touchy area, Unlike automotive, there are very few insurers to cover specific types of AI-related scenarios in health care. Providers and insurers are quite wary of litigation threats related to AI. In fact, recent FDA guidance suggests that the FDA wants to push the liability for cyber and any software issues, including AI, to manufacturers rather than hospitals and providers. 
Brooks: And I guess this is where an additional obstacle arises, correct? Manufacturers would naturally be wary of assuming this liability.
Harvey: Yes, you got it right. They're fearful because there's currently no insurance structure in place to cover such scenarios. This resistance could cause a big hindrance to the promising innovation and usage of AI, which could bring considerable benefits to the health care system.
Brooks: And given these issues with transparency, how do these concerns open the way for a potentially more defensible validation system?
Harvey: You've hit the nail on the head, A defensible validation system to withstand an FDA audit with the data volumes and interdependencies dictated by AI is an extremely crucial aspect. It's fundamental to ensuring that medical technology companies move forward with new innovations. However, it would be a shame to curtail AI’s potential simply because it takes time for insurers to “catch up.”
Brooks: It's clear that a careful balance needs to be struck when it comes to AI in healthcare. Given these risks, what kind of approach can we adopt to optimally leverage AI?
Harvey: The key is to take a risk-based approach, You've got to understand that no medicine is 100% effective and safe for every single patient. It’s well known that patients react differently to different treatments. AI-powered medical devices and systems should be evaluated based on a careful assessment of whether the benefits outweigh the risks.
Brooks: A mix of caution and optimism is what it's sounding like. 
Harvey: Indeed. Greater transparency between manufacturers and health care professionals will facilitate the assignment of liability and establish trust, ultimately leading to a broader acceptance and utilization of AI. 
Brooks: Well, it's clear that we are on the cusp of a paradigm shift in healthcare brought on by AI. There's a lot of promise, but also uncertainty and risks that need to be navigated carefully. 
Harvey: Well said, This dialogue is just the beginning, but it’s one we need to have as we tackle these challenges head-on. And that's all from us today at the AI in Healthcare series on GPT Podcast.com Stay tuned for our next episode as we continue exploring the future of healthcare.