Addressing Health Disparities in The Real World: Lessons Learned From AI

Improving health equity has become a driving force within the medical community, US Congress, and the Department of Health and Human Services, and is even starting to affect reimbursement.¹ While there are many reasons for avoidable health inequities, lack of equitable access to diagnosis and treatment are prominent in disease states ranging from breast cancer to depression and diabetic eye disease.^2-7 Currently, fostering health equity is a goal of all health care stakeholders: patients, providers, ethicists, payors, regulators, legislators, and even AI creators.

Autonomous AI—where the medical decision is made by the AI without human oversight or clinician input—has received broad stakeholder support, including from retina specialists, considering the first device cleared by the FDA provides a diabetic retina examination.⁸ Where rigorously validated and appropriately implemented in real-world clinic workflows, AI tools can improve clinician productivity, health equity and efficacy, and clinical outcomes, all while reducing cost.^9-13

THE PROBLEM

When first encountering autonomous AI, clinicians raised many concerns, including job loss, potential bias, and the effect on health equity, even though such issues already affect non-AI-related health care processes and interactions.¹⁴ This is especially true when the autonomous AI (eg, LumineticsCore, Digital Diagnostics) claims that it is intentionally designed to improve access, outcomes, and health equity for underserved populations—and paves the way, ethically, for other autonomous AI systems on the market.

Such concerns have led to an explosion of studies on the risks and benefits of AI and how to address them. In response, we and others created an ethical framework for AI as the foundation upon which autonomous AI regulation and AI reimbursement is built.^13,15,16

Provider concerns of bias, patient benefit, cost, liability, and effect on health equity led to the reexamination, from an ethics perspective, of all health care interactions and processes, even those performed solely or mostly by specialists. Using our ethical framework as a foundation, we, together with the FDA and other health care stakeholders, recently completed a careful analysis of how bias can be introduced—and mitigated and addressed—during the conceptualization, design, engineering, training, deployment, regulation, and monitoring of AI in the real world, and it easily translates to any health care process.¹⁷

MEASURING ETHICS

The three central bioethical principles are beneficence/maleficence (patient benefit or “do no harm”), justice (ie, equity), and autonomy (Figure).¹⁸ Any provider, medical process, or treatment is unable to meet each bioethical principle fully. Rather, everything requires a balance between each ethical principle. For example, maximizing outcomes for lung cancer (beneficence) may be reached by banning smoking, which negatively affects the bioethical principle of patient autonomy.¹⁵

Click to view larger

Figure. Physicians must always strive to find the best balance between the three central bioethical principles of beneficence/maleficence, justice, and autonomy.

With AI, the creators can measure how much each bioethical principle is being met through the principle of metrics for ethics.¹⁶ For example, we (with the FDA) created metrics for measuring bias in AI algorithms and the effect on a given population. One example is the concept of population-achieved sensitivity, which measures how many patients a diagnostic autonomous AI can identify in an entire population. If an AI system is highly sensitive but works only on a small subset of patients, the population-achieved sensitivity will be lower compared with an autonomous AI system that works for the majority of the population but has a slightly lower sensitivity—more of the total number of true cases are identified.

ETHICS IN HEALTH CARE

Any given health care process can improve outcomes for a patient or an entire population, thereby maximizing the bioethical principle of beneficence. If this assumed improvement is not evenly distributed across the patient group or population, the bioethical principle of justice (ie, equity) is negatively affected, and health inequity is the result. When stakeholders within the health care system take a more active role in allocation to improve beneficence, justice, or both, the bioethical principle of autonomy may be infringed upon. The trick is finding a balance between the three principles.

Bias in any part of the health care process may lead to inequity and, in the past, has resulted in poorer health outcomes for specific underrepresented, underserved, and underresourced groups.^19,20

For example, a recent study found that providers’ charts documented Black patients’ symptoms and signs in a more pejorative manner,²¹ which has the potential to exacerbate health disparities. Other studies suggest physician bias in caring for other populations as well.^22-24 In retina, a recent study showed that reading center confidence in evaluating fundus images for diabetic eye disease was lower in more pigmented retinas.²⁵

Such bias in clinical practice reduces the bioethical principle of justice as described by Char et al and Abramoff et al.^15,16 Using the metrics for ethics, bias and its effect on beneficence and justice can be quantified to better understand the differential effect of a health care process on a particular group.

IMPROVING ETHICS IN PRACTICE

To mitigate health care bias, we can apply our ethical AI framework to our daily practice as retina specialists. To start, we can translate the AI creation phases into descriptions relevant to clinical practice as follows:

• Conception: target disease and population
• Design: management and treatment (ie, how we choose to treat the disease)
• Development and validation: clinical training
• Access and marketing: practice characteristics
• Monitoring: follow-up and reporting

Using this approach, we can analyze the potential bias in each of these aspects; we can also assess the potential mitigation strategies, although the capability of the individual retina specialist to affect these may vary. The primary goal of such analysis is to provide transparency about potential sources of bias and health inequity, and the sustainability of any mitigation based on the financial and time resource constraints that must be adequately balanced.

Target disease and population. Invisible populations are the often-large segments of the population that are underrepresented, underserved, underresourced, and rarely or never get proper eye care. For example, only 15.3% of adults with diabetes who are recommended to get an annual examination actually get it.²⁶ This means that more than 80% of Americans are invisible in this regard, and this is the case with many, especially chronic, eye diseases. Typically, these invisible populations are underrepresented in (phase 3) clinical trials validating new treatments, limiting our ability to understand their efficacy, if any, in these patients. Mitigating such bias is a primary concern in the design and evaluation of clinical trials, and not so much something that the individual specialist can control.

Management and treatment. Mitigating potential bias in the efficacy of any given therapy for certain culturally, genetically, or otherwise characterized subgroups is another concern for those engineering the therapies themselves, not necessarily individual practitioners.

Clinical training. Medical school, residency, fellowship, and exposure to various populations can exacerbate or mitigate bias when dealing with patients in these populations. For example, during a study of children with diabetes, my colleagues and I found anecdotal evidence that diabetic eye disease may manifest differently in young patients. Although this is now the subject of further research, the current literature pertaining to diabetic eye disease in the pediatric population is lacking—a clear example of age bias. As another example, learning how to diagnose retinal disease in eyes with various levels of retinal pigmentation or different amounts of pupil dilation can limit unconscious bias in clinic and allow better care across many populations.

Practice characteristics. Where we decide to practice, and which populations we see (often decided by which payer contracts we engage in), can introduce bias.

Monitoring. Reporting in systems such as the AAO’s IRIS registry itself can be influenced by the populations in which we practice and how we treat and manage our patients. Such a bias in the registry can be mitigated by ensuring diversity among practitioners.

FROM AI TO CLINICAL PRACTICE

There are many sources of bias in clinical practice that have the potential to affect health equity, and they can be analyzed using our AI framework. Improving health equity starts with awareness. The good thing is that the bias that creates invisible populations can be addressed directly through highly scalable autonomous AI diagnostics, turning them into visible populations and providing them much-needed specialized retina care.¹⁰