This year, Circle U. is supporting 13 seed-funded projects. Over the next few months, each project leader will present their initiative. This month, we are taking a closer look at the project Algorithmic Regulation Before Medical Liability. Advancing Doctor-Patient Autonomy in AI-Driven Healthcare (AlgoNomy).
The project AlgoNomy is led by Nicolò Amore, Assistant Professor in Criminal Law at the University of Pisa. We’ve asked Nicolò Amore to share some insights about his project and what his team is aiming to achieve this year. Let’s have a closer look at AlgoNomy.
What is your project about?
AlgoNomy aims at tackling the emerging issue of “digital paternalism” in healthcare, an adaptation of medical paternalism, where healthcare decisions are primarily made by algorithms rather than jointly by patients and physicians. To this aim, we will bring together scholars from the fields of law, medicine, and computer science to investigate the impact of AI on the autonomy of patients and healthcare professionals, while also considering their potential liability implications. AlgoNomy shall investigate how current AI medical applications fail to accommodate individual patient preferences and limit the physician’s ability to comprehend and influence treatment outcomes. Moreover, it will seek to identify strategies to mitigate these risks by leveraging the fruitful interplay among the disciplines involved, as an essential instrument to safeguard and fully realize the core principles of autonomy for both physicians and patients within the emerging landscape of AI-driven medicine.
Why is this topic important?
The increasing adoption of AI-based medical devices presents a critical challenge for healthcare: maintaining human agency in an increasingly data-driven medical environment. Clinical autonomy and patient self-determination are fundamental to ensuring that future healthcare remains genuinely human-centered, despite its growing reliance on artificial intelligence. Current AI medical devices often generate therapeutic recommendations through opaque algorithmic processes, analyzing only objective clinical data with limited opportunities for physician and patient interaction. Moreover, their implementation has frequently followed a simplistic approach, treating them merely as data collection tools while overlooking their significant impact on clinical decision-making processes and patients’ fundamental rights. This situation calls for innovative solutions in both the design and implementation of AI medical devices. AlgoNomy’s interdisciplinary approach aims to enhance the control and interaction capabilities of both healthcare providers and patients over clinical processes and outputs. This advancement is crucial as it would enable key stakeholders in the therapeutic journey to meaningfully influence both the process and outcomes of their medical care, thereby preserving human agency in AI-enhanced healthcare environments.
What are you aiming to achieve with this project over the year?
Over the course of the year, AlgoNomy aims to achieve significant milestones through a series of concrete activities. First of all, two interdisciplinary call for papers will be launched by the end of January, targeting both doctoral students and senior researchers, with a strong emphasis on co-authored contributions from diverse disciplines. The selected papers will be presented in May, during a lecture series at Pisa’s renowned Festival of Robotics, fostering public engagement and interdisciplinary dialogue. Also, to support early-career researchers, a summer doctoral workshop will be organized at one of the partner universities, providing them with an opportunity to refine their ideas under the supervision of the Scientific Committee. The SC, comprising experts from all project partners, will oversee these initiatives and will also edit the project’s proceedings for publication in an open-access, international journal by September. Additionally, the project will actively involve non-academic stakeholders, including businesses, patient organizations, and healthcare professionals, in public discussions. These efforts aim to build a collaborative network within the Circle-U alliance, fostering long-term partnerships and laying the groundwork for future European-funded research proposals. Through these activities, AlgoNomy seeks to advance interdisciplinary collaboration and promote innovative research on the human dimension of AI-driven medicine.
Explain in which way your project has an interdisciplinary approach.
In our vision, interdisciplinarity is not only beneficial, but rather it is crucial to meaningfully address the different domains intersected by AI application in healthcare. Indeed, the presence of experts in all the domains involved is crucial, because digital paternalism arises from programming choices and implementation methods that are inconsistent with the legal framework governing healthcare services. Therefore, a synergistic approach across all three disciplines is essential to ensure that the fundamental principles of autonomy for both the physician and the patient are fully realized also in the new AI-driven clinical setting. To achieve this objective, the SC composition will mirror both the consortium’s structure and the diverse expertise integral to the project. This diverse team will ensure that all aspects concerning the autonomy of both patients and clinicians in AI-driven medical settings are comprehensively examined from multiple points of view. Moreover, the call for paper will strongly encourage interdisciplinary and co-authored contributions. In addition, the speakers and the papers will be selected to evenly cover all the scientific perspectives involved. Finally, The PhD summer workshop will strengthen the project’s interdisciplinarity by facilitating discussions between young scholars and project partners from diverse fields.
