Table of Contents
I. Background and Scope............................................................................................... 1
II. Current and Potential Uses of AI/ML in Drug Development ....................................... 3
A. Drug Discovery................................................................................................... 3
1. Drug Target Identification, Selection, and Prioritization ....................................... 3
2. Compound Screening and Design....................................................................... 4
B. Nonclinical Research ......................................................................................... 5
C. Clinical Research ............................................................................................... 5
1. Recruitment ......................................................................................................... 6
2. Selection of Trial Participants .............................................................................. 6
3. Dose/Dosing Regimen Optimization .................................................................... 6
4. Adherence ........................................................................................................... 7
5. Retention ............................................................................................................. 7
6. Site Selection....................................................................................................... 7
7. Clinical Trial Data Collection, Management, and Analysis................................... 7
8. Clinical Endpoint Assessment.............................................................................. 8
D. Postmarket Safety Surveillance ......................................................................... 9
1. Case Processing.................................................................................................. 9
2. Case Evaluation................................................................................................. 10
3. Case Submission............................................................................................... 10
E. Advanced Pharmaceutical Manufacturing........................................................ 10
1. Optimization of Process Design......................................................................... 11
2. Advanced Process Control ................................................................................ 11
3. Smart Monitoring and Maintenance ................................................................... 12
4. Trend Monitoring................................................................................................ 12
F. FDA Experience with AI/ML for Drug Development ......................................... 12
III. Considerations for the Use of AI/ML in Drug Development..................................... 15
A. Overarching Standards and Practices for the Use of AI/ML............................. 15
B. Discussion of Considerations and Practices for AI/ML in Drug Development .. 17
IV. Next Steps: Engagement and Collaboration ........................................................... 23
Glossary ........................................................................................................................ 24
References.................................................................................................................... 27
1 I. Background and Scope
2
3 To fulfill its mission of protecting, promoting, and advancing public health, the Food and
4 Drug Administration’s (FDA’s) Center for Drug Evaluation and Research (CDER), in
5 collaboration with the Center for Biologics Evaluation and Research (CBER) and the
6 Center for Devices and Radiological Health (CDRH), including the Digital Health Center
7 of Excellence (DHCoE), is publishing this document to facilitate a discussion with
8 stakeholders on the use of artificial intelligence (AI)
1
and machine learning (ML)
2
in
9 drug development,
3,4
including in the development of medical devices intended to be
10 used with drugs, to help inform the regulatory landscape in this area.
11
12 FDA helps to ensure that drugs are safe and effective while facilitating innovations in
13 their development. Recent, rapid technological innovations in data collection and
14 generation tools, combined with robust information management and exchange systems
15 and advanced computing abilities, may transform the way drugs are developed and
16 used (ElZarrad, Lee, Purcell, & Steele, 2022). This evolving ecosystem presents
17 unique opportunities and challenges, and FDA is committed to working across its
18 medical product centers with partners domestically and internationally to ensure that the
19 full potential of these innovations is realized for the benefit of the public.
20
21 Developers, manufacturers, regulators, academic groups, and other stakeholders are
22 working to develop a shared understanding of where and how specific innovations, such
23 as AI and ML, can best be used throughout the drug development process. FDA is
24 publishing this discussion paper as part of a multifaceted approach to enhance mutual
25 learning and to establish a dialogue with FDA stakeholders on this topic. AI can
26 generally be described as a branch of computer science, statistics, and engineering that
27 uses algorithms or models to perform tasks and exhibit behaviors such as learning,
28 making decisions, and making predictions.
5
ML is considered a subset of AI that allows
29 ML models to be developed by ML training algorithms through analysis of data, without
30 models being explicitly programmed.
6
Additionally, there are a variety of ML methods
31 and different types of algorithms that may be utilized in a given context. For purposes
32 of this document, AI and ML will be referenced together as AI/ML, and references to
1
Words and phrases in bold italics are defined in the Glossary.
2
There are multiple definitions for AI and ML, and the Glossary includes several definitions from federal
legislation and agencies.
3
For purposes of this discussion paper, all references to drug or drugs include both human drugs and
biological products.
4
FDA is focusing this discussion paper on drug development. However, many of the AI/ML scientific and
regulatory science principles outlined in this document may be applicable across all medical products,
including in the development of medical devices intended to be used with drugs (including, but not limited
to, combination products, companion devices, and complementary devices). Some medical devices
intended to be used with drugs are intended for use only in clinical investigations; others are intended to
be marketed for use outside of clinical investigations. Examples include medical devices that help identify
side effects of drugs as well as medical devices that assist in drug dosing.
5
See IMDRF/AIMD WG/N67 Machine Learning-enabled Medical Devices: Key Terms and Definitions,
final document, May 6, 2022. https://www.imdrf.org/documents/machine-learning-enabled-medical-
devices-key-terms-and-definitions
6
Ibid.
1
33 drug development and the drug development process include a wide scope of activities
34 and phases, including manufacturing and postmarket drug safety monitoring, among
35 others.
7,8
36
37 This discussion paper, which considers the application of AI/ML in the broad context of
38 the drug development process, is not FDA guidance or policy and does not endorse a
39 specific AI/ML use or approach in drug development. Rather, this discussion paper is
40 an initial communication with stakeholders, including academic groups, researchers,
41 and technology developers, that is intended to promote mutual learning and discussion.
42 It is particularly beneficial for those new to drug development and human subjects
43 research, to recognize some of the initial thinking and considerations involved with
44 utilizing these technologies, including having familiarity with FDA’s current activities,
45 initiatives, practices, and potentially applicable regulations. FDA is soliciting feedback
46 on the opportunities and challenges with utilizing AI/ML in the development of drugs, as
47 well as in the development of medical devices intended to be used with drugs. This
48 feedback will provide an additional resource to help inform the regulatory landscape in
49 this area.
50
51 In this discussion paper, three main topics are discussed:
52
53 • Landscape of current and potential uses of AI/ML: FDA recognizes the
54 potential for AI/ML to enhance drug development in many ways, including to help
55 bring safe and effective drugs to patients faster; provide broader access to drugs
56 and thereby improve health equity; increase the quality of manufacturing;
57 enhance drug safety; and develop novel drugs and drug classes, as well as
58 personalized treatment approaches. Section II provides examples of the use of
59 AI/ML to highlight the potential impact of AI/ML across the drug development
60 process and includes a brief description of FDA’s experience with AI/ML in drug
61 development. The list of examples in this section is not comprehensive of all
62 AI/ML uses, and it includes uses where FDA oversight may or may not be
63 applicable. The purpose of this section is to promote shared learning and to
64 identify areas where future regulatory clarity may be helpful.
65
66 • Considerations for the use of AI/ML: FDA is also aware of the potential
67 concerns and risks with emerging innovations such as AI/ML and will share initial
68 considerations and solicit feedback on how to help ensure the responsible
69 utilization of AI/ML in drug development. Section III briefly describes several key
70 efforts to develop general principles, standards, and practices for the use of
71 AI/ML across diverse applications and then explores the principles and
72 considerations that may be particularly applicable when using AI/ML for drug
73 development activities. FDA understands that AI/ML use in drug development is
7
See The Drug Development Process, January 2018. https://www.fda.gov/patients/learn-about-drug-and-
device-approvals/drug-development-process
8
In this discussion paper, the topic of clinical investigations focuses on the drug development process,
however, many other activities and phases included as part of the drug development process may also be
part of the development process for other medical products; see footnote 4.
2
75
80
85
90
95
100
105
110
115
74 diverse, and careful assessments that consider the specific context of use are
needed. Taking a risk-based approach to evaluate and manage the use of AI/ML
76 can help facilitate innovations and protect public health.
77
78 • Next steps and stakeholder engagement: FDA is interested in mutual
79 opportunities to learn and engage with all stakeholders to establish a shared
understanding of AI/ML systems and their rapidly evolving potential uses and
81 considerations in drug development. As part of this ongoing effort, FDA
82 welcomes feedback on this discussion paper and any AI/ML-related issues
83 pertaining to drug development. Specifically, to initiate a broader dialogue with
84 stakeholders, Section III includes several key questions to which interested
parties can provide perspectives and Section IV outlines opportunities for future
86 engagement.
87
88 II. Current and Potential Uses of AI/ML in the Drug Development Process
89
This section provides a high-level overview of the diverse and evolving uses of AI/ML
91 being employed throughout the drug development process. These examples are not
92 comprehensive of all AI/ML uses and include uses where FDA oversight may or may
93 not be applicable.
9
Additionally, while some of the uses of AI/ML described in this
94 section may also have utility in clinical practice, this paper is focused on uses of AI/ML
in the drug development process. The purpose of this section is to promote shared
96 learning and to identify areas where future FDA regulatory clarity may be beneficial.
97
98 Although the overall drug development process is an iterative continuum of activities
99 and not strictly linear in nature, for simplicity, this section utilizes different phases of
drug development to highlight several uses of AI/ML, ranging from drug discovery and
101 clinical research to postmarket safety surveillance and advanced pharmaceutical
102 manufacturing. The section also includes references to how AI/ML is being applied to
103 real-world data (RWD) and data from digital health technologies (DHTs) in support
104 of drug development. Some of the general challenges and considerations with utilizing
AI/ML in different drug development use cases are discussed in Section III.
106
107 A. Drug Discovery
108
109 Early drug discovery is one of the areas with significant interest and activity in utilizing
AI/ML. Included below is a brief discussion of the current and potential uses of AI/ML
111 for drug target identification, selection, and prioritization, as well as compound
112 screening and drug design in drug discovery.
113
114 1. Drug Target Identification, Selection, and Prioritization
9
The examples listed were not necessarily submitted to FDA for review or approval and are not meant to
suggest an endorsement of any specific approach. The FDA does not endorse any particular use of
AI/ML.
3
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
The early stages of drug development generally rely on the initial identification of a
suitable biological target for drug candidates. As a starting point, the process of
identifying biological targets and elucidating disease relationships can utilize AI/ML to
analyze and synthesize significant amounts of information from existing scientific
research, publications, and other data sources. The growth of available genomic,
transcriptomic, proteomic, and other data sources from healthy persons and those with
a specific disease of interest provide a significant opportunity to inform biological target
selection. These datasets are often complex and originate from disparate sources,
which can be well-suited for the utilization of AI/ML approaches (Fumagalli et al., 2023).
Building from existing validated data, AI/ML can be applied to mine and analyze these
large multi-omics and other datasets to provide information on the potential structure
and function of biological targets to predict their role in a disease pathway (Vamathevan
et al., 2019; Weissler et al., 2021). While early target identification and prioritization is a
critical step where AI/ML could help improve the efficiency and effectiveness of drug
development, it is important to validate the role of the biological target in the disease of
interest through subsequent studies (Fumagalli et al., 2023).
2. Compound Screening and Design
The discovery of potential drug candidates that modify the function of the identified
biological targets of interest generally involves significant in silico or experimental
screening of compound libraries, followed by subsequent refinement of a compound’s
specificity and selectivity for the biological target. In the area of compound screening,
potential AI/ML uses include predicting the chemical properties and bioactivity of
compounds and predicting efficacy and potential adverse events based on the
compound’s specificity and affinity for a target (Chan, Shan, Dahoun, Vogel, & Yuan,
2019; Schneider et al., 2020).
AI/ML approaches used to further elucidate drug-target interactions could also help
provide predictions about classes of drugs potentially interacting with the same targets
or having a similar mechanism of action, which may help predict the toxicity of a
molecule based on specific known features. This strategy can help guide drug
repurposing efforts that could utilize previously characterized compounds. Drug
repurposing efforts utilizing AI/ML can also potentially benefit from the increased
availability of suitable RWD from a variety of sources (e.g., electronic health records
(EHRs), registries, and DHTs) to identify previously unknown effects of drugs on
disease pathways (Z. Liu et al., 2022).
Finally, AI/ML could accelerate advances in de novo drug design (Mouchlis et al., 2021).
For example, AI/ML may be applied to help predict the 3D structure of target proteins,
informing chemical synthesis and the potential effect of a drug candidate on the target,
including predicting affinity and potential toxicity (Chan et al., 2019; Jumper et al., 2021;
Vamathevan et al., 2019). It is worth noting that one must be cautious with the use of
AI/ML in 3-D structure prediction, as many proteins that are developed for
pharmaceutical applications are codon optimized (with many synonymous mutations
4
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
incorporated), the impact of which on protein structure is still an area of active research
(Fumagalli et al., 2023; Jumper et al., 2021).
B. Nonclinical Research
Nonclinical research refers to in vitro and in vivo studies and is designed to further
advance potential therapeutics towards clinical research in humans. Nonclinical
studies, in support of new drug development, can be conducted at all phases of
development: prior to clinical studies, in parallel with clinical development, and even in
postmarketing environments. Data from pharmacokinetic, pharmacodynamic, and
toxicologic studies conducted in animals; exploratory in vitro and in vivo mechanistic
studies conducted in animal models; organ-on-chip and multi-organ chip systems; and
cell assay platforms may be leveraged using AI/ML (e.g., computational modeling and
simulation techniques) for evaluating toxicity, exploring mechanistic models, and
developing in vivo predictive models (Bulitta et al., 2019; Harrison & Gibaldi, 1977; Hsu
et al., 2014; Mager, Woo, & Jusko, 2009; Shroff et al., 2022).
Pharmacokinetics (PK) describes the time course of drug absorption, distribution,
metabolism, and excretion. Pharmacodynamics (PD) explores the body’s biological
response to drugs. When PK and PD are integrated in a model, the model can describe
how the drug effect will change with time when a certain dose or dosing regimen is
used. Pharmacokinetic/pharmacodynamic (PK/PD) modeling has been used in drug
development for decades and can be applied at both the nonclinical and clinical stages
(Daryaee & Tonge, 2019). Along with the advances in computational tools and
technology and the availability of modeling platforms, use of physiologically-based
pharmacokinetic (PBPK) and physiologically-based PK/PD (PBPK-PD) modeling is also
increasing (Sager, Yu, Ragueneau-Majlessi, & Isoherranen, 2015). There are current
efforts to explore the use of more novel AI/ML algorithms (e.g., artificial neural network
models and tree-based models) for PK/PD modeling. For example, a recurrent neural
network, an ML algorithm commonly used for analyzing time series data, may be used
to complement traditional PK/PD models in the area of highly complex PK/PD data
analysis, and possibly lead to improved accuracy for nonclinical and clinical
applications (Liu et al., 2021).
C. Clinical Research
Clinical research typically involves a series of phases of clinical trials in increasing
numbers of human subjects to assess the safety and effectiveness of a drug. One of
the most significant applications of AI/ML in drug development is in efforts to streamline
and advance clinical research. For example, AI/ML is being utilized to analyze vast
amounts of data from both interventional studies (also referred to as clinical trials) and
non-interventional studies (also referred to as observational studies) to make inferences
regarding the safety and effectiveness of a drug. Additionally, AI/ML has the potential to
inform the design and efficiency of non-traditional trials such as decentralized clinical
trials, and trials incorporating the use of RWD extracted from EHRs, medical claims, or
other data sources. AI/ML may also have a role in analyzing and interpreting data
5
207 collected from DHTs used in clinical studies. Finally, AI/ML could also be used to
208 improve the conduct of clinical trials and augment operational efficiency. The following
209 subsections will highlight some of the uses and potential uses of AI/ML during the
210 design and conduct of clinical research.
211
212 1. Recruitment
213
214 AI/ML is increasingly being developed and used to connect individuals to trials for
215 investigational treatments from which participants may benefit. Specifically, AI/ML is
216 being used to mine vast amounts of data, such as data from clinical trial databases, trial
217 announcements, social media, medical literature, registries, and structured and
218 unstructured data in EHRs, which can be used to match individuals to trials (Harrer,
219 Shah, Antony, & Hu, 2019). While these algorithms are trained on high volumes of
220 patient data and enrollment criteria from past trials, it is important to ensure adequate
221 representation of populations that are likely to use the drug (e.g., gender, race, and
222 ethnicity) as matching algorithms are created and, when used, to confirm that equitable
223 inclusion was achieved during the recruitment process. In the future, these
224 technologies, if properly validated, may continue to play an increasing role in matching
225 individuals with investigational treatments.
226
227 2. Selection and Stratification of Trial Participants
228
229 Enrichment strategies can aid participant selection in clinical investigations designed to
230 demonstrate the effectiveness of drug and biological products.
10
AI/ML has been
231 explored and used as part of a clinical investigation in the prediction of an individual
232 participant’s clinical outcome based on baseline characteristics (e.g., demographic
233 information, clinical data, vital signs, labs, medical imaging data, and genomic data)
234 (Aerts et al., 2016; Athreya et al., 2019; Dercle et al., 2020; Harrer et al., 2019;
235 Kawakami et al., 2019). Such predictive models can be used to enrich clinical trials
236 (e.g., identifying high-risk participants or participants more likely to respond to the
237 treatment). When these types of AI/ML algorithms are used for patient evaluation and
238 selection before randomization, it may be possible to reduce variability and increase
239 study power (Y. Wang, Carter, Li, & Huang, 2022).
240
241 In addition to utilization in enrichment strategies, such predictive models can also be
242 used for participant stratification, for example, if an AI/ML model could predict the
243 probability of a serious adverse event before an investigational treatment is
244 administered. Based on their predicted risk for these serious adverse events,
245 participants can be stratified into different groups and then monitored accordingly (or
246 excluded depending on predicted severity of the adverse event).
247
248 3. Dose/Dosing Regimen Optimization
249
10
See the guidance for industry Enrichment Strategies for Clinical Trials to Support Determination of
Effectiveness of Human Drugs and Biological Products (March 2019).
https://www.fda.gov/media/121320/download
6
250
255
260
265
270
275
280
285
290
AI/ML can be used to characterize and predict PK profiles after drug administration. It
251 can also be used to study the relationship between drug exposure and response, taking
252 into consideration confounding factors. These kinds of models can be used to optimize
253 the dose/dosing regimen selection for a study (Liu et al., 2021; Lu, Deng, Zhang, Liu, &
254 Guan, 2021). This could potentially include aiding in dose optimization in special
populations where there may be limited data (e.g., rare disease studies, pediatric and
256 pregnant populations).
257
258 4. Adherence
259
AI/ML can be used to monitor and improve adherence during a clinical trial through
261 tools, such as smartphone alerts and reminders, eTracking of medication (e.g., smart
262 pillboxes and tools for visual confirmation) (Mason et al., 2022), and eTracking of
263 missed clinical visits, which trigger non-adherence alerts. Examples of AI/ML used in
264 clinical research to improve medication adherence include applications using digital
biomarkers, such as facial and vocal expressivity, to monitor adherence remotely.
266
267 5. Retention
268
269 AI/ML has the potential to improve the participants’ access to relevant trial information
by enabling tools, such as AI chatbots, voice assistance, and intelligent search. AI/ML
271 can also be used to reduce the burden for participants by using passive data collection
272 techniques and by extracting more information from available data generated during
273 clinical practice or by study activities (Weissler et al., 2021). Additionally, data from
274 DHTs and other systems can be used to develop patient profiles to potentially predict
dropouts and adverse events to ensure participant retention.
276
277 6. Site Selection
278
279 Trial operational conduct could also be optimized by utilizing AI/ML to help identify
which sites have the greatest potential for a successful trial and to aid sites in identifying
281 process gaps. For example, algorithms can be used to evaluate site performance and
282 to help determine which sites may have a higher risk of running behind schedule based
283 on data from other trials at that site.
284
7. Clinical Trial Data Collection, Management, and Analysis
286
287 a. Data Collection
288
289 DHTs, such as wireless and smartphone-connected products, wearables, implantables,
and ingestibles, are increasingly being used in clinical trials to collect objective,
291 quantifiable, longitudinal, and continuous physiological data.
11
In addition, many of
292 these DHTs enable the use of AI/ML, either as embedded algorithms within the DHT or
11
See the draft guidance for industry, investigators, and other stakeholders Digital Health Technologies
for Remote Data Acquisition in Clinical Investigations (December 2021). When final, this guidance will
represent FDA’s current thinking on this topic. https://www.fda.gov/media/155022/download
7
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
employed upon the data generated after the data are collected from the DHT, and have
been used to predict the status of a chronic disease and its response to treatment
(Stehlik et al., 2020) or to identify novel characteristics of an underlying condition
(Avram et al., 2020). AI/ML can be utilized to analyze the large and diverse data
generated from the continuous monitoring of persons using these technologies. This
could include using AI/ML to aid in the evaluation of multimodal data and composite
measures that may combine individual measures collected through multiple DHTs
(Cohoon & Bhavnani, 2020).
b. Data Management
AI/ML can be used for a range of data cleaning and curation purposes, including
duplicate participant detection and imputation of missing data values (Zhang, Yan, Gao,
Malin, & Chen, 2020), as well as the ability to harmonize controlled terminology
across drug development programs. Use of AI/ML could also significantly enhance data
integration efforts by using supervised and unsupervised learning to help integrate data
submitted in various formats and perform data quality assessments. Additionally, AI/ML
can be used for data curation via masking and de-identification of personal identifiable
information, metadata creation, and search and retrieval of stored data. These
applications can potentially increase data accuracy and improve the speed at which
data are prepared for analyses.
c. Data Analysis
AI/ML has been used to analyze high volumes of diverse and complex RWD extracted
from EHRs, medical claims, and disease registries, among other sources. Additionally,
the use of AI/ML in predictive modeling and counterfactual simulation to inform clinical
trial designs is being actively explored. For example, in silico clinical trials utilize
computational modeling and simulation to evaluate drug candidates using a virtual
cohort of simulated participants with realistic variability of traits representing the desired
participant population (Pappalardo, Russo, Tshinanu, & Viceconti, 2019). AI/ML could
be employed in these situations to aid in evaluating a vast number of counterfactual
simulations and to predict trial outcomes before human trials.
At an even more personalized level, AI/ML can also be used in the context of digital
twins of patients, an emerging method that could potentially be used in clinical research.
To create digital twins of patients, AI/ML can be utilized to build in silico representations
or replicas of an individual that can dynamically reflect molecular and physiological
status over time (European Medicines Agency, 2022; Laubenbacher, Sluka, & Glazier,
2021; Schuler et al., 2021). In comparison to a participant in a clinical trial that received
an investigational treatment, the digital twin could potentially provide a comprehensive,
longitudinal, and computationally generated clinical record that describes what may
have happened to that specific participant if they had received a placebo.
8. Clinical Endpoint Assessment
8
340
345
350
355
360
365
370
375
380
339 Clinical endpoint assessment is a key part of evaluating safety and efficacy of medical
interventions in clinical trials. AI/ML-enabled algorithms could detect clusters of signs
341 and symptoms to identify a potential safety signal, as well as help detect cases with
342 safety issues in real time (Pierce et al., 2017; Routray et al., 2020). AI/ML could be
343 used to assist in the assessment of outcomes captured from diverse sources (e.g.,
344 DHTs, social media) during a clinical trial, including those consisting of large amounts of
data for which manual review may be impractical.
346
347 D. Postmarketing Safety Surveillance
348
349 For purposes of this paper, pharmacovigilance (PV) refers to the science and activities
related to the detection, assessment, understanding, and prevention of adverse events
351 or any other drug-related problems (including medication errors and product quality
352 issues).
12
Postmarketing safety surveillance, or PV activities in the post-approval
353 period, includes postmarketing safety reporting of adverse events associated with use
354 of human drug and biological products. An individual case safety report (ICSR) is used,
as applicable, for the postmarketing reporting of adverse events to FDA and serves as
356 an important data source of potential drug safety issues for postmarket safety
357 surveillance. The clinical information in ICSRs can include suspect product or products,
358 and temporal information related to use of the product and occurrence of the adverse
359 event(s) in the patient’s medical history, clinical course, and outcome. Complete and
accurate reporting of ICSRs is critical to the understanding of a drug’s safety profile.
361 For reasons including increases in ICSR volume, AI/ML applications are being explored
362 to help process and evaluate ICSR submissions within regulatory agencies (Ball & Dal
363 Pan, 2022; Bate & Hobbiger, 2021).
364
1. Case Processing
366
367 There are potential opportunities to use AI/ML for automation during ISCR processing.
368 The number and complexity of data sources of adverse events for ICSRs have
369 increased, including from spontaneous reports, clinical trials, EHRs, social media,
phone calls, emails, literature, patient registries, claims data, and post-approval safety
371 studies (Beninger, 2020). The use of AI/ML to detect information from source
372 documents could help identify adverse events for ICSR submission. For instance, the
373 use of AI/ML to detect and evaluate drug event associations from literature and to
374 screen social media for adverse events has been explored (Comfort, Dorrell, Meireis, &
Fine, 2018; Negi, Pavuri, Patel, & Jain, 2019; S. V. Wang et al., 2017; W. Wang et al.,
376 2011).
377
378 After an adverse event is identified from a data source, AI/ML could be used for case
379 validity, case prioritization, duplicate check, coding, and quality control. The use of
AI/ML can help identify whether a case is a valid case, which includes determining
12
See the guidance for industry Good Pharmacovigilance Practices and Pharmacoepidemiologic
Assessment (March 2005). Accessed September 30, 2022. https://www.fda.gov/media/71546/download
See also, Council for International Organizations of Medical Sciences (CIOMS) Pharmacovigilance
definition. Accessed September 29, 2022. https://cioms.ch/pharmacovigilance/
9
381 whether a case contains the minimum reporting requirements, such as an identifiable
382 patient, suspect drug or biological product, adverse event(s), and identifiable reporter
383 (Abatemarco et al., 2018; Schmider et al., 2019). During case intake, to assist in the
384 prioritization of cases, AI/ML has been used to classify adverse events by expectedness
385 (whether an adverse event is known and in the product labeling) (Abatemarco et al.,
386 2018; Routray et al., 2020). Automated duplicate checks using AI/ML are being
387 conducted to identify whether the case is a true duplicate, a follow up version of a prior
388 case, or a new case (Kassekert 2022). Another area in which AI/ML has been applied
389 is the coding of adverse events described in ICSRs to structured medical dictionary
390 terms and for quality control purposes (Ghosh 2020).
391
392 2. Case Evaluation
393
394 Adverse event cases undergo clinical assessment. Case evaluation includes assessing
395 the possibility of a causal relationship between the drug and adverse event, as well as
396 assessing the outcome of the case. An AI model was developed based on relevant
397 features used in causality assessments; it was trained, validated, and tested to classify
398 cases by the probability of a causal relationship between the drug and adverse event
399 (Comfort et al., 2018). AI/ML has also been applied to determine seriousness of the
400 outcome of ICSRs (Routray, et al., 2020), which not only supports case evaluation, but
401 also the timeliness of individual case submissions that require expedited reporting.
402
403 3. Case Submission
404
405 Generally, the final step after case processing is the submission of ICSRs. AI/ML
406 algorithms have been used to automate reporting rules for submission of ICSRs to FDA.
407 The reporting of ICSRs is required on an individual basis, as well as in aggregate
408 (Ghosh et al., 2020). The aggregate reporting of adverse events generally involves the
409 compilation of safety data for a product that is submitted at regular time intervals as
410 specified. AI/ML can be used to develop aggregate reports that include multiple
411 adverse events for particular products that occur within a time period for reporting
412 purposes (Lewis & McCallum, 2020).
413
414 E. Advanced Pharmaceutical Manufacturing
13
415
416 A critical aspect of drug development includes the methods, facilities, and controls used
417 in manufacturing, processing, packing, and holding of a drug to help ensure that the
418 drug meets the requirements of safety and effectiveness, has the identity and strength it
419 is represented to possess, and meets quality and purity characteristics. Advanced
13
The examples in this section are based on the review of general published information that projects or
forecasts how AI/ML may be currently used in the pharmaceutical manufacturing space. In the continued
spirit of FDA’s recent engagement through the Quality Metrics Feedback Program and CDER’s Emerging
Technology Program, FDA has been able to solicit valuable feedback demonstrated by industry
interactions on several AI/ML use cases in the pharmaceutical manufacturing space, such as optimal risk-
based supply chain modeling, business forecasting, process optimization, application of natural language
processing (NLP) algorithms for complaints reduction, use of predictive analytics for non-conformance
(NC) reduction, and corrective and preventive action (CAPA) effectiveness.
10
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
analytics leveraging AI/ML in the pharmaceutical manufacturing industry offers many
possibilities, including, but not limited to, enhancing process control, increasing
equipment reliability and throughput, monitoring early warnings or signals that the
manufacturing process is not in a state of control, detecting recurring problem clusters,
and preventing batch losses. The use of AI/ML to support pharmaceutical
manufacturing can be deployed together with other advanced manufacturing
technologies (e.g., process analytical technology, continuous manufacturing) to achieve
the desired benefits. AI/ML is an enabler for the implementation of Industry 4.0, a term
that refers to the fourth industrial revolution that brings together rapidly evolving
technologies, and could result in a well-controlled, hyper-connected, digitized
ecosystem and pharmaceutical value chain for the manufacturer (Arden et al., 2021).
AI/ML could also be used to improve the reliability of the manufacturing supply chain
through forecasting product demand, analyzing production schedules, estimating and
mitigating the impact of potential disruptions, and optimizing inventory. Use of AI/ML-
based approaches in pharmaceutical manufacturing can be broadly grouped into the
areas outlined below that cover the entire drug manufacturing life cycle, from design to
commercial manufacturing.
1. Optimization of Process Design
Digital twins can also be used in process design optimization. In this context, a digital
twin of a process is a digital replica of the physical process used to better understand,
analyze, predict, and optimize process performance. The digital twin could be
especially beneficial for analyzing manufacturing processes characterized by a limited
amount of development data, where AI/ML models could potentially leverage prior
knowledge of the product and process (e.g., from previous studies, development
programs, and scientific literature) to more quickly identify the optimal processing
parameters, thus reducing design time and waste.
2. Advanced Process Control
Process controls have been implemented in pharmaceutical manufacturing for several
decades. Traditional process controls maintain input process parameters at set points,
but are not capable of simultaneously changing multiple input parameters to maintain
the output parameters at desired levels to optimize the process. On the other hand,
advanced process control (APC) allows dynamic control of the process to achieve a
desired output (Huang et al., 2021). AI/ML techniques such as neural networks, with
real-time process data as inputs, can be used to implement APC. These methods can
also be used to develop process controls that can predict whether a process is
performing under a state of control by using AI/ML tools in combination with real-time
sensor data, including, in conjunction with smart monitoring of production lines, to
improve existing manufacturing line efficiency and output. In the near term, APC
approaches that combine physics and chemistry knowledge with AI/ML techniques are
expected to be increasingly adopted and have already been reported by several
pharmaceutical manufacturers (National Academies of Sciences, 2021). In these APC
applications, high quality model inputs inform process understanding and, model
11
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
structure. These robust inputs, when combined with data-driven modeling, allow
derivation of model parameters. These models leverage data required for model
development while improving model robustness.
3. Smart Monitoring and Maintenance
Manufacturing processes can be automated and monitored in real time, leading to more
efficient inventory management with shorter lead times and increased production
output, without impacting product quality. AI/ML methods can be used to monitor
equipment and detect deviations from normal performance that can trigger maintenance
activities, thus reducing process downtime. Another example is the use of computer
vision-based quality control that uses images (e.g., images of packaging, labels, or
glass vials) that are analyzed by AI/ML-based software to detect deviations and to
ensure images match the requirements of a given quality attribute of a product.
Augmenting human visual inspection of drug products and packaging with such AI/ML-
based methods can improve the accuracy and efficiency of visual inspection controls.
4. Trend Monitoring
AI/ML can be used in many ways to make manufacturing more effective and efficient
with faster output, less waste, more informed decision-making, and enhanced quality
control. Current practice for the analysis of deviations in the process is primarily done
by quality personnel and relevant subject matter experts. AI/ML could be utilized to
assist in examination of deviation reports that mostly contain large volumes of data or
text to analyze manufacturing-related deviation trends, cluster problem areas, and
prioritize areas for proactive continual improvement. This offers the advantage of
expediting the process of identifying root causes, as solely manual review of deviation
trends can be very time-consuming. AI/ML methods integrated with process
performance (Ppk) and process capability (Cpk) metrics can be used to proactively
monitor manufacturing operations for trends and out-of-control events, and predict
thresholds for triggering CAPA effectiveness evaluations.
F. FDA Experience with AI/ML for Drug Development
FDA recognizes the increased use of AI/ML throughout the drug development life cycle
and its potential to accelerate the development of safe and effective drugs. AI/ML is
increasingly integrated in areas where FDA is actively engaged, including clinical trial
design, DHTs, and RWD analytics. Over the last few years, FDA has seen a rapid
growth in the number of submissions that reference AI/ML. Submissions across drug
and biological product applications that include AI/ML have increased over the last few
years to more than 100 submissions in 2021 (Q. Liu et al., 2022). These submissions
cut across a range of therapeutic areas, and the uses of AI/ML within the submissions
cover the many different areas of the drug development process highlighted in this
section, from drug discovery and clinical trial enrichment to endpoint assessment and
postmarket safety surveillance. Inclusion of AI/ML in the clinical development/research
phase represents the most common stage for AI/ML uses in submissions.
12
512
513 One of the ways FDA has been supporting the development of innovative and robust
514 AI/ML is through the establishment of the CDER AI Steering Committee (AISC), which
515 coordinates efforts around AI/ML uses across therapeutic development. Leveraging its
516 commitment to advancing innovative approaches and promoting collaborative efforts
517 across the Agency, CDRH, including the DHCoE, have provided consults for drug
518 submissions that involve AI/ML, and are developing a framework for AI/ML-based
519 devices, including predetermined change control plans for devices incorporating
520 AI/ML,
14
as well as a foundation for Good Machine Learning Practices for medical
521 device development.
15
In addition, FDA has organized various workshops
16,17
and held
522 a Patient Engagement Advisory Committee (PEAC) meeting on DHT and AI/ML-related
523 topics
18
and has fostered regulatory science research, including on robustness, user-
524 centered transparency, and bias identification and management, through external
525 academic and clinical partnerships to evaluate the safety and effectiveness of emerging
526 AI/ML products.
19
527
528 Additionally, CDER has developed the Innovative Science and Technology Approaches
529 for New Drugs (ISTAND) Pilot Program, which is designed to expand drug
530 development tool (DDT) types included in the DDT qualification programs, including
531 tools that leverage DHTs. Applications of AI/ML may represent novel DDTs or could be
532 used to aid in the interpretation and analysis of traditional DDTs (such as biomarkers
533 or clinical outcome assessments), potentially speeding novel therapeutics to patients
534 by enhancing the evidence available for decision-making.
20
In the area of model-
535 informed drug development (MIDD), FDA’s CDER and CBER have established a MIDD
536 Pilot Program to facilitate the development and application of exposure-based,
537 biological, and statistical models derived from nonclinical and clinical data sources.
21
In
14
Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-
Based Software as a Medical Device (SaMD) – Discussion Paper and Request for Feedback, April 2019.
https://www.fda.gov/files/medical%20devices/published/US-FDA-Artificial-Intelligence-and-Machine-
Learning-Discussion-Paper.pdf
15
Good Machine Learning Practice for Medical Device Development: Guiding Principles, October 2021.
https://www.fda.gov/medical-devices/software-medical-device-samd/good-machine-learning-practice-
medical-device-development-guiding-principles
16
See the Virtual Public Workshop – Transparency of Artificial Intelligence/Machine Learning-enabled
Medical Devices, October 14, 2021. https://www.fda.gov/medical-devices/workshops-conferences-
medical-devices/virtual-public-workshop-transparency-artificial-intelligencemachine-learning-enabled-
medical-devices
17
See the Public Workshop – Evolving Role of Artificial Intelligence in Radiological Imaging, February
25–26, 2020. https://www.fda.gov/medical-devices/workshops-conferences-medical-devices/public-
workshop-evolving-role-artificial-intelligence-radiological-imaging-02252020-02262020
18
See the Patient Engagement Advisory Committee Meeting Announcement, October 22, 2020.
https://www.fda.gov/advisory-committees/advisory-committee-calendar/october-22-2020-patient-
engagement-advisory-committee-meeting-announcement-10222020-10222020
19
See CERSI research projects, October 2022. https://www.fda.gov/science-research/advancing-
regulatory-science/cersi-research-projects
20
See the guidance for industry and FDA staff Qualification Process for Drug Development Tools
(November 2020). https://www.fda.gov/media/133511/download
21
See the Model-Informed Drug Development Paired Meeting Program, October 2022.
https://www.fda.gov/drugs/development-resources/model-informed-drug-development-pilot-program
13
538 the context of MIDD, AI/ML could be employed to help improve clinical trial simulations,
539 optimize dose selection or estimations, or enhance predictive or mechanistic safety
540 evaluations.
541
542 In the area of postmarket safety surveillance, the FDA’s Sentinel Initiative, including
543 CDER’s Sentinel System,
22
CBER’s Biologics Effectiveness and Safety (BEST)
544 system,
23
and CDRH’s National Evaluation System for health Technology (NEST)
24
545 efforts, are exploring AI/ML approaches to improve existing systems. The FDA outlined
546 its goals for using linked claims and EHR data supported by advanced analytics in the
547 5-year Sentinel System strategic plan.
25
The Sentinel System Innovation Center has
548 outlined a four-pronged approach to implement this plan by incorporating emerging data
549 science innovations and EHR data for medical product safety surveillance: (1) data
550 infrastructure, (2) feature engineering, (3) causal inference, and (4) detection analytics
551 (Desai et al., 2021). Examples of AI/ML applications in this approach include natural
552 language processing (NLP) and automated feature extraction from unstructured EHR
553 clinical notes for computable phenotyping and improved confounding adjustment from
554 EHR-based variables using advanced statistical and ML approaches, such as
555 algorithms created to enhance performance or “Super Learner” and targeted maximum
556 likelihood estimation (Naimi & Balzer, 2018).
557
558 CBER’s BEST system is designed to provide better data sources, methods, tools,
559 expertise, and infrastructure to conduct surveillance and epidemiological studies.
26
Part
560 of this program is an effort to use AI/ML methods to analyze EHRs to predict or better
561 understand adverse events associated with the use of biological products and other
562 products that CBER regulates. This work may also enhance FDA’s understanding of
563 the use of AI/ML methods for generating real-world evidence about product efficacy.
564
565 CDER is also exploring the application of AI to enhance the evaluation of ICSRs
566 submitted to the FDA Adverse Event Reporting System (FAERS) (Ball & Dal Pan,
567 2022). The Information Visualization Platform (InfoViP) was developed with AI/ML to
568 detect duplicate ICSRs, classify ICSRs by level of information quality, and derive
569 visualization of the timeline of clinical events to aid in analysis of reported adverse
570 events (Kreimeyer et al., 2022; Kreimeyer et al., 2021; Spiker et al., 2020). AI/ML
571 methods have been investigated to automate the identification of adverse events in drug
572 product labeling to support safety reviewers in the triaging of ICSRs to facilitate the
573 identification of unknown or unexpected safety issues (Bayer et al., 2021; Ly et al.,
574 2018). Another AI-based tool that focuses on drug product labeling and is currently in
22
See FDA’s Sentinel Initiative, December 2022. https://www.fda.gov/safety/fdas-sentinel-initiative
23
See the CBER Biologics Effectiveness and Safety (BEST) System, March 2022.
https://www.fda.gov/vaccines-blood-biologics/safety-availability-biologics/cber-biologics-effectiveness-
and-safety-best-system
24
See the National Evaluation System for health Technology (NEST), October 2019.
https://www.fda.gov/about-fda/cdrh-reports/national-evaluation-system-health-technology-nest
25
See the FDA Sentinel System Five-Year Strategy, January 2019.
https://www.fda.gov/media/120333/download
26
See the CBER BEST System, March 2022. https://www.fda.gov/vaccines-blood-biologics/safety-
availability-biologics/cber-biologics-effectiveness-and-safety-best-system
14
575
580
585
590
595
600
605
610
615
use is the Computerized Labeling Assessment Tool (CLAT), which serves to automate
576 the review of label and labeling (e.g., prescribing information, carton and container
577 labeling). NLP and ML are also being explored to classify free-text narratives in FAERS
578 ICSRs into structured medical dictionary medication error terminologies to support the
579 human review of coding quality. Additionally, through the FDA Quality Metrics Reporting
Program,
27
CDER’s Emerging Technology Program, and CBER’s Advanced
581 Technologies Team (CATT) Program,
28
FDA has been able to engage industry and
582 gain valuable feedback on AI/ML use cases in pharmaceutical manufacturing.
583
584 The FDA also utilizes mechanisms such as a Broad Agency Announcement to solicit
extramural proposals that address emerging regulatory science priorities, including
586 leveraging external expertise and infrastructure to provide insight on the methods used
587 to integrate and evaluate AI/ML in drug development.
588
589 III. Considerations for the Use of AI/ML in Drug Development
591 As shown in Section II, AI/ML has been applied to a broad range of drug development
592 activities and continues to evolve. The use of AI/ML has the potential to accelerate the
593 drug development process and make clinical trials safer and more efficient. However, it
594 is important to assess whether the use of AI/ML introduces specific risks and harms.
For example, AI/ML algorithms have the potential to amplify errors and preexisting
596 biases present in underlying data sources and, when the findings are extrapolated
597 outside of the testing environment, raise concerns related to generalizability and ethical
598 considerations. Additionally, an AI/ML system may exhibit limited explainability due to
599 its underlying complexity or may not be fully transparent for proprietary reasons. These
concerns have resulted in a focus on developing standards for trustworthy AI that
601 address specific characteristics in areas such as explainability, reliability, privacy,
602 safety, security, and bias mitigation. This section begins with an overview of
603 considerations and good practices for the general application of AI/ML and ends with
604 questions to solicit feedback from stakeholders on these considerations and to further
identify potential good practices in the context of drug development. This will aid FDA in
606 further identifying opportunities and challenges with utilizing AI/ML throughout the drug
607 development process.
608
609 A. Overarching Standards and Practices for the Use of AI/ML
611 There has been an increased commitment by the Federal Government and the
612 international community to facilitate AI innovation and adoption, which includes
613 promoting trustworthy and ethical AI (Exec. Order No. 13859, Maintaining American
614 Leadership in Artificial Intelligence, February 11, 2019; Exec. Order No. 13960,
Promoting the Use of Trustworthy Artificial Intelligence in the Federal Government,
616 December 3, 2020; Lander & Nelson, October 22, 2021; Notice of Request for
27
See the Quality Metrics for Drug Manufacturing, October 2022.
https://www.fda.gov/drugs/pharmaceutical-quality-resources/quality-metrics-drug-manufacturing
28
See the CBER Advanced Technologies Team (CATT) Program, June 27, 2019.
https://www.fda.gov/vaccines-blood-biologics/industry-biologics/cber-advanced-technologies-team-catt
15
617 Information on Public and Private Sector Uses of Biometric Technologies, October 8,
618 2021; Organisation for Economic Co-operation and Development, 2019; Vought, 2020).
619 As a result, efforts for the development of cross-sector and sector-specific standards to
620 facilitate the technological advancement of AI have rapidly increased in both domestic
621 and international forums. For example, in August 2019, the National Institute for
622 Standards and Technology (NIST) released “U.S. Leadership in AI: A Plan for Federal
623 Engagement in Developing Technical Standards and Related Tools” to help ensure the
624 use of technical standards and to advance innovation, trust, and confidence in the use
625 of AI (National Institute of Standards and Technology, 2019). The plan identified
626 several areas of focus for AI standards development, including data and knowledge,
627 performance testing and reporting methodology, risk management, and trustworthiness,
628 among others. Other standards organizations, such as the International Organization
629 for Standardization (ISO), the Institute of Electrical and Electronics Engineers (IEEE),
630 and the International Electrotechnical Commission (IEC), are also developing relevant
631 AI/ML standards and work products addressing fundamental issues of data quality,
632 explainability, and performance, in addition to examining applications that are specific to
633 certain industries. The Verification and Validation (V&V 40) risk-informed credibility
634 assessment framework was initially developed by the American Society of Mechanical
635 Engineers (ASME) for the assessment of credibility of computational models used for
636 medical devices (American Society of Mechanical Engineers, 2018) and was later
637 adopted into model-informed drug development
29
(Kuemmel et al., 2020; Viceconti et
638 al., 2021). As AI/ML is also used for computational models, the V&V 40 framework
639 potentially serves to inform whether the AI/ML model is credible for use in drug
640 development.
30
The V&V 40 Standard, which is not specific to AI/ML and does not
641 specify activities or define criteria required to establish model credibility for a particular
642 context of use or application, has been adapted for medical devices and for model-
643 informed drug development.
31,32
644
645 In addition to the V&V 40 Standard for evaluating the predictive capability of
646 computational models for medical devices, FDA, Health Canada, and the United
647 Kingdom’s Medicines and Healthcare products Regulatory Agency (MHRA) jointly
648 published 10 guiding principles to inform the development of Good Machine Learning
649 Practices (GMLP) for medical devices that use AI/ML.
33
The guiding principles include
29
Promoting Innovation in Medical Product Assessment: A Risk-based Framework for Evaluating
Computational Models for Regulatory Decision-Making, October 2020. https://www.fda.gov/drugs/news-
events-human-drugs/promoting-innovation-medical-product-assessment-risk-based-framework-
evaluating-computational-models
30
A V&V 70 Subcommittee has been established for Verification and Validation of Machine Learning.
31
See the draft guidance for industry and FDA staff Assessing the Credibility of Computational Modelling
Simulation in Medical Device Submissions (December 2021). When final, this guidance will represent
FDA’s current thinking on this topic. https://www.fda.gov/media/154985/download
32
Promoting Innovation in Medical Product Assessment: A Risk-based Framework for Evaluating
Computational Models for Regulatory Decision-Making, October 2020. https://www.fda.gov/drugs/news-
events-human-drugs/promoting-innovation-medical-product-assessment-risk-based-framework-
evaluating-computational-models
33
Good Machine Learning Practice for Medical Device Development: Guiding Principles, October 2021.
https://www.fda.gov/medical-devices/software-medical-device-samd/good-machine-learning-practice-
medical-device-development-guiding-principles
16
650 adopting a total product life cycle approach in which multidisciplinary expertise is
651 leveraged throughout product development, with an in-depth understanding of how the
652 model is integrated into the clinical workflow. The principles also emphasize the
653 importance of adequate representation of age, gender, sex, race, and ethnicity within
654 the clinical study population to manage bias, improve generalizability, and provide
655 sufficient transparency with clear and essential information, such as the product’s
656 intended use and indications, the data used to test and train the model, and known
657 limitations. Finally, these GMLP highlight the importance of monitoring deployed
658 models for performance while managing the risk of model retraining. FDA’s CDRH had
659 previously discussed the role of GMLP for medical devices, and in 2019 issued a
660 proposed framework for modifications to AI/ML-based SaMD. The framework proposed
661 a predetermined change control plan mechanism—whereby a sponsor can proactively
662 specify intended modifications to device software incorporating AI/ML and the methods
663 that will be used to ensure their safety and effectiveness—thereby laying the foundation
664 for AI/ML-enabled devices with improved capacity for adaptation.
34
665
666 Although the standards and practices described in this section were not tailored
667 specifically for drug development, the utility and applicability of these standards to drug
668 development and the development of medical devices intended to be used with drugs,
669 will be explored to ensure alignment and consistency.
670
671 B. Discussion of Considerations and Practices for AI/ML in Drug Development
672
673 Informed by the diverse applications of AI/ML in drug development (see Section II),
674 FDA is considering approaches to provide regulatory clarity around the use of AI/ML in
675 drug development, supported by an expanding body of knowledge and a clear
676 appreciation of the opportunities and challenges with utilizing AI/ML in drug
677 development. While certain standards and practices outlined in Section III.A can
678 potentially be adapted to address the use of AI/ML in the context of drug development,
679 the use of AI/ML in drug development may raise specific challenges that could highlight
680 additional considerations. As noted above, this document is not FDA guidance or policy
681 and does not endorse any specific approaches for the use of AI/ML in drug
682 development. However, the feedback and future discussions with stakeholders can
683 help inform future regulatory activities.
684
685 Adapting the overarching principles of the General Accountability Office AI
686 accountability framework
35
below, FDA’s CDER, CBER, CDRH, including DHCoE, aim
687 to initiate a discussion with stakeholders and solicit feedback on three key areas in the
688 context of AI/ML in drug development:
689
34
Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-
Based Software as a Medical Device (SaMD) – Discussion Paper and Request for Feedback, April 2019.
https://www.fda.gov/files/medical%20devices/published/US-FDA-Artificial-Intelligence-and-Machine-
Learning-Discussion-Paper.pdf
35
See Artificial Intelligence: An Accountability Framework for Federal Agencies and Other Entities (June
2021). https://www.gao.gov/assets/gao-21-519sp.pdf
17
690 (1) human-led governance, accountability, and transparency;
691
692 (2) quality, reliability, and representativeness of data; and
693
694 (3) model development, performance, monitoring, and validation.
695
696 In each of these areas, a risk-based approach could include measures commensurate
697 with the level of risk posed by the specific context of use for AI/ML.
698
(1) Human-led governance, accountability, and transparency
Human-led AI/ML governance can help ensure adherence to legal and ethical values,
where accountability and transparency are essential for the development of
trustworthy AI. Such governance and clear accountability may extend across the
spectrum of planning, development, use, modification, and discontinuation (as
applicable) of AI/ML in the drug development process.
As part of governance, a risk management plan that considers the context of use may
be applied to identify and mitigate risks. This approach can help guide the level of
documentation, transparency, and explainability, with tracking and recording of key
steps and decisions, including the rationale for any deviations and procedures that
enable vigilant oversight and auditing. Transparency and documentation can provide
critical insight on the initial planning, development, function, and any modifications of
the AI/ML in the specific context of use, while explainability can provide
accompanying evidence or reason for the outputs.
Questions:
• In what specific use cases or applications of AI/ML in drug development are
there the greatest need for additional regulatory clarity?
• What does transparency mean in the use of AI/ML in drug development (for
example, transparency could be considered as the degree to which appropriate
information about the AI/ML model—including its use, development,
performance, and, when available, logic—is clearly communicated to
regulators and/or other stakeholders)?
36
• In your experience, what are the main barriers and facilitators of transparency
with AI/ML used during the drug development process (and in what context)?
• What are some of the good practices utilized by stakeholders for providing risk-
based, meaningful human involvement when AI/ML is being utilized in drug
development?
36
Adapted from ISO/IEC JTC1/SC42 DIS 25059 (draft).
https://www.iso.org/standard/80655.html?browse=tc
18
• What processes are in place to enhance and enable traceability and
auditability?
• How are pre-specification activities managed, and changes captured and
monitored, to ensure the safe and effective use of AI/ML in drug development?
(2) Quality, reliability, and representativeness of data
AI/ML is particularly sensitive to the attributes or characteristics of the data used for
training, testing, and validation. Although not unique to AI/ML, missing data, bias, and
data drift are typically important considerations. Ensuring data quality, reliability, and
that the data are fit for use (i.e., relevant for the specific intended use and population)
can be critical. Potential data-related issues to consider include:
Bias: AI/ML can potentially amplify preexisting biases that exist in the underlying
input data. NIST published a document characterizing three categories of bias
(human, systemic, and statistical/computational) and “how they may occur in the
commission, design, development, and deployment of AI technologies that can be
used to generate predictions, recommendations, or decisions (e.g., algorithmic
decision systems), and how AI systems may create societal harms.”
37
Integrity: The completeness, consistency, and accuracy of data.
38
Privacy and security: The protection and privacy of data, linked to data
classifications and the technical features of the system.
Provenance: Record trail that accounts for the origin of a piece of data (in a
database, document, or repository) together with an explanation of how and why it
got to the present place.
39
Provenance describes “the metadata, or extra
information about data, that can help answer questions such as who created the
data and when.”
40
Relevance: Adequate data are available and are appropriate for the intended use.
Replicability: Obtaining consistent results across studies aimed at answering the
same question, each of which has obtained its own data.
41
It is important to clarify
data access early in the process.
37
NIST Special Publication 1270, March 2022. https://doi.org/10.6028/NIST.SP.1270
38
For additional considerations related to data integrity see the guidance for industry Data Integrity and
Compliance with Drug CGMP (December 2018). https://www.fda.gov/media/119267/download
39
Encyclopedia of Database Systems, definition of data provenance.
https://link.springer.com/referenceworkentry/10.1007%2F978-0-387-39940-9_1305
40
21st Century Cures Act: Interoperability, Information Blocking, and the ONC Health IT Certification
Program (March 2019). https://www.federalregister.gov/documents/2019/03/04/2019-02224/21st-century-
cures-act-interoperability-information-blocking-and-the-onc-health-it-certification
41
Ibid.
19
Reproducibility: Obtaining consistent results using the same input data,
computational steps, methods and code, and conditions of analysis
42
(while not
confirming validity, the transparency required to demonstrate reproducibility
permits evaluation of the validity of design and operational decisions (S. V. Wang
et al., 2017)).
Representativeness: Confidence that a sample from which evidence is generated
is sufficiently similar to the intended population. In the context of patient
experience data, representativeness includes the extent to which the elicited
experiences, perspectives, needs, and priorities of the sample are sufficiently
similar to those of the intended patient population.
43
Questions:
• What additional data considerations exist for AI/ML in the drug development
process?
• What practices are developers, manufacturers, and other stakeholders
currently utilizing to help assure the integrity of AI/ML or to address issues,
such as bias, missing data, and other data quality considerations, for the use of
AI/ML in drug development?
• What are some of the key practices utilized by stakeholders to help ensure
data privacy and security?
• What are some of the key practices utilized by stakeholders to help address
issues of reproducibility and replicability?
• What processes are developers using for bias identification and management?
(3) Model development, performance, monitoring, and validation
The use of the model may be important to consider in evaluating AI/ML model
development and performance, including through practices of pre-specification steps
and clear documentation of criteria for developing and assessing models. It may also
be important to consider the model risk and credibility; model risk drives the selection
of credibility goals and activities.
44
Model risk is determined by two factors, which are
42
National Academies of Sciences, Engineering, and Medicine, 2019, Reproducibility and Replicability in
Science. https://doi.org/10.17226/25303
43
See discussion document for Patient-focused Drug Development Public Workshop Collecting
Comprehensive and Representative Input, December 2017.
https://www.fda.gov/media/109179/download
44
Credibility refers to trust in the predictive capability of a computational model for a particular context of
use (Kuemmel et al., 2020). This includes steps to document performance and approaches to measure
uncertainty at the component level (e.g., model and non-level components, including metrics and
20
shaped by the context of use: model influence (the weight of the model in the totality
of evidence for a specific decision) and decision consequence (the potential
consequences of a wrong decision).
In balancing performance and explainability, it may be important to consider the
complexity of the AI/ML model. In situations where complex models (e.g., artificial
neural network models) are determined to have similar performance, there may be
overall advantages to selecting the more traditional and parsimonious (i.e., fewer
parameters) model.
It may also be important to monitor and document monitoring efforts of the AI/ML
model to ensure it is reliable, relevant, and consistent over time. This includes
documentation of the results of monitoring and any corrective action taken to ensure
that the AI/ML produces intended results. Subsequent assessments (e.g., postmarket
safety monitoring, surveillance) can provide valuable feedback on processes and real-
world model performance. Real-world model performance includes applications that
may be supported by collection and monitoring of RWD (e.g., electronic health
records, product and disease registries). Potential re-training based on real-world
performance could provide important insights to model performance, and following
such re-training, it may be important to monitor and document the AI/ML model to
appropriately manage risks.
Data considerations also include providing the details of the training dataset utilized to
develop the AI/ML model, along with the performance, when employing independent,
external testing data to support verification and validation (“external validity”). It is
generally important for data of sufficient quality for the particular context of use to be
representative of the population where the AI/ML method will be utilized. It is
important to help ensure AI/ML models are validated to produce results that are
credible for the model’s use. Credibility activities include verification of the software
code and calculations, validation of the model, and evaluation of the applicability of
validation assessments to the context of use. These activities include considerations
of measuring the level of uncertainty of the model predictions. Upon completion of
credibility activities, an assessment can be made to determine whether the model is
sufficiently credible for its use and whether the model may be acceptable for a given
regulatory purpose.
Questions:
• What are some examples of current tools, processes, approaches, and best
practices being used by stakeholders for:
assessing performance and outcome of each component) and system level (e.g., methods for
assessment, performance metrics, and outcomes), where feasible. Demonstration of credibility often
includes a risk-based approach, where uses presenting the highest risk generally require the greatest
standard of evidence, with a gradient of evidence needed based on the associated risk (i.e., informing
early-stage drug development for non-serious medical condition versus evaluating drug safety and
effectiveness for critical medical condition).
21
- Documenting the development and performance of AI/ML models that can
be applied in the context of drug development (e.g., CONSORT-AI (Liu et
al., 2020) and SPIRIT-AI (Cruz Rivera et al., 2020))?
- Selecting model types and algorithms for a given context of use?
- Determining when to use specific approaches for validating models and
measuring performance in a given context of use (e.g., selecting relevant
success criteria and performance measures)?
- Evaluating transparency and explainability and increasing model
transparency?
- Addressing issues of accuracy and explainability (e.g., scenarios where
models may provide increased accuracy, while having limitations in
explainability)?
- Selecting open-source AI software for AI/ML model development? What
are considerations when using open-source AI software?
- The use of RWD performance in monitoring AI/ML?
• What practices and documentation are being used to inform and record data
source selection and inclusion or exclusion criteria?
• In what context of use are stakeholders addressing explainability, and how
have you balanced considerations of performance and explainability?
• What approaches are being used to document the assessment of uncertainty
in model predictions, and how is uncertainty being communicated? What
methods and standards should be developed to help support the assessment
of uncertainty?
699
700 As outlined above, many of the overarching principles and standards related to the
701 characteristics of trustworthy AI can help inform considerations or key practice areas for
702 the application of AI/ML in the context of drug development. In addition to meeting
703 current requirements to support regulatory decision-making regarding a drug’s safety
704 and effectiveness, the use of AI/ML in drug development raises challenges related to
705 human-led AI/ML governance, accountability, and transparency; data considerations;
706 and model development, performance, monitoring, and validation. Transparency and
707 documentation across the entire product life cycle can help build trust in the use of
708 AI/ML. In this regard, it may be important to consider pre-specification and
709 documentation of the purpose or question of interest, context of use, risk, and
710 development of AI/ML. While not unique to the use of AI/ML in drug development, there
711 are also a broad range of data quality, relevance, and reliability-related considerations.
22
712 Related to the area of model development, performance, monitoring, and validation, the
713 V&V 40 risk-informed credibility assessment framework may be a helpful guide when
714 considering the specific use for AI/ML. In general, use of a risk-based approach may
715 guide the level of evidence and record keeping needed for the verification and validation
716 of AI/ML models for a specific context of use. Engagement with the FDA early in the
717 process can also help inform and address these considerations.
718
719 IV. Next Steps: Engagement and Collaboration
720
721 The release of this initial discussion paper is part of a broader effort to communicate
722 with a range of stakeholders and to explore the relevant considerations for the use of
723 AI/ML in the development of human drugs and biological products. Coupled with this
724 document, FDA has included a series of questions for feedback, and a workshop with
725 stakeholders is planned to provide an opportunity for further engagement. The FDA will
726 also provide several other mechanisms to engage with stakeholders, sponsors, and
727 developers on this topic, and these can be utilized to address questions before
728 conducting a study that utilizes AI/ML. In addition to formal meetings where these
729 methods can be discussed, the Critical Path Innovation Meetings (CPIM),
45
ISTAND
730 Pilot Program,
46
Emerging Technology Program,
47
and Real-World Evidence Program
48
731 meetings are examples of additional avenues for communicating and discussing a
732 relevant AI/ML methodology or technology and improving efficiency and quality in drug
733 development. Additionally, communication and engagement with patients and the
734 public regarding considerations for AI/ML in drug development is critical to ensure
735 patient-centered approaches and policies.
736
737 Building on this discussion paper, FDA will continue to solicit feedback and engage a
738 broad group of stakeholders to further discuss considerations for utilizing AI/ML
739 throughout the drug development life cycle. These discussions and future
740 collaborations with stakeholders may provide a foundation for a future framework or
741 guidance.
45
See CPIM, November 11, 2022. https://www.fda.gov/drugs/new-drugs-fda-cders-new-molecular-
entities-and-new-therapeutic-biological-products/critical-path-innovation-meetings-cpim
46
See the ISTAND Pilot Program, February 10, 2021. https://www.fda.gov/drugs/drug-development-tool-
ddt-qualification-programs/innovative-science-and-technology-approaches-new-drugs-istand-pilot-
program
47
See Emerging Technology Program, February 22, 2022. https://www.fda.gov/about-fda/center-drug-
evaluation-and-research-cder/emerging-technology-program
48
See Framework for FDA’s Real World Evidence Program, April 14, 2020.
https:/fda.gov/media/120060/download
23
742 Glossary
743
744 Accuracy: The level of agreement between the measured value and the true value of
745 the clinical event or characteristic.
746
747 Artificial Intelligence (AI): A branch of computer science, statistics, and engineering
748 that uses algorithms or models to perform tasks and exhibit behaviors such as learning,
749 making decisions, and making predictions.
49
750
751 Biomarker: A defined characteristic that is measured as an indicator of normal
752 biological processes, pathogenic processes, or biological responses to an exposure or
753 intervention, including therapeutic interventions. Biomarkers may include molecular,
754 histologic, radiographic, or physiologic characteristics. A biomarker is not a measure of
755 how an individual feels, functions, or survives.
50
756
757 Clinical Outcome Assessment (COA): A measure that describes or reflects how a
758 patient feels, functions, or survives. There are four types of COAs: patient-reported
759 outcome, observer-reported outcome, clinician-reported outcome, and performance
760 outcome.
51
761
762 Context of Use: A statement that fully and clearly describes the way AI/ML is to be
763 used and the drug development-related purpose of the use.
52
764
765 Controlled Terminology: A finite set of values (e.g., codes, text, numeric) that
766 represent the only allowed values for a data item. Generally, controlled terminology
767 standards specify the key concepts that are represented as definitions, preferred terms,
768 synonyms, and code systems.
53
769
770 Decentralized Clinical Trial: A clinical investigation where some or all of the trial-
771 related activities occur at a location separate from the investigator’s location.
54
772
773 Digital Health Technology (DHT): A system that uses computing platforms,
774 connectivity, software, and/or sensors for health care and related uses. These
775 technologies span a wide range of uses, from applications in general wellness to
776 applications as a medical device. They include technologies intended for use as a
49
See IMDRF/AIMD WG/N67 Machine Learning-enabled Medical Devices: Key Terms and Definitions,
final document, May 6, 2022. https://www.imdrf.org/documents/machine-learning-enabled-medical-
devices-key-terms-and-definitions
50
See BEST (Biomarkers, EndpointS, and other Tools) Resource Glossary, 2016.
https://www.ncbi.nlm.nih.gov/books/NBK338448
51
See Clinical Outcome Assessment (COA), December 2020. https://www.fda.gov/about-fda/clinical-
outcome-assessment-coa-frequently-asked-questions
52
CDISC Glossary, 2022. https://evs.nci.nih.gov/ftp1/CDISC/Glossary/CDISC%20Glossary.html
53
Ibid.
54
See the draft guidance for industry, investigators, and other stakeholders Digital Health Technologies
for Remote Data Acquisition in Clinical Investigations (December 2021). When final, this guidance will
represent FDA’s current thinking on this topic. https://www.fda.gov/media/155022/download
24
777 medical product, in a medical product, or as an adjunct to other medical products
778 (devices, drugs, and biologics). They may also be used to develop or study medical
779 products. Data captured by DHTs can often be transmitted directly to investigators,
780 sponsors, and/or other authorized parties, with the capability to maintain blinding or
781 masking when appropriate. The ability to transmit data remotely increases opportunities
782 for patients to participate in clinical investigations at locations remote from the
783 investigator’s site.
55
.
784
785 Digital Twins: An integrated multi-physics, multiscale, probabilistic simulation of a
786 complex system that uses the best available data, sensors, and models to mirror the
787 behavior of its corresponding twin. A fully developed digital twin consists of a physical
788 component (e.g., unit operations), a virtual component, and automated data
789 communications between the two. The development and application of digital twins are
790 now being extended to manufacturing and complex products to assess sensitivities of
791 material attributes and process parameters, reliability of control strategies, and
792 effectiveness of mitigation plans for potential disturbances.
56
793
794 Drug Development Tool (DDT): A biomarker, COA, or any other method, material, or
795 measure determined to aid drug development and regulatory review. Animal models
796 developed to be used for product development under the Animal Rule
57
have been
797 determined by FDA to be DDTs under section 507 of the FD&C Act.
58
798
799 Endpoint: A precisely defined variable intended to reflect an outcome of interest that is
800 statistically analyzed to address a particular research question. A precise definition of
801 an endpoint typically specifies the type of assessments made, the timing of those
802 assessments, the assessment tools used, and possibly other details, as applicable,
803 such as how multiple assessments within an individual are to be combined.
59
804
805 Machine Learning (ML): A subset of AI that allows ML models to be developed by ML
806 training algorithms through analysis of data, without being explicitly programmed.
60
807
808 Natural Language Processing (NLP): The branch of computer science, specifically
809 the branch of AI, concerned with giving computers the ability to understand text and
810 spoken words in much the same way human beings can.
61
55
Ibid.
56
See Modeling & Simulation at FDA, November 16, 2022. https://www.fda.gov/science-research/about-
science-research-fda/modeling-simulation-fda
57
See Animal Rule Approvals, June 2022. https://www.fda.gov/drugs/nda-and-bla-approvals/animal-rule-
approvals
58
See the guidance for industry and FDA staff Qualification Process for Drug Development Tools
(November 2020). https://www.fda.gov/media/133511/download
59
See BEST (Biomarkers, EndpointS, and other Tools) Resource Glossary, 2016.
https://www.ncbi.nlm.nih.gov/books/NBK338448
60
See IMDRF/AIMD WG/N67 Machine Learning-enabled Medical Devices: Key Terms and Definitions,
final document, May 6, 2022. https://www.imdrf.org/documents/machine-learning-enabled-medical-
devices-key-terms-and-definitions
61
“What is natural language processing?” Accessed September 8, 2022.
https://www.ibm.com/cloud/learn/natural-language-processing#toc-what-is-na-jLju4DjE
25
811
812 Neural Network: A commonly used form of AI/ML that is used for categorization
813 applications and has been loosely likened to the way that neurons in the brain process
814 signals. Neural networks typically consist of at least three layers of neurons: input layer
815 (which receives information), hidden layer (responsible for extracting patterns and
816 conducting the internal processing), and output layer (produces and presents the final
817 network output).
62
818
819 Real-World Data (RWD): The data relating to patient health status and/or the delivery
820 of health care routinely collected from a variety of sources. Examples of RWD include
821 data derived from electronic health records (EHRs); medical claims and billing data;
822 data from product and disease registries; patient-generated data, including from in-
823 home-use settings; and data gathered from other sources that can inform on health
824 status, such as mobile devices.
63
825
826 Real-World Evidence (RWE): The clinical evidence about the usage and potential
827 benefits or risks of a medical product derived from analysis of RWD. RWD sources
828 (e.g., registries, collections of EHRs, administrative and medical claims databases) can
829 be used for data collection and, in certain cases, to develop analysis infrastructure to
830 support many types of study designs to develop RWE, including, but not limited to,
831 randomized trials (e.g., large simple trials, pragmatic clinical trials) and observational
832 studies (prospective or retrospective).
64
833
834 Recurrent Neural Network: A type of artificial neural network that uses sequential
835 data or time series data to exhibit temporal dynamic behavior. These algorithms are
836 commonly used for ordinal or temporal problems, such as language translation, NLP,
837 speech recognition, and image captioning.
65
62 See the Executive Summary for the Patient Engagement Advisory Committee Meeting: Artificial
Intelligence and Machine Learning in Medical Devices, October 22, 2020.
https://www.fda.gov/media/142998/download
63 See the draft guidance for industry, investigators, and other stakeholders Real-World Data: Assessing
Electronic Health Records and Medical Claims Data to Support Regulatory Decision-Making for Drug and
Biological Products (September 2021). https://www.fda.gov/media/152503/download
64 Ibid.
65 Adapted from https://www.ibm.com/cloud/learn/recurrent-neural-networks
26
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
References
Abatemarco, D., Perera, S., Bao, S. H., Desai, S., Assuncao, B., Tetarenko, N., . . . Mingle, E.
(2018). Training Augmented Intelligent Capabilities for Pharmacovigilance: Applying
Deep-learning Approaches to Individual Case Safety Report Processing. Pharmaceut
Med, 32(6), 391-401. doi:10.1007/s40290-018-0251-9
Aerts, H. J. W. L., Grossmann, P., Tan, Y., Oxnard, G. R., Rizvi, N., Schwartz, L. H., & Zhao, B.
(2016). Defining a Radiomic Response Phenotype: A Pilot Study using targeted therapy
in NSCLC. Sci Rep, 6, 33860. doi:10.1038/srep33860
American Society of Mechanical Engineers. (2018). Assessing Credibility of Computational
Modeling through Verification and Validation: Application to Medical Devices. In.
Arden, N. S., Fisher, A. C., Tyner, K., Yu, L. X., Lee, S. L., & Kopcha, M. (2021). Industry 4.0 for
pharmaceutical manufacturing: Preparing for the smart factories of the future. Int J
Pharm, 602, 120554. doi:10.1016/j.ijpharm.2021.120554
Athreya, A. P., Neavin, D., Carrillo-Roa, T., Skime, M., Biernacka, J., Frye, M. A., . . . Bobo, W.
V. (2019). Pharmacogenomics-Driven Prediction of Antidepressant Treatment
Outcomes: A Machine-Learning Approach With Multi-trial Replication. Clin Pharmacol
Ther, 106(4), 855-865. doi:10.1002/cpt.1482
Avram, R., Olgin, J. E., Kuhar, P., Hughes, J. W., Marcus, G. M., Pletcher, M. J., . . . Tison, G.
H. (2020). A digital biomarker of diabetes from smartphone-based vascular signals. Nat
Med, 26(10), 1576-1582. doi:10.1038/s41591-020-1010-5
Ball, R., & Dal Pan, G. (2022). "Artificial Intelligence" for Pharmacovigilance: Ready for Prime
Time? Drug Saf, 45(5), 429-438. doi:10.1007/s40264-022-01157-4
Bate, A., & Hobbiger, S. F. (2021). Artificial Intelligence, Real-World Automation and the Safety
of Medicines. Drug Saf, 44(2), 125-132. doi:10.1007/s40264-020-01001-7
Bayer, S., Clark, C., Dang, O., Aberdeen, J., Brajovic, S., Swank, K., . . . Ball, R. (2021). ADE
Eval: An Evaluation of Text Processing Systems for Adverse Event Extraction from Drug
Labels for Pharmacovigilance. Drug Saf, 44(1), 83-94. doi:10.1007/s40264-020-00996-3
Beninger, P. (2020). Signal Management in Pharmacovigilance: A Review of Activities and
Case Studies. Clin Ther, 42(6), 1110-1129. doi:10.1016/j.clinthera.2020.03.018
Bulitta, J. B., Hope, W. W., Eakin, A. E., Guina, T., Tam, V. H., Louie, A., . . . Hoover, J. L.
(2019). Generating Robust and Informative Nonclinical In Vitro and In Vivo Bacterial
Infection Model Efficacy Data To Support Translation to Humans. Antimicrob Agents
Chemother, 63(5), e02307–02318. doi:10.1128/AAC.02307-18
Chan, H. C. S., Shan, H., Dahoun, T., Vogel, H., & Yuan, S. (2019). Advancing Drug Discovery
via Artificial Intelligence. Trends Pharmacol Sci, 40(8), 592-604.
doi:10.1016/j.tips.2019.06.004
Cohoon, T. J., & Bhavnani, S. P. (2020). Toward precision health: applying artificial intelligence
analytics to digital health biometric datasets. Per Med, 17(4), 307-316. doi:10.2217/pme-
2019-0113
Comfort, S., Dorrell, D., Meireis, S., & Fine, J. (2018). MOdified NARanjo Causality Scale for
ICSRs (MONARCSi): A Decision Support Tool for Safety Scientists. Drug Saf, 41(11),
1073-1085. doi:10.1007/s40264-018-0690-y
Cruz Rivera, S., Liu, X., Chan, A. W., Denniston, A. K., Calvert, M. J., & and the SPIRIT-AI and
CONSORT-AI Working Group; SPIRIT-AI and CONSORT-AI Steering Group; SPIRIT-AI
and CONSORT-AI Consensus Group. (2020). Guidelines for clinical trial protocols for
interventions involving artificial intelligence: the SPIRIT-AI extension. Nat Med, 26(9),
1351-1363. doi:10.1038/s41591-020-1037-7
Daryaee, F., & Tonge, P. J. (2019). Pharmacokinetic-pharmacodynamic models that incorporate
drug-target binding kinetics. Curr Opin Chem Biol, 50, 120-127.
doi:10.1016/j.cbpa.2019.03.008
27
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
Dercle, L., Fronheiser, M., Lu, L., Du, S., Hayes, W., Leung, D. K., . . . Zhao, B. (2020).
Identification of non-small cell lung cancer sensitive to systemic cancer therapies using
radiomics. Clin Cancer Res, 26(9), 2151-2162. doi:10.1158/1078-0432.CCR-19-2942
Desai, R. J., Matheny, M. E., Johnson, K., Marsolo, K., Curtis, L. H., Nelson, J. C., . . .
Schneeweiss, S. (2021). Broadening the reach of the FDA Sentinel system: A roadmap
for integrating electronic health record data in a causal analysis framework. NPJ Digit
Med, 4(1), 170. doi:10.1038/s41746-021-00542-0
ElZarrad, M. K., Lee, A. Y., Purcell, R., & Steele, S. J. (2022). Advancing an agile regulatory
ecosystem to respond to the rapid development of innovative technologies. Clin Transl
Sci, 15(5), 1332–1339. doi:10.1111/cts.13267
European Medicines Agency. (2022). DRAFT Qualification opinion for Prognostic Covariate
Adjustment (PROCOVA™). European Medicines Agency Retrieved from
https://www.ema.europa.eu/documents/other/draft-qualification-opinion-prognostic-
covariate-adjustment-procovatm_en.pdf
Exec. Order No. 13859, Maintaining American Leadership in Artificial Intelligence. (2019-
02544). (February 11, 2019). 84 Fed. Reg. 3967 Retrieved from
https://www.federalregister.gov/documents/2019/02/14/2019-02544/maintaining-
american-leadership-in-artificial-intelligence
Exec. Order No. 13960, Promoting the Use of Trustworthy Artificial Intelligence in the Federal
Government. (2020-27065). (December 3, 2020). 85 Fed. Reg. 78939 Retrieved from
https://www.federalregister.gov/documents/2020/12/08/2020-27065/promoting-the-use-
of-trustworthy-artificial-intelligence-in-the-federal-government
Fumagalli, S. E., Padhiar, N. H., Meyer, D., Katneni, U., Bar, H., DiCuccio, M., . . . Kimchi-
Sarfaty, C. (2023). Analysis of 3.5 million SARS-CoV-2 sequences reveals unique
mutational trends with consistent nucleotide and codon frequencies. Virol J, 20(1), 31.
doi:10.1186/s12985-023-01982-8
Ghosh, R., Kempf, D., Pufko, A., Barrios Martinez, L. F., Davis, C. M., & Sethi, S. (2020).
Automation opportunities in pharmacovigilance: An industry survey. Pharmaceut Med,
34(1), 7-18. doi:10.1007/s40290-019-00320-0
Harrer, S., Shah, P., Antony, B., & Hu, J. (2019). Artificial Intelligence for Clinical Trial Design.
Trends Pharmacol Sci, 40(8), 577-591. doi:10.1016/j.tips.2019.05.005
Harrison, L. I., & Gibaldi, M. (1977). Physiologically based pharmacokinetic model for digoxin
disposition in dogs and its preliminary application to humans. J Pharm Sci, 66(12), 1679-
1683. doi:10.1002/jps.2600661206
Hsu, V., de, L. T. V., M., Zhao, P., Zhang, L., Zheng, J. H., Nordmark, A., . . . Huang, S. M.
(2014). Towards quantitation of the effects of renal impairment and probenecid inhibition
on kidney uptake and efflux transporters, using physiologically based pharmacokinetic
modelling and simulations. Clin Pharmacokinet, 53(3), 283-293. doi:10.1007/s40262-
013-0117-y
Huang, J., O'Connor, T., Ahmed, K., Chatterjee, S., Garvin, C., Ghosh, K., . . . Warman, M.
(2021). AIChE PD2M Advanced Process Control workshop-moving APC forward in the
pharmaceutical industry. Journal of Advanced Manufacturing and Processing, 3(1),
e10071. doi:10.1002/amp2.10071
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., . . . Hassabis, D.
(2021). Highly accurate protein structure prediction with AlphaFold. Nature, 596(7873),
583-589. doi:10.1038/s41586-021-03819-2
Kawakami, E., Tabata, J., Yanaihara, N., Ishikawa, T., Koseki, K., Iida, Y., . . . Okamoto, A.
(2019). Application of atificial intelligence for preoperative diagnostic and prognostic
prediction in epithelial ovarian cancer based on blood biomarkers. Clin Cancer Res,
25(10), 3006-3015. doi:10.1158/1078-0432.CCR-18-3378
28
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
Kreimeyer, K., Dang, O., Spiker, J., Gish, P., Weintraub, J., Wu, E., . . . Botsis, T. (2022).
Increased Confidence in Deduplication of Drug Safety Reports with Natural Language
Processing of Narratives at the US Food and Drug Administration. Frontiers in Drug
Safety and Regulation, 2. doi:10.3389/fdsfr.2022.918897
Kreimeyer, K., Dang, O., Spiker, J., Munoz, M. A., Rosner, G., Ball, R., & Botsis, T. (2021).
Feature engineering and machine learning for causality assessment in
pharmacovigilance: Lessons learned from application to the FDA Adverse Event
Reporting System. Comput Biol Med, 135, 104517.
doi:10.1016/j.compbiomed.2021.104517
Kuemmel, C., Yang, Y., Zhang, X., Florian, J., Zhu, H., Tegenge, M., . . . Zineh, I. (2020).
Consideration of a credibility assessment framework in model-informed drug
development: Potential application to physiologically-based pharmacokinetic modeling
and simulation. CPT Pharmacometrics Syst Pharmacol, 9(1), 21-28.
doi:10.1002/psp4.12479
Lander, E., & Nelson, A. (October 22, 2021). ICYMI: WIRED (Opinion): Americans Need a Bill of
Rights for an AI-Powered World. https://www.whitehouse.gov/ostp/news-
updates/2021/10/22/icymi-wired-opinion-americans-need-a-bill-of-rights-for-an-ai-
powered-world/
Laubenbacher, R., Sluka, J. P., & Glazier, J. A. (2021). Using digital twins in viral infection.
Science, 371(6534), 1105-1106. doi:10.1126/science.abf3370
Lewis, D. J., & McCallum, J. F. (2020). Utilizing advanced technologies to augment
pharmacovigilance systems: Challenges and opportunities. Ther Innov Regul Sci, 54(4),
888-899. doi:10.1007/s43441-019-00023-3
Liu, Q., Huang, R., Hsieh, J., Zhu, H., Tiwari, M., Liu, G., . . . Huang, S. M. (2022). Landscape
analysis of the application of artificial antelligence and machine learning in regulatory
submissions for drug development from 2016 to 2021. Clin Pharmacol Ther.
doi:10.1002/cpt.2668
Liu, X., Cruz Rivera, S., Moher, D., Calvert, M. J., Denniston, A. K., & the SPIRIT-AI and
CONSORT-AI Working Group. (2020). Reporting guidelines for clinical trial reports for
interventions involving artificial intelligence: the CONSORT-AI extension. Nat Med,
26(9), 1364-1374. doi:10.1038/s41591-020-1034-x
Liu, X., Liu, C., Huang, R., Zhu, H., Liu, Q., Mitra, S., & Wang, Y. (2021). Long short-term
memory recurrent neural network for pharmacokinetic-pharmacodynamic modeling. Int J
Clin Pharmacol Ther, 59(2), 138-146. doi:10.5414/CP203800
Liu, Z., Chen, X., Carter, W., Moruf, A., Komatsu, T. E., Pahwa, S., . . . W., T. (2022). AI-
powered drug repurposing for developing COVID-19 treatments. In Reference Module in
Biomedical Sciences: Elsevier.
Lu, J., Deng, K., Zhang, X., Liu, G., & Guan, Y. (2021). Neural-ODE for pharmacokinetics
modeling and its advantage to alternative machine learning models in predicting new
dosing regimens. iScience, 24(7), 102804. doi:10.1016/j.isci.2021.102804
Ly, T., Pamer, C., Dang, O., Brajovic, S., Haider, S., Botsis, T., . . . Ball, R. (2018). Evaluation of
Natural Language Processing (NLP) systems to annotate drug product labeling with
MedDRA terminology. J Biomed Inform, 83, 73-86. doi:10.1016/j.jbi.2018.05.019
Mager, D. E., Woo, S., & Jusko, W. J. (2009). Scaling pharmacodynamics from in vitro and
preclinical animal studies to humans. Drug Metab Pharmacokinet, 24(1), 16-24.
doi:10.2133/dmpk.24.16
Mason, M., Cho, Y., Rayo, J., Gong, Y., Harris, M., & Jiang, Y. (2022). Technologies for
medication adherence monitoring and technology assessment criteria: Narrative review.
JMIR Mhealth Uhealth, 10(3), e35157. doi:10.2196/35157
29
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
Mouchlis, V. D., Afantitis, A., Serra, A., Fratello, M., Papadiamantis, A. G., Aidinis, V., . . .
Melagraki, G. (2021). Advances in de Novo Drug Design: From Conventional to Machine
Learning Methods. Int J Mol Sci, 22(4). doi:10.3390/ijms22041676
National Academies of Sciences, E., and Medicine. (2021). Innovations in Pharmaceutical
Manufacturing on the Horizon: Technical Challenges, Regulatory Issues, and
Recommendations. Retrieved from https://doi.org/10.17226/26009.
National Institute of Standards and Technology. (2019). U.S. Leadership in AI: A Plan for
Federal Engagement in Developing Technical Standards and Related Tools. Retrieved
from
https://www.nist.gov/system/files/documents/2019/08/10/ai_standards_fedengagement_
plan_9aug2019.pdf
Negi, K., Pavuri, A., Patel, L., & Jain, C. (2019). A novel method for drug-adverse event
extraction using machine learning. Informatics in Medicine Unlocked, 17, Article 100190.
doi:10.1016/j.imu.2019.100190
Notice of Request for Information on Public and Private Sector Uses of Biometric Technologies.
(2021-21975). (October 8, 2021). 86 Fed. Reg. 56300 Retrieved from
https://www.federalregister.gov/documents/2021/10/08/2021-21975/notice-of-request-
for-information-rfi-on-public-and-private-sector-uses-of-biometric-technologies
Organisation for Economic Co-operation and Development. (2019). Recommendation of the
Council on Artificial Intelligence. Retrieved from
https://legalinstruments.oecd.org/en/instruments/OECD-LEGAL-0449
Pappalardo, F., Russo, G., Tshinanu, F. M., & Viceconti, M. (2019). In silico clinical trials:
concepts and early adoptions. Brief Bioinform, 20(5), 1699-1708.
doi:10.1093/bib/bby043
Pierce, C. E., Bouri, K., Pamer, C., Proestel, S., Rodriguez, H. W., Van Le, H., . . . Dasgupta, N.
(2017). Evaluation of Facebook and Twitter monitoring to detect safety signals for
medical products: An analysis of recent FDA safety alerts. Drug Saf, 40(4), 317-331.
doi:10.1007/s40264-016-0491-0
Routray, R., Tetarenko, N., Abu-Assal, C., Mockute, R., Assuncao, B., Chen, H., . . . Mingle, E.
(2020). Application of augmented intelligence for pharmacovigilance case seriousness
determination. Drug Saf, 43(1), 57-66. doi:10.1007/s40264-019-00869-4
Sager, J. E., Yu, J., Ragueneau-Majlessi, I., & Isoherranen, N. (2015). Physiologically based
pharmacokinetic (PBPK) modeling and simulation approaches: A systematic review of
published models, applications, and model verification. Drug Metab Dispos, 43(11),
1823-1837. doi:10.1124/dmd.115.065920
Schmider, J., Kumar, K., LaForest, C., Swankoski, B., Naim, K., & Caubel, P. M. (2019).
Innovation in pharmacovigilance: Use of artificial intelligence in adverse event case
processing. Clin Pharmacol Ther, 105(4), 954-961. doi:10.1002/cpt.1255
Schneider, P., Walters, W. P., Plowright, A. T., Sieroka, N., Listgarten, J., Goodnow, R. A., . . .
Schneider, G. (2020). Rethinking drug design in the artificial intelligence era. Nat Rev
Drug Discov, 19(5), 353-364. doi:10.1038/s41573-019-0050-3
Schuler, A., Walsh, D., Hall, D., Walsh, J., Fisher, C., & for the Critical Path for Alzheimer's, D.,
the Alzheimer's Disease Neuroimaging Initiative and the Alzheimer's Disease
Cooperative Study. (2021). Increasing the efficiency of randomized trial estimates via
linear adjustment for a prognostic score. Int J Biostat. doi:10.1515/ijb-2021-0072
Shroff, T., Aina, K., Maass, C., Cipriano, M., Lambrecht, J., Tacke, F., . . . Loskill, P. (2022).
Studying metabolism with multi-organ chips: new tools for disease modelling,
pharmacokinetics and pharmacodynamics. Open Biol, 12(3), 210333.
doi:10.1098/rsob.210333
30
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
Spiker, J., Kreimeyer, K., Dang, O., Boxwell, D., Chan, V., Cheng, C., . . . Botsis, T. (2020).
Information visualization platform for postmarket surveillance decision support. Drug Saf,
43(9), 905-915. doi:10.1007/s40264-020-00945-0
Stehlik, J., Schmalfuss, C., Bozkurt, B., Nativi-Nicolau, J., Wohlfahrt, P., Wegerich, S., . . .
Pham, M. (2020). Continuous Wearable Monitoring Analytics Predict Heart Failure
Hospitalization: The LINK-HF Multicenter Study. Circ Heart Fail, 13(3), e006513.
doi:10.1161/circheartfailure.119.006513
Vamathevan, J., Clark, D., Czodrowski, P., Dunham, I., Ferran, E., Lee, G., . . . Zhao, S. (2019).
Applications of machine learning in drug discovery and development. Nat Rev Drug
Discov, 18(6), 463-477. doi:10.1038/s41573-019-0024-5
Viceconti, M., Pappalardo, F., Rodriguez, B., Horner, M., Bischoff, J., & Musuamba Tshinanu, F.
(2021). In silico trials: Verification, validation and uncertainty quantification of predictive
models used in the regulatory evaluation of biomedical products. Methods, 185, 120-
127. doi:10.1016/j.ymeth.2020.01.011
Vought, R. T. (2020). Guidance for Regulation of Artifical Intelligence Applications. Retrieved
from https://www.whitehouse.gov/wp-content/uploads/2020/11/M-21-06.pdf
Wang, S. V., Schneeweiss, S., Berger, M. L., Brown, J., de Vries, F., Douglas, I., . . . the joint
ISPE-ISPOR Special Task Force on Real World Evidence in Health Care Decision
Making. (2017). Reporting to improve reproducibility and facilitate validity assessment for
healthcare database studies V1.0. Pharmacoepidemiol Drug Saf, 26(9), 1018-1032.
doi:10.1002/pds.4295
Wang, W., Haerian, K., Salmasian, H., Harpaz, R., Chase, H., & Friedman, C. (2011). A drug-
adverse event extraction algorithm to support pharmacovigilance knowledge mining from
PubMed citations. AMIA Annu Symp Proc, 2011, 1464-1470. Retrieved from
https://www.ncbi.nlm.nih.gov/pubmed/22195210
Wang, Y., Carter, B. Z., Li, Z., & Huang, X. (2022). Application of machine learning methods in
clinical trials for precision medicine. JAMIA Open, 5(1), ooab107.
doi:10.1093/jamiaopen/ooab107
Weissler, E. H., Naumann, T., Andersson, T., Ranganath, R., Elemento, O., Luo, Y., . . .
Ghassemi, M. (2021). The role of machine learning in clinical research: transforming the
future of evidence generation. Trials, 22(1), 537. doi:10.1186/s13063-021-05489-x
Zhang, X., Yan, C., Gao, C., Malin, B. A., & Chen, Y. (2020). Predicting Missing Values in
Medical Data via XGBoost Regression. J Healthc Inform Res, 4(4), 383-394.
doi:10.1007/s41666-020-00077-1
31
