Developing a quality indicator system for pediatric clinical trials management of research centers in China: a modified Delphi-Analytic Hierarchy Process study (2025)

Background

Pediatric clinical trials face many challenges that hinder the development of safe and effective pediatric drugs. Enhancing risk control and management quality of pediatric clinical trials in research centers is essential to address these challenges. While existing studies focus on risk management from the perspective of sponsors, research specifically addressing risk management strategies in research centers is limited. There is a lack of systematic exploration and expert consensus on risk-based management system of pediatric trials for institutions. This study seeks to fill this gap by developing a risk assessment index system designed to enhance the management quality of pediatric clinical trials in research centers. The system is built upon a theoretical framework that integrates insights from experts across various domains of pediatric clinical trials.

Methods

The Delphi-analytic hierarchy process method was used for expert consultation to develop a risk assessment index system in pediatric clinical trials management of research centers. The questionnaire was distributed to 20 experts with extensive experience in pediatric clinical trials. These experts were selected from various regions and work units across China. Experts with more than 5 years of working experience and bachelor degree or above were included for consultation. The mean value, proportion of experts with full marks, and variation coefficient of important score and feasibility score of the indicators were applied to include the indexes. The analytic hierarchy process was used to investigate the weight of each indicator.

Results

The consultation outcomes exhibited good response rate and credibility, the experts’ positive coefficients were 100% and 95%, respectively, and the authoritative coefficients were 0.885 and 0.892. The final quality management indicator system for research centers consisted of four primary, sixteen secondary, and thirty-two tertiary indicators. Protocol design and subject protection accounted for the highest weight of 0.3741 and 0.3148 respectively in the index system, and next came process management and data collection of 0.1879 and 0.1232 in quality management of pediatric clinical trials.

Conclusions

This study develops a systematic expert consensus on a risk-based quality system tailored to the management of pediatric clinical trials in domestic research centers. The framework identifies critical risk indicators in the management process, providing a potential reference for quality control and internal audits within pediatric clinical trial research centers in China. However, further research is required to assess its practical implementation and adaptability to diverse trial conditions.

Due to the unique vulnerability of children as research subjects, clinical research involving children face challenges of informed consent [1], subject enrollment, specimen collection and visits, as well as the high drop-out rate. Lack of safety and efficacy data in pediatric population leads to lag in pediatric drug development. Currently, inadequacy of exclusive medication and drug annotation, and off-label use for children are some of the problems commonly faced [2, 3]. However, the urgent clinical needs and a series of policies initiated by governments are promoting the development of pediatric clinical trials [3, 4]. The regulations governing pediatric research conduct include special requirements that call for a higher level of scrutiny and more stringent thresholds of subject protection [5, 6]. The notable policies issued by Chinese regulatory departments include the “Technical Guidelines for Drug Clinical Trials in Pediatric Population” ([2016] No.48), “Technical Guidelines for Extrapolation of Adult Medication Data to Pediatric Population” ([2017] No.79), “Technical Guidelines for Real-World Research to Support Drug Development and Evaluation in Children (pilot edition)” ([2020] No.22), and “Quantitative Methodological Guidelines for Extrapolation of Adult Medication Data to Pediatric Population (pilot edition)” ([2023] No.27), etc [6,7,8,9,10,11]. These pivotal documents lay a robust groundwork for research centers to augment the management quality of clinical trials in children.

In China, clinical trials must be carried out in institutions with the requisite qualifications. As the implementation site of pediatric clinical trials, it is crucial for research centers to optimize the pediatric research process and improve the quality of trial management. Minimal risk is the basic principle of national ethical norms and guidelines in pediatric clinical trials [5, 12,13,14]. To minimize risks, the National Academy of Sciences proposed 14 types of issues that need attention when conducting risk assessments for clinical trials involving children [15]. Chinese scholars also explored the evaluation path of minimal risk for biomedical research in children based on the national conditions, as well as the safety of new traditional Chinese medicine research [16,17,18]. For clinical trial institutions, effectively identifying and evaluating risk points are essential to safeguard pediatric subjects and enhance the overall quality of clinical research. However, research on the adoption of risk management strategies and methods in pediatric clinical trial institutions remains limited. According to the International Conference on Harmonisation of Good Clinical Practice guidelines, sponsors should implement a risk-based approach to quality management of clinical trials [19, 20]. As this requirement is sponsor-focused, most existing studies investigate risk management strategies from the perspective of sponsors or regulatory agencies, such as assessing the impact of risk-based data monitoring on trial outcomes [21], evaluating the importance of standard document preparation during trial process [22], and developing flexible tools to help sponsors track key risk indicators across multiple sites [23]. Studies specifically address risk management strategies in research centers are scarce. Notable examples include the 2014 work by Catrin Tudur Smith and colleagues, which introduced a risk assessment framework using a matrix to score potential risks for each trial [24]. But this structured form was not specifically designed for pediatric trials and lacked weighted analyses of risk indicators. Another study conducted in 2018 utilized surveys and principal component analysis to preliminarily develop a risk control model for clinical trials in children [25]. These contributions have yet to establish a systematic expert consensus on quality management indicators tailored to pediatric clinical trials institutions. This study aims to address this gap by leveraging a modified Delphi method to develop a risk management system for pediatric trial research centers, providing a foundation for optimizing risk-based management practices in these centers.

Delphi analysis reaches a consensus and establish standards basing on anonymous questionnaires [26]. The modified Delphi process with Analytic Hierarchy Process (AHP), which is used to determine the relative significance of the finalized quality indicators, can be applied to construct the quality assessment index system of clinical research in children, and further play the vital role of research centers in minimizing risks and improving quality of pediatric trials. Therefore, modified Delphi method was adopted in this study to build the quality index system of pediatric trials in Chinese research centers. We divided the quality indexes into four key parts: scheme design, subject protection, process management, and data collection, which can provide a basis for the high-quality management of pediatric research in trial institutions.

Ethics approval

Ethics Committee approval was obtained from the Institutional Ethics Committee of Capital Institute of Pediatrics for the commencement of the study. The reference number of the ethical approval is SHERLL2023043.

Constructing preliminary indicator pool

A systematic search was conducted to develop an initial indicator pool of pediatric clinical trials. We searched PubMed and China National Knowledge Infrastructure to gather potential indicators in quality management of pediatric clinical trials. The terms including [“pediatric clinical trials” OR “childhood trials” OR “pediatric research”] AND [“risk factor*” OR “risk assess*” OR “risk indicator*” OR “minim* risk”] AND [“quality indicator*” OR “quality management” OR “quality evaluat*”]. Boolean operators were used to combine or exclude keywords in the search. A total of 36 relevant articles were identified and included in the study (di Table A1). These articles, combined with recommendations on focus points of pediatric clinical trials from relevant management regulations and guidelines, served as foundation for the initial indicator selection. Additionally, we incorporated insights from practical experience in routine quality control of pediatric clinical trials. Then the research group, consisting of national clinical trial inspectors, ethics committee members, quality control specialists, research investigators, and clinical research coordinators (CRC), conducted multiple rounds of in-depth discussions to define the initial indicators and ensure their comprehensibility and applicability. The experts have extensive experience in pediatric clinical trials and none of them participated in the subsequent Delphi consultations. Afterwards, a small-scale pre-survey was carried out, during which no suggestions regarding the indicators were raised. This process resulted in the finalization of initial set of 73 indicators.

Inclusion criteria of experts

The experts were identified through the research team’s professional networks and collaborations. They were selected based on their active engagement and professional expertise in pediatric clinical trials. The inclusion criteria for consulting experts from different work units in China were as following: (a) being conversant with pediatric clinical trials——investigators, members of ethics committee, trial managers, governmental supervisors, CRCs, and experts in other related fields; (b) having more than 5 years of working experience; (c) having bachelor degree or above; (d) being willing to participate in this research, and completing the questionnaire promptly as required. In the first round of consultation, the recovered questionnaires were 20, and the active coefficient of experts was 100%. In the second round of the Delphi survey, 20 questionnaires were distributed, with 19 successfully retrieved. The respondents also participated in the subsequent AHP consultation, resulting in an expert engagement rate of 95%. Experts offered positive feedback to this survey.

Delphi process

The questionnaire consists of informed consent form on the cover page and the main text. The informed consent form included the research background, method, risks, rights and interests of experts, information privacy, and contact information of the researcher. The main text contained five parts: (a) experts’ assessment of importance and feasibility of indicators, scoring from 1 to 5; (b) blank columns for experts proposing addition, deletion and amendments of these indicators; (c) experts’ basic information, including the name, gender, age, professional title, highest academic credential, specialized field, work unit, working years and principle role in clinical trials; (d) experts’ familiarity with the survey content, ranging from 0.2 (not familiar) to 1 (very familiar); (e) the basis of expert judgement, evaluating the impact of theoretical analysis, practical experience, knowledge of literature and expert intuition on their scoring.

In the first round of Delphi consultation, the significance and feasibility of primary indicators were analyzed, and inappropriate indexes were added, deleted or modified according to the experts’ suggestions. Then the revised questionnaires were sent out for second round of survey. After final indicators confirmation, the experts were invited to complete the comparison matrix. For the primary and secondary indexes, judgement matrices for pairwise comparisons were constructed using Satty’s 1–9 scale to assess the importance degree of each indicator relative to others at the same level [27, 28]. The online application software SPSSPRO 1.0.11 (www.spsspro.com) was used to calculate the weight of each index and conduct consistency testing. Under the same secondary indicator, the weight of each tertiary indicator was obtained by multiplying its importance ratio of the sum of all tertiary indexes by the weight of the secondary indicator [26]. The study procedure is shown in Fig.1.

Inclusion criteria of indicators

The mean value, percentage of experts with full marks, and coefficient of variation (CV) of the important score and feasibility score were applied to include the indexes. The inclusion criteria were as follows. The importance score and feasibility score both accords with the following criteria——the mean value > 3.5, the proportion of experts with full marks > 20%, and the CV value < 0.25. The indexes which did not meet the inclusion criteria were further evaluated by the experts not involved in consultation. These indicators were deleted or adjusted after discussion.

Data analysis

The Microsoft Excel 2013 and IBM SPSS Statistics (version 21) was used for the analysis. Descriptive statistics are typically represented using proportions, means, and standard deviations. The authority coefficient (Cr) represents the authority level of experts. Cr is the arithmetic mean of judgement basis (Ca) and the level of experts’ familiarity with the indicators (Cs) [29]. The Ca value is calculated based on the criteria outlined in Table1, and Cs value of expert ranges from 1.0 (very familiar) to 0.2 (unfamiliar). A Cr value of 0.7 or higher is indicative of a higher reliability in the inquiry’s outcomes [26, 29]. The Kendall’s coefficient of concordance (Kendall’s W) and chi-square test are used for consensus level among experts. The judgment matrix is deemed to possess satisfactory consistency when its Consistency Ratio (CR) is less than or equal to 0.1 [28].

Characteristics of experts

The experts in the study are between the ages of 30 and 52, and the average age of the consulted experts was 40 years, and the vast majority of them held a Master’s degree or higher, along with mid to senior professional titles. The details of these experts are summarized in Table2. The expert work units and geographic locations are listed as Appendix Table A2 in Additional file 2.

Authority coefficients of experts

The authority coefficients of experts were calculated based on their self-assessment scores. The average authority coefficients in the first and second rounds were 0.885 and 0.892, respectively, indicating the credibility of the consultation outcomes.

Degree of consensus among experts

Generally, the Kendall’s coefficient ranges from 0 to 1, where a higher value indicates a greater degree of coordination among experts [28]. The results of Kendall’s test in the two rounds of consultation are shown in Table3. All p-values were < 0.001, suggesting that the degree of consensus among experts was acceptable.

Index screening

The indicator screening process is summarized in the Table4. Finally, the quality management indicators of pediatric clinical trials for research centers included 4 primary indicators, 16 secondary indicators and 32 tertiary indicators.

Weight of indicators

The weight of each indicator was calculated using AHP method and percentage weight method (Table5). The CR value of each judgment matrix was lower than 0.1, indicating that all judgment matrices exhibited good consistency. For primary indicators, “Protocol design”, “Subject protection”, “Process management”, and “Data collection”, the weight values were 0.3741, 0.3148, 0.1879, and 0.1232, respectively. The weights of secondary indicators ranged from 0.0215 to 0.1355. For tertiary indicators, the weights ranged from 0.01075 to 0.1188.

In China, pediatric clinical trials must be conducted at qualified clinical trial institutions. As the venues for clinical research, Effective identification and management of risk points in research centers are essential for protecting pediatric subjects and improving clinical research quality. However, limited attention has been given to risk management strategies within pediatric clinical trial institutions, with most studies focusing on sponsor-driven approaches. Existing research on risk management in research centers is scarce and has not developed a systematic expert consensus on quality management indicators specifically designed for pediatric clinical trial institutions. Our study addresses this gap by using a modified Delphi method to construct a risk management system for pediatric research centers.

This system contains four key areas, protocol design, subject protection, process management, and data collection. In primary indicators, “Protocol design” holds the highest weight, which fully demonstrates the vital role of scientific and ethical study design in the entire phase of drug clinical development. Multiple guidelines from the International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use are related to scheme design, notably ICH E6(R3), which put forward incorporation quality into the design of the clinical trial, namely quality by design [20]. It emphasizes that quality of trials should be built on good design and execution, rather than overly relying on the inspection, supervision, audit or verification of retrospective documents. In China, the Center for Drug Evaluation of the National Medical Products Administration is responsible for overseeing drug clinical trials and handling drug marketing authorization applications. It not only encourages sponsors to communicate with it prior to finalizing their study designs, but also provides technical guidelines on the pediatric medication section of its official website. These documents focus on the differences in etiology, clinical course, and treatment responsiveness between pediatric and adult patients, offering technical specifications for the project design of specific diseases [30,31,32].

In this process, the secondary indicators “Scientific and standardized protocol design process” and “Comprehensive rationale for clinical trial” hold substantial weight. Key areas of focus include the participation of pediatricians, the documentation of investigator meeting deliberations for protocol modification, the inclusion of risk management plans, and analyses based on the efficacy and safety data in adults. For participants aged 16 to 18 years old, their transition from minor to adult status during long-term clinical trials requires close monitoring. It should be uniformly managed with a particular emphasis on ethical risks. Besides, in the discussion process of trial protocols, the inclusion of nursing staff is controversial. While the insights of nursing personnel are valuable, integrating their participation into current protocol discussion presents practical challenges. In addition, although the experts recognize the significance of the following tertiary indicators including “Presence of juvenile animal study data in the protocol and investigator’s manual”, “analysis of racial variations in pediatric populations domestically and internationally” and “Establishment of a long-term follow-up observation plan for the impact of the investigational drug on growth, development and organ function of subjects”. However, the practical implementation of these indicators in trials also faces feasibility challenges. These indexes were therefore eliminated.

Following closely in importance as a primary indicator is “Subject protection”. Given the distinct characteristics of pediatric subjects and adult participants, the informed consent process for children requires the involvement of their guardians. Under this category, the secondary indicators of “Rational and clear informed consent process” and “Minimizing invasive procedures” account for significant weight. For children of various age groups, the modes of informed consent differ. Beyond obtaining signed informed consent form from the subject’s guardians, the investigators also need to acquire the verbal consent or signed informed consent form from pediatric subjects over the age of 8. The process should be accurately reflected in the medical records, which is helpful for the trial verification and data provenance. Furthermore, employing child-friendly tools like illustrations, smiley faces, and sad faces can enhance the pediatric participants’ comprehension of the research purposes. To observe the effects of clinical trial drugs on children’s growth, development and organ function, sponsors often need to design companion or extension studies [33, 34]. In the operational process of clinical research, many projects integrate these companion or extension studies with the primary research and require only one informed consent form. According to the consulted experts, this practice is inappropriate as it merges research that can be distinctly separated, posing a potential risk of inadequate description of clinical trial information to pediatric participants and their guardians.

In the process of collecting biological samples from participants, using indwelling needles instead of multiple venipunctures can indeed reduce the risk of invasive procedures in pediatric subjects. However, this practice currently lacks feasibility in clinical diagnosis and treatment. Therefore, the indicator of using indwelling needles was removed based on expert opinions. Regarding the frequency and total volume of biological sample collection, traditional pharmacokinetics requiring blood collection at multiple time points is challenging in the pediatric patient population and often not well accepted by children and their guardians. But population pharmacokinetics [35,36,37] utilizes sparse data for analysis and significantly decreases the frequency and total volume of blood sampling for individual pediatric subjects, which holds substantial importance for clinical studies in pediatric populations.

Concerning the indicator of participant compensation, there exists a debate among experts. This indicator was supposed to be eliminated by standard criteria. But the newly released “Ethical Review Methods for Life Science and Medical Research Involving Humans” and the latest version of “Good Clinical Practice for Drug Trials” both highlight the significance and necessity of rational subject compensation [38, 39]. This indicator was therefore modified. At present, the practical execution of subject compensation still lacks specific and operational guidelines, and the forms vary across different countries and regions [40, 41]. In view of the inherent characteristics of pediatric clinical trials, the visits in the scheme for subjects necessitate the companion of either one or both guardians, which leads to the income loss, time consumption and travel expenses. Additionally, the collection of biological samples from pediatric subjects is more difficult. All these reasons contribute to the poor compliance and high dropout rate in pediatric trials. After deliberation with the research group, the indicator “Providing compensation for guardians’ lost wages and transportation costs” was not removed but modified to “Providing appropriate compensation based on the risk exposure of subjects in the trial”. This revision can not only help to improve the subject adherence and decrease the shedding rate, but also diminish the potential inducement issues that may arise from compensating for lost wages and travel expenses.

Compared with study design and subject protection, the risks of process management and data collection in clinical trials occupy a relatively small proportion. Because normally the qualified and experienced managers of research centers have established stable and standardized management process. Regarding the frontline personnel of clinical research, it is essential to select investigators with rich clinical experience and risk coping capability and hire CRCs with expertise in pediatric clinical trials. Initially, our research group opined that on-site CRCs trained and assessed by trial institutions should be preferentially employed, but the experts had different views. Although some research centers have completed the optimal selection of site management organization companies, as well as the assessment and management of on-site CRCs, it is still lack of feasibility to maintain CRCs stationed in many institutions due to regional and level disparities. Therefore, this criterion was ultimately removed.

In terms of pediatric subject rights, the sponsors are required to purchase clinical trial insurance based on the risk level of projects. The standardized procedures as well as the records for handling biological samples collected from subjects should be well established and submitted to the research centers. To enhance adherence of pediatric participants, the drug formulations in trials are encouraged to improve and become more suitable for children. The remote visits basing on internet medical service should also be included to reduce the difficulty and cost of subjects participating in the trial. In addition, for centralized management of trial materials, there was a divergence of opinions among experts. We revised the indicator from “Centralized management of trial materials by the institutional office” into “Centralized management of investigational drugs by the institutional pharmacy” with the adoption of experts’ suggestions. Storing items like test kits and injection needles in individual departments are more convenient, but the trial drugs are different. The centralized management of trial drugs by experienced personnel in the central pharmacy can significantly decrease potential risks such as medication errors and losses. In the data collection phase, we primarily concentrate on the completion of diaries, scales, and professional forms by participants and their guardians. Providing guardians with timely training conducted by investigators, accompanied by appropriate record-keeping, can improve the credibility and precision of self-reported information from participants or guardians. The documentation, handling, follow-up, and establishment of clear and definitive criteria of adverse events are also essential actions to safeguard participant rights and interests.

The two indicators of “promoting the construction of pediatric clinical research institutions to strengthen the rules and systems related to standardized research responsibilities” and “the risk responsibility bearing capacity of institutions in multi-center research” are critical for the quality management of pediatric clinical trial institutions. However, they are not included in the indicator set constructed in this study. Under current circumstances, implementing these indicators presents significant challenges. The effective development of standardized systems and responsibilities in research institutions requires substantial structural and regulatory changes, which may not be easily achievable in the short term. Additionally, enhancing institutional capacity to bear risks in multi-center research would necessitate extensive coordination and collaboration among various stakeholders, including regulatory agencies, medical consortiums, research networks, and other relevant organizations. The collective efforts of these entities are crucial to ensuring that resources, guidelines, and risk management strategies are effectively integrated into clinical research practices. While the immediate application of these indicators may be limited, they remain highly relevant to the long-term development and quality improvement of pediatric clinical trials. These aspects will be addressed in our ongoing and future research.

Our research offers a preliminary framework that could serve as a potential reference for regulatory authorities during project verification processes. Policymakers might consider leveraging the proposed risk indicator system as a basis for developing guidelines and regulatory policies to better safeguard pediatric subjects. Additionally, the management system may have the potential to assist sponsors in conducting pre-risk assessments of trial designs and support institutional self-evaluation, government regulation, cross-sectional and longitudinal comparisons of pediatric clinical trial quality across hospitals, as well as sponsor-driven project quality control. However, this study represents only an initial step in establishing a systematic expert consensus on risk-based management system tailored to pediatric clinical trial research centers, and several limitations should be acknowledged. First, the number of consulted experts was limited. Their geographic regions and areas of expertise were diverse, differing opinions on certain indicators emerged. Furthermore, the inclusion of numerous indicators in the evaluation system contributed to the concordance coefficient not achieving the desired benchmark. Second, the consulted experts were exclusively from China, the risk management system would require further consultation and revision of indicators before it can be extrapolated to other countries. Lastly, Since the system has not undergone pilot testing, further validation is necessary to determine its practical utility in improving management capabilities and reducing risks in pediatric clinical trials. The feasibility and adaptability of this framework across diverse trial characteristics and conditions require further investigation.

Authors and Affiliations

1. Drug Clinical Trial Institution, Children’s Hospital, Capital Institute of Pediatrics, No.2 Yabao Road, Beijing, 100020, China

   Chang Liu,Yi Liu,Bo Xuan,Yuenan Qi,Ling Ou&Jianmin Zhang

Contributions

L.C. conceptualized and designed the study, designed the consulted indexes, analyzed and interpreted the data, wrote the manuscript and performed funding acquisition. L.Y. reviewed and revised the consulted indexes, carried out the investigation and performed funding acquisition. X.B., Q.Y. and O.L. help to review and revise the indicators. Z.J. conceptualized the study, revised the indexes and performed funding acquisition. All authors read and approved the final manuscript.

Corresponding authors

Correspondence to Chang Liu or Jianmin Zhang.

Ethics approval and consent to participate

Ethics Committee approval was obtained from the Institutional Ethics Committee of Capital Institute of Pediatrics for the implement of the study. All experts signed the informed consent before participating in the investigation.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

Reprints and permissions