Research on the Pragmatic Clinical Trial Design Based on Real-World Study

Li Gan,Zhang Yu,Yuan Xiaoliang,,Chen Yuwen,＊

(1.School of Business Administration,Shenyang Pharmaceutical University,Shenyang 110016,China;2.Research Institute of Drug Regulatory Science,Shenyang Pharmaceutical University,Shenyang 110016,China)

Abstract Objective To introduce the concept and plan design of pragmatic clinical trials based on real-world study (RWS)so as to provide both guidance for clinical trials and reference for relevant personnel studying RWS.Methods A retrospective study was made on the design of real-world clinical trials at home and abroad in recent years to summarize their design process and applications.Results and Conclusion The advantages of combining randomization and real-world data in clinical trials are important methods for evaluating medical interventions in the real clinical environment.Practical clinical trials can evaluate the therapeutic effects of interventions in real or close to real medical environments to obtain the best real-world evidence.

Keywords:RWS;pragmatic clinical trial;design

Real-world study (RWS) refers to collecting a variety of data in actual clinical,community or family to evaluate the impact of a certain diagnosis and treatment on the health of patients (subjects)[1].The basic design of RWS usually includes intervention and observation.The general way to carry out interventional trials in the real world is pragmatic clinical trials (PCTs).This article intends to promote a better understanding and design of PCTs in RWS.

1 Definition,application and advantages of PCTs

According to the different research purposes,PCTs can be divided into trials to select a treatment plan to verify the effects or biological mechanisms of various interventions[2].PCT refers to the that compares the treatment results (including actual effects,reference effects,safety and effectiveness,etc.) of various interventions in actual diagnosis and treatment using random,reference and other designs in actual or nearly actual clinical environment[3].PCT is a method for comparing the therapeutic effects of various treatment programs under actual or close to actual conditions.Under the condition close to the actual treatment,it mainly answers the question:which treatment plan is better for the patient (subject).It is used to evaluate the practicality of the treatment plan[4].PCT is often used to get the effect of intervention measures taken in actual or near real environments,and it is the best evidence for evaluating the effect of drugs and medical devices after they are marketed.

Randomized controlled trials (RCTs) are often regarded as the gold standard for evaluating the safety and effectiveness of drugs,which are widely used in clinical trials of drugs. Schwartz et al.first stated that RCT could prove whether a diagnosis effect exist,and how the interventions played their role.However,they didn’t tell us whether the research conclusions could be extended to a real situation and other people[5].On the other hand,PCT compares different medical programs under real conditions of diagnosis and treatment with the purpose of providing the best medical decision.Compared with RCT,PCT has many advantages in actual clinical practice,which are not limited to the following aspects.Applicability:PCT is often used to evaluate the actual effect,safety and effectiveness of drugs or medical devices after they are marketed,which is the best evidence for evaluating the effect of drugs and medical devices.Utility research:PCT is a study of the clinical effect (effectiveness) of a drug or a certain therapy.The effectiveness is the true effect of the therapy,which is the research result observed under actual conditions.Individualized diagnosis and treatment:PCT emphasizes practical diagnosis and treatment,and puts more emphasis on routine treatment plan.The treatment plan can include the patients’ syndrome differentiation treatment,which reflects the individualized diagnosis and treatment for patients (subjects).External authenticity:The external authenticity means that the conclusions of one diagnosis and treatment can be applied to other people,that is,the general applicability of the study[4].PCT requires the test conditions should be consistent with the daily clinical medical conditions as much as possible,emphasizing the actual effects of the treatment methods observed in the conventional medical environment.PCT often has high external authenticity in clinical practice.

2 PCT design in a real world research

In the RWS,PCT design is very important.This article discusses PCT design through the following aspects.

2.1 Research purpose

Normally,the purpose of PCT is to evaluate the effect of the intervention conditions imposed in the trial in the real or near real environment,and to provide the best evidence for the implementation and decision-making of actual diagnosis and treatment or medical policies.Eventually,the intervention measures imposed in the trial will be applied to the real-world clinical environment[6]. Therefore,in the stage of research planning and design,the research purpose should be determined,and the suitable design type should be selected according to the research purpose.

2.2 Type and selection of research design

The designs of PCT include following types,but are not limited to them.Individual PCT (iPCT):It refers to the clinical trial conducted with individuals as the test and observation unit.The test in PCT generally refers to iPCT.Cluster RCT (cRCT):It is a controlled trial with cluster as the trial and intervention unit[7].It should be noted that when cRCT conducts data analysis,it is necessary to consider the group effect as well as evaluate the effect as a unit.Stepped wedge RCT (swRCT):It is a special group trial that is often used to evaluate medical and health services and policy interventions[8].However,there are many factors that need to be considered in the selection of individual and group trials in PCT,such as purpose,problems and conditions of research.

2.3 Research site and environment

Research site and the environment means that clinical trial institutions should have the corresponding qualifications,the technical capabilities required by PCT,and accept the supervision and inspection of the sponsor organization and drug regulatory authority.The research site in PCT design usually refers to conventional medical institutions or general hospitals with common therapies.Besides,the selected research site should be fully used by researchers.We know that some rare diseases and special cases can only be tested in specific medical places.In such case,the corresponding research site and environment should be selected according to the actual situation.

2.4 Research elements

2.4.1 Patients (subjects)

From the practical perspective,in order to make the results of PCT test more popularized,patients(subjects) should be selected in a wider range and fit the actual diagnosis and treatment environment.Therefore,it is necessary to take into account the different conditions of patients’ disease or complex complications,different age groups,various physiological and pathological conditions,TCM syndromes,or differences in compliance degrees.

2.4.2 Recruitment,selection and enrollment of patients(subjects)

In iPCT,the selection conditions of patients(subjects) are generally wide since the purpose is to increase the population of the results.Therefore,the selection process is not significantly different from traditional explanatory RCT (eRCT).However,in cRCT and swRCT,patients (subjects) are recruited into groups,but they should all meet the predetermined selection conditions of groups and individuals.When selecting individuals,patients in cRCT and swRCT should meet the selection criteria of their group and individual at the same time.They are usually recruited and selected based on their research sites.Patients (subjects) should meet the selection criteria and consent before entering the group.They can voluntarily withdraw from the study during the test time.When the research is in progress,researcher needs to specify the methods and procedures of recruiting patients (subject),and try to avoid selection bias due to recruitment methods and program design[8].

2.4.3 Intervention measures and patients (subjects)

The interventions in the trial include drugs,medical devices,and medical policies.Patients(subjects) are the targets of interventions in PCT and they are usually divided into individuals and groups according to the actual environment.The general intervention is carried out by individuals.Only when different intervention programs interfere with the control program,the group is used as a unit to conduct the trial.

2.4.4 Control measures

Usually,placebos and other similar comfort measures are not used as the control of the trial,which can ensure that the trial is in the real clinical environment,and the untreated control can be used under conditions permitted by ethical principles.The selection of control in PCT generally uses currently recognized conventional treatment.

2.4.5 Outcome indicators

The outcome indicators in PCT need to be selected according to the purpose of the research,generally including safety evaluation,effectiveness evaluation,patient satisfaction,and quality of life.PCT generally emphasizes the use of patient-oriented diagnosis and treatment indicators,which include the outcome of overall life benefits closely related to patient (subject).Among them,the safety evaluation includes but is not limited to the description of safety parameters,the method and time selection for evaluating,collecting and analyzing safety parameters,procedures for collecting and reporting adverse events and concurrent diseases,procedures for issuing reports,and the form and time of followup of patients (subjects) after adverse events.Besides,the effectiveness evaluation includes but is not limited to the parameters of the relevant effectiveness description,methods and time selection for evaluating,collecting and analyzing effectiveness parameters,etc.

2.4.6 Sample size estimation

Sample size estimation includes the following two items.The first is iPCT sample size estimation.Its basic principle is the same as RCT.It combines with the design type,comparison type,selection of controls,and data attributes of the main outcome indicators to put forward hypotheses for the statistical test of effect.Then,inspection parameters are set for calculation,and appropriate adjustments are made according to the reality of the situation.The second is the sample size estimation of cRCT and swRCT.It is usually based on iPCT to calculate the iPCT sample size.Then,appropriate adjustments are made according to the design of the PCT study.Basically,it can be divided into two situations:the specified group size and the specified number of groups for estimation.Sample size estimation generally includes intervention measures,control effects and statistical methods,as well as coefficients related to the group,group conditions,and the number of outcome measurements.After estimating the overall sample size through a certain method,researchers should adjust the number of groups/group size based on the difficulty and cost of the trial while recruiting patients.The number of groups should be appropriate,generally about 50 cases.If adaptive design is used in the PCT,then the sample size estimation should also conform to its design principle and specification[8].Because PCT adopts a wider selection of conditions and practical diagnosis and treatment plans,the trial may produce related differences based on individual conditions.Therefore,the sample size should be appropriately expanded after the evaluation before the implementation of the research[9].

2.4.7 Data management and quality control

(1) Data source and collection.PCT data sources not only include traditional experimental data,data for specific research purposes,but also the data from conventional electronic health databases such as hospital information system (HIS),health records,and census information.Although these data sources are similar to observational RWS,they are different in actual use.The specific manifestations are as follows.First is the patients (subjects) selection and grouping.The inclusion criteria of PCT are relatively wide,and the exclusion criteria are narrow.Generally,HIS can be used to initially select patients (subjects) and obtain some basic information.Second is random grouping of patients (subjects).After confirming that they meet the selection criteria,patients (subjects) are assigned to the corresponding treatment groups and control groups through a separate random allocation system.Researchers will intervene on patients (subjects)according to the assigned corresponding diagnosis and treatment plan.The intervention information of patients (subjects) will be stored in the electronic medical record data through routine records (such as hospital admission records)[10].Third is research follow-up.According to the different situations of the research follow-up,researchers can obtain the diagnosis and treatment data and outcome index data of patients (subjects) from clinics,relevant medical insurance and other institutions.For example,the patient’s disease information can be obtained,such as whether he is admitted to the hospital again from HIS.When collecting data through different electronic systems,the commonality of data from different electronic systems is one of the factors that affect data collection.When certain conditions are met,health level seven international (HL7)[11]or clinical data interchange standard consortium (CDISC)[12]data standards can be used for collection and conversion.Then,general coding dictionary or related coding system will be applied.The unified data will greatly improve efficiency.Among them,HL7 is to develop hospital data information transmission protocols and standards,standardizing the format of clinical medicine and management information,reducing the cost of interconnection of hospital information systems,and improving the degree of data information sharing between hospital information systems.CDISC standard is a set of standards on how to collect data,what type of data should be collected,and how to submit the data to the agency responsible for approving new drugs.Because the purpose of PCT research is to evaluate the effect of interventions in a real environment,it is characterized by diversified interventions and outcomes of patients (subjects),and wider coverage of trials.Therefore,when collecting data,all relevant data should be collected as much as possible,and these data should be analyzed and processed.

(2) Data management and quality control.Any test data management and quality control are important.There are many data in PCT,and data management and processing are difficult.Therefore,in the design phase,pre-preparation is particularly crucial.Institution (organization) responsible for PCT needs to establish an internal PCT data management team under the existing quality assurance and control system,which is mainly to formulate data management plans,test operation standards and corresponding quality control documents.Besides,it should ensure the prior preparations are strictly implemented in the test process.In data collection and management,if an electronic information system is used,attention must be paid to ensure the traceability of information,data confidentiality and reliability.The data quality control process includes but is not limited to:pre-training of the experiment,the organization of the experiment,the test documents such as data collection or patient(subject) case report forms,as well as the allocation of test institutions and related resources.The data management process includes,but is not limited to:trial inspection,supervision,monitoring and reporting of adverse events,data problem correction and formulation of relevant measures to prevent bias,etc.In the design stage,the experiment needs to clarify the responsibilities of the corresponding positions to ensure the high efficiency and normal operation of the data management and control system[8].

2.5 Recruitment of researchers

Researchers should first meet but not limited to the following conditions.They should be able to undertake the responsibility of implementing the experiment in terms of education,training and experience.Besides,they meet all the conditions specified in the applicable management requirements,and obtain the qualification certificates required by sponsor or other relevant documents.All trial participants should have PCT professional skills and abilities,and at the same time,they should pass relevant training on drug clinical trial management practices.The recruitment of PCT researchers varies according to the types of experimental design.In iPCT,researchers are often recruited as individuals.Particular attention should be paid to cRCT and swRCT because groups are the intervention unit,and researchers are generally recruited to participate in the trial as the whole medical institution.

2.6 Random allocation of patients

In iPCT,patients are grouped for trials,and interventions are implemented on an individual basis.But in cRCT and swRCT,patients (subjects) are grouped for trials and interventions are implemented on a group basis.When individuals in the intervention group and the control group are prone to mutual interference or contamination,the group is chosen as the unit for the experiment grouping.In general,group allocation methods for cRCT include pairing,stratification and randomness.In some swRCTs,the research location and environment of the patients (subjects) and the implementation arrangements determine the order of grouping in the trial.

2.7 Research follow-up

Research follow-up is an important part to ensure the quality of the trial.Generally,in order to ensure the quality of follow-up,detailed information transmission and preaching to patients (subjects)are carried out at the initial stage of the trial,and a standardized follow-up plan is designed in advance for patients (subjects).The personnel responsible for the follow-up of patients (subjects) in the research team should receive comprehensive training on relevant content.Besides,a complete follow-up reminder system for patients (subjects) should be established.Due to the large sample size of PCT,it is difficult to carry out the standard follow-up for patients.In the implementation process,the follow-up period can be adjusted according to specific issues,and multiple information and big data network systems can be fully integrated to ensure the quality of follow-up of patients (subjects)[9].

2.8 Selection of statistical analysis methods

The statistical analysis of PCT is consistent with the basic principle of RCT,that is,the principle of intention-to-treat (ITT) analysis is adopted.Meanwhile,the full analysis set (FAS) is used as the main analysis data set.In addition,per-protocol set(PPS) analysis,subgroup analysis,and sensitivity analysis are also used.For statistical analysis,an analysis plan must be formulated in advance.For the discrepancies found,it is necessary to give explanations.Supplementary analyses that are not in the plan also need to be explained.The choice of analysis method varies according to the type of research.The performance is as follows.When comparing effects of cRCT,group coefficients,group effects,and time effects are often estimated using fixed-effects models.Covariate adjustments at the group or individual level are generally made in the analysis.Mixed-effect models and multi-level modeling techniques are often used for data analysis at the individual level.Its purpose is to simultaneously consider the related effects of groups,individual levels,and group characteristics.In swRCT,it is generally analyzed from two perspectives:horizontal and vertical.Horizontal analysis is to compare the intervention and control in different periods,while the vertical analysis is to compare the difference of group assigned to intervention or control within a certain time between consecutive interaction points.The study is mainly on the comparison between intervention and control.If the results of the study are feasible,a comparative analysis of group conversion control and intervention period can be added according to the situation.In fact,most trials use the group random effect model for analysis,and adjust the time effect regardless of its statistical significance.Then,they incorporate the horizontal and vertical comparison information into the intervention effect analysis[8].

2.9 Results and analysis

The results and analysis of PCT should be consistent with the trial protocol,including but not limited to the following aspects.The basic conditions between the different diagnosis and treatment groups should be studied to determine whether they can be compared.The original and final number of cases in each diagnosis and treatment group should be compared to find out cases excluded during the research process and the reasons.The effectiveness and safety of treatment groups should be shown by means of words,graphics,tables,research parameter,andPvalues.The degree of recognition and differences among treatment groups should be studied,and the statistical values of each group must be tested as well.Serious adverse events reported in the trials should be evaluated and discussed.

2.10 Discussion of results

Based on real-world data,PCT comprehensively applies multidisciplinary methods such as pharmacoepidemiology,biostatistics,and evidencebased medicine,combining the mechanism of the disease and drugs to evaluate the effectiveness and safety of drugs.By determining the research purpose,comparing the treatment results of different interventions in clinical practice with random and controlled design,the overall adverse reactions and curative effects of drugs are analyzed.Finally,PCT illustrates the limitations of existing research,including a description of the potential bias and inaccurate sources,and briefly summarizes its findings to provide the best evidence for the effect evaluation of after the listing of drugs and medical devices,and to provide support for regulatory authorities to make decisions.

2.11 Code of ethics

(1) Basic principles.RWS is an integrated research that combines scientific research with practice.Participants in the research include not only clinicians but also experimental researchers.From the doctor’s point of view,the research goals of RWS are the same as routine diagnosis and treatment.Objectively speaking,the risk of patients(subjects) should be less than the risk of diagnosis and treatment.PCT is still within the scope of clinical trial research,and it must be evaluated on ethical issues in accordance with the normal clinical ethics review principles and specifications[13-15].Some specifications can be based on the “Guiding Principles for Ethical Review of Drug Clinical Trials” issued by National Medical Products Administration (NMPA) for ethical approval of RWS[16-17].

(2) Management of informed consent.The specific management methods of informed consent mainly include the consent to individual patients(subjects).Researchers must obtain the voluntarily acknowledged informed consent of the patients(subjects).If patients (subjects) cannot give the informed consent in the special case,the consent or permission of the legally obligated agent must be obtained in accordance with the requirements of the current Chinese law.Additionally,patients (subjects)should fully understand the reasons for participating in the study and possible consequences.Sufficient time and opportunity should be given to patients (subjects)to consider whether to participate so as to seek their consent.PCT research also needs to obtain the approval of the national medical and health authorities and the ethics committee.Patients (subjects) of the research also need to give relevant informed consent and sign their written informed consent with exact date.Another is the informed consent of the patients (subjects) in a group.In order to consider the interests of the entire group of patients (subjects),the trial investigator should strictly formulate a group plan.Then they should inform patients (subjects)of the group and carry out the trial.Individual informed consent in cRCT is usually signed after group consultation,group notification and random assignment[8].Among them,whether individuals in the group can be exempted from informed consent should be submitted to the ethics committee according to their actual conditions,and the ethics committee will approve the exemption of the patient’s personal informed consent[3].

(3) Privacy protection and information security.When PCT involves the collection of personal identity,contact information,disease diagnosis,and other personal information through various systems,it needs to be explained in the research plan and approved by the ethics committee.Strict measures should be taken to protect individual privacy and avoid disclosure.

3 Conclusion

As an important type of RWS design,PCT has the advantage of integrating randomization and real-world data,and it can better control the bias in research.PCT is an important way to evaluate the results,safety and cost of medical interventions in the real clinical environment.Through scientific evaluation of the therapeutic effect of intervention measures in real or close to real medical environment,the best real-world evidence is obtained.