Pragmatic Free Trial Meta is a non-commercial, 프라그마틱 환수율 open data platform and infrastructure that supports research on pragmatic trials. It collects and distributes cleaned trial data, ratings and evaluations using PRECIS-2. This permits a variety of meta-epidemiological studies to evaluate the effects of treatment across trials with different levels of pragmatism.
BackgroundPragmatic studies provide real-world evidence that can be used to make clinical decisions. However, the use of the term "pragmatic" is not consistent and its definition and assessment requires clarification. Pragmatic trials must be designed to inform clinical practice and policy decisions, rather than to prove a physiological or clinical hypothesis. A pragmatic study should strive to be as close to real-world clinical practice as is possible, including the participation of participants, setting up and design, the delivery and execution of the intervention, as well as the determination and analysis of the outcomes, and primary analysis. This is a significant difference from explanatory trials (as described by Schwartz and Lellouch1) which are intended to provide a more complete confirmation of a hypothesis.
The most pragmatic trials should not conceal participants or clinicians. This could lead to an overestimation of the effects of treatment. Pragmatic trials will also recruit patients from various healthcare settings to ensure that their results can be applied to the real world.
Additionally, pragmatic trials should focus on outcomes that are vital to patients, such as quality of life or functional recovery. This is particularly relevant when it comes to trials that involve invasive procedures or those with potential serious adverse events. The CRASH trial29 compared a 2 page report with an electronic monitoring system for patients in hospitals with chronic cardiac failure. The catheter trial28 however was based on symptomatic catheter-related urinary tract infection as its primary outcome.
In addition to these characteristics pragmatic trials should reduce the requirements for data collection and trial procedures to cut down on costs and time commitments. Additionally pragmatic trials should try to make their findings as applicable to clinical practice as they can by making sure that their primary analysis is the intention-to-treat approach (as described in CONSORT extensions for pragmatic trials).
Many RCTs that do not meet the criteria for pragmatism, but have features that are in opposition to pragmatism, have been published in journals of varying types and incorrectly labeled pragmatic. This could lead to misleading claims of pragmaticity, and the usage of the term must be standardized. The development of a PRECIS-2 tool that provides an objective, standardized evaluation of the pragmatic characteristics is a first step.
Methods
In a pragmatic research study the aim is to inform clinical or policy decisions by showing how an intervention can be integrated into routine care in real-world contexts. Explanatory trials test hypotheses concerning the cause-effect relationship within idealised environments. Consequently, pragmatic trials may have less internal validity than explanatory trials and may be more susceptible to bias in their design, conduct, and analysis. Despite these limitations, pragmatic trials can provide valuable information to decision-making in healthcare.
The PRECIS-2 tool scores an RCT on 9 domains, with scores ranging between 1 and 5 (very pragmatic). In this study, the recruitment, organization, flexibility in delivery and follow-up domains scored high scores, however the primary outcome and the method of missing data were below the limit of practicality. This suggests that a trial could be designed with well-thought-out practical features, yet not harming the quality of the trial.
It is, however, difficult to assess the degree of pragmatism a trial really is because the pragmatism score is not a binary characteristic; certain aspects of a trial can be more pragmatic than others. A trial's pragmatism can be affected by modifications to the protocol or the logistics during the trial. In addition 36% of 89 pragmatic trials discovered by Koppenaal and co. were placebo-controlled or conducted prior to approval and a majority of them were single-center. Therefore, they aren't as common and are only pragmatic when their sponsors are accepting of the lack of blinding in these trials.
A common feature of pragmatic studies is that researchers attempt to make their findings more relevant by studying subgroups within the trial sample. However, this often leads to unbalanced comparisons with a lower statistical power, which increases the chance of not or misinterpreting the results of the primary outcome. In the case of the pragmatic studies that were included in this meta-analysis this was a significant problem because the secondary outcomes weren't adjusted for variations in baseline covariates.
Furthermore, pragmatic studies may pose challenges to collection and interpretation safety data. This is due to the fact that adverse events are usually self-reported and are susceptible to errors, delays or coding variations. It is therefore important to enhance the quality of outcomes ascertainment in these trials, in particular by using national registries rather than relying on participants to report adverse events in the trial's database.
Results
While the definition of pragmatism doesn't require that all clinical trials are 100% pragmatic, there are benefits when incorporating pragmatic components into trials. These include:
By including routine patients, the results of trials can be translated more quickly into clinical practice. However, pragmatic studies can also have disadvantages. For instance, the appropriate type of heterogeneity can help the trial to apply its findings to a variety of settings and patients. However, the wrong type of heterogeneity can reduce assay sensitivity and therefore decrease the ability of a trial to detect small treatment effects.
Many studies have attempted classify pragmatic trials using different definitions and scoring methods. Schwartz and Lellouch1 have developed a framework for distinguishing between explanatory trials that confirm the clinical or physiological hypothesis, and pragmatic trials that inform the selection of appropriate therapies in clinical practice. The framework consisted of nine domains that were scored on a 1-5 scale, with 1 being more explanatory while 5 was more practical. The domains were recruitment and setting, delivery of intervention, flexible adherence, follow-up and primary analysis.
The original PRECIS tool3 was based on a similar scale and domains. Koppenaal et al10 created an adaptation of this assessment dubbed the Pragmascope that was simpler to use in systematic reviews. They discovered that pragmatic systematic reviews had a higher average scores across all domains, but lower scores in the primary analysis domain.
The difference in the primary analysis domains could be due to the way in which most pragmatic trials approach data. Certain explanatory trials however don't. The overall score was lower for pragmatic systematic reviews when the domains on organisation, flexible delivery and follow-up were combined.
It is important to note that the term "pragmatic trial" does not necessarily mean a poor quality trial, and there is an increasing rate of clinical trials (as defined by MEDLINE search, but this is not sensitive nor specific) that employ the term 'pragmatic' in their abstract or title. These terms may indicate an increased appreciation of pragmatism in abstracts and titles, however it's unclear whether this is evident in content.
Conclusions
In recent times, pragmatic trials are increasing in popularity in research because the importance of real-world evidence is increasingly recognized. They are randomized trials that compare real world treatment options with clinical trials in development. They involve patient populations more closely resembling those treated in regular medical care. This approach has the potential to overcome the limitations of observational research that are prone to biases that arise from relying on volunteers and the lack of availability and coding variability in national registry systems.
Pragmatic trials offer other advantages, including the ability to draw on existing data sources and a higher chance of detecting significant distinctions from traditional trials. However, these trials could have some limitations that limit their validity and generalizability. For example the participation rates in certain trials may be lower than expected due to the healthy-volunteer effect as well as incentives to pay or compete for participants from other research studies (e.g., industry trials). A lot of pragmatic trials are limited by the need to enroll participants in a timely manner. Additionally some pragmatic trials do not have controls to ensure that the observed differences are not due to biases in the conduct of trials.
The authors of the Pragmatic Free Trial Meta identified RCTs that were published between 2022 and 2022 that self-described themselves as pragmatic. The PRECIS-2 tool was used to assess the pragmatism of these trials. It includes domains such as eligibility criteria, recruitment flexibility, adherence to intervention, and follow-up. They found that 14 of these trials scored pragmatic or highly practical (i.e. scoring 5 or higher) in any one or more of these domains, and that the majority were single-center.
Studies that have high pragmatism scores tend to have more lenient criteria for eligibility than conventional RCTs. They also include populations from many different hospitals. According to the authors, may make pragmatic trials more relevant and useful in everyday practice. However they do not guarantee that a trial will be free of bias. The pragmatism characteristic is not a fixed characteristic; a pragmatic test that doesn't have all the characteristics of an explanation study can still produce valid and useful outcomes.
