Why Do These Different Designs Matter?

There are potential adverse impacts to participants when protocols are unclear or a study is improperly described as an observational study, when in fact it is an interventional clinical trial, and also when a protocol is written as if it is a clinical trial when it really an observational study. The correct risks and benefits of the research may not be identified and analyzed, consent documents may fail to identify research procedures and research risks, thus participants would not be adequately informed as to what their participation involves, and the appropriate regulatory standards may not be applied.

An example of an interventional trial, that may not be identified as such, is a study where participants are assigned to be implanted with an FDA-approved hip system.  The research is conducted as a post-marketing study. The protocol indicates that participants meet criteria to enroll in the study based on their need to receive a hip implant, and meeting the indications for use of the device. These participants are then implanted with the device. Data are collected before, during, and after implantation.

Investigators state that participants will receive a device regardless of whether they are in the research study.  This could easily be misconstrued as an observational study.  However, when the protocol inclusion criteria prescribe use of the device as part of the research and implantation of the device is a research procedure, the protocol is an interventional clinical trial.

Thus the consent form would need to include procedures for implantation of the device, as well as the risks of the implant procedure, and the risks of the device itself. If the inclusion criteria were modified to enroll participants whose surgeons decided to use the device independent of the research, and the procedures only included that data would be collected, before, during, and after implantation, then the protocol would now be an observational study.

When Clinical Trials are Treated as Observational Studies:

When risks associated with the intervention are not identified as risks of the research, the research may improperly be categorized as research involving only minimal risk. In these cases, the protocol may be reviewed by a IRB Chair or and experienced IRB member designated by the IRB Chair (Expedited Review) rather than being reviewed at a fully convened meeting (Full Board Review), and would therefore be out of compliance with FDA/OHRP regulations.

Misclassification of interventional trials as observational studies may lead to the research being conducted without fulfilling FDA requirements for an IND or IDE, in cases where an IND or IDE may be required. For example, a study of an approved drug being used outside of its approved dosing range may be improperly evaluated as being exempt from IND requirements if the research is misclassified as an observational study when administration of the drug is a research procedure.

When Observational Studies are Treated as Clinical Trials:

When protocols are unclear or a study is improperly described as an interventional trial, when in fact it is an observational study, the converse of the above adverse impacts can occur. Participants may be told that the research involves risks, benefits and procedures that are not risks, benefits or procedures of the research, but rather are things that would have occurred as part of their usual medical care. The research may be required to be IRB-reviewed through a Full Board Review because of the risk of the intervention (which is not actually part of the study), when it could more appropriately be classified as a minimal risk study that can be reviewed through an Expedited Review pathway.

Observational Studies vs. Interventional Trials

To distinguish between observational studies and interventional trials, the protocol needs to be clear as to what procedures are mandated by the protocol, and how/when participants are selected to participate. The protocol title, the purpose, the background and a statement of whether procedures are “standard of care” are usually not sufficient to make this distinction clear. So what does the protocol need to include to make this distinction clear?

1. Inclusion Criteria

For the protocol to be clear, the inclusion criteria are critical. In an observational study, the inclusion criteria should indicate that the clinical decision to administer the drug, device, or procedure is independent (made outside) of the decision to take part in the research. Interventional trials would not have this criterion, but would instead have medical criterion that would qualify the participant to be eligible to receive the drug, device or procedure.

Comparative examples of inclusion criteria for observational studies and interventional trials are shown below:

2. Study Procedures

The description of study procedures is Just as important as the inclusion criteria, if not more important. In an observational study, the research procedures do not describe or specify the administration of the drug, device, or procedure under study, although they can specify other procedures such as blood tests and radiographic studies. In an interventional trial, the research procedures describe the use of the drug, device, or procedure under study.

Comparative examples of study procedures for observational studies and interventional trials are shown below:

Conclusion

Differentiating observational studies and interventional trials is important. It is necessary to ensure the protocol is clearly written to distinguish what is occurring as part of the research to adequately protect research participants, and to ensure proper regulatory compliance. To distinguish between these two, the inclusion criteria and the procedures involved in the research should clearly indicate what is within the research study and what is outside of the research. In a clearly designed protocol, this can cut down on IRB review time, prevent questions from the IRB, and ensure proper compliance and participant protection.

Parsing out the definition of “pragmatic clinical trial” can be difficult. There is overlap between the classic randomized clinical trial and the pragmatic clinical trial. Instead of focusing on the definitions and dividing clinical trials into two baskets, it can be better to think about how the concept of a pragmatic clinical trial will affect future trends in the conduct of all clinical trials. In this regard, there are two major differences. Pragmatic clinical trials are focused on care delivered to the patient versus focused on a specific treatment or procedure. Pragmatic clinical trials are also focused on effectiveness versus efficacy.

Clinical trials are classically conducted to evaluate a specific intervention, such as a new drug, a new device, or new surgical procedure. Payors, government agencies, and researchers are putting greater emphasis on clinical trials that focus on health care in a broader sense. Today, a clinical trial might evaluate whether a new drug lowers hemoglobin A1c without evaluating the acceptability of the treatment to the patient or the effect of non-pharmacologic interventions. In the future, we can expect more trials that evaluate whether a combination of diet, drug treatment, education, and
supervision lowers hemoglobin A1c in a manner that is readily adopted by the patient.

Another trend is to have study questions developed by clinicians and patients versus researchers and sponsors. Currently, a clinical trial is usually focused on gathering data to obtain approval from regulatory agencies. In the future, we might see more clinical trials that gather data to answer frequent questions of patients and clinicians, sometimes while also designed to support regulatory approvals. A typical patient question might be, “If I have lung disease, what level of supplemental oxygen needed at home indicates that it is no longer safe for me to fly?” A typical clinician question might be, “If I treat this disease aggressively, which patients will live longer with an improved quality of life, and which patients will just live longer?” In this sense, the trend is for clinical trials to become more patient focused rather than disease-focused, and to compare and evaluate real-world situations and alternatives rather than comparing an intervention to a placebo.

Although most clinical trials start out with a statement that the purpose of the research is to evaluate the safety and effectiveness of an intervention, clinical trials typically evaluate the safety and efficacy of an intervention.

Efficacy is the performance of an intervention under ideal circumstances, whereas effectiveness is the performance of an intervention under real-world circumstances. In the setting of a classic clinical trial, many variables are controlled to maximize the probability of getting clear answers to a few, narrowly defined research questions, and to minimize the probability of adverse outcomes. Staff are carefully trained in the intervention and the use of the intervention is highly controlled. Not infrequently, an intervention will be efficacious but will fail to be effective for reasons such as steep learning curve for clinician competency, or inability of patients to comply with the treatment. Sometimes, the inclusion criteria of a protocol demonstrating efficacy for a disease are so strict that the study results don’t apply to most patients with that disease. In this regard, the trend is for clinical trials that embrace heterogeneous subject populations to avoid studying a homogeneous population of no practical application.

Along with the trend to focus on effectiveness versus efficacy, is a trend for research that studies data gathered by clinicians versus data gathered by investigators. The goal is to capture data that the clinicians feel is necessary to evaluate their patients, rather than collecting structured data. The focus is on data collection dictated by healthcare versus data collection dictated by the protocol. There is also resignation to the fact that actual medical practice cannot be rigorously structured without the intensive resources used in many of today’s clinical trials. Therefore, there is a trend to allow protocol flexibility versus a machine-like rigidity. The data gathered by clinicians is often called “real-world data” and enormous amounts of these data are embedded in electronic medical records. These data are much less structured than the data collected in a classic protocol, but the hope is that machine learning techniques (“big data” or “artificial intelligence”) can be brought to bear to discover a strong signal despite the added noise.

Summary

In the United States, healthcare costs are skyrocketing while objective measures of quality lag far behind other industrialized nations. This has led clinicians, medical administrators, and thought leaders to seek objective answers to questions about what forms of healthcare are cost-effective. Pragmatic clinical trials have been proposed to get these answers. We can expect to see more research in the future that is focused on real-world care, designed to compare the effectiveness of combinations of interventions without conducting placebo-controlled trials, and are focused on effectiveness in the real-world rather than efficacy under artificial conditions.