Non-Experimental Claims Checklist

 1. Key Benefits:

Claims based on observational evidence can provide additional information to aid HCPs in making appropriate therapeutic choices that may not be captured in claims based solely on experimental evidence, such as randomized trials. Observational research can reliably answer clinically relevant questions in a complimentary way. It can also provide important information for clinical decisions in an easier fashion than experimental research. Some important information for clinical decision-making cannot be reliably answered with experimental studies.

2. Key Pitfalls:

Claims of comparative effectiveness from observational studies rely on more elaborate techniques for measuring and analyzing data. Findings from observational research may be largely influenced by the scientific judgments of the researchers analyzing the data. There is no real consensus on what statistical adjustment techniques are most reliable. Although there is no compelling evidence that observational studies are consistently less reliable than those of experimental studies, the reliability of findings from a single observational study is much more suspect than that of a similarly sized randomized trial and requires sufficient details regarding the analytic approach to be properly scrutinized.

Managing pitfalls:

As a general principle, the degree of evidence must support the degree of claim. In general observational studies may be appropriate for claims relating to adherence, persistence, preference, and as additional support of efficacy/safety claims established by evidence based on randomized controlled trials. In general, the use of observational studies alone is not sufficient to support claims of efficacy and safety. 

A consensus checklist has been developed for Strengthening The Reporting of OBservational studies in Epidemiology (STROBE) that are intended to appear in biomedical journals. Most of the elements of the checklist are also relevant to allow scrutiny of claims based on observational data submitted to the PAAB.

The checklist should be used to guide industry and to assist the PAAB staff in determining whether findings from observational studies may appear within advertising/promotional systems (APS). The checklist relates to factors specific to the reporting of observational studies that attempt to make comparisons (i.e. causal claims).  Before using the checklist, applicants should be sure the claim is consistent with the Health Canada approved Terms of Market Authorization (TMA).  Observational studies cannot be used to support observations that contradict anything in the TMA (with respect to magnitude, direction, or duration of clinical effect).

Observational Claims Checklist (Adapted from the STROBE Checklist)

Principles, Rationale and Application of Checklist Items

An Explanation and Elaboration article discusses each checklist item and gives methodological background and published examples of transparent reporting.

This checklist is best used in conjunction with the article, freely available at

http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0040297

Information on the STROBE Initiative is available at www.strobe-statement.org .