Bayesian Statistics Checklist
| Item No |
Checklist Item (clients can use this tool to help make decisions regarding use of Bayesian statisitcs in advertising claims) |
✓ |
| What Must Appear in the Advertisement |
| 3.1 |
Consistency with Terms of Market Authorization |
|
| 3.3 |
Describe the comparator and central tendency for the posterior probability |
|
| 3.4 |
Describe the variance for the posterior probability |
|
| What Must Appear in the Published Study for Claim Validation |
| 3.1 |
Consistency with Terms of Market Authorization |
|
| 3.2 |
Describe the prior probability and how it was derived |
|
| 3.3 |
Describe the comparator and central tendency for the posterior probability |
|
| 3.4 |
Describe the variance for the posterior probability |
|
| 3.5 |
Describe whether alternative prior probabilities, Bayesian models or analytic assumptions were tested and the sensitivity of the results to these alternative assumptions |
|
1. Key Benefits:
Bayesian statistics provide some measure of the degree to which a claim of effectiveness is true as opposed to traditional frequentist statistics, which is less intuitive.
2. Key Pitfalls:
Although increasingly accepted, there may be inconsistencies in approaches to reporting these statistics which undermines their usefulness for clinical decision making.
3. Managing pitfalls:
The checklist provides 5 helpful principles to guide industry and PAAB staff in determining whether reporting of Bayesian statistics may appear within advertising/promotional systems (APS). The checklist relates only to factors specific to the reporting of Bayesian statistics. It is assumed these statistics would be derived from comparisons using study designs that are deemed acceptable to the PAAB, such as randomized controlled trials. Refer to the PAAB code for general factors relating to acceptability of a study.
❐ 3.1 Consistency with Terms of Market Authorization
Principles:
Drug advertising should be consistent with the Health Canada approved Terms of Market Authorization (TMA).
Rationale:
Advertising content which is inconsistent with the TMA would contravene section 9.1 of the Food and Drugs Act.
Application:
- Bayesian statistics cannot be used to support observations that contradict anything in the TMA (with respect to magnitude, direction, or duration).
❐ 3.2 Describe the prior probability and how it was derived
Principle:
Unlike frequentist approaches, Bayesian statistics do not use consistent assumptions for testing differences.
Rationale:
The estimate of observed difference is sensitive to assumptions about prior knowledge which are embodied in the prior probability.
Application:
Describe whether the source of the prior is from the study itself or from information outside the study. Describe the central tendency (e.g. mean), variance (e.g. 95% confidence interval) and distributional form of the prior probability.
❐ 3.3 Describe the comparator and central tendency for the posterior probability
Principle:
Probability is useful for clinical decision making.
Rationale:
Claims of differences in effectiveness must be supported by a quantitative estimate of the magnitude of difference between clinical strategies. An estimate of the increased probability of success is most useful for clinical decision making compared to estimates of probability of effectiveness or no quantitative estimate.
Application:
Describe the difference in terms of a mean, median or other appropriate measure of central tendency. For example, “Compared to X, Drug Y reduced the incidence of outcome Z by 3%”. It is inappropriate to state “The probability that Drug Y reduced the incidence of outcome Z compared to X is 95%”.
❐ 3.4 Describe the variance for the posterior probability
Principle:
Measures of precision are useful for clinical decision making.
Rationale:
Understanding the range of probabilities that may reasonably occur given current knowledge is useful for decision making. Decisions may be influenced by the precision of the estimate.
Application:
Applicants are encouraged to report a 95% credibility (also called probability) interval for the posterior probability, along with the measure of central tendency. For example, “Compared to X, Drug Y reduced the incidence of outcome Z by 3% (95% credibility interval 0.01% to 5%)”. For example, “Drug Y was associated with an average HbA1c reduction of 0.9 (95% CrI 0.8 to 1.0)”. Higher levels of precision (e.g. 98% credibility intervals) are also acceptable.
❐ 3.5 Describe whether alternative prior probabilities, Bayesian models or analytic assumptions were tested and the sensitivity of the results to these alternative assumptions
Principle:
The reliability of the findings must be demonstrated.
Rationale:
Bayesian statistics do not reduce opportunities for chance findings. There are opportunities to exaggerate findings using different assumptions and the complexity of Bayesian analysis may make this harder to discern. Findings are most likely to be sensitive to alternate assumptions about the prior probability used but different analytic approaches and Bayesian models may also lead to differences in posterior probability estimates.
Application:
Describe the effect of alternate assumptions about prior probability has on the main estimate of difference. Describe the known effect of any other alternate analytic or modeling assumptions.
View in Full Forum
link to change the temporary password to one of their choosing”
Community Forum Guide