Conjugate Models

BayesianExperiments.ConjugateBernoulliType
ConjugateBernoulli <: ConjugateModel

Bernoulli likelihood with Beta distribution as the conjugate prior.

ConjugateBernoulli(α, β)              # construct a ConjugateBernoulli

update!(model, stats)             # update model with statistics from data
samplepost(model, numsamples)    # sampling from the posterior distribution
samplestats(model, numsamples)   # sampling statistics from the data generating distribution
source
BayesianExperiments.ConjugateExponentialType
ConjugateExponential <: ConjugateModel

Exponential likelihood with Gamma distribution as the conjugate prior.

ConjugateExponential(α, β)            # construct a ConjugateExponential

update!(model, stats)             # update model with statistics from data
samplepost(model, numsamples)    # sampling from the posterior distribution
samplestats(model, numsamples)   # sampling statistics from the data generating distribution
source
BayesianExperiments.ConjugateNormalType
ConjugateNormal <: ConjugateModel

Normal likelihood and Normal Inverse Gamma distribution as the conjugate prior.

Parameters

  • μ: mean of normal distribution
  • v: scale variance of Normal
  • α: shape of Gamma distribution
  • θ: scale of Gamma distribution
ConjugateNormal(μ, v, α, θ)           # construct a ConjugateNormal

update!(model, stats)             # update model with statistics from data
samplepost(model, numsamples)    # sampling from the posterior distribution
samplestats(model, numsamples)   # sampling statistics from the data generating distribution

References

  • The update rule for Normal distribution is based on this lecture notes.
source
BayesianExperiments.ConjugateLogNormalType
ConjugateLogNormal(μ, v, α, θ)

A model with Normal likelihood and Normal Inverse distribution with log transformed data. Notice LogNormal in Distributions.jl takes mean and standard deviation of $\log(x)$ instead of $x$ as the input parameters.

ConjugateLogNormal(μ, v, α, θ) # construct a ConjugateLogNormal

lognormalparams(μ_logx, σ²_logx) # convert normal parameters to log-normal parameters
update!(model, stats)              # update model with statistics from data
samplepost(model, numsamples)     # sampling from the posterior distribution
samplestats(model, numsamples)    # sampling statistics from the data generating distributio
source
BayesianExperiments.ChainedModelType
ChainedModel <: ProbabilisticModel

ChainedModel is a combination of ConjugateModels chained by the specified operator. It can be used to model a multiple step process.

source

Bayes Factor Models

BayesianExperiments.NormalEffectSizeType

NormalEffectSize <: EffectSizeModel

A standard effect size model has two hypotheses: $H_0$(null) an $H_1$(alternative):

  1. $H_0$: $\mu = m_0$
  2. $H_1$: $\mu ≠ m_0$

with the population mean $\mu$ and pre-specified standard deviation $\sigma$. We want to test whether $\mu$ is equal to $\mu_0$ or not.

The prior of the standard effect size is

$\delta | H_1 \sim \text{Normal}(0, \sigma_0^2)$

where $\delta$ is the standard effect size. When $\sigma_0 = 1$the prior is called unit-information prior.

The standard effect size $\delta$ is defined as

$\delta = \frac{\mu - \mu_0}{\sigma}.$

In practice, the standard deviations are unknown but in large sample scenario we assume they are known and use their estimates.

Fileds

  • μ0: mean of null hypothesis.
  • σ0: The prior standard deviation of the effect size.
  • p0: prior belief of $H_0$. This is used to calculate the prior odds. Default $0.5$.

Methods

bayesfactor(model, stats)    # calculate Bayes factor from one group statistics
bayesfactor(model, twostats) # calculate Bayes factor from two group's statistics

References

  1. Chapter 5 hypothesis Testing with Normal Populations in An Introduction to Bayesian Thinking.
  2. Deng, Alex, Jiannan Lu, and Shouyuan Chen. "Continuous monitoring of A/B tests without pain: Optional stopping in Bayesian testing." 2016 IEEE international conference on data science and advanced analytics (DSAA). IEEE, 2016.
source
BayesianExperiments.StudentTEffectSizeType
StudentTEffectSize <: BayesFactorModel

A model with Bayes factor from the Student's t distributions. We have a standard effect size model has two hypotheses: $H_0$(null) an $H_1$(alternative):

  1. $H_0$: $\mu = m_0$
  2. $H_1$: $\mu ≠ m_0$

The model uses the Jeffreys-Zellener-Siow (JZS) prior. More specifically, we use a Cauchy prior on $\mu$ for $H_1$

$\mu | \sigma^2 \sim \text{Cauchy}(0, r^2 \sigma^2)$

and a Jeffrey's prior on $\sigma$:

$p(\sigma^2) \propto \frac{1}{\sigma2}$

for both $H_0$ and $H_1$.

Fields

  • $r$: Prior standard deviation of the effect size.
  • $rtol$: Numerical tolerence fo the $quadgk$ function used in the denominator calculation.
  • $p0$: prior belief of $H_0$. This is used to calculate the prior odds. Default $0.5$.

References

  • Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009). Bayesian t tests for accepting and rejecting the null hypothesis. Psychonomic bulletin & review, 16(2), 225-237.
source

Stopping Rules

BayesianExperiments.OneSidedBFThreshType
OneSidedBFThresh <: BayesFactorThresh

The bayes factor itself is interpretable as the comparative evidence of data under the two competing hypotheses. Higher bayes factor, as defined $\text{BF}_{10}$ favours the alternative hypothesis.

In practice, a threshold can be used to make decision in bayes factor experiment. The experiment will stop when $\text{BF}_{10}$ > threshold.

In this case, the bayes factor of alternative over null is above the threshold, we can accept the alternative hypothesis. Otherwise, we don't have enough evidence to accept alternative hypotheses.

source
BayesianExperiments.TwoSidedBFThreshType
TwoSidedBFThresh <: BayesFactorThresh

The bayes factor itself is interpretable as the comparative evidence of data under the two competing hypotheses. Higher bayes factor, as defined $\text{BF}_{10}$ favours the alternative hypothesis.

In practice, a threshold can be used to make decision in bayes factor experiment. The experiment will stop when

  1. $\text{BF}_{10}$ > threshold
  2. $\text{BF}_{10}$ < 1/threshold

In (1), the bayes factor of alternative over null is above the threshold, we can accept the alternative hypothesis. In (2), the bayes factor is below the inverse of the threshold, we can accept the null hypothesis. Otherwise, we don't have enough evidence to accept any of these hypotheses.

source

Experiment

BayesianExperiments.ExperimentBFType
ExperimentBF{M} <: Experiment

ExperimentBF is an experiment using a Bayes Factor between the null and alternative hypothesis as the stopping rule.

Constructors

ExperimentBF(kwargs...)

Keywords

  • model::M: Prior of effect size of alternative hypothesis
  • p0::Float64: Probablity of null hypothesis
  • winner::Union{String, Nothing: Decision to reject the null hypothesis or not
  • rule::BayesFactorThresh: Stopping rule using Bayes Factor as the threshold
  • stats: Statistics for calculating the bayes factor. Default is nothing.
  • modelnames: Names of the hypotheses. Default is ["null", "alternative"].
source

Simulation

BayesianExperiments.SimulationType
Simulation(experiment, parameters, datagendists, maxsteps, onestepsizes, minsteps)

A simulation setup includes the experiment, data generating distributions, max number of steps and minimum number of steps.

source