Bayesian Analysis of Preference Tests

The issue: Preference tests are simple to carry out and routinely done, but standard analysis using a “frequentist” approach, means the decisions are based only on data collected for today’s study, whereas the Bayesian approach offers the chance to combine historical data with current data, providing a formal (rather than subjective) way of incorporating prior knowledge into decision making. In addition, the frequentist approach depends on the concept of repeated sampling and is heavily reliant on statistical significance making the chosen sample size crucial. The Bayesian approach eliminates the reliance on p-values and instead quantifies decision risk in an intuitive and business-relevant manner and provides a basis for reduction in sample size given prior knowledge and agreed risk tolerance.

The problem: Bayesian analysis is new to the sensory and consumer world and is not currently available in many stats software package and requires coding or specialist software.

The solution: The Qi Bayesian app allows the user to perform a simple Bayesian analysis of a preference test with minimal data input and clear graphical outputs, which can be compared to desired target performance goals and business decisions.

Bayes Preference Test Application

The user first sets a prior belief of how the data should look (a prior distribution for p) based on historical data or expert opinion, in the app. Then the results of the actual preference test are input. These 2 items are then combined to come up with a Beta posterior distribution graph
for preference.

The app gives the user the parameters and characteristics of the posterior distribution and compares these to a user-defined target performance and action standards.

The ‘app’ works extremely quickly, and so the user can change the basic parameters and see instantly how changing the shapes of the held beliefs or previous results (prior distributions) will affect the resulting posterior distribution.

In addition performance against different business targets can also be assessed interactively. The output is shown clearly with colour coding for ‘good and ‘bad’ results compared to target performance.

Once the analyses are finalised the results can be exported.

Key features

• Web based
• Simple to use and input data
• Complex stats method made easy
• Easy to calculate many scenarios and try different priors
• Clear visualisations with interactive functions for comparing to ‘target’ deliverables
• Results displayed and exported simply
• Colour coding to clearly identify ‘good’ and ‘bad’ results compared to target performance of the product