Abstract

The ranking of products (e.g., formulations, molecules...) according to pre-specified criteria (efficacy, safety, etc.) can be key for companies developing new technologies. For example, in the pharmaceutical industry, it can be used for molecule screening at early phase development to identify the best candidates. In the cosmetic industry, a ranking can be useful for positioning all the solutions that has been tested on a given indication (anti-dandruff; skin aging; …).

Rankings which involve both a positive and a negative reference assess the ability of products to approach an ideal case, while being the farthest away from the worst case, according to a given distance measurement (position in the ranking, value of an efficacy parameter…). The complexity lies on how to define the said distance measure. Indeed, one could simply take the products whose mean value is closest to the positive reference and consider it as the best. However, this basic method omits a major factor of discrimination among products, the uncertainty of their effects. A higher variability around the mean would translate into a higher risk of being comparable to the worst case. Thus, the ranking methodology must account for the uncertainty in two ways, (i) to determine how close the product is from the ideal case and (ii) to determine the probability to outperform the worst case.

Here we proposed a new Bayesian methodology to conjugate these two objectives and compute a unique metric allowing to rank products efficiently. It can be applied to multiple domains involving different kinds of efficacy criteria. It is primarily intended and defined to be used in a meta-analytical framework. It aims to rank products tested in different Randomized Clinical Trials (RCTs), compared to a positive and negative reference, and according to homogeneous protocols.

For each product, the posterior probability to perform as well as the positive reference is computed and is penalized by the posterior probability of performing like the negative reference. This computation is then compared to a hypothetical case of no knowledge as represented by a uniform distribution. The final metric obtained shows the improvement of a product from this state of ignorance. This allows to eliminate products that achieve good scores by chance. The product that achieves the highest metric is thus the product that obtains a high probability to perform as well as the positive reference, that has a low chance of performing like the negative reference, and that strays the furthest away from ignorance.

This Bayesian methodology provides an objective criterion to rank products that appropriately accounts and penalizes for the uncertainty around a product estimate when positive and negative references are used. Moreover, this metric has the benefit to be directly interpretable. It shows how much better a product is from a state of ignorance with the aim to answer the following question: Is my product performing better than any product taken at random for which I do not have any information?