A formal comparison of different methods for establishing cut points to distinguish positive and negative samples in immunoassays

Abstract

Biotechnology derived therapeutics may induce an unwanted immune response leading to the formation of anti-drug antibodies (ADA). As a result the efficacy and safety of the therapeutic protein could be impaired. Neutralizing antibodies may, for example, affect pharmacokinetics of the therapeutic protein or induce autoimmunity. Therefore a drug induced immune response is a major concern and needs to be assessed during drug development. It is therefore crucial to have assays available for the detection and characterization of ADAs. These assays are used to classify samples in positive and negative samples based on a cut point. In this manuscript we investigate the performance of established and newly developed methods to determine a cut point in immunoassays such as ELISA through simulation and analysis of real data. The different methods are found to have different advantages and disadvantages. A robust parametric approach generally resulted in very good results and can be recommended for many situations. The newly introduced method based on mixture models yields similar results to the robust parametric approach but offers some additional flexibility at the expense of higher complexity.

Introduction

Biotechnology derived therapeutics may induce an unwanted immune response resulting in the formation of anti-drug antibodies (ADA). As a consequence of the development of ADA efficacy and safety of the therapeutic protein could be impaired. For example, binding or neutralizing antibodies may affect pharmacokinetics or functionality of the therapeutic protein or even induce autoimmunity when the ADA cross-react with endogenous counterparts. In addition, unwanted immune responses may lead to allergic reactions. As a result, drug induced immune responses to a therapeutic protein are a major concern and need to be assessed during drug development.

Consequently there is a need to develop appropriate assays for the detection and characterization of ADA. In 2007, the European Medicines Agency (EMA) published a guideline that describes the general strategy for the development and validation of assays for immunogenicity assessment of biotechnology derived therapeutic proteins [1]. A multi-tiered approach for the testing of patient samples is recommended. In the first instance a screening assay is used for rapid identification of positive samples while subsequently an additional confirmatory assay is used to confirm the results of the screening assay. As a third step, a functional assay for assessment of the neutralizing capacity of antibodies is recommended. Screening, confirmatory and functional assays for detection and characterization of ADA need to be validated [2], [3].

A critical step during assay development and validation is the definition of an appropriate cut-off that can be used to distinguish between positive and negative samples in the screening assay. This initial assay needs to be as sensitive as possible to maximize the detection of true positive samples and should be designed to avoid classifying positive samples as negative. A proportion of false positive samples is acceptable as they can be identified by the following confirmatory assay while costs and time urge to take few samples to this second stage. This approach ensures that the assays will detect as many patients who have indeed developed antibodies.

A valid statistical approach needs to be elaborated to define a reliable cut-off value used in screening and confirmatory assays [4]. For defining an appropriate cut point usually control samples obtained from healthy subjects or untreated patients are used. Such a pool of control samples is in most cases of heterogeneous composition, containing sub-populations consisting of true negative samples as well as true and false positive samples. The portion of each sub population has impact on the final cut-off value if one assumes that indeed all samples are truly negative. For example, a high content of true positives in the sample population due to specific pre-existing antibodies used for calculating the cut-off would result in a high number of false negative evaluation of samples. Consequently it is crucial to use statistical methods that deal with potential (false) positive samples appropriately when determining a cut point. Different strategies to detect and characterize ADA's have been discussed in [4], [5] but no formal evaluation of the methods has yet been undertaken.

In this paper we evaluate a variety of established and less established methods for cut point determination. We will introduce the methods in Section 2 before we compare them thoroughly via simulation (Section 3). We conclude with an in-depth discussion and some future directions.