Skip to main content
Download PDF
  • Methodology article
  • Open access
  • Published:

Precision and linearity targets for validation of an IFNγ ELISPOT, cytokine flow cytometry, and tetramer assay using CMV peptides

BMC Immunology volume 9, Article number: 9 (2008) Cite this article

Abstract

Background

Single-cell assays of immune function are increasingly used to monitor T cell responses in immunotherapy clinical trials. Standardization and validation of such assays are therefore important to interpretation of the clinical trial data. Here we assess the levels of intra-assay, inter-assay, and inter-operator precision, as well as linearity, of CD8+ T cell IFNγ-based ELISPOT and cytokine flow cytometry (CFC), as well as tetramer assays.

Results

Precision was measured in cryopreserved PBMC with a low, medium, or high response level to a CMV pp65 peptide or peptide mixture. Intra-assay precision was assessed using 6 replicates per assay; inter-assay precision was assessed by performing 8 assays on different days; and inter-operator precision was assessed using 3 different operators working on the same day. Percent CV values ranged from 4% to 133% depending upon the assay and response level. Linearity was measured by diluting PBMC from a high responder into PBMC from a non-responder, and yielded R2 values from 0.85 to 0.99 depending upon the assay and antigen.

Conclusion

These data provide target values for precision and linearity of single-cell assays for those wishing to validate these assays in their own laboratories. They also allow for comparison of the precision and linearity of ELISPOT, CFC, and tetramer across a range of response levels. There was a trend toward tetramer assays showing the highest precision, followed closely by CFC, and then ELISPOT; while all three assays had similar linearity. These findings are contingent upon the use of optimized protocols for each assay.

Background

Validation of immunological assays can take a number of forms, and is required for compliance with Good Laboratory Practice (GLP), or for submission of data to licensing agencies. Two basic components of assay validation are the demonstration that an assay performs with adequate reproducibility for the intended purpose, and that the assay readout is linear over a useful range of data [1].

Specific guidelines exist for validation of traditional immunoassays such as ELISA, and the expected levels of precision and linearity of these assays are well-known [2, 3]. Less well-characterized are cellular immunity assays, of which the single-cell assays like tetramer staining [4], cytokine flow cytometry (CFC) [5, 6], and ELISPOT [7] are among the most popular. Some data has been published regarding the precision of individual assays [8–10], and there is very limited data on the linearity of CFC [11]. However, precision and linearity have not been compared across assays, and expected levels of precision and linearity of these assays have not been determined in a side-by-side fashion.

Precision and linearity are important aspects of cellular immunity assays, since (a) cellular assays are inherently more complex, and thus less reproducible, than traditional immunoassays; and (b) cellular immunity assays are frequently used to detect rare antigen-specific T cell populations, which may be present at or near the assay detection limit. It is thus crucial to demonstrate that an assay is reproducible enough to generate reliable data in the response range expected for, say, a vaccine clinical trial, and that linearity is adequate to quantitatively compare results between treatment groups or between trials.

Fortunately, we now know that at least some vaccines to HIV and cancer, for example, can generate readily detectable T cell responses by assays such as tetramer, CFC, and ELISPOT [12, 13]. Still, there is wide variability in the performance of such assays between labs [14]. Compliance with GLP thus requires that a given lab demonstrate its proficiency for a given assay, preferably with reference to an accepted standard.

Here we compare results from optimized protocols for tetramer staining, CFC, and ELISPOT, performed on shared cryopreserved PBMC specimens, with expert laboratories performing the individual assays. From this data, we derive target values for those who wish to determine precision and linearity of these assays in their own laboratory, and we also facilitate comparison of the three assays with regard to their relative precision and linearity.

Results

Study design and response levels of donors

In order to allow meaningful comparisons between assays, this study was performed using a format previously published [15], in which three laboratories, each expert at an individual assay, performed their assay of expertise in parallel on the same cryopreserved PBMC. PBMC from healthy CMV seropositive donors were chosen to represent a high, medium, and low responder to CMV pp65495–503 peptide [16] and/or a CMV pp65 peptide mix [17]. Actual mean responses across all the assays as obtained in this study are shown in Table 1. Assays were performed with six replicates in order to determine intra-assay precision. They were repeated on eight different days in order to determine inter-assay precision. Three operators performed assays in parallel on a single day in order to determine inter-operator precision. And triplicate samples from the high responder were serially diluted into non-responsive PBMC in order to determine linearity. Results were then collated across the three laboratories.

Table 1 Mean response levels of the three CMV-responsive donors
Full size table

Precision of individual assays

When the coefficient of variation (CV) of six parallel replicates was plotted against the mean response level, a characteristic non-linear relationship was observed for all three assays (Figure 1), whereby the CV rose dramatically as the mean approached zero. Since the six replicates were performed on eight separate days, a standard deviation (SD) could be determined for the intra-assay CV. Taking the mean+SD of the intra-assay CV at any given response level allows one to determine an "acceptable zone" wherein a laboratory validating these assays might expect their data to lie (gray zone in Figure 1). For example, at a response level of 0.25%, one would expect a CV no higher than about 0.15 (15%) for either tetramer or CFC, based on these data. Similarly, for a response level of about 30 SFC in ELISPOT, one would expect a CV no higher than about 0.60 (60%).

Figure 1
figure 1

Intra-assay CVs. The mean CV for six replicates was plotted for samples from three donors and two different antigen stimulations in tetramer, CFC, and ELISPOT assays. Circles represent donor 41; triangles, donor 68; and squares, donor 43. Open symbols represent CMV pp65495–503 responses; closed symbols represent CMV pp65 peptide mix responses (CFC and ELISPOT assays only). Note that certain responses were very similar, so some symbols overlap. Error bars represent the SD of 10 times that the six replicates were repeated. The gray zones indicate the area within which a laboratory doing validation could expect their data to lie.

Full size image

Because antigen-specific assays are often used to analyze data in a range where CV is non-linear with the mean, we chose to use SD as a measure of variability in further analyses. As shown in Figure 2, the SD was relatively linear with mean for all three assays, when comparing six parallel replicates (intra-assay precision, left panels), or eight separate experiments (inter-assay precision, middle panels), or three different operators (inter-operator precision, right panels). For ELISPOT, the SD was significantly lower for intra-assay precision compared to inter-assay (p = 0.03) and inter-operator (p = 0.03) precision. There was a similar trend for CFC, though the comparisons did not reach statistical significance (p = 0.06). Tetramer assays had low SD in all three situations, with no significant differences among intra-assay, inter-assay, and inter-operator precision (0.25 < p < 0.75).

Figure 2
figure 2

SD of each assay. The mean response for each sample (as per Figure 1) was plotted versus the SD of six replicates (intra-assay precision), eight assays on different days (inter-assay precision), or three operators on the same day (inter-operator precision). Circles represent donor 41; triangles, donor 68; and squares, donor 43. Open symbols represent CMV pp65495–503 responses; closed symbols represent CMV pp65 peptide mix responses (CFC and ELISPOT assays only). Error bars in the intra-assay graphs represent the SD of 10 times that the six replicates were repeated. Lines represent linear regression of the combined data (both antigens, where used), with 95% confidence intervals of the regression shown with dotted lines.

Full size image

Comparative precision of the three assays

In Figure 2, it is difficult to compare the precision across all three assays, because the readout for ELISPOT (SFC per 2.5 × 105 PBMC) is different than the readout for CFC and tetramer staining (percent of CD8+ T cells). In order to compare all three assays on the same scale, the ELISPOT values were converted to a percent scale (since the percentage of CD8+ T cells in each PBMC sample was known). The result (Figure 3) allows visual comparison of the precision between assays. For all three conditions (intra-assay, inter-assay, and inter-operator precision), ELISPOT tended to have higher SD than CFC or tetramer. This trend was greatest for inter-assay and inter-operator studies.

Figure 3
figure 3

Comparison of precision across assays. ELISPOT means and SD for each sample were converted to percent of CD8+ T cells using the formula SFC/2.5 × 105/(CD8 percent of PBMC for that donor)×100. This allowed SD to be compared on the same scale. Note that the regression line for ELISPOT (dotted) is consistently higher than the regression line for CFC (dashed) or tetramer (solid). Due to complexity, data points for different donors and different antigens are not distinguished in this Figure; please refer to Figure 2 for relative responses of individual donors and antigens.

Full size image

Linearity

Figure 4 shows the results of linearity studies, in which PBMC from a CMV-responder (donor 43) were diluted into PBMC from a known CMV non-responder. This design was chosen to mimic physiological conditions of donors with few responsive cells in the context of many non-responsive cells. It was also possible to take this approach, since allogeneic responses were not detected using this particular donor pair, stimulation time, and cytokine readout (i.e., backgrounds were as low as those seen in precision experiments with the CMV-positive donor alone). All three assays showed highly significant linearity (p < 0.0001). For the pp65495–503 peptide system, CFC and tetramer showed almost identical R2 values (R2 = 0.99), while ELISPOT was lower (R2 = 0.85). This could relate to the relatively low pp65495–503 response of the chosen donor in ELISPOT versus the other two assays. For the pp65 peptide mix system, where the ELISPOT response of this donor was higher, ELISPOT and CFC showed similar R2 values (0.97–0.98).

Figure 4
figure 4

Linearity of assays. Triplicate samples of PBMC from a HLA-A2+ high-responding donor (#43) were serially diluted into PBMC from a known HLA-A2+ non-responder. The same non-responding donor was used for all assays. Unstimulated background was subtracted for each dilution point in CFC and ELISPOT assays. This background was uniformly low (< 0.08% for CFC and < 15 SFC per 2.5 × 105 PBMC for ELISPOT) despite the used of allogeneic PBMC for the dilution. Note that the pp65495–503 peptide response of this donor (#43) was much lower in ELISPOT compared to the other two assays.

Full size image

Discussion

A number of studies have been published comparing ELISPOT, CFC, and tetramer staining [15, 18–27]. However, while these studies have addressed issues such as relative sensitivity of the assays, direct comparisons of precision and linearity have not been done. The best previous assessments of inter-laboratory precision came from standardization studies carried out for ELISPOT [14, 28] or CFC [10]. The present study was designed to compare these assays with regard to precision and linearity, and in so doing, to provide target values for laboratories wishing to validate their own protocols for any of these assays. It should be noted that these targets may be specific to IFNγ and to the antigen systems used (CMV pp65495–503 and pp65 peptde mix), and can not necessarily be generalized to other cytokines and antigens.

In comparing assays, the choice of protocol is crucial; one wishes to compare fully-optimized versions of each assay. As such, we chose individual labs recognized as experts in each type of assay (tetramer, CFC, and ELISPOT) and allowed them each to use their own optimized protocol. In the case of ELISPOT, this protocol did not use costimulatory antibodies, while in CFC it did. Validation data in the lab doing ELISPOT showed that inclusion of costimulatory antibodies was not warranted based on increased backgrounds (data not shown); while similar data in the CFC lab supported the use of costimulation. The net result, that the assays might not give quantitatively equivalent results, was accepted in favour of the idea that each assay was optimally configured for its own best performance.

Often, CV has been used as a measure of precision that is independent of the units in which data is expressed (e.g., SFC or percent positive cells) [29]. However, the stability of the CV deteriorates at very low means (Figure 1), so it is difficult to use it as a robust measurement of precision for low-level responses. Since antigen-specific assays are often used to assess very low response values, we chose instead to focus on SD, which we found to be highly linear with the response mean (Figure 2).

In general, there was a trend toward higher precision in intra-assay measurements compared to inter-assay or inter-operator measurements, in agreement with a previous study on CFC [11]. Since the inter-assay and inter-operator precision studies were not repeated multiple times, we could not derive statistical tolerances around the SD values in Figure 2. However, a laboratory wishing to validate these assays could use this figure as a guide, comparing their own internal validation data to the figure for any given response level.

In order to compare precision across all three assays, ELISPOT values were converted to the same scale as CFC and tetramer staining (Figure 3), and the SD was plotted versus mean. Deriving a regression line through this data for each assay, we observed that ELISPOT SD tended to be higher, for any given response level, than CFC or tetramer SD. While we did not validate this trend statistically, it is concordant with previous studies that reported relatively low inter-laboratory precision for ELISPOT [14] and relatively high inter-laboratory precision for CFC [10], at least when data were analyzed centrally.

Why would ELISPOT assays have poorer precision in this study? One relevant factor could be the number of cells collected per data point, since assay precision is dependent upon counting statistics. The ELISPOT assay was optimized such that 2 × 105 PBMC per well gave the highest counting efficiency. CFC assays collected 40,000 CD3+CD8+ cells per sample, and tetramer assays collected 30,000 CD8+ T cells. Assuming CD3+CD8+ cells to be about 15% of PBMC, this means that roughly 30,000 CD3+CD8+ cells were present in each ELISPOT well. While the difference relative to CFC (30,000 versus 40,000) could explain some difference in precision, it is unlikely to be a major factor, at least at the higher response levels tested.

Another factor in ELISPOT assays is the fact that precision is dependent upon cell counting and pipetting. All calculation is based upon the assumption that 2 × 105 PBMC were in fact plated in every well. If the original PBMC count were in error, this would introduce a systematic bias in that assay's results that would potentially affect inter-assay or inter-operator reproducibility. And if pipetting were not precise, intra-assay reproducibility would be affected as well. In contrast, tetramer and CFC assays have a percentage readout that is less dependent upon the true number of cells plated per sample.

The ELISPOT assay is also unique in that it is a monolayer assay, and cell stimulation could be limited by the ability of T cells and APC to interact when they are spread out over a filter-bottom well. This could affect the reproducibility of stimulation, and could also explain undercounting of responsive T cells in ELISPOT relative to CFC. Lack of costimulatory antibodies in the ELISPOT assay, which were used in CFC, could also cause undercounting. Fnally, undercounting could result from coincident spots or other imprecisions in the computer-aided ELISPOT counting system. We saw such undercounting to a variable degree in this study, when ELISPOT results were converted to the same scale as CFC and tetramer results (see Table 1). ELISPOT has been shown in other studies to undercount positive events by 4- to 10-fold relative to CFC or tetramer staining [23, 24], even in the same CMV system used for the current study [15]. Thus, the ELISPOT regression line for SD versus mean (Figure 3) could be expected to shift leftward relative to CFC and tetramer, if SD for a given sample were actually equal between the three assays.

For all the above reasons, it is not surprising that ELISPOT precision was not as high as CFC and tetramer. In fact, common practice in the field takes this into account, in that ELISPOT assays are typically done with 3–6 replicates per sample, while this is rarely done in CFC and tetramer assays. It was only for purposes of fair comparison that a single well was considered as one data point in this study.

With regard to linearity, we used a dilution methodology in which responder PBMC were diluted into non-responder PBMC. There was the potential for confounding artifacts with this method, including the possibility of loss of relevant antigen-presenting cells with serial dilution, and the possibility of allogeneic responses. Fortunately, these were not observed as significant factors. Linearity of CFC and ELISPOT were similar to tetramer, except in the instance where ELISPOT response was disproportionately low compared to the CFC or tetramer response. This suggests that antigen-presenting cell frequency was not a major limiting factor. In terms of allogeneic responses, unstimulated CFC background at each dilution was 0–0.07% of CD8+ T cells, which was not different from background in the assays that did not contain allogeneic cells. For ELISPOT, the average background in the linearity studies was 4.7 SFC per 2.5 × 105 PBMC, compared to 5.0 SFC when donor 43 (high responder) PBMC were tested alone.

In terms of assay comparison, it is worth noting that the ELISPOT assay does not easily allow for phenotyping of responsive cells as CD4+ or CD8+, etc. In our study, we fortuitously chose donors whose pp65 peptide mix responses were almost entirely CD8-restricted (pp65495–503 peptide responses should be CD8-restricted by nature). We thus focused on CD8+ T cell responses for all CFC assays. However, when using either CMV pp65495–503 or peptide mix stimulation, we observed that the rank order of donors was different for ELISPOT compared to the other two assays. CD4 responses in CFC were 0.2–0.4% for all donors (data not shown), so this was not sufficient to account for the difference. One outcome of this anomaly is that the pp65495–503 linearity data for ELISPOT was generated with a donor whose ELISPOT response was many fold lower than that donor's CFC or tetramer response. We thus focused more attention on the pp65 peptide mix ELISPOT response, which was at least somewhat higher, and which showed comparable linearity to CFC and tetramer.

Tetramer staining is by far the simplest of the three assays, requiring only phenotypic staining of PBMC. It is therefore not unexpected that it would have the highest precision, as the complexity of activation and sample processing should negatively impact precision of CFC and ELISPOT. However, it is interesting that CFC precision and linearity were not much different from that of tetramer staining. This is consistent with the notion that precision of CFC assays is largely related to gating and analysis, rather than activation or processing variables [10].

Although our focus in this paper is on precision and linearity, a limit of detection for each assay can also be determined from our data. This limit could be defined, for example, as 2 SD above the mean of replicate negative control samples. Such a limit is of course dependent upon the number of events collected, as previously described for CFC [30], as well as the negative control background, which varies between donors. In our study, we calculated an average lower limit of detection of 0.08% for CD8+ T cell IFNγ production in CFC, and 64 SFC per 2.5 × 105 PBMC in IFNγ ELISPOT. Note that these detection limits were the average across three donors, and were lower for those donors with low background. A similar calculation could not be done for tetramer staining, because a negative control (irrelevant tetramer) was not used.

Conclusion

From this comparison study, we provide target values for precision and linearity of tetramer, CFC, and ELISPOT assays, using cryopreserved PBMC. We conclude that all three assays can be performed with reasonable precision and linearity. Intra-assay precision was generally lower than inter-assay or inter-operator precision. Tetramer staining tended to have the highest precision and linearity, followed closely by CFC and then ELISPOT.

Methods

Donors and CMV responses

Three HLA-A2+, CMV seropositive healthy subjects were chosen from previous work, which suggested that they represented low (~0.1% tetramer+ or IFNγ+ cells, or ~30 SFC per 2.5 × 105 PBMC), medium (~0.3% tetramer+ or IFNγ+ cells, or ~100 SFC per 2.5 × 105 PBMC), and high (~1% tetramer+ or IFNγ+ cells, or ~200 SFC per 2.5 × 105 PBMC) responders to CMV pp65495–503 peptide. Actual mean values obtained in this study are shown in Table 1.

Collection and cryopreservation of PBMC

PBMC from leukapheresis were isolated using Ficoll gradient separation as previously described [15]. To cryopreserve PBMC, 2× freezing media was first prepared, containing 20% DMSO in RPMI (Sigma Chemical Co., St. Louis, MO) containing 12.5% human serum albumin (HSA) (Gemini Bioproducts, Woodland, CA), and cooled on ice for a minimum of 30 minutes. Ficolled PBMC at 2 × 107 viable lymphocytes/ml were resuspended in cold RPMI+12.5% HSA with no DMSO. An equal volume of chilled 2× freezing media was added to the cell suspension dropwise, while gently swirling the tube. One ml of this cell suspension was aliquoted into each cryovial (Sarstedt, Inc., Newton, NC). Once aliquoted, cryovials were placed on ice and then transferred into a freezing container (Nalgene, Rochester, NY), and stored at -80°C for 24 hours. Cryovials were then transferred into liquid nitrogen for long-term storage. After 30 days, cryovials were overnight shipped on dry ice to the recipient laboratories.

Thawing

Cryopreserved PBMC were stored at -80°C until thawing to set up the assays. Cryopreserved cells were thawed and slowly diluted with 8 ml of warm RPMI+10% fetal bovine serum+antibiotics (cRPMI-10, all components from Sigma). The cells were centrifuged for 7 minutes at 250 × G, then resuspended as described below for each assay. Viability and recovery were checked using Trypan blue, and were > 80% and > 50%, respectively, in all samples.

Antigens

A common source of peptide antigens was shared among laboratories for CFC and ELISPOT assays. These included CMV pp65 peptide mix (BD Biosciences, San Jose, CA; used at a final concentration of 1.7 μg/ml/peptide); and CMV pp65495–503 peptide (SynPep Corp., Dublin, CA; used at a final concentration of 10 μg/ml).

Tetramer assays

Staining was done using the Multiple Antibody Single Color protocol (iMASC, Beckman Coulter Inc., Fullerton, CA) as previously described [15]. Briefly, PBMC were resuspended at 1 × 107 per mL in HBSS+0.1% bovine serum albumin+0.02% sodium azide. 100 μL of PBMC per sample were stained for 30 minutes at room temperature using CD8 FITC, HLA-A2 tetramer loaded with CMV pp65495–503 PE, and CD4, CD13, and CD19 PE-Cy5 (all from Beckman Coulter). Samples were washed and analyzed on a FACS Calibur flow cytometer (BD Biosciences). 30,000 CD8+ T cells were collected, and results were reported as the percentage of CD8+, PE-Cy5-negative cells that were tetramer-positive. No correction for background staining with irrelevant tetramer was used, but the test tetramer showed undetectable staining on CMV-negative donors (not shown).

CFC assays

Activation and processing were done as previously described [15]. Briefly, PBMC were resuspended at 1 × 107 per mL in cRPMI-10, and 200 μL were plated per well in 96-well round-bottom plates. After overnight rest at 37°C, activation reagents (stimulus+brefeldin A+costimulatory antibodies to CD28 and CD49d) were added and the cells incubated at 37°C for 6 hours. The cells were then fixed and permeabilized as per reference [15], followed by staining with IFNγ FITC/CD69 PE/CD8 PerCPCy5.5/CD3 APC (BD Biosciences) for 1 hour at room temperature in dark. Plates were washed and cells resuspended in 1% paraformaldehyde in PBS, then acquired on a FACSCalibur flow cytometer (BD Biosciences). 40,000 CD3+CD8+ lymphocytes were collected per sample. Data were reported as the net percent of CD3+CD8+lymphocytes that were CD69+IFNγ+ after subtracting the average response of unstimulated samples.

ELISPOT assays

Activation and processing were done largely as previously described [15]. Briefly, Multiscreen-HA 96-well plates (Millipore, Bedford, MA) were coated with mouse anti-human IFNγ mAb, and the plates washed and blocked as per the referenced publication. PBMC were resusupended at 1 × 106 per mL, and 200 μL plated per well along with the appropriate antigen. Stimulation was for 18–24 hours at 37°C in 5% CO2. Plates were washed and developed as in reference [15], and the number of spots per well was determined using a KS ELISPOT Automated Reader System with KS ELISPOT 4.2 Software (Carl Zeiss, Inc., Thornwood, NY). SFC per 2.5 × 105 PBMC were reported after subtracting the average response of unstimulated samples.

Design of precision and linearity studies

Data for each assay were acquired on a single instrument, using the same settings throughout the study. For flow cytometry-based assays, an optimized gating template and instrument settings file were used throughout the study, including inter-operator precision studies. All data for linearity, intra-assay, and inter-assay precision were performed by a single operator. For conversion of ELISPOT data to percent of CD3+CD8+ cells, the percent of CD3+CD8+ cells was determined from a representative subset of the CFC data, and was found to be very consistent across days and across operators. However, it varied significantly from one donor to the next (see Table 1). The ELISPOT data was converted to percent response as follows: SFC/2.5 × 105/(mean CD3+CD8+ percent for that donor)×100.

Statistical analyses

For comparison of SD among samples in intra-assay versus inter-assay or inter-operator studies, a Wilcoxon Matched Pairs test was used. Statistical analyses were carried out in GraphPad Prism software (San Diego, CA).

References

  1. Owens MA, Vall HG, Hurley AA, Wormsley SB: Validation and quality control of immunophenotyping in clinical flow cytometry. J Immunol Methods. 2000, 243: 33-50. 10.1016/S0022-1759(00)00226-X.

    Article  CAS  PubMed  Google Scholar 

  2. Borg L, Kristiansen J, Christensen JM, Jepsen KF, Poulsen LK: Evaluation of accuracy and uncertainty of ELISA assays for the determination of interleukin-4, interleukin-5, interferon-gamma and tumor necrosis factor-alpha. Clin Chem Lab Med. 2002, 40: 509-19. 10.1515/CCLM.2002.089.

    Article  CAS  PubMed  Google Scholar 

  3. Findlay JW, Smith WC, Lee JW, Nordblom GD, Das I, DeSilva BS, Khan MN, Bowsher RR: Validation of immunoassays for bioanalysis: a pharmaceutical industry perspective. J Pharm Biomed Anal. 2000, 21: 1249-73. 10.1016/S0731-7085(99)00244-7.

    Article  CAS  PubMed  Google Scholar 

  4. Altman JD, Moss PAH, Goulder PJR, Barouch DH, McHeyzer-Williams MG, Bell JI, McMichael AJ, Davis MM: Phenotypic analysis of antigen-specific T lymphocytes. Science. 1996, 274: 94-6. 10.1126/science.274.5284.94.

    Article  CAS  PubMed  Google Scholar 

  5. Suni MA, Picker LJ, Maino VC: Detection of antigen-specific T cell cytokine expression in whole blood by flow cytometry. J Immunol Methods. 1998, 212: 89-98. 10.1016/S0022-1759(98)00004-0.

    Article  CAS  PubMed  Google Scholar 

  6. Waldrop SL, Pitcher CJ, Peterson DM, Maino VC, Picker LJ: Determination of antigen-specific memory/effector CD4+ T cell frequencies by flow cytometry: evidence for a novel, antigen-specific homeostatic mechanism in HIV-associated immunodeficiency. J Clin Invest. 1997, 99: 1739-50. 10.1172/JCI119338.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  7. Czerkinsky C, Andersson G, Ekre HP, Nilsson LA, Klareskog L, Ouchterlony O: Reverse ELISPOT assay for clonal analysis of cytokine production. I. Enumeration of gamma-interferon-secreting cells. J Immunol Methods. 1988, 110: 29-36. 10.1016/0022-1759(88)90079-8.

    Article  CAS  PubMed  Google Scholar 

  8. Lathey J: Preliminary steps toward validating a clinical bioassay: case study of the ELISpot assay. Biopharm Intl. 2003, 42-50.

    Google Scholar 

  9. Lathey J, Sathiyaseelan J, Matijevic M, Hedley ML: Validation of pretrial ELISspot measurements. BioProcess Intl. 2003, 34-41.

    Google Scholar 

  10. Maecker HT, Rinfret A, D'Souza P, Darden J, Roig E, Landry C, Hayes P, Birungi J, Anzala O, Garcia M, et al.: Standardization of cytokine flow cytometry assays. BMC Immunol. 2005, 6: 13-10.1186/1471-2172-6-13.

    Article  PubMed Central  PubMed  Google Scholar 

  11. Nomura LE, Walker JM, Maecker HT: Optimization of whole blood antigen-specific cytokine assays for CD4(+) T cells. Cytometry. 2000, 40: 60-68. 10.1002/(SICI)1097-0320(20000501)40:1<60::AID-CYTO8>3.0.CO;2-J.

    Article  CAS  PubMed  Google Scholar 

  12. Maino VC, Suni MA, Wormsley SB, Carlo DJ, Wallace MR, Moss RB: Enhancement of HIV type 1 antigen-specific CD4+ T cell memory in subjects with chronic HIV type 1 infection receiving an HIV type 1 immunogen. AIDS Res Hum Retroviruses. 2000, 16: 539-47. 10.1089/088922200308954.

    Article  CAS  PubMed  Google Scholar 

  13. Smith JW, Walker EB, Fox BA, Haley D, Wisner KP, Doran T, Fisher B, Justice L, Wood W, Vetto J, et al.: Adjuvant immunization of HLA-A2-positive melanoma patients with a modified gp100 peptide induces peptide-specific CD8+ T-cell responses. J Clin Oncol. 2003, 21: 1562-73. 10.1200/JCO.2003.09.020.

    Article  CAS  PubMed  Google Scholar 

  14. Cox JH, Ferrari G, Kalams SA, Lopaczynski W, Oden N, D'Souza M P: Results of an ELISPOT proficiency panel conducted in 11 laboratories participating in international human immunodeficiency virus type 1 vaccine trials. AIDS Res Hum Retroviruses. 2005, 21: 68-81. 10.1089/aid.2005.21.68.

    Article  CAS  PubMed  Google Scholar 

  15. Maecker HT, Moon J, Bhatia S, Ghanekar SA, Maino VC, Payne JK, Kuus-Reichel K, Chang JC, Summers A, Clay TM, et al.: Impact of cryopreservation on tetramer, cytokine flow cytometry, and ELISPOT. BMC Immunol. 2005, 6: 17-10.1186/1471-2172-6-17.

    Article  PubMed Central  PubMed  Google Scholar 

  16. Kern F, Surel IP, Brock C, Freistedt B, Radtke H, Scheffold A, Blasczyk R, Reinke P, Schneider-Mergener J, Radbruch A, et al.: T-cell epitope mapping by flow cytometry. Nat Med. 1998, 4: 975-8. 10.1038/nm0898-975.

    Article  CAS  PubMed  Google Scholar 

  17. Kern F, Bunde T, Faulhaber N, Kiecker F, Khatamzas E, Rudawski IM, Pruss A, Gratama JW, Volkmer-Engert R, Ewert R, et al.: Cytomegalovirus (CMV) phosphoprotein 65 makes a large contribution to shaping the T cell repertoire in CMV-exposed individuals. J Infect Dis. 2002, 185: 1709-16. 10.1086/340637.

    Article  CAS  PubMed  Google Scholar 

  18. Asemissen AM, Nagorsen D, Keilholz U, Letsch A, Schmittel A, Thiel E, Scheibenbogen C: Flow cytometric determination of intracellular or secreted IFNgamma for the quantification of antigen reactive T cells. J Immunol Methods. 2001, 251: 101-8. 10.1016/S0022-1759(01)00312-X.

    Article  CAS  PubMed  Google Scholar 

  19. Heeger PS, Greenspan NS, Kuhlenschmidt S, Dejelo C, Hricik DE, Schulak JA, Tary-Lehmann M: Pretransplant frequency of donor-specific, IFN-gamma-producing lymphocytes is a manifestation of immunologic memory and correlates with the risk of posttransplant rejection episodes. J Immunol. 1999, 163: 2267-75.

    CAS  PubMed  Google Scholar 

  20. Helms T, Boehm BO, Asaad RJ, Trezza RP, Lehmann PV, Tary-Lehmann M: Direct visualization of cytokine-producing recall antigen-specific CD4 memory T cells in healthy individuals and HIV patients. J Immunol. 2000, 164: 3723-32.

    Article  CAS  PubMed  Google Scholar 

  21. Hobeika AC, Morse MA, Osada T, Ghanayem M, Niedzwiecki D, Barrier R, Lyerly HK, Clay TM: Enumerating antigen-specific T-cell responses in peripheral blood: a comparison of peptide MHC Tetramer, ELISpot, and intracellular cytokine analysis. J Immunother. 2005, 28: 63-72. 10.1097/00002371-200501000-00008.

    Article  CAS  PubMed  Google Scholar 

  22. Karlsson AC, Martin JN, Younger SR, Bredt BM, Epling L, Ronquillo R, Varma A, Deeks SG, McCune JM, Nixon DF, et al.: Comparison of the ELISPOT and cytokine flow cytometry assays for the enumeration of antigen-specific T cells. J Immunol Methods. 2003, 283: 141-53. 10.1016/j.jim.2003.09.001.

    Article  CAS  PubMed  Google Scholar 

  23. Kuzushima K, Hoshino Y, Fujii K, Yokoyama N, Fujita M, Kiyono T, Kimura H, Morishima T, Morishima Y, Tsurumi T: Rapid determination of Epstein-Barr virus-specific CD8(+) T-cell frequencies by flow cytometry. Blood. 1999, 94: 3094-100.

    CAS  PubMed  Google Scholar 

  24. Moretto WJ, Drohan LA, Nixon DF: Rapid quantification of SIV-specific CD8 T cell responses with recombinant vaccinia virus ELISPOT or cytokine flow cytometry. Aids. 2000, 14: 2625-7. 10.1097/00002030-200011100-00034.

    Article  CAS  PubMed  Google Scholar 

  25. Pahar B, Li J, Rourke T, Miller CJ, McChesney MB: Detection of antigen-specific T cell interferon gamma expression by ELISPOT and cytokine flow cytometry assays in rhesus macaques. J Immunol Methods. 2003, 282: 103-15. 10.1016/j.jim.2003.08.003.

    Article  CAS  PubMed  Google Scholar 

  26. Sun Y, Iglesias E, Samri A, Kamkamidze G, Decoville T, Carcelain G, Autran B: A systematic comparison of methods to measure HIV-1 specific CD8 T cells. J Immunol Methods. 2003, 272: 23-34. 10.1016/S0022-1759(02)00328-9.

    Article  CAS  PubMed  Google Scholar 

  27. Whiteside TL, Zhao Y, Tsukishiro T, Elder EM, Gooding W, Baar J: Enzyme-linked immunospot, cytokine flow cytometry, and tetramers in the detection of T-cell responses to a dendritic cell-based multipeptide vaccine in patients with melanoma. Clin Cancer Res. 2003, 9: 641-9.

    CAS  PubMed  Google Scholar 

  28. Samri A, Durier C, Urrutia A, Sanchez I, Gahery-Segard H, Imbart S, Sinet M, Tartour E, Aboulker JP, Autran B, et al.: Evaluation of the interlaboratory concordance in quantification of human immunodeficiency virus-specific T cells with a gamma interferon enzyme-linked immunospot assay. Clin Vaccine Immunol. 2006, 13: 684-97. 10.1128/CVI.00387-05.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  29. Motulsky H: Intuitive Biostatistics. 1995, Oxford: Oxford Univ. Press, 29-

    Google Scholar 

  30. Maecker HT: The role of immune monitoring in evaluating cancer immunotherapy. Cancer Drug Discovery and Development: Immunotherapy of Cancer. Edited by: Disis ML. 2005, Totowa, NJ: Humana Press, 59-71.

    Google Scholar 

Download references

Acknowledgements

This work was supported by grant #U54 CA090818 from the National Cancer Institute, National Institutes of Health.

Author information

Authors and Affiliations

  1. BD Biosciences, San Jose, CA, USA

    Holden T Maecker, Sonny Bhatia, Smita A Ghanekar & Vernon C Maino

  2. Cancer Research and Biostatistics, Seattle, WA, USA

    Jeffrey Hassler

  3. Beckman-Coulter, San Diego, CA, USA

    Janice K Payne

  4. Departments of Surgery, Medicine, Pathology, and Immunology, and Duke Comprehensive Cancer Center, Duke University Medical Center, Durham, NC, USA

    Amanda Summers, Karrie Comatas, Manar Ghanayem, Michael A Morse, Timothy M Clay & Herbert K Lyerly

  5. Tumor Vaccine Group, Division of Oncology, University of Washington, Seattle, WA, USA

    Corazon delaRosa & Mary L Disis

Authors
  1. Holden T Maecker
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jeffrey Hassler
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Janice K Payne
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Amanda Summers
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Karrie Comatas
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Manar Ghanayem
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Michael A Morse
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Timothy M Clay
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Herbert K Lyerly
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Sonny Bhatia
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Smita A Ghanekar
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Vernon C Maino
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Corazon delaRosa
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Mary L Disis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Holden T Maecker.

Additional information

Authors' contributions

HTM, JH, TMC, HKL, SAG, VCM, and MLD designed and supervised the study. JKP, AS, KC, MG, MAM, SB, and CdR obtained patient samples and collected the data. JH collated data and did statistical analyses. HTM drafted the manuscript and figures, and all authors edited and/or approved the final version.

Authors’ original submitted files for images

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Maecker, H.T., Hassler, J., Payne, J.K. et al. Precision and linearity targets for validation of an IFNγ ELISPOT, cytokine flow cytometry, and tetramer assay using CMV peptides. BMC Immunol 9, 9 (2008). https://doi.org/10.1186/1471-2172-9-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/1471-2172-9-9

Keywords

BMC Immunology

ISSN: 1471-2172

Contact us