Introduction

An estimated 2.1 million children were living with HIV in 2007, 90% of whom were in Sub-Saharan Africa [1]. Combination antiretroviral therapy (ART) has transformed pediatric HIV from a disease with 50% mortality before 2 years of age into a chronic illness in high-income settings [2, 3] and increasingly in the developing world [4].

Adherence to therapy is vital to preventing resistance and maintaining the effectiveness of ART regimens [5]. Adherence to ART is a challenge, particularly for children [6]. Children rely on a caregiver, who often is also infected, to administer medication. Syrups are difficult to measure and frequently unpalatable. These challenges may be exacerbated in resource-limited settings (RLS) that are most affected by the HIV epidemic. Often elderly family members have to care for children whose biological parents have passed away, and end up being burdened with multiple dependents while being frail themselves. Extreme financial insecurities force caregivers to work in multiple temporary employments, making it difficult to stick to a consistently managed medication schedule. The pressures of guaranteeing day-to-day survival in poor areas may overshadow the long-term goal of adherence to medication. With this in mind, adherence is even more crucial in most parts of the developing world, where only one or two ART regimens are available, leaving limited or no options once resistance has developed.

Although there is literature surrounding pediatric adherence in the developed world [7–9], fewer data exist from sub-Saharan African settings [10]. Data from Cape Town suggest that adherence, as measured by Medication Event Monitoring System (MEMS), may be as good or better than in the developed world [11]. In other African countries adherence was similarly good, however these studies primarily used caregiver report or pill count as adherence measures [12–14]. A heterogeneous array of pediatric adherence measures and metrics has been used in RLS, leading to incomparable study outcomes [10].

Although viral load is often used as a surrogate marker of good adherence, viral load data is not always available in RLS, and some common regimens, especially those containing protease inhibitors, can be more “forgiving” of poor adherence to the extent that VL may still be suppressed in the face of poor short-term adherence [15–17]. While, in adults, pharmacy refill data seems to correlate well with virologic suppression and time-to-death in African settings [18, 19], this measure has not been evaluated for liquid formulations. Of all adherence measures regularly used, MEMS appear to provide the closest correlation with viral load [20] and was used as the reference method for measuring adherence in this study. MEMS has been shown to be a more accurate adherence measure than self-report and provides the exact time that the cap was removed [7], provided that it is used correctly; i.e. closed promptly after removal without periods of it being kept open.

There have been few comparisons of adherence measures among children [21] and almost none in sub-Saharan Africa. Self-report is commonly used, and although readily available and inexpensive, its validity has often been questioned [21]. Objective measures like MEMS, on the other hand, are either not readily available or simply prohibitively expensive to be used outside of well-resourced research settings [11].

To our knowledge, there has been no cost analysis of adherence measures for children. With the ongoing expansion of ART programs, an accurate, cost-effective method for measuring adherence among children in RLS is needed. Adherence needs to be closely monitored to detect early non-adherence and prevent the subsequent development of resistance—even more so in settings with limited treatment options.

This study compared four readily available measures of pediatric adherence to MEMS in order to assess adherence behavior and find the most accurate and feasible method for a RLS. It aims to identify adequate methods of adherence measurement for ongoing and future pediatric ART programs and program-related research.

Methods

Setting

The study took place at the Hannan Crusaid Treatment Center (HCTC) in the Nyanga district of Cape Town, which has approximately 300,000 inhabitants and a 28% antenatal HIV seroprevalence [22]. When meeting the criteria to start ART (recurrent hospitalizations for HIV-related disease, modified WHO stage II or III disease, CD4 < 20% or CD4 < 15% in children under or over 18 months old respectively [23]), most children are commenced on non-nucleoside reverse transcriptase inhibitor-based regimens, usually stavudine (d4T), lamivudine (3TC) and efavirenz (EFV) or nevirapine (NVP). A lopinavir/ritonavir-based therapy (LPV/r) is offered for those who had NNRTI-based Prevention of Mother-To-Child-Transmission Therapy (PMTCT), fail first line treatment or are on Rifamycin-based antituberculous treatment and are below 3 years of age. All regimens, with the exception of EFV, are administered twice daily. Routine follow-up is monthly to month 4 and then every 4 months. VL and CD4 counts are obtained every 4 months. At the time of this study, 185 children and 1750 adults were receiving ART at the center.

Adherence support is a central component of pediatric HIV care. Caregivers attend three group “treatment readiness” sessions in order to ensure treatment preparedness. Trained therapeutic counselors are responsible for up to 120 patients each. Home visits occur at set intervals, with frequency decreasing over time and with success on therapy.

Design

We conducted a 1-month longitudinal study, which was approved by the Health Sciences Research Ethics Committee of the University of Cape Town. Children and caregivers who attended the clinic during July and August 2007 received information about the study in their home language and were enrolled after written informed consent. The caregiver contact was managed by a research-trained nurse in the preferred language of the caregiver. This nurse did not belong to the regular clinic staff, in order to minimize the risk of obtaining answers influenced by social desirability or fear of negative treatment repercussions.

Sample

In June and July 2007, 55 eligible caregiver-child dyads were approached to participate in the study. Inclusion criteria were child age less than 7 years (to focus on the specific adherence challenges in this young age group which makes up 69% of the clinic’s pediatric patients), having one designated caregiver with appropriate cognitive ability who consented to be enrolled, and receiving either lamivudine or abacavir in syrup form (as all other liquid antiretrovirals required large volumes and did not fit into one MEMS bottle).

Measurements

Clinical histories were collected from the medical records. These included age, ART regimen and date of initiation, WHO stage at treatment initiation, changes to ART regimen, and most recent CD4 percentages and VL (within 6 months of enrolment). Caregivers completed a basic demographic questionnaire at the enrollment visit, recording caregiver age, gender, ethnic and language background (black, “coloured” or white South African, primary language at home), relationship to child (biological mother/father/grandmother, extended family or other) and ART status.

We used five adherence measures, of which four could routinely be used in clinical settings with limited resources: MEMS as the method of reference; Visual Analog Scales (VAS); 3 Day Recall of missed doses (3DR); Pharmacy Refill data (PR); and measurement of returned syrups (RS). Adherence was described as a continuous variable as well as a binary variable with a 95% cut-off, based on current literature [5]. The next routine viral load and CD4 percentage taken post-study were also recorded.

Medication Event Monitoring System (MEMS)

Each participant was equipped with a MEMS cap (Aardex Corp., Zug, Switzerland) for a 1-month period. The cap records time and date of all bottle-opening events as presumptive doses taken by the patient; the data is stored in the chip until downloaded onto a computer. The cap was fitted on one of the three liquid antiretroviral (ARV) drugs. Decanting of this drug into the MEMS-equipped bottle was performed at the clinic pharmacy when patients filled their monthly prescriptions. Caregivers received information on the correct handling and were informed that their adherence would be monitored by MEMS. The specific mechanism was not revealed to avoid deliberate cap opening without drug administration.

At the follow-up visit caregivers completed a questionnaire about their MEMS handling habits, based on the tool used by Deschamps et al. [24]. Data were downloaded using PowerView 3.3 (Aardex Corp., Zug, Switzerland), reviewed manually, compared to the handling questionnaire, and adjusted if necessary. For example, days with reported non-use of MEMS (but with ART from another container) were excluded from the analysis as were times when the MEMS-monitored drug was not immediately administered after dispensing from the bottle. If two opening events occurred within an hour, the latter was excluded (as introduced by Bova et al. [25]). Caregivers who reported contemporaneous use of the MEMS-monitored ARV from an additional medication bottle were excluded from the analysis. MEMS has recently been used in a similar population in Cape Town where it was significantly correlated to virologic outcome [11]. Adherence was defined as:

$$ {\frac{{{\text{doses\,prescribed }} - {\text{ doses \,missed}}}}{\text{doses \,prescribed}}} \, \times 100 $$

Adherence descriptions created were

  1. (i)

    The overall percentage of doses taken during the 4 week monitoring interval (overallMEMS) and

  2. (ii)

    The percentage of doses that were taken at the prescribed 12-h intervals during the whole monitoring period (with a 2-h window allowed: 1 h before and 1 h after the target dose time; timedMEMS).

Visual Analogue Scale (VAS)

At the 1-month follow-up visit and following the MEMS monitoring period, all caregivers were asked to rate their adherence on a Visual Analogue Scale for the previous month, ranging from 0 to 100%, in bands of 10%, marked on a vertical line. The scale was administered by the research assistant and available in English and isiXhosa. VAS has shown valid results in adults in Uganda [26], however it seemed less reliable in a pediatric cohort in Cape Town [11].

Caregiver 3-Day Recall (3DR)

At the 1-month follow-up visit after the MEMS monitoring, caregivers were also asked about their medication-administering behavior over the past 3 days. For this information, an adapted and translated version of the recall questionnaire developed by the Pediatric AIDS Clinical Trials Group (PACTG) [27] was used. Our version in isiXhosa recorded information on the number of doses missed for each ARV drug over the past 3 days and the reasons for missing them. To our knowledge, the PACTG had not been used in resource-limited settings before. Data were expressed both in comparison to the MEMS reports and as a percentage of adherence calculated:

$$ {\frac{{{\text{doses\, prescribed }} - {\text{ doses \,missed}}}}{\text{doses \,prescribed}}} \, \times 100 $$

Pharmacy Refill (PR)

For all participants, data on previously dispensed medication was collected for the 4 months preceding enrollment (thus expecting 5 dispenses) using the clinic pharmacy’s Intelligent Dispensing of ART software system (iDart, open source software developed by Cell-Life: www.cell-life.org) and medical records. We recorded both the number of times that medication was dispensed and the interval between dispensing visits in days. All children received their ART at the clinic’s in-house pharmacy. There were no other points of ART provision for our patients and ART was dispensed once a month. Unused medication was returned to be measured by the adherence counselors and then disposed. There were no stock-outs during the monitored period. Adherence was calculated as:

$$ {\frac{\text{occasions \,when \,medication\, was\, dispensed}}{\text{occasions \,when\, medication\, was\, supposed \,to \,be \,dispensed}}} \, \times 100 { } $$

Nachega and colleagues demonstrated that pharmacy refill-measured adherence predicted survival times in a cohort of adults in Soweto, South Africa [18].

Measurement of Returned Syrups (RS)

Patients were dispensed originally packaged medication containers, even if these exceed their amount of medication needed for the subsequent month. Caregivers were asked to return all of the previously dispensed medication containers and bottles to assess the amount of medication left. Adherence counselors were trained to measure the amount of RS using a measuring cylinder. Adherence was calculated as:

$$ {\frac{{\left( {{\text{amount \,of\, syrup \,dispensed}} - {\text{amount \,of\, syrup \,returned}}} \right)\,{\text{in \,ml}}}}{\text{amount\, of \,syrup \,dispensed \,in \,ml}}} \, \times 100 \, $$

This method has been used in conjunction with pill count in another South African study, where adherence was associated with virologic outcome [28].

Treatment Outcome

At the study site, CD4% and VL specimens were routinely collected every 4 months. Since funding restraints did not allow for us to perform study-specific blood tests, we collected the latest CD4% and VL results before study enrolment as well as the earliest ones after finishing the study from the patient records. HIV RNA was measured using the Bayer HIV-1 RNA 3.0 assay. Children were defined as having achieved virologic suppression if there were fewer than 50 copies of HIV RNA/ml in the blood sample (HIV RNA log <1.69).

Cost Calculation

Costing of the adherence measures used an ingredients-based methodology in order to most accurately assess the cost of the measures [29]. This involves measuring the utilization of person-time and goods (the ingredients) required to perform the task, and the cost of each ingredient. For each adherence measure all tasks performed by staff members over and above standard practice in the absence of adherence measurement were determined. Measurement of returned syrups was excluded from the costing analysis for logistical reasons—the research assistant could not measure more than one action at the time. For each task, the research assistant observed a consecutive convenience sample of events, recording the time taken to perform the task and all consumables and other resources used. The costs of staff time were based on the annual cost of employment of the staff grade performing the task, divided by the number of minutes they were expected to work in each year. The costs of other resources were based on commercial prices and the costs of durable items were depreciated across their expected lifetime. It was assumed that each patient was measured at each of their three scheduled visits to the clinic annually and where items were shared across all patients (i.e. the MEMS USB communicator and software) the cost per patient was based on the total clinic population at the end of August 2008 (2,304). Overhead costs of building infrastructure and program management were not included in this costing since these would be very similar across all adherence measures and comprise a very small proportion of the total cost of this clinic [30]. The total annual cost of each adherence measure was calculated by summing the annual costs for all ingredients. All costs were converted to US dollars at the average exchange rate for the South African Rand and Euro in 2007 (http://www.oanda.com/convert/fxhistory). In the absence of any gold standard against which to evaluate effectiveness, no attempt to conduct a cost-effectiveness analysis was made, and a cost comparison was performed instead.

Statistical Analyses

Descriptive statistics were used to analyze children’s and caregivers’ characteristics and variables related to treatment outcome. Normally distributed continuous variables were described using means and standard deviations (SD); non-normally distributed ones using medians and interquartile ranges (IQR). Discrete data were described using counts and percentages. To express adherence as a binary variable, the defined cut-off value was at 95% of the continuous data for all measures. Patients above this cut-off were defined as adherent, patients below as non-adherent. We used Pearson correlations to compare adherence measures and linear regression to assess the associations between adherence and treatment outcome. The performance criteria for all adherence measures were calculated with two-by-two tables and receiver operating characteristic curves. The level of significance for all analyses was P < 0.05. All statistical analyses were performed using STATA 9.0 (StataCorp, College Station, TX).

Results

Characteristics of Caregivers and Children

Two caregivers declined participation such that 53 caregiver-child dyads were enrolled. These 53 dyads constituted 56% of the 95 children coming in for a physician visit during these 2 months, and 29% of all pediatric patients. The median age of children was 3.7 years (IQR, 3–7) and 53% were female (n = 28). They had been on ART for a median time of 20 months (IQR, 15–32). Most children had started ART with late stage HIV-disease—WHO stage 3 or 4 (n = 37, 71%)—and were receiving first-line ART (n = 50, 94%; regimen details see Table 1). The median VL was <50 copies/ml both before and after the study (IQR 50 for both, P < 0.5; for number of children with undetectable viral load see Table 1). There was no significant change in CD4 count pre-and post-study (t = −3.4, P = 0.9; Table 1). CD4% were available for all children at enrollment and follow-up visits and the virologic data for all but one child at follow-up due to insufficient sample for analysis. MEMS adherence data was available for 46 children, VAS and RS data for 48, 3DR data for 49 and PR for all 53.

Table 1 Demographic characteristics of the children
Full size table

Of the caregivers, 40 were the children’s biological mothers (75%), eight were grandmothers (15%), two each were fathers and aunts (4%), and one was an older sibling (2%). The median age of caregivers was 30 years (IQR, 26–37), and 33% (n = 17) were enrolled on ART at the center themselves. All caregivers spoke isiXhosa as their first language.

Adherence

Medication Event Monitoring System

MEMS cap data were available for 46 children with a mean monitoring period of 29 days (SD 4.8). For 7 children, MEMS data were unavailable because of reported loss of cap (n = 2), of cap non-return due to a different caregiver at follow-up (n = 3) and because caregivers had refilled the MEMS syrup into a smaller container for travel (n = 2). In six cases (13%) data were adjusted as described above. The median percentage of doses taken during the monitoring period was 92% (overallMEMS; IQR, 85–99; Table 2). Only 66% of doses (IQR, 50–79) were administered within the recommended 12-h intervals (timedMEMS). When MEMS was adjusted for correct timing, it showed that 73% of children (n = 33) had adherence levels of less than 80%. Adherence rates by all measures did not differ between children with and without MEMS data.

Table 2 Reported adherence and groups of above 95% and below 80% adherence, by measure
Full size table

Visual Analogue Scale

Medication-administering adherence as self-rated by caregivers on the VAS was 100% for all participants (Table 2).

Three Day Recall

Ninety-two percent of caregivers (n = 45) reported not having missed any dose in the 3 days prior to the follow-up visit. One caregiver reported to have missed one dose, one had missed 3 doses, one 4 doses and one 5 doses. The mean number of missed doses for all children over the whole monitoring period was 0.3 (SD, 1.0). Median 3DR adherence was 100% (IQR, 100; Table 2).

Pharmacy Refill

Median adherence by pharmacy records was 100% (IQR, 100). The expected interval between refill visits was 28 days, and 24 caregivers (45%) adhered to their scheduled visits. Seven caregivers (13%) were 1 day late, three caregivers (6%) were 3 days late and five caregivers (9%) were more than 4 days late. Thirteen caregivers (25%) picked up their refills ahead of the scheduled time (Table 2).

Measurement of Returned Syrups

In 32 children (67%), RS showed an adherence rate greater than 100%. Adherence for the MEMS-monitored ARV drug was 104% (IQR, 98–116), 100% (IQR, 93–101) for the second ARV drug and 101% (IQR, 99–107) for the third ARV drug (Table 2).

Adherence as a Categorical Variable

Table 2 shows the classification of patients into adherence groups for each adherence measure. MEMS was the only measure that reported median adherence rates below 100%, and the only method that detected substantial numbers of caregivers with low adherence rates. Only 4 caregivers (9%) were >95% adherent with regards to correct timing of doses within the 12-hourly interval.

Comparison of Adherence Measures

Correlations Between Measures

Both MEMS measures were highly correlated with one another (r = 0.80, P < 0.05). RS and 3DR both correlated with VAS, which correlated with timedMEMS (Table 3).

Table 3 Correlation between adherence measures, rho correlation coefficients
Full size table

3-Day Recall and MEMS

The number of doses that caregivers reportedly missed by 3DR was compared with the MEMS data for these days (n = 46 due to the smaller group of children with MEMS data; MEMS data was available for all caregivers who reported less than 100% adherence). While 85% of the caregivers had reported not to have missed any dose, this was confirmed by MEMS in only 43% of cases. MEMS consistently showed higher rates of missed doses than 3DR caregiver report; 17% for one missed dose (2% by 3DR), 9% for 2 missed doses (2% by 3DR), and 15% for more than 2 missed doses (0% by 3DR).

MEMS and Treatment Outcome

MEMS adherence, adjusted for the correct timing of doses and at a cut-off of 95%, showed a specificity of 100% in predicting virologic suppression (AUC 0.57; sensitivity 13%). For unadjusted MEMS, sensitivity was 27%, specificity 47% and AUC 0.43.

Composite Measures of Adherence

In an attempt to find an easy and cost-effective measure of adherence, the value of the single measures as well as a combined measure of adherence was assessed by creating receiver-operating characteristics (ROC) curves, using MEMS adherence greater than 95% as the reference. For each combination of measures, an additive score was developed (e.g. VAS adherence + 3DR adherence). These are shown in Table 4. The largest AUC was reached by a combination of 3DR and VAS, the two subjective caregiver-based measures (AUC = 0.62). The difference in AUCs did not reach statistical significance for any of the additive scores (P > 0.05 for all). Using a combination of three measures did not result in larger AUCs, nor did it show statistical significance (P > 0.05 for all).

Table 4 Sensitivity, specificity, positive and negative predictive value and area under curve for all adherence measures in predicting MEMS adherence >95%
Full size table

Costing of Four Methods of Adherence Measurement

As shown in Table 5, the MEMS adherence measure was considerably more expensive than any of the other methods. This was primarily due to the purchasing cost every 3 years ($120 per cap/bottle unit; the battery life limits the length of use) but also due to the additional expected input from the pharmacist (annual wage $37,341) and pharmacist assistant (annual wage $8,487) in managing the adherence measurement process. The average length of time taken by a pharmacist assistant to measure adherence in the PR method explains the higher costs for this method compared to the VAS and PACTG questionnaire methods, which can be administered by a counselor (annual wage $4,532).

Table 5 Annual per patient costs of adherence monitoring methods
Full size table

Discussion

In this cohort of children on ART we sought to determine the most practical and cost-effective method of measuring adherence in a RLS. MEMS was used as the reference method against which two subjective and two objective adherence assessments were compared. Of note, adherence as measured by MEMS (adjusted for dosing interval) in this study correlated with virologic suppression, which remained unchanged pre-and post-study.

Overall adherence by all measures was excellent: both subjective measures (VAS and 3DR) showed a median adherence of 100%. Overestimation of adherence is a common issue with subjective measures, which is why they are often not considered useful alone [31]. Objective measures (PR and SR) however, had similarly high median adherence values in this study. All of the subjective and objective measures appeared to overestimate adherence when compared to MEMS.

The adherence rates in this young age group are similar to findings in older children [7], except for PR adherence, which was higher in our cohort. It has recently been shown that caregiver motivation and knowledge of ART are important predictors of adherence [32],

The fact that adherence rates were not lower in our very young cohort, where medication administration was further complicated by liquid formulations, suggests that motivational factors are of higher predictive importance than regimen complexity.

MEMS data showed that the timing of doses was often deviant from prescription—a finding that no other adherence measure could detect. Unfortunately, at an annual cost of $57 per patient including equipment and staff time, MEMS caps are not affordable in most RLS. While both subjective measures are easy to use and low cost at less than $0.40 per patients per annum per measure, neither correlated significantly with overall MEMS adherence. Only VAS showed significant correlation with timedMEMS—the correlation, however, was weak. Given the fact that VAS showed very little variance, this correlation’s relevance should be interpreted with caution. Similarly, the objective measures showed poor correlation with MEMS. Syrup returns were difficult to manage and required more staff time than simply calculating PR data.

For the objective measures, although PR has been shown to be a highly effective means of monitoring adherence in adults [18], in our study PR overestimated adherence when compared to MEMS. This difference may be due to the different dispensing practices for syrups versus pills. For adults, exact numbers of pills can be counted and dispensed, whereas syrups can usually only be dispensed according to the manufacturers prepackaged volume. Since the dose of antiretroviral drugs changes with patient weight in children, no pharmacist is able to dispense an exact amount of syrup. The other objective measure, syrup returns, also overestimated adherence when compared to MEMS, possibly due to liquid spillage at dosing times. Another possibility is that of medication dumping—caregivers might deliberately empty medication containers before attending the clinic in order to appear adherent. Regardless, although PR is not as useful a tool in children as in adults, it is the more simple and cost effective measure if an objective method is to be used.

These results should be interpreted in light of several limitations. Due to limited financial and staff resources, we were only able to follow mother–child dyads for a short period of time, and it is possible that self-reported adherence was inflated (relative to MEMS adherence) over this short period. Overestimation of adherence by all measures but PR could also have occurred through the Hawthorne effect, due to the short monitoring period. It is also important to note that we measured only one antiretroviral drug of the regimen of three. Even though we included virologic data from subsequent visits after the adherence-monitoring period, the study cohort showed very little heterogeneity in their virologic response, and this limits the analyses. It is further noteworthy that the wider time frame during which specimen were collected could have influenced results. It is also important to note the limitations of our sample size—even though it is relatively large in comparison with other pediatric MEMS studies and with regards to our clinic population, its size is limiting in interpreting the significance of the detected associations and the overall statistical power of the data.

MEMS can over- or underestimate adherence, and even though we adjusted our data for these limitations, it cannot be guaranteed that all the caregiver reports that the adjustments were based on were correct.

Measuring adherence remains complex and elusive. The data here does highlight some areas where more intensive education should occur in the treatment support process: training with regard to amount and method of dosing syrups should be emphasized, as should timing of dosing. Although MEMS gave realistic adherence data in our study, no single adherence method is ideal. Our data suggest that PR and VAS would give an inexpensive assessment of objective and subjective adherence, but only VAS showed any correlation with MEMS usage. It is unfortunate that the expense of the best method renders it unavailable in RLS. As MEMS is repeatedly shown to be the best method of monitoring adherence, despite its limitations, perhaps more effort should be directed at the exploration of cheaper electronic alternatives for such RLS.