ABSTRACT
Background:
Widespread and effective use of molecular diagnostic tests is indispensable for protecting public health and containing the severe respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic. More than 1 year into the pandemic, as resources have reached a point of depletion, grouping samples in pools of certain sizes appears to be a reasonable method to reduce both the costs and the processing time without necessitating additional training, equipment, or materials.
Aims:
To assess whether the pooling strategy that was used in past outbreaks and is used in blood tests prior to transfusion for screening large populations can also be used in SARS CoV-2 tests.
Study Design:
Diagnostic accuracy study.
Methods:
This prospective study was conducted with 2815 samples, sent to the coronavirus disease 2019 (COVID-19) Laboratory of our hospital between February 12 and 21, 2021, to be tested for the presence of SARS-CoV-2. The samples were examined individually and in pools of five 100 μl taken from each sequential sample, using 3 different SARS-CoV-2 reverse transcription-polymerase chain reaction (RT-PCR) kits, the Allplex™ 2019-nCoV Assay kit (Seegene, Republic of Korea), the GeneMAP™ 2019-nCoV detection V.3 kit (GenMark, Türkiye), and the Bio-Speedy™ SARS-CoV-2 Double Gene™ RT-qPCR kit (Bioeksen, Türkiye) on the BioRAD CFX96™ Touch (Bio-Rad Laboratories Inc., Hercules, CA, USA) platform available in our laboratory.
Results:
Following the extraction of serial dilutions prepared from the SARS-CoV-2 RNA positive (cycle of threshold: 20) sample, the standard curves of RT-PCR were analyzed. By evaluating the efficiency (E) values, all 3 kits showed high sensitivity and similar results; while the highest level was detected with the Allplex™ 2019-nCoV Assay kit in the nucleocapsid (N) gene (E: 124%), the lowest was detected with the Double Gene™ RT-qPCR kit in the N and ORF 1ab genes (E: 90%). Of the samples included in the study, only 1 positive sample with low viral load was found to be negative when studied by pooling. The total number of kits to be used in pooled tests and then to individually retest the 5 samples in positive pools was calculated as 827 and the savings rate as 69.91% (1968/2815).
Conclusion:
The pooling strategy is an effective approach to extend the impact of limited testing resources and reagents available in certain periods of the COVID-19 pandemic. Testing by pooling samples requires improvement of RNA extraction methods and careful monitoring of RT-PCR test sensitivity to avoid missing low-positive entities. Therefore, based on the prevalence of COVID-19 in their regions, laboratories should conduct their own validation of pooling studies for RNA extraction and amplification methods they use.
INTRODUCTION
Since its first detection in Wuhan, China in December 2019 the severe respiratory syndrome coronavirus 2 (SARS-CoV-2), which is the causative agent of the coronavirus disease 2019 (COVID-19), spread across the continents by human-to-human transmission and was defined as a pandemic.1 Given that COVID-19 cases can be contagious 3 days before the onset of symptoms and a considerable portion of patients remain asymptomatic or minimally symptomatic throughout the course of the disease is responsible for the spread of the virus, the World Health Organization recommends increasing the number of tests as much as possible as the most effective way to limit the epidemic.2 Although, intensive efforts aimed at developing nucleic acid-based or viral antigen detection-based formats to enable faster results and ease of application in the field have begun to yield positive results, these new methods do not allow for testing large numbers of samples in a short period of time.3,4 Another approach is the pooling technique, where the samples are grouped to reduce the number of tests required. As the pooling strategy reduces both the costs and the processing time without necessitating additional training, equipment, or material, this method is more attractive. In this way, both laboratory facilities and human resources can be used more efficiently.5,6
The real-time reverse transcription polymerase chain reaction (RT-PCR) tests developed for detecting SARS-CoV-2 are widely used during the pandemic by laboratories authorized by the Ministry of Health. In these tests, the samples collected by nasopharyngeal and oropharyngeal swabs undergo reverse transcription and amplification with real-time RT-PCR after sample inactivation and lysis. The sample pooling technique, on the other hand, has already been used in past outbreaks and is being used prior to blood product transfusion for detecting human immunodeficiency virus, hepatitis B and C viruses.7,8 This technique, which is implemented according to the pooling guidelines of the The European Center for Disease Prevention and Control, was first suggested by Dorfman in 1943 and improved over the years.9 Here, test samples are combined in equal volumes in a single pool, and if the pooled test result is negative, then all the samples can be presumed negative. If the pooled test result is positive, then each of the samples in the pool are tested individually. Successful implementation of the pooling technique essentially requires knowing the detection limit of the test, its sensitivity and specificity, and the prevalence of the disease in the population. Many studies that have focused on determining pool sizes concluded that an effective pooling study could be carried out without affecting test specificity and sensitivity with a pool size of up to 8 samples.10,11 Recent guidelines recommend leaving the decision of pool size to the laboratories given the sensitivity and specificity of different extraction and PCR methods, as well as the disease prevalence rate that can vary among populations.12 Furthermore, since SARS-CoV-2 is a newly identified pathogen, it is still unclear how much dilution would affect the detection of the RNA-containing sample, hence leading to false negative results. Therefore, modifications and extensive validation studies are needed before the worldwide adaptation of the test.13,14
While the pooling technique has generally been used to screen large asymptomatic populations, the data obtained with this technique provides a reliable resource that can then be used to estimate the actual prevalence of the virus in the population. Determining the prevalence of SARS-CoV-2 is crucial for an accurate estimation of mortality. Considering that 85% of cases are asymptomatic, the mortality rate calculated only from the symptomatic cases will be higher than the actual rate. This can lead to an unnecessary panic in societies. In fact, it was recently found that previous serological studies estimating the prevalence of the SARS-CoV-2 virus have calculated the mortality rate 5 times higher than the actual.15 In this respect, the pooling technique will help to obtain more accurate prevalence data.
In this study, we aimed to assess whether a strategy, which was used in past outbreaks and is used in blood tests prior to transfusion for screening large populations, can also be used in SARS CoV-2 tests with a pool size that enables the best use of resources while maintaining the reliability of the tests.
MATERIAL and METHODS
This study was performed in line with the principles of the Declaration of Helsinki. Approval was granted by the Ethics in Non-Pharmaceutical Clinical Research Ethics Committee of İstanbul University-Cerrahpaşa (date: August 4, 2021 and decision no: A-23).
Sample Collection
This prospective study was conducted with 2815 samples, sent to the COVID-19 Laboratory of the İstanbul University-Cerrahpaşa, Cerrahpaşa School of Medicine, between February 12 and February 21, 2021, to be tested for the presence of SARS-CoV-2. Naso-oropharyngeal swab samples taken and placed in 2 ml vNAT (viral nucleic acid buffer) transfer tubes by our hospital’s healthcare staff were delivered under suitable conditions to the laboratory where routine SARS-CoV-2 RT-PCR tests were done, and the remaining parts were portioned into 1.5 ml RNAse-free microcentrifuge tubes. The samples were renumbered and examined anonymously on the same day.
Nucleic Acid Isolation and Amplification Validation Tests
To quantitatively determine the limit of detection (LoD) of the RT-PCR kits, the lowest detectable copy number was determined using 10-fold serial dilutions prepared from the extraction of virus culture supernatant obtained from an external cell culture laboratory with the virus concentration predetermined sample (107 copy/ml) by nano-drop spectrophotometer. In addition, to determine the lowest concentration level at which the 3 different RT-PCR kits to be used would be able to detect SARS-CoV-2, serial dilutions prepared from a SARS-CoV-2 RNA positive (cycle of threshold (Ct): 20) sample were extracted and the RT-PCR test was done.
The slope (the standard curve-Ct against log/DNA copy), is a measure of efficiency (E) and depends on the dilution factors. Using the same standards, if operated many times, the slope should stay the same. The intercept (y-int) is the value of Y when X equal zero and is a measure of the sensitivity of the assay. As long as threshold is used with the same value, during the whole assay, a longer PCR product gives more fluorescence per copy and we get higher Ct values and therefore higher intercepts. Therefore, keeping the number of cycles longer than the intercept value on the y-axis when the x-axis is zero will not yield more PCR products, and the fluorescence obtained above this value will be false positive. Kits were compared for the efficiency and sensitivity using standard curves drawn individually for each of the E, nucleocapsid (N), RdRP, and ORF 1ab gene regions targeted by all 3 kits.
Determining the Most Effective Pool Size for Detecting SARS-CoV-2 RNA
The optimal pool size was computed using the https://www.chrisbilder.com/shiny platform which uses mathematical modeling based on the parametric values of the prevalence of the disease in the region, the sensitivity and specificity of the test used, and the number of pooling stages.12 While we were using Chris Bilder’s calculator, we compared the prevalence of SARS-CoV-2 in Turkish population and SARS-CoV-2 positivite detection rate in our own hospital in the period of study, and used whichever was higher in the formula, in order to keep the pooling size at a safe value, since the prevalence rate in the population declared by the government is not always up-to-date.
Nucleic Acid Isolation and Amplification Tests
The study was planned to be carried out over 10 days. The nucleic acid isolations of the samples alone and as sequential pools of 5, created by taking 100 µl from each sample in vNAT transfer tubes were done with the SEEPREP32™ automated extraction system (Seegene, Republic of Korea), according to the manufacturer’s instructions. While the volume put into the system was 200 µl, the final elution volume was 60 µl. All samples were run on the BioRAD CFX96™ Touch (Bio-Rad Laboratories Inc., USA) platform available in our laboratory, separately with all 3 SARS-CoV-2 RT-PCR kits, the Allplex™ 2019-nCoV Assay kit (LOD: 4.167 copy/ml, sensitivity: 99.97%) (Seegene, Republic of Korea), the GeneMAP™ 2019-nCoV detection V.3 kit (LOD: 100 copy/ml, sensitivity: 99.81%) (GenMark, Türkiye), and the Bio-Speedy™ SARS-CoV-2 Double Gene™ RT-qPCR kit (LOD: 1000 copy/ml, sensitivity: 99.53%) (Bioeksen, Türkiye) alone and as sequential pools of 5. The Allplex™ 2019-nCoV kit is multiplex real-time PCR assay for simultaneous detection of 3 target genes of SARS-CoV-2. The assay is designed to detect RdRP (in the Cal Red 610 channel) and N (in the Quasar 670 channel) genes specific for SARS-CoV-2, and E gene (in the FAM channel) for all of sarbecovirus including SARS-CoV-2. The GeneMAP™ 2019-nCoV detection V.3 kit target gene regions are RdRp and N in 2 dye channels of FAM and VIC, respectively. The Bio-Speedy™ SARS-CoV-2 Double Gene™ RT-qPCR kit is an 1-step real-time RT-PCR assay for the qualitative detection of SARS-CoV-2 specific ORF1ab gene and N gene in a single FAM channel. The comparison of sensitivity, specificity, positive predictive value, negative predictive value, and accuracy values of the 3 kits used in the study is summarized in Table 1.
All detected Ct values were recorded. Correlations were evaluated by comparing the Ct values obtained separately for each gene region. While in groups containing positive samples with high-Ct (low viral load) levels, the lowest viral copy detection limit of the pooling technique for the diagnosis of COVID-19 and the possible risks for false negative results were evaluated. The number of test kits used for pooled tests was compared with the number of those that would have been used if tests were done individually to find the reduction in test kit consumption.
Statistical Analysis
Statistical analysis was done using the Statistical Package for Social Sciences (SPSS) v22.0. Results of the parametric data were expressed as mean and standard deviation, and results of non-parametric data were expressed as median and minimum-maximum (median, min-max). The confidence interval was 95% and P < 0.05 was considered as significant.
RESULTS
LoDs calculated with the SARS-CoV-2 RT-PCR kits using the lowest detectable 10-fold serial dilutions prepared from extract of virus culture supernatant were <10 copies/ml for Allplex™ and GeneMAP™ and <100 copies/ml for Double Gene™. The standard curves of RT-PCR were analyzed after the extraction of serial dilutions prepared from the SARS-CoV-2 RNA positive (Ct: 20) sample. Based on the intercept values, we interpreted that false positive could be detected with the Allplex™ 2019-nCoV Assay kit in the FAM channel (E gene) after 41.70 Ct, in the Quasar 670 channel (N gene ) after 39.64 Ct, in the Cal Red 610 channel (RdRp gene) after 44.34 Ct; with the GeneMAP™ 2019-nCoV detection V.3 kit in the FAM channel (RdRp gene) after 43.32 Ct, in the Cy5 channel (N gene) after 43.32 Ct; and with the SARS-CoV-2 Double Gene™ RT-qPCR kit in the FAM channel (N and ORF 1ab genes) after 45.21 Ct; and therefore, the cycles should not exceed the indicated values. By evaluating the E values, all 3 kits showed high sensitivity (over 90%) and similar results. While the highest level was detected with the Allplex™ 2019-nCoV Assay kit in the N gene (E: 124%), the lowest was detected with the Double Gene™ RT-qPCR kit in the N and ORF 1ab genes (E: 90%) (Figure 1).
However, an analytical sensitivity difference of 3-6 Ct was noticed between the RT-PCR kits used, which increased especially as the Ct value was rising (Figure 2).
Failure in RNA extraction can create a significant limitation as positive samples will not be detected. To prevent this, internal control (human transcript) was additionally performed on the samples before extraction. Human gene amplification was present in all of the samples included in the study, and no false negatives were detected that could potentially be associated with extraction errors.
SARS-CoV-2 RNA positivity rate was 2.3% (66/2815) in the samples included in the study. Of these, 22 were weak positive (Ct≥30), 26 were moderate positive (30>Ct≥25), and 18 were strong positive (25>Ct). Of the positive pools, 46 had 1 positive sample and 10 had 2 positive samples (66 positives, totally). However, out of 567 pools of 5 samples, 512 were negative and 55 were positive, false negatives occurred in 1 pool, and 1 positive sample could not be detected (The potential risk of missing positive samples was calculated as 0.33%). The mean Ct values of the positive samples and pools are summarized in Table 2.
When the difference between the Ct value of the positive samples and the Ct values of the pools containing these samples was calculated, a median decrease of 4.31 (3-5) Ct was found. Of the samples included in the study, only 1 positive sample with low-viral load (Allplex™ E gene: 36.15 Ct, N gene: 36.16 Ct, RdRp gene: 39.61 Ct, GeneMAP™ RdRp gene: 38.25 Ct, N gene: 36.62 Ct, and SARS-CoV-2 Double Gene™ RT-qPCR kit N and ORF 1ab genes: 38.55 Ct) was found to be negative. The number of kits to be used when samples are tested in pools of 5 and then positive pools are retested was calculated as 827 and the savings rate as 69.91% (1968/2815) (Figure 3).
DISCUSSION
Grouping samples in pools of a certain size has been successfully applied for many years in blood banking and screening of infectious diseases. Today, this strategy is becoming more widespread with the development of highly sensitive molecular-based tests; and mathematical models have been designed to determine the appropriate pool sizes for use in cases of new diseases and outbreaks. In this study, we examined the possibility of a pooling strategy that would allow for more efficient use of the testing resources, which have reached a point of depletion more than 1 year into the COVID-19 pandemic, by comparing 3 RT-PCR tests that target the E, N, RdRp, and ORF 1ab gene regions in a large sample size.
According to a study, higher sample pool sizes provide more gain in conditions when the prevalence rate is 1% or lower, and pool sizes of 5 or more samples would be inefficient if the prevalence rate is 5% or higher as this would increase the risk of dilution and thereby the risk of false negative.16 In the referred study, the authors conclude that a pool size of 5 samples would be suitable when prevalence rate is lower than 5%, but the sample pooling method should not be used when the rate exceeds 20%. While there are no epidemiological study data available on the prevalence of COVID-19 in Turkey, the positivity rate in the population tested in our hospital in the study period remained below 5%. Our study showed that the pool size of 5 samples that we calculated using the mathematical modeling tool at https://www.chrisbilder.com/shiny maintained both testing sensitivity and specificity and helped save from reagent consumption as well as staff time, and thereby provided high efficiency.
The calculated increase in efficiency is only for RT-PCR and does not include the viral isolation and reporting processes. Since, the routine workflow for molecular diagnosis of SARS-CoV-2 in Turkey includes only the viral inactivation and lysis steps without a complete extraction, the process will inevitably take longer to complete when RNA extraction and pool creation are added to this typical algorithm. Pooling-dependent groups such as individuals from the same family or increasing the efficiency of the RNA isolation process with automated extraction devices or using primary kits that screen more gene regions, as we did in our study, can help to enhance testing sensitivity and thereby to increase efficiency and tackle the issue of inefficiency due to loss of time.5,17 Because test results are automatically sent to the data network, the pooling scheme should be adapted to this system to obviate the need for additional time and work. Since this approach will allow to screen more people in shorter period of time by providing increased efficiency, it will enable isolation and prevent the spread of the disease within the group as soon as positivity is detected in closed communities such as nurseries or factories.
The major issue which is likely to be encountered in the pooling strategy is false positive and false negative. We had no false positives in the pools of our 2815 samples. False negatives, however, are more likely. Because amplification could not be obtained for 1 weak positive sample in the pool of 5 samples, the potential risk of missing positive samples in our pooled-testing protocol was calculated as 0.33%. This was deemed tolerable considering the potentially high rates of false negative results due to especially preanalytical errors (e.g., insufficient sampling, transport times, and impact of temperature). Large-scale implementation of the pooling strategy must be done carefully so that preanalytical processes do not lead to major signal losses.
Since, the prevalence of COVID-19 in the tested population is not always known, it may affect the decision of an optimum pool size. In such an event, data such as patient histories, serological data, and the number of symptomatic cases can be used to determine the pool size.18,19 Once the pooling strategy is routinely implemented, it will also be possible to adapt the pool size according to the dynamic prevalence data.
Ultimately, in a rapidly changing pandemic, it is essential that testing strategies are adapted to the conditions brought about by the potential increases in positive test rates. Testing by pooling samples requires improvement of RNA extraction methods and careful monitoring of RT-PCR test sensitivity to avoid missing low-positive entities. Therefore, based on the prevalence of COVID-19 in their regions, laboratories should conduct their own validation pool studies for the RNA extraction and amplification kits they use. Our study demonstrated that the pooling strategy is an effective approach to extend the impact of limited testing resources and reagents available in certain periods of the COVID-19 pandemic.