Introduction

We contribute to the literature on the proportion of the population who remain COVID-naïve (neither vaccinated nor previously infected) and to the technical literature on the performance of COVID-19 spike protein and nucleocapsid tests in detecting prior vaccination or infection, including the extent to which test sensitivity wanes with time since vaccination or infection.

In early 2020, SARS-CoV-2, the virus that causes the COVID-19 disease, spread rapidly among an initially naïve population. Infections were initially measured using RT-PCR (reverse transcription polymerase chain reaction) tests. However, many infections went often undetected1,2. Undercounting had several causes: many infections were asymptomatic or only mildly symptomatic3, and those infected didn’t realize they might have COVID-19; in early and mid-2020, tests were in short supply; some people were tested either too soon (before the RT-PCR test could detect infection) or too late, after the infection had subsided; and some infected persons avoided testing to avoid quarantine requirements.

The pattern of more actual than reported infections persisted in 2021, after vaccines became available. In early 2022, with the emergence of the Omicron variant, the percent of the population who had been infected rose rapidly4. However, the ratio of known to actual infections fell at the same time. First, rapid, at-home antigen tests became widely available. Only a small but unknown fraction of positive antigen test results were confirmed with RT-PCR tests or otherwise reported to public health authorities. Second, the Omicron variant was much more infectious than the prior Delta variant, but on average milder5,6, with more asymptomatic or mildly symptomatic infections7. Mildly symptomatic people may not realize they have COVID-19 or may not view testing as important.

This leads to research questions: What proportion of the population remains both unvaccinated and uninfected, thus lack antibodies to the SARS-CoV-2 virus, and are therefore more vulnerable to severe COVID-19 infection or death? How many people were infected, at a level sufficient so that their bodies generated measurable antibodies, without knowing this? We study these questions using data from an antibody study, conducted on a diverse population in Milwaukee County, Wisconsin, from May 2021 to June 2022. All participants completed a survey and reported their vaccination history and whether they had been infected, to their knowledge. All participants received a SARS-CoV-2 spike protein antibody test (below, “spike”), which is intended to detect antibodies produced either by vaccination or past infection. Those who reported being vaccinated also received a SARS-CoV-2 nucleocapsid antibody test (below, “capsid”), which is intended to detect antibodies produced by past infection, but not those produced by vaccination. We used only the more sensitive spike test for unvaccinated persons.

We use the combined survey and antibody test information to report the proportion of survey participants who were vaccinated; the proportions of vaccinated and unvaccinated persons who tested positive for spike antibodies; the proportion of vaccinated persons who tested positive on a capsid test; the proportions of vaccinated and unvaccinated persons who reported having been infected, confirmed through a COVID-19 test; and for those who were neither vaccinated nor, to their knowledge, infected, the proportion who nonetheless tested positive on the spike test. Much of the analysis below focuses on the period from April-June 2022, shortly after the large Omicron infection wave during January-March 2022.

Our study is related to the series of studies of repeat blood donors by Jones et al.2,8,9 The most recent of these studies found 96% seroprevalence (from infection or vaccination) through September 2022, including a 70% probability of prior infection. Our study is smaller, but has the important advantage that we have information from the surveys on which participants were vaccinated, infected to their knowledge, or both. This let us use a highly sensitive spike test to assess the prior infection rate among unvaccinated persons, rather than a less sensitive capsid test. We can also report spike test positivity rates separately for participants who did versus did not know they had been infected. The Jones et al. studies used less-sensitive capsid antibody tests to detect prior infection.

With regard to test performance, we provide new evidence on the sensitivity of the Abbott Laboratories and Roche Diagnostics spike and capsid tests to capture vaccination or infection, a substantial period after vaccination or infection.

Data and methods

This project was reviewed and approved by the Medical College of Wisconsin Institutional Review Board (IRB). We complied with the research protocol as submitted to the IRB, including obtaining informed consent from all participants.

Data

We conducted an antibody seroprevalence study of 2,697 adults, aged 18+, who were residents of Milwaukee County, Wisconsin (below, “Milwaukee”). All participants received antibody tests over May 2021-June 2022. Milwaukee is racially, ethnically, and economically diverse. We engaged in extensive community outreach to reach the large Milwaukee Black and Hispanic communities, including using study locations in community health centers in predominantly Black and Hispanic areas, and outreach through a consortium of Black churches.

Study participants completed a short survey governing sociodemographic information. The survey also asked participants whether they had been vaccinated, infected (to their knowledge, confirmed by a positive COVID-19 test), or both. Participants then received a spike antibody test, a capsid antibody test, or both. We used tests from Abbott Laboratories, Roche Diagnostics, or both. See Appendix for test details.

The inclusion criteria for participation in the study were only that one needed to be a Milwaukee County resident, at least 18 years old, and able to visit one of our testing sites. Participants who completed the initial survey and test were invited to be retested and complete a shorter, second survey, beginning four months after their first test.

We used Abbott tests through February 2022, then switched to Roche tests for new participants, but generally used Abbott tests for second tests of the same person. The Abbott tests were Alinity SARS-CoV-2 IgG II (spike test) and Alinity SARS-CoV-2 IgG (without the “II”, capsid test), where Alinity is the testing machine. The Roche tests were cobas Elecsys Anti-SARS-CoV-2 S (spike test) and cobas Elecsys Anti-SARS-CoV-2 (without the “S”; capsid test), where cobas Elecsys is the testing machine. For 118 participants in February and March 2022, we assessed concordance between the Abbott and Roche test results by running both companies’ tests on the same blood samples.

Calculation of Confidence Intervals

We did not find in the literature on COVID-19 tests a satisfactory approach to computing confidence intervals for test specificity. Some studies do not report 95% confidence intervals (CIs). Some report CIs with an upper bound greater than 1.00, which is clearly incorrect. Some appear to use confidence intervals drawn from a binomial distribution based on the observed specificity. This is also incorrect, since the observed specificity is an estimate, drawn with error from an underlying population. This approach will also fail when measured specificity is 100%, as was the case for some of our analyses.

We therefore developed our own approach. We summarize this approach here and provide details in the Appendix. Our approach is based on the cumulative distribution function of the Binomial distribution (call this CDFbinomial). The binomial CDF has an analytical formula and can be computed for a given number of positive tests, npos, out of a sample of N individuals, and thus an observed probability p = npos/N. We analytically calculated plow such that there is a 97.5% chance that the observed proportion of positive tests out of N tests will be at least plow; that is, where CDFbinomial(npos, N, plow) = 0.025. We also calculated phigh, such that there is only a 2.5% chance that the observed proportion of positive tests will be at least phigh; that is, we computed the value of phigh at which CDFbinomial(npos, N, phigh) = 0.975. The 95% confidence interval around the observed proportion p will be [plow, phigh]. For test results, in which npos = N (all tested persons were positive), we calculated plow as 0.025(1/N) and set phigh = 100%. We used simulation to confirm the accuracy of the analytical confidence intervals; see Appendix for details.

Results

The Milwaukee adult population in 2021 (age 18+) was 24.11% Black, 13.21% non-Black Hispanic, 55.46% White, 4.22% Asian, and 3.01% other (American Community Survey). Our community outreach to minority populations resulted in a study sample with racial/ethnic composition reasonably representative of Milwaukee (Table 1). We were not successful, however, in enrolling many unvaccinated persons. Instead, our study appears to have attracted mostly health-conscious persons, with a disproportionate share of women (64.3%; Table 1), versus 52.4% for all Milwaukee adults and a high percentage of persons who received at least one vaccine dose (92.5%; Table 2), versus a vaccination rate for all Milwaukee adults of 64.3% as of June 30, 2022. Participants also tended to be older than the population as a whole (mean age of 55.5 years versus 46.2 for all Milwaukee adults).

Table 1 Table provides summary statistics for antibody study participants. Panel A. Full study period from May 2021-June 2022. For 117 people who received both Abbott and Roche spike protein tests in February-March 2022, 2 were positive for Roche but not Abbott; 0 were positive for Abbott but not Roche. For 112 vaccinated people who received both Abbott and Roche nucleocapsid tests in Feb-March 2022, 16 were positive for Roche but not Abbott; 0 were positive for Abbott but not Roche. The one unvaccinated but infected person (based on the survey) with a negative antibody test received the Abbott capsid test. Panel B. Period from April-June 2022. All participants received Roche tests. Both panels. Totals for male and female columns do not sum to total for all persons due to missing values.
Full size table

Table 2 reports monthly data for our principal measures over the sample period.

Table 2 Summary statistics by month. Table shows data by month for Milwaukee antibody study participants over May 2021- June 2022, for indicated participant counts and percentages. The table shows counts for initial antibody testing; some participants also received follow-up tests, generally four months after the first test.  All participants received spike test, which detects either vaccination or prior infection.  Vaccinated persons also received capsid test, which detects only prior infection.
Full size table

Results for vaccinated persons

Spike test results. The proportion of vaccinated participants who tested positive for spike protein is close to 100% (Table 2). For the more sensitive Roche test, it is exactly 100% (611/611; 95% CI = [99.40%,100%]). We used the Roche test during February-June 2022. All vaccinated persons retained sufficient spike antibodies to be detectable, including those who were vaccinated in early 2021 and did not thereafter receive a booster shot or become infected, and 29 immune-compromised persons. We thus confirm both the durability of detectable vaccine-induced antibodies and high sensitivity for the Roche spike test.

Abbott versus Roche spike tests. Over the full sample period, the sensitivity of the Abbott spike test for vaccinated persons (both with and without prior infection) was 98.2% (1,954/1,989, CI = [97.62%, 98.77%] versus 100% for the Roche test, even though the Roche test was administered later in the sample period, and thus at a time often more distant from vaccination. The Abbott percentage was similar for the 112 vaccinated persons who received both spike tests in February and March 2022, at 110/112 (98.2%). If we separate vaccinated persons into those with prior infection (either reported or determined through a positive capsid test) and thus hybrid immunity, versus those with only vaccination, the Abbott spike test is 100% sensitive for hybrid immunity (373/373; CI = [99.02%, 100%]) versus 97.8% for vaccination without infection (1581/1616; CI = [97.07%, 98.49%]).

Capsid test results. All vaccinated persons also received a capsid test. Over the last three months of the sample period, from April-June 2022, 205/497 tested capsid positive, including 122/141 (86.5%; CI = [80.62%, 91.71%]) who reported prior infection and 83/356 (23.3%; CI = [19.29%, 28.05%]) who did not report prior infection.

Abbott versus Roche capsid tests. The capsid test results for vaccinated persons are strongly affected by test sensitivity. Over the full sample period, the Abbott capsid test captured only 135/279 (48.4%; CI = [42.77%, 54.44%]) of vaccinated persons who reported prior infection, versus 152/169 (89.9%; CI = [85.05%, 94.04%]) for the Roche capsid test. For the 29 vaccinated persons who reported being infected and received both capsid tests, 28 were capsid positive on the Roche test (96.6%, CI = [88.06%, 99.91%]) versus 22 for the Abbott test (75.9%; CI = [60.34%, 89.71%]). As discussed below, the Abbott test also exhibited substantial waning; the Roche test did not. The limited sensitivity of the capsid tests, especially the Abbott test, is an important practical constraint on our ability to determine the proportion of vaccinated persons who were infected.

Results for unvaccinated persons: Spike test results

Unvaccinated persons generally received only the spike test. Of 52 persons who reported a prior infection, all tested positive for spike antibodies.

We also assessed spike test results for unvaccinated persons who did not know of a prior infection. This percentage generally increased over time. Over the last three sample months, April-June 2022, 19/22 (86%; CI = [70.89%, 97.09%]) of these persons tested spike positive.

Overall antibody test results for April-June 2022. Overall, during April-June 2022, following the Omicron wave, 523/526 (99.4%; CI = [98.63%, 99.88%]) persons tested antibody positive.

Time trends

Vaccinated persons. Figure 1 shows time trends, separately for unvaccinated and vaccinated persons, in (i) the percentage reporting prior infection at the time of the antibody test (dashed gray line); (ii) the percentage not reporting prior infection but testing antibody positive, based on a spike test for unvaccinated persons and a capsid test for vaccinated persons (dotted orange line); and (iii) the sum of the two (solid blue line).

Fig. 1
figure 1

Vaccinated and Unvaccinated Persons: Proportions with Reported Prior Infection or Positive Antibody Test.

Figure shows percentage of participants who reported a prior infection, confirmed with COVID-19 test (dashed gray line), percentage who reported no prior infection, but tested positive on an antibody test (dotted orange line); and percent of participants with either reported infection or positive antibody test (top solid blue line). Panel A. unvaccinated participants, with prior infection determined using spike antibody test. Panel B. vaccinated participants, with prior infection determined using capsid antibody test. Sample (summed across the sample period) is 2,494 vaccinated and 203 unvaccinated participants.

Full size image

For unvaccinated persons (Panel A), the overall percentage of reported or detected prior infection generally rises during the sample period and averages 90% during the last three sample months (April-June 2022). For vaccinated persons (Panel B), this percentage is under 20% during 2021, consistent with vaccine effectiveness against symptomatic infection during this period (see reviews by Feikin et al. and Black and Thaw)10,11. This percentage rises during the Omicron wave in early 2022, and averages around 50% during the last three sample months.

The gap in overall percentages for reported plus antibody-detected infections between unvaccinated and vaccinated persons has several possible sources. First, even in the Omicron era, vaccination may continue to provide meaningful protection against symptomatic infection. Second, vaccinated persons may be more health-conscious and more careful to limit infection risk, despite being vaccinated. Third, as we explore in Fig. 2, the capsid test that we used to detect infection in vaccinated persons was less sensitive than the spike test used for the unvaccinated.

Fig. 2
figure 2

Spike Protein and Capsid Antibody Test Sensitivity Conditional on Reported Past Infection.

Figure shows, for vaccinated and unvaccinated participants who reported prior infection, confirmed with COVID-19 test, the percentage of infections that were confirmed with an antibody test. For unvaccinated participants, confirmation was through a spike test (this percentage is 100% and is shown with a dotted black line). For vaccinated participants, confirmation was through a capsid test, either Abbott (orange dashed line); or Roche (solid blue line). Both tests were used during a limited overlap period in February-March 2022. Sample (summed across the sample period) is 52 unvaccinated participants who reported prior infection received spike test; 279 vaccinated participants who reported prior infection and received the Abbott capsid test; and 169 vaccinated participants who reported prior infection and received the Roche capsid test.

Full size image

In Fig. 2, we report antibody test sensitivity for persons who reported infections. For unvaccinated persons, all 52 reported infections were confirmed with a spike test, shown as the top, dotted black line at 100%. Sensitivity was much lower for vaccinated persons, for whom we used a capsid test to detect prior infection, especially for the Abbott test. The dashed orange line shows sensitivity by month for the Abbott capsid test. During 2021, the percentage of vaccinated persons, with reported prior infection who tested positive on the Abbott capsid test is never above 60% and sometimes as low as 20%. Given that the Abbott test missed many reported infections, it surely also missed unreported infections, many of which were asymptomatic or only mildly symptomatic, and hence may have generated weaker antibody response. Vaccination could have also led to milder infection, lower capsid antibody production and thus lower test sensitivity.

Abbott capsid test sensitivity rises sharply to 60–80% in January-March 2022, with the emergency of the Omicron variant. We replaced the Abbott tests with Roche tests in February-March 2022; Roche capsid test sensitivity is shown with the solid blue line. Roche capsid sensitivity is well above Abbott for the overlap period, and averages around 90% for the full period in which we used the Roche test. However, the Roche capsid test sensitivity is still well below 100% for reported infections.

For April-June 2022, we can estimate the number of infections missed by the Roche capsid test by assuming that the roughly 90% sensitivity for people with known infection also applies to for those without known infection. Of 354 vaccinated persons without known infection during this period, 82 had positive Roche capsid tests. If we assume 90% test sensitivity, then the true number infected would be roughly (82/0.90) = 91, and the true percent of vaccinated persons who were infected, without known infection, would be roughly 26%.

Figure 3 shows the percentage of unvaccinated persons without known infection, who tested positive on the spike test. This percentage is around 20% during most of 2021, but rises sharply beginning in December 2021, during the late-2021 Delta wave, and continues to rise in early 2022, which Omicron replacing Delta as the dominant variant in early 2022. As noted above, this percentage averaged 86% over April-June 2022 (19/22).

Fig. 3
figure 3

Percentage of unvaccinated participants, without reported prior infection, who tested antibody positive.

Figure shows monthly percentages of unvaccinated study participants who reported no prior infection but tested positive on a spike antibody test over the sample period. Sample (summed across the sample period) is 151 participants.

Full size image

Second test results

We offered repeat antibody tests to participants over July 2021-September 2022, generally four months after initial testing. Of 2,697 study participants, 1,545 received follow-up testing. Above, we did not report combined first and second test results because we are not aware of a simple way to report results or calculate confidence intervals for draws from a binomial distribution, when some draws are independent (first test results) and some are not (second test results). However, the second test results are consistent with the results reported above for the initial test. In particular, if we consider both first and second tests over April-June 2022 for both vaccinated and unvaccinated participants, 723/728 (99.3%) participants have positive spike tests, including 30/34 (88.2%) unvaccinated participants who did not know of prior infection, Thus, the second tests confirm the main finding that by April-June 2022, very few participants were Covid-19 naïve.

Capsid test waning. We found substantial waning of test sensitivity for the Abbott capsid test. Of 98 vaccinated participants who tested positive on a first Abbott capsid test, only 50 (51%) tested positive on a second test four months later. Waning sensitivity helps to explain the weak performance of the Abbott capsid test to detect known prior infection on the first tests. Our results imply that the Abbott capsid test is a bad choice for detecting prior COVID-19 infection, sometime after infection.

The Roche capsid test, in contrast, showed no evidence of waning. It remained positive for 100% (63/63; CI = [94.31%, 100%]) of vaccinated persons with a prior positive capsid test (either Roche or Abbott). Of course, waning is still possible over a longer time period.

Discussion

Very small percentage of COVID-naïve participants

Over April-June 2022, following the Omicron infection wave in early 2022, almost all study participants (523/526; 99.4%) tested positive for spike antibodies. While the number of unvaccinated participants was small, 90% of these persons (27/30) tested positive for spike protein antibodies, including 100% (8/8) of those who reported being infected with COVID-19 and 86.4% (19/22) of those who did not report prior infection. Thus, by spring 2022, there were almost no COVID-19 naïve persons left in our sample.

The small proportion of remaining COVID-19 naïve persons likely contributes importantly to the reduced toll of COVID-19 in hospitalizations and deaths, observed after the early-2022 Omicron wave, both in our own research in Milwaukee (Black et al.,12 Supp. Appx) and elsewhere. Infections of non-COVID-naïve persons are less likely to lead to severe disease.

Limited sensitivity and waning of capsid tests

A strength of this study is that we have survey data on who was vaccinated. The survey responses on vaccination and prior known infection appear to be highly reliable. For vaccinated recipients who received the more sensitive Roche spike test, 611/611 tested positive. For unvaccinated recipients who reported prior infection, 52/52 tested positive for spike antibodies. The availability of the surveys let us use the more sensitive spike test to assess prior infection, instead of the less sensitive capsid test.

The performance of the Abbott capsid test was notably weak, with rapid waning of ability to detect infection. Peluso et al.13 also report rapid waning of the Abbott capsid test. The Roche capsid test was more sensitive but also failed to detect some reported infections. Follmann et al. also used the Roche capsid test and report that it captured only a fraction of breakthrough infections for vaccinated persons who participated in the Moderna COVID-19 vaccine trial14. However, we found evidence suggesting that capsid test sensitivity is higher for the Omicron variant than for prior variants.

Our analysis suggests that any assessment of the percent of vaccinated persons who were infected requires a sensitive capsid test, such as the Roche test, and potentially adjusting for the limited sensitivity and potential waning of the capsid test.

Comparison to other studies

Our results differ from Jones et al.,9 who report 3.6% Covid-naïve as of September 2022, among a national sample of repeat blood donors. This is substantially higher than the 0.6% that we find. Both samples suffer from selection bias. On the one hand, Jones et al. study only repeat blood donors, who tend to be healthier than the general population15 and, in this and other ways, may not be population representative. They use weighting to make their sample more population representative on demographic characteristics, but cannot address the potential for blood donors to be different than demographically similar non-donors on behavioral factors, such as the propensity to be vaccinated or to be infected. On the other hand, the participants in our antibody study are highly vaccinated and not population representative, even for Milwaukee.

We suspect that an important contributor to the difference in results may be imperfect test sensitivity. In our study, the Roche spike test was 100% sensitive in detecting spike antibodies for vaccinated persons. However, the Abbott spike test was less sensitive at 98.6% across all tests. Jones et al. do not report the sensitivity of the Ortho spike test they used. If sensitivity was even modestly below 100%, their study will underestimate the proportion of unvaccinated but infected persons, and therefore overestimate the proportion of COVID-19 naïve persons. There is reason to question the sensitivity of the Ortho test. Wiegand et al. (2023) use the Roche capsid test during the Omicron period on a national convenience sample of blood samples collected for other purposes and report positive capsid test rates approaching 80% in February 2022 and rapidly increasing during the Omicron period, versus the 70% reported by Jones et al. for a later period, July-September 20229.

Our results are closer to Brown et al.,16 who study a representative sample of Canadian adults over January 24-March 15, 2022, in the middle of the early-2022 Omicron wave. This study assessed both spike and capsid antibodies using highly sensitive tests developed by the authors. They found a 2% Covid-naïve rate for the entire sample, and 1% for ages 60+; recall that our mean age is 55.5.

Limitations

We have a substantial sample of vaccinated persons, but a small number of unvaccinated persons. The study participants are also not population representative, with a much higher proportion of vaccinated persons than Milwaukee as a whole. The generally older, more often female, and mostly health-conscious participants in our sample may also have been more careful than the average person in limiting their exposure to COVID-19 infection. If so, our results for the unvaccinated may, if anything, understate the true population percentage of unvaccinated persons who were infected, often without knowing this.

The participants in our study were also older than the Milwaukee population as a whole. Age correlates with lower infection rates;4,7,9 a tendency which we confirm for our sample (results not reported). Moreover, our data is from only one mid-sized Midwestern city and surrounding suburbs.

All of these factors suggest caution in extrapolating from our sample to the general adult population. However, we know of no other comparable studies that combine antibody testing with survey-based knowledge of which participants were vaccinated and which, to their knowledge, had been previously infected.

Our study ended in June 2022. Infection rates have continued to be substantial since then, so the proportion of Covid-naïve persons is likely lower than reported in our study.

Conclusion

We found that by the second quarter of 2022, the proportion of Covid-naïve participants in an antibody study was only 0.6% (CI = [0.21%, 1.66%]). The proportion of naïve persons is surely lower today. Most people have antibody protection against severe disease (hospitalization or death), and an increasing percentage of vaccinated persons have hybrid immunity.