.jpg)
Carbon Health’s COVID Ready program is helping businesses across the country get their employees back to work safely. As a first step, our data team has built a simulation model that mimics transmission in the workplace and quantifies the number of cumulative infections over a given period of time. We can then use this simulation to tune parameters, taking into account certain constraints that the business may have — budget or otherwise — to determine the best policies and protocols for businesses looking to get their employees back to work safely.
Not only will this simulation allow us to assess how efficient we would be at stopping outbreaks within the workplace, but it will also allow us to inform certain discrete actions for those businesses. These actions include decisions around which type of diagnostic tests to use, at which cadence to test employees, what quarantine practices to implement, and how to best prioritize who to test.
We created an agent-based SIR model that simulates the actions and dynamics of individual agents and assesses their effects on the overall system. These individual agents progress between S, I, R (Susceptible, Infectious, Recovered) compartments in the model depending on their interactions with the population. One of the benefits of agent-based models is their ability to model highly complex systems. This model is capable of incorporating a level of randomness while simulating individual actions and interactions, and then tracking the pattern on the whole aggregate system as the data is generated from the simulation. The main framework for this model is from a post investigating the effects of different types of COVID-19 diagnostic tests. [1]
The parameters used in the simulation are:
These parameters can all be adjusted for each specific business. For example, depending on the workplace policies on masks, on distancing, and on sanitation, the `beta` value may increase or decrease. The industry sector of the business will also affect these values — a meatpacking plant will likely have a higher `beta` value than would a tech company. Similarly, the `external_infection_rate` is highly dependent on both location and employee behavior.
We start with an `N` number of employees. We then take the following steps:
The output of the simulation on each simulated day is 1) `Q`, the number of employees quarantining, and 2) `I`, is the number of (non-quarantining) infected employees. We then calculate the probability of contracting COVID-19 for each employee using the following formula:
In short, the probability of contracting COVID-19 is one subtracted by the probability of coming in contact with only safe, non-infectious, employees.
Note that it would be ideal to test everybody, but due to limitations in the number of tests or constraints in budget allocation, this might not be a possibility. In order to decide who to test, we included a few filters to Step #4 above. We prioritize patients based on the following heuristics:
If additional tests remain after all the filters are applied, we randomly select who to test out of the remaining employees.
For COVID-19 testing, we currently use two different types of tests:
The PCR non-rapid tests are believed to provide greater testing sensitivity (i.e. more reliable), but take longer to deliver results. The greater the number of tests administered, the longer this delay will be. This is because a large influx of tests leads to bottlenecks in the laboratory process. By contrast, antigen tests and PCR rapid tests have much quicker turnaround times — results within 15 minutes — but are believed to be less reliable, displaying a higher number of false negatives.
So which is preferable of the two? Do we prefer greater sensitivity at the expense of longer turnaround times, or vice versa?
As a first step, we run the simulation with the following default values:
For the antigen and rapid PCR tests, we use a test sensitivity value of 0.75 and a testing delay value of 0 days (since results are returned within 15 minutes), while for the non-rapid PCR test, we use a testing sensitivity value of 0.98 and a testing delay value of 5 days. It should be noted that in reality, this testing delay is often even longer, up to 9 days long. Additionally, while the exact testing sensitivity is out of the scope of this study, this model will allow us to investigate the relative benefit of faster turnaround times compared to lower sensitivity.
We ran the model through 100 iterations for each configuration type, with an iteration meaning a full run of the 130-day simulation, and then observed the average number of cumulative infections for each day over a given period of time. This allowed us to plot and compare the different parameter configurations to determine the configuration leading to less cumulative infections.
First we compared the impact of faster turnaround times versus greater sensitivity. As shown in the plot below, faster turnaround times result in lower numbers of infections even at the expense of lower reliability. This insight is valuable and non-intuitive. It is important to consider this tradeoff when determining the best diagnostic tool.
We also plotted the best case scenario of zero intra-workplace infections, where a slight increase in transmission rate is visible due to purely external infections that have occurred outside the workplace.
However, in the above plot we were assuming a `risk_behavior` of 0, meaning that all simulated employees were choosing to head to the office regardless of whether they displayed symptoms or not. Hopefully this is not the case — individuals with symptoms are likely to self-isolate in most cases.
We then decided to investigate how transmission rate will be affected by different `risk_behavior` values. Testing values of 0.0, 0.25, 0.5, 0.75, and 1.0, results showed that even low `risk_behavior` values greatly decreased outbreaks. When `risk_behavior` was just 0.25 (indicating that 25% of cases with symptoms will self-isolate), cumulative infections dropped from 31.20% to just 17.08% of total employees infected.
Note that here we are using the `sensitivity` and `testing_delay` values of the antigen test, so the black line (‘0% of those with symptoms remain home’) in the below plot corresponds to the blue line ( ‘Antigen test every 4 days’) from the previous plot.
We then tuned the `risk_behavior` parameter and tested antigen vs PCR testing again. This time, we used a `risk_behavior` value of 0.4 (meaning 40% of cases displaying symptoms will choose to self-isolate) and a value of 0.9 (90% of cases displaying symptoms choosing to self-isolate).
Even when 40% of those with symptoms were choosing to remain home, antigen tests were outperforming PCR non-rapid tests in diminishing transmission. When the value goes up to 90%, the difference between the tests diminishes, but antigen tests still outperform PCR tests, showing slightly fewer cumulative infections.
Keep in mind here that the y-axis scales differ between the two plots above. We see a higher number of infections with the configuration where `risk_behavior` = 0.4 than we do with the configuration where `risk_behavior` = 0.9. Although antigen tests result in less cumulative infections than do PCR tests in both cases, PCR testing with 90% of employees self-isolating with symptoms yields less infections than does antigen testing with only 40% of employees self-isolating with symptoms. This further showcases the importance of employees choosing to remain home with the onset of symptoms.
The next parameter we experimented with was the `testing_cadence` values. We kept the default antigen test values for the `testing_delay` and `sensitivity` parameters, and then observed how transmission rate was affected when we tested employees every 2 weeks, every week, every 4 days, 3 days, 2 days, everyday. Not surprisingly, the more frequent the testing, the lower the transmission rate.
It should be noted that in all of the above cases, we were randomly selecting who to test. Given that people with COVID-19 are more likely than not to display symptoms, we next observed how the testing prioritization criteria affects transmission. Instead of randomly selecting who to test, we observed how outbreaks would be affected by prioritizing symptomatic or asymptomatic cases first.
As seen above (top), when all employees are coming into the workplace, it makes sense to first test symptomatic cases. Does this change if more people are choosing to remain isolated when displaying symptoms? Yes, the difference diminishes when the `risk_behavior` value is changed to 0.9. If people with symptoms remain home, the testing prioritization method is less relevant.
In summary, the measures that can be taken to prevent COVID-19 outbreaks in the workplace include:
This current configurable model is the framework for organizations looking to assess relative risk depending on certain parameters. However, figuring out the specific values to use within the model is not a straightforward task. For the above analyses, we used a value of 0.2 for the `beta` parameter. This specific value will depend on a myriad of factors though, and will differ among different work sectors — a meatpacking plant will likely have a higher `beta` value than would a tech company. Policies on wearing masks, protocols around sanitization and ability to social distance in the workplace will affect this value. Vaccinations will decrease the value of this parameter as well — the amount by which depends both on the number of employees vaccinated, as well as the efficacy of the vaccines.
Similarly, the `external_infection_rate` will depend on variables such as the individuals’ locations and behaviors (i.e. do they take public transit regularly). To reliably calculate these values, we will further need to look at current literature on the subject as well as our Carbon Health dataset. We are prioritizing this research to make sure we are able to provide a reliable model, with up-to-date information.
Next steps for this project include creating a web application. This would allow an organization to input certain information and constraints, including budget, number of employees, sanitization practices, work from home and sick leave policies, distance between employees, etc. The application would then recommend testing cadence and testing prioritization strategies for that particular organization.
Although it is difficult to reliably gauge the exact values for certain parameters (for example, it is difficult to know the exact value for the `beta` parameter), this can still be used to assess relative risk and to compare two different policies (for e.g. testing prioritization strategies).
Ensuring workplaces and businesses are opening safely is important. With more data and careful analysis, we are able to better select between testing options and better inform] employees to self-isolate, thereby reducing or diminishing outbreaks.
If you have any suggestions or questions regarding this study, please reach us at data-science@carbonhealth.com.
Thanks to my colleagues at Carbon Health for their valuable feedback on this study:
Data Science Team: erenbali, Raymond Deng, Neha Kumar, Nosheen Moosvi, Hoda Noorian, Pardis Noorzad
Medical Team: Caesar Djavaherian, MD, Sujal Mandavia, Aaron Weinberg, Roger Wu
Clinical Informatics: Haritha Atluri
Marketing Team: Caitlin Heinz, Loren Mattia
[1] https://chris-said.io/2020/07/15/why-inaccurate-antigen-tests-are-better-than-slow-pcr-tests/
%2520(1).jpeg)
