computer vision can help us be ready for the next pandemic
The World Context
It’s not like we weren't warned
In 2015, Bill Gates delivered a TED Talk about the global lack of preparedness for a novel viral pandemic. During this talk, he said:
“If anything kills over 10 million people in the next few decades, it’s most likely to be a highly infectious virus rather than a war.”
Prescient words and Gates was by no means the only voice delivering such warnings, yet it’s now clear the global community was woefully unprepared for Covid-19. Now the true implications of a global pandemic are no longer theoretical, a question we should be asking ourselves is:
‘How are our societies going to prepare for another pandemic now we know they can happen in this modern era?’
It’s not been the best start to a decade
It’s August 2020 and here we are 8 months into the first Pandemic of this century (yes there will probably be others) and it’s not gone well so far:
Nearly 700,000 (confirmed) people have died as a result of Covid-19 infection
The drop in World GDP is now greater than during the 2012 Global Financial Crisis.
There’ll probably be a vaccine but even when there is one, it will take years to manufacture enough to inoculate 7 billion people
What this means for the world in the long run isn’t clear but it would be difficult to dispute the following:
There’s going to be a ‘deep’ global recession with an unknown recovery period.
Some things we’ve taken for granted over the past 20-50 years are never going to be the same again.
Pandemics in our modern world are now no longer a theoretical problem.
Assuming there’s a vaccine….
So, let’s imagine a world where we have a vaccine and/or most nations have achieved herd immunity - Covid-19 is unlikely to have been eliminated but it is now manageable like influenza. International travel has recovered to some extent and people are allowed to gather in groups again
Assuming the above to be true, we are asking ourselves this question:
How does a society get to behave normally while being ready to react to a new pandemic?
There’s probably a couple of doctoral theses to write to even make a stab at answering that question properly but let’s use bullet points instead!
A society should:
Have systems to quickly identify any potential new pathogens and the risk they pose
Have controls in place to quickly limit the opportunity for community spread commensurate with the degree of risk the pathogen presents
Develop rigorous and well-tested mechanisms of isolating potential cases
Retain an array of rapidly deployable and scalable testing facilities and associated services
Have plans in place to allow as many critical social, supportive and economic activities to continue during a pandemic response as practical. eg. Homeschooling, home retail deliveries, etc.
In the context of the above, Rush is thinking about how software (and in particular computer vision) can help support these goals.
Measuring Social Distancing Behaviours
Planning for a yet-to-happen event is difficult if you can’t first establish the likelihood and impact of the event. In order to do this, you need to create a ‘risk profile’ that helps you define the magnitude of your planned response to the event.
If a government is to understand what the transmission risk profile is for a given pathogen, the key questions about the pathogen are:
How easily does the pathogen spread?
How serious are the effects of the pathogens?
They then need to put these into the context of operational and behavioural baselines such as:
Macroscopic social behaviour of the populace
Overall healthiness and pathogen susceptibility of the populace
The health system's capacity to manage an influx of infected people
These baselines are critical to understanding the risk profile. For example, if the pathogen is only communicable by physical contact, a society which is more prone to casual physical contact would have a much higher risk profile than one that wasn’t and those societies should plan accordingly.
How do we measure behavioural baselines?
Our focus in this section is going to be on the area where it’s currently hardest to get quality, up-to-date data, eg the Macroscopic social behaviour of a whole society.
One of the defining challenges of assigning metrics to human social behaviours is that the metrics are dramatically affected by so many secondary factors and (somewhat like schrodinger's cat) the overt act of observation will usually affect a human’s behaviour thereby biasing the observation.
However with the approach detailed below, Rush believes that all the pieces of the jigsaw are there to create a continuously updated baseline for macroscopic human social behaviour.
What do we mean by Macroscopic Social Behaviour?
For the sake of brevity, we’ll just concentrate on the key behaviours related to spreading diseases, the specific behaviours we’re interested in are:
How close people are getting to each other
How long they are spending in close proximity to each other
What surfaces are people touching repeatedly
What are the hygiene habits of the populace as a whole?
For example, the chance of contracting Covid-19 from a contagious person increases dramatically as you spend more time within a one-meter radius of them and this is the principal driver for using social distancing to reduce the rate of spread of the virus as illustrated below:
Measuring en masse
So, what are the requirements for measuring the physical social behaviour of a given group or community?
You need to sample enough of the community to have confidence you can establish a normalised baseline of their behaviour.
You need to avoid influencing the populace’s behaviour when gathering the data.
You need to sample frequently enough to adjust for any short or long term changes in behaviour.
With a goal to measure the behaviour of a significant section of your population, it would be financially and logistically impractical to develop and deploy any type of new physical infrastructure to manage this so we must focus instead on how we can utilise existing infrastructure.
At Rush, we see the obvious candidate for this type of measurement as CCTV cameras.
There’s a raft of reasons why cameras make sense for this purpose but the principal one is that wherever there are large numbers of people, you will find cameras eg:
What about privacy?
We can’t put this idea forward without first evaluating the human rights and privacy consequences of mass surveillance and we will cover our ideas on how to fully anonymise surveillance imagery in a future post. But for the sake of brevity we want to emphasise that nothing we are proposing would be concerned with identifying individuals or storing imagery.
In terms of creating a ‘social distancing metric’, we only care about how close people are getting to each other, not who they are. You’ll see in the next section how surveillance footage can quickly be ‘washed’ of any information that might allow an individual to be identified while retaining the critical proximity information.
Transforming imagery into metrics
By applying a combination of state-of-the-art machine learning and some clever post processing, we can quickly establish how close people are getting to each other in a given image. An excellent example of this approach is this post by Basile Roth. The gif below is taken from his post.
In brief, the approach Basile has taken is as follows:
A ground plane for the images established. This allows the location of any people detected in the image to be estimated relative to each other.
A ‘human detection’ machine learning algorithm is used to detect any humans in the scene and evaluate where their feet are.
The information about where the detected people’s feet are (relative to the ground plane) can be transformed into a top-down view of the area
The position of each human relative to each other can be evaluated - In this case by drawing a 2meter radius circle around each human.
You’ll see in the example above that only 2 of 6 people in the scene are maintaining a 2m social distance.
Turning metrics into baselines
Now, let’s assume we have an analytic like this running on every street camera in a city 24/7-365. Using the acquired data, as shown above. We can generate a social distancing metric for every area covered by a camera, detailing the number of humans detected during a given time period and the average time they spend within a given radius of other humans.
A visualisation of that might look something like this on a dashboard:
A solution like this could allow a city operations team to see, in real-time where people are getting too close to each other and thereby focus their attention in these areas, making the best use of their resources for the maximum effect.
Extrapolating past a manual intervention, with the introduction of some simple automatic systems, pedestrians could be informed that they are getting too close to each other in the area in question using digital signage, push notifications or even loud-speaker/tannoy systems. With this kind of system, you might be able to reduce the likelihood of contamination by actively reminding people to maintain their distance before they become infected.
As we look towards a ‘pandemic aware’ and more austere future we are going to need to find ways to utilise the infrastructure we already have to better understand our societies behavioural trends and to ultimately better inform and prepare ourselves for future pandemics.