A new Yale School of Public Health study found that slot machines, or “one-armed bandits,” may offer a clue to how AIDS programs can better locate persons living with undiagnosed HIV infection.
“When you walk into a casino and see a row of slot machines how do you decide which one to play and when it’s time to switch to another? What’s the best strategy to maximize your winnings?” asks Gregg Gonsalves, Ph.D., assistant professor of epidemiology at Yale, and lead author on the paper published in Medical Decision Making. “Mathematicians have created strategies called “bandit” algorithms to answer these questions – and we’ve used one of them as the basis for our approach to HIV testing.”
In the quest to diagnose individuals who don’t know that they are HIV infected and can unknowingly transmit the virus, some AIDS programs have started to focus on “hotspots” of HIV infection. However, pinpointing the hotspots in which to conduct testing remains elusive. “Different methods have been proposed to identify hotspots, but there is no consensus on where to look,” explains Gonsalves. “Even setting up where common sense dictates — like places with high HIV prevalence or concentrations of people at risk of infection -- doesn’t guarantee a high yield of new diagnoses.”
So, Gonsalves says they decided to try something different: to see if they could get hotspots to reveal themselves by using the logic of a bandit algorithm called Thompson Sampling (TS). A gambling strategy based on TS has gamblers making bets based on current information, while constantly collecting new information from successes or failures at the slot machine in use. Instead of applying this to slot machines, the bandit algorithm offers AIDS programs the ability to select a hotspot based and up to date understanding of the likelihood of success in that area – or the best odds.
Yale researchers applied this bandit algorithm to different geographic zones targeted for HIV testing. The hope is that TS can help AIDS programs decide where to test for HIV based on information in different zones, which gets updated every time a testing team comes back with new information on HIV positive and HIV negative tests in a specific area.
When you walk into a casino and see a row of slot machines how do you decide which one to play and when it’s time to switch to another? Mathematicians have created strategies called “bandit” algorithms to answer these questions – and we’ve used one of them as the basis for our approach to HIV testing.
Gregg Gonsalves, Ph.D
The study, in the format of a computer simulation, is an online competition between TS and several other strategies, including one that simply sends testing teams to locales with the highest HIV prevalence from the previous year, and another based on doing limited, preliminary testing in all the areas under consideration and then picking the one with the most new diagnoses to test in for the duration of the competition. Findings showed TS located more new HIV cases then the other two strategies. “If you were using Thompson Sampling in a casino, a player wouldn’t make a large bet on one machine, they would play for a while to hone in on the best machine to play their quarters.”
While bandit methods have been applied successfully to a variety of problems where finding hidden objects is important, such as searching for downed airplanes or good places for oil exploration, they haven’t been used in the context of screening for infectious disease, which makes the application of these methods unique.
Gonsalves says the next step for this project is to put the algorithm to test in the real world, to see if it indeed can offer a better way for AIDS programs to choose where to test for new HIV cases, using mobile vans, or outreach teams.
According to the Center for Disease Control and Prevention, 14% of the 1.2 million individuals in the United States living with HIV infection are undiagnosed. With as many as two-thirds of new HIV transmissions attributable to individuals who are unaware of their HIV infection. Moreover, undiagnosed individuals initiate antiretroviral therapy later in their disease compromising their health and well-being.