LW - What happens if you present 500 people with an argument that AI is risky? by KatjaGrace

The Nonlinear Library

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What happens if you present 500 people with an argument that AI is risky?, published by KatjaGrace on September 4, 2024 on LessWrong.
Recently, Nathan Young and I wrote about arguments for AI risk and put them on the AI Impacts wiki. In the process, we ran a casual little survey of the American public regarding how they feel about the arguments, initially (if I recall) just because we were curious whether the arguments we found least compelling would also fail to compel a wide variety of people.
The results were very confusing, so we ended up thinking more about this than initially intended and running four iterations total. This is still a small and scrappy poll to satisfy our own understanding, and doesn't involve careful analysis or error checking. But I'd like to share a few interesting things we found. Perhaps someone else wants to look at our data more carefully, or run more careful surveys about parts of it.
In total we surveyed around 570 people across 4 different polls, with 500 in the main one. The basic structure was:
1.
p(doom): "If humanity develops very advanced AI technology, how likely do you think it is that this causes humanity to go extinct or be substantially disempowered?" Responses had to be given in a text box, a slider, or with buttons showing ranges
2.
(Present them with one of eleven arguments, one a 'control')
3.
"Do you understand this argument?"
4.
"What did you think of this argument?"
5.
"How compelling did you find this argument, on a scale of 1-5?"
6.
p(doom) again
7.
Do you have any further thoughts about this that you'd like to share?
Interesting things:
In the first survey, participants were much more likely to move their probabilities downward than upward, often while saying they found the argument fairly compelling. This is a big part of what initially confused us. We now think this is because each argument had counterarguments listed under it. Evidence in support of this: in the second and fourth rounds we cut the counterarguments and probabilities went overall upward.
When included, three times as many participants moved their probabilities downward as upward (21 vs 7, with 12 unmoved).
In the big round (without counterarguments), arguments pushed people upward slightly more: 20% move upward and 15% move downward overall (and 65% say the same). On average, p(doom) increased by about 1.3% (for non-control arguments, treating button inputs as something like the geometric mean of their ranges).
But the input type seemed to make a big difference to how people moved!
It makes sense to me that people move a lot more in both directions with a slider, because it's hard to hit the same number again if you don't remember it. It's surprising to me that they moved with similar frequency with buttons and open response, because the buttons covered relatively chunky ranges (e.g. 5-25%) so need larger shifts to be caught.
Input type also made a big difference to the probabilities people gave to doom before seeing any arguments. People seem to give substantially lower answers when presented with buttons (Nathan proposes this is because there was was a <1% and 1-5% button, so it made lower probabilities more salient/ "socially acceptable", and I agree):
Overall, P(doom) numbers were fairly high: 24% average, 11% median.
We added a 'control argument'. We presented this as "Here is an argument that advanced AI technology might threaten humanity:" like the others, but it just argued that AI might substantially contribute to music production:
This was the third worst argument in terms of prompting upward probability motion, but the third best in terms of being "compelling". Overall it looked a lot like other arguments, so that's a bit of a blow to the model where e.g. we can communicate somewhat adequately, 'arguments' are more compelling than rando...