One Scary Way Face-Capture Apps Like FaceApp Can Be Used to Manipulate Voters: By Morphing a User's Face With the Face of a Preferred Candidate

Above: Images created in Balienson and Yee's Stanford study, subtly morphing the face of 2008 Presidential candidates with the test subject's face, significantly changed the political choice of many test subjects

There's been much speculation and satirical humor over the viral popularity of FaceApp, the Russian-owned "see you how you look old" app with a dubious Terms of Service which claim a "transferable sub-licensable license to use, reproduce, modify, adapt, publish" users' photos. But can modifying user photos really be used for nefarious goals -- for example, to secretly influence the next US Presidential election?

Yes, it can.

Last decade, some highly-regarded Stanford academics, including Jeremy Balienson and Nick Yee, conducted an amazing if fairly terrifying experiment which did just that.

"Between 2004-2008," as Nick Yee recently explained in this Facebook post, "we ran a series of studies where we morphed people's faces into the faces of presidential candidates (at a 35:65 ratio). We found that people are more likely to vote for candidates that look more like them (even in high-stakes/information races like the 2004 Kerry/Bush election), and this photo manipulation was largely undetectable at a 35% ratio."

Emphasis mine. As you can see above in the images under the 35:65 Blend column, the photos of candidates Hillary Clinton and John Edwards have been very subtly morphed with the test subjects in Balienson's experiment.

Doing that was enough to change their overall choice:

In one experiment Nick and his colleagues conducted before the 2004 election, this face morphing changed a plurality preference for John Kerry to a majority preference for George W. Bush. (See excerpt above, from the study).

"Looks like a 9% difference," Yee tells me now, adding some caveats: "We had a control condition that matched actual vote outcome. So it depends on whether you're comparing with non-manipulated control, or manipulation on both sides. Of the 3 studies we ran, that one had the strongest effect. So it may be something about two older white men running against each other. In the other studies that had broader mix of gender and ethnicity, the effect was lower."

So theoretically, could an app like FaceApp micro-target a specific user with a campaign ad where the hackers' preferred candidate has been subtly morphed to look like the user?

"Theoretically ... yes," Nick Yee tells me. "With social network data, it could also theoretically morph a candidate with someone you know (who is the right gender for example). Like say, you're male, but the candidate is female. We could morph the candidate with your mom's photo."

Balienson and his colleagues conducted their study before the growth of social networks and the rise of smartphones -- but now, the technology and mass market penetration exists to capture tens of millions of peoples' photos.

"I'm not saying that FaceApp is actually doing this," Nick emphasizes in his Facebook post, "but it strikes me that they've solved the hardest part of what we had to do manually--getting people to upload multiple hi-res photos of themselves with specific angle/lighting/pose parameters.

"I think we're at the point where this entire process can be automated: from photo acquisition to face blending in a targeted ad. In 2008, it was totally not obvious how you would acquire the faces of people at scale.

"In 2019, we've figured out a way to get millions of people to do it for fun."

Much more here.

Comments

[Maddy]

I think John Edwards is definitely prettier.