Zuckerberg fired the fact-checkers. We tested their replacement.

Washington Post

6 days ago

When a hoax about Donald Trump went viral at the funeral of Pope Francis, I went on social media to try to set the record straight. I'm a volunteer for community notes, a program Mark Zuckerberg announced in January that replaces fact-checkers with users to counter falsehoods on Facebook, Instagram and Threads.
I drafted a note debunking images claiming to show Trump sleeping during the ceremony. I cited live-stream footage and corroborating time stamps. I linked to research by Snopes.
None of that mattered. My community note never got added to a post because not enough other users voted it was 'helpful.'
Over four months, I've drafted 65 notes debunking conspiracy theories on topics ranging from airplane crashes to Ben & Jerry's ice cream. I've tried to flag fake artificial intelligence-generated video clips, viral hoax security threats and false reports about an ICE partnership with DoorDash.
Only three of them got published, all related to July's Texas floods. That's an overall success rate of less than 5 percent. My proposed notes were on topics other news outlets — including Snopes, NewsGuard and Bloomberg News — had decided were worth publishing their own fact checks about.
Zuckerberg fired professional fact-checkers, leaving users to fight falsehoods with community notes. As the main line of defense against hoaxes and deliberate liars exploiting our attention, community notes appear — so far — nowhere near up to the task.
Feeds filled with inaccurate information matter for the 54 percent of American adults who, according to Pew Research Center, get news from social media.
Zuckerberg's decision to fire fact-checkers was widely criticized as a craven attempt to appeal to President Donald Trump. He said Meta was adopting the crowdsourced community notes system used by Elon Musk's X because users would be more trustworthy and less biased than fact-checkers. Before notes get published to posts, enough users have to agree they're helpful. But agreement turns out to be more complicated than it sounds.
Meta says my test can't be used to evaluate its notes program, which has been public in the United States for more than four months. 'Community Notes is a brand new product that's still in the test-and-learn phase, and it takes time to build a robust contributor community. While there are notes continuously publishing across Threads, Instagram and Facebook, not every note will be broadly rated as helpful by the community — even if those notes were written by a Washington Post columnist,' said spokeswoman Erica Sackin.
Meta declined to answer my questions about how many notes it has published, how many participants are in the program or whether there's evidence it is making an impact, despite promising to be transparent about the program.
Alexios Mantzarlis, director of the Security, Trust and Safety Initiative at Cornell Tech, published another independent evaluation of community notes in June and said it was 'not yet ready for prime time.' He found that only a handful of proposed notes provided truly valuable context and some were inaccurate.
'It is concerning that four months in, they have shared no updates,' Mantzarlis told me.
This matters because community notes programs are spreading beyond X and Meta as a way for Big Tech to outsource the politically fraught work of moderating content. YouTube said it was testing a version last year. And TikTok said in April it was testing a system called Footnotes.
If community notes are becoming a standard for fighting falsehoods, we need to be honest about what they can and cannot do.
I volunteered to join Meta's community notes, and was initially allowed into the program on Threads, and eventually Instagram and Facebook, too.
Volunteers are able to tap a few buttons on any post from a U.S. user and suggest a note, complete with text and a link as proof. I deliberately drafted notes that crossed the political spectrum. For example, I suggested notes on a fabricated image of Pam Bondi seen over half a million times, as well as a false claim about the wealth of Alexandria Ocasio-Cortez seen a quarter of a million times.
I also voted on dozens of notes drafted by others, rating them 'helpful' or 'not helpful.'
Community notes have upsides. I've seen users trying to tackle a wider set of topics than a traditional fact-checker might be an expert in. Some are trying to fight lies in plain, easy-to-understand language.
But I discovered problems quickly. Sometimes, posts I identified for notes wouldn't accept them because they were written by accounts outside the U.S. (which are excluded from Meta's initial program) or had other technical problems. I've seen notes suggested by others that were low quality, some with more opinions than facts, or that sourced to 'Google it.'
The biggest challenge has been cutting through. I started contributing in April, and by early July nothing I had proposed had been published. Only one note that I had rated as helpful, written by someone else, had been published.
Meta says it is not cherry picking which notes get published. It uses a 'bridging algorithm' to try to determine which notes are actually helpful, as opposed to just popular. This formula requires contributors who have disagreed with each other on past notes to agree that a new note is helpful.
In theory, this is a good thing. You don't want to publish notes that contain falsehoods or are simply attacks on particular people or ideas.
However, agreement is tough to find. Notes I couldn't get published included facts that shouldn't be up for debate, including identifying AI deepfakes. This system also doesn't lend itself to the unique risks of breaking news and fast-moving viral conspiracies. (Meta does still remove some falsehoods directly, but in very specific instances when it can lead to physical harm or interfere with elections.)
Meta says it is using the same algorithm as X, which the rival social network has published. 'It's a very, very conservative system,' said Kolina Koltai, who helped develop community notes at Twitter, now called X. The algorithm is better at avoiding bad stuff than ensuring the good stuff actually gets published, she says.
Koltai, who is now a Bellingcat journalist, says her own personal publishing rate on X is about 30 percent — low, but still multiples higher than mine on Meta's social networks.
Is it possible my notes were too one-sided? I shared what I'd been proposing with Mantzarlis, who was also the founding director of the International Fact-Checking Network. He said they used 'non-polarizing language, clear context, and high quality sources.'
It's possible Meta hasn't been able to recruit a wide enough variety of users to meet the bridging algorithm's requirements. It could also be that the community note volunteers that Meta does have just aren't very engaged. I kept contributing for months, even though nothing I wrote was getting published. I can't imagine most unpaid users would bother to stick around.
Some of these factors could improve as Meta's program matures. But since Zuckerberg already fired the professional fact-checkers, the community notes system isn't just a test — it's our current main line of defense.
For users, it's a good idea to bolster our own defenses to social media lies.
But our best hope is that the teams inside Meta who care about the truth can evolve community notes to make them more effective.
It could begin allowing people to draft notes on posts written by overseas accounts. It could develop a triage system to put notes on certain topics or likely to cause more harm higher in the queue. It could also improve engagement by giving contributors clout — maybe a badge? — for repeat submissions.
Still, the reality is, randomly selected volunteers just can't do this work alone.
Twitter originally launched community notes alongside professional fact-checkers. 'It was never, ever meant to be the solo thing,' says Koltai.
Musk also eventually ended X's paid fact-checking program, and community notes have struggled to keep up with the tide of falsehoods. An analysis last year ahead of the presidential election found the majority of accurate notes proposed by X users on political posts were never shown to the public.
Fact-checkers might make mistakes, as Zuckerberg has said, but they can also check each other as part of a community.
Meta could give experts specialized profiles within community notes once they've cleared some kind of credential — and maybe also let them get paid for their work. 'Professionals and the crowd are not contradictory, they can be complementary,' says Mantzarlis.
Andrea Jimenez contributed this report.