We know there are many challenges involved in building a community-driven system like Birdwatch — from making it resistant to manipulation attempts to ensuring it isn’t dominated by a simple majority or biased based on its distribution of contributors.
Throughout our development process, we’ve conducted feedback sessions with Twitter customers and subject area experts to identify critical risks to monitor and mitigate. We will be closely focused on understanding and identifying solutions to prevent such risks during our initial pilot phase.
We have intentionally designed our pilot to mitigate potential risks as we seek to learn about them and iterate on solutions carefully. We are starting with a small number of participants. Further, in this first phase, notes will only be visible on a separate Birdwatch site and will not affect Tweets or recommendations systems on Twitter.
Here are a handful of particular challenges we are aware of as well as our thoughts on how we might address them over time:
Attempts at coordinated manipulation represent a crucial risk for community rating systems. We expect such attempts to occur, and for Birdwatch to be effective, it needs to be sufficiently resistant to them.
During this initial phase, we are taking some steps to reduce this type of abuse in the pilot: First, people must meet eligibility criteria to participate in the pilot. Additionally, we will prioritize admitting new participants who tend to follow and engage with different accounts and Tweets than existing participants do, so as to reduce the likelihood of bias from predominantly one ideology, background, or interest space.
As we iterate on Birdwatch, we plan to test new ways to address adversarial coordination. For example, Birdwatch can factor in not just a count of how many people said a note is helpful but also the diversity of those inputs. Additionally, we plan for Birdwatch to have a reputation system in which one earns reputation for contributions that people from a wide range of perspectives find helpful. We believe requiring a diverse range of inputs and giving more weight to inputs from contributors with consistent track records can reduce the impact of coordinated manipulation attempts.
Birdwatch will only be effective if the Tweet context it produces is found to be helpful and appropriate by a wide range of people with diverse views. To achieve this, we’re designing Birdwatch to encourage contributions from people with diverse perspectives, and to reward contributions that are found helpful by a wide range of people.
For example, rather than ranking and selecting top notes by a simple majority vote, Birdwatch can consider how diverse a note’s set of ratings is and determine whether additional inputs are needed before a consensus is reached. Additionally, Birdwatch can proactively seek ratings from contributors who are likely to provide a different perspective based on their previous ratings.
Further, we plan for Birdwatch to have a reputation system in which one earns reputation for contributions that people from a wide range of perspectives find helpful.
We are also focused on increasing the diversity of participants in our initial pilot phase. If we have more applicants than pilot slots, we will randomly admit accounts, prioritizing accounts that tend to follow and engage with different audiences and content than those of existing participants, so as to reduce the likelihood that participants would be predominantly from one ideology, background, or interest space. We will do the same for new sets of participants admitted after the initial pilot phase.
Finally, to assess Birdwatch’s effectiveness, we will evaluate whether people find Birdwatch context to be helpful and appropriate. This can serve as an indicator of its ability to reflect and be of value to people from a wide range of perspectives.
It’s crucial that people feel safe contributing to Birdwatch, and this is something to which we will be paying close attention throughout the pilot. While community moderation can be a powerful tool, it can also result in bias and harassment. One of the reasons Birdwatch is starting very small is so we can understand contributors’ experience and make changes if needed.
Pilot participants have an open communication line with the Birdwatch team to report any issues they experience (they can reach us by DM @Birdwatch). Birdwatch will also have a dedicated community manager who will be gathering and responding to contributors’ feedback or concerns via this handle. Additionally, people have the option to participate in Birdwatch under any account, including secondary accounts that use a pseudonym.
As Birdwatch grows and evolves, we will continue to iterate to ensure it remains a place for healthy conversation, where our contributors feel safe participating.
We welcome feedback on these or additional risks and challenges, as well as ideas for addressing them. Please DM us at @Birdwatch.