We know there are many challenges involved in building aan open, participatory system like Birdwatch — from making it resistant to manipulation attempts to ensuring it isn’t dominated by a simple majority or biased based on its distribution of contributors.
Throughout our development process, we’ve conducted feedback sessions with Twitter customers and subject area experts to identify critical risks to monitor and mitigate. We will be closely focused on understanding and identifying solutions to prevent such risks during our initial pilot phase.
We have intentionally designed our pilot to mitigate potential risks as we seek to learn about them and iterate on solutions carefully. We are starting with a small number of contributors. Further, in this first phase, notes will not affect Tweets or recommendations systems on Twitter, and are only visible within the app to Birdwatch contributors and small test groups — those not in the pilot can see notes on a separate Birdwatch site.
Here are a handful of particular challenges we are aware of as well as our thoughts on how we might address them over time:
Attempts at coordinated manipulation represent a crucial risk for open rating systems. We expect such attempts to occur, and for Birdwatch to be effective, it needs to be sufficiently resistant to them.
During this initial phase, we are taking some steps to reduce this type of abuse in the pilot: First, people must meet eligibility criteria to become a Birdwatch contributor. Additionally, we will prioritize admitting new contributors who tend to follow and engage with different accounts and Tweets than existing contributors do, so as to reduce the likelihood of bias from predominantly one ideology, background, or interest space.
As we iterate on Birdwatch, we plan to test new ways to address adversarial coordination. For example, Birdwatch factors in not just a count of how many people said a note is helpful but also the diversity of those inputs. Additionally, Birdwatch has a reputation system in which one earns reputation for contributions that people from a wide range of perspectives find helpful. We believe requiring a diverse range of inputs and giving more weight to inputs from contributors with consistent track records can reduce the impact of coordinated manipulation attempts.
Birdwatch will only be effective if the Tweet context it produces is found to be helpful and appropriate by a wide range of people with diverse views. To achieve this, we’re designing Birdwatch to encourage contributions from people with diverse perspectives, and to reward contributions that are found helpful by a wide range of people.
For example, rather than ranking and selecting top notes by a simple majority vote, Birdwatch considers how diverse a note’s set of ratings is and determine whether additional inputs are needed before a consensus is reached. Additionally, Birdwatch can proactively seek ratings from contributors who are likely to provide a different perspective based on their previous ratings.
Further, Birdwatch has a reputation system in which one earns reputation for contributions that people from a wide range of perspectives find helpful.
Finally, to assess Birdwatch’s effectiveness, we regularly evaluate whether people who use Twitter find Birdwatch context to be helpful and informative. This can serve as an indicator of its ability to reflect and be of value to people from a wide range of perspectives.
It’s crucial that people feel safe contributing to Birdwatch. Because of that, all Birdwatch contributors get a new, auto-generated display name (or alias) when they join Birdwatch. These aliases are not publicly associated with contributors’ Twitter accounts, so everyone can write and rate notes privately.
You can learn more about aliases here.
Contributors also have an open communication line with the Birdwatch team to report any issues they experience (they can reach us by DM @Birdwatch). Birdwatch has a dedicated community manager who gathers and responds to contributors’ feedback or concerns via this handle.
As Birdwatch grows and evolves, we will continue to iterate to ensure it remains a place for healthy conversation, where our contributors feel safe participating.
We welcome feedback on these or additional risks and challenges, as well as ideas for addressing them. Please DM us at @Birdwatch.