Post approvals in online communities
By: Justin Cheng, Manoel Horta Ribeiro, Robert West
Justin Cheng is a Research Scientist on Facebook Core Data Science (CDS), Manoel Horta Ribeiro is a PhD student at EPFL and 2021 Meta Research PhD Fellow, and Robert West is an Assistant Professor at EPFL and 2013 Meta Research PhD Fellow.
Moderation in online communities increases the quality of contributions and decreases antisocial behavior, and the tools that online platforms provide are one way that moderation can occur. In Facebook Groups, post approvals allow administrators and moderators to proactively filter undesired content by requiring posts to be approved before publication.
Figure 1: Post approvals allow posts that violate a community's guidelines (in red) to be filtered before other members in the community see them. Without post approvals, these posts can only be moderated after they are posted in the group.
In our latest paper, “Post Approvals for Online Communities,” we explored why groups chose to adopt this feature, and how it affected user behavior in these groups. This paper was presented at the International Conference on Web and Social Media (ICWSM).
What we did
We analyzed data from over 200 thousand Facebook groups between March and July 2021, comparing groups that enabled post approvals during this period with those that did not. For each group, we analyzed factors related to the way moderation was carried out (e.g., the number of posts removed per day in each group) and to the overall activity in the group (e.g., number of posts in the group per day). To ensure that groups were comparable, we used a technique called propensity score matching — for each group in our study that went on to enable post approvals during the study period, we found a comparable group that did not enable the setting.
With this data, we set out to explore three research questions:
- When and why do communities adopt post approvals?
- How does post approvals impact the communities that adopt it? and;
- How does the setting impact different kinds of communities (e.g., small vs. bigger groups)?
What we found
Our findings paint a nuanced picture of how participation controls such as post approvals shape the way online communities function. As shown below, communities that adopted post approval experienced greater growth in user and moderation activity in the weeks leading up to the enabling of post approvals. And right before post approvals were enabled, communities experienced a sudden increase in moderation, suggesting that administrators may have turned on post approvals in response to a significant negative event.
Figure 2: Average values for user activity- and moderation-related variables in the four weeks before communities enabled post approvals. Values for communities that enabled post-approvals (PA-ON) are in red and those for communities that did not (PA-OFF) are in blue. The shaded area represents the period used for matching PA-ON and PA-OFF communities. Error bars represent 95% confidence intervals.
To examine what happened in communities that adopted the setting, we compared the changes in activity in communities that enabled post approvals with the changes in activity in similar communities that did not. After adopting post approvals, communities became centered around fewer, higher-quality posts, while not significantly increasing the average time community leaders spent in those groups.
Figure 3: Standardized effect of enabling post approvals on user activity- and moderation-related variables. Error bars represent 95% confidence intervals. The data shows that post approvals reduces the number of posts in the group but also reduces the likelihood of a post being reported or deleted.
Last, we found that the communities may react differently to the adoption of post approvals. For example, group size mattered in determining whether community leaders spent more or less time in the community following the adoption of the feature. In larger communities, enabling post approvals increased the time administrators and moderators spent in the group; in smaller communities, enabling post approvals decreased time spent.
Next steps
This work represents an initial exploration of how one moderation tool – post approvals – may affect the way that communities work. Beyond deeper research on post approvals, understanding their long-term effects or understanding how other tools impact communities on different platforms remains important future work.
