Who Uses Bots? A Statistical Analysis of Bot Usage in Moderation Teams

This quantitative study examines how the use of user-innovated moderation bots is influenced by social and organizational factors, focusing on communities that transitioned from Reddit to Discord. Conducted as part of my work with the University of Washington's Department of Communication and the Community Data Science Collective, the research combines qualitative and quantitative methods to test hypotheses about the relationship between community size, team structure, and bot adoption. 

• Understand how community size and team dynamics impact bot adoption.

• Identify specific features of user-innovated bots that facilitate moderation tasks.

• Provide actionable insights for platform designers and community leaders to enhance moderation capabilities.

I served as the Lead Researcher on this project:

• Study Design: Developed hypotheses based on prior qualitative research.

• Data Collection: Sampled 300 Discord communities connected to Reddit, analyzing community size, team structure, and bot usage.

• Quantitative Analysis: Performed statistical modeling to test hypotheses about bot adoption.

• Publication: Authored the paper published in CHI 2020 Late Breaking Works and disseminated findings to both academic and practitioner audiences.

• Hypothesis Testing: Examined three key propositions:

  • Larger communities are more likely to adopt moderation bots.

  • Smaller moderation teams are more likely to use bots due to resource constraints.

  • Experience moderating on Reddit correlates with bot adoption.

• Data Management: Conducted rigorous data collection and organization from public and private sources (e.g., Reddit sidebars, Discord bot documentation).

  • Built subreddit data web scraper with Python and Reddit API.

• Tool Analysis: Investigated the prevalence of three bot features: automod, mod logs, and mod mail.

• Collaboration: Partnered with research colleagues for feedback and refinement of findings.

• Found that larger communities are significantly more likely to adopt bots with features such as mod logs and mod mail.

• Discovered no strong relationship between team size or prior Reddit experience and bot adoption, contrary to initial hypotheses.

• Highlighted the utility of bots in scaling governance tasks, especially for communities managing extensive membership bases.

• Provided recommendations for designing platform features that better support custom moderation tools.