Catching them red-handed: Real-time aggression detection on social media

Abstract

Aggression on social media has evolved into a major point of concern. However, recently proposed machine learning (ML) approaches to detect various types of aggressive behavior fall short, due to the fast and increasing pace of content generation as well as evolution of such behavior over time. This work introduces the first, practical, real-time framework for detecting aggression on Twitter via embracing the streaming ML paradigm. This method adapts its ML binary classifiers in an incremental fashion, while receiving new annotated examples, and achieves similar performance as batch-based ML models, with 82-93% accuracy, precision, and recall. Experimental analysis on real Twitter data reveals how this framework, implemented in Spark Streaming, easily scales to process millions of tweets in minutes.