The proliferation of harmful content on online platforms is a significant societal problem that takes various forms, including hate speech, offensive language, bullying and harassment, misinformation, spam, violence, graphic content, sexual abuse, self-harm and more. Online platforms strive to moderate such content to limit societal harm, comply with legislation and create a more inclusive environment for users. However, there is currently a dichotomy between the types of harmful content that online platforms aim to curb and the research efforts dedicated to automatically detecting such content. To address this issue comprehensively, researchers have developed different methods for automatically detecting harmful content. These methods often focus on specific sub-problems or narrow communities since what is considered harmful can vary depending on the platform and context. Despite these efforts, there has been no comprehensive work covering all aspects of harmful content detection. In terms of datasets used for training and evaluation purposes in this field of research Vidgen & Derczynski (2021) categorized 64 datasets primarily focused on abusive language detection. They found that around half of these datasets were limited to English only; some datasets covered European languages like Hindi and Indonesian while six datasets specifically targeted Arabic language content. Twitter was the primary source of data for most datasets; however other social networks were also utilized. When it comes to specific tasks within harmful content detection research:
- Hate Speech Detection: This task has received significant attention in research (Waseem & Hovy 2016; Kwok & Wang 2013; Burnap & Williams 2015). Several shared tasks have been organized in this area as well (e.g., HateEval Basile et al 2019). Multimodal approaches have also been explored using memes as a medium for hate speech detection (Sabat et al 2019). - Offensive Language Detection: The dataset by Davidson et al (2017) containing over 24000 English tweets has been widely used for this task. Multiple shared tasks and associated datasets have focused on offensive language identification in various languages (e.g., OffensEval 2019–2020 GermEval 2018 HASOC 2019). - Aggression Detection: The TRAC shared task on Aggression Identification provided a dataset for discriminating between non-aggressive covertly aggressive and overtly aggressive content (Kumar et al 2018).
- The proliferation of harmful content on online platforms is a significant societal problem.
- Harmful content takes various forms, including hate speech, offensive language, bullying and harassment, misinformation, spam, violence, graphic content, sexual abuse, self-harm, and more.
- Online platforms strive to moderate such content to limit societal harm and comply with legislation.
- There is currently a dichotomy between the types of harmful content that online platforms aim to curb and the research efforts dedicated to automatically detecting such content.
- Researchers have developed different methods for automatically detecting harmful content.
- These methods often focus on specific sub-problems or narrow communities since what is considered harmful can vary depending on the platform and context.
- There has been no comprehensive work covering all aspects of harmful content detection.
- Around half of the datasets used for training and evaluation in this field are limited to English only.
- Some datasets cover European languages like Hindi and Indonesian while six datasets specifically target Arabic language content.
- Twitter is the primary source of data for most datasets in this field, but other social networks are also utilized.
- Hate Speech Detection has received significant attention in research and several shared tasks have been organized in this area. Multimodal approaches using memes as a medium for hate speech detection have also been explored.
- Offensive Language Detection has a widely used dataset containing over 24,000 English tweets. Multiple shared tasks and associated datasets have focused on offensive language identification in various languages.
- Aggression Detection had a shared task on Aggression Identification that provided a dataset for discriminating between non-aggressive covertly aggressive and overtly aggressive content.
Key points
1. Harmful content on online platforms is a big problem in society.
2. Harmful content can be things like mean words, bullying, lies, violence, and more.
3. Online platforms try to stop harmful content and follow the rules.
4. Researchers are trying to find ways to automatically detect harmful content.
5. Different methods focus on different problems or groups of people.
Definitions
- Proliferation: When something spreads quickly and becomes more common.
- Harmful: Something that can hurt or cause problems for people.
- Content: Things like words, pictures, videos, or information that is shared online.
- Platforms: Websites or apps where people can share things with others.
- Moderation: When someone checks and controls what is shared on a platform to make sure it's okay.
- Legislation: Laws or rules made by the government that everyone has to follow.
- Dichotomy: A difference between two things that are opposite or very different from each other.
- Detecting: Finding or discovering something using special methods or tools.
- Datasets: Collections of information used for research or testing purposes.
- Multimodal approaches: Ways of doing something that use different types of media (like pictures and words) together.
Harmful Content Detection on Online Platforms: A Comprehensive Overview
Online platforms are increasingly becoming a part of our everyday lives, and with that comes the need to ensure user safety. Harmful content on these platforms can take many forms, such as hate speech, offensive language, bullying and harassment, misinformation, spam, violence, graphic content and more. As a result, online platforms strive to moderate this type of content in order to limit societal harm and create an inclusive environment for users. However, there is currently a dichotomy between the types of harmful content that online platforms aim to curb and the research efforts dedicated to automatically detecting such content. To bridge this gap comprehensively researchers have developed different methods for automatically detecting harmful content.
Datasets Used in Research
In terms of datasets used for training and evaluation purposes in this field of research Vidgen & Derczynski (2021) categorized 64 datasets primarily focused on abusive language detection. They found that around half of these datasets were limited to English only; some datasets covered European languages like Hindi and Indonesian while six datasets specifically targeted Arabic language content. Twitter was the primary source of data for most datasets; however other social networks were also utilized.
Specific Tasks Within Harmful Content Detection Research
When it comes to specific tasks within harmful content detection research several approaches have been explored:
- Hate Speech Detection: This task has received significant attention in research (Waseem & Hovy 2016; Kwok & Wang 2013; Burnap & Williams 2015). Several shared tasks have been organized in this area as well (e.g., HateEval Basile et al 2019). Multimodal approaches have also been explored using memes as a medium for hate speech detection (Sabat et al 2019).
- Offensive Language Detection: The dataset by Davidson et al (2017) containing over 24000 English tweets has been widely used for this task. Multiple shared tasks and associated datasets have focused on offensive language identification in various languages (e.g., OffensEval 2019–2020 GermEval 2018 HASOC 2019).
- Aggression Detection: The TRAC shared task on Aggression Identification provided a dataset for discriminating between non-aggressive covertly aggressive and overtly aggressive content (Kumar et al 2018).
In conclusion it is clear that there is still much work needed when it comes to developing comprehensive methods for automatically detecting all aspects of harmful online platform activity due to its varying nature across different contexts and communities but progress is being made through initiatives such as shared tasks which provide access to large amounts of data necessary for machine learning models used in these areas
