Startup Develops AI Tool to Detect Email Harassment

2020-01-09

Since the rise of the Me Too movement at the end of 2017, various types of harassment incidents have received increasing attention, including workplace harassment and harassment through email or instant messaging. According to The Guardian, AI researchers and engineers have been developing a tool called MeTooBots to detect harassment through text communication. To identify potential harmful and harassing communication, companies around the world are implementing MeTooBots. One example is a robot created by Nex AI, which is currently being used by approximately 50 different companies. The robot utilizes an algorithm to examine company documents, chats, and emails, comparing them with training data on bullying or harassment messages, and sends harmful information to the human resources manager for review. On the other hand, Brian Subirana, an AI professor at MIT and Harvard University, believes that there are limitations in using AI to detect harassment. Harassment can be very subtle and difficult to detect, often manifesting as a pattern that only becomes apparent after examining a large amount of data. So far, robots have not been able to go beyond detecting certain trigger words. Despite the complexity of detecting harassment, Subirana believes that robots can play a role in combating online harassment. A potential problem information database can be created through robots, allowing for big data comparison and screening of harassment messages. Data and privacy confidentiality are major concerns. If this technology is abused, it could create an atmosphere of mistrust and suspicion. Methods for using robots to detect harassment while protecting anonymity and privacy must be jointly developed by robot developers, companies, and regulatory agencies. While protecting privacy, some methods can be used to maintain communication anonymity by utilizing the predictive capabilities of robots and AI. For example, robots can generate reports that only include the presence of potentially harmful language and the frequency of possible harassment language. In this way, the human resources department can determine whether the use of harassing language has decreased after awareness workshops or, conversely, whether there is an increase in harassment behavior that should be monitored.

RECENT AI NEWS

RECENT AI TOOLS