The Invisible Threat: How AI-Generated Harmful Content Erodes Trust and Drains Ad Budgets

AI-generated fake videos are spreading rapidly, misleading vulnerable viewers like the elderly. These videos evade traditional filters, causing ads to appear on harmful content and wasting ad spend. A healthier ad ecosystem requires better content detection and clearer insight into where ads are placed.

In today's digital landscape, video content has become an indispensable part of our daily lives, projected to make up 80-90% of global internet traffic by 2027. Yet, lurking in the shadows of this vast digital space is a new and unsettling threat: harmful or deceptive content generated by artificial intelligence (AI) technology. Recent reports of "AI fake documentaries" circulating on online platforms, with outlandish titles like "Man Who Impregnated 32 Women" or "50-Something Korean Man Impregnated Three Mongolian Beauties," reveal a shocking reality.

Source: YouTube

These videos, crafted by combining AI image generators, synthesized narration, and provocative captions and thumbnails, are racking up millions of views despite their preposterous narratives. The alarming issue is that some elderly viewers are mistaking these fabricated videos for real events. Comments such as "Lucky man, I'm envious" and "Solution to population decline" underscore the severity of this misunderstanding. This phenomenon cunningly exploits the low digital literacy among the elderly and their vulnerability to information. Furthermore, the societal impact of AI-generated content cannot be ignored, as these videos are even consumed in public spaces, causing discomfort to those nearby. The spectrum of harm these videos inflict is vast, presenting themselves as "life wisdom" or "life lessons," while also incorporating racist tropes, excessive sexual objectification, and distorted fantasies about age gaps.

The Proliferation of AI-Generated Content and the Advertiser's Dilemma

The unchecked proliferation of such AI content presents advertisers with a significant, often unseen dilemma. In the digital advertising market, the "context" in which an ad appears remains a challenging issue for many advertisers. Traditional brand safety measures have primarily relied on keywords or metadata, but AI-generated content cleverly sidesteps these filters. This is because even sensational content can be disguised with ordinary titles and descriptions, or by subtly rephrasing specific words to bypass filtering mechanisms. Moreover, certain AI audio novels, which have minimal visual harmfulness, are difficult to detect with existing visual-centric filtering systems. 

This opaque and imprecise video inventory environment directly leads to financial losses for advertisers. According to an analysis by Pyler utilizing a Video Understanding AI model on approximately 10,000 campaigns and over 100 million ad placements, it's estimated that a typical brand spends 20% to 30% of its ad budget on video ads placed within unsafe or unsuitable content. This exposure to harmful videos directly translates into wasted ad spending and brand safety concerns for advertisers. Beyond mere budgetary waste, sponsoring creators of harmful videos can draw criticism regarding a company's corporate social responsibility (CSR). Ultimately, this can damage brand value and negatively impact long-term profitability.

The advancement of AI technology is a double-edged sword. While it offers groundbreaking opportunities, it also serves as a tool for mass-producing harmful or deceptive content. Now, more than ever, we must clearly recognize what to trust, how to protect ourselves in the "fake world" created by AI, and precisely where our valuable advertising budgets are being spent. Acknowledging and understanding these complex, multi-layered issues is the crucial first step toward building a healthy and responsible digital advertising environment.