‘Tool for grifters’: AI deepfakes push bogus sexual cures

“AI is a useful tool for grifters looking to create large volumes of content slop for a low cost,” misinformation researcher Abbie Richards told AFP. (Reuters)
Short Url
Updated 12 May 2025
Follow

‘Tool for grifters’: AI deepfakes push bogus sexual cures

  • The trend underscores how rapid advances in artificial intelligence have fueled what researchers call an AI dystopia, a deception-filled online universe designed to manipulate unsuspecting users into buying dubious products

WASHINGTON: Holding an oversized carrot, a brawny, shirtless man promotes a supplement he claims can enlarge male genitalia — one of countless AI-generated videos on TikTok peddling unproven sexual treatments.

The rise of generative AI has made it easy — and financially lucrative — to mass-produce such videos with minimal human oversight, often featuring fake celebrity endorsements of bogus and potentially harmful products.

In some TikTok videos, carrots are used as a euphemism for male genitalia, apparently to evade content moderation policing sexually explicit language.

“You would notice that your carrot has grown up,” the muscled man says in a robotic voice in one video, directing users to an online purchase link.

“This product will change your life,” the man adds, claiming without evidence that the herbs used as ingredients boost testosterone and send energy levels “through the roof.”

The video appears to be AI-generated, according to a deepfake detection service recently launched by the Bay Area-headquartered firm Resemble AI, which shared its results with AFP.

“As seen in this example, misleading AI-generated content is being used to market supplements with exaggerated or unverified claims, potentially putting consumers’ health at risk,” Zohaib Ahmed, Resemble AI’s chief executive and co-founder, told AFP.

“We’re seeing AI-generated content weaponized to spread false information.”

The trend underscores how rapid advances in artificial intelligence have fueled what researchers call an AI dystopia, a deception-filled online universe designed to manipulate unsuspecting users into buying dubious products.

They include everything from unverified — and in some cases, potentially harmful — dietary supplements to weight loss products and sexual remedies.

“AI is a useful tool for grifters looking to create large volumes of content slop for a low cost,” misinformation researcher Abbie Richards told AFP.

 

“It’s a cheap way to produce advertisements,” she added.

Alexios Mantzarlis, director of the Security, Trust, and Safety Initiative at Cornell Tech, has observed a surge of “AI doctor” avatars and audio tracks on TikTok that promote questionable sexual remedies.

Some of these videos, many with millions of views, peddle testosterone-boosting concoctions made from ingredients such as lemon, ginger and garlic.

More troublingly, rapidly evolving AI tools have enabled the creation of deepfakes impersonating celebrities such as actress Amanda Seyfried and actor Robert De Niro.

“Your husband can’t get it up?” Anthony Fauci, former director of the National Institute of Allergy and Infectious Diseases, appears to ask in a TikTok video promoting a prostate supplement.

But the clip is a deepfake, using Fauci’s likeness.

Many manipulated videos are created from existing ones, modified with AI-generated voices and lip-synced to match what the altered voice says.

“The impersonation videos are particularly pernicious as they further degrade our ability to discern authentic accounts online,” Mantzarlis said.

Last year, Mantzarlis discovered hundreds of ads on YouTube featuring deepfakes of celebrities — including Arnold Schwarzenegger, Sylvester Stallone, and Mike Tyson — promoting supplements branded as erectile dysfunction cures.

The rapid pace of generating short-form AI videos means that even when tech platforms remove questionable content, near-identical versions quickly reappear — turning moderation into a game of whack-a-mole.

Researchers say this creates unique challenges for policing AI-generated content, requiring novel solutions and more sophisticated detection tools.

AFP’s fact checkers have repeatedly debunked scam ads on Facebook promoting treatments — including erectile dysfunction cures — that use fake endorsements by Ben Carson, a neurosurgeon and former US cabinet member.

Yet many users still consider the endorsements legitimate, illustrating the appeal of deepfakes.

“Scammy affiliate marketing schemes and questionable sex supplements have existed for as long as the Internet and before,” Mantzarlis said.

“As with every other bad thing online, generative AI has made this abuse vector cheaper and quicker to deploy at scale.”


Two Palestinian journalists killed in less than 24 hours as Israeli strikes intensify in Gaza

Updated 14 July 2025
Follow

Two Palestinian journalists killed in less than 24 hours as Israeli strikes intensify in Gaza

  • Fadi Khalifa was killed while inspecting the remnants of his house
  • In a separate strike on their tent, Hussam Saleh Al-Adlouni was murdered alongside his wife Suad and their three children

LONDON: Two Palestinian journalists were killed in separate Israeli airstrikes on Sunday, in the latest deadly escalation targeting media workers in the Gaza Strip, according to local sources.

Fadi Khalifa was killed along with at least one other person while inspecting the remains of his home in the Al-Zaytoun neighborhood of southeastern Gaza City, multiple sources reported.

In a separate strike, photojournalist Hussam Saleh Al-Adlouni was killed alongside his wife Suad and their three children when an Israeli airstrike hit their tent in the Al-Qarara area, north of Khan Yunis in southern Gaza.

According to WAFA news agency, the family had been sheltering there after being displaced.

The Palestinian Journalists Syndicate condemned the killings, saying they added to Israel’s “abysmal record” of violence against members of the press.

Their deaths bring the number of Palestinian journalists killed in Gaza since Oct. 7, 2023, to 231, according to official figures.

The Committee to Protect Journalists puts the toll at 178: 176 Palestinians and two Israelis.

Medical sources reported that at least 92 civilians were killed across Gaza on Sunday alone, including 52 in central and southern areas of the Strip.


BBC’s Gaza documentary breached accuracy guideline, review finds

Updated 14 July 2025
Follow

BBC’s Gaza documentary breached accuracy guideline, review finds

  • “Gaza: How To Survive A War Zone” was pulled after it was discovered that the 13-year-old son of a Hamas official was the narrator of the documentary
  • Review found the program breached a guideline on accuracy that deals with misleading audiences but no other breaches of the BBC’s editorial guidelines, including on impartiality

LONDON: A BBC documentary about children’s lives in Gaza narrated by the 13-year-old son of a Hamas official breached its editorial guidelines on accuracy, an internal review by the British public broadcaster said on Monday.
The investigation, however, found there were no other breaches of the BBC’s editorial guidelines, including on impartiality, and no evidence that outside interests “inappropriately impacted on the program.”
The BBC removed “Gaza: How To Survive A War Zone” from its online platform in February, five days after it was broadcast, saying it had “serious flaws.” The documentary was made by independent production company HOYO Films.
A review found the program breached a guideline on accuracy that deals with misleading audiences.
The background on the narrator’s father — a minister in the Hamas-run government in Gaza — was “critical information,” which was not shared with the BBC before broadcast, the review found.
Gaza’s health ministry says more than 58,000 people have been killed since the start of the war on Oct. 7 2023, when Hamas-led militants stormed into Israel, killing about 1,200 people and taking 251 hostages into Gaza.
The BBC’s coverage of the war has been heavily scrutinized throughout the conflict, with both supporters of Israel and its critics saying the broadcaster had failed to strike the right balance.
“Regardless of how the significance or otherwise of the Narrator’s father’s position was judged, the audience should have been informed about this,” said the report by Peter Johnston, BBC Director of Editorial Complaints and Reviews.
BBC Director-General Tim Davie said the report identified a significant failing in relation to accuracy.
“We will now take action on two fronts – fair, clear and appropriate actions to ensure proper accountability and the immediate implementation of steps to prevent such errors being repeated,” Davie said in a statement.


Indonesian boy’s ‘aura farming’ dance brings global spotlight to centuries-old tradition

Updated 11 July 2025
Follow

Indonesian boy’s ‘aura farming’ dance brings global spotlight to centuries-old tradition

  • Pacu jalur is a boat tradition from Indonesia’s Riau province that can be traced back to the 17th century
  • Dika was named tourism ambassador of his home province after going viral with his dance moves

JAKARTA: An Indonesian boy dancing on the front of a boat has become an Internet sensation in recent weeks, setting a global trend of “aura farming” that has been recreated by famous athletes and thousands of others worldwide.

“Aura farming” is an Internet expression popularized in 2024, largely in reference to anime characters and celebrities. It refers to the act of consistently looking cool to build one’s “aura.”

Dressed in a black traditional costume and wearing sunglasses, 11-year-old Rayyan Arkhan Dikha from Indonesia’s Riau province has been dubbed “the ultimate aura farmer” on social media for performing a series of repetitive movements calmly on the bow of a thin boat, videos of which have amassed millions of views globally.

The Indonesian boy who goes by the name of Dika was participating in a local event known as “pacu jalur,” which roughly translates to “boat race.” A tradition that dates back to the early 17th century, the event is now held every August to commemorate the Indonesian Independence Day.

“Pacu jalur has been one of Indonesia’s Intangible Cultural Heritage since 2015,” Indonesia’s Culture Minister Fadli Zon said in a statement, after hosting Dika in his office in Jakarta on Wednesday.

The water sport tradition originated in Riau’s Kuantan Singingi regency at a time when boats were the main mode of transportation for the communities living along the local river.

“It has always been a part of life for people in (Kuantan Singingi), including to celebrate the most important Islamic holidays and also our independence day,” Fadli said.

During the race, each of the long, canoe-like boats and its large crew has an “anak coki,” a dancer who moves with rhythmic hand movements and body waves to provide inspiration for the rowers.

While every anak coki brings their own charm to the race, Dika — who has participated in the races since he was 9 — has since become the face of the pacu jalur tradition.

Though the original clip featuring Dika was posted to TikTok in January by a user named Lensa Rams and the event itself was held last August, the boy shot to global popularity over the past few weeks, as various creators on Instagram and TikTok have tried their own hand at Dika’s dance.

The list includes soccer team Paris Saint-Germain and Travis Kelce, American football star and boyfriend of pop singer Taylor Swift. When the US men’s national soccer team won against Guatemala last week, American soccer player Diego Luna copied Dika’s moves to celebrate a goal.

The massive impact of the video garnered him special attention from the government in Riau, where the governor on Tuesday named Dika as a tourism ambassador for the province and awarded him a scholarship for 20 million rupiah (about $1,200) for his education.

“Today, almost everyone opened their eyes to the vibrant and thriving culture of Riau, especially pacu jalur. This is why I wanted to show my appreciation to Dika,” Governor Abdul Wahid said.

In a statement, the local government confirmed that Dika will participate in the races next month.

Speaking to reporters in Jakarta following his meeting with the culture minister, Dika said: “I’m happy that I’ve gone viral globally.”


Israeli journalist arrested over post praising death of 5 IDF soldiers in Gaza

Updated 11 July 2025
Follow

Israeli journalist arrested over post praising death of 5 IDF soldiers in Gaza

  • Israel Frey, who frequently posts his criticism of the Israeli army’s actions in Gaza, is being held in the Tel Aviv Magistrate’s Court
  • The Committee to Protect Journalists condemned Frey’s arrest

LONDON: An Israeli court on Thursday extended the detention of journalist Israel Frey over a post on X that hailed “the world is a better place” following the death of five soldiers in an explosion in Gaza.

Frey, who frequently posts his criticism of the Israeli army’s actions in Gaza, is being held in the Tel Aviv Magistrate’s Court over charges of inciting and supporting terrorism.

“The world is better this morning without five young people who participated in one of the cruelest crimes against humanity,” the Israeli journalist said, referring to five Israeli soldiers who were killed by an explosive device during their fight with the militant group Hamas in northern Gaza earlier this week.

He added: “Sadly, for the boy in Gaza now being operated on without anesthesia, the girl starving to death and the family huddling in a tent under bombardment — this is not enough.

“This is a call to every Israeli mother: Do not be the next to receive your son in a coffin as a war criminal. Refuse.”

Frey was previously questioned over his critical posts in the past. In March, he was interrogated on suspicion of inciting terrorism over several pro-Palestinian posts.

“A Palestinian who hurts an IDF soldier or a settler in the apartheid territories is not a terrorist. And it’s not a terror attack. He’s a hero fighting against an occupier for justice, liberation and freedom,” he once wrote.

In December 2022, he was questioned over posts in which he said that “targeting security forces is not terrorism” and called a Palestinian who was planning an attack a “hero.”

Frey fled into hiding on Oct. 16, 2023, about a week into the Gaza war, after his home was attacked by a far-right Israeli mob when he expressed solidarity with Palestinians in Gaza.

On Thursday, he told the Israeli newspaper Haaretz that he will not be “bowing his head” to his persecution, adding that “we have already caused enough suffering, blood and tears. Liberate Gaza. Enough.”

According to Israeli media reports, Judge Ravit Peleg Bar Dayan ruled that Frey’s remarks “offend public sensibilities and are deeply disturbing,” asking, “How can the deaths of young soldiers, who fell in the line of duty defending their homeland, possibly be considered good?”

She added that extending Frey’s detention was necessary due to “investigative actions susceptible to obstruction,” as she denied bail to Frey.

In a statement, the Committee to Protect Journalists condemned Frey’s arrest and said his detention “underscores authorities’ growing intolerance of freedom of expression since the start of the war on October 7, 2023.”

CPJ Regional Director Sara Qudah called for Frey’s immediate release along with “all detained Palestinian journalists” and for an end to the “ongoing crackdown on the press and dissenting voices.”


Pakistani father kills daughter over TikTok account: police

Updated 11 July 2025
Follow

Pakistani father kills daughter over TikTok account: police

  • TikTok is wildly popular in Pakistan, in part because of its accessibility to a population with low literacy levels
  • Pakistani women have found both audience and income on the app, which is rare in the country

RAWALPINDI: Pakistan police on Friday said a father shot dead his daughter after she refused to delete her account on popular video-sharing app TikTok.

In the Muslim-majority country, women can be subjected to violence by family members for not following strict rules on how to behave in public, including in online spaces.

“The girl’s father had asked her to delete her TikTok account. On refusal, he killed her,” a police spokesperson said.

According to a police report shared with AFP, investigators said the father killed his 16-year-old daughter on Tuesday “for honor.” He was subsequently arrested.

The victim’s family initially tried to “portray the murder as a suicide” according to police in the city of Rawalpindi, where the attack happened, next to the capital Islamabad.

Last month, a 17-year-old girl and TikTok influencer with hundreds of thousands of online followers was killed at home by a man whose advances she had refused.

Sana Yousaf had racked up more than a million followers on social media accounts including TikTok, where she shared videos of her favorite cafes, skincare products, and traditional outfits.

TikTok is wildly popular in Pakistan, in part because of its accessibility to a population with low literacy levels.

Women have found both audience and income on the app, which is rare in a country where fewer than a quarter of the women participate in the formal economy.

However, only 30 percent of women in Pakistan own a smartphone compared to twice as many men (58 percent), the largest gap in the world, according to the Mobile Gender Gap Report of 2025.

Pakistani telecommunications authorities have repeatedly blocked or threatened to block the app over what it calls “immoral behavior,” amid backlash against LGBTQ and sexual content.

In southwestern Balochistan, where tribal law governs many rural areas, a man confessed to orchestrating the murder of his 14-year-old daughter earlier this year over TikTok videos that he said compromised her “honor.”