AI companies are reportedly still scraping websites despite protocols meant to block them

  • 📰 engadget
  • ⏱ Reading Time:
  • 21 sec. here
  • 7 min. at publisher
  • 📊 Quality Score:
  • News: 29%
  • Publisher: 63%

Robots.Txt File News

AI Company,Perplexity,Web Crawlers

Mariella Moon has been a night editor for Engadget since 2013, covering everything from consumer technology and video games to strange little robots that could operate on the human body from the inside one day. She has a special affinity for space, its technologies and its mysteries, though, and has interviewed astronauts for Engadget.

Perplexity, a company that describes its product as "a free AI search engine," has been under fire over the past few days. Shortly afterreported that Perplexity has been ignoring the Robots Exclusion Protocol, or robots.txt, and has been scraping its website and other Condé Nast publications.

Srinivas defended his company's practices, telling the publication that the Robots Exclusion Protocol is "not a legal framework" and suggesting that publishers and companies like his may have to establish a new kind of relationship. He also reportedly insinuated thatdeliberately used prompts to make Perplexity's chatbot behave the way it did, so ordinary users will not get the same results.

 

Thank you for your comment. Your comment will be published after being reviewed.
Please try again later.
We have summarized this news so that you can read it quickly. If you are interested in the news, you can read the full text here. Read more:

 /  🏆 276. in US

United States United States Latest News, United States United States Headlines