Skip to main content

The Open-Source Software Saving the Internet From AI Bot Scrapers

2 months 1 week ago
An anonymous reader quotes a report from 404 Media: For someone who says she is fighting AI bot scrapers just in her free time, Xe Iaso seems to be putting up an impressive fight. Since she launched it in January, Anubis, a "program is designed to help protect the small internet from the endless storm of requests that flood in from AI companies," has been downloaded nearly 200,000 times, and is being used by notable organizations including GNOME, the popular open-source desktop environment for Linux, FFmpeg, the open-source software project for handling video and other media, and UNESCO, the United Nations organization for educations, science, and culture. [...] "Anubis is an uncaptcha," Iaso explains on her site. "It uses features of your browser to automate a lot of the work that a CAPTCHA would, and right now the main implementation is by having it run a bunch of cryptographic math with JavaScript to prove that you can run JavaScript in a way that can be validated on the server." Essentially, Anubis verifies that any visitor to a site is a human using a browser as opposed to a bot. One of the ways it does this is by making the browser do a type of cryptographic math with JavaScript or other subtle checks that browsers do by default but bots have to be explicitly programmed to do. This check is invisible to the user, and most browsers since 2022 are able to complete this test. In theory, bot scrapers could pretend to be users with browsers as well, but the additional computational cost of doing so on the scale of scraping the entire internet would be huge. This way, Anubis creates a computational cost that is prohibitively expensive for AI scrapers that are hitting millions and millions of sites, but marginal for an individual user who is just using the internet like a human. Anubis is free, open source, lightweight, can be self-hosted, and can be implemented almost anywhere. It also appears to be a pretty good solution for what we've repeatedly reported is a widespread problem across the internet, which helps explain its popularity. But Iaso is still putting a lot of work into improving it and adding features. She told me she's working on a non cryptographic challenge so it taxes users' CPUs less, and also thinking about a version that doesn't require JavaScript, which some privacy-minded disable in their browsers. The biggest challenge in developing Anubis, Iaso said, is finding the balance. "The balance between figuring out how to block things without people being blocked, without affecting too many people with false positives," she said. "And also making sure that the people running the bots can't figure out what pattern they're hitting, while also letting people that are caught in the web be able to figure out what pattern they're hitting, so that they can contact the organization and get help. So that's like, you know, the standard, impossible scenario."

Read more of this story at Slashdot.

BeauHD

Waymo Starts Robotaxi Testing In Philadelphia and NYC

2 months 1 week ago
Waymo has launched new "road trips" to Philadelphia and New York City, "signaling the Alphabet-owned company's interest in expanding into Northeastern cities," reports TechCrunch. While these trips don't guarantee commercial launches, they follow a pattern that previously led to deployments in cities like Los Angeles. Other road trips this year are planned for Houston, Orlando, Las Vegas, San Diego, and San Antonio. From the report: Typically, the trips involve sending a small fleet of human-driven vehicles equipped with Waymo's autonomous driving system to map out the new city. Then Waymo tests the vehicles autonomously, though still with a human behind the wheel, before taking any data and learnings back to its engineers to improve the AI driver's performance. In some cases, these road trips have led to commercial launches. In 2023, the company made a road trip to Santa Monica, a city in Los Angeles County. The company now operates a commercial service in Los Angeles, including Santa Monica, Beverly Hills, and Hollywood. For its Philadelphia trip, Waymo plans to place vehicles in the most complex parts of the city, including downtown and freeways, according to a spokesperson. She noted folks will see Waymo vehicles driving "at all hours throughout various Philadelphia neighborhoods, from North Central to Eastwick, University City, and as far east as the Delaware River." In NYC, Waymo will drive its cars manually in Manhattan just north of Central Park down to The Battery and parts of Downtown Brooklyn. The company will also map parts of Jersey City and Hoboken in New Jersey. Waymo applied last month for a permit to test its AVs in New York City with a human behind the wheel. The company has not yet received approval.

Read more of this story at Slashdot.

BeauHD