Skip to main content

AI Crawlers Haven't Learned To Play Nice With Websites

2 months 3 weeks ago
SourceHut, an open-source-friendly git-hosting service, says web crawlers for AI companies are slowing down services through their excessive demands for data. From a report: "SourceHut continues to face disruptions due to aggressive LLM crawlers," the biz reported Monday on its status page. "We are continuously working to deploy mitigations. We have deployed a number of mitigations which are keeping the problem contained for now. However, some of our mitigations may impact end-users." SourceHut said it had deployed Nepenthes, a tar pit to catch web crawlers that scrape data primarily for training large language models, and noted that doing so might degrade access to some web pages for users. "We have unilaterally blocked several cloud providers, including GCP [Google Cloud] and [Microsoft] Azure, for the high volumes of bot traffic originating from their networks," the biz said, advising administrators of services that integrate with SourceHut to get in touch to arrange an exception to the blocking.

Read more of this story at Slashdot.

msmash

Time to ditch US tech for homegrown options, says Dutch parliament

2 months 3 weeks ago
Trump administration 'has made the call for tech sovereignty an urgent geopolitical issue'

Not content to wait for open letters to influence the European Commission, Dutch parliamentarians have taken matters into their own hands by passing eight motions urging the government to ditch US-made tech for homegrown alternatives.…

Brandon Vigliarolo

More Than 150 'Unprecedented' Climate Disasters Struck World in 2024, Says UN

2 months 3 weeks ago
The devastating impacts of the climate crisis reached new heights in 2024, with scores of unprecedented heatwaves, floods and storms across the globe, according to the UN's World Meteorological Organization. From a report: The WMO's report on 2024, the hottest year on record, sets out a trail of destruction from extreme weather that took lives, demolished buildings and ravaged vital crops. More than 800,000 people were displaced and made homeless, the highest yearly number since records began in 2008. The report lists 151 unprecedented extreme weather events in 2024, meaning they were worse than any ever recorded in the region. Heatwaves in Japan left hundreds of thousands of people struck down by heatstroke. Soaring temperatures during heatwaves peaked at 49.9C at Carnarvon in Western Australia, 49.7C in the city of Tabas in Iran, and 48.5C in a nationwide heatwave in Mali. Record rains in Italy led to floods, landslides and electricity blackouts; torrents destroyed thousands of homes in Senegal; and flash floods in Pakistan and Brazil caused major crop losses. Storms were also supercharged by global heating in 2024, with an unprecedented six typhoons in under a month hitting the Philippines. Hurricane Helene was the strongest ever recorded to strike the Big Bend region of Florida in the US, while Vietnam was hit by Super Typhoon Yagi, affecting 3.6 million people. Many more unprecedented events will have passed unrecorded.

Read more of this story at Slashdot.

msmash