X

Aggregator

Should US Open-Weight AI Labs 'Distill' Frontier Models Too?

Slashdot
1 month 2 weeks ago
Silicon Valley giants and national security experts "are calling for action against Chinese companies engaged in model distillation," reports CNBC. But "I would do nothing," says Y Combinator CEO Garry Tan. "We could argue that there should be an American distillation regime." Distillation is the process of using the outputs of a more capable AI model to train a smaller or less capable one, sometimes illicitly... Anthropic has accused Chinese companies such as Moonshot AI, DeepSeek, and MiniMax of the practice, while OpenAI believes DeepSeek's V3 and R1 model architectures were distilled from its own GPT-4 and GPT-4o models. In the midst of this, the U.S.'s National Security Agency, Cybersecurity and Infrastructure Security Agency, and Federal Bureau of Investigation released an official cyber security advisory warning on the topic on Tuesday... But Tan believes regulators should focus less on curbing distillation and more on creating an equilibrium between open weight models and frontier models — as long as frontier models retain a price premium that allows their business model to remain feasible. "This is actually the ideal case. You want open weight models to give people freedom and access," he explained. "If I were a regulator, that's what I would go after." Tan acknowledged that this is a hard balance to strike, calling it "a tightrope." Nevertheless, he says it's a balance worth pursuing — saying it "could result in the best possible outcome." Tan later told TechCrunch he'd like to see America with more open-weight options that aren't Chinese, built by smaller U.S. open-weight AI labs using those same training techniques on products from America's frontier AI labs: Anthropic CEO Dario Amodei had previously publicly called on U.S. regulators to crack down on distillation. It's notable that the commander of Silicon Valley's prestigious and prolific startup accelerator doesn't agree. To be clear, Tan isn't advocating for American AI labs to use stolen credentials to distill. He wants them to be free to come in the front door. In fact, his argument is twofold. He feels it's an overreach for AI labs to dictate what their customers can do with the information their models share with them. He also notes that the proprietary AI labs didn't ask permission when they vacuumed up as much human knowledge as they could to train their models. They famously ingested plenty of copyrighted material without the permission of those intellectual property holders. "Controlling what users and customers do with API calls to closed weight models feels constraining, and there's a role government can play here to normalize the fact that access to intelligence that was trained on broad public access data should itself also be more a form of a public good than something locked away behind restrictive terms of service," he told TechCrunch when asked why American labs should be free to distill, too... To him, the true AI doomer scenario is for all the immense power of frontier AI to wind up in the hands of a single powerful, proprietary provider. "The nightmare scenario, the doomer scenario for AI is that there's just one company," he said. "It has the best access to capital. It has the best AI researchers. It runs away with it and suddenly there's one company that's monolithic. And that would be bad."

Read more of this story at Slashdot.

EditorDavid

CodeSOD: I Exist

The Daily WTF
1 month 3 weeks ago

In addition to using an ancient development environment, with terrible UX, Greta also has the misfortune of working in Pascal.

Recently, she was diagnosing a bug. The program was reporting that files didn't exist when they definitely existed. She traced the problem down into the system library. Let's see if you can spot what's wrong:

{ Delphi / Kylix Cross-Platform Runtime Library } { System Utilities Unit } { } { Copyright (c) 1995-2001 Borland Softwrare Corporation } ... function FileAge(const FileName: string): Integer; {$IFDEF MSWINDOWS} var Handle: THandle; FindData: TWin32FindData; LocalFileTime: TFileTime; begin Handle := FindFirstFile(PChar(FileName), FindData); if Handle <> INVALID_HANDLE_VALUE then begin Windows.FindClose(Handle); if (FindData.dwFileAttributes and FILE_ATTRIBUTE_DIRECTORY) = 0 then begin FileTimeToLocalFileTime(FindData.ftLastWriteTime, LocalFileTime); if FileTimeToDosDateTime(LocalFileTime, LongRec(Result).Hi, LongRec(Result).Lo) then Exit; end; end; Result := -1; end; {$ENDIF} function FileExists(const FileName: string): Boolean; {$IFDEF MSWINDOWS} begin Result := FileAge(FileName) <> -1; end; {$ENDIF}

The first function here is FileAge, which returns the last modified timestamp on a file. Note the use of FileTimeToDosDateTime, which is a Windows API function. It converts LocalFileTime and stores the date part in the first output parameter (LongRec(Result).Hi) and the time part in the second output parameter (LongRec(Result).Lo). Result, in this case, is our return value. If anything goes wrong, we return -1.

The FileExists function then, simply calls FileAge. If it doesn't return a -1, there must be a file there.

That's an awkward, weird solution to the problem. There has to be a system call that can answer that question more obviously. But it doesn't seem like it should be blowing up- it looks like it should work.

But note that FileTimeToDosDateTime also returns a boolean value. If it succeeds, great, but if it fails, it returns false and sets an error code you can check. An error code that definitely isn't being checked.

And this brings us to the root cause of Greta's bug: the process that's writing the files isn't setting the "last write time", so while the file exists, the attempt to check its age fails, so FileExists believes that the file doesn't exist, just because it doesn't have a valid timestamp.

This is the kind of high quality softwrare that sometimes infects our system libraries.

[Advertisement] Keep the plebs out of prod. Restrict NuGet feed privileges with ProGet. Learn more.
Remy Porter