Skip to main content

OpenAI Tests Its AI's Persuasiveness By Comparing It to Reddit Posts

3 months 1 week ago
Friday TechCrunch reported that OpenAI "used the subreddit, r/ChangeMyView to create a test for measuring the persuasive abilities of its AI reasoning models." The company revealed this in a system card — a document outlining how an AI system works — that was released along with its new "reasoning" model, o3-mini, on Friday.... OpenAI says it collects user posts from r/ChangeMyView and asks its AI models to write replies, in a closed environment, that would change the Reddit user's mind on a subject. The company then shows the responses to testers, who assess how persuasive the argument is, and finally OpenAI compares the AI models' responses to human replies for that same post. The ChatGPT-maker has a content-licensing deal with Reddit that allows OpenAI to train on posts from Reddit users and display these posts within its products. We don't know what OpenAI pays for this content, but Google reportedly pays Reddit $60 million a year under a similar deal. However, OpenAI tells TechCrunch the ChangeMyView-based evaluation is unrelated to its Reddit deal. It's unclear how OpenAI accessed the subreddit's data, and the company says it has no plans to release this evaluation to the public... The goal for OpenAI is not to create hyper-persuasive AI models but instead to ensure AI models don't get too persuasive. Reasoning models have become quite good at persuasion and deception, so OpenAI has developed new evaluations and safeguards to address it. Reddit's "ChangeMyView" subreddit has 3.8 million human subscribers, making it a valuable source of real human interactions, according to the article. And it adds one more telling anecdote. "Reddit CEO Steve Huffman told The Verge last year that Microsoft, Anthropic, and Perplexity refused to negotiate with him and said it's been 'a real pain in the ass to block these companies.'"

Read more of this story at Slashdot.

EditorDavid

Slashdot Asks: Do You Remember Your High School's 'Computer Room'?

3 months 1 week ago
Bill Gates' blog has been updated with short videos about his upcoming book, including one about how his school ended up with an ASR-33 teletype that could connect their Seattle classroom to a computer in California. "The teachers faded away pretty quickly," Gates adds, "But about six of us stayed hardcore. One was Paul Allen..." — the future co-founder of Microsoft. And the experience clearly meant a lot to Gates. "Microsoft just never would've happened without Paul — and this teletype room." In a longer post thanking his "brilliant" teachers, Gates calls his teletype experience "an encounter that would shape my entire future" and "opened up a whole new world for me." Gates also thanks World War II Navy pilot and Boeing engineer Bill Dougall, who "was instrumental in bringing computer access to our school, something he and other faculty members pushed for after taking a summer computer class... The fascinating thing about Mr. Dougall was that he didn't actually know much about programming; he exhausted his knowledge within a week. But he had the vision to know it was important and the trust to let us students figure it out." Gates shared a similar memory about the computer-room's 20-something overseer Fred Wright, who "intuitively understood that the best way to get students to learn was to let us explore on our own terms. There was no sign-up sheet, no locked door, no formal instruction." Instead, Mr. Wright let us figure things out ourselves and trusted that, without his guidance, we'd have to get creative... Some of the other teachers argued for tighter regulations, worried about what we might be doing in there unsupervised. But even though Mr. Wright occasionally popped in to break up a squabble or listen as someone explained their latest program, for the most part he defended our autonomy... Mr. Wright gave us something invaluable: the space to discover our own potential. Any Slashdot readers have a similarly impactful experience? Share your own thoughts and memories in the comments. Do you remember your high school's computer room?

Read more of this story at Slashdot.

EditorDavid