Skip to main content

Securely Useless

1 week 5 days ago

Submitter Chops relates a tale of chasing "The Standard" at any cost:

A company had been forced, kicking and screaming, into the 20th century, and then into the 21st. Their range of dumb hardware was no longer being bought by customers who expected at least some way to get the data off it, and a USB stick wouldn't cut it.

The techno-fearful hired an external IT company and instructed them to ensure that the company adhered to the UK's "CyberEssentials" programme of IT security.

Unfortunately, this meant that the nascent software development capability was immediately hamstrung, as people were denied the right to install and run unapproved software (including software that needed admin level rights in order to work, such as process monitor tools used as standard in software dev and test), could not easily connect across networks (also vital for the software dev), could not add various hardware to the network (such as the company's own products, which made testing them awfully difficult) and so on.

A criminal underworld of mystery PCs slowly grew into place, plugging into the network to get things done, disavowed by anyone who saw them. Tunnels grew up, IP addresses were handed round on notepaper (no DNS approvals), a shadow world of software development in the back alleys of the network. Until finally it was too much, and approval was given for a second, less secure network. The plan was that the real core business, the really important parts, they would stay on the high-security side. There would be a second network, with a single bridge, which would serve the various software development and test systems, and also host the myriad new internal services growing up (wikis, servers, tools and so on). This way, the real core business would definitely remain secure, and compliant, and the software side of it and all that would not compromise the heart of the business.

The heart of the business ended up being a network share containing a decade of out-of-date, unsearchable documentation for products that were mostly no longer sold. It is super-secure, and the business heads confirm whole-heartedly that the company adheres to the CyberEssentials programme 100% with zero compromise.

There's just a carefully separated set of servers and PCs, not the true core of the business but about eight times the size and full of essential activity, on a secondary network.

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!
Ellis Morning

Microsoft Announces Open Source Tool Evaluating Your Software's Optimization For AI Coding Agents

1 week 5 days ago
The problem? Understanding how AI coding agents interact with the software you build. "Whether you're maintaining an SDK, building a command-line interface, or publishing an MCP server or skills... you need to understand whether agents can discover and use what you ship, and what your team can change." Results can vary, and configurations keep changing, with behaviors influenced by everything from custom instructions and MCP servers to skills and plugins. So after a year of development Microsoft has created Scope, "an open-source platform built on Kubernetes for turning those interactions into evidence product teams can act on." Details are available on... the Microsoft open source blog: Available under the MIT License, Scope supports a continuous Developer Agent Experience (DevAx) learning loop grounded in repeatable experiments. The Azure Cosmos DB team's experience shows what that loop can deliver... "The evaluations led to 20 Agent Kit issues and 27 documentation issues resolved," [says Jay Gordon, Senior Program Manager, Azure Cosmos DB]... These results connect evaluation to decisions a product team can make: which documentation needs clarification, where tooling needs a fix, and whether an update changes agent behavior.... [The gains] demonstrate how targeted changes, informed by observed behavior, can improve the way agents use a product. For more on the shift from building for human developers to supporting agents, watch Your Product Was Built for Humans. Is It Ready for Agents? or join us at AI DevCon NYC conference in November. Scope surfaces everything from incorrect assumptions in documentation (or recurring missed instructions) to whether agents select the intended SDK, initialize it correctly, and use its APIs, leading to clearer descriptions for command-line tool and more actionable errors. Microsoft's open source blog also explains why AI agent evaluation needs an open-source foundation: The challenges behind agent experience aren't unique to one product or organization. SDK maintainers, tool authors, researchers, and agent developers are all asking how agents behave and what helps them succeed. No single team can represent every language, workflow, or integration. An open project can bring more scenarios, more failure cases, and more viewpoints into the evaluation process... Opening Scope's execution and evaluation code gives contributors a way to question the methods, investigate unexpected results, and improve the evaluation itself. A shared foundation also reduces duplicated infrastructure. Agent integrations, reusable criteria, and trace-analysis improvements can help other teams study their own products without rebuilding the same machinery. The goal isn't one universal benchmark or definition of success. It's a common starting point that teams can adapt to their needs. The MIT License lets teams use and extend Scope, while self-managed deployments let them evaluate private workloads without publishing their code or run data.... Scope streams activity as the agent works and retains interactions and workspace snapshots for later review. Your team can follow tool calls, inspect generated files, and connect a failed requirement to the steps that preceded it. Those traces provide observable evidence of what the agent did, helping turn a failure into a concrete documentation, SDK, or tooling improvement.

Read more of this story at Slashdot.

EditorDavid

25% of Incoming Pull Requests May Be AI Slop, Finds Study From Linux Foundation Research

1 week 5 days ago
"59% of maintainers report growing contribution volume since the rise of GenAI," according to a new study published by Linux Foundation Research. But 25% of their incoming pull requests "are estimated to be low-quality, also known as 'AI slop'." And on large projects, 69% see growing contributions — and estimate that 31% are low-quality/AI slop. So what do these results show us? "Generative AI is fundamentally changing how people participate in open source," says the head of Open Source Ecosystem Development at Huawei's R&D subsidiary Futurewei, which helped support the study. Specifically GenAI is "lowering barriers to contribution" but also "creating new challenges" for maintainers and communities. Their recommendation? "Build the governance, transparency and trust mechanisms that allow AI-assisted contributions to strengthen, but not overwhelm, the open source ecosystem." "Open source projects need to design for a growing AI-generated contributor base, not simply react to it," said Hilary Carter, SVP of Research at the Linux Foundation. "That means setting clear expectations for GenAI, putting responsibility for testing and disclosure on contributors and giving maintainers better tools to manage incoming pull requests. If we get those fundamentals right, AI can expand participation without overburdening and increasing costs for maintainers." More from the blog It's FOSS: LF Research is the Linux Foundation's research arm. It was set up in 2021 and publishes studies on how open source is built and used, drawing on surveys and project data. Its latest report, Navigating Open Source Software Development in the Generative AI Era, surveyed 234 contributors and maintainers on how generative AI is affecting the way open source gets made... 64% say reviews now take longer. Burnout is their top concern at 63%, narrowly ahead of contribution quality at 60%. You will also find that policies have not caught up to the AI-assisted coding and vulnerability reporting trend. Of the 119 respondents who answered the question, 35% said their project has a formal policy permitting AI use. Another 32% have none and no plans for one, 23% plan to write one, and 10% ban AI contributions outright.... Most maintainers (71%) say AI makes it easier for newcomers to start contributing. Other interesting findings: "Respondents estimate average time reductions across all six OSS development activities measured, including proposal discussions (19%) and code authoring (52%). For high-complexity tasks, however, 21% of respondents report taking more time when using GenAI." "71% of maintainers agree GenAI reduces barriers to entry for OSS, but 53% of maintainers disagree that GenAI has improved community trust." Developers are reinvesting the time they save back into open source: 66% report using productivity gains to address technical debt, 63% to develop new features and 58% for learning and upskilling. Continued investment in human mentorship remains important: 46% of maintainers say GenAI increases the need for mentorship of new contributors, compared with 32% who say it decreases that need. OpenSSF and LF AI & Data also helped support the study.

Read more of this story at Slashdot.

EditorDavid