KBSS

We design and build knowledge-based information systems

1 minute read

The Open mic session starts on Friday, 14 November 2025, at 10:30 via this link. Miroslav Blaško is going to talk about the findings from the METR study “Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity” (July 2025), which should be followed by a discussion on the implications for software engineering.

Abstract

In their randomized controlled trial, METR recruited 16 experienced open-source developers (with long-term familiarity with their repositories) to complete 246 issue tasks. Each task was randomly assigned to allow or disallow the use of state-of-the-art AI development tools (e.g., Cursor Pro with Claude 3.5/3.7) during February to June 2025. Contrary to prevailing expectations, the study found that when developers used AI tools, they took, on average, 19% longer to complete tasks compared to when they did not use AI. Developers had initially forecasted a 24% speed-up, and even after experiencing the slowdown, they still believed they had improved by ~20%. During this session, we will review the study’s methodology and key findings, discuss potential explanations for the unexpected outcomes, and consider how these insights might generalize to our own software development practices.

Further reading:

You may also enjoy