Introduction
In the world of open-source development, pull requests (PRs) are the backbone of collaboration and innovation. However, like any open system, it is constantly under the threat of abuse. Today, PR spam mirrors the email spam explosion that we witnessed in the early 2000s. Let's explore how these phenomena are similar and how the industry can adapt.
The OpenClaw Phenomenon
Consider the example of OpenClaw, a project that experienced a surge in PR submissions. Last December, OpenClaw received two PRs per week. By February, this number skyrocketed to 3,400 per week. This massive increase is largely due to AI-generated PRs of often questionable quality.
For instance, one contributor submitted 106 PRs in a single day, with a median interval of three seconds between submissions. These figures clearly demonstrate the scale of the current problem.
Comparison with Email Spam
In the 2000s, the ILOVEYOU worm infected 45 million computers in 24 hours, exploiting the low cost of sending emails and users' trust in the platform. Today, the cost of submitting a PR is similarly low, leading to a massive increase in the quantity of contributions, often with little relevance.
Current Challenges
With a merge rate of only 9.3% following the PR surge, quality is clearly an issue. First-time PRs have an acceptance rate of 8.2%, compared to 18.6% for those with more than five contributions.
Mitchell Hashimoto, creator of Ghostty, had to limit AI-generated contributions due to overload. He introduced Vouch, a trust management system to filter out unreliable contributors.
Possible Solutions
To counter PR spam, we can draw inspiration from email anti-spam solutions:
- Blocklists: Similar to email blacklists, these can block problematic contributors.
- Trust-based Filters: Evaluate the reputation of contributors to determine the relevance of their PRs.
- Reputation Infrastructure: Just like reputation scores for emails, contributors could be rated based on their history.
Conclusion
The proliferation of AI-generated PRs is a challenge that requires immediate attention. By leveraging past experiences with email spam, we can develop robust tools to protect the integrity of open-source projects.
Let's discuss your project in 15 minutes.