You must log in or register to comment.
Love this guys, hope he lives forever!
I got weirdly invested in this, and by the end I was kinda happy that it was “just” a bug in the tooling and not anything actually malicious.
If you’re not familiar with reading mailing lists or don’t follow what is happening, Brodie Robertson on YT did a good video on this: https://youtu.be/GhfhzTDQdUU
TL;DR: Some tooling script caused the problem, but it initially seemed like a malicious pull request from kernel developer. It wasn’t and the issue was resolved. The tooling script will be updated with better error messages so this kind of problem should be obvious when it occurs.
Brodie has a good read on the pulse of Linux, worth following if you want to keep up with linux news.
Readers: make sure you read the replies. This happened four days ago and has since been resolved.
Im a little unfamiliar with navigating this particular mailing list, where was this resolved?
I read through it by clicking the “next” link at the bottom. There isn’t a single email explaining it; it’s a story you have to read through to understand.
If you jump to the last message, it’s someone saying they had the same issue.
But, TL;DR a tool kernel devs use has surprising behavior that’s biting people, and can alter the commit history to be a pack of lies that looks suspiciously like malicious intent.
The thread doesn’t mention how or if the tool has been changed. The tool is
b4.Towards the bottom of that page is a tree with all the replies in the chain.
Here is one where they determined it was not malicious by examining the ref logs
https://lore.kernel.org/all/20250601-pony-of-imaginary-chaos-eaa59e@lemur/
See 2025-06-01 14:40 from Konstantin Ryabitsev. Navigation is just below the post.
It’s pretty annoying to read the mailing list, I agree. There’s a very small hyperlink that says “next” that’s right below the message body. If you click that, you can read the next message in the chain. Keep doing that until you get to the end, and yeah, it looks like this was resolved and wasn’t actually malicious.
Good thing Linus remembers all his sha1 hashes
Bring the anubis girl back!
Found the furry.
> Welp, that precisely recreated it -- even identical shas! Looking at > the b4 output, I do see a suspicious "39 commits" listed for some reason. Well, that's the point where the user, in theory, goes "this is weird, why is it 39 commits," and does Ctrl-C, but I'm happy to accept blame here -- we should be more careful with this operation and bail out whenever we recognize that something has gone wrong. To begin with, we'll output a listing of all the commits that will be rewritten, just to make it more obvious when things are about to go wrong. > So, I assume the "git-filter-repo" invocation is what mangled it. I will > try to dig into what b4 actually asked it to do in the morning... Thanks for looking into this. Linus, this is accurate and I am 100% convinced that there was no malicious intent. My apologies for being part of the mess through the tooling. I will reinstate Kees's account so he can resume his work. -KI have also been done in many times by git-filter-repo. My condolences to the chef.
I’ll say here that one of the less discussed differences between git and Mercurial is that Mercurial does not allow commited history to be changed, and git does. Git users call this a “feature,” and it leads to situations like this which are utterly impossible in Mercurial.
Git allows rewriting history by design. The kernel team uses it liberally. It is debatable whether this is a good thing, but it’s one reason I stick with Mercurial.
Unless commits are signed, you can always rewrite history. No matter the tool. Extreme example demonstrating that this is possible is the fact that I can change my machine’s time, change my user name and reply the tool’s commands to construct whatever history I want.
If you have access to the actual files themselves you can even edit them with a text, binary, or hex editor depending on the format.
Unless you go in with a byte editor, you can’t change Mercurial’s commit history. I didn’t say “fabricate”, I said “change”.
You can, as you say, configure your user name and email to be “Linus Torvalds” and change your computer date and fabricate whatever history you want. You might also be able to go in with a byte editor and fiddle bits and change history that way; Mercurial provides no blockchain-like cryptographic guarantees. But, unlike git, rewriting history is not supported by Mercurial; history is immutable. Rebase doesn’t change history; the commit index only ever increments. Squash and rebasing create new commits, and there history of what happened is always in the repo.
There’s a distinct and clear difference between Mercurial’s immutable history and git’s de jour history rewriting, which can literally - with the git command - change published history to make a commit made 3 years ago look like it was committed by someone else. The git workflow used by the kernel team, and the b4 tool, use this history rewriting in the standard workflow.
If you wanted to do the same thing with Mercurial, you’d have to get a byte editor and start hacking the on-disk format, and it would have to be entirely outside of any Mercurial tooling. And there is some sequential hash verification you’d have to work around, even if it’s not cryptographically auditable.
The point is, with Mercurial it would be hard and the result would be utterly incompatible with any other clone of the repo: there would be no way to propagate your changes to other clones. With git, this is a standard workflow.
Unless you go in with a byte editor, you can’t change Mercurial’s commit history. I didn’t say “fabricate”, I said “change”.
In git you also cannot change history of a commit. You can only create a new commit with a new history. You’re arguing about semantics which don’t change the end result.
The point is, with Mercurial it would be hard and the result would be utterly incompatible with any other clone of the repo: there would be no way to propagate your changes to other clones. With git, this is a standard workflow.
As the example under discussion demonstrates, it’s also impossible to propagate the changes to git clones. Since history changed, merging the pull requests shows all the differences. That’s how Linus noticed the issue.
Did you read the same thread I did?
Looks like Mercurial can change the history just fine using the hg command. You just need to enable it first.
https://book.mercurial-scm.org/read/changing-history.html
Git can also be configured to disable history rewrites.
https://stackoverflow.com/questions/2085871/strategy-for-preventing-or-catching-git-history-rewrite
So the difference between git and hg really just comes down to the defaults.
Safely changing history
How exactly does Mercurial manage to keep you from getting in trouble while changing history?
Mercurial actually keeps track of something called phases. Every time you push changesets to another repository, Mercurial analyses the phases for those changesets and adapts them if necessary.
There are three phases, each resulting in different behaviour:
- secret: This phase indicates that a changeset should not be shared with others. If someone else pulls from your repository, or you push to another repository, these changesets will not be pushed along.
- draft: This is the default phase for any new changeset you create. It indicates that the changeset has not yet been pushed to another repository. Pushing this changeset to another repository will change its phase to public.
- public: This phase indicates that a changeset was already shared with others. This means changing history should not be allowed if it includes this changeset.
Mercurial uses this information to determine which changesets are allowed to be changed. It also determines how to select changes to rebase or histedit automatically. For example: using histedit without any arguments will only show draft and secret changesets for you to change.
You can not change history for any published changes - like I said, doing so makes your repository incompatible with any other clone.
You can not change history for any published changes - like I said, doing so makes your repository incompatible with any other clone.
That’s the same on Git.
It is not standard workflow in git to change the commit history for a branch on the remote. You have to use
--force, and the next time someone pulls they also have to--forcetheir any local tracking branch to follow the remote. Every git guide on the internet warns against pushing a rebase for this reason.Locally you can do whatever. I’m not familiar with Mercurial, but I assume it must work the same as git: I can do whatever I want locally, and only what I push matters. And when I’m doing stupid stuff locally as I organize my changes, rebase is handy.
Read the thread. The Kernel maintainers use b4, which rewrites history.
Mercurial does not work like git, and history is immutable: there are no commands for changing history.
Yes, it rewrote history, and thanks to Git’s robustness it was extremely easy to notice and identify. Forced rewrites are not an issue if you trust people on your repo, and if you don’t (and honestly you shouldn’t, everyone fucks up), you can disable force rewrites in the remote
I’m responding to the literal words you said that were inaccurate. Cheers.
Wait was that Anubis without an anime girl? YOU MONSTERRRRS!
what’s the reference here?
Anubis is an anti bot protection measure that gives your browser a proof-of-work challenge to solve before giving you access to the website. When I opened the link the website briefly showed Anubis but the anime girl mascot wasn’t there 😭
I’ve used these tools to remove stuff from git history (e.g. someone accidentally committed a password or key that wasn’t noticed for a while) and they are powerful but scary. Good discussion on what when wrong and how to avoid it or at least notice it before it gets this far
WHOOOOOA. If Linus is not mistaken (doubtful), there wasn’t an intrusion in the repo, or there wasn’t some fucked up merge somewhere, this is crazy as hell. This is a huge deal. Good on Linus for catching it.
If you read the whole thread, it turns out to be an undesirable behaviour of a tool called b4, which was rewriting not just author information but committer information. The consensus seems to be that this tool needs to be updated not to do that.
It was in fact a microscopic deal. Linus overreacted. Lemmy and Reddit milked the drama.
Linus’ tone was disproportionate, but he wasn’t wrong. This could easily have been a compromised account trying to sneak code into the kernel.
His tone is almost always disproportionate. It’s just who he is.
If it was compromised account trying to sneak code into the kernel, the attacker wouldn’t rewrite history since that would be obviously flagged when Linus tries to merge the pull request; as demonstrated by Linus in fact noticing the rewritten history. There was virtually no chance that this was an attack.
Well, that’s kind of his personality though. Just reading the message it seemed like quite an event though. The mailing list is generally very transactional and uneventful.
Well, that’s kind of his personality though.
Yes. Linus is known to overreact and use colourful language.
Linux Torvalds / Tim Apple
Pam from The Office: they are the same (picture)
Sorry, couldn’t be bothered to fire up a meme generator for that joke
why do they keep those email public?
same as why they keep source code public
They are a record of the process of adding to the Linux kernel. Such background can be used to trace the history of contributions if those contributions turn out to have had malicious intent or were derived from code that came from sources that were not compatible with the GNU license that the kernel is released under.
I
The thread reads like Git is just as awesome to use as one would suspect from the outside, even for the original target audience.
Once a user starts performing merges and trying to fix conflicts by hand, simplicity goes out the windows. It’s not git, but any tree system that has to track and merge content. Same thing would happen with any other tool if the user is trying to “recreate” revisions that had bad merges.
A script called b4 was at fault but sure blame git
More like git blame amiright?!
fatal: not a git repository (or any parent up to mount point /) Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).(u r not rite)
