Although only mentioned in a single line in this article, `git bisect` is well worth reading about by itself. I found the following tutorial to be extremely helpful:
> Finding yourself in a situation with both a working and a buggy system is quite common.
Man, I wish it were common -- it's pretty rare, at least in my experience. From my experience, most bugs are introduced together with new features/functionality, and there was no time when the bug wasn't present.
But when you do find yourself with the working/buggy versions, a lot of the time source-control will make it quite clear what changed! Either that, or a hardware/os configuration changed...
https://www.kernel.org/pub/software/scm/git/docs/git-bisect....