GitLab B.V. CEO here. The proposed diff is interesting and I love the way they explained the idea. During the development of GitLab (over 10.000 commits) we never experienced the problem that was mentioned. Maybe it is because our codebase contains many tests. One related idea that a GitLab user proposed is not testing the feature branch but testing the merge of the feature branch into master. This would require retesting of all feature branches on every commit to master. We're open to proposals to add this to GitLab CI. But so far we didn't experience this problem frequently.
The idea of testing the merge commit is a great one. Even if the test suite is able to catch the 'logical conflict' mentioned in the article, current solutions will only catch the conflict after it's merged into master.
Of course, it's not too hard for the person responsible for merging the pull request to replicate this themselves (branch off master, merge in feature branch, run tests).
I hear GitLab EE has a 'rebase before merge' checkbox next to the 'merge' button. What I'd like to see is have it enforced, that is, the UI won't allow a merge if the MR has not been rebased onto master's tip. Enforcing this policy has saved us a few headaches, including the article's case which happened tree times on a project counting ~2000 commits.
(BTW, we love GitLab, congrats on the new UI layout, which has rave reviews coming back to me mere hours after the upgrade)
It is exactly what Travis CI does for auto-PR testing on Github: they pull in master (or whatever base) and fetch the feature branch. They create the commit themselves and test the result. That way, you always get the tests based on the merged result which gives far more insights than tests on your (local) feature branch.
