Regression analysis - 5.1.0 (fixed 5.1.1)
What happened?
A patch release was created for 5.1 (5.1.1) that included a fix for a regression that occurred relating to FULL_GROUP_BY mode for mysql. The fix did not include a pre-requisite patch that was already in the rc and in master, and hence it caused a more serious, wide-ranging regression
How & When was it addressed
The new regression was fixed & a new patch release was issued within 24 hours
How did the regression fit with our processes?
Our processes say that we should issue patch releases to address any regressions that have occurred in the current release (so that we are raising the bar each release). Technically the regression we were trying to fix was actually from the prior release (5.0) and would not meet that bar. The regression was tested by the submitter but there was confusion about the combination of patches tested, ultimately leading to the belief that the one released had been confirmed when it hadn’t (dev/core#106 (comment 4507))
Context This happened on the second release on the 5.x series. We are definitely still bedding in processes for the new patch release system and communications / timeframes & expectations are still in flux. We have a bottleneck in Tim for the actual releasing of them & it’s hard to know when he will allocate time for what could be described as a ‘regular patch release’.
We put out a patch for a fix that was not actually a 5.1 regression in part because there is a specific intent to get the ‘current 5.x version’ to being really solid & hence we have been very sensitive to regressions, which might have made us jump the gun. We had also had a user complain fairly vigorously that there was a bug fixed in 5.1 that was not released as a patch release on 5.0.
The fix itself was a FULL_GROUP_BY fix (support for recent MySQL & MariaDB releases) - which is an area in which we have had a lot of whack-a-mole and we feel we have a viable approach to this technical problem now (of which this patch is part). Basically the plan is We have enabled FULL_GROUP_BY mode on the CI box We are now disabling FULL_GROUP_BY mode for specific problematic queries as a first line of defence We are trying to fix the queries to comply with the FULL_GROUP_BY standard only in the context of writing unit tests & are trying not to do further fixes on the QUERY builder for them due to it’s complexity & brittleness.
Recommendations going forwards The whole patch release process is in a lot of flux so it’s expected we will have some teething issues. Developing a clearer process / set of expectations is the next step.