A composer plugin created to make it easy for humans to upgrade their patched Drupal websites.
If you ever wanted to know if you still needed that patch after an update, or if your custom patch would still apply to the new version of the module, or if your 15 levels patch tower still applies after the update, this is the tool for you. Install the composer plugin and get started using the remote API.
$ composer require --dev tresbientech/drupal-patch-checkIf you want the short version, feel free to read the package readme.
What is it?
The composer plugin will help with patches on public drupal.org modules. It doesn't check or even see the private or custom modules used on your website. This works by calling api.tresbien.tech/v1/composer/scan with a filtered version of your composer.json, composer.lock files as well as the contents of all the associated patches, only for public drupal.org modules. The endpoint is fully controlled by myself, a Drupal core committer, and the data processed will be used to inform how we can prioritize issues for example. There is an option to obfuscate patch paths to avoid disclosing ticket numbers and such.
The API in turns respond with a rich set of data we're going to explore on a real life composer.json I had to clean up earlier this year by hand.
Starting the patch clean-up
The initial composer.json was broken, a composer install that would not install, and when it finally did patches would barely apply. First step in that situation once the composer install actually finishes:
$ composer drupal-patch-check
// or the cuter alternative
$ composer drupatch:checkWith that I know what will need work and what can be quickly sorted out. Let's review the output:
Summary
Drupal Patch Check: 51 patches against the releases this site installs
simplesamlphp/simplesamlphp 1 patch skipped (not a drupal.org project)
acquia/cohesion 8.2.1 6 patches skipped (not a drupal.org project)Tells you how many patches total you have in the composer.json, and all the modules and patches that will not be reviewed because they're not against public Drupal modules. To be clear this data is from your local composer plugin, the API never see the list of private modules. The names of custom modules are never sent away. If you want to review what is sent out before using this tool, you can run composer drupatch:check --dry-run, it will output the exact JSON sent to the API.
Patch data
There are various situation your patch can be in. Starting with the easy ones.
Everything is fine, or fine enough:
drupal/entity_reference_revisions 1.12.0 1 applies
#1 · applies Draft translations should be based on the lat… moderation-translation-err-rc1-…
drupal/permissions_by_term 3.1.40 1 applies
#1 · applies Extend permissions by edit and create permiss… access_for_view_nodes.patch
context drifted, needed: git apply -p1 -C1 --ignore-whitespace --recountFirst patch applies no problems, next patch targets an older version of the module and a few lines of context changed, not a problem, this will install without warning.
Looks bad, great news:
drupal/geoip dev-3.x 1 conflicts
#1 ! conflicts Automated Drupal 10 compatibility fixes gepop.3.0.x-update-to-d10.patch
geoip.info.yml:1: patch failed
tests/src/Kernel/GeoLocatorPluginTest.php:31: already in the release, not needed
tests/src/Kernel/RequirementsTest.php:19: already in the release, not needed
tests/src/Kernel/RequirementsTest.php:31: already in the release, not needed
tests/src/Unit/CdnGeoLocatorTest.php:2: patch failed
tests/src/Unit/CdnGeoLocatorTest.php:17: patch failed
tests/src/Unit/GeoLocationTest.php:2: patch failed
run composer drupatch:reroll to see if the release has the restLook at that last line, the whole reason I started working on this tool. If parts of the patch are in the release, there is a good chance the patch is not needed anymore. We'll get confirmation on the next command. Spoiler alert, it's not needed anymore.
The patch tower™:
drupal/core 11.2.5 4 conflicts, 11 applies
#1 ! conflicts Fix Drupal Views filters boolean-no-value-2862828.patch
core/modules/views/src/Plugin/views/filter/BooleanOperator.php:2: patch failed
core/modules/views/src/Plugin/views/filter/BooleanOperator.php:108: patch failed
core/modules/views/src/Plugin/views/filter/BooleanOperator.php:149: patch failed
#2 · applies Error after saving node drupal 8.7 issue ignore_width_height_on_untransl…
judged after #1 applied in part
(…)
#7 · applies Argument 1 passed 3243387-argument-1-passed-8.pat…
judged with only the part of #1 that applied
#8 ! conflicts TIC-316: Patched parent links (https://www.dr… 3110371-56.patch
core/modules/menu_link_content/src/Form/MenuLinkContentForm.php:92: patch failed
judged after #1 applied in part
(…)
#15 · applies MenuTreeStorage shouldn't invalidate cache ta… 10023.diff
judged after #1 applied in partThe classic 15 patch Drupal core install where one patch introduces a change, and a second or third patch modifies the contents of the previous patch. Usually it's a custom change applied on top of a public patch from drupal.org and it's pretty common. The very first patch fails to apply, awesome… Since this will not stop other patches from being applied on composer install, we continue with the parts that could apply properly. This is explained by the "judged after #1 applied in part" line. In the end we have 11 patches that apply properly and 4 where there are conflicts on the drupal/core package.
Footer
patches: 35 applies, 16 conflicts
composer applied these patches at install, so the files on disk show them
Next: composer drupatch:reroll writes the 16 re-rollsOur 35 patches to review turned into 16 patches to review. Not bad. Now to deal with the 16 patches remaining we apparently need to be calling drupatch:reroll, we're doing that next.
Fixing problems
Let's see how much of the remaining 16 patches can we figure out automatically
$ composer drupatch:rerollBig picture
Drupal Patch Check: 51 patches against the releases this site installs
patches: 2 now apply, 13 conflicts left, 1 to dropThat's 3 patches taken care of, 13 to have a look at. Much better than the 51 from the start.
Not needed anymore:
already in the release, drop it:
drupal/geoip: Automated Drupal 10 compatibility fixes
https://www.drupal.org/files/issues/2023-06-16/gepop.3.0.x-update-to-d10.patchThe whole point of this exercise. We found a patch we don't need and that can be removed. It didn't show up on the initial scan because there is some extra processing done when calling reroll that would be a bit too expensive to run on the initial scan all the time.
Rerolled automatically:
re-rolled:
patches/core/boolean-no-value-2862828.patch (verified against the release)
the merge kept both additions in 2 regions, check it:
core/modules/views/src/Plugin/views/filter/BooleanOperator.php:194
core/modules/views/src/Plugin/views/filter/BooleanOperator.php:356
patches/allowed_languages/access-denied-page-builder.patch (verified against the release)Taking a closer look at the "2 now apply" patches. Git was able to figure out the merge, excellent. Patches contents are updated, no more errors on composer install for these two.
Conflits:
re-rolled with conflicts:
patches/acquia_dam/acquia_dam_versioning_cron_timestamp_fix.conflict.patch (1 region to decide)
src/Cron.php region 0
patches/addtoany/add_SRI_to_JS_file.conflict.patch (1 region to decide)
addtoany.libraries.yml region 0
patches/diff/3359192-display-current-revision-11.conflict.patch (3 regions to decide)
src/Form/RevisionOverviewForm.php region 0
src/Form/RevisionOverviewForm.php region 1
src/Form/RevisionOverviewForm.php region 2
patches/moderated_content_bulk_publish/fix_publish_latest_revision.conflict.patch (1 region to decide)
src/AdminModeratedContent.php region 0
the merge kept both additions in 1 region, check it:
src/AdminModeratedContent.php:258That's where we start to need to look at some code and make decisions about what to keep or remove from the patch. The tool will point to the patch, the file, and region that needs to be resolved. Once it's resolved another command will apply them to the patch we'll see that afterwards.
Can't reroll:
Sometimes patches are not well enough structured to get them evaluated, the patches need some work to get the service to be able to do anything with them. There is some guidance about what is wrong with the patches for the service to be able to do something with them.
not re-rolled:
it is declared as a URL, so there is no file to replace
https://git.drupalcode.org/project/drupal/-/merge_requests/4098.diff drupal/core: Warning: Undefined array key "#access" in language_form_alter()
https://www.drupal.org/files/issues/2024-01-11/3110371-56.patch drupal/core: TIC-316: Patched parent links (https://www.drupal.org/node/3250632)
cannot re-roll: no release takes this patch, and it does not say which one it was made from; remake it with git diff or re-roll it by hand
patches/core/fix-content-language-expansion-logic.patch drupal/core: Fix content language expansion logic https://www.drupal.org/project/drupal/issues/3414415
patches/acquia_dam/set-placeholder-images.patch drupal/acquia_dam: Set placeholder image if DAM is not responding
patches/acquia_dam/TIC-1341_Fix_SVG_dam_rendering_with_no_width_height.patch drupal/acquia_dam: Fix SVG width height error
patches/acquia_dam/TAC-1415___set_dam_assets_cache_to_1_month.patch drupal/acquia_dam: Set DAM assets cache to 1 month
patches/acquia_dam/SVG_as_Acquia_DAM__Image.patch drupal/acquia_dam: SVG_as_Acquia_DAM__Image
patches/simple_sitemap/TAC-1421___remove_translation_set.patch drupal/simple_sitemap: Remove translation set
error: repository lacks the necessary blob to perform 3-way merge.
patches/tmgmt/tic_648_metatags_tokens_and_fix.patch drupal/tmgmt: TIC-648: Fixed metatag tmgmt + filter tokensFooter
Next: composer drupatch:reroll sends the regions you decide in the 4 conflict files
composer drupatch:reroll --update drops the shipped entry from composer.json and adopts the 2 patches declared as URLsNow we're almost out of the woods. Resolve the conflicts, deal with the 9 patches the service can't help with. Eventually after running composer drupatch:reroll enough times patch status will be clean.
After some work I got to a clean composer install state. The site is still outdated, new core and contrib releases went out, what happens when we try to update the site? which patches will break?
Updating to major version
Clean-up is done, our Drupal 10 version is clean, no patch problems. To go from there to Drupal 11 we can run the drupatch command with a target version:
$ composer drupatch:check --target=11.4.6The header is the same pretty much, a few patches changed to make it work so the count is different, not really interesting. The upgrade data is though:
drupal/core 10.2.4 → 11.4.6 5 conflicts, 10 applies
#1 ! conflicts Fix Drupal Views filters boolean-no-value-2862828.patch
core/modules/views/src/Plugin/views/filter/BooleanOperator.php:2: patch failed
core/modules/views/src/Plugin/views/filter/BooleanOperator.php:108: patch failed
core/modules/views/src/Plugin/views/filter/BooleanOperator.php:149: patch failed
(…)
drupal/yoast_seo 2.0.0-alpha10 → 2.2.0 1 conflicts, 1 merged
#1 ✓ merged Uncaught DOMException: Failed to execute 'rem… 3394487-failed-to-execute-remov…
#2 ! conflicts Metatags depending on URL cause errors for un… yoast_seo-3110455-22.patch
src/EntityAnalyser.php:211: patch failed
drupal/search_api_solr_multilingual 3.1.0 1 unknown
#1 ? unknown Drupal 10 compatibility Drupal-10-Compatibility-3370449…
drupal/search_api_solr_multilingual has no release for 11.4.6: the package blocks the upgrade, so its patches cannot be judged
drupal/menu_item_extras 2.19.0 1 unknown
! 3.1.2 supports 11.4.6; the site requires ^2.19. Widen it to ^3.1.
#1 ? unknown Custom fix for client menu_item__extras_client_fix_1.…
3.1.2 supports 11.4.6; the site requires ^2.19. Widen it to ^3.1. A patch that was shipped in the newest release! we can drop this patch and update, nice. We also have some problems with updating a patched module, there are no releases for Drupal 11, impossible to check the patch. We can see that the menu_item_extras has a release that supports Drupal 11, but we can't check it because the composer constraints don't allow it. And we end up with:
patches: 27 applies, 24 conflicts, 1 merged, 4 unknown
composer already applied these patches to your files
Next: composer drupatch:reroll --target 11.4.6 writes the 24 re-rolls
composer drupatch:reroll --target 11.4.6 --update drops the shipped entry from composer.jsonwe should first do a reroll to get the 24 conflict files, resolve the problems and call reroll --update to create the conflict files, resolve what can be and remove patches that ship with the new release.
Reroll please:
The only difference with the command earlier is that we have a couple of patches dropped:
Drupal Patch Check: 56 patches for a move from core 10.2.4 to 11.4.6
patches: 2 now apply, 21 conflicts left, 4 unknown
already in the release, dropped:
drupal/geoip: Automated Drupal 10 compatibility fixes
https://www.drupal.org/files/issues/2023-06-16/gepop.3.0.x-update-to-d10.patchAnd now after resolving the 21 conflicts the site will be fine. Faster than reviewing the 56 patches manually. Process is the same as before, resolve conflicts, and keep calling composer drupatch:reroll --target 11.4.6 until the output is ok.
Wrapping up
Why a remote service?
Conceptually there is no need to have the plugin send data out, it could be all resolved locally. It's not going to take a second or two though. It would need some pretty extensive processing and guards to make sure it's all safe and correct. It is much easier and a few orders of magnitude faster since all the heavy data is already available at hand on my end. For a few years now I've built an index of a lot of metadata about Drupal core and contrib, turning that data into a useful service seemed easy enough.
TL:DR;
$ composer drupatch:check
$ composer drupatch:check --target=latest
$ composer drupatch:reroll --target=latestYou can also run this in CI, as a hook after every composer update, filter by package, or by patch, and a few more things the endpoint can do but it's a long enough post. I hope this will be useful. I know I'll be using it at least.
Help needed
Thanks for reading so far. In order to improve the tool I need more examples of real and vetted upgrades I have a handful of examples to use myself but I would need a bit more examples. I need people to supply me with the --dry-run data for the site before an upgrade and after the upgrade so they can be compared and build-in some of the special Drupal sauce in the service that can benefit everyone.
If you do not want to have the patch path submitted as part of using this tool, you can obfuscate private paths with this in your composer.json:
{ "extra": { "drupal-patch-check": { "private-paths": true } } }# In the checkout from before the upgrade
composer drupatch:check --dry-run > /tmp/drupatch-before.json
# In the checkout from after
composer drupatch:check --dry-run > /tmp/drupatch-after.json
# One file to send
tar czf /tmp/drupatch-state.tgz -C /tmp drupatch-before.json drupatch-after.jsonAnd send me that file through slack or any other means you're comfortable with. Thank you!