CiviCRM Shortcode issue with YoastSEO

On one of our sites we are using a contribution form via a shortcode on a std WP page. The form is loading just fine on the page, but when the user hits the "Review Contribution" button, the shortcode itself is rendered on the page instead of the review form for the contribution page. After doing some testing on the site, it seems there is a conflict with YoastSEO. I tried to downgrade YoastSEO back to 16.6.1, and still have the issue.

This was previously working on an older version of CiviCRM for WP. Version 5.38 is working just fine and so was 5.37 of CiviCRM. But when upgrading to 5.39, the contribution shortcode is failing on the review step. Below are two vanilla instances of WP with CiviCRM and YoastSEO installed.

Both are on WP 5.7 and using the CiviCRM Demo data and WP theme. The contribution page for both of these is the default demo Membership contribution page, ID 2. The CiviCRM v5.38 install is working as expected, but v5.39 is breaking. If I remove YoastSEO, this works again on both. Since it was working before with Yoast installed, there is something introduced in 5.39 that is causing issues while having Yoast installed.

CiviCRM v5.38 https://civcrmtest.graydigitaldev.net/sample-page/

CiviCRM v5.39 https://civicrm.graydigitaldev.net/sample-page/