Let me share with you something baffling about this issue as I've been testing it since yesterday as I am in the process of releasing an app.
I've so far tried on
1. Android Studio v.4.2.2, and Android Studio Arctic Fox patch 2 and patch 3
2. use NDK v.2.2.0, v.2.3.0 and v.2.3.1
3. combine with the keystore file generated from GMS2 IDE v.2.3.5 STABLE and GMS2 IDE v.2.3.6 BETA.
4. Target API 31 Min 16, Target API 31 Min 19, Target API 30 Min 16, Target API 30 Min 19
Always end result is BUILD FAILED with the bad key decryption message as you received.
But when I test using keystore from different projects that was generated using IDE v.2.3.4 or older, on the IDE v.2.3.5 with either of those 2 Android Studio mentioned above ..... BUILD SUCCESS! How weird can that be? LOL
I think something is messed up with the build.gradle, but I can't seems to pinpoint what it is.
Edit:
I've finally identified the root of the problem.
The problem is the KEYSTORE file itself! While no problem generating and show hash using 2 different password for Common Name and Alias. Our Alias Password in the keystore file doesn't match the one in the above input box being read by the keystore tool, why? because the decryption tool is instead looking for our Alias Password in the common name's password column when building apk/aab/test on android target. This mismatch causes it to fail to decrypt to build our app. How I found out was simple;
I did a test with 2 copies of Keystore files; 1 from v.2.3.5 the other from a project I made in v..2.3.4. I ran it on a keystore editor, did a simple change from PKCS12 to JKS, it ask for password to my alias, I input the alias password it tells me it was wrong, I input the password to my common name, it works. So apparently GMS2 create our keystore the way we want/expected based on what we input in the above, but when it read the keystore for building process, the decryption process tried reading our common name's password for the alias instead of use the alias password.
That said, to solve the build fail:
If lets say currently if your already generated KEYSTORE FILE, the PASSWORD and ALIAS PASSWORD are different (like it is in my case), after SHOW HASH KEY you delete the ALIAS PASSWORD column and type in your Common Name's PASSWORD, then you press APPLY, and then do the build. OR, just make a new KEYSTORE with the PASSWORD and ALIAS PASSWORD set the same if it's a new project. I know it adds to the risk of being easier to hack. While for project already released to Play Store, if using its keystore made using older IDE, if it fail to build on v.2.3.5 Stable/v.2.3.6 Beta, I suggest wait until a fix is released.
If you go to C:\Users\[PCNAME]\AppData\Roaming\GameMakerStudio2\Cache\GMS2CACHE\[PROJECTNAME]\Android\Default\[PROJECTNAME] after successful build, then look into the "build.gradle", file you will see that both PASSWORD and ALIAS PASSWORD will be the same. While the FAILED one will show the 2 password aren't the same (which would be what at least mine was originally). That said, we're currently being forced to implement same password for both columns.
But again it's strange that there's no problem with it recognizing 2 different passwords in PASSWORD and ALIAS PASSWORD when come to keystore file generated in older IDE - Runtime. LOL
That said, nudging
@Dan and
@rwkay to fix this since I've detected the same issue in v.2.3.6 Beta as well.
