So, the OSX app had a problem where attempting to execute it would yield a popup saying "The application can't be opened" because of stripped execution rights in the package's contents.

I first thought it was the new Catalina gatekeeper being more aggressive but it was impossible to allow the app in the system preferences.

It turned out to be an issue caused by the auto-unzipping by the web browser upon download when the archive was produced by zipping the .app directory itself. 
Creating the archive by putting the app in an otherwise empty directory fixed the issue. The app still needs to be either notarized or manually authorized in the system preferences after the gatekeeper catches a first attempt at running it.