This note seems to be the problem, as the class path doesn't change when the script is being ran by the *.app, but works when being ran when pms isn't running.
Note: Since applications do access the defaults system while they're running, you shouldn't modify the defaults of a running application. If you change a default in a domain that belongs to a running application, the application won't see the change and might even overwrite the default.
I am now looking how to inject all plugins (in a static way) when building the app file, as suggested by chocolateboy.
Anyone knows of another way the jars could be included in the class path? It takes away some sexyness of the plugin system if the plugins have to be known beforehand, but would be a viable solution.