Automating Code Signing with multiple E-Business Suite instances - Part 3

Automating Code Signing with multiple E-Business Suite instances - Part 3

Johannes Michler PROMATIS Horus Oracle


Executive Vice President – Head of Platforms & Development

 

As you are probably aware by now, since June 1st 2023 all well-known / public Certificate Authorities (CA) no longer provide Code Signing Certificates using pure software based private keys (see https://www.linkedin.com/posts/johannes-michler-099892ab_code-signing-key-storage-requirements-will-activity-7090432157688492032-jGvC).

Since I prefer using such a trusted / public CA to sign Java Applets (that are still crucial for Oracle E-Business Suite or Oracle Forms) I've recently had a look into how we can now sign those Java JAR files. Part 1 of this blog series introduced the topic and an available "Cloud Based" Code Signing Certificate provider: Signing EBS/Forms - Part 1

In a second part I covered how the code signing can be done on an E-Business Suite Application Server running on Oracle Linux 7 on Oracle Cloud Infrastructure (OCI).

This third post will look how we can further automate this by installing the Certum tools onto the E-Business Suite Cloud Manager VM. First, we'll cover the latest changes from Certum, then we'll look into some scripts that can be used on multiple E-Business Suite Application servers to send the .jar files for signing to that central signing instance.

Certum Tool updates (April 2024)

Back in the previous post, I've complained about the incomplete translation of the Certum tools still revealing a lot of polish error messages. While it seems this is fixed at least partially, I realized that the 2.9.9 versions available over there https://files.certum.eu/software/SimplySignDesktop/Linux-RedHat/ leads to fatal crashes (segmentation fault). That is why for now I stuck with the 2.9.8 release.

Installing Certum SimplySignDesktop as a non-root user

When installing the SimplySignDesktop tool according to the official documentation it is necessary to do so globally/as the root user. Since I didn't like the tool to modify my cloud manager VM in that massive way, I've investigated what the installer actually does. With that I was able to get the tool running with a way less privileged user (that I call certum). Run the following as root:

yum install https://rpmfind.net/linux/epel/8/Everything/x86_64/Packages/s/stalonetray-0.8.3-15.el8.x86_64.rpm
yum install libxslt.x86_64 pulseaudio-libs-glib2.x86_64 libwebp.x86_64 xkeyboard-config
useradd certum
sudo su – certum
mkdir .ssh
vi .ssh/authorized_keys
# add the SSH public key(s) of your oracle@ebs-appserver
chmod 700 .ssh
chmod 600 .ssh/authorized_keys

Then connect a SSH Session with X-Forwarding as certum:

wget https://files.certum.eu/software/SimplySignDesktop/Linux-RedHat/2.9.8-9.1.6.0/SimplySignDesktop-2.9.8-9.1.6.0-x86_64-prod-centos.bin
sh SimplySignDesktop-2.9.8-9.1.6.0-x86_64-prod-centos.bin --target /home/certum/
cp /home/certum/SSD-2.9.8-dist/SimplySignDesktop.xml /home/certum/

Create a /home/certum/provider_simplysign.cfg file as follows:

name=SimplySignDesktop/SimplySignPKCS
library=/home/certum/SSD-2.9.8-dist/SimplySignPKCS_64-MS-1.0.20.so
slot=-1

Furthermore, create a script startGUI.sh as follows:

export LD_LIBRARY_PATH=/home/certum/SSD-2.9.8-dist/
export QT_QPA_PLATFORM_PLUGIN_PATH=/home/certum/SSD-2.9.8-dist/plugins
export OPENSSL_CONF=/etc/ssl/
stalonetray &
/home/certum/SSD-2.9.8-dist/SimplySignDesktop

Finally start the Script and sign in with a one-time-token.

Do a test as follows (in new SSH Session):

/home/certum/SS-9.1.6.0-dist/jre/bin/keytool -list -keystore NONE -storetype PKCS11 -providerclass sun.security.pkcs11.SunPKCS11 -providerArg /home/certum/provider_simplysign.cfg -v

This will provide an alias, in our case: 4F4F410D1234A9110B16DA9C83BD6F59

Furthermore, create a /home/certum/mychain.pem file as described in the previous episode.

Passing the jars

On the E-Business Apps-Server first create a ~/sign_1.sh script as follows:

folderstamp=$(date +%Y-%m-%d-%H:%M)
mkdir -p /home/oracle/sign_bkp/${folderstamp}
jar=$1
# Remove Signature from jar files created through ADADMIN in EBS
echo " ** Removing EBS signature from: ${jar} "
cp -i ${jar} /home/oracle/sign_bkp/${folderstamp}/
zip -d ${jar} 'META-INF/*.SF' 'META-INF/*.RSA'
scp ${jar} certum@10.1.2.199:/tmp/signing-dummy.jar

ssh certum@10.1.2.199 "/home/certum/SS-9.1.6.0-dist/jre/bin/jarsigner -keystore NONE -tsa \"http://time.certum.pl\" -certchain /home/certum/mychain.pem -storetype PKCS11 -providerClass sun.security.pkcs11.SunPKCS11 -providerArg /home/certum/provider_simplysign.cfg -storepass 12345 /tmp/signing-dummy.jar 4F4F410D1234A9110B16DA9C83BD6F59"

scp certum@10.1.2.199:/tmp/signing-dummy.jar ${jar}

The script first creates a backup of the jar, then un-signs the .jar files and copies it to the cloud-manager VM (in my case with IP 10.1.2.199). There the jar is signed and finally the signed .jar is copied back to the E-Business Suite Apps Tier.

This allows signing a single .jar file; the script may be helpful when applying a patch with "options=nojarsigning". Then in there should be a file such as /u01/install/APPS/fs_ne/EBSapps/log/adop/176/20240327_132920/apply/mastebsapp01/36177213/log/jarlist.txt containing all the .jar files that require re-signing.

For the initial signing the procedure in the previous episode can be combined with the copying of the .jar to the Cloud Manager VM.

Verifying and patching

As an alternative to signing "just" the files in $NE_BASE/EBSapps/log/adadmin/log/jarlist.txt I found it useful to just sign all .jar files under $JAVA_TOP. For this the following script proved helpful:

folderstamp=$(date +%Y-%m-%d-%H:%M)
mkdir -p /home/oracle/sign_bkp/${folderstamp}
# Select the jar files from jarlist.txt
for jar in $(find $JAVA_TOP/oracle/apps -name \*.jar)
do
# Remove Signature from jar files created through ADADMIN in EBS
echo " ** Removing EBS signature from: ${jar} "
cp -i ${jar} /home/oracle/sign_bkp/${folderstamp}/
zip -d ${jar} 'META-INF/*.SF' 'META-INF/*.RSA'
scp ${jar} certum@10.1.2.199:/tmp/signing-dummy.jar
ssh certum@10.1.2.199 "/home/certum/SS-9.1.6.0-dist/jre/bin/jarsigner -keystore NONE -tsa \"http://time.certum.pl\" -certchain /home/certum/mychain.pem -storetype PKCS11 -providerClass sun.security.pkcs11.SunPKCS11 -providerArg /home/certum/provider_simplysign.cfg -storepass 12345 /tmp/signing-dummy.jar 4F4F410D1234A9110B16DA9C83BD6F59"
scp certum@10.1.2.199:/tmp/signing-dummy.jar ${jar}
done

It is helpful to first to a check if the $jar is already signed as follows:

result=`jarsigner -verify -certs ${jar}| tr -d '[:space:]'`
if [[ "jarverified." != "$result" ]]
then
echo ${jar} needs re-sign; $result
# put the signing here
fi

Summary

Using above scripts, it is amazingly easy to sign all .jar files both initially as well as after applying a patch. The version using a "find" on $JAVA_TOP may sign "a bit more than needed", but in my experience that does not do any harm.

I am still hoping that Oracle will provide a way to "Hook" a script such as sign_1.sh into the signing process called during patching or through adadmin. This would probably be announced in "Signing EBS Jar Files With HSM (Hardware Security Module) - (Doc ID 2806640.1)".

PROMATIS receives Certificate: Expertise in Oracle E-Business Suite Applications to Oracle Cloud in Western Europe

PROMATIS receives Certificate: Expertise in Oracle E-Business Suite Applications to Oracle Cloud in Western Europe

We’re delighted to announce that we’ve achieved Service Expertise in Oracle E-Business Suite Applications to Oracle Cloud in Western Europe. This further demonstrates our commitment to provide holistic and seamless Oracle Cloud implementations.

This award not only testifies to our high level of expertise in Oracle E-Business Suite, but it also presents the satisfaction of our customers.

The Expertise Initiative was launched by Oracle to provide more transparency on the specific capabilities of Oracle partner companies. To receive a certification, an Oracle partner company must meet strict criteria: On the one hand, the requirements include the availability of enough certified consultants in the company, and on the other hand, corresponding customer references must be submitted, which confirm the company’s own project successes for the respective focus field.

Oracle named a Leader in the Forrester Wave for Digital Experience Platforms 2021 report

This year’s report evaluated the 13 most significant digital experience platform providers based on 26 criteria. Oracle Advertising and Customer Experience (CX) was one of only four Leaders and received the highest possible score in 19 of the 26 criteria including: vision, market approach, and partner ecosystem, as well as campaign management, digital commerce, customer analytics, and customer journey management.

Oracle Advertising and CX includes a wide range of digital experience capabilities within a suite of connected applications that help you build a complete view of your customer and their every interaction across advertising, marketing, sales, service, and ecommerce.

Read here the Forrester Report.

Latest E-Business Suite Cloud Manager 20.2.1.1

Latest E-Business Suite Cloud Manager 20.2.1.1

Johannes Michler PROMATIS Horus Oracle


Senior Vice President – Head of Platforms & Development

Oracle released a new release of the E-Business Suite last week (on April 23, to be more exact). While so far there are no official announcements in the Oracle Blog, the My Oracle Support Note describing Cloud manager (2517025.1) gives an overview of the new features – so let’s have a closer (first) look at the new release.

Support for automatic time zone setting

In the past every E-Business Suite instance provisioned through Cloud Manager was set up with an “UTC” timezone. For many customers, this did not work well obviously. So far the workaround we used with our customers was to manually run the following command:

timedatectl set-timezone Europe/Berlin on every newly provisioned (e.g restored from a backup) instance. With the latest release this is no longer necessary: The “backup utility” saves the timezone set up in the source system and then during restore, this timezone is automatically preserved:

Support for Environment Rediscovery after 12.2 upgrades

The second new feature is even more important. When previously performing an upgrade from E-Business Suite 12.1.3 to the latest 12.2.10 version the Cloud Manager kind of “lost track” of the E-Business Suite instance it is managing: The tool is still under the impression, that it is managing a 12.1.3 version of E-Business Suite residing in different directories. When then trying to e.g. “scale out” (add a second node) or create a backup or clone of the instance, the operation was failing with all kinds of errors.

This is now fixed with the 20.2.1.1 release of Cloud Manager and it is now possible to rediscover the environment after an upgrade.

In the past we had worked around this by performing a manual backup with the “lift and shift” utility and then restoring it as a new environment. This workaround though has a significant drawback by adding appx. 6-8 hours (depending on the size of the environment) to the downtime needed for the upgrade from 12.1.3 to 12.2.

Summary

Besides the new major features as usual a bunch of smaller bugs have been fixed. And more important: At least for the two customers where we have already given the release a try no new regressions came up and basic backup/restore or cloning operations are still working as expected without any problems.