Recently, I described that starting with Release 22.2.1, the Oracle E-Business Suite Cloud Manager now supports OCI File Storage Service (FSS) besides block storage. Furthermore, the usage of FSS is mandatory for (new) multi-node environments: First Experience with E-Business Suite Cloud Manager 22.2.1

Given the advantages of FSS described in my previous blog one of my customers decided to migrate their apps-tier to File Storage Service. Let’s see how we did this:

Pre-Downtime

In preparation for the move first of all we had to create a new file system:

Creating the new file system.

This file system then has to be attached to a mount target and needs the proper export options:

NFS Export Options

Then, the mount point can be mounted in a temporary directory:

sudo mkdir /mnt/fss
sudo vi /etc/fstab
10.22.9.97:/ENTW220811 /mnt/fss nfs rw,bg,hard,timeo=600,nfsvers=3 0 0

Then:

sudo mount /mnt/fss
sudo chown oracle:oinstall /mnt/fss
sudo yum install -y fss-parallel-tools/

Downtime Operations

After these preparatory steps, we stop the entire environment and copy the content from the previous /u01 block volume over to the new FSS mount point /mnt/fss. Performance can be greatly improved by using fss-parallel-tools and parcp for this. For volumes of considerable size, an incremental operation can also be handy; but for the 300 GB one usually used for the apps-tier doing the migration in a 1-2 hours downtime is usually not a problem:

. setenv_run.s
adstpall.sh -mode=allnodes
~/stop_apex122.sh

sudo parcp --restore /u01/install /mnt/fss/

If working in a multi-node environment, then on the second apps-tier the mount through NFS to the primary apps-tier then needs to be replaced with a mount to the FSS mount target.

After the copy is finished, the old /u01/install mountpoint is retired and FSS is mounted to /u01:

sudo umount /u01
sudo vi /etc/fstab
10.22.9.97:/ENTW220811 /u01 nfs rw,bg,hard,timeo=600,nfsvers=3 0 0
sudo mount /u01

Then, start the apps-tier again:

. setenv_run.s
adstrtal.sh -mode=allnodes
~/start_apex122.shh

Post Downtime

After the downtime the old block volume should be detached through the OCI console. Furthermore, it is necessary to re-discover the environment in the E-Business Suite Cloud Manager. For this, either unregister the existing environment or (as I did in order to pick up the latest operating system image and add compatibility with the latest OCI shapes for the Cloud Manager) setup a new Cloud Manager environment. Make sure the Cloud Manager network profile is aware of the FSS Mount Target (usually, you have to create a new network profile for this).

Then, issue a re-discovery request:

Rediscovery of the moved environment

Setup FSS Snapshots

FSS is a highly durable service. However, that does not help against e.g. user errors (issuing “rm -rf /u01”). To remedy this, it is helpful to setup FSS Snapshots with e.g. an hourly snapshot (kept for a day) plus a daily snapshot (kept for a week). Unfortunately, so far this cannot be setup from the OCI console. Furthermore, for FSS it is currently not possible to perform automatic backups to OCI object storage (this is on the roadmap, though). However, using the utility fss-scheduler, such a backup policy can be easily setup.

Costs

As described in my previous post, FSS is considerably more expensive “per gigabyte” (see the oci price list) compared to block storage: 30 cents compared to just around 4.5 cents per GB and month. However due to the “sparse” nature of FSS, for an environment with production plus 3 clones for dev/test our storage usage changed from 4x400GB=1200GB with block storage (equaling roughly 54 USD/month) to 200GB + (3*15) GB = 245 GB (which equals 74 USD/month). The relative increase in costs may still seem significant, compared with the total costs for hosting 4 E-Business Suite environments with terabytes of block storage for the database; these are, however, peanuts. The ratio might even improve if in the future, Oracle will eventually use FSS clones for creating the patch file system (see, and vote for my idea over there).

Summary

By performing the above steps, you can migrate an E-Business Suite apps-tier easily to File Storage Service in a downtime of about 1 hour. By doing so, this it is e.g. possible to create very fast clones that are furthermore “sparse”.