Solving intermittent errors with adop fs_clone on OCI File Storage Service (FSS)

January 17, 2024

Johannes Michler PROMATIS Horus Oracle


Executive Vice President – Head of Platforms & Development

Recently for multiple customers I ran into issues while running "adop phase=fs_clone".

The error message itself is a bit strange, the error message is as follows:

/u01/install/APPS/fs_ne/EBSapps/log/adop/52/20240116_210955/fs_clone/prodebsapp01/TXK_SYNC_create/txkADOPPreparePhaseSynchronize.log:
-------------------------------------------------------------------------------------------------------------------------------------

Lines #(424-426):
TIME : Tue Jan 16 22:02:45 2024
FUNCTION: main::runFSCloneApply [ Level 1 ]
ERRORMSG: /u01/install/APPS/fs2/EBSapps/comn/adopclone_prodebsapp01/bin/adclone.pl did not go through successfully.

/u01/install/APPS/fs_ne/EBSapps/log/adop/52/20240116_210955/fs_clone/prodebsapp01/TXK_SYNC_create/fsclone_apply/FSCloneApplyAppsTier_01162152.log:
--------------------------------------------------------------------------------------------------------------------------------------------------

Lines #(1176-1180):

ERROR: RC-50410: Fatal: OUI Registration failed with an Abnormal Termination error during call from ouicli.pl. Please check the log file /u01/install/APPS/fs1/inst/apps/UPG_prodebsapp01/admin/log/ohclone.log for more details.
Raised by oracle.apps.ad.clone.ApplyAppsTechStack

Lines #(1179-1183):
Raised by oracle.apps.ad.clone.ApplyAppsTechStack

WARNING: RC-50208: Warning: Exception in method deepOUICheck

instantiate file:

Lines #(1226-1233):
AutoConfig is exiting with status 1

ERROR: RC-50013: Fatal: Instantiate driver did not complete successfully.
/u01/install/APPS/fs1/EBSapps/10.1.2/appsutil/driver/regclone.drv
ERROR: RC-50004: Fatal: Error occurred in ApplyAppsTechStack:
RC-50013: Fatal: Failed to instantiate driver /u01/install/APPS/fs1/EBSapps/10.1.2/appsutil/driver/regclone.drv

START: Inside exitClone....

Analyzing the error further I realized that this is around the (rather old) version of Oracle Installer trying to find the available space on the disk. With E-Business Suite running on OCI File Storage Service (FSS) this amount of available disk storage is basically "unlimited"; df reveals 8 Exabyte of free storage.

df showing tremendous amount of free storage

There are two solutions for this:

  • The support note "Applying the Latest AD and TXK Release Update Packs to Oracle E-Business Suite Release 12.2" (Doc ID 1617461.1) suggests to first apply a patch and then set the new parameter s_forms_oui_tmpdir to a directory with e.g. less than 200 GB of free storage space. By default that is set to /tmp which should be fine since it isn't on FSS.
  • As an alternative you can set the free space reported by FSS within the OCI console as described over there: https://docs.oracle.com/en-us/iaas/Content/File/Tasks/change-file-system-size-mt.htm

Modified "Reported size" and "Reported inodes" in OCI Console

We went for the second approach since it does not require to apply a patch and set more parameters. Setting the size to 600 GB worked both to

  • improve the reporting of space used in our checkmk based monitoring tool
  • solve the issue described above.

On one environment I still ran into issues with 600 GiB/Gil; there 490 worked.