Homepage 5 Techblog 5 Solving issues with Weblogic-Password change of Cloud Manager 23.3.1

Solving issues with Weblogic-Password change of Cloud Manager 23.3.1

December 23, 2023

Johannes Michler PROMATIS Horus Oracle


Executive Vice President – Head of Platforms & Development

This week, Oracle released the version 23.3.1 of Cloud Manager. More details can be found in my previous blog post New E-Business Suite Cloud Manager Release 23.3.1 as well as in https://blogs.oracle.com/ebsandoraclecloud/post/introducing-ebs-cloud-manager-2331

The new release finally provides the possibility to change the apps, ebs_system and weblogic password during cloning. This is great, and for most of my customers it works "out-of-the box".

However with one customer I ran into the issue as follows:

[10.22.14.8] * ruby_block[20231223_02-50-20 : Start Admin Server on Run file system : execution] action run
[10.22.14.8] [2023-12-23T02:50:20+01:00] INFO: Processing ruby_block[20231223_02-50-20 : Start Admin Server on Run file system : execution] action run (oracle-ebs::alter-weblogic-password line 105)
[10.22.14.8] 
***** Start of STDOUT *****
E-Business Suite Environment Information
----------------------------------------
RUN File System : /u01/install/APPS/fs2/EBSapps/appl
PATCH File System : /u01/install/APPS/fs1/EBSapps/appl
Non-Editioned File System : /u01/install/APPS/fs_ne

DB Host: prodebsora.intern.dns Service/SID: UPG

Sourcing the RUN File System ...

You are running adadminsrvctl.sh version 120.10.12020000.11

Starting WLS Admin Server...
Refer /u01/install/APPS/fs2/inst/apps/UPG_prodebsapp01/logs/appl/admin/log/adadminsrvctl.txt for details

AdminServer logs are located at /u01/install/APPS/fs2/FMW_Home/user_projects/domains/EBS_domain/servers/AdminServer/logs

adadminsrvctl.sh: exiting with status 1

adadminsrvctl.sh: check the logfile /u01/install/APPS/fs2/inst/apps/UPG_prodebsapp01/logs/appl/admin/log/adadminsrvctl.txt for more information ...

***** End of STDOUT *****

Digging into the AdminServer.out logfiles I realized that cloud manager starts and stops the Admin Server multiple times. The first shutdown of the AdminServer didn't work cleanly/fast enough though. Then Cloud Manager did ignore this and tried to bring the AdminServer up again - which failed obviously due to the parallel shutdown progress.

Investigating the AdminServer logs during the shutdown revealed Stuck Threads around the EmbeddedLdap server. This brought me to "Admin Server Take a Long Time to Shutdown (Doc ID 2992284.1)" which fits a Cloning Situation quite good.

This lead me to modifying the Oracle provided script to change the passwords as follows:

[oracle@ebscm2 cloudmanager_custom]$ diff /u01/install/APPS/apps-unlimited-ebs/resources/chef-repo/cookbooks/oracle-ebs/recipes/change-datasource-passwords.rb /u01/install/APPS/apps-unlimited-ebs/resources/chef-repo/cookbooks/oracle-ebs/recipes/change-datasource-passwords.rb.org
44c44
< startAdminServerCmd = "#{sourceEnvCmd}; rm /u01/install/APPS/fs?/FMW_Home/user_projects/domains/EBS_domain/servers/AdminServer/data/ldap/conf/replicas.prop; sh $INST_TOP/admin/scripts/adadminsrvctl.sh start -nopromptmsg"
---
> startAdminServerCmd = "#{sourceEnvCmd}; sh $INST_TOP/admin/scripts/adadminsrvctl.sh start -nopromptmsg"

So I just delete the replicas.prop before the initial startup (on RUN and PATCH Filesystem). This will be fixed during startup anyways.

Then just do:

knife cookbook upload -a -V -o /u01/install/APPS/apps-unlimited-ebs/resources/chef-repo/cookbooks

After that, also on this environment cloning worked again with Cloud Manager 23.3.1.

I guess that this might be related with this being a Multi-Zone environment with many managed servers on multiple nodes.

I hope Oracle will incorporate this into the next Cloud Manager release; it shouldn't harm to recreate that file during Cloning.