Oracle SOA Suite 12.2.1.3 HA Setup- Per Host Nodemanager Startup Issue

This post is focused on my recent engagement with production setup of Oracle SOA Suite 12.2.1.3 and issue that encountered during nodemanager startup. Below are my environment details-
  1. Oracle SOA Suite- 12.2.1.3
  2. Oracle Database- 12.2.1.0
  3. Oracle Linux- 7.5
  4. Nodemanager type- Manual (Per Host Node Manager Setup)

TOPOLOGY
-----------------------------------------------------------------
SOAHOST1
     -/u01/oracle/config/domains/soabpm_domain [ASERVER_HOME] [SAN Storage mounted on SOAHOST1 as /u01/oracle]
     -/u01/oracle/products/soa_oracle_home [ORACLE_HOME]
     -/u01/oracle/products/soa_oracle_home/wlserver [WLS_HOME]
     -/u02/oracle/config/domains/soabpm_domain [MSERVER_HOME]
     -/u02/oracle/config/nodemanager/ [NM_HOME]
          --startNodeManager.sh
          --nodemanager.domains
          --nodemanager.properties
SOAHOST2
     -/u01/oracle/config/domains/soabpm_domain [ASERVER_HOME] [SAN Storage mounted on SOAHOST2 as /u01/oracle]
     -/u01/oracle/products/soa_oracle_home [ORACLE_HOME]
     -/u01/oracle/products/soa_oracle_home/wlserver [WLS_HOME]
     -/u02/oracle/config/domains/soabpm_domain [MSERVER_HOME]
     -/u02/oracle/config/nodemanager/ [NM_HOME]
          --startNodeManager.sh
          --nodemanager.domains
          --nodemanager.properties


I have selected manual nodemanager setup during the configuration.

After the successful installation and configuration I have tried to start the node manager from NM_HOME location on SOAHOST1 which caused below error-

Oct 21, 2018 12:16:58 PM weblogic.nodemanager.server.NMServer main 
SEVERE: Fatal error in NodeManager server 
weblogic.nodemanager.common.ConfigException: Could not load log formatter: weblogic.nodemanager.server.LogFormatter 
at weblogic.nodemanager.server.NMServerConfig.loadFormatter(NMServerConfig.java:619) 
at weblogic.nodemanager.server.NMServerConfig.<init>(NMServerConfig.java:371) 
at weblogic.nodemanager.server.NMServer.<init>(NMServer.java:156) 
at weblogic.nodemanager.server.NMServer.getInstance(NMServer.java:134) 
at weblogic.nodemanager.server.NMServer.main(NMServer.java:589) 
at weblogic.NodeManager.main(NodeManager.java:31) 
Caused by: java.lang.ClassNotFoundException: weblogic.nodemanager.server.LogFormatter 
at com.oracle.classloader.PolicyClassLoader.findClass(PolicyClassLoader.java:398) 
at com.oracle.classloader.PolicyClassLoader.loadClass(PolicyClassLoader.java:371) 
at com.oracle.classloader.weblogic.LaunchClassLoader.loadClass(LaunchClassLoader.java:55) 
at java.lang.ClassLoader.loadClass(..... 
+ status=1 
+ set +x 
/u02/oracle/config/nodemanager 

Error Codes 
-----------------------------------------------------------------
weblogic.nodemanager.common.ConfigException: Could not load log formatter: weblogic.nodemanager.server.LogFormatter 

nodemanager.properties content-
DomainsFile=/u02/oracle/config/nodemanager/nodemanager.domains 
LogLimit=0
PropertiesVersion=12.2.1.3.0
AuthenticationEnabled=true 
NodeManagerHome=/u02/oracle/config/nodemanager
#Include the specific JDK home 
#JavaHome=/u01/oracle/products/jdk 
JavaHome=/usr/lib/java/jdk1.8.0_181
LogLevel=INFO 
DomainsFileEnabled=true 
StartScriptName=startWebLogic.sh 
#Leave blank for listening on ANY 
ListenAddress=
NativeVersionEnabled=true 
ListenPort=5556
LogToStderr=true
SecureListener=false
LogCount=1
StopScriptEnabled=false
QuitEnabled=false
LogAppend=true
StateCheckInterval=500
CrashRecoveryEnabled=true
StartScriptEnabled=true 
LogFile=/u02/oracle/config/nodemanager/nodemanager.log 
LogFormatter=weblogic.nodemanager.server.LogFormatter 
ListenBacklog=50

nodemanager.domains content
soabpm_domain=/u02/oracle/config/domains/soabpm_domain;/u02/oracle/config/domains/soabpm_domain

Observation and Workaround
  1. Initially, it looks like library missing or path issue associated with the requested library 'weblogic.nodemanager.server.LogFormatter'.
  2. Since the current nodemanager has a new location and might not be able to load the requested library.
  3. As a workaround, I have commented the library inclusion of   "LogFormatter" variable.
  4. #LogFormatter=weblogic.nodemanager.server.LogFormatter
  5. Started the nodemangaer with rest of unchanged settings now and it's starting normally on both HOST machines i.e. SOAHOST1 & SOAHOST2.
  6. Upon close observation of nodemanager startup log, it shows inclusion of LogFormatter=weblogic.nodemanager.server.LogFormatter as nodemanager properties.
  7. In addition, I have checked and corrected all environment specific vairaibles on SOAHOST1 and SOAHOST2 for respective user's bash profile-
    1. JAVA_HOME=/usr/lib/jdk/jdk1.8.0_181
    2. ASERVER_HOME=/u01/oracle/config/domains/soabpm_domain
    3. MSERVER_HOME=/u02/oracle/config/domains/soabpm_domain
    4. ORACLE_RUNTIME=/u01/oracle/runtime
    5. ORACLE_HOME=/u01/oracle/products/soa_oracle_home
    6. ORACLE_COMMON_HOME=/u01/oracle/products/soa_oracle_home/oracle_common
    7. WL_HOME=/u01/oracle/products/soa_oracle_home/wlserver
    8. NM_HOME=/u02/oracle/config/nodemanager
    9. APPLICATION_HOME=/u01/oracle/config/applications
    10. DEPLOY_PLAN_HOME=/u01/oracle/config/dp
References-
1. https://docs.oracle.com/en/middleware/lifecycle/12.2.1.3/soedg/enterprise-deployment-guide-oracle-soa-suite.pdf
2. https://community.oracle.com/thread/4179613.pdf
3. https://www.youtube.com/watch?v=YICIWjQBLUs













Comments

Popular posts from this blog

Oracle SOA Suite- Implementing Email Notification

Oracle SOA Suite 12c- PKIX path building failed & unable to find valid certification path to requested target

Migration of Oracle SOA Suite Composite from 11g to 12c