When things go bad in Weblogic, finding the problem can be extremely tricky. This post outlines some troubleshooting techniques that can save you some serious amounts of time and frustration.

Introduction

In order to deploy an application on WebLogic (WLS from here on), you must first create a domain. You can do this by running:

val@thinkpad:/opt/Oracle/Middleware/wlserver_10.3/common/bin$ ./config.sh

After that you need to create a “Machine” (Environment -> Machines) and point it to a running instance of a Node Manager. Then you create a Server (Environment -> Servers) and point it to the Machine. Then you can specify Deployments (EARs, WARS, etc) and run them on a specific server.

Deployments can either be archive files, or an exploded version of an archive on the file system. This is especially useful if you are tweaking your deployment by making modifications to it from Eclipse.

For example, let’s say you wanted to make some changes and redeploy the EAR. The steps would be:

1. Stop your Server: Environments -> Servers -> Example_Server -> Control -> Shutdown
2. Make the changes in Eclipse
3. Export your ear or weblogic jar. The structure of a Weblogic application is explained in confusing detail here: http://docs.oracle.com/cd/E13222_01/wls/docs81/programming/environment.html
4. Go to http://localhost:7001/console/ -> Deployments -> Update. This step assumes that you have previously deployed the app.
5. Start the server

Starting Weblogic

val@thinkpad:/opt/Oracle/Middleware/wlserver_10.3/server/bin$ ./setWLSEnv.sh
val@thinkpad:/opt/Oracle/Middleware/wlserver_10.3/server/bin$ ./startNodeManager.sh
val@thinkpad:/opt/Oracle/Middleware/user_projects/domains/base_domain$ ./startWebLogic.sh

Changing Java interpreter

/opt/Oracle/Middleware/wlserver_10.3/common/bin/commEnv.sh:

@@ -129,7 +129,7 @@ if [ -z "${JAVA_HOME}" -o -z "${JAVA_VENDOR}" ]; then
   JAVA_HOME="/opt/Oracle/Middleware/jrockit_160_14_R27.6.5-32"
   # Set up JAVA VENDOR, possible values are
   #Oracle, HP, IBM, Sun ...
-  JAVA_VENDOR=Oracle
+  JAVA_VENDOR=Sun
   # PRODUCTION_MODE, default to the development mode
   PRODUCTION_MODE=""
 fi

Tweaking VM memory arguments

Go to /opt/Oracle/Middleware/user_projects/domains/base_domain/bin/setDomainEnv.sh and find the section for the chosen jdk and architecture. PermGen options are in the same file.

Logs

The logs you will be interested in are located under the server directory. For example:

/opt/Oracle/Middleware/user_projects/domains/base_domain/servers/Example_Server/logs

There will be .log files, which are less verbose and .out files, which is what you’ll want most of the time.

By default WLS is configured to provide as little information as possible in those logs, which is why it is unlikely that you’ll be able to fix your problem with default configuration. However, it is possible to make those logs much more verbose.

Debug Logging

-Dweblogic.StdoutDebugEnabled=true 
-Dweblogic.log.RedirectStdoutToServerLogEnabled=true

Classloader Logging

If you are dealing with NoClassDefFoundError, or other classloading issues, turn on classloader logging:

-Dweblogic.utils.classloaders.ClasspathClassFinder=true 
-Dweblogic.utils.classloaders.GenericClassLoader.Verbose=true 
-Dweblogic.utils.classloaders.ChangeAwareClassLoader.Verbose=true

General Debug Flags

-Dlog4j.debug=true
-Dweblogic.debug.DebugConfigurationEdit=true
-Dweblogic.debug.DebugDeploymentTaskRuntime=true
-Dweblogic.debug.DebugDeploymentManagerAdmin=true
-Dweblogic.debug.DebugDeploymentManagerTarget=true
-Dweblogic.debug.DebugDeploymentOperationsAdmin=true
-Dweblogic.debug.DebugDeploymentManagerTargetOperations=true
-Dweblogic.debug.DebugDeploymentServiceApiTargetCalls=true
-Dweblogic.debug.DebugDeploymentServiceApiAdminCalls=true
-Dweblogic.debug.DebugDeploymentServiceApiAdminCallback=true
-Dweblogic.debug.DebugDeploymentServiceApiTargetCallback=true
-Dweblogic.debug.DebugDeploymentServiceStatusUpdatesAdmin=true
-Dweblogic.debug.DebugDeploymentServiceTransport=true
-Dweblogic.debug.DebugDeploymentServiceStatusUpdatesTarget=true

Even more debug flags

http://middlewaremagic.com/weblogic/?page_id=1096

Configuring Logging

These settings need to be passed in as VM arguments to the server JVM. This is done here:

Environment -> Servers -> Example_Server -> Server Start -> Arguments

For example, my Arguments field looks like this right now:

-Dweblogic.StdoutDebugEnabled=true -Dweblogic.log.RedirectStdoutToServerLogEnabled=true -Xms512m -Xmx1024m -XX:MaxPermSize=512m

Debugging with Eclipse

If increased logging did not help you, it’s time to get serious :). We’ll connect to the running WLS JVM with the Eclipse Debugger and debug the internals of WLS in order to find the problem.

• Make sure that JadClipse plugin is installed and working properly.

• Add the following parameters to Server -> Server Start -> Arguments:

-Xdebug -Xrunjdwp:transport=dt_socket,address=1044,server=y,suspend=y

suspend=y means that when you start the server JVM, it will immediately get suspended and wait for a debugger to attach.

You might also want to add other VM arguments during this process. For example, my Arguments field looks like this:

-Xdebug -Xrunjdwp:transport=dt_socket,address=1044,server=y,suspend=y -Dweblogic.utils.classloaders.ClasspathClassFinder=true -Dweblogic.utils.classloaders.GenericClassLoader.Verbose=true -Dweblogic.utils.classloaders.ChangeAwareClassLoader.Verbose=true -Dweblogic.StdoutDebugEnabled=true -Dweblogic.log.RedirectStdoutToServerLogEnabled=true -Xms512m -Xmx1024m -XX:MaxPermSize=512m

• Go to Eclipse, open your project’s Build Path and include /opt/Oracle/Middleware/wlserver_10.3/server/lib/weblogic.jar.

• Create a Remote Java Application debug launcher. Address is localhost:1044

• Start the server from the WLS Admin console. The status of the server will change to STARTING and nothing else will happen, b/c the server process is now suspended.

• Run the Remote Java Application debug launcher you created earlier. The WLS server will continue booting and you should now be able to place breakpoints into WebLogic code!

That’s it – happy debugging!