Friday, September 5, 2008

Websphere Application Server Community Edition (WAS CE) Clustering using WADI

by Rockfield.

Working on the latest version of BizAgi, we were looking for an app server that supported JDK 1.6, EJB 3.0 and, of course, clustering. One of our targets was WAS CE, and as the WAS CE Cluster through WADI is explained at the IBM website, you might think it's easy to build. However, as long as you start following the instructions you'll soon realize it's not as simple, clear and quick as shown in there. So this will be a "for dummies" walkthrough for most people who are not expert enough to overcome the possible problems that might occur when building the cluster.


Previous considerations
  • For this specific example WAS CE 2.1 was previously installed on 2 machines called server1 and server2. No special configuration was used, just the plain default installation.

  • Default ports are used, and they are the same for both machines.

  • In order to check the cluster behavior, a load balancing HTTP server is necessary. A plain Apache HTTP Server installation and configuration is provided at the end of this tutorial.

  • <WASCE_HOME> is the WAS CE installation directory.

WAS CE cluster configuration

NOTE: These steps must be executed on both servers!
  • Go to the file <WASCE_HOME>/var/config/config-substitutions.properties and assign a name to each node of the cluster replacing the value of the clusterNodeName property (by default this property's value is "NODE"). For this example we used "node1" on server1 and "node2" on server2.
    clusterNodeName=node1
  • On the same config-substitutions.properties file go to the RemoteDeployHostname property and replace the default value "localhost" with the actual server name or IP address.
    RemoteDeployHostname=server1
  • Go to the <WASCE_HOME>/var/config/config.xml file (NOTE: Make sure the server is NOT RUNNING while editing the config.xml, because when stopped all changes made while running will get lost) and look for the following tags in order to switch the load property to "true" if it does exist and is set to "false":
    <module name="org.apache.geronimo.configs/tomcat6-clustering-wadi/2.1.1/car" load="true"/>
    <module name="org.apache.geronimo.configs/wadi-clustering/2.1.1/car" load="true">
    <module name="org.apache.geronimo.configs/farming/2.1.1/car" load="true">
    NOTE: If the value is set to "true" while not running, when the server starts up, the load property disappears. It is only shown explicitly when its value is set to "false".
  • On the same config.xml file, look for the org.apache.geronimo.configs/farming/2.1.1/car key. You must add all the cluster nodes except for the current one which is already there by default. This is how the key would look like for our server1:
    <module name="org.apache.geronimo.configs/farming/2.1.1/car">
    <gbean name="NodeInfo">
    <attribute name="name">${clusterNodeName}</attribute>
    </gbean>
    <gbean name="ClusterInfo">
    <attribute name="name">${clusterName}</attribute>
    </gbean>
    <gbean name="org.apache.geronimo.configs/farming/2.1.1/car?ServiceModule=org.apache.geronimo.configs/farming/2.1.1/car,j2eeType=NodeInfo,name=NodeInfo2" gbeanInfo="org.apache.geronimo.farm.config.BasicNodeInfo">
    <attribute name="name">node2</attribute>
    <attribute propertyEditor="org.apache.geronimo.farm.config.BasicExtendedJMXConnectorInfoEditor" name="extendedJMXConnectorInfo">
    <ns:javabean class="org.apache.geronimo.farm.config.BasicExtendedJMXConnectorInfo" xmlns="" xmlns:ns4="http://geronimo.apache.org/xml/ns/attributes-1.2" xmlns:ns="http://geronimo.apache.org/xml/ns/deployment/javabean-1.0">
    <ns:property name="username">system</ns:property>
    <ns:property name="password">manager</ns:property>
    <ns:property name="protocol">rmi</ns:property>
    <ns:property name="host">server2</ns:property>
    <ns:property name="port">1099</ns:property>
    <ns:property name="urlPath">JMXConnector</ns:property>
    <ns:property name="local">false</ns:property>
    </ns:javabean>
    </attribute>
    </gbean>
    </module>
  • NOTE: On the other hand, for our server2's configuration the <attribute name="name"> tag's value would be "node1", and the <ns:property name="host"> tag's value would be "server1".

Starting the cluster

Once the node configuration is done, start the cluster nodes as you usually would if they were stand alone. In order to verify that they are clustered, on the console screen of the first running node will appear the following notification when the second node would had started:
Dec 12, 2012 11:11:53 AM org.codehaus.wadi.tribes.WadiMemberInterceptor memberAdded
INFO: memberAdded:tcp://{-84, 16, 2, 16}:4000

Application configuration

  • Add a <distributable/> tag in your application's web.xml file.

  • Check inside your deployment plan xml file, at the war definition, and add the <dep:tomcat-clustering-wadi> tag so it would look like this:
    <dep:environment>
    <dep:moduleId>
    <dep:groupId>com.javaxcross.tests</dep:groupId>
    <dep:artifactId>WarTest-war</dep:artifactId>
    <dep:version>1.0.0.0</dep:version>
    <dep:type>war</dep:type>
    </dep:moduleId>
    </dep:environment>
    <dep:tomcat-clustering-wadi></dep:tomcat-clustering-wadi>
    where, of course, you should replace the groupId, artifactId and version tags with you own app's data.

Application Deployment

In order to deploy an application on the cluster, the web console does not have the necessary options, so you must open a command console window and execute the following instruction:

deploy.bat --user <name> --password <word> <command> --targets <target> <module> <plan>

where...

  • you'll find deploy.bat (or deploy.sh for unix like o.s.) at <WASCE_HOME>/bin/deploy.bat

  • <name> is the username and <word> is the password you use to login the web console. Default values are: system/manager.

  • <command> is the required instruction to execute, in this case could be deploy or distribute. The difference relies on deploy starting the application after the distribution, while distribute doesn't.

  • Replace <target> with the destination repository. which must be MasterConfigurationStore, because of the cluster. If you need to know the exact data you should use on the <target> tag, run GShell (<WASCE_HOME>/bin/gsh.bat), provided by WAS CE installation, and run the following command
    deploy/list-targets
    to list all the posible destination repositories. Just in case you don't find MasterConfigurationStore, execute on GShell the following command
    deploy/start org.apache.geronimo.configs/farming//car
    copy the whole MasterConfigurationStore value and add it to the command replacing the
    <target> tag, the default value is:
    org.apache.geronimo.configs/farming/2.1.1/car?ServiceModule=org.apache.geronimo.configs/farming/2.1.1/car,j2eeType=ConfigurationStore,name=MasterConfigurationStore
    The fist command used on GShell will ask for username and password, use the username/password combination you usually use for connecting through the web console.

  • <module> is the application (ear, war or jar) you want to deploy on the cluster.

  • <plan> is the xml file with the deployment plan.

Verification of the application deployment on cluster.

Once the application has been deployed, you may check if the process ended successfully verifying the following items:
  • Check the virtual repository MasterConfigurationStore for the application in the machine where the deploy command was executed on, what means, look for the application in the <WASCE_HOME>/master-repository/ directory.
  • Check the actual repository ClusterStore for the application in all of the cluster nodes including the one where the deploy command was executed on, in other words, look for the application in the <WASCE_HOME>/cluster-repository/ directory

Starting/stopping the application

Using the web console you can start the application on each node separately. But if you want to start the application on all nodes of the cluster at the same time, you must open a command console window and execute:
deploy.bat --user <name> --password <word> <command> <module>
Where...

  • you'll find deploy.bat (or deploy.sh for unix like o.s.) at <WASCE_HOME>/bin/deploy.bat

  • <name> is the username and <word> is the password you use to login the web console. Default values are: system/manager.

  • <command> is the required instruction to execute, in this case could be start or stop.

  • <module> is the name wich the application was registered with when deployed on the master repository, for instance, com.javaxcross.tests.WASCEtest/1219953325295/car