Oracle WebLogic Server - Part 2 - Node Manager and its interaction with WebLogic Server instances

In this post, I am going to describe node manager process and its interaction with WebLogic server instances. This post covers mostly theoretical parts, it does not includes instructions on  how to perform tasks related to node manager.

 In WebLogic Server environment, the Admin Server does not connect to the managed servers directly. A separate utility named "node manager" should be running in each machine for helping this purpose.

The main task of node manager is to provide a remote controller interface for controlling of the WebLogic Server instances (Admin Server and managed server instances) which are located in the same machine as the node manager process. It is not mandatory to have a node manager in a machine if you do not want to control  managed server instances through Admin Server. Also it is possible to have only one node manager for all the domains in a machine.

It is recommended to install node manager as an operating system service or daemon. Then the node manager will be automatically restarted when a system failure or reboot of the machine happens.

Node manager is required if you want following features,

• If you want to start managed servers from Admin console.
• For supporting High Availability.
• For Whole Server Migration.
• For Automatic Service Migration.

Purpose of Node Manager Process:

• Allow a remote or a local client to stop/start WebLogic server instances in the machine which runs the Node Manager process (mainly, but not necessarily from the Admin Server)
• start, shutdown Admin Server instance
• start, suspend, shutdown managed server instances
• Automatically restart failed WebLogic server instances which were abnormally exited, if following conditions met,
• the failed server instance was started by the node manager.
• AutoRestart value of this server instance is set to true. (AutoRestart property of nodemanager's startup.properties file in <DOMAIN_HOME>/servers/<server_name>/data/nodemanager folder )
• Number of max allowed restart is not yet passed. (RestartMax property of nodemanager's startup.properties file).
• Allow monitoring of the health status of WebLogic servers from a remote or the local machine. (mainly, but not necessarily from the Admin Server)
• Retrieve server logs and node manager logs.

Usually, the node manager gets the start/shutdown commands for starting/shut downing managed servers from the Admin Server. Then node manager is the process that actually start or shutdown the relevant managed server.

Tip: you can access the node manager (tested for a Plain Node Manager in WL 10.3.6) through a program like netcat, and then execute plain text commands to control the servers and monitor servers.

e.g:

~# nc <MACHINE_IP> 5556
domain <your_domain_name>
+OK Current domain set to ‘<your_domain_name>’
getStates
-ERR Must login to a domain first
user <weblogicuser>
+OK NM usr set to '<weblogicuser>'
pass <weblogicuser_password>
+OK Password accepted
getStates
+OK managed1=RUNNING managed2=UNKNOWN AdminServer=UNKNOWN

There are two versions of node manager processes.

Versions of Node Manager

Java based node manager

Java-based node manager process runs in a JVM instance. A Java-based node manager provides more security than a script based node manager.

• Java-based node manager supports both Windows and UNIX operating systems.
• Database-less leasing (consensus leasing) is faster when using a Java-based node manager.
• Using inetd in UNIX, java-based node managers can be configured to restart automatically when receiving a request on a configured port.
• Determines its configuration from nodemanager.properties file.

See https://docs.oracle.com/cd/E14571_01/web.1111/e13740/java_nodemgr.htm#NODEM177 for more details.

Script based node manager

You can managed remote server instances in a network through ssh using a script-based node manager (a script named wlscontrol.sh), using the scripts provided in this node manager version.

• Script-based node manager only supports UNIX operating systems.
• Automatic server migration is only supported by script-based node manager.
• The node manager can control servers in remote machines through a set of scripts copied to those remote machine.
• It requires only a smaller foot print than java-based node manager.
• It supports SSH for security and setting up security is easier than setting up SSL for java based node manager.
• It supports SSH or RSH for connection.

Following command line option should be added to Admin Server startup parameters in order to use SSH type of script based node manager.

-Dweblogic.nodemanager.ShellCommand='ssh -o PasswordAuthentication=no %H wlscontrol.sh -d %D -r %R -s %S %C'

where %H = host name of ssh server, %S = WebLogic server name, %D = WebLogic domain name, %R = domain directory, %C = node manager script command. For RSH type, the ShellCommand to use is relevant RSH command.

See http://docs.oracle.com/cd/E12840_01/wls/docs103/nodemgr/script_nodemgr.html  and "Running Script-based Node Manager" in https://docs.oracle.com/cd/E24329_01/web.1211/e21050/starting_nodemgr.htm#NODEM214 for more details about Script based node manager.

Node Manager Types

There are several node manager types which supports different level/type of security for node manager communication.

Type	Description	Java Based Node Manager	Script Based Node Manager
Plain	Communication is not encrypted.	✓	✘
SSL	Use SSL for encrypting communication. You need to setup a certificate for this. See http://www.veriton.co.uk/roller/fmw/entry/ssl_for_weblogic_server_11g and http://weblogic-wonders.com/weblogic/2015/04/23/configure-ssl-nodemanager-12c/	✓	✘
SSH	Use SSH session to do the communication. An OS user should be created for connecting using SSH in each machine and setup SSH settings for connection. See https://docs.oracle.com/cd/E23943_01/web.1111/e13740/script_nodemgr.htm#NODEM208	✘	✓
RSH	This type of node manager uses RSH for communication. But RSH is an older protocol which does not use encryption when sending data (such as passwords). Therefore RSH is not secure. It seems it is now rare to use this type of node manager, as not much information about it available in the internet.	✘	✓

Node Manager Operations

Please see https://docs.oracle.com/cd/E28280_01/web.1111/e13740/overview.htm#NODEM137 for more detailed explanation.

Starting Admin Server using Node Manager

1. A node manager client authenticate to the Node Manager using node manager user name and password (e.g. nmConnect from WLST). Node Manager authenticates the given node manager user credentials using nm_password.properties file.
2. Then Node Manager receives a command to start Admin Server from the node manager client (e.g. nmStart from WLST). This command provides the server name, domain name, server user credentials and any additional start-up properties.
3. Node Manager determines the domain directory for the Admin Server from nodemanager.domains file and authenticates the given user credentials using a local file which contains the encrypted user credentials.
4. Node Manager creates relevant directories (server_name/logs, server_name/security, server_name/data/nodemanager etc.) and obtains start-up properties for the Admin Server.
• Then it saves encrypted user name, password etc in boot.properties file.
• It saves the startup properties obtained from Admin Server configuration in startup.properties.
5. Node Manager creates Admin Server process and start monitoring it.
6. Admin Server process gets domain configuration data from its config folder.

Starting a managed server using Node Manager

This process can be done either from Admin Console, WLST console or another node manager client. Following steps mentions how to start the server from Admin console.

1. A user issues a start command from Admin Console to start a managed server.
2. Then Admin Server connects to the node manager which is located on the same machine as the targeted managed server, authenticate to the node manager.
3. Then Admin Server sends the start command for starting the managed server to the node manager (with server name, username/password for connecting to server and other start-up properties).
4. Node manager starts the managed server process.
5. Managed server contacts the Admin Server to check for updates to its configuration information.
6. If there is any update to the domain configuration, managed server updates its local cache of configuration from the update it got from the Admin Server.

Restart Admin Server using Node Manager

1. Node Manager determines from the exit code of Admin Server process that it needs to restart the Admin Server.
2. Node Manager obtains the user name and password value needed for starting the Admin server from the boot.properties file, and the start-up properties from <servername>/data/nodemanager/startup.properties file.
3. Node Manager uses the above retrieved information and starts the Admin Server process.
4. Admin Server reads its configuration data and starts up.

Restart a managed server using Node Manager

1. Node Manager determines from the last known status of the managed server process (+ other node manager settings) that it needs to restart that managed server.
2. Node Manager retrieves the user name and password value needed for starting the managed server from the boot.properties file, and the startup properties from <servername>/data/nodemanager/startup.properties file.
3. Node Manager uses the above retrieved information and starts the managed Server process.
4. Managed server contacts the Admin server to check for updates of the configuration data. If it get any updates, it update its local cache of config directory with the received updates.
5. If managed server fails to contact the Admin Server and if the Managed Server Independence Mode (MSI) is enabled, the managed server uses its locally cached configuration data.

Shutdown a server instance using Node Manager

1. An authorized user connects to the Admin console and issues the shutdown command for a managed server.
2. Then Admin Server issues shutdown command directly to the managed server. Then managed server perform the shutdown sequences for Graceful Shutdown procedure. 
3. If Admin server failed to contact the managed server in above step, then it connects to the node manager and issues the shutdown command to the node manager for shutting down the managed server.
4. Node manager asks Operating System to kill the managed server process.
5. Operating system kills the managed server process.

Important files and folders related to node manager

File	Folder	Purpose
nodemanager.properties	<NODEMANAGER_HOME>/	The configuration file used by Java based node manager
nodemanager.domains	<NODEMANAGER_HOME>/	Contains domain names and corresponding directories of the domains managed by the node manager
nm_data.properties	<NODEMANAGER_HOME>/	This file stores a symmetric encryption key. It is stored in encrypted form.
nm_password.properties	<DOMAIN_HOME>/config/nodemanager	This file stores the username and password of the node manager
boot.properties	<DOMAIN_HOME>/servers/<server_name>/data/nodemanager	This files contains the user credentials needed for starting server instances.
startup.properties	<DOMAIN_HOME>/servers/<server_name>/data/nodemanager	Each managed server instance contains its own copy of startup.properties file which contains the properties that control the startup/shutdown/restart of the server instance by the node manager.
<server_name>.addr	<DOMAIN_HOME>/servers/<server_name>/data/nodemanager	This file contains the IP address added when the server started or migrated.
<server_name>.pid	<DOMAIN_HOME>/servers/<server_name>/data/nodemanager	This file contains the process id of the server instance. Node manager checks this process id during crash recovery. This is generated by the server instance.
<server_name>.state	<DOMAIN_HOME>/servers/<server_name>/data/nodemanager	This file contains the current state of the server instance. This file is generated by the server instance and monitored by the node manager. Without this file, node manager cannot determine the current state of the server instance.
<server_name>.lck	<DOMAIN_HOME>/servers/<server_name>/data/nodemanager	This file is generated by the server instance and contains a internal lock id.
nodemanager.log	<NODEMANAGER_HOME>/	Node manager log file (only for Java based node manager). This file contains logs about all of domains controlled by the node manager. This log can be viewed using nmLog command of WLST and "Machines --> Monitoring --> Node Manager Log" in Admin Console.
<server_name>.out	<DOMAIN_HOME>/servers/>server_name</logs	Node Manager Server instance log file. This file contains the messages written to stderr and stdout by the server instance. You can view this log by either nmServerLog WLST command or "Diagnostics --> Log Files".
<server_name>.log	<DOMAIN_HOME>/servers/<server_name>/logs	WebLogic server log files.

References:

• https://docs.oracle.com/cd/E28280_01/web.1111/e13740/overview.htm#NODEM127
• http://docs.oracle.com/cd/E13222_01/wls/docs90/server_start/nodemgr.html
• Java-based Node Manager - https://docs.oracle.com/cd/E14571_01/web.1111/e13740/java_nodemgr.htm#NODEM177
• Script-based Node Manager - http://docs.oracle.com/cd/E12840_01/wls/docs103/nodemgr/script_nodemgr.html
• Running Script-based Node Manager - https://docs.oracle.com/cd/E24329_01/web.1211/e21050/starting_nodemgr.htm#NODEM214
• Running Script-based Node Manager - http://docs.oracle.com/cd/E13222_01/wls/docs90/server_start/nodemgr.html#1081981
• WLST commands for Node Manager - https://docs.oracle.com/middleware/1212/wls/WLSTC/reference.htm#WLSTC470

Class Loading in Java - Concept, Class Loader Types, and Related Exceptions

The other day one of my colleagues asked me whether I could explain to him about Java class loading. Then I thought "I know basics about Java Class Loading, but is it complete and accurate?". So I decided to do some search about the topic and it ended with writing this post.

When you launch a Java application using "java" command, one of major operation that happens behind it is the loading of required classes. There are 3 types of classes normally needed for running a Java application.

1. Classes that defines the implementation of Java platform. These classes are necessary for running basic Java Runtime Environment (JRE).
2. Classes that implements the functionalities for Java Extension mechanism.
3. Classes that implements the functionality of your application and the 3rd party libraries required for running your application code.

Process of finding the byte code for a given class name and then converting that byte code to a JVM internal representation of a class, is known as class loading. JVM has 3 separate class loaders for loading each of above mentioned sets of classes. In addition to those, developers can create their own class loaders.

1. Bootstrap class loader
2. Extension class loader
3. System classpath class loader

More details about these class loaders are explained in below sections. Few important things to remember is, 

• Each of those class loaders has different set of classpaths (directories and jar files to find the classes) to check when it tries to find/load a class. (e.g.: Bootstrap class loader loads class located in "sun.boot.class.path" system property which normally point to a set of jar files in jre/lib folder.)
• JVM does not load all classes at the start-up, it only loads the required set of classes, and then load the classes dynamically as needed. 
• The same class loaded by two separate class loaders are not equal. Therefore you cannot cast one of those class to other class, although those were loaded from same class byte code. A loaded class is internally identified by [the package name, class name, the name of the class loader which it loaded]. (Two different class loaders mentioned here does not have parent-child relationship. if that is the case, the class is loaded by same parent.)

The classes loaded by Class Loaders are stored in method area in PermGen area of JVM. Method Area stores the per-class information such as 

• Class Loader Reference
• Runtime Constant Pool
• Field Data
• Method Data
• Method Code (Byte code + other information)

All threads share the same Method Area.

Java has following class loader types in addition to user built class loaders.

Class Loader Types

Bootstrap Class Loader (aka Primordial or Null Class Loader)

    Bootstrap Class Loader is the parent class loader of all other class loaders. It is responsible for loading the key classes (from java.* packages etc.) at the beginning of the class loading. It loads the pre-requisite classes needed for supporting a basic Java Runtime Environment. It has access to a set of trusted classes which can be run by JVM without "verification". Bootstrap Class Loader is implemented using native code (mostly in C), not using Java code. But all other Class Loaders are implemented using Java code.
    So when you execute a java program with -verbose:class JVM argument, You can see that first few lines in the output that starts with "[Loaded" are related to the class loading done by Bootstrap Class Loader from jre\lib\rt.jar file and other important jar files the jre\lib directory in your Java installation. This location is known as bootstrap classpath and you can read it from sun.boot.class.path system property.
    You can use -Xbootclasspath non-standard command line option to change the directories of bootstrap class path, but it is not recommended and it violates JRE binary code license. There are two more variation of this command line option; -Xbootclasspath/a  to append to existing bootclasspath, and -Xbootclasspath/p to prepend to existing bootclasspath.

Extension Class Loader

Extension Class Loader is immediate child of the Bootstrap Class Loader. Extension Class Loader is responsible for loading classes from Java extensions directories (value of java.ext.dirs system property including "jre\lib\ext"). These locations has the jar files which implements the Java extension APIs.
Extension Class Loader in JVM is implemented by sun.misc.Launcher$ExtClassLoader. When Extension Class Loader get a call to load a class, it first tries to delegate the loading to the Bootstrap class loader (because it is the parent class loader). If Bootstrap class loader cannot load the class, then Extension Class Loader tries to load the class from "jre\lib\ext" directory or other directories listed in "java.ext.dirs" system property.

System Class Loader (System Classpath Class Loader)

System Class Loader is the immediate child of the Extension Class Loader. This class loader loads the classes from classes and jar files in CLASSPATH environment variable or given in -classpath command line option.
Some important points about Classpath of the System Class Loader,

• Default classpath is "." aka current directory.
• You can use CLASSPATH environment variable to define the classpath.
• You can use -cp or -classpath command line option to define the classpath, this will override the value defined in CLASSPATH environment variable.
• When you use -jar command line option, it will ignore the values in -cp and -classpath command line options, and also the value of CLASSPATH environment variable. It will only consider the Class-Path attribute defined in the META-INF/mainfest.mf file of the jar file.
• If two classes with the same name exists in the classpaths, the class loader will load the first found class.
• Current value of the classpath being used by a java program can be retrieved from java.class.path system property.

System Class Loader is implemented by sun.misc.Launcher$AppClassLoader class and it is a child of Extension Class Loader. You can change the System Classpath Class Loader to use your own class loader implementation instead of sun.misc.Launcher$AppClassLoader, by using -Djava.system.class.loader command line option.

User Defined Class Loaders

Developers can create their own class loaders to support specific requirements of their application. Possible examples for implementing user defined class loaders,

• to load classes through network.
• to load classes which are auto generated at runtime.
• to load classes from encrypted files.
• to separate class loading of different applications which runs in same JVM.
• etc.

Class Loading Algorithm

Class loaders (at least JVM class loaders) follow following algorithm when it wants to load a class. (This algorithm is based on delegating the task to parent first.)

1. Check whether class is already available in the local cache of the current class loader.
2. If the class was not found in the cache, ask (delegate) the parent class loader to load the class. (Please remember that Bootstrap Class Loader does not have a parent.)
3. Parent class loader also does above steps recursively until the class found in its cache or no parent class loader found. (Ask parent's parent class loader to load, if parent class loader did not find the class in its cache.).
4. If the parent class loader can find and load the class, it store it in its cache and return it to its child class loader.
5. If parent class loader cannot load the class, the class loader tries to load the class from its classpaths (aka code sources).
6. If the class loader cannot find the class in its local cache and cannot get the class from parent class loaders and cannot load the class from its local code sources, it throws ClassNotFoundException.

Example: When system classpath class loader wants to load javax.imageio.ImageIO class (a class belong to Java Extensions),

1. System Classpath Class Loader check whether it has javax.imageio.ImageIO class loaded in its local cache.
2. It does not find it in local cache, so it ask its parent class loader (which is Extensions Class Loader) to load the javax.imageio.ImageIO class.
3. Extension Class Loader checks its local cache whether it has javax.imageio.ImageIO class.
4. If it found the javax.imageio.ImageIO class in its local cache, it returns it to the System Classpath Class Loader.
5. If it did not find the javax.imageio.ImageIO class in its local cache, it ask its parent class loader (which is Bootstrap Class Loader) to load the javax.imageio.ImageIO class.
6. Bootstrap Class Loader checks whether it has javax.imageio.ImageIO in its local cache. If it has the class in local cache, it returns it to Extension Class Loader method call. (But in our example, Bootstrap class loader should not find the class in its local cache, because the class is in extension classpath).
7. If Bootstrap Class Loader did not find it in its local cache, and because it does not have any parent class loader, it tries to load the javax.imageio.ImageIO class from its classpath aka its local code sources (bootstrap classpath).
8. Because the class is belong to Java Extensions, Bootstrap Class Loader does not find it in bootstrap classpath. Then it return the method call to its child class loader (Extension Class Loader) without any class loaded.
9. Then Extension Class Loader tries to find the javax.imageio.ImageIO class in its classpath aka its local code sources.
10. Because the class belongs to Java Extensions, Extension Class Loader finds the class in its classpath and loads it. Then put the class in its local cache. Then it returns the class to its child class loader which is System Classpath Class Loader.

    So when a class loader wants to load a class, it first check whether class is already loaded in its local cache, then delegates the loading to the parent class loader if it is not in the current class loader's cache. The class loader will try to load the class itself, only if the parent class loader cannot load the class.
    Therefore the classes loaded by parent class loaders are visible to the children class loaders of that parent class loader. But the classes loaded by a child class loader are not visible to any of parent class loader of the child. Due to this algorithm, a class loaded by parent class loader will not be reloaded by a child class loader of it.

Stages in Class Loading

There are 3 main stages in loading a class from byte code to proper JVM internal representation.

1. Loading - Class Loader search for the class file (byte code) in the classpath and the byte code is loaded to memory (as a ClassFile structure). This step creates a basic structure for the class, but it does not have all the important data filled yet.
2. Linking - This step tries to convert byte code data to proper runtime information. This step is sub divided into 3 more sub steps.
1. Byte Code Verification - the byte code is checked for the correct format and other checks. This step ensures that loaded class is structurally correct.
2. Class Preparation - This step prepare the data structures to represent methods, fields, and implemented interfaces etc. from the byte code. This step create static fields and initialize those to their default values.
3. Resolving - This step do the resolving of other classes referenced by this class. (e.g.: Super classes, field types, method param types, method local variable types)
3. Initializing - This step executes the static initializer blocks.

Class Data Sharing (CDS) feature

 Class Data Sharing (CDS) is a feature introduced in JavaSE 5.0 (in Oracle/Sun hotspot JVM) in order to reduce the java application start-up time by improving class loading time. When the JRE is installed in your machine, JRE loads the classes from system jar files into internal representations and then dump those loaded internal representations to a file known as "shared archive" (which is located in jre/lib/[arch]/client/classes.jsa in UNIX and in jre/bin/client/classes.jsa in Windows). During subsequent JVM invocations, the "shared archive" is used by those JVM processes by memory mapping to it, so reducing the loading time of classes loaded by Bootstrap class loader (which also means this memory mapped file is shared among multiple JVM processes).

If this feature is enabled, you will see many lines with "[Loaded className from shared objects file]" when you run a java application with "-verbose:class" command line argument. The list of classes in the "shared archive" can be found in "jre/lib/classlist" file.

You can use "-XShare" command line option to either enable or disable class data sharing for your java application. 

In JDK 8U40, Oracle has introduced an experimental feature named Application Class Data Sharing (AppCDS) by extending CDS, to allow developers to place classes from Standard Extensions directories and from Application classpath in the "shared archive".

See http://docs.oracle.com/javase/7/docs/technotes/guides/vm/class-data-sharing.html for more details about Class Data Sharing.

Common Errors/Exceptions related to class loading

ClassNotFoundException

java.lang.ClassNotFoundException is thrown when your program dynamically tries to load the class at runtime and the class loader did not find the requested class. Main reason is, the class is not available in the CLASSPATH.
This is normally thrown by methods like Class.forName(), ClassLoader.loadClass(), ClassLoader.findSystemClass() when the java program tries to load a class dynamically, using its string name. Due to this reason, the compiler does not detect the unavailability of the class at the compile time.

NoClassDefFoundError

java.lang.NoClassDefFoundError happens when JVM or a class loader instance trying to load a class definition, but this class definition cannot be found or loaded due to some reason.
Most of the time, the class was available in the classpath during the compile time, but it is no longer available during the runtime. Another possible reason is, the class is still available in the classpath, but the class has static initialization blocks which fails during the loading of the class.

References:

• http://docs.oracle.com/javase/7/docs/technotes/guides/security/spec/security-spec.doc5.html
• http://docs.oracle.com/javase/7/docs/technotes/tools/findingclasses.html
• http://docs.oracle.com/javase/tutorial/ext/basics/install.html  
• http://javapapers.com/core-java/java-jvm-run-time-data-areas/
• http://www.artima.com/insidejvm/ed2/jvm2.html
• http://what-when-how.com/enterprise-javabeans-3/exploring-class-loading-ejb-3/
• http://blog.jamesdbloom.com/JVMInternals.html#method_area
• https://docs.oracle.com/javase/tutorial/essential/environment/paths.html 
• http://docs.oracle.com/javase/1.5.0/docs/tooldocs/windows/classpath.html 
• http://docs.oracle.com/javase/7/docs/technotes/guides/extensions/index.html
• http://docs.oracle.com/javase/tutorial/ext/
• https://blogs.oracle.com/sundararajan/entry/understanding_java_class_loading
• https://docs.oracle.com/javase/specs/jvms/se7/html/jvms-5.html
• http://www.techjava.de/topics/2008/01/java-class-loading/
• http://docs.oracle.com/cd/E11035_01/wls100/programming/classloading.html

Oracle WebLogic Server - Part 1 - Main components and their relationship

In this post series, I am planning to write about the important things I learned about the WebLogic server during past few years (specially about version 10.3.6). Main purpose of these posts is to keep it as a future reference.

The WebLogic Server is an "application server" developed by the Oracle Corporation, and it supports many powerful features such as high availability, scalability, high performance, reliability, etc.

 The WebLogic Server has following main components.

Domain

A WebLogic domain is a logically related group of WebLogic server resources. A WebLogic domain has a special server instance known as “Admin Server”, and zero or more server instances known as “managed servers”.

All WebLogic server instances in a WebLogic domain must have same WebLogic major and minor version. The server instances in a WebLogic domain can be distributed through out several machines.

Admin Server

This is the component which helps the configuration, management, and monitoring of the servers and resources in the whole WebLogic Server domain. You can access the Admin Server through,

• The "WebLogic Admin Console" web application Or
• An UNIX shell like command line interface known as "WLST" Or
• Through JMX API

 All the configurations settings related to the Admin Server, managed servers, clusters, and machines are stored and managed in the Admin Server, mainly in a file named config.xml.

Using Admin Server, You can

• Start/stop managed servers,
• Deploy/update/delete applications,
• Update settings of managed servers/admin server/cluster/etc.,
• Create/edit/delete more managed servers/clusters/machines etc.
• Monitor health of the servers.

Managed Servers

In a WebLogic domain, you can have zero or more servers known as "managed servers".  These are separate JVM processes.

A "managed server" is the server process where usually you deploy your application. Although it is possible to deploy an application in Admin Server, it is not the intended usage of the Admin Server.

A “managed server” can be a part of WebLogic cluster, but it is not mandatory.

Clusters

A cluster is a set of managed servers grouped to achieve Scalability (Horizontal scalability) and High Availability. A cluster appears to the client as a single WebLogic server instance (when client access the cluster through a load balancer or use the cluster address to connect to the EJBs in the cluster). All managed server instances in a cluster must belong to the same WebLogic domain. You can create a WebLogic cluster distributed in one or more server machines.

WebLogic cluster has following capabilities in order to achieve the Scalability and High Availability.

• Application Fail Over Protection
• Load Balancing
• Supporting automatic and manual migration of resources between servers in the cluster.

Machines

A “machine” in WebLogic is a logical representation of a physical machine or a virtual machine, which contains one or more WebLogic server instances (Admin Server or managed servers).

In each machine, there is a special process known as “Node Manager” in order for Admin Console to start/stop “managed server” instances in the respective machine, without actually login to that machine. “Machine” configuration includes the listen address and the port of the Node Manager of the respective “machine”.

Machine Types:

• Unix
• Virtual
• Other

Node Manager

In WebLogic Server environment, the Admin Server does not connect to the managed servers directly. In each machine, a separate process named "node manager" should be running for helping this purpose.

A "node manager" process is necessary for managing server instances through Admin Server or similar remote client and to support restarting of failed server instances.

But it is not mandatory to have a node manager process if you do not want above features.

Purpose of Node Manager Process:

• Allow to stop/start/restart WebLogic server instances in the machine which has the Node Manager process from a remote/local machine (mainly, but not necessarily from the Admin Server)
• If a server instance started using a node manager fails and AutoRestart attribute of the server set to true, the node manager automatically restart that server instance.
• Allow monitoring of the health status of WebLogic servers from a remote machine. (mainly, but not necessarily from the Admin Server)
• Retrieve server logs and node manager logs.

(In the above diagram, Admin Server can be at only one location depicted above in a given domain, not in the both places at once).

Please see my next blog post "Oracle WebLogic Server - Part 2 - Node Manager and its interaction with WebLogic Server instances" for more information about the node manager and its interaction with other components.

References:

• http://www.oracle.com/us/products/middleware/application-server/oracle-weblogic-server-ds-1391360.pdf
• https://docs.oracle.com/cd/E13222_01/wls/docs61/intro/chap1.html
• https://docs.oracle.com/cd/E13222_01/wls/docs92/server_start/nodemgr.html
• http://tekslate.com/weblogic-node-manager-configuration/
• https://oraclestacksupport.wordpress.com/2012/04/27/node-manager/