¯\_(ツ)_/¯: Class Loading in Java - Concept, Class Loader Types, and Related Exceptions

The other day one of my colleagues asked me whether I could explain to him about Java class loading. Then I thought "I know basics about Java Class Loading, but is it complete and accurate?". So I decided to do some search about the topic and it ended with writing this post.

When you launch a Java application using "java" command, one of major operation that happens behind it is the loading of required classes. There are 3 types of classes normally needed for running a Java application.

Classes that defines the implementation of Java platform. These classes are necessary for running basic Java Runtime Environment (JRE).
Classes that implements the functionalities for Java Extension mechanism.
Classes that implements the functionality of your application and the 3rd party libraries required for running your application code.

Process of finding the byte code for a given class name and then converting that byte code to a JVM internal representation of a class, is known as class loading. JVM has 3 separate class loaders for loading each of above mentioned sets of classes. In addition to those, developers can create their own class loaders.

Bootstrap class loader
Extension class loader
System classpath class loader

More details about these class loaders are explained in below sections. Few important things to remember is,

Each of those class loaders has different set of classpaths (directories and jar files to find the classes) to check when it tries to find/load a class. (e.g.: Bootstrap class loader loads class located in "sun.boot.class.path" system property which normally point to a set of jar files in jre/lib folder.)
JVM does not load all classes at the start-up, it only loads the required set of classes, and then load the classes dynamically as needed.
The same class loaded by two separate class loaders are not equal. Therefore you cannot cast one of those class to other class, although those were loaded from same class byte code. A loaded class is internally identified by [the package name, class name, the name of the class loader which it loaded]. (Two different class loaders mentioned here does not have parent-child relationship. if that is the case, the class is loaded by same parent.)

The classes loaded by Class Loaders are stored in method area in PermGen area of JVM. Method Area stores the per-class information such as

Class Loader Reference
Runtime Constant Pool
Field Data
Method Data
Method Code (Byte code + other information)

All threads share the same Method Area.

Java has following class loader types in addition to user built class loaders.

Class Loader Types

Bootstrap Class Loader (aka Primordial or Null Class Loader)

    Bootstrap Class Loader is the parent class loader of all other class loaders. It is responsible for loading the key classes (from java.* packages etc.) at the beginning of the class loading. It loads the pre-requisite classes needed for supporting a basic Java Runtime Environment. It has access to a set of trusted classes which can be run by JVM without "verification". Bootstrap Class Loader is implemented using native code (mostly in C), not using Java code. But all other Class Loaders are implemented using Java code.
    So when you execute a java program with -verbose:class JVM argument, You can see that first few lines in the output that starts with "[Loaded" are related to the class loading done by Bootstrap Class Loader from jre\lib\rt.jar file and other important jar files the jre\lib directory in your Java installation. This location is known as bootstrap classpath and you can read it from sun.boot.class.path system property.
    You can use -Xbootclasspath non-standard command line option to change the directories of bootstrap class path, but it is not recommended and it violates JRE binary code license. There are two more variation of this command line option; -Xbootclasspath/a to append to existing bootclasspath, and -Xbootclasspath/p to prepend to existing bootclasspath.

Extension Class Loader

Extension Class Loader is immediate child of the Bootstrap Class Loader. Extension Class Loader is responsible for loading classes from Java extensions directories (value of java.ext.dirs system property including "jre\lib\ext"). These locations has the jar files which implements the Java extension APIs.
Extension Class Loader in JVM is implemented by sun.misc.Launcher$ExtClassLoader. When Extension Class Loader get a call to load a class, it first tries to delegate the loading to the Bootstrap class loader (because it is the parent class loader). If Bootstrap class loader cannot load the class, then Extension Class Loader tries to load the class from "jre\lib\ext" directory or other directories listed in "java.ext.dirs" system property.

System Class Loader (System Classpath Class Loader)

System Class Loader is the immediate child of the Extension Class Loader. This class loader loads the classes from classes and jar files in CLASSPATH environment variable or given in -classpath command line option.
Some important points about Classpath of the System Class Loader,

Default classpath is "." aka current directory.
You can use CLASSPATH environment variable to define the classpath.
You can use -cp or -classpath command line option to define the classpath, this will override the value defined in CLASSPATH environment variable.
When you use -jar command line option, it will ignore the values in -cp and -classpath command line options, and also the value of CLASSPATH environment variable. It will only consider the Class-Path attribute defined in the META-INF/mainfest.mf file of the jar file.
If two classes with the same name exists in the classpaths, the class loader will load the first found class.
Current value of the classpath being used by a java program can be retrieved from java.class.path system property.

System Class Loader is implemented by sun.misc.Launcher$AppClassLoader class and it is a child of Extension Class Loader. You can change the System Classpath Class Loader to use your own class loader implementation instead of sun.misc.Launcher$AppClassLoader, by using -Djava.system.class.loader command line option.

User Defined Class Loaders

Developers can create their own class loaders to support specific requirements of their application. Possible examples for implementing user defined class loaders,

to load classes through network.
to load classes which are auto generated at runtime.
to load classes from encrypted files.
to separate class loading of different applications which runs in same JVM.
etc.

Class Loading Algorithm

Class loaders (at least JVM class loaders) follow following algorithm when it wants to load a class. (This algorithm is based on delegating the task to parent first.)

Check whether class is already available in the local cache of the current class loader.
If the class was not found in the cache, ask (delegate) the parent class loader to load the class. (Please remember that Bootstrap Class Loader does not have a parent.)
Parent class loader also does above steps recursively until the class found in its cache or no parent class loader found. (Ask parent's parent class loader to load, if parent class loader did not find the class in its cache.).
If the parent class loader can find and load the class, it store it in its cache and return it to its child class loader.
If parent class loader cannot load the class, the class loader tries to load the class from its classpaths (aka code sources).
If the class loader cannot find the class in its local cache and cannot get the class from parent class loaders and cannot load the class from its local code sources, it throws ClassNotFoundException.

Example: When system classpath class loader wants to load javax.imageio.ImageIO class (a class belong to Java Extensions),

System Classpath Class Loader check whether it has javax.imageio.ImageIO class loaded in its local cache.
It does not find it in local cache, so it ask its parent class loader (which is Extensions Class Loader) to load the javax.imageio.ImageIO class.
Extension Class Loader checks its local cache whether it has javax.imageio.ImageIO class.
If it found the javax.imageio.ImageIO class in its local cache, it returns it to the System Classpath Class Loader.
If it did not find the javax.imageio.ImageIO class in its local cache, it ask its parent class loader (which is Bootstrap Class Loader) to load the javax.imageio.ImageIO class.
Bootstrap Class Loader checks whether it has javax.imageio.ImageIO in its local cache. If it has the class in local cache, it returns it to Extension Class Loader method call. (But in our example, Bootstrap class loader should not find the class in its local cache, because the class is in extension classpath).
If Bootstrap Class Loader did not find it in its local cache, and because it does not have any parent class loader, it tries to load the javax.imageio.ImageIO class from its classpath aka its local code sources (bootstrap classpath).
Because the class is belong to Java Extensions, Bootstrap Class Loader does not find it in bootstrap classpath. Then it return the method call to its child class loader (Extension Class Loader) without any class loaded.
Then Extension Class Loader tries to find the javax.imageio.ImageIO class in its classpath aka its local code sources.
Because the class belongs to Java Extensions, Extension Class Loader finds the class in its classpath and loads it. Then put the class in its local cache. Then it returns the class to its child class loader which is System Classpath Class Loader.

So when a class loader wants to load a class, it first check whether class is already loaded in its local cache, then delegates the loading to the parent class loader if it is not in the current class loader's cache. The class loader will try to load the class itself, only if the parent class loader cannot load the class.
Therefore the classes loaded by parent class loaders are visible to the children class loaders of that parent class loader. But the classes loaded by a child class loader are not visible to any of parent class loader of the child. Due to this algorithm, a class loaded by parent class loader will not be reloaded by a child class loader of it.

Stages in Class Loading

There are 3 main stages in loading a class from byte code to proper JVM internal representation.

Loading - Class Loader search for the class file (byte code) in the classpath and the byte code is loaded to memory (as a ClassFile structure). This step creates a basic structure for the class, but it does not have all the important data filled yet.
Linking - This step tries to convert byte code data to proper runtime information. This step is sub divided into 3 more sub steps.

Byte Code Verification - the byte code is checked for the correct format and other checks. This step ensures that loaded class is structurally correct.
Class Preparation - This step prepare the data structures to represent methods, fields, and implemented interfaces etc. from the byte code. This step create static fields and initialize those to their default values.
Resolving - This step do the resolving of other classes referenced by this class. (e.g.: Super classes, field types, method param types, method local variable types)

Initializing - This step executes the static initializer blocks.

Class Data Sharing (CDS) feature

Class Data Sharing (CDS) is a feature introduced in JavaSE 5.0 (in Oracle/Sun hotspot JVM) in order to reduce the java application start-up time by improving class loading time. When the JRE is installed in your machine, JRE loads the classes from system jar files into internal representations and then dump those loaded internal representations to a file known as "shared archive" (which is located in jre/lib/[arch]/client/classes.jsa in UNIX and in jre/bin/client/classes.jsa in Windows). During subsequent JVM invocations, the "shared archive" is used by those JVM processes by memory mapping to it, so reducing the loading time of classes loaded by Bootstrap class loader (which also means this memory mapped file is shared among multiple JVM processes).

If this feature is enabled, you will see many lines with "[Loaded className from shared objects file]" when you run a java application with "-verbose:class" command line argument. The list of classes in the "shared archive" can be found in "jre/lib/classlist" file.

You can use "-XShare" command line option to either enable or disable class data sharing for your java application.

In JDK 8U40, Oracle has introduced an experimental feature named Application Class Data Sharing (AppCDS) by extending CDS, to allow developers to place classes from Standard Extensions directories and from Application classpath in the "shared archive".

See http://docs.oracle.com/javase/7/docs/technotes/guides/vm/class-data-sharing.html for more details about Class Data Sharing.

Common Errors/Exceptions related to class loading

ClassNotFoundException

java.lang.ClassNotFoundException is thrown when your program dynamically tries to load the class at runtime and the class loader did not find the requested class. Main reason is, the class is not available in the CLASSPATH.
This is normally thrown by methods like Class.forName(), ClassLoader.loadClass(), ClassLoader.findSystemClass() when the java program tries to load a class dynamically, using its string name. Due to this reason, the compiler does not detect the unavailability of the class at the compile time.

NoClassDefFoundError

java.lang.NoClassDefFoundError happens when JVM or a class loader instance trying to load a class definition, but this class definition cannot be found or loaded due to some reason.
Most of the time, the class was available in the classpath during the compile time, but it is no longer available during the runtime. Another possible reason is, the class is still available in the classpath, but the class has static initialization blocks which fails during the loading of the class.

¯\_(ツ)_/¯

Friday, October 21, 2016

Class Loading in Java - Concept, Class Loader Types, and Related Exceptions