Monday, September 2, 2013

How to find what is the current working directory of a remotely logged ssh session to a Linux server and disconnect remote sessions

First you need to get the list of users currently connected to your server. You can use "last" command to achieve that. "last" command shows the list of users currently logged in or recently logged in to the system through various terminals. If the user still is connected to the system through the relevant terminal, you can see "still logged in" part in the output row related to that user.
If the user is connected remotely (for example, by ssh), you can see the his terminals related to remote sessions has the prefix "pts" (Pseudo Terminals) and in the third column of output, you can see the remote IP of the machine which the user connected from.

[root@10 ~]# last | grep "still logged"
root     pts/2        192.168.18.1     Mon Sep  2 21:42   still logged in   
root     pts/1        :0.0             Mon Sep  2 21:12   still logged in   
root     :0                            Mon Sep  2 21:11   still logged in 

Then you can find the processes originated by  the user by a connected pseudo terminal using "ps" command and grep command.

[root@10 ~]# ps -ef | grep -v grep | grep "pts/2"
root      6224  3971  0 21:42 ?        00:00:00 sshd: root@pts/2 
root      6229  6224  0 21:42 pts/2    00:00:00 -bash

In the above output, you can see the processes initiated by the remote connection. The first process "sshd" is the process used to initiate the remote connection to this machine. The next process is the shell the remote user has logged into. Therefore the process we needs to check is the second process which "-bash" shell process.

By killing "sshd: root@pts/2" or "-bash" process (process ID 6224 or 6229), you can disconnect this user from the server.

We can use one of following command to get the current working directory of the remotely connected user in "pts/2".

[root@10 ~]# pwdx 6229
6229: /product/softwares

Or

[root@10 ~]# readlink /proc/6229/cwd
/product/softwares

Or

[root@10 ~]# lsof -p 6229 
COMMAND  PID USER   FD   TYPE DEVICE     SIZE    NODE NAME
bash    6229 root  cwd    DIR  253,0     4096  397609 /product/softwares
bash    6229 root  rtd    DIR  253,0     4096       2 /

How to get the current working directory of a process in Linux

Sometimes we encounter  situations where you need to  find the current working directory of a running process. There are several ways to get this information in a Linux machine.

Method 1:

We can get the current working directory (and more details) of a process with pid $PID, by checking the contents of /proc/$PID folder.

Example:

For a process with process ID 3551

[root@10 ~]# readlink /proc/3551/cwd
/etc/sysconfig/network-scripts

Or with more details,

[root@10 ~]# ll /proc/3551
total 0
dr-xr-xr-x 2 root root 0 Sep  2 21:15 attr
-r-------- 1 root root 0 Sep  2 21:15 auxv
-r--r--r-- 1 root root 0 Sep  2 21:05 cmdline
-rw-r--r-- 1 root root 0 Sep  2 21:15 coredump_filter
-r--r--r-- 1 root root 0 Sep  2 21:15 cpuset
lrwxrwxrwx 1 root root 0 Sep  2 21:15 cwd -> /etc/sysconfig/network-scripts
-r-------- 1 root root 0 Sep  2 21:15 environ
lrwxrwxrwx 1 root root 0 Sep  2 21:05 exe -> /sbin/dhclient
dr-x------ 2 root root 0 Sep  2 21:15 fd
dr-x------ 2 root root 0 Sep  2 21:15 fdinfo
-r--r--r-- 1 root root 0 Sep  2 21:15 io
-r--r--r-- 1 root root 0 Sep  2 21:15 limits
-rw-r--r-- 1 root root 0 Sep  2 21:15 loginuid
-r--r--r-- 1 root root 0 Sep  2 21:15 maps
-rw------- 1 root root 0 Sep  2 21:15 mem
-r--r--r-- 1 root root 0 Sep  2 21:15 mounts
-r-------- 1 root root 0 Sep  2 21:15 mountstats
-rw-r--r-- 1 root root 0 Sep  2 21:15 oom_adj
-r--r--r-- 1 root root 0 Sep  2 21:15 oom_score
lrwxrwxrwx 1 root root 0 Sep  2 21:15 root -> /
-r--r--r-- 1 root root 0 Sep  2 21:15 schedstat
-r--r--r-- 1 root root 0 Sep  2 21:15 smaps
-r--r--r-- 1 root root 0 Sep  2 21:05 stat
-r--r--r-- 1 root root 0 Sep  2 21:15 statm
-r--r--r-- 1 root root 0 Sep  2 21:12 status
dr-xr-xr-x 3 root root 0 Sep  2 21:15 task
-r--r--r-- 1 root root 0 Sep  2 21:15 wchan

Method 2:

Use pwdx command.

Example:

For a process with process ID 3551


[root@10 ~]# pwdx 3551
3551: /etc/sysconfig/network-scripts

Method 3:

Use lsof command.

Example:

For a process with process ID 3551

[root@10 ~]# lsof -p 3551
dhclient  3551      root  cwd       DIR      253,0     4096    1769476 /etc/sysconfig/network-scripts
dhclient  3551      root  rtd       DIR      253,0     4096          2 /
....

Monday, July 22, 2013

Different behaviour of Source or (dot) or (period) operator when passing arguments in Linux and Solaris

The source operator is used for including another file in current script and execute the lines in that file in the current shell context. Therefore the final result is same as that the lines in the sourced file were physically presented in the parent file. I recently found out that the behavior of source operator is different in Linux and Solaris.
If you pass command line arguments to a file you are sourcing inside another shell script, and when you retrieve the passed command line arguments in child (sourced) file,

In Linux,
You will get the parameters you passed as command line parameters when you sourcing the file.

In Solaris,
You will get the command line parameters passed to the parent file as the command line parameters for the child file also.

Therefore if you have a file named test.sh with following lines, (testEcho.sh will echo all the command line arguments passed to it).

. "./testEcho.sh" ted
sh ./testEcho.sh ted


and if you execute it like below,
sh ./test.sh param1 param2 param3


Output in linux,
ted
ted

Output in Solaris,
param1 param2 param3
      ted

Saturday, July 20, 2013

How to get Java system properties and JVM flags of a running Java process - Using jinfo

I wanted to check the current system properties and JVM flags of a Weblogic managed server, so I first used "wlst" (Weblogic scripting tool) to access that managed server. It was a little time consuming task and you need to have a knowledge about the inside of Weblogic server. But later I found out about this jinfo tool. jinfo is a very useful tool for retrieving Java system properties and JVM flags (JVM command line flags)  (and changing values of JVM flags) of a running local Java process or a remote server process. It was lot easier than accessing these properties using "wlst" or "jconsole". According to the "jinfo" documentation, This tool can also be used to get system properties and JVM flags of a remote server process.

But I am going to focus only about how to get those details of a local Java process in a Linux environment.
You can get the process ID of the specific process by doing a command like below in Linux. (Please replace "managed1" with a word (main class name, for example) in the command used for starting the Java process.)

ps -ef | grep "managed1"

In the output of above command, you can get the PID of the relevant Java process.
Then execute the following command. Please replace <PID> with the actual PID of the process.(I am assuming that path to JDK bin folder is in PATH environment variable).

jinfo <PID>

It will attach to the process related to that PID and will show the Java system properties and JVM flags like below.

Attaching to process ID 25965, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 11.0-b17
Java System Properties:

java.vendor = Sun Microsystems Inc.
org.xml.sax.parser = weblogic.xml.jaxp.RegistryParser
sun.java.launcher = SUN_STANDARD
com.sun.xml.ws.api.streaming.XMLStreamReaderFactory.woodstox = true
sun.management.compiler = HotSpot Tiered Compilers
os.name = Linux
sun.boot.class.path = /opt/jdk1.6.0_11/jre/lib/resources.jar:/opt/jdk1.6.0_11/jre/lib/rt.jar...
weblogic.threadpool.MinPoolSize = 50
java.vm.specification.vendor = Sun Microsystems Inc.
.....
VM Flags:

-Dweblogic.Name=managed1 ..................
...................... 
   
Other features of this tool:
(I didn't have a chance to experiment with these yet):
1. Get only JVM flags of the relevant Java process.
jinfo -flags <PID>
2. Get only Java system properties
jinfo -sysprops <PID>
3. Get a value of a specific JVM flag of the relevant Java process.
jinfo -flag <JVM_FLAG_NAME> <PID>
4. Change the value of a specific JVM flag of the relevant Java process.
        a) Enabling  a boolean JVM flag,
         jinfo -flag +<JVM_FLAG_NAME> <PID>
        b) Disabling  a boolean JVM flag,
         jinfo -flag -<JVM_FLAG_NAME> <PID>
        c) Setting a value of a JVM flag
         jinfo -flag  <JVM_FLAG_NAME>=<NEW_VALUE> <PID>

If you get VMVersionMismatchException,

Sometimes you may get following VMVersionMismatchException. 

Attaching to process ID 6461, please wait...
Exception in thread "main" java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at sun.tools.jinfo.JInfo.runTool(JInfo.java:79)
        at sun.tools.jinfo.JInfo.main(JInfo.java:53)
Caused by: sun.jvm.hotspot.runtime.VMVersionMismatchException: Supported versions are 17.1-b03. Target VM is 20.5-b03
        at sun.jvm.hotspot.runtime.VM.checkVMVersion(VM.java:223)
        at sun.jvm.hotspot.runtime.VM.<init>(VM.java:286)
        at sun.jvm.hotspot.runtime.VM.initialize(VM.java:350)
        at sun.jvm.hotspot.bugspot.BugSpotAgent.setupVM(BugSpotAgent.java:594)
        at sun.jvm.hotspot.bugspot.BugSpotAgent.go(BugSpotAgent.java:494)
        at sun.jvm.hotspot.bugspot.BugSpotAgent.attach(BugSpotAgent.java:332)
        at sun.jvm.hotspot.tools.Tool.start(Tool.java:163)
        at sun.jvm.hotspot.tools.JInfo.main(JInfo.java:128)
        ... 6 more

The reason for this is, you are using a "jinfo" tool related to a different java version than the JVM used to start this targeted process. If you get above exception, please find the JDK home of the JVM used for the targeted process, and use the "jinfo" tool in the bin folder of that JDK home.

According to the jinfo tool documentation, it seems this is not a standard tool of the JDK. In the documentation, there is a note saying
"NOTE - This utility is unsupported and may or may not be available in future versions of the JDK. In Windows Systems where dbgent.dll is not present, 'Debugging Tools for Windows' needs to be installed to have these tools working. Also the PATH environment variable should contain the location of jvm.dll used by the target process or the location from which the Crash Dump file was produced.
For example, set PATH=\jre\bin\client;%PATH%
"

How to avoid "java.lang.NoClassDefFoundError: Could not initialize class" when using JFreeChart in a headless server

Recently I encountered following exception when using JFreeChart to generate a graphical chart in a product deployed in a weblogic server.

]] Root cause of ServletException.
java.lang.NoClassDefFoundError: Could not initialize class sun.awt.X11GraphicsEnvironment
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:169)
        at java.awt.GraphicsEnvironment.getLocalGraphicsEnvironment(GraphicsEnvironment.java:68)
        at sun.awt.X11.XToolkit.(XToolkit.java:89)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:169)
        at java.awt.Toolkit$2.run(Toolkit.java:834)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.awt.Toolkit.getDefaultToolkit(Toolkit.java:826)
        at sun.swing.SwingUtilities2$AATextInfo.getAATextInfo(SwingUtilities2.java:126)
        at javax.swing.plaf.metal.MetalLookAndFeel.initComponentDefaults(MetalLookAndFeel.java:1556)
        at javax.swing.plaf.basic.BasicLookAndFeel.getDefaults(BasicLookAndFeel.java:130)
        at javax.swing.plaf.metal.MetalLookAndFeel.getDefaults(MetalLookAndFeel.java:1591)
        at javax.swing.UIManager.setLookAndFeel(UIManager.java:541)
        at javax.swing.UIManager.setLookAndFeel(UIManager.java:581)
        at javax.swing.UIManager.initializeDefaultLAF(UIManager.java:1343)
        at javax.swing.UIManager.initialize(UIManager.java:1432)
        at javax.swing.UIManager.maybeInitialize(UIManager.java:1420)
        at javax.swing.UIManager.getDefaults(UIManager.java:660)
        at javax.swing.UIManager.getColor(UIManager.java:702)
        at org.jfree.chart.JFreeChart.(JFreeChart.java:261)
        at org.jfree.chart.ChartFactory.createTimeSeriesChart(ChartFactory.java:1918)
 

 Reason:
During the issue investigation, I found out that the above mentioned Weblogic server is located inside a headless server. (A headless server is a server which does not have a monitor, a key board and a mouse. The control and access to a headless server is done using a network card (remote connection).) It turned out the reason for the above exception is that.

The problem happens because JFreeChart class has some static variables which are initialized using local graphic environment properties retrieved from javax.swing.UIManager.
javax.swing.UIManager in turn tries to get these default properties from local graphic environment.
But because there is no graphic environment in a headless server , initializing sun.awt.X11GraphicsEnvironment class (local graphic environment class) fails and then the loading of JFreeChart class also fails.

Solution:
The solution for this problem is to add -Djava.awt.headless=true to the list of server start arguments of the managed server.  You can change it by,
  1. Login to the Weblogic admin console.
  2. Expand Environment and select Servers.
  3. In the Servers table, click the name of the managed server you want to configure.
  4. Select Configuration > Server Start.
If this is another server or some other java application, you can add this property to wherever place you can specify the JVM arguments for the start of that server or application.



Read more about Java headless mode from: http://www.oracle.com/technetwork/articles/javase/headless-136834.html

Friday, June 21, 2013

How to solve "javax.naming.CommunicationException [Root exception is java.net.ConnectException" thrown when connecting to a EJB in weblogic server

Recently during a test done on an EJB, I got the following exception.

javax.naming.
CommunicationException [Root exception is java.net.ConnectException: t3://server01:7101: Bootstrap to server01/10.10.58.26:7101 failed. It is likely that the remote side declared peer gone on this JVM]
javax.naming.
CommunicationException [Root exception is java.net.ConnectException: t3://server01:7101: Bootstrap to server01/10.10.58.26:7101 failed. It is likely that the remote side declared peer gone on this JVM]
    at weblogic.jndi.internal.
ExceptionTranslator.toNamingException(ExceptionTranslator.java:40)
   ......

Caused by: java.net.ConnectException: t3://
server01:7101: Bootstrap to server01/10.10.58.26:7101 failed. It is likely that the remote side declared peer gone on this JVM
    at weblogic.rjvm.RJVMFinder.
findOrCreateInternal(RJVMFinder.java:216)
..............

    ... 25 more
Caused by: java.rmi.ConnectException: Bootstrap to
server01/10.10.58.26:7101 failed. It is likely that the remote side declared peer gone on this JVM
    at weblogic.rjvm.
ConnectionManager.bootstrap(ConnectionManager.java:334)
    at weblogic.rjvm.RJVMManager.
findOrCreateRemoteInternal(RJVMManager.java:254)
    at weblogic.rjvm.RJVMManager.
findOrCreate(RJVMManager.java:197)
    at weblogic.rjvm.RJVMFinder.
findOrCreateRemoteServer(RJVMFinder.java:238)
    at weblogic.rjvm.RJVMFinder.
findOrCreateInternal(RJVMFinder.java:200)
    ... 31 more

 It turned out that the reason for this exception is, the listen address (lets say "lstnr01") declared in the managed server in Weblogic Admin console is different from server name (although it is a correct address to that machine, I can use it to connect to weblogic console and do ssh) I used for initiating the connection to EJB. You can check the listen address of the server by browsing to "Environments"->Servers->YourServer->Configuration->General. In that page, there is a text box to enter "Listen Address".

Now after finding the reason for the problem, next problem I got is I cannot use this listen address "lstnr01" to connect to my machine, because that is not registered in a DNS server in the network. The mapping of "lstnr01" to that machine was only configured in the "/etc/hosts" file of the server.
Therefore to fix that issue, I added the same IP host name mapping entry to map "lstnr01" to server IP in the C:/Windows/system32/drivers/etc/hosts/hosts file.

10.10.58.26  lstnr01  

Then I executed the "ipconfig /flushdns" command using the command line. It refreshed the DNS cache of the local machine and updated the cache with newly added entry in the hosts file.
Then I used "lstnr01" as the remote server address instead of "server01" when connecting to the EJB.
 Then the above exception was gone and I was able to connect to the EJB and run the EJB method successfully.

How to highlight a searching text in "tail -f" output

When we are investigating an software issue of a software run on a unix machine, we normally use "tail -f" command to see the live log messages logged by the software (for e.g: checking managed server logs in a weblogic server). If you are looking the log for finding a specific log message, it will be easier if you can make the search word highlighted in some color in the output.

Following method is a one of way you can achieve that task. Please note that to write ^[ character, you should press Ctrl+V and then Ctrl+[

tail -f testEcho.txt | sed "s/\\(search\\)/^[[33;1m\\1^[[0m/g"

 



Wednesday, June 12, 2013

Writing a custom JUnit runner for running a single method using command line

Recently I needed to create a custom JUnit runner for executing a single method using the command line.  This is how I implemented that.


package testpackage;

import org.junit.internal.TextListener;
import org.junit.runner.JUnitCore;
import org.junit.runner.Request;
import org.junit.runner.Result;
import org.junit.runner.notification.RunListener;

public class SingleJUnitTestRunner {

    /**
     * Custom built junit runner for running single method.
     * Usage:
     * java -cp ...... testpackage.SingleJUnitTestRunner testpackage.TestClass#testMethod
     */
    public static void main(String... args) throws ClassNotFoundException {
        if (args.length == 0){
            System.out.println("Usage:");
            System.out.println("java -cp ...... testpackage.SingleJUnitTestRunner testpackage.TestClass#testMethod");
            System.exit(-1);
        }
        String[] classAndMethod = args[0].split("#");

        if (classAndMethod .length != 0){
            System.out.println("Usage:");
            System.out.println("java -cp ...... testpackage.SingleJUnitTestRunner testpackage.TestClass#testMethod");
                     System.exit(-1);
        }
        System.out.println("======================================");
        System.out.println("Test Execution started for " + classAndMethod[1] + " method in class " + classAndMethod[0] + ".");
        System.out.println("=====================================");

        Request request = Request.method(Class.forName(classAndMethod[0]),
                classAndMethod[1]);
        JUnitCore core = new JUnitCore();
        // Use the text listener to get the output from test cases to console
        RunListener listener= new TextListener(System.out);
        core.addListener(listener);
        Result result = core.run(request);


        System.out.println("RunCount: " + result.getRunCount());

        System.out.println("FailureCount: " + result.getFailureCount());
        System.out.println("IgnoreCount: " + result.getIgnoreCount());
        System.exit(result.wasSuccessful() ? 0 : -1);
    }

}