1.
What is Thread dump?
2.
When we will take Thread dump? (Scenarios)
4.
Thread Dump Generating Procedure
5.
What can I Analysis with Thread Dump?
6.
How can I analysis thread dump?
7.
Actions taken for Issue resolving
8.
References
Coming to step by step learning:
--------------------------------
What is Thread dump?
Thread Dump is a textual dump of all active threads and monitors of Java
apps running in a Virtual Machine.
When we will take Thread dump? (Scenarios)
1. Scenario 1: when server is hang Position, i.e.
that time server will not respond to coming requests.
2. Scenario 2: While sever is taking more time to
restart
3. Scenario 3: When we are Getting exception like
“java.lang.OutOfMemoryException”
4. Scenario 4: Process running out of File
descriptors. Server cannot accept further requests because sockets cannot be
created
5. Scenario 5: Infinite Looping in the code
How many ways take Thread Dumps?
Many types we have to take a Thread dumps. As per your flexibility you
can choose one Procedure. For analyzing take dumps some Intervals (like every
10mins, 10mins etc.).
Generating Dump Talking Procedures
1. Take Thread dump from Console by Using of below command
$kill -3 PID
(For Getting PID, Use this Command ps –ef | grep “java”)
Here The Output of the Thread Dump will be generated in the Server
STDOUT.
(Note: If a process is not responding to kill -3 then it’s a JVM bug.)
2. Generation Thread Dump via Admin Console
a. login to Admin Console(with Admin Username/Password)
b. Click on Server, after choose your server
c. Goto Monitoring TAB
d. Goto Threads TAB, after click on “Dump Thread Stack” Button
e. Now you can view the all the Threads in Same page
f. Copy and paste in a txt file.
3. We can Collect Thread Dump Using “WebLogic.Admin” which
is deprecated, but still available or may be available in near future as well
As i think because it is one of the best debugging utility for Admins.
java WebLogic.Admin -url t3://hostname: port -username Weblogic
-password Weblogic THREAD_DUMP
This Thread Dumps will be generated in Servers STDOUT file
4. Getting Thread Dumps by using Jstack Utility
a.jstack –m (to connect to a live java process)
b. jstack –m [server_id@]
(to connect to a remote debug server)
(-m Means print both java and native frames (mixed
mode))
process ID: jps
[soaosb]-->jps
19234 Server
17786 Server
11653 TMMain
18087 NodeManager
23300 Jps
18414 Server
18886 Server
example: -->$JAVA_HOME/bin/jstack -l 19234 > osb1.tdump
5. By Using WLST Script, can contain extension of (.py)
connect(‘weblogic’,'weblogic’,'t3://hostname:port′) cd
(”Servers’) ls()cd (‘AdminServer’) ls() threadDump()
Execute this Script in console.
What can I Analysis with Thread Dumps?
We need to analyze the thread dumps for analyzing running threads and
their states to identifying.
How can I analysis thread dumps?
For analyze thread dumps we have lots of tools to understand easily
thread states
1. samurai tool :
In this tool you can identify all the Thread states
by identifying colors. We need to take care about
Deadlocks and waiting state threads.
More Details:
$ java -jar samurai.jar
After running we will get a Screen like below
Goto Thread dump tab
When Samurai detects a thread dump in your log, a tab
named "Thread Dump" will appear.
You can just click "Thread dumps" tab to see the
analysis result. Samurai colors idle threads in gray, blocked threds in red and
running threds in green. There are three result views and Samurai shows
"Table view" by default. In many case you are just interested in the
table view and the sequence view. Use the table view to decide which thread
needs be inspected, the sequence view to understand the thread's behavior. You
should takecare especially threds always in red.
2. TDA Tool :
(Visit: https://tda.dev.java.net/)
Actions taken for Issue resolving
1. Classic Dead Locks : Look for the threads
waiting for monitor entry
For Example :
"ExecuteThread: '95' for queue: 'default'" daemon prio=5
tid=0x411cf8 nid=0x6c waiting for monitor entry [0xd0f80000..0xd0f819d8]
at
weblogic.common.internal.ResourceAllocator.release(ResourceAllocator.java:766)
at weblogic.jdbc.common.internal.ConnectionEnv.destroy(ConnectionEnv.java:590)
Reason: The above thread is waiting to acquire lock on Resource Allocator
object. The next step is to identify the thread that is holding the Resource
Allocator object
"ExecuteThread: '0' for queue: '__weblogic_admin_rmi_queue'"
daemon prio=5 tid=0x41b978 nid=0x77 waiting for monitor entry
[0xd0480000..0xd04819d8]
at
weblogic.jdbc.common.internal.ConnectionEnv.getPrepStmtCacheHits(ConnectionEnv.java:174)
at weblogic.common.internal.ResourceAllocator.getPrepStmtCacheHitCount
(ResourceAllocator.java:1525)
Reason: This thread is holding lock on source Allocator object, but is waiting
for Connection Env object. This is a classic deadlock.
2. Threads in wait() state:
A sample dump:
"ExecuteThread: '10' for queue: 'SERV_EJB_QUEUE'" daemon
prio=5 tid=0x005607f0 nid=0x30 in Object.wait() [83300000..83301998]
at java.lang.Object.wait(Native Method)
- waiting on (a weblogic.ejb20.pool.StatelessSessionPool)
at weblogic.ejb20.pool.StatelessSessionPool.waitForBean(StatelessSessionPool.java:222)
Reason: The above thread would come out of wait() under two conditions
(Depending on application logic)
1) One of the thread available in the execute queue pool would call
notify() on this object when an instance is available. (If the wait() is
indefinite).
This can cause the thread to hang for ever if server never
does a notify() to this object.
2) If the timeout exceeds, the thread would throw an exception and back
to execute queue thread pool.
