Java Memory Leaks
April 17, 2008 – 7:20 pmFinding (and fixing) memory leaks in java can sometimes be a tricky process. I recently had to do some analysis on a leaking application and hit a snag.
My game plan was to do the following:
Along the way, I ran into a problem with jmap and discovered a couple new tools.
Problem with jmap
The first problem I had was actually generating the heap dumps. My application was running 32 bit JVM version 1.5.0_11-b03 on Solaris 10, and every time I tried running
jmap -heap:format=b $pid
, the process would start, but never finish. I could see that the heap.bin file was created and growing in size, but the growth rate would get slower and slower until, eventually, it would fail with:
Exception in thread "main" java.lang.OutOfMemoryError:
requested 8192 bytes for jbyte in
/BUILD_AREA/jdk1.5.0_11/hotspot/src/share/vm/prims/jni.cpp. Out of swap space?
Well, after multiple attempts and trying to run on machines with more than 2 GB of swap space, I started to monitor the jmap process and found that it would grow and grow in size, until almost 4 GB when it would fail with the OOME. Ok, now this is looking like a bug. I found a similar bug for an older version of the JDK, but the symptoms were similar.
After installing JDK 1.5.0_15, I tried again and bingo, worked like a charm. Damn, wasted a good few hours there.
Next… Analysis
I had originally planned on doing all of the leak analysis using jhat, but I stumbled upon the SAP Memory Analyzer while researching the jmap problem and boy, am I glad I did.
It looks like jhat has potential to be quite powerful with OQL (object query language), but it was terribly slow with my 600 MB heap dumps. Although jhat is distributed with JDK 1.6, it can handle heap dumps created in 1.5 with no problem.
In contrast, the SAP Memory Analyzer was fantastic. It had no problem with large heap dumps, it was fast and it led me almost directly to the leaking culprits. The documentation provided was also very helpful.
Conclusion
In summary, the process works like this:
- Capture Heap Dump
- Analyze Heap Dump
For Step 1, you have option of using jmap against a running process or a core file, which can be created using gcore. The syntax is slightly different depending on your JDK version (see Resources below).
For Step 2, I would highly recommend the SAP Memory Analyzer. I actually switched back and forth between it and jhat, but if you were going to choose just one, go with the SAP tool.
Resources
jmap - JDK 1.5
jmap - JDK 1.6
jhat - Java Heap Analysis Tool
HAT Java 1.5 Heap Analysis Tool
SAP Memory Analyzer
Edward Chou’s Blog - provides some good jhat tips
8 Responses to “Java Memory Leaks”
here is my practice based on my daily testing work, not that hard if you make it systemiclly:
1.jconsole is a very good tool for detecting java memory leak issue at first.
2.narrow down the test cases or senarios to find which part may cause the problem
3.also i love jmap to get heap dump snapshot to get deep analysis after we are clear we have an issue on memory.
4. use some tool to read the binary heap dump file if necessary
–Joychester,performance testing engineer
By joychester on May 12, 2008
As you point out, jconsole can be good for detection, but once you’ve identified a memory leak exists, you have to move to other tools to find root cause.
In general, the java tools are much improved in Java 6, but a lot of people aren’t on Java 6 in production systems. So I’m stuck with Java 5 and the problems that jmap has in some older revisions (at least updates 11 and 12 are known to have the issue described above).
By Charlie on May 13, 2008
Charlie,
You can try this way to use Jconsole 6 remotely to detect your application memory leak, even your application is under JDK5, not JDK6. (That means your remote dasktop or laptop is under JDK6)I am using this way very well
(BTW,Jconsole in JDK5 is not good at all.)
For further analysis(personally speaking, I perfered to use Netbean heap walker) we should use other tools to support. if application is using JDK6, everything is good, you can extract memory heap snapshot banary file using “Mbean” tab. if application is under JDK5, then you need other tools to support.
Here we just talk about the open source solution.
—Joychester, Performance engineer
By joychester on May 18, 2008
UPDATE:
The SAP Memory Analyzer is now an Eclipse project
By charlie on Jun 17, 2008
I tried running your command (jmap -heap:format=b $pid) but all I get is the following:
Attaching to process ID 6535, please wait…
Debugger attached successfully.
Server compiler detected.
JVM version is 1.5.0_15-b04
heap.bin (Permission denied)
Anyone got any idea of what is happening here?
By John P. Anderson on Oct 13, 2008
I suspect it is either you don’t have permission to write to the current directory, or you aren’t running jmap as the owner of the process.
By charlie on Oct 13, 2008
Hi,
I am using jmap on Linux (JVM is 1.5.0_15-b04).
I use the following command to take the heap dump:
jmap -heap:format=b
This creates a heap dump file in my current directory as “heap.bin”.
I then copied this heap.bin from Linux to windows using Winscp.
But from windows when I open this file in SAP Memory Analyzer(with open snapshot) it gives me the following error:
org.eclipse.core.runtime.CoreException: Text editor does not have a document provider
at org.eclipse.ui.texteditor.AbstractTextEditor.doSetInput(AbstractTextEditor.java:3928)
at org.eclipse.ui.texteditor.StatusTextEditor.doSetInput(StatusTextEditor.java:190)
at org.eclipse.ui.texteditor.AbstractDecoratedTextEditor.doSetInput(AbstractDecoratedTextEditor.java:1225)
at org.eclipse.ui.editors.text.TextEditor.doSetInput(TextEditor.java:168)
at org.eclipse.ui.texteditor.AbstractTextEditor$19.run(AbstractTextEditor.java:2994)
at
I even renamed the dump file to “heap.hprof” as in the sample snapshots of SAP memory Analyzer the filenames are: “HeapDumpSample.hprof”
Please help me in opening the heap dump file from jmap in SAP Memory Analyzer.
Where did I make a mistake?
Nirmal…..
By Nirmal on Oct 14, 2008
This is a good tool too ->
http://www.alphaworks.ibm.com/tech/heapanalyzer
–
“No trees were destroyed in the sending of this message.
However, a large number of electrons were terribly inconvenienced.”
By loolek on Feb 24, 2009