This document describes a heap dump file for Java programs, and a tool to analyze these files. This tool can help you to debug an analyze the objects in a running Java program. It is particularly useful when debugging unintentional object retention.
Unintentional object retention, a phenomenon sometimes called "memory leaks," can come about in several ways. I call it a unintentional object retention when an object that is no longer needed is referenced through some path from the rootset. This can happen, for example, if an unintentional static reference to an object remains after the object is no longer needed, if an Observer or Listener fails to de-register itself from its subject when it is no longer needed, or if a Thread that refers to an object does not terminate when it should.
This tool helps to debug unnecessary object retention by providing a convenient means to browse the object topology in a heap snapshot that is generated by the Java VM. This shapshot is in a JDK 1.2 heap profile file (generated with -Xhprof). The tool allows a number of queries, including "show me all reference paths from the rootset to this object". It is this query that is most useful for finding unnecessary object retention.
This document will discuss each of the queries that the analysis tool offers.
To generate a .hprof file, you should run java with the -Xhprof flag. The format recognized by this program first appeared in JDK 1.2 beta3. HAT must be given a binary hprof file. This is done with the "format=b" modifier to the -Xhprof command. For example, to run the program Main and generate a binary heap profile in the file dump.hprof, use the following command:
java -Xhprof:file=dump.hprof,format=b Main
With JDK 1.2, you can add a heap dump to a .hprof file with control-backslash under Unix, or control-break in Win32. A sample .hprof file is provided in this directory (slotCar.hprof). This was produced with a small slot car animation program I wrote in a former life. The source for that program can be found under http://jovial.com/slotCar/.
To run the hat server program, use the run shell script provided in the bin directory. For example, to analyze slotCar.hprof, use the following command:
./hat -port=7002 ../../doc/slotCar.hprof
If a port is not specified, it defaults to 7000.
The server can compare two heap dumps, and it's possible to specify a numbered heap dump in the file. For example, to compare the first and second heap dump in slotCar.hprof, use this command:
./hat -baseline=../../doc/slotCar.hprof:1 ../../doc/slotCar2.hprof:2
You can specify a file that influences the reachable objects query. This file lists the data members that should be excluded when generating this query. This can be used to determined the size of a given object and all objects reachable from it, excluding observers and other objects that aren't "owned" by the target object. To specify a set of data members to exclude, use the -exclude command line option:
./hat -exclude=../../doc/slotCar.exclude ../../doc/slotCar.hprof
This file should contain the fully-qualified name of each data member to exclude, with a newline between each data member. If this file is modified while a server is running, it will be re-read.
The program sets itself up as an http server on the port you specify. You access it by bringing up the default page in the browser of your choice. For example, if your computer were called "mango", you could access the server started above by navigating to the URL "http://mango:7002/". From there, you can run other queries by clicking on hyperlinks, or by directly typing in URLs.
This document contains links to sample HTML pages generated by the server. The links within these sample pages will not work. To generate a working example, run the server program with the provided .hprof file.
The default page shows you all of classes present on the heap. This list is sorted by fully-qualified class name, and broken out by package. By clicking on the name of a class, you are taken to the Class query. At the bottom of the page is a link you can click to be taken to the All Roots query.
The Class query shows you information about a class. This includes its superclass, any subclasses, instance data members, and static data members. From this page you can navigate to any of the classes that are referenced, or you can navigate to an Instances query.
The instances query shows you all instances of a given class. The "allInstances" variant includes instances of subclasses of the given class as well. From here, you can navigate back to the source class, or you can navigate to an Object query on one of the instances.
The object query gives you information about an object that was on the heap. From here, you can navigate to the class of the object, to the value of any data members of the object. You can also navigate to objects that refer to this object. Perhaps the most valuable query is at the end: The Roots query ("Reference Chains from Rootset").
Note that the object query also gives you a stack backtrace of the point of allocation of the object.
The roots query gives you reference chains from the rootset to a given object. It will give you one chain for each member of the rootset from which the given object is reachable. When calculating these chains, it does a depth-first search, so that it will provide reference chains of minimal length.
There are two kinds of roots query: One that excludes weak references ("roots"), and one that includes them ("allRoots"). A weak reference is a reference object that does not prevent its referent from being made finalizable, finalized, and then reclaimed. Weak references are subclasses of sun.misc.Ref, or java.lang.ref.WeakReference as of JDK 1.2. If an object is only refered to by a weak reference, it usually isn't considered to be retained, because the garbage collector can collect it as soon as it needs the space.
The server sorts rootset reference chains by the type of the root, in this order:
This query, accessable from the Object query, shows the transitive closure of all objects reachable from a given object. This list is sorted in decreasing size, and alphabetically within each size. At the end, the total size of all of the reachable objects is given. This can be useful for determining the total runtime footprint of an object in memory, at least in systems with simple object topologies.
This query is most valuable when used in conjunction with the "-exclude" command line option, described above. This is usefule, for example, if the object being analyzed is an Observable. By default, all of its Observers would be reachable, which would count against the total size. -exclude allows you to exclude the data members java.util.Observable.obs and java.util.Observable.arr.
This query shows the count of instances for every class in the system. It is sorted in decending order, by instance count.
This query shows all members of the rootset.
The New Instances query is only available if you invoke the hat server with two heap dumps. It is like the Instances query, but it only shows "new" instances. An instance is considered new if it is in the second heap dump, and there is no object of the same type with the same ID in the baseline heap dump. An object's ID is a 32 bit integer that uniquely identifies the object, and it is assigned by the VM. In the VM implementations I know about, an object's ID is the address of its handle. Because handle address can be re-used, the new instances query could give inaccurate results, and it might not work well with other VM implementations. With the current VM, and comparing snapshots that are taken within a relatively short interval, you can generally expect good results.
The hat server can be valuable for debugging and understanding programs. It allows you to naviagate object structures to learn how objects are interconnected in a program. It also allows you to trace the references to a given object from the rootset, which can be valuable for tracking down unnecessary object retention. We found the hat server to be a valuable debugging aid, and hope that you will, too.
For more information, please consult the HAT page at http://java.sun.com/people/billf/heap/.