HAT -- The Java Heap Analysis Tool
Bill Foote
bill.foote at sun d o t com, billf at jovial d o t com
Introduction
This document describes a heap dump file for Java programs, and
a tool to analyze these files. This tool can help you to debug
an analyze the objects in a running Java program. It is particularly
useful when debugging unintentional object retention.
Unintentional object retention, a phenomenon sometimes called "memory
leaks,"
can come about in several ways. I call it a
unintentional object retention when an object that is no longer needed is
referenced through some path from the rootset. This can happen, for
example, if an
unintentional static reference to an object remains after the object
is no longer needed, if an Observer or Listener fails to de-register
itself from its subject when it is no longer needed, or if a Thread
that refers to an object does not terminate when it should.
This tool helps to debug unnecessary object retention
by providing a convenient means
to browse the object topology in a heap snapshot that is generated by
the Java VM. This shapshot is in a JDK 1.2 heap profile file (generated
with -Xrunhprof).
The tool allows a number of
queries, including "show me all reference paths from the rootset
to this object". It is this query that is most useful
for finding unnecessary object retention.
This document will discuss each of the queries that the analysis tool
offers.
Donations to Charities
HAT was done mostly nights and weekends, starting when I was
at JavaSoft. To cover Sun there is a
license, but as the author
may I make a small request?
Could you please read the words I've written on the
charitable donations page, and
do as you feel appropriate? Thank you.
Generating a .hprof File and Running the Program
To generate a .hprof file, you should run java with the -Xrunhprof
flag. The format recognized by this program first appeared in JDK 1.2
beta3. HAT must be given a binary hprof file. This is
done with the "format=b" modifier to the -Xrunhprof command. For
example, to run the program Main and generate a binary heap profile in
the file dump.hprof, use the following command:
java -Xrunhprof:file=dump.hprof,format=b Main
As of JDK 1.2, you can add a heap dump to a .hprof file with control-backslash
under Unix or OS/X, or control-break in Win32. Some sample .hprof files are
provided in the hprof_files subdirectory (doc/hprof_files). They were
produced with a small slot car
animation program I wrote in a former life. The source for that program
can be found under
http://jovial.com/slotCar/.
To run the hat server program, use the run shell script or batch file
provided in the bin directory. You'll probably want to copy it and
hat.jar somewhere sensible (I use ~/bin on Unix and OS/X, and c:/bin
on Windows). Once that's done, to analyze a slot car hprof file, you
can do the following:
cd doc/hprof_files
hat -port=7002 slot_car_jdk_1_5_0_beta2.hprof
If a port is not specified, it defaults to 7000.
The server can compare two heap dumps, and it's possible to specify a
numbered heap dump in the file. For example, to compare the first and second
heap dump in slotCar.hprof, use this command:
hat -baseline slot_car_from_jdk_1_2_beta3.hprof#1 slot_car_from_jdk_1_2_beta3.hprof#2
You can specify a file that influences the reachable objects query. This
file lists the data members that should be excluded when generating
this query. This can be used to determined the size of a given object
and all objects reachable from it, excluding observers and other objects
that aren't "owned" by the target object. To
specify a set of data members to exclude, use the -exclude command line option:
./hat -exclude slotCar.exclude slot_car_from_jdk_1_2_beta3.hprof
This file should contain the fully-qualified name of each data member
to exclude, with a newline between each data member. If this file
is modified while a server is running, it will be re-read.
The program sets itself up as an http server on the port you specify.
You access it by bringing up the default page in the browser of your
choice. For example, if you're running hat on the same computer
as your browser, you can probably call it "localhost",
if not "127.0.0.1" will do. Assuming your OS supports
calling it localhost, you
could access the server started above by navigating to the URL
"http://localhost:7002/". From there, you can run other queries
by clicking on hyperlinks, or by directly typing in URLs.
This document contains links to sample HTML pages generated by the
server. The links within these sample pages will not work.
To generate a working example, run the server program with the
provided .hprof file.
Note on JDK versions
The feature of generating a .hprof file was considered experimental
in JDK 1.2 through 1.4. It worked well with the classic VM on JDK 1.2,
but some VMs and some versions of the JDK might
not produce .hprof files compatible with HAT. As of JDK 1.5, .hprof
files are produced by an experimental agent that connects to the VM
via a supported profiling interface. As such, it's more stable and
dependable, though I'm told there were some issues with version prior
to update 3. I recommend using a JDK version of 1.5 update 3 or better with
HAT.
Sample HAT queries
The All Classes Query
Example: http://localhost:7002/
Example: http://localhost:7002/allClassesWithPlatform/
The default page shows you all of classes present on the
heap, excluding platform classes. This list is sorted by
fully-qualified class name,
and broken out by package. By clicking on the name of a class,
you are taken to the Class query. At the bottom of the page is
a link you can click to be taken to other queries.
The second variant of this query
includes the platform
classes. Platform classes include classes whose fully-qualified
names start with things like "java.", "sun.", "javax.swing.",
or "char[". The list of prefixes is in a system resource file
called "resources/platform_names.txt". If you want to override
this list, just replace it in the jar file, or arrange for your
replacement to occur first on the classpath when hat is invoked.
|
The Class Query
Example: http://localhost:7002/class/jovial.slotCar.Car
The Class query shows you information about a class. This includes its
superclass, any subclasses, instance data members, and static data members.
From this page you can navigate to any of the classes that are referenced,
or you can navigate to an Instances query.
|
The Object Query
Example: http://localhost:7002/object/ee3077c8
The object query gives you information about an object that was on the
heap. From here, you can navigate to the class of the object,
to the value of any data members of the object. You can also navigate
to objects that refer to this object. Perhaps the most valuable
query is at the end: The Roots query
("Reference Chains from Rootset").
Note that the object query also gives you a stack backtrace of the
point of allocation of the object.
|
The Roots Query
Example: http://localhost:7002/roots/ee3077c8
Example: http://localhost:7002/allRoots/ee3077c8
The roots query gives you reference chains from the rootset to a given
object. It will give you one chain for each member of the rootset from
which the given object is reachable. When calculating these chains, it
does a depth-first search, so that it will provide reference chains of
minimal length.
There are two kinds of roots query: One that excludes weak references
("roots"), and one that includes them ("allRoots").
A weak reference is a reference object that does not prevent
its referent from being made finalizable, finalized, and then
reclaimed. Weak references are subclasses of sun.misc.Ref, or
java.lang.ref.WeakReference as of JDK 1.2. If an object is only
referred to by a weak reference, it usually isn't considered to be
retained, because the garbage collector can collect it as soon as it
needs the space.
The server sorts rootset reference chains by the type of the root, in this
order:
- Static data members of Java classes.
- Java local variables. For these roots, the thread responsible
for them is shown. Because a Thread is a Java object, this link
is clickable. This allows you, for example, to easily navigate
to the name of the thread.
- Native static values.
- Native local variables. Again, such roots are identified with their
thread.
This is probably the most valuable query in HAT, if you're debugging
unintentional object retention. Once you find an object that's
being detained, this query tells you why it's being retained.
|
The Reachable Objects Query
Example: http://localhost:7002/reachableFrom/ee3077c8
This query, accessible from the Object query, shows the transitive closure
of all objects reachable from a given object. This list is sorted in
decreasing size, and alphabetically within each size. At the end,
the total size of all of the reachable objects is given. This can be useful for
determining the total runtime footprint of an object in memory, at least
in systems with simple object topologies.
This query is most valuable when used in conjunction with the
"-exclude" command line option, described above. This
is useful, for example, if the object being analyzed is an
Observable. By default, all of its Observers would be reachable,
which would count against the total size. -exclude allows you to
exclude the data members java.util.Observable.obs and
java.util.Observable.arr.
|
The Instance Counts for All Classes Query
Example: http://localhost:7002/showInstanceCounts/
Example: http://localhost:7002/showInstanceCounts/includePlatform/
This query shows the count of instances for every class in the system,
excluding platform classes. It
is sorted in descending order, by instance count. A good way to spot
a problem with unintentional object retention is to run a program
for a long time with a variety of input, then request a heap dump.
Looking at the instance counts for all classes, often a class you
know will jump out at you, because there are more than you expect.
Then, you can analyze them to determine why they're being retained
(possibly using the roots query).
There's also a variant of this that
includes platform classes. See the notes under the All Classes query
for notes about what is considered to be a platform class.
|
The All Roots Query
Example: http://localhost:7002/showRoots/
This query shows all members of the rootset.
The New Instances Query
Example: http://localhost:7002/newInstances/java.awt.Point
Example: http://localhost:7002/allNewInstances/java.lang.Object
The New Instances query is only available if you invoke the hat server with
two heap dumps. It is like the Instances query, but it only shows
"new" instances. An instance is considered new if it is in the
second heap dump, and there is no object of the same type with the same ID in
the baseline heap dump. An object's ID is a 32 bit integer that uniquely
identifies the
object, and it is assigned by the VM. In the VM implementations that were
current when HAT was written (JDK 1.1.x and betas of JDK 1.2), an object's
ID was the address of its handle, so it was pretty stable. Because
handle address could be re-used, the new instances query could give
inaccurate results. In more modern VM implementations, the object ID
could well be less table, and the results might be unusable.
With betas of JDK 1.2, comparing snapshots taken within a relatively
short interval, the results were pretty good. YMMV.
|
Summary
The hat server can be valuable for debugging and understanding programs.
It allows you to navigate object structures to learn how objects are
interconnected in a program. It also allows you to trace the references
to a given object from the rootset, which can be valuable for tracking
down unnecessary object retention. We found the hat server to be
a valuable debugging aid, and hope that you will, too.
For more information, please consult the HAT page at
http://hat.dev.java.net/.
|
MAIN PAGE |
SENDING COMMENTS |
DONATIONS |
HISTORY
