tag:blogger.com,1999:blog-38570071297151686622024-03-13T17:05:23.681+02:00Le Café Performant: A Blog On Java PerformanceThis blog contains a fine blend of Java and performance, seasoned with a light touch <br>of software architectureLifeyhttp://www.blogger.com/profile/10586718302147663938noreply@blogger.comBlogger50125tag:blogger.com,1999:blog-3857007129715168662.post-69693739211140049612014-05-23T10:00:00.002+03:002014-05-23T10:02:09.499+03:00MJStack -> MJProfTwo months ago I have started the MJStack project (<a href="http://blog.performize-it.com/2014/03/mjstack-monadic-jstack-analyzer.html">see previous blog post</a>). This project continues to evolve (I am getting help from anatmaor@github). As time passes I have realised at some point that this is turning into a sampling profiler so I have decided to rename the project into mjprof.<br />
<br />
So what is new ?<br />
<br />
First of all data source concept. MJStack collected its data from standard input only. MJProf can collect stack dumps via invocation of JStack, via JMX (including remote JMX), from standard input and directly from a file. Since mjprof is extensible via plugins you can add additional data sources.<br />
<br />
In addition merging of stack traces (into profiles) can be done over time based on thread id.<br />
<br />
This is a great leap forward. I will appreciate your feedback and contribution.<br />
Stay tuned and<br />
fork me on github <a href="https://www.blogger.com/%C2%A0https://github.com/lifey/mjprof"> https://github.com/lifey/mjprof</a><br />
<br />
<br />Lifeyhttp://www.blogger.com/profile/10586718302147663938noreply@blogger.com0tag:blogger.com,1999:blog-3857007129715168662.post-8005829091523163702014-03-01T15:10:00.002+02:002014-03-01T15:10:19.260+02:00MJStack- The Monadic JStack Analyzer <div style="text-align: justify;">
When you come to debug a JVM in production(or in a performance lab) one of the available techinques is to grab a thread dump using either a JMX or the jstack command line tool. Sometimes it is all you have, thats why it is called the 'poor man's profiler'. JDK (1.6 and up) includes the <b><i>jstack</i></b> command line tool. In order to use it you need to discover the relevant process id by using either ps (UNIX) / task manager (Windows) or by using <b><i>jps</i></b> which is another command line tool found in the jdk. Usually you will execute the following command:</div>
<div>
<i><span style="color: #274e13;"><br /></span></i></div>
<div>
<i><span style="color: #741b47;">jstack -l {your process i} > stacktrace.txt </span></i></div>
<div>
<br /></div>
<div style="text-align: justify;">
This will create a file with stack traces for all the thread in the process. If you run your JVM in UNIX you need to make sure that you are using the same user (or root). In Windows you will need to make sure you are running on the same session as well. </div>
<div style="text-align: justify;">
The <i>stacktrace.txt</i> contains a short header and a per thread section. For each thread we will have some information: Thread name, priority, thread id, native thread id, status, state, stacktrace and "locked ownable synchronizers" (java.util.concurrent ReentrantLocks which the thread is locking) </div>
<div>
<br /></div>
<div>
Following an example for a thread which is named ISCREAM3 and is currently sleeping.</div>
<div>
<div class="p1">
<br /></div>
<div class="p1">
<i><span style="color: #741b47;">"ISCREAM3" prio=5 tid=0x00007fd48a9b2800 nid=0x9203 waiting on condition [0x000000010ddec000]</span></i></div>
<div class="p1">
<i><span style="color: #741b47;"> java.lang.Thread.State: TIMED_WAITING (sleeping)</span></i></div>
<div class="p1">
<i><span style="color: #741b47;"> at java.lang.Thread.sleep(Native Method)</span></i></div>
<div class="p1">
<i><span style="color: #741b47;"> at com.performizeit.jpo.ex.cpuprofile.threadClasses.Sleeper.rest(Sleeper.java:44)</span></i></div>
<div class="p1">
<i><span style="color: #741b47;"> at com.performizeit.jpo.ex.cpuprofile.threadClasses.Sleeper.singleCycle(Sleeper.java:23)</span></i></div>
<div class="p1">
<i><span style="color: #741b47;"> at com.performizeit.jpo.ex.cpuprofile.threadClasses.ExceptionThrower.singleCycle(ExceptionThrower.java:19)</span></i></div>
<div class="p1">
<i><span style="color: #741b47;"> at com.performizeit.jpo.ex.cpuprofile.threadClasses.Sleeper.longerStackTrace(Sleeper.java:30)</span></i></div>
<div class="p1">
<i><span style="color: #741b47;"> at com.performizeit.jpo.ex.cpuprofile.threadClasses.Sleeper.longerStackTrace(Sleeper.java:32)</span></i></div>
<div class="p1">
<i><span style="color: #741b47;"> at com.performizeit.jpo.ex.cpuprofile.threadClasses.Sleeper.longerStackTrace(Sleeper.java:32)</span></i></div>
<div class="p1">
<i><span style="color: #741b47;"> at com.performizeit.jpo.ex.cpuprofile.threadClasses.Sleeper.longerStackTrace(Sleeper.java:32)</span></i></div>
<div class="p1">
<i><span style="color: #741b47;"> at com.performizeit.jpo.ex.cpuprofile.threadClasses.Sleeper.longerStackTrace(Sleeper.java:32)</span></i></div>
<div class="p1">
<i><span style="color: #741b47;"> at com.performizeit.jpo.ex.cpuprofile.threadClasses.Sleeper.longerStackTrace(Sleeper.java:32)</span></i></div>
<div class="p1">
<i><span style="color: #741b47;"> at com.performizeit.jpo.ex.cpuprofile.threadClasses.Sleeper.loopForever(Sleeper.java:37)</span></i></div>
<div class="p1">
<i><span style="color: #741b47;"> at com.performizeit.jpo.ex.cpuprofile.threadClasses.Sleeper.run(Sleeper.java:51)</span></i></div>
<div class="p1">
<i><span style="color: #741b47;"> at com.performizeit.jpo.ex.cpuprofile.threadClasses.Recurse.doRecursion(Recurse.java:25)</span></i></div>
<div class="p1">
<i><span style="color: #741b47;"> at com.performizeit.jpo.ex.cpuprofile.threadClasses.Recurse.doRecursion(Recurse.java:25)</span></i></div>
<div class="p1">
<i><span style="color: #741b47;"> at com.performizeit.jpo.ex.cpuprofile.threadClasses.Recurse.singleCycle(Recurse.java:16)</span></i></div>
<div class="p1">
<i><span style="color: #741b47;"> at com.performizeit.jpo.ex.cpuprofile.threadClasses.Sleeper.longerStackTrace(Sleeper.java:32)</span></i></div>
<div class="p1">
<i><span style="color: #741b47;"> at com.performizeit.jpo.ex.cpuprofile.threadClasses.Sleeper.longerStackTrace(Sleeper.java:32)</span></i></div>
<div class="p1">
<i><span style="color: #741b47;"> at com.performizeit.jpo.ex.cpuprofile.threadClasses.Sleeper.loopForever(Sleeper.java:37)</span></i></div>
<div class="p1">
<i><span style="color: #741b47;"> at com.performizeit.jpo.ex.cpuprofile.threadClasses.Sleeper.run(Sleeper.java:51)</span></i></div>
<div class="p1">
<i><span style="color: #741b47;"> at java.lang.Thread.run(Thread.java:744)</span></i></div>
<div class="p1">
<i><span style="color: #741b47;"><br /></span></i></div>
<div class="p1">
<i><span style="color: #741b47;"> Locked ownable synchronizers:</span></i></div>
<div class="p1">
</div>
<div class="p1">
<i><span style="color: #741b47;"> - None</span></i></div>
</div>
<div>
</div>
<div style="text-align: justify;">
This is really cool, you take a quick glance and you are able to see what the thread is doing. The problem starts when you come to analyze real applications. These application are huge and may have hundreds of such threads (and from time to time even thousands) with long stacktraces so you will probably need to read through a file with tens of thousands of lines. </div>
<div>
<br /></div>
<div style="text-align: justify;">
Thats where mjstack gets into the picture. It enables you to transform the thread dump into something that may be useful to you. It does not have a fancy UI, as a matter of fact it has no user interface at all. The way to work with it, is to compose several small steps that let you focus on what you really want. With this tool you are able to filter out threads, sort them, eliminate some of the data in them and group them by certain attribute.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Each such step will be called a monad and there three kind of such monads:</div>
<div style="text-align: justify;">
<ul>
<li>Filters - filters out certain threads </li>
<li>Mappers - Change thread info</li>
<li>Terminals - performs final transformation and subsequent monads can't be applied after it.</li>
</ul>
Sounds like project lambda ? Well that is not a coincidence. I was inspired by the concept of functional programming streams closures and lambda expressions. You can take those monads and compose them as you wish.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
The argument passed to mjstack are a list of monads seperated by period(.) each one of them contains a list of arguments inside two slashes (//) seperated by a comma (,).</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
For example the monad<i> contains/name,mythread/ </i> will filter out any thread which its name does not contain the string "mythread". The terminal <i>group/state/</i> will create a histogram of different states. We can also combine them together <i>contains/name,mythread/.group/state/</i> this will histogram by state only the threads with name that contains mythread</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<br /></div>
<div>
<br /></div>
<div>
Lets see some examples:</div>
<div>
<br /></div>
<div>
1. Focus on runnable threads:</div>
<div>
<i><span style="color: #741b47;"> cat stacktrace.txt | ./mjs.sh contains/state,RUNNABLE/</span></i></div>
<div>
<i><span style="color: #741b47;"><br /></span></i></div>
<div>
2. Show only the names of the runnable threads</div>
<div>
<span style="color: #741b47;"> <i>cat stacktrace.txt | ./mjs.sh contains/state,RUNNABLE/.group/name/</i></span></div>
<div>
<span style="color: #741b47;"><i><br /></i></span></div>
<div>
3. Focus on the threads which belong to your ISCREAM thread pool and group them by state:</div>
<div>
<i> <span style="color: #741b47;">cat stacktrace.txt | ./mjs.sh contains/name,ISCREAM/.group/state/ </span></i></div>
<div>
<i><span style="color: #741b47;"><br /></span></i></div>
<div>
4. Look for threads that actually run your application code</div>
<div>
<span style="color: #741b47;"><i> cat stacktrace.txt | ./mjs.sh contains/stack,myconpany/</i></span></div>
<div>
<span style="color: #741b47;"><i><br /></i></span></div>
<div>
<div>
4. Look for threads that actually run your application code and take only the 5 topmost stack-frames:</div>
<div>
<span style="color: #741b47;"><i> cat stacktrace.txt | ./mjs.sh contains/stack,myconpany/.keeptop/5/</i></span></div>
</div>
<div>
<span style="color: #741b47;"><i><br /></i></span></div>
<div>
And so much more ! You have great power in your hands since you can compose monads as you like.</div>
<div>
<br /></div>
<div>
If you reached to this point and still interested, don't hesitate the project is open source and you can download it from : </div>
<div>
<a href="https://github.com/lifey/mjstack">https://github.com/lifey/mjstack</a></div>
<div>
<br /></div>
<div>
Disclaimer: This is an initial version so it may contain bugs and it was not optimized yet. I would love to get feedback.</div>
<div>
<br /></div>
<div>
Enjoy !</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
Lifeyhttp://www.blogger.com/profile/10586718302147663938noreply@blogger.com0tag:blogger.com,1999:blog-3857007129715168662.post-46076331934890657622013-09-24T14:58:00.000+03:002014-03-02T15:05:31.494+02:00JVM Params Everyone Should Have in Production <div style="text-align: justify;">
I know I did not published anything useful in last months. As a matter of fact, I had this blog post almost ready for quite a long time. Unfortunately, I have a lot to say on procrastination but I will leave it for tomorrow:) Anyway after the JVM hotspot sessions in JavaOne and after a direct question in JClarity friends newsletter and after suffering from severe jet lag I have decided to finalise this post. </div>
<div style="text-align: justify;">
<br /></div>
<br />
<div style="text-align: justify;">
When talking about JVM configuration I find myself breaking the DRY principle so I it about time to write a blog post on this topic. Following list of a the must have JVM params for (almost) every server installation:<br />
<br />
(parameterised values are specified in curly braces)</div>
<br />
<span style="color: #3d85c6;">-Xms{#MB}m -Xmx{#MB}m</span><br />
<span style="color: #3d85c6;">-XX:PermSize={#MB}m -XX:MaxPermSize={#MB}m</span><br />
<span style="color: #3d85c6;">-XX:+HeapDumpOnOutOfMemoryError </span><br />
-XX:+PrintFlagsFinal<br />
-server<br />
<span style="color: #6aa84f;">-XX:+PrintGCDetails <br />-XX:+PrintGCDateStamps </span><br />
<div style="background-color: white;">
<span style="color: #6aa84f; font-family: Times, Times New Roman, serif;">-XX:+PrintTenuringDistribution</span></div>
<div style="background-color: white;">
<span style="color: #6aa84f; font-family: Times, Times New Roman, serif;">-XX:+<wbr></wbr>PrintGCApplicationStoppedTime</span></div>
<div style="background-color: white;">
<span style="color: #6aa84f; font-family: Times, Times New Roman, serif;">-XX:+<wbr></wbr>PrintGCApplicationConcurrentTi<wbr></wbr>me</span></div>
<span style="color: #6aa84f;"> -XX:+UseGCLogFileRotation</span><br />
<span style="color: #6aa84f;">-XX:NumberOfGCLogFiles={#files}</span><br />
<span style="color: #6aa84f;">-XX:GCLogFileSize={#MB}M<br />-Xloggc:{some gc log file}.gc </span><br />
<span style="color: #a64d79;">-Dcom.sun.management.jmxremote<br />-Dcom.sun.management.jmxremote.port={a port}<br />-Dcom.sun.management.jmxremote.ssl=false<br />-Dcom.sun.management.jmxremote.authenticate=false</span><br />
<span style="color: #a64d79;"><br /></span>
<span style="color: #a64d79;"><br /></span>
<br />
<div style="text-align: justify;">
<span style="color: #3d85c6;">Memory params (blue)</span>: You probably need to set minimal and maximal heap size to suite your needs. In most cases ms=mx but that is not always the case. IMO it is really important to have a memory dump whenever you encounter an OutOfMemoryError mainly if you want to be able to isolate fast "spike" memory depletion bugs. </div>
<div>
<br /></div>
<div>
<div style="text-align: justify;">
-XX:+PrintFlagsFinal - Displays the configuration of the JVM all parameters available with their default value or the overridden one.<br />
-server is not really needed it is just a safety net to make sure you are not running a client compiler or something stupid like that :) </div>
</div>
<div>
<br /></div>
<div>
<div style="text-align: justify;">
<span style="color: #6aa84f;">GC logging configuration(green)</span>: If you want to be able to estimate GC efficiency GC logging is the way. Starting from J7 log file rotation is supported use it to keep at least two-four weeks of GC activity.</div>
</div>
<div>
<br /></div>
<div>
<div style="text-align: justify;">
<span style="color: #a64d79;">Remote JMX configuration(purple)</span> : Opening a JMX port for remote monitoring can help you debug and monitor your server with tools like VisualVM and JConsole. If security is an issue you may also require password and encrypt communications. </div>
<br />
A few more notes:<br />
<div style="text-align: justify;">
1. I am not getting into GC tuning configuration params as these params vary from one application to another and tuning GC is not in the scope of this post. However if you are using CMS I think that <br />
-XX:+ExplicitGCInvokesConcurrent should be your default unless proven otherwise.</div>
2. J8 has the new MetaSpace instead of the PermGen space so perm gen params are no longer required.<br />
<br />
What is your common set of params ?</div>
<div>
<br /></div>
Lifeyhttp://www.blogger.com/profile/10586718302147663938noreply@blogger.com6tag:blogger.com,1999:blog-3857007129715168662.post-90415470809972155502013-07-16T09:22:00.002+03:002016-01-13T09:26:52.623+02:00 Scalapeño 2013Tomorrow I am going to give the session "Tales about Scala Performance" in <a href="http://scalapeno.underscore.co.il/" target="_blank">Scalapeño 2013</a> conference in Tel Aviv see ya there :)<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/--bHMjOBAHf8/UeTmhctZNgI/AAAAAAAAjdw/PFIa5lxYp8U/s1600/bartpremature.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="285" src="http://1.bp.blogspot.com/--bHMjOBAHf8/UeTmhctZNgI/AAAAAAAAjdw/PFIa5lxYp8U/s400/bartpremature.jpg" width="400" /></a></div>
<br />
<br />
<br />Lifeyhttp://www.blogger.com/profile/10586718302147663938noreply@blogger.com0tag:blogger.com,1999:blog-3857007129715168662.post-19692780221906626382011-12-17T08:56:00.003+02:002011-12-18T09:34:13.094+02:00A tale about a JVM crash<span class="Apple-style-span" style="border-collapse: collapse; color: #222222; font-family: arial,sans-serif; font-size: 13px;"></span><br />
<div>It happened again :) </div><div>After a long time it happened again....</div><span class="Apple-style-span" style="border-collapse: collapse; color: #222222; font-family: arial,sans-serif;">I was called to investigate a JVM crash. Usually it is the sign of a long adventure along side with the Java platform. A</span><span class="Apple-style-span" style="color: #222222; font-family: arial,sans-serif;"><span class="Apple-style-span" style="border-collapse: collapse;"> thrilling and frustrating</span></span><span class="Apple-style-span" style="color: #222222; font-family: arial,sans-serif;"><span class="Apple-style-span" style="border-collapse: collapse;"> event in the same time. This time I was lucky it took me 4 hours to reach a valid solution ....</span></span><br />
<span class="Apple-style-span" style="color: #222222; font-family: arial,sans-serif; font-size: x-small;"><span class="Apple-style-span" style="border-collapse: collapse;"><br />
</span></span><br />
<span class="Apple-style-span" style="font-family: arial,sans-serif;"><span class="Apple-style-span" style="border-collapse: collapse;"><span class="Apple-style-span" style="color: #741b47; font-size: large;">Chapter I : First Encounter</span></span></span><br />
<div><span class="Apple-style-span" style="font-family: arial,sans-serif; font-size: x-small;"><span class="Apple-style-span" style="border-collapse: collapse;"><span class="Apple-style-span" style="color: #222222;"><br />
</span></span></span></div><div><span class="Apple-style-span" style="font-family: arial,sans-serif;"><span class="Apple-style-span" style="border-collapse: collapse;"><span class="Apple-style-span" style="color: #222222;">First of all I looked at the directory from which the process started running. I started looking at the </span><span class="Apple-style-span" style="color: #38761d;">hs_err_pidXXXXX.log </span><span class="Apple-style-span" style="color: #222222;">files that existed there about... 50 such files as there is a watchdog process which relaunch the application immediately after the crash. In this case just for the sake of crashing again.</span></span></span></div><div><span class="Apple-style-span" style="color: #222222; font-family: arial,sans-serif;"><span class="Apple-style-span" style="border-collapse: collapse;">I was happy to discover they accumulate very fast the process needs merely 45 seconds to crash. Someone who is not familiar with those kind of problems </span></span><span class="Apple-style-span" style="border-collapse: collapse; color: #222222; font-family: arial,sans-serif;">may be startled by fact that the process crashes so fast but for me that was heaven! 45 seconds to reproduction.</span></div><div style="border-collapse: collapse; color: #222222; font-family: arial, sans-serif;">I started looking at the crash files. They were very disappointing, each one them crashed on different address the thread was applicative. In addition stack trace was missing . I snooped around to make sure no native code was running on this process(no native code good news). </div><div style="border-collapse: collapse; color: #222222; font-family: arial, sans-serif;">All I had was a pale description of a JVM frustration: </div><div style="border-collapse: collapse; font-family: arial, sans-serif;"><span class="Apple-style-span" style="color: #274e13;">#</span></div><div style="border-collapse: collapse; font-family: arial, sans-serif;"><span class="Apple-style-span" style="color: #274e13;"># A fatal error has been detected by the Java Runtime Environment:</span></div><div style="border-collapse: collapse; font-family: arial, sans-serif;"><span class="Apple-style-span" style="color: #274e13;">#</span></div><div style="border-collapse: collapse; font-family: arial, sans-serif;"><span class="Apple-style-span" style="color: #274e13;"># EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000000000000000, pid=4384, tid=8364</span></div><div style="border-collapse: collapse; font-family: arial, sans-serif;"><span class="Apple-style-span" style="color: #274e13;">#</span></div><div style="border-collapse: collapse; font-family: arial, sans-serif;"><span class="Apple-style-span" style="color: #274e13;"># JRE version: 6.0_26-b03</span></div><div style="border-collapse: collapse; font-family: arial, sans-serif;"><span class="Apple-style-span" style="color: #274e13;"># Java VM: Java HotSpot(TM) 64-Bit Server VM (20.1-b02 mixed mode windows-amd64 compressed oops)</span></div><div style="border-collapse: collapse; font-family: arial, sans-serif;"><span class="Apple-style-span" style="color: #274e13;"># Problematic frame:</span></div><div style="border-collapse: collapse; font-family: arial, sans-serif;"><span class="Apple-style-span" style="color: #274e13;"># C 0x0000000000000000</span></div><div style="border-collapse: collapse; font-family: arial, sans-serif;"><span class="Apple-style-span" style="color: #274e13;">#</span></div><div style="border-collapse: collapse; font-family: arial, sans-serif;"><span class="Apple-style-span" style="color: #274e13;"># If you would like to submit a bug report, please visit:</span></div><div style="border-collapse: collapse; font-family: arial, sans-serif;"><span class="Apple-style-span" style="color: #274e13;"># <a href="http://java.sun.com/webapps/bugreport/crash.jsp" target="_blank">http://java.sun.com/webapps/<wbr></wbr>bugreport/crash.jsp</a></span></div><div style="border-collapse: collapse; font-family: arial, sans-serif;"><span class="Apple-style-span" style="color: #274e13;">#</span></div><div style="border-collapse: collapse; font-family: arial, sans-serif;"><span class="Apple-style-span" style="color: #274e13;"><br />
</span></div><div style="border-collapse: collapse; font-family: arial, sans-serif;"><span class="Apple-style-span" style="color: #274e13;">--------------- T H R E A D ---------------</span></div><div style="border-collapse: collapse; font-family: arial, sans-serif;"><span class="Apple-style-span" style="color: #274e13;"><br />
</span></div><div style="border-collapse: collapse; font-family: arial, sans-serif;"><span class="Apple-style-span" style="color: #274e13;">Current thread (0x000000001029e000): JavaThread "XXXXXXXXXXXXX" daemon [_thread_in_Java, id=8364, stack(0x000000001a840000,<wbr></wbr>0x000000001a940000)]</span></div><div style="border-collapse: collapse; font-family: arial, sans-serif;"><span class="Apple-style-span" style="color: #274e13;"><br />
</span></div><div style="border-collapse: collapse; font-family: arial, sans-serif;"><span class="Apple-style-span" style="color: #274e13;">siginfo: ExceptionCode=0xc0000005, ExceptionInformation=<wbr></wbr>0x0000000000000008 0x0000000000000000</span></div><div style="border-collapse: collapse; color: #222222; font-family: arial, sans-serif;"><br />
</div><div style="border-collapse: collapse; color: #222222; font-family: arial, sans-serif;">The fact that crashing thread was an applicative thread was indicating that the crash was not related to GC.I tried switching the GC mechanism anyhow with no change in behavior. I also removed compressedOops and crash remain there standing still.</div><div><span class="Apple-style-span" style="border-collapse: collapse; color: #222222; font-family: arial,sans-serif;">My next guess was the JIT compiler so as a first step I added the following command line option </span><span class="Apple-style-span" style="border-collapse: collapse; color: #38761d; font-family: arial,sans-serif;">-Djava.compiler=NONE</span><span class="Apple-style-span" style="font-family: arial,sans-serif;"><span class="Apple-style-span" style="border-collapse: collapse;">. That guess was working like a charm JVM stopped crashing. That was still not a sign for euphoria, running interpreted mode is not a very good choice so the next step included divide and conquer.</span></span></div><div><span class="Apple-style-span" style="font-family: arial,sans-serif; font-size: x-small;"><span class="Apple-style-span" style="border-collapse: collapse;"><br />
</span></span></div><div></div><div style="border-collapse: collapse; font-family: arial, sans-serif;"><span class="Apple-style-span" style="color: #741b47; font-size: large;">Chapter II: Lion hunting</span></div><div style="border-collapse: collapse; color: #222222; font-family: arial, sans-serif; font-size: 13px;"><br />
</div><div><span class="Apple-style-span" style="font-family: arial,sans-serif;"><span class="Apple-style-span" style="border-collapse: collapse;">I added the following flags to the JVM </span></span><span class="Apple-style-span" style="border-collapse: collapse; font-family: arial,sans-serif;"><span class="Apple-style-span" style="color: #38761d;">-XX:+LogCompilation -XX:+<wbr></wbr>UnlockDiagnosticsVMOptions</span></span></div><div style="border-collapse: collapse; font-family: arial, sans-serif;"><span class="Apple-style-span" style="color: #222222;">When a HotSpot JVM is running with log compilation flag it is creating </span><span class="Apple-style-span" style="color: #38761d;">hotspot.log</span><span class="Apple-style-span" style="color: #222222;"> file which includes detailed info regarding all of its JIT activity. For a detailed document regarding this log file take a look </span><a href="http://wikis.sun.com/display/HotSpotInternals/LogCompilation+overview" style="color: #222222;" target="_blank">here</a><span class="Apple-style-span" style="color: #222222;">.</span></div><div style="border-collapse: collapse; color: #222222; font-family: arial, sans-serif;">Truth be told I still don't fully understand the generated XML but I was looking for the following lines :<br />
<span style="color: #38761d;"><task_queued backedge_count="5000" bytes="452" comment="count" compile_id="1" count="2" hot_count="2" iicount="2" method="java/util/Properties$LineReader readLine ()I" stamp="0.094"></task_queued></span><br />
</div><div style="border-collapse: collapse; color: #222222; font-family: arial, sans-serif;"><br />
</div><div><span class="Apple-style-span" style="color: #38761d; font-family: arial,sans-serif;"><span class="Apple-style-span" style="border-collapse: collapse;"><task_queued backedge_count="5000" bytes="452" comment="count" compile_id="1" count="2" hot_count="2" iicount="2" method="java/util/Properties$LineReader readLine ()I" stamp="0.094"></task_queued></span></span></div><div style="border-collapse: collapse; color: #222222; font-family: arial, sans-serif;"></div><div><span class="Apple-style-span" style="font-family: arial,sans-serif;"><span class="Apple-style-span" style="border-collapse: collapse;"><span class="Apple-style-span" style="color: #222222;">Compilation of a method is performed asynchronously by specific JVM threads and every request for JIT is queued before compilation. Grabbing all these lines will give me a full list of all compilation attempts done by the JVM. This list now was taken to a text editor and manipulated so it can be fed to the </span><span class="Apple-style-span" style="color: #38761d;">.hotspot_compiler </span>file.</span></span></div><div><span class="Apple-style-span" style="font-family: arial,sans-serif;"><span class="Apple-style-span" style="border-collapse: collapse;"><br />
</span></span></div><div><span class="Apple-style-span" style="font-family: arial,sans-serif;"><span class="Apple-style-span" style="border-collapse: collapse;"><span style="color: #38761d;">.hotspot_compiler</span> is a JIT exclusion file which should be placed in the working directory from which the JVM is launching. It contains lines of the following format:</span></span></div><div style="border-collapse: collapse; color: #222222; font-family: arial, sans-serif;"><span class="Apple-style-span" style="color: #38761d;">exclude</span><span class="Apple-style-span" style="color: #222222;"> <span class="Apple-style-span" style="color: #38761d;">java/util/Properties$LineReader readLine ()I</span></span></div><div style="border-collapse: collapse; color: #222222; font-family: arial, sans-serif;"><br />
</div><div style="border-collapse: collapse; color: #222222; font-family: arial, sans-serif;">Telling the JVM|: </div><div style="border-collapse: collapse; color: #222222; font-family: arial, sans-serif;">"Dear JVM </div><div style="border-collapse: collapse; color: #222222; font-family: arial, sans-serif;"> Do not compile the method realLine which resides in the inner class LineReader of the class java.util.Properties, Thanks in advance,</div><div style="border-collapse: collapse; color: #222222; font-family: arial, sans-serif;"> A desperate programmer"</div><div style="border-collapse: collapse; color: #222222; font-family: arial, sans-serif;"><br />
</div><div style="border-collapse: collapse; font-family: arial, sans-serif;"><span class="Apple-style-span" style="color: #222222;">I started with approx. 2000 such lines and fed them to the </span><span class="Apple-style-span" style="color: #38761d;">.hotspot_compiler</span><span class="Apple-style-span" style="color: #222222;"> file (after removing the compile=NONE flag of course ) and the JVM did not crash. After the divide and conquer process I was left with a single method which its compilation made the JVM freak out.</span></div><div style="border-collapse: collapse; color: #222222; font-family: arial, sans-serif;">Everybody is happy crash eliminated with minimal effect on performance.</div><div style="border-collapse: collapse; font-family: arial, sans-serif;"><span class="Apple-style-span" style="color: #741b47; font-size: large;"><br />
</span></div><div style="border-collapse: collapse; font-family: arial, sans-serif;"><span class="Apple-style-span" style="color: #741b47; font-size: large;">Chapter III: After-max</span></div><div><div style="border-collapse: collapse; color: #222222; font-family: arial, sans-serif; font-size: 13px;"><br />
</div><div style="border-collapse: collapse; color: #222222; font-family: arial, sans-serif;">Everybody is happy, except of me.... a lot of questions remain unanswered.</div><div><span class="Apple-style-span" style="color: #222222; font-family: arial,sans-serif;"><span class="Apple-style-span" style="border-collapse: collapse;">The method which aggravated the crash looks like this: </span></span></div><div class="p1" style="border-collapse: collapse; color: black; font-family: arial, sans-serif; font: normal normal normal 15px/normal Calibri; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><span class="Apple-style-span" style="color: #6aa84f; font-size: small;">protected boolean isXXXXXXXXX(double stdDev,double mean){</span></div><div class="p1" style="border-collapse: collapse; color: black; font-family: arial, sans-serif; font: normal normal normal 15px/normal Calibri; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><span class="Apple-style-span" style="color: #6aa84f; font-size: small;"> //|stddev/mean| > 10^-5</span></div><div class="p1" style="border-collapse: collapse; color: black; font-family: arial, sans-serif; font: normal normal normal 15px/normal Calibri; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><span class="Apple-style-span" style="color: #6aa84f; font-size: small;"> if(stdDev==0.0){</span></div><div class="p1" style="border-collapse: collapse; color: black; font-family: arial, sans-serif; font: normal normal normal 15px/normal Calibri; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><span class="Apple-style-span" style="color: #6aa84f; font-size: small;"> return mean==0.0;</span></div><div class="p1" style="border-collapse: collapse; color: black; font-family: arial, sans-serif; font: normal normal normal 15px/normal Calibri; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><span class="Apple-style-span" style="color: #6aa84f; font-size: small;"> }</span></div><div class="p1" style="border-collapse: collapse; color: black; font-family: arial, sans-serif; font: normal normal normal 15px/normal Calibri; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><span class="Apple-style-span" style="color: #6aa84f; font-size: small;"> if(Double.isNaN(stdDev) || Double.isNaN(mean)){</span></div><div class="p1" style="border-collapse: collapse; color: black; font-family: arial, sans-serif; font: normal normal normal 15px/normal Calibri; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><span class="Apple-style-span" style="color: #6aa84f; font-size: small;"> return true;</span></div><div class="p1" style="border-collapse: collapse; color: black; font-family: arial, sans-serif; font: normal normal normal 15px/normal Calibri; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><span class="Apple-style-span" style="color: #6aa84f; font-size: small;"> }</span></div><div class="p1" style="border-collapse: collapse; color: black; font-family: arial, sans-serif; font: normal normal normal 15px/normal Calibri; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><span class="Apple-style-span" style="color: #6aa84f; font-size: small;"> return Math.abs(mean/stdDev) < 100000.0;</span></div><div style="border-collapse: collapse; color: #222222; font-family: arial, sans-serif;"><span class="Apple-style-span" style="color: #6aa84f; font-family: Calibri;"> }</span> </div><div style="border-collapse: collapse; color: #222222; font-family: arial, sans-serif;">What? That's a method which can cause the JVM to crash ? I don't think so... </div><div style="border-collapse: collapse; color: #222222; font-family: arial, sans-serif;">My guess was that it is related to some kind of interaction.....</div><div style="border-collapse: collapse; color: #222222; font-family: arial, sans-serif;">I tried to reproduce the crash on an isolated test and as expected, nothing.</div><div style="border-collapse: collapse; color: #222222; font-family: arial, sans-serif;">Could not find anything in the internet since I don't even know how to ask the correct question.</div></div><div style="border-collapse: collapse; color: #222222; font-family: arial, sans-serif;"><br />
</div><div style="border-collapse: collapse; color: #222222; font-family: arial, sans-serif;">What am I going to do next ?:</div><div><ul><li><span class="Apple-style-span" style="color: #222222; font-family: arial,sans-serif;"><span class="Apple-style-span" style="border-collapse: collapse;">Try to upgrade the JDK sub version though I could not find a fixed bug related to this in the release notes.</span></span></li>
<li><span class="Apple-style-span" style="color: #222222; font-family: arial,sans-serif;"><span class="Apple-style-span" style="border-collapse: collapse;">Try JDK 1.7</span></span></li>
<li><span class="Apple-style-span" style="color: #222222; font-family: arial,sans-serif;"><span class="Apple-style-span" style="border-collapse: collapse;">Try 32 bit JVM </span></span></li>
<li><span class="Apple-style-span" style="color: #222222; font-family: arial,sans-serif;"><span class="Apple-style-span" style="border-collapse: collapse;">Try to disable certain optimizations such as Escape analysis </span></span></li>
<li><span style="font-size: small;"><span class="Apple-style-span" style="color: #333333; display: inline ! important; float: none; font-family: 'Lucida Console',Monaco,'DejaVu Sans Mono',monospace; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px;">Use -XX:+PrintOptoAssembly</span></span><span class="Apple-style-span" style="color: #222222; font-family: arial,sans-serif;"><span class="Apple-style-span" style="border-collapse: collapse;"><span style="font-size: small;"> </span></span></span></li>
</ul><span class="Apple-style-span" style="color: #222222; font-family: arial,sans-serif;"><span class="Apple-style-span" style="border-collapse: collapse;">But the most important of all hoping for some input from the community......</span></span></div><div><span class="Apple-style-span" style="color: #222222; font-family: arial,sans-serif; font-size: x-small;"><span class="Apple-style-span" style="border-collapse: collapse;"><br />
</span></span></div><div class="yj6qo ajU" style="border-collapse: collapse; color: #222222; cursor: pointer; font-family: arial, sans-serif; font-size: 13px; outline-color: initial; outline-style: none; outline-width: initial; padding-bottom: 10px; padding-left: 0px; padding-right: 0px; padding-top: 10px; width: 22px;"></div>Lifeyhttp://www.blogger.com/profile/10586718302147663938noreply@blogger.com2tag:blogger.com,1999:blog-3857007129715168662.post-2624338387272956152011-11-14T18:55:00.000+02:002011-11-14T18:55:19.029+02:00Java 7 and String.intern()For a long time interning strings in HotSpot-VM would store the result in permanent generation. Since Java 7 this is no longer true. Interned strings are stored in the heap. As a result if you have an application where relied on intern() you may need to re-size your perm and your heap. Heap need to be larger Perm need to be smaller.<br />
I understood that permanent generation is going to be removed from the HotSpot-VM in futures releases of Java.<br />
Maybe this is the first step in this direction :)<br />
Reference<br />
<a href="http://www.oracle.com/technetwork/java/javase/jdk7-relnotes-418459.html">http://www.oracle.com/technetwork/java/javase/jdk7-relnotes-418459.html</a>Lifeyhttp://www.blogger.com/profile/10586718302147663938noreply@blogger.com0tag:blogger.com,1999:blog-3857007129715168662.post-76372971680377620562011-04-27T10:53:00.003+03:002011-04-27T20:09:25.998+03:00Java 6u25 releasedTwo new operations in ThreadingMBean<br />
- getThreadCPUTime - over threadID array<br />
- <span class="Apple-style-span" style="font-family: arial, helvetica, sans-serif; font-size: 12px;">getThreadAllocateBytes - allocated bytes on thread basis </span><br />
<br />
<a href="http://www.oracle.com/technetwork/java/javase/6u25releasenotes-356444.html">http://www.oracle.com/technetwork/java/javase/6u25releasenotes-356444.html</a><br />
<br />
- Tiered compilation is in...<br />
<br />
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif; font-size: 11px;"></span><br />
<div style="font-size: 12px; line-height: 16px; list-style-image: initial; list-style-position: initial; list-style-type: none; margin-bottom: 17px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;">HotSpot diagnostic information has been expanded in several ways:</div><ul style="color: black; font-family: arial, helvetica, sans-serif; font-size: 12px; list-style-image: initial; list-style-position: initial; list-style-type: none; margin-bottom: 10px; margin-left: 0px; margin-right: 0px; margin-top: 10px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;"><li style="background-attachment: initial; background-clip: initial; background-color: initial; background-image: url(http://www.oracleimg.com/us/assets/bullet1.gif); background-origin: initial; background-position: 0px 5px; background-repeat: no-repeat no-repeat; color: black; font-family: arial, helvetica, sans-serif; font-size: 12px; list-style-image: initial; list-style-position: initial; list-style-type: none; margin-bottom: 10px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 10px; padding-right: 0px; padding-top: 0px;">Tracking of cumulative Java heap bytes allocated on a per-thread basis (Threading mbean was added an operation getThreadAllocateBytes</li>
<li style="background-attachment: initial; background-clip: initial; background-color: initial; background-image: url(http://www.oracleimg.com/us/assets/bullet1.gif); background-origin: initial; background-position: 0px 5px; background-repeat: no-repeat no-repeat; color: black; font-family: arial, helvetica, sans-serif; font-size: 12px; list-style-image: initial; list-style-position: initial; list-style-type: none; margin-bottom: 10px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 10px; padding-right: 0px; padding-top: 0px;"><br />
</li>
<li style="background-attachment: initial; background-clip: initial; background-color: initial; background-image: url(http://www.oracleimg.com/us/assets/bullet1.gif); background-origin: initial; background-position: 0px 5px; background-repeat: no-repeat no-repeat; color: black; font-family: arial, helvetica, sans-serif; font-size: 12px; list-style-image: initial; list-style-position: initial; list-style-type: none; margin-bottom: 10px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 10px; padding-right: 0px; padding-top: 0px;">On <code style="color: #666666; font-family: courier, 'courier new', monaco; font-size: 12px; list-style-image: initial; list-style-position: initial; list-style-type: none; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;">OutOfMemoryError</code>, indication of the faulting thread in the heap dump</li>
<li style="background-attachment: initial; background-clip: initial; background-color: initial; background-image: url(http://www.oracleimg.com/us/assets/bullet1.gif); background-origin: initial; background-position: 0px 5px; background-repeat: no-repeat no-repeat; color: black; font-family: arial, helvetica, sans-serif; font-size: 12px; list-style-image: initial; list-style-position: initial; list-style-type: none; margin-bottom: 10px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 10px; padding-right: 0px; padding-top: 0px;">Improved handling of unexpected exceptions in application native code</li>
<li style="background-attachment: initial; background-clip: initial; background-color: initial; background-image: url(http://www.oracleimg.com/us/assets/bullet1.gif); background-origin: initial; background-position: 0px 5px; background-repeat: no-repeat no-repeat; color: black; font-family: arial, helvetica, sans-serif; font-size: 12px; list-style-image: initial; list-style-position: initial; list-style-type: none; margin-bottom: 10px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 10px; padding-right: 0px; padding-top: 0px;">Better indication of native heap exhaustion</li>
<li style="background-attachment: initial; background-clip: initial; background-color: initial; background-image: url(http://www.oracleimg.com/us/assets/bullet1.gif); background-origin: initial; background-position: 0px 5px; background-repeat: no-repeat no-repeat; color: black; font-family: arial, helvetica, sans-serif; font-size: 12px; list-style-image: initial; list-style-position: initial; list-style-type: none; margin-bottom: 10px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 10px; padding-right: 0px; padding-top: 0px;">More details in hs_err files</li>
</ul>Lifeyhttp://www.blogger.com/profile/10586718302147663938noreply@blogger.com0tag:blogger.com,1999:blog-3857007129715168662.post-81190003350773496192011-04-06T20:45:00.001+03:002011-04-06T20:45:16.991+03:00On JDK 7 G1 enables compressed Oops by defaultDidn't understand why CompressedOops are not enabled by default....<br />
Take a look at this<br />
<a href="http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2011-February/002312.html">http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2011-February/002312.html</a>Lifeyhttp://www.blogger.com/profile/10586718302147663938noreply@blogger.com0tag:blogger.com,1999:blog-3857007129715168662.post-85099440180184850792011-03-22T13:56:00.000+02:002011-03-22T13:56:14.974+02:00Thread TopHi,<br />
I have finally found time to write a small tool which can help profiling Java applications.<br />
Its name is threadTop and it connects to your application via JMX.<br />
It can measure<br />
<br />
<ul><li> thread contention </li>
<li> thread CPU usage</li>
</ul><div>I think it can be very useful when combined with jstack.</div><div>Waiting for your feedback :</div><div><br />
</div><div><a href="http://www.performize-it.com/tools/thread-top">http://www.performize-it.com/tools/thread-top</a></div>Lifeyhttp://www.blogger.com/profile/10586718302147663938noreply@blogger.com4tag:blogger.com,1999:blog-3857007129715168662.post-88718748717474247002011-03-15T19:31:00.000+02:002011-03-15T19:31:47.230+02:00Controlling GC logs at runtimeGC logging can be controlled using the following command line options:<br />
<ul><li><span class="Apple-style-span" style="color: #38761d;">-XX:+PrintGC</span> (same as -verbose:gc) - Displays basic GC details </li>
<li><span class="Apple-style-span" style="color: #38761d;">-XX:+PrintGCDetail</span>s - Displays advanced GC details </li>
<li><span class="Apple-style-span" style="color: #38761d;">-XX:+PrintGCTimeStamps</span> - Displays the time stamp for every GC event reported</li>
<li><span class="Apple-style-span" style="color: #38761d;">-XX:+PrintGCDateStamps</span> - Displays the date for every GC event reported </li>
<li><span class="Apple-style-span" style="color: #38761d;">-Xloggc:c:\gc.log</span> - Writes GC logging to the specified file.</li>
</ul>It is usually recommended to run with <span class="Apple-style-span" style="color: #38761d;">-XX:+PrintGC -XX:+PrintGCTimeStamps -Xloggc:c:\gc.log</span> even in production. The amount of data written to those files is not huge and it can help to analyze GC throughput ,pause time and even to expose bugs in the GC mechanism.<br />
On the other hand these files are not managed by a "log4j style" rolling appender and will not be trimmed by default. So what would we choose ?<br />
The good news is that logging GC can be configured at runtime. It can be enabled and disabled at will via JMX.<br />
The JMX object <span class="Apple-style-span" style="color: #38761d;">com.sun.management:type=HotSpotDiagnostic</span> has an attribute named <span class="Apple-style-span" style="color: #38761d;">DiagnosticOptions</span>. This attribute contains 12 boolean sub-attributes (as for Java 1.6.0_24). All of them are very interesting but for the GC logging we need to take a look on 4 of them:<br />
<ul><li><span class="Apple-style-span" style="color: #38761d;">PrintGC</span></li>
<li><span class="Apple-style-span" style="color: #38761d;">PrintGCDetails</span></li>
<li><span class="Apple-style-span" style="color: #38761d;">PrintGCTimeStamps</span></li>
<li><span class="Apple-style-span" style="color: #38761d;">PrintGCDateStamps</span></li>
</ul><br />
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">The bad news is that <span class="Apple-style-span" style="color: #38761d;">-Xloggc</span> cannot be configured via JMX so you will need to configure it on startup.</div><div><br />
</div><br />
All these can be modified using jconsole or VisualVM using the operation <span class="Apple-style-span" style="color: #38761d;">setVMOption</span>.<br />
<br />
My recommendation is that even if you do not want to log GC by default add the following params to your command line <span class="Apple-style-span" style="color: #38761d;">-Xloggc:c:\gc.log </span><span class="Apple-style-span" style="color: #38761d;">-XX:-PrintGC -XX:+PrintGCTimeStamps </span><br />
(Notice that the PrintGC has a - sign before it)<br />
When you will need to analyze GC just turn on PrintGC and possible the PrintGCDetails.<br />
<br />
<div><br />
</div>Lifeyhttp://www.blogger.com/profile/10586718302147663938noreply@blogger.com0tag:blogger.com,1999:blog-3857007129715168662.post-36019284453653852872011-03-03T11:50:00.001+02:002011-03-03T11:54:39.764+02:00-XX:PrintFLSStatistics=1When trying to understand what is going on with GC logging on G1GC. I found an undocumented JVM param that can really help when using CMS. <span class="Apple-style-span" style="color: #505050; font-family: monospace; font-size: 14px; line-height: 20px;">-XX:PrintFLSStatistics=1 </span>will report extra data for every GC:<br />
<br />
<i><span class="Apple-style-span" style="color: #274e13; font-size: x-small;">8.790: [ParNew: 72703K->6341K(76672K), 0.0067245 secs] 72703K->6341K(816832K)After GC:</span></i><br />
<i><span class="Apple-style-span" style="color: #274e13; font-size: x-small;">Statistics for BinaryTreeDictionary:</span></i><br />
<i><span class="Apple-style-span" style="color: #274e13; font-size: x-small;">------------------------------------</span></i><br />
<i><span class="Apple-style-span" style="color: #274e13; font-size: x-small;">Total Free Space: 94724096</span></i><br />
<i><span class="Apple-style-span" style="color: #274e13; font-size: x-small;">Max Chunk Size: 94724096</span></i><br />
<i><span class="Apple-style-span" style="color: #274e13; font-size: x-small;">Number of Blocks: 1</span></i><br />
<i><span class="Apple-style-span" style="color: #274e13; font-size: x-small;">Av. Block Size: 94724096</span></i><br />
<i><span class="Apple-style-span" style="color: #274e13; font-size: x-small;">Tree Height: 1</span></i><br />
<i><span class="Apple-style-span" style="color: #274e13; font-size: x-small;">After GC:</span></i><br />
<i><span class="Apple-style-span" style="color: #274e13; font-size: x-small;">Statistics for BinaryTreeDictionary:</span></i><br />
<i><span class="Apple-style-span" style="color: #274e13; font-size: x-small;">------------------------------------</span></i><br />
<i><span class="Apple-style-span" style="color: #274e13; font-size: x-small;">Total Free Space: 0</span></i><br />
<i><span class="Apple-style-span" style="color: #274e13; font-size: x-small;">Max Chunk Size: 0</span></i><br />
<i><span class="Apple-style-span" style="color: #274e13; font-size: x-small;">Number of Blocks: 0</span></i><br />
<i><span class="Apple-style-span" style="color: #274e13; font-size: x-small;">Tree Height: 0</span></i><br />
<i><span class="Apple-style-span" style="color: #274e13; font-size: x-small;">, 0.0068238 secs] [Times: user=0.06 sys=0.00, real=0.01 secs] </span></i><br />
<br />
This can really help when trying to understand CMS concurrent mode failure due to fragmentation<br />
<br />
Pity it is not reporting data in a single line though :-)Lifeyhttp://www.blogger.com/profile/10586718302147663938noreply@blogger.com3tag:blogger.com,1999:blog-3857007129715168662.post-29248570575822932022011-02-25T10:17:00.000+02:002011-02-25T10:17:33.363+02:00JDK1.6.0_24 releasedNothing interesting though.... JDK1.6.0_23 included the latest version of VisualVM but this is old news by now.Lifeyhttp://www.blogger.com/profile/10586718302147663938noreply@blogger.com0tag:blogger.com,1999:blog-3857007129715168662.post-7700153366098622312011-02-25T10:16:00.000+02:002011-02-25T10:16:05.876+02:00First impressions from G1GCI was trying to see how G1GC will cope with an application that I am investigating.<br />
Here are my disappointing results:<br />
1. With JDK 1.6.0_20 G1GC is not reporting memory correctly via JMX. problem is solved in later jdks but I am not able at this stage to upgrade the JVM<br />
2. VisualVM is not reporting memory correctly with G1GC need to open a bug for this issue.<br />
3. I cannot find thorough documentation regarding what the GC logs mean (when using G1GC with -Xloggc)<br />
4. The application crashed every few minutes. Again this was with 160_20 and I need to recheck with 160_24Lifeyhttp://www.blogger.com/profile/10586718302147663938noreply@blogger.com0tag:blogger.com,1999:blog-3857007129715168662.post-29286206411745420972010-07-12T18:05:00.000+03:002010-07-12T18:05:23.049+03:00Java6u21 is released 4 days agoThat's good news from performance perspective since it includes the latest VisualVM version (1.2.2)<br />
According to Oracle release notes this include the following functionality<br />
<br />
<ul style="font-family: Arial, Helvetica, FreeSans, Luxi-sans, 'Nimbus Sans L', sans-serif; font-size: 12px; margin-left: 13px; padding-left: 0px;"><li style="list-style-image: url(http://java.sun.com/im/ul_bullet.gif); margin-left: 13px; padding-left: 0px;">HeapWalker performance improvements</li>
<li style="list-style-image: url(http://java.sun.com/im/ul_bullet.gif); margin-left: 13px; padding-left: 0px;">VisualVM-Sampler performance improvements</li>
<li style="list-style-image: url(http://java.sun.com/im/ul_bullet.gif); margin-left: 13px; padding-left: 0px;">BTrace4VisualVM plugin introduces <a href="http://kenai.com/projects/btrace/forums/forum/topics/2262-BTrace-1-1-Released)" style="color: black;">BTrace 1.1</a></li>
<li style="list-style-image: url(http://java.sun.com/im/ul_bullet.gif); margin-left: 13px; padding-left: 0px;">Profiling engine bugfixes</li>
<li style="list-style-image: url(http://java.sun.com/im/ul_bullet.gif); margin-left: 13px; padding-left: 0px;">Built on NetBeans Platform 6.8</li>
</ul><div><span class="Apple-style-span" style="font-family: Arial, Helvetica, FreeSans, Luxi-sans, 'Nimbus Sans L', sans-serif; font-size: small;"><span class="Apple-style-span" style="font-size: 12px;"></span></span></div><div><span class="Apple-style-span" style="font-family: Arial, Helvetica, FreeSans, Luxi-sans, 'Nimbus Sans L', sans-serif; font-size: small;"><span class="Apple-style-span" style="font-size: 12px;"><br />
</span></span></div><div><span class="Apple-style-span" style="font-family: Arial, Helvetica, FreeSans, Luxi-sans, 'Nimbus Sans L', sans-serif; font-size: small;"><span class="Apple-style-span" style="font-size: 12px;">It still not a full replacement to a commercial profiler but I hope it will be soon :-)</span></span></div><div><span class="Apple-style-span" style="font-family: Arial, Helvetica, FreeSans, Luxi-sans, 'Nimbus Sans L', sans-serif; font-size: small;"><span class="Apple-style-span" style="font-size: 12px;"><br />
</span></span></div>Lifeyhttp://www.blogger.com/profile/10586718302147663938noreply@blogger.com0tag:blogger.com,1999:blog-3857007129715168662.post-33967567667033758032010-05-18T10:25:00.000+03:002010-05-18T10:25:00.438+03:00Page file size on machines with large amounts of memoryThere is a rule of thumb that page file size (and swap space partition) should be 1.5 times the physical memory of the machine. Now that we have 64 bit machines with 16-64GB of memory this looks like an absurd.<br />
<br />
<ul><li>We do not want to reserve so much disk space</li>
<li>It does not make sense to rely so heavily on disk </li>
</ul><br />
I found two nice references in the internet regarding Linux and Windows<br />
<br />
<a href="http://www.cyberciti.biz/tips/linux-swap-space.html">http://www.cyberciti.biz/tips/linux-swap-space.html</a><br />
<a href="http://support.microsoft.com/kb/889654">http://support.microsoft.com/kb/889654</a><br />
<br />
So my new rule of thumb page file should not be more than 70% full<br />
<span class="Apple-style-span" style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 11px; line-height: 16px;"><table cellspacing="1" class="table" style="-webkit-box-sizing: border-box; background-attachment: initial; background-clip: initial; background-color: #7fbae2; background-image: initial; background-origin: initial; background-position: initial initial; background-repeat: initial initial; border-bottom-width: 2px; border-left-width: 3px; border-right-width: 3px; border-top-width: 0px;"><tbody style="-webkit-box-sizing: border-box;">
<tr style="-webkit-box-sizing: border-box;"><td style="-webkit-box-sizing: border-box; background-attachment: initial; background-clip: initial; background-color: #f7f7ff; background-image: initial; background-origin: initial; background-position: initial initial; background-repeat: initial initial; font-size: 11px; padding-bottom: 5px; padding-left: 5px; padding-right: 5px; padding-top: 5px; vertical-align: top;">Paging File\\% Usage Peak _Total < 70%<br />
</td></tr>
</tbody></table></span>Lifeyhttp://www.blogger.com/profile/10586718302147663938noreply@blogger.com0tag:blogger.com,1999:blog-3857007129715168662.post-32505397709460023962010-05-12T20:43:00.002+03:002010-05-12T20:43:52.873+03:00iPad Performance<span class="Apple-style-span" style="font-family: Verdana, sans-serif; font-size: 11px; line-height: 17px;">Just received my new iPad and I must say that I really amazed by its performance. Everything is so snappy kudos to apple. I do want to understand how they were able to create this experience on such a device.</span><br />
<div style="font-family: Verdana, sans-serif; font-size: 11px; line-height: 17px;">Can someone arrange me a meeting with Steve?</div>Lifeyhttp://www.blogger.com/profile/10586718302147663938noreply@blogger.com0tag:blogger.com,1999:blog-3857007129715168662.post-91516091513862668982010-04-25T08:28:00.000+03:002010-04-25T08:28:11.652+03:00Dangling LocksHappy about Java not suffering from dangling pointers ? Think again the Java5 concurrency utils while very powerful looks like a language flaw:<br />
The following code is an example where a lock cannot be released anymore the thread which holds it is dead. Moreover there is no indication who is holding this lock ...... jstack won't help you......<br />
Oracle please fix it !!!<br />
<br />
<br />
<span class="Apple-style-span" style="color: #134f5c;"><b><i>public class DanglingLock {</i></b></span><br />
<span class="Apple-style-span" style="color: #134f5c;"><b><i> static ReadWriteLock l = new ReentrantReadWriteLock();</i></b></span><br />
<span class="Apple-style-span" style="color: #134f5c;"><b><i> public static void main(String [] args) throws InterruptedException {</i></b></span><br />
<span class="Apple-style-span" style="color: #134f5c;"><b><i> Runnable r = new Runnable() {</i></b></span><br />
<span class="Apple-style-span" style="color: #134f5c;"><b><i><br />
</i></b></span><br />
<span class="Apple-style-span" style="color: #134f5c;"><b><i> public void run() {</i></b></span><br />
<span class="Apple-style-span" style="color: #134f5c;"><b><i> l.writeLock().lock();</i></b></span><br />
<span class="Apple-style-span" style="color: #134f5c;"><b><i> try {</i></b></span><br />
<span class="Apple-style-span" style="color: #134f5c;"><b><i> Thread.sleep(20000);</i></b></span><br />
<span class="Apple-style-span" style="color: #134f5c;"><b><i> } catch (InterruptedException ex) {</i></b></span><br />
<span class="Apple-style-span" style="color: #134f5c;"><b><i> Logger.getLogger(DanglingLock.class.getName()).log(Level.SEVERE, null, ex);</i></b></span><br />
<span class="Apple-style-span" style="color: #134f5c;"><b><i> }</i></b></span><br />
<span class="Apple-style-span" style="color: #134f5c;"><b><i> }</i></b></span><br />
<span class="Apple-style-span" style="color: #134f5c;"><b><i><br />
</i></b></span><br />
<span class="Apple-style-span" style="color: #134f5c;"><b><i> };</i></b></span><br />
<span class="Apple-style-span" style="color: #134f5c;"><b><i> Thread t = new Thread(r);</i></b></span><br />
<span class="Apple-style-span" style="color: #134f5c;"><b><i> t.start();</i></b></span><br />
<span class="Apple-style-span" style="color: #134f5c;"><b><i> Thread.sleep(5000);</i></b></span><br />
<span class="Apple-style-span" style="color: #134f5c;"><b><i> l.writeLock().lock();</i></b></span><br />
<span class="Apple-style-span" style="color: #134f5c;"><b><i> }</i></b></span><br />
<span class="Apple-style-span" style="color: #134f5c;"><b><i><br />
</i></b></span><br />
<span class="Apple-style-span" style="color: #134f5c;"><b><i>}</i></b></span>Lifeyhttp://www.blogger.com/profile/10586718302147663938noreply@blogger.com0tag:blogger.com,1999:blog-3857007129715168662.post-21807431779116559272010-04-08T10:36:00.000+03:002010-04-08T10:36:52.174+03:001 gadget per year policyWell as I am trying to convince myself that I really need an iPad I use the following arguments<br />
- I must try new technology<br />
- I must understand the hype around tablets<br />
<br />
and the most absurd argument is that I need a policy for buying gadgets...<br />
The policy is one gadget per year .....<br />
<br />
Sounds reasonable no ?Lifeyhttp://www.blogger.com/profile/10586718302147663938noreply@blogger.com0tag:blogger.com,1999:blog-3857007129715168662.post-91008619083038850312010-04-04T11:48:00.000+03:002010-04-04T11:48:45.926+03:00I really want an iPad but do I really need it ?<span class="Apple-style-span" style="font-family: Arial; font-size: small;"><span class="Apple-style-span" style="font-size: 13px;">It is cool I fantasize reading books with it viewing movies on the go and browsing the internet in the park....</span></span><div><span class="Apple-style-span" style="font-family: Arial; font-size: small;"><span class="Apple-style-span" style="font-size: 13px;">I am not sure i will continue using it after more than few weeks?</span></span></div><div><span class="Apple-style-span" style="font-family: Arial; font-size: small;"><span class="Apple-style-span" style="font-size: 13px;">Will I start carrying another device ? not sure.....</span></span></div><div><span class="Apple-style-span" style="font-family: Arial; font-size: small;"><span class="Apple-style-span" style="font-size: 13px;">I guess that I will continue to hold my breath and wait for Hebrew keyboard, jailbreak and the 3G version....</span></span></div><div><span class="Apple-style-span" style="font-family: Arial; font-size: small;"><span class="Apple-style-span" style="font-size: 13px;"> </span></span><br />
</div>Lifeyhttp://www.blogger.com/profile/10586718302147663938noreply@blogger.com0tag:blogger.com,1999:blog-3857007129715168662.post-72444871229355034992010-04-04T11:44:00.000+03:002010-04-04T11:44:32.705+03:00JDK1.6.0_19 is releasedNo major news just a few bug fixes...<br />
BTW JDK1.6.0_18 included a new version of the VisualVM which is much more powerful and also includes a sampler..... so I was very anxious to see if 1.6.0_19 includes visualvm1.2.2 but it does not ....<br />
Never mind we can always download it directly from the website....Lifeyhttp://www.blogger.com/profile/10586718302147663938noreply@blogger.com0tag:blogger.com,1999:blog-3857007129715168662.post-80678836207474386362010-02-11T17:04:00.002+02:002010-02-11T17:09:03.768+02:00AWS is offering consolidated billing systemIt took them one year but finally......<br />If you are a company and wanted to use AWS you every user needed to create an acoount and enter billing details.<br />This is very cumbersome and does not enable centralized management and tracking of cloud costs.<br />Since not it is going to be changed. You will be able to consolidate all your acoount and get aggregated report for your organization.<br />I guess that now it will be also possible to limit an acoount budget according to the organization policy.....<br />Kudos AmazonLifeyhttp://www.blogger.com/profile/10586718302147663938noreply@blogger.com0tag:blogger.com,1999:blog-3857007129715168662.post-7888013036514285272009-12-30T13:19:00.003+02:002009-12-30T13:25:50.114+02:00Cloud era: Pay for your poor performanceI have read an interesting post which claims that in the cloud computing era poor performance directly converts to higher operational costs.<a href="http://www.infoq.com/news/2009/11/Price-for-Performance"> http://www.infoq.com/news/2009/11/Price-for-Performance</a><br />I agree with this to some extent. Paying for machines is not exactly as paying for CPU but the general idea is interesting.<br />Thinking about it a bit more it makes me wonder that it will be nice to be able to find the equilibrium point of performance investigation investment. The amount of money that you invest of performization should be less than the amount of time you loose for inefficiency.<br />A new research area is born :-)Lifeyhttp://www.blogger.com/profile/10586718302147663938noreply@blogger.com0tag:blogger.com,1999:blog-3857007129715168662.post-85830417434465532392009-12-08T17:40:00.000+02:002009-12-08T17:41:30.901+02:00diagnosis tools for JDK 1.6A nice Sun document on diagnosis tools for JDK 1.6<br /><a href="http://java.sun.com/developer/technicalArticles/J2SE/monitoring/">http://java.sun.com/developer/technicalArticles/J2SE/monitoring/</a>Lifeyhttp://www.blogger.com/profile/10586718302147663938noreply@blogger.com0tag:blogger.com,1999:blog-3857007129715168662.post-3052642415430320122009-11-01T17:48:00.004+02:002010-03-17T09:50:07.815+02:00KSE2 is around the cornerHi,<br />
On the 17th of November the second Knowledge share event is going live.<br />
It will include 3 sessions:<br />
1. Fork/Join Framework<br />
2. What makes good performance<br />
3. Short stories on simple tools<br />
<br />
For more info and registration<br />
<a href="http://www.performize-it.com/kse/kse2">http://www.performize-it.com/kse/kse2</a>Lifeyhttp://www.blogger.com/profile/10586718302147663938noreply@blogger.com0tag:blogger.com,1999:blog-3857007129715168662.post-38799021482363724482009-09-12T21:06:00.002+03:002009-09-12T21:14:18.582+03:00Mac OSX Snow leopard better performanceI have upgraded my Macbook to Snow Leopard this weekend and indeed experienced better performance. I have bumped into an interesting article which explains why : (<a href="http://www.appleinsider.com/articles/08/09/04/road_to_snow_leopard_twice_the_ram_half_the_price_64_bits.html">follow link</a> )<br />To make short story shorter Mac maps 32bit processes to two 4GB address spaces one for user mode and one for kernel mode. As a result any kernel mode system call will require a TLB flush.<br />Windows by limiting user address space to 2GB and kernel address space to 2GB do not suffer from this problem (but suffer from much lower memory limit). Snow leopard is full 64 bit which solves this issue. User mode and kernel mode address space share the same TLBLifeyhttp://www.blogger.com/profile/10586718302147663938noreply@blogger.com0