Yet another reason to move to JDK 1.6

My first production memory leak analysis with Java 1.6 was almost a very pleasant experience.

While in previous versions of java I  needed to restart the JVM and set appropriate profiler command line option,  Java 6 comes with a built in alternative.  jmap which was not available for Windows distributions is now available for use. Taking a memory snapshot is a point and click task:

jmap -dump:format=b,file=dump.hprof

Take  the hprof dump generated and use you favorite profiler to analyze it.

If you are only interested in a short summary of memory usage by class:

jmap -histo

So, why was it only an almost pleasant experience ? Because you need to make sure that you are connected to the same windows session as the process you are trying to dump. If you are using terminal services make sure you use the /console command line option, otherwise jmap will not work. Beats me why.. 

10 People - a Billion USD revenue

I have been attending this week to the"World summit of cloud computing" which was quite an impressive conference compared to other Israeli conferences I have been to with representatives from Google,HP,Intel,Amazon,EBay, Yahoo,SUN, Salesforce, GridGain, Forrester,GigaSpaces, MerrilLynch and more. Each one of those vendor has a different definition and point of view so after those two days I am ready to give my own definition and what it means to me.
So, Cloud computing is a deployment option for your application which has the following four characteristics:

Outsourced
Your run your stuff on other people machines and infrastructures(IaaS). These resources can be conזsumed either from the the intranet for small large enterprises and government use or reside in the in the Internet for small/midsized companies. But the main issue here is that all you pay for the infrastructure in that case is money.

Economical
Using this deployment option should save you money and possibly a lot of money.

• Reduced CAPEX:, meaning money you will need to invest upfront inoder to support the expected and unexpected changes in capacity.

• Reduced OPEX: Let cloud providers do what they do the best, maintain large machine infrastructure. Hopefully (with the right amount of competition) that will reduce your IT OPEX.
  
• New financial model: All the ?aaS concept is presuming a different(and more fair) revenue model compared to what we are used to in the traditional software industry you only pay for the resources you use no payment upfront and not commitment for long term.
  

Elastic
You are able to adjust seamlessly the amount of infrastructure you consume. This helps you handle peaks more efficiently. You no longer need to choose between compromising on your SLA in peak times or let your servers rot in the cellars when you really don't need them.

SelfService
You as a developer(and I am intentionally focusing on developers) are able to allocate and deallocate resources by yourself through an API no need to sign forms no paperwork. You can write on use software which will do the job for you.


And not to my thoughts:
So I have a crazy vision regarding this new concepts what I see in my mind is a billion USD internet company with a size of 10 people which altogether bring unique (probably patented) value and all their job is to orchestrate outsourced resources to the benefit of the company.
A CEO - orchestrating vision and execution with a set of VAs
VP sales - orchestrate sales operation through an effective network affiliation
VP infrastructure - manages the application in the cloud
VP support - orchestrate outsourced call center and
VP finance
VP R&D -
4 developers - developing only core business logic consuming open source projects for the other stuff.

I know it is a bit extreem but I don't care I am sure it will happen one day.

IGT 2008 next week

The world summit of cloud computing is taking place in Israel next week.
I have big expectations toward this summit as I believe that the future is in the cloud.
This is not an affiliation link :-) you are welcome to register I'd love to see you there!
http://www.cloudcomputing.org.il/index.html

Last post Recall or why use HTTP1.1

Today I saw that Ford are recalling 1000 cars due to problems in steering wheel liquid. This makes me feel a bit more comfortable about the need to take back what I said just few days ago :-).

So why do I need to recall the post. Not because SSL connection establishment is not expensive. It is indeed. The issue is that HTTP1.1 (opposed to HTTP1.0)  can pool the connections and reuse them for later requests thus diminishing the effect of HTTPS latency).

So why the Apache web server was not behaving well (and put me into confusion)? Configuration .....

The AWS was configured to use HTTP1.0 for HTTPS connection under Internet explorer.

I did not understand yet why(an issue for a later post)  is it so, probably bugs in older versions of IE. Anyway commenting out the following lines in httpd-ssl.conf  brings back the missing .1:

BrowserMatch ".*MSIE.*" \

   nokeepalive ssl-clean-shutdown \

   downgrade-1.0 force-response-1.0

And voila after that only one roundtrip for SSL requests.....

Absolutely great :-)

Speed of light vs SSL

(Please see next post for update)

I have been trying to measure WAN efffect over an application I am working on. Due to a bug the application was running in SSL and I was quite surprised by the results.
After investigating it a little I want to share with you the following information. https (SSL) requests to the server involve two extra round trips for certificate authentication and key exchange. As a result a short HTTP request will require 3 times the network round trip time (6 times the latency). For example if latency is 90 ms and server time is 20ms https request will be 90*6+20= 560ms. While an http request will require 200ms. Quite impressive difference.
Saying that it is extremely important to reduce number of https requests to a server combining https requests is 3 times more important than regular http requests.

TSSJS Day 3

I want to cover two sessions from the third day:

The busy Java developer’s Guide to Scala

Ted Newman is definitely a charismatic presenter. With great passion he reviewed basic functionality of Scala DSL. Well Scala is a functional language and after the session I really want to learn it.

Performance Tuning a web shop with open source tools

Dr. Jeroen Borgers showed how performance improvement process can be done using Open Source tools. He was using JMeter (while I usually use LoadRunner) for load generation. Jprobe for profiling (I prefer YourKit BTW). JAMon for continuous monitoring (I used SiteScope commercial tool). For reporting purposes they wrote their own tool and placed on SourceForge (JaRep).

TSSJS Day 2

I want to cover 3 sessions on the secion day

Languages-Oriented Programming: Shifting Paradigms
Computer languages evolution enables us to pave over disturbing problems (e.g. GC as a mechanism to pave over error-prone memory allocation, Neal Ford introduced a very important observation.. One of the most powerful aspects in Java platform is the community and the amount of open source frameworks. This creates a new problem to pave over. Each framework has its own jargon and adapting this to the java language leads to a complicated syntax which usually is very wet (not DRY – Don’t Repeat Yourself). Niel offer the use of DSLs to approach this problem. In other words frameworks will be transformed to a carefully designed DSL. AntLR as a lexical analyzer and environments like Jetbrains MPS can help reaching this with less efforts.


Extreme Transaction Processing, Low latency and performance
John Davis a banking expert gave a startling session on design criteria in the online trading arena. In this world a 1ms delay in processing a message can lead to losses of $100M. As a result for example banks try to locate the trading infrastructure as close as possible the trading backbone (usually in London) in order to reduce latency. In addition GC can be a real problem we are used to think of GC taking 200ms as something reasonable but again it is x200 from the 1ms threshold. You do not want to loose money on GC. This leads to weird solutions such as restarting the VM before the first GC and redirecting to a different cluster member in the meanwhile. Another point mentioned is that traditional RDBMS are not capable of handling tens of thousands of transaction on a reasonable price. The solutions will be to use in memory databases or caching mechanisms (e.g. GigaSpaces, Oracle Coherence, Terracotta etc… )

Concurrency and High Performance
Kirk Pepperdine session had an important punch-line. Processors clock speed is stuck at the 3Ghz boundary and this situation is not likely to change in the near future. CPU vendors are going to achieve Moore’s law by doubling the number of cores every 18 months. This is a fact developers can’t afford to ignore. Leading to the inevitable conclusion (punch-line ahead) Is your application ready to double it’s concurrency in the next 18 months?

The inevitable delay

A professional blogger will never let this happen. I admit that as a newbie when my wife joined me I preferred traveling and not blogging… Here is a link to the pictures. When I came back to Israel I was dead tired. So I continue to write only now (I do not regret it though)

Yourkit 7.5 is released

Yourkit is my favorite profiler. It is a bit of old news but I am happy to hear that Yourkit 7.5 is released (on the 22-May-08). It includes of usability improvements and bug fixes. The most important thing I think is the CPU measurements for stack trace snapshots. It can actually show you which treads consume CPU before starting to profile. Previously I used intrnal tools for such purposes... I did not try this feature yet and I wonder if it can be used with out configuration of the profiler agent (great value for production)

TSSJS day 1

A: Beautiful

B: Who?

A: The City.

B: Which city?

A: Prague

B: So why don’t you say Prague? Why are you going around and around?

A: Well, since I have got here I have an unstoppable urge to stroll in the beautiful streets.

B: So why don’t you?

A: Well....

I have landed in Prague Yesterday in 08:45 directly into the conference. (Comment: Try to avoid these scenarios as much as you can, I have been dead tired all day long). To the conference I have arrived two hours late and missed the first two sessions :-(.

While I really want to share as much knowledge as I can I am reluctant of writing long descriptions and summaries on sessions. A lecture heard can enlighten one’s mind but it is hard to transfer the essence of this enlightenment to a blog . Therefore, I am going to share with you a single most important piece of information I took with me from sessions.

The two most effective sessions I did attend on day 1 are (Drums !!!!):

Monitoring Management and Troubleshooting in the Java SE6 Platform

Jean-Francois Denis from SUN gave a very interesting session on JDK 6 new JMX abilities and tools. The lecture which started with the very basics did move to more advanced issues. The most useful piece of information from my point of view is the VisualVM. A real open source lightweight Java profiler!!

Java Performance Tooling

Dr Holly Cummins from IBM is a very colorful person and a funny lecturer. She gave a nice introductory session on performance troubleshooting. Holly exposed me to the term ‘lock-bound’ which is a brilliant terminology for saying ‘Well we use locking mechanism extensively and there is a lot of contention on these locks… this is why our application sucks’. From this lecture I am have learned that IBM have set of nice free tools which one can use even if he is not using IBM JVM.Follow this link for more information.

TSSJS

Hi,
I am on my way to TSSJS. This is the first time I am attending this conference and I am really thrilled as I heard great deal about it.
I will update my blog during the conference whenever I will have something interesting to say :-)
If you attend the conference come and say hello :-)

I have a flight to catch....

Client side Memory Leaks IE and GWT - A practical guide

By now I guess that you read the previous post and you are eager to get some practical knowledge.
So I will summarize my knowledge regarding this issue. While I agree it is a bit shallow it is much better than nothing at all.
Tools like IESieve and IEdrip proven to be inefficient when coming to GWT leaks the code is too big for them and the transformation from Java to JS complicates things.
I will be more that delighted to be proven an idiot, if someone have a better (proven) approach please let me know.

Code Review
When coming to solve /prevent memory leaks the best way will always be code review. Hence I will give a list of guidelines for reviewing GWT code:
First list of guidelines Stating the obvious:

1. Avoid writing JSNI code. Google made quite a good job in writing "almost" leak free it is so easy to ruin it if you do not know what you are doing. Remember every JSNI code you write will lower your productivity.
2. Do not use the DOM.* methods (except the setStyle... which are safe) . Manipulating the DOM yourself will lead you directly toward a memory leak.
   

Second list of guidelines - less obvious

1. Static variables containing(even indirectly) references to widgets and dom objects may cause a leak.
2. According to Google it should not happen :-) but in some cases event listeners may leak. unregistered them when window unloads.

Lion in the desert approach
When code review by it self is insufficient, the desperate developer should try the following approach. As stated in the previous post when a memory leak in GWT occur the entire JavaScript is locked in the memory. This will lead to a huge memory leak (megabytes in large modules). This major drawback which forces us to deal with memory leaks is also our savior. Since the memory leak is so evident it is possible to defferentiate between cases where it is existant and cases where it is eliminated. Track the VM size of the IE process while refreshing the page for several times. If the VM size inceases dramatically you have got a memory leak. Vice versa, memory do not increase constantly memory leak is solved.
Saying that all you need to do is comment out pieces of code until memory leak is eliminated.

This process is not easy nor fun some tips to improve its effectiveness:

• Memory leaks are ilusive sometimes removing an irellevant piece of code will stop the memroy leak. As a result you should try to reach to the smallest peice of code which is still leaking before eliminating it. This will ensure you do not shoot at the wrong target .
• Work directly on javascript files. It takes some experitse to be able to understand what you are doing but it worth it. GWT comiplation phase is long and working directly on the JS file shortens the search cycle dramatically.

Thats all folks :-)
Enjoy

Client side Memory Leaks IE and GWT - Introduction

I believe that software development should be fun. This is why for a long time I avoided the wild world of client side and JavaScript. Even though there is a big traction toward AJAX and WEB2.0 I am still not convinced that a group of average developers can develop a large scale, maintainable, enterprise ready and long lasting project based on this technology. JavaScript in just not that kind of language and the freakishly incompatible Browsers world which do not supply comprehensive set of monitoring tools is absolutely a nightmare. Moreover this world is ruled by IE, the two headed beast, which encompasses two GC mechanisms each one of them is not aware to the other. It is so easy to introduce new memory leaks and it is so hard to eliminate them.

I am currently involved in a project written in GWT. I must admit that on the first look this ambitious technology looks quite promising. Developing in Java is much better than JavaScript, GWT offers mitigation to browser incompatibility and IE memory leaks.

Indeed, development phase was quite productive developers did enjoy to develop the code and delivered fast results. (Complications came back when JavaScript native code needed to be written in order to supply needed functionality not supported by GWT--> we were back in the JS alley)

When we approached stabilization phase things became nasty. On one hand performance optimizations where required and we could not ignore the fact that the target engine was JavaScript engine. Moreover the biggest nightmare was memory leaks in IE 6/7. Each indirect circular reference between the DOM and the JS engine caused the entire JS to be locked in memory leading to megabytes of leak. The effort needed to solve memory leaks was huge and involved most of the code written. I will not delve into project constraints and design criteria that might have made this problem so acute. I want to focus on some guidelines to help in avoiding and solving memory leaks. Since this post is getting a bit long I will split those guidelines to the next one.


And one good news to close this post: I opened a bug to Google regarding a memory leak when closing a window 5 months ago. I forgot all about it, buts suddenly out of the blue I got an email. Bug was verified and fixed. Fix will be available for GWT1.5

Moore's Law for the average Mosquito

I really planned to go to sleep, but I was really amazed by the amount of mosquitoes that one have to hunt before he can go to sleep quietly. It suddenly occurred to me that the average mosquito is becoming twice as fast every approx two years. It is maneuvering much better. It's flight is much more silent and its bite is becoming stronger.

When are we going to see dual core mosquitoes?

System.out.println("Hello World!");

Hello world

I wanted to open a professional blog for a long time. It took me a few months to convince myself that I can transform this desire into commitment. I have a lot of thoughts related to java and performance that I want to share with you all. I really hope it will be a trigger an effective and lively discussion.

There goes away one hour of sleep a day... I hope that I will not fall asleep in the middle of the posts :-)

So, now I will check again that post title is not empty and....
System.out.flush();