5 Things a Java Developer consider start doing this year

This post is about 5 things that I am planning to do this year. I have created it for me to track my progress then thought it can be a good direction for anybody similar.

1. Create an application using a NoSQL Data store and connect it with Java

If you have yet not understood the NoSQL databases its the best time. Lots of choices on the NOSQL side, may be MongoDB or Hadoop can be a starting point. We can create applications using Spring Data or the native Java adapters to connect to the Data Store.

2. Get the first app on Java PaaS on Cloud and ask your 5 friends to use it.

You have many platforms available including Openshift backed by JBoss and Redhat , CouldFoundry backed by Spring source and VMware. Cloud is the future of application deployments and Software as service gaining more popularity. From a developer point of view nothing really changes apart from the configurations and deployment.

3. What really is Software Design?

Read the GOF Design Pattern catalog and Search your project for the usage of it. If you are not using them check if you have similar patters. If you have a Java enterprise application you can check for Java EE patterns. Take a existing use case and think of possible alternative implementations.

4. Learn a new Programming Language and create a sample project

I think here you have 2 broad choices, Ruby or a JVM functional language. There are a number of functional languages available. It will help you become a polygot programmer.

5. Contribute to the community

You should be doing it already if not it is the time to start. There are a number of ways including Community Forums, Stack overflow or write a blog

on how your leanings.

11 Online Learning websites that you should check out

Planning to start something new in this year? You can try online learning!!. Online education is gaining popularity over the last few years, as it should. I have tried few of them last year and will share my experience with them in this post. Since I am a software developer we are going to focus on the materials related to it.

So, here is a list of 15 sites that you should visit before you make you make your choice. I have not used much on all of them, but worth a spending some time. The italic text below the title are taken from the respective sites. The order of names are arbitrary and does not carry any significance.

1. MIT's OpenCourseWare

OCW makes the materials used in the teaching of MIT's subjects available on the Web.

Add caption

2. Coursera

We are a social entrepreneurship company that partners with the top universities in the world to offer courses online for anyone to take, for free. We envision a future where the top universities are educating not only thousands of students, but millions. Our technology enables the best professors to teach tens or hundreds of thousands of students.

3. Khan Academy

Learn almost anything for free

4. Google Developers

5. Code Academy

6. Code School

Code School teaches web technologies in the comfort of your browser with video lessons, coding challenges, and screencasts.

7. LearnStreet

Whether you are completely new to coding or a developer looking to learn a new language, LearnStreet courses make it engaging and fun to create with code. Our interactive courses are designed to help you learn by actually writing real code and getting immediate feedback.

8. wibit.net

WiBit.net is a video tutorial web site offering cutting edge programming and computer tutorials. We specialize in focused and linear content. WiBit is a great place to start learning how to program, or to pick up new skills even if you've been at it a while.

9. Interview street

Solve interesting challenges and also get connected to amazing tech companies if you are interested. Join in to be among the elite group of talent in the world.

10. PeepCode

Specialize in screencast tutorials for professional web developers and designers. Learn advanced Ruby, JavaScript, Ruby on Rails, Node, Git, Design, and connect with developers you admire in our popular Play by Play series.

11. P2PU.org

At P2PU, people work together to learn a particular topic by completing tasks, assessing individual and group work, and providing constructive feedback.

Related articles on the web:

1. http://mashable.com/2011/05/07/learn-to-code-online/

2. http://thenextweb.com/dd/2012/10/21/so-you-want-to-be-a-programmer-huh-heres-25-ways-to-learn-online/

3. http://www.makeuseof.com/tag/learn-code-10-free-fantastic-online-resources-hone-skills/

4. http://techcrunch.com/2012/08/21/5-ways-to-learn-code-from-the-comfort-of-your-own-browser/

5. http://thenextweb.com/dd/2012/01/21/7-ways-to-start-learning-how-to-code-right-now-for-free/

Using Jasper Reports to create reports in Java

Last week I was trying to create a report using Jasper. In this post I will document some of the resources and links so that it will be useful for any one looking for similar information. I will cover life cycle of Jasper reports, examples and Dynamic Jasper. 

The Jasper Reports is the world's most popular open source reporting engine. It is entirely written in Java and it is able to use data coming from any kind of data source and produce pixel-perfect documents that can be viewed, printed or exported in a variety of document formats including HTML, PDF, Excel, OpenOffice and Word.

JasperReport Life Cycle

Image from Jasper Library Wiki

As in the image the life cycle has 3 distinct phases,

1. Designing the Report

In this step involves creation of the JRXML file, which is an XML document that contains the definition of the report layout. We can use the either iReport Designer or a text editor to manually create it. Using iReport Designer, the layout is completely designed in a visual way, so you can ignore the real structure of the JRXML file.

Here is the detailed tutorial on designing a report using iReport. We can also use Dynamic Jasper described later in the article to design a report.

2. Executing the report.

Before executing a report, the JRXML must be compiled in a binary object called a Jasper file(*.jasper). This compilation is done for performance reasons. Jasper files are what you need to ship with your application in order to run the reports. Once the report is compiled it is filled with data from the application. The class net.sf.jasperreports.engine.JasperFillManager provides necessary functions to fill the data in the reports.

The report execution is performed by passing a Jasper file and a data source to JasperReports. There are plenty of types of data sources, it's possible to fill a Jasper file from an SQL query, an XML file, a csv file, an HQL (Hibernate Query Language) query, a collection of Java Beans, etc... If you don't find a suitable data source, JasperReports is very flexible and allows you to write your own custom data source.

JasperFillManager.fillReportToFile( "MasterReport.jasper" , parameters, getDataSource());

This operation creates a Jasper print file (*.jrprint), which used to either print or export the report.

3. Exporting to the desired format

Using the Jasper print file created in the previous step we shall be able to export it into any format using JasperExportManager. Jasper provides various forms of exports. This means with the same input we can create multiple representation of the data. Jasper inernally uses different APIs to create documents. But these complexity are hidden by the simpler JasperExportManager.

JasperExportManager. exportReportToPdfFile( "MasterReport.jrprint" );

In a nutshell the life cycle can be summarized in the below image

Image from Ramki Tech

References and other good articles on Jasper Reports Life Cycle

1.  Jasper Library Wiki
2.  Jasper Reports Wiki

3.  Jasper Reports in Ramki Java Blog

4.  JasperReport – Open Source Java Reporting Framework

Examples

I have found it really hard to find a working example of Jasper report. But it is right there inside the package shipment!. Once you have downloaded the Jasper Library go to demo\samples, you will find a lot of sample programs. Many of these needs a working HSQL DB connection, to activate it go to demo\hsqldb and start the server. Every folder has a readme.txt file which will help you in understanding how to run it. All the examples can be executed using ant tasks.

Here is a list of few other sources.

1. Samples from the Jasper Library

2. Java Reporting With Jasper Reports - Part 2

3. Jasper Reports - Example

4. Spring MVC 3.1 and JasperReports

Simplify report creation using Dynamic Jasper

DynamicJasper (DJ) is an open source free library that hides the complexity of Jasper Reports, it helps developers to save time when designing simple/medium complexity reports generating the layout of the report elements automatically.

The project homepage provides lots of examples and code snippets on how to use the library. I have been using it for some time and it is a pretty stable replacement for the JRXML file.While using dynamic jasper the report design is coded in Java. Which means every time the report is compiled, filled and exported. By using dynamic jasper we are replacing the first step in the above mentioned jasper life cycle. Even with dynamic jasper you need the jasper library and other dependent files.

Here is some more examples of Dynamic Jasper usage.

1. HOW TO page on Dynamic Jasper

2. Spring 3 - DynamicJasper - Hibernate Tutorial: Concatenating a DynamicReport

3. Spring 3 - DynamicJasper - Hibernate Tutorial: Using Plain List

Top 7 tips for succeeding in a technical interview for software engineers

In this post I would like to write on how to succeed in a technical interview based on my experience as an interviewer. Most of the interviews follows some patterns. If you understand it and frame your response in the same way you can clear any interview. If you don't know stuff this might not help you, but if you are prepared, this article will help you show of your full potential.

If you are skillful the only reason you can loose an interview is by lack of preparation. You may know all the stuff but you still needs to prepare by reading books, article etc.. Theses may not teach you anything new but will help in organizing things that you already know. Once you have organized information it is really easy to access it. You should read not only for interviews, make it a practice and get better at your job.

Most of the time interviewer is looking for a candidate who can work with him. The vacancy may be in other teams but they use this parameter to judge. Mostly this article contains general tips. These are targeted for 2 to 6 years experienced candidates.

1. Be honest and don't bluff

Answer what you know, confidently. If you have been asked a question that you don't know, Start by telling "I am not sure, but I think It is .....". Never tell a wrong answer confidently. That will make them doubt your correct answers also or may feel that they were guesses. You can't use this technique for every question, but I would think 25% is a good amount. Most importantly this shows your ability to think and a never die attitude. No one wants to work with people says "I can't do this". Try to do some thing about all the questions.

2. Be ready to write Code

If you are been asked to write some code, be careful and follow some basic standards. I heard people telling me "I forgot the syntax..." and this for the syntax of a for loop. No one expect you to remember everything but basics like looping, if conditions, main method, exceptions are never to be forgotten. If you did, brush them up. Always write the code with good indentation using lots of white spaces. That might make up for your bad handwriting!!

3. Get ready to explain about your project

As engineers you have to understand the business before you start code it. So you should be able to explain what is being done in your project. Write down 3-4 lines that will explain the project in high level. By hearing the lines some one out side your team should get an idea about it. Because we always works inside on features, most of the time it is difficult to frame these. Check your client's internal communications how they are marketing and get some clue from it. Practice what your are going to say with friends make make sure you are on to the point.

Once you have explained about the business needs then you will be asked about the technical architecture of the project. You have to be prepared with a architecture diagram that shows how the interaction of components in your project. It don't have to be in any specific UML format, but make sure you can explain stuff relating to the diagram you have drawn. For example if you are working in a web application show how the data is flow from UI to DB. You can show different layers involved, technologies used etc.. The most important part is you should be clear in your mind about what you are currently working on. 

4. Convert arguments to conversation

Even if you know that that person is wrong do not argue and try to continue the conversation saying "Ok, But I am not so sure if that is correct, I will check that out". This keeps the person in good terms. Be an active listener during the interview use reference to your experience when you are answering. 

5. Be prepared for the WHY question

Good interviews focus on the question "Why?". It might start with "What" but will end in "Why?". For example in Java typical question would be "What is the difference between String and StringBuffer?". A follow-up why question will be like "Why is String has so-and-so" or "How is it done..?". Be ready to give inside information by answering "How?" and "Why" parts of he question.

6. Tell about your best achievement

During your work there might be something that you consider as your best achievement. It is important to describe it in such a way that interviewer feels that you have did something extraordinary there. So, prepare a believable story on how your abilities helped you complete that task. It is important to prepare this because it takes time to dig your memory and find situations.

7. Do you have any questions for me?

This question gets repeated in every single interview. Use this space to get to know more about the organization style and role you have been interviewed for. This article will help you in this.

Update:
Here is a gem of an article and I could not stop myself from adding it here.

Java Memory Profiling Simplified

As a typical Java developer I never monitored the memory usage of my application apart from following typical best practices like closing the connections, streams etc.. Recently we were struck with few issues in our JBoss servers that I had to dig in to the memory management. 

One of the best thing in java is that the developers are not required to handle the memory allocations when the objects are created. The JVM does that for us. Most of the time we just need the outer layer knowledge of heap memory and garbage collector. I will share some really interesting investigations that I had. I is a huge topic and I am writing from a point of view of a web application developer, what all minimum I thought we should understand about it.

The Tools

There are number of good tools available to profile the java applications some of them are,

1. Your Kit Java Profiler
2. JProfiler 
3. Eclipse MAT
4. Visual VM

Out of these Your Kit and JProfilers needs licences and others are free to use products. We are going to use VisualVM. It is a simple yet powerful tool and comes bundled inside the JDK. It has power list of plugins that you can download and use. To start using VisualVM, go to your <JDK_HOME>\bin and run the jvisualvm.exe. I found the below articles it useful to get going. 


1.Profiling With VisualVM
2.VisualVM performance tuning tool
3.How to Get VisualVM to Profile JBoss Without Crashing


Since we are talking about memory here make sure you install the Visual GC plugin on the VisualVM as told in this article.

Setting the stage - JVM Memory Structure

The JVM Memory is divided in to 3 parts as shown in the below image. In our applications we are concerned about the Heap Memory. We can input this value to JVM using the parameters,


-Xmx<size> - to set the maximum Java heap size
-Xms<size> - to set the initial Java heap size

JVM Memory Structure

The non-Heap memory stores per-class structures such as runtime constant pool, field and method data, and the code for methods and constructors, as well as interned Strings.

Here is a nice article with more details on the JVM memory sizes. Read Javin's article on JVM Heap space here. 

The one common confusion is about the stack memory and heap memory. This is well explained here.Stack values only exist within the scope of the function they are created in. Once it returns, they are discarded. Java only stores primitives on the stack. This keeps the stack small and helps keeping individual stack frames small, thus allowing more nested calls. Objects are created on the heap, and only references (which in turn are primitives) are passed around on the stack.

Now, Lets get real. Given below the image from Visual GC, a plugin inside the VisualVM as told earlier. We see many graphs here a detailed decription of the output is available here. 

The Game begins - What happens when the application runs

When the objects are created, they reside inside the Eden. When the Garbage collector(GC) runs, if the object is dead (means they are no active references) it is flushed out otherwise they are moved to S1(Survivor Space 1) or S2. This is called a GC cycle. Internal GM algorithm decides the frequency of the GC cycle. Eden + S1 + S2 section of Heap memory is called as Young generation. Objects that survives a fixed number of GC cycles are moved in to Old Gen space. Most number of java objects die as infant and never reach Old Gen. This typically includes local variables which are flushed after the methods are executed.

The frequency of GC cycles inside the Old Gen is much lesser than Young Gen. Typical examples of Old Gen objects are singletons, cached objects and other application wide used data.


When things do not go as per the plan

In a typical application there will be less variation inside the Old Gen space. If the Old Gen space grows linearly with time even after the GC cycle that would lead to a OutOfMemoryError. This might be a indication of a memory leak inside the code. However we might need to use a profiler to find out the exact reason for the same. Here is a Dzon article on some of the Causes of Java EE Enterprise Performance Problems.


These are the basic building blocks of how JVM memory is organized and reacts when the application is executed. From this point there are lots of topics including tuning the memory parameters and garbage collector. I will add some of the useful resources related to this.

1. Java Performance Tuning, Profiling, and Memory Management
2. InfoQ Presentation : Diagnosing Web Application OutOfMemoryErrors
3. InfoQ Presentation : Everything I Ever Learned about JVM Performance Tuning @twitter
4. InfoQ Presentation : Extreme Performance Java  
5. Java theory and practice: Garbage collection and performance
6. Understanding Java Garbage Collection

Update on 27 Sept 2013:

More on JVM Garbage Collection

Recently I have given a talk at Eclipse Day India event, I am adding the slides from the talk and details of the Demo here so that you can make use of it.

Steps for the Demo:
If you use eclipse and test the Visual GC plugin with some controlled load in the memory, download the plugin and put it under the "dropins" folder in eclipse. Restart your eclipse, you should be getting a new item in the Menu bar "Memory Tester". By clicking the Click on "Add Memory Load" you can load some objects in the memory and observe the behavior of Visual GC.

I have recently presented the content of this article and there are the slides!

JVM Garbage Collection Explained from Manu Pk

All about Hibernate Second Level Cache

Recently I have experimented with hibernate cache. In this post I would like share my experience and point out some of the details of Hibernate Second Level Cache. On the way I will direct you to some articles that helped me implement the cache. Let's get started from the ground.

Caching in hibernate

Caching functionality is designed to reduces the amount of necessary database access. When the objects are cached they resides in memory. You have the flexibility to limit the usage of memory and store the items in disk storage.The implementation will depend on the underlying cache manager. There are various flavors of caching available, but is better to cache non-transactional and read-only data.

Hibernate provides 3 types of caching.

1. Session Cache

    The session cache caches object within the current session. It is enabled by default in Hibernate. Read more about Session Cache. Objects in the session cache resides in the same memory location.

2. Second Level Cache

     The second level cache is responsible for caching objects across sessions. When this is turned on, objects will be first searched in cache and if they are not found, a database query will be fired. Read here on how to implement Second Level Cache. Second level cache will be used when the objects are loaded using their primary key. This includes fetching of associations. In case of second level cache the objects are constructed and hence all of them will reside in different memory locations.

3. Query Cache

Query Cache is used to cache the results of a query. Read here on how to implement query cache.When the query cache is turned on, the results of the query are stored against the combination query and parameters. Every time the query is fired the cache manager  checks for the combination of parameters and query. If the results are found in the cache they are returned other wise a database transaction is initiated.  As you can see, it is not a good idea to cache a query if it has number of parameters or a single parameter can take number of values. For each of this combination the results are stored in the memory. This  can lead to extensive memory usage.

Finally, here is a list of good articles written on this topic, 

1. Speed Up Your Hibernate Applications with Second-Level Caching
2. Hibernate: Truly Understanding the Second-Level and Query Caches
3. EhCache Integration with Spring and Hibernate. Step by Step Tutorial
4. Configuring Ehcache with hibernate

Doing more as a Programmer

In this post I want to write about my first talk titled Doing more as a Programmer. This presentation is about maximizing your value as a programmer by enhancing the way you interact with yourself, your surroundings, with community and to your career.


I have attached the presentation here, it is available on SlideShare.

Doing more as a Programmer

I had fun presenting it in my office, hope you will enjoy it too.