Why You Shouldn’t Use Complex Objects as HashMap Keys

I’m a big believer in learning from my mistakes, but I’m an even bigger believer in learning from other people’s mistakes.  Hopefully someone else will be able to learn from my mistakes.

mistakes

 

This post is inspired by an issue that took me a number of days to track down and pin point the root cause.  It started with NullPointerExceptions randomly be thrown in one of my applications.  I wasn’t able to consistently replicate the issue, so I added an indiscriminate amount of logging to the code to see if I could track down what was going on.

What I found was that when I was attempting to pull a value out of a particular hashmap, the value would sometimes be Null, which was a bit puzzling because after initializing the map with the keys/values, there were no more calls to put(), only calls to get(), so there should have been no opportunity to put a null value in the map.

Below is a code snippet similar (but far more concise) to the one I had been working on.

There is a ProductSummaryBean with a short summary of the product, and a ProductDetailBean with further product details.  The summary bean is below and contains four properties.

 

Any guesses what happens when the code above is run?

Exception in thread "main" java.lang.NullPointerException
 at com.lynden.mapdemo.TestClass.runTest(TestClass.java:34)
 at com.lynden.mapdemo.TestClass.main(TestClass.java:50)

 

So what happened?  The HashMap stores its keys by using the hashcode of the key objects.  If we print out the hashcode when the ProductSummaryBean is first created and also after its read out of the DB we get the following.

SummaryBean hashcode before: -298224643
SummaryBean hashcode after: -298224679

 

We  can see that the hashcode before and after are different, so there must be something different about the two objects.

SummaryBean before: ProductBean{priceFormatter=java.text.DecimalFormat@67500, price=19.95, name=MyWidget, upcCode=Z332332}
SummaryBean after: ProductBean{priceFormatter=java.text.DecimalFormat@674dc, price=19.95, name=MyWidget, upcCode=Z332332}

 

Printing out the entire objects shows that while name, upc code, and price are all the same, the DecimalFormatter used for the price is different.  Since the DecimalFormatter is part of the hashcode() calculation for the ProductSummaryBean, the hashcodes between the before and after versions of the bean turned out different.  Since the hashcode was modified, the map was not able to find the corresponding ProductDetailBean which in turned caused the NullPointerException.

Now one may ask, should the DecimalFormat object in the bean been used as part of the equals() and hashcode() calculations?  In this case, probably not, but this may not be true in your case.  The safer way to go for the hashmap key would be to have used the product’s upc code as the HashMap key to avoid the danger of the keys changing unexpectedly.

 

 

 

 

 

Advertisements

Automatically Cloning a Wildfly Instance Using Chef

As we have started moving to a service based architecture, we have been developing processes to create and configure our infrastructure in a predictable and repeatable way using Vagrant and Chef.  One challenge that we have faced is trying to replicate a production Wildfly server on a dev box, including the applications are installed on it and their correct versions.

Ideally, we’d like the developer to be able to specify which server they want to clone when kicking off the Chef process.  Chef would then create a new Wildfly instance and download and install all the web applications running on the specified instance.

The first question Chef will need to know, is “what what Wildfly servers are running on the network?”  The next question is then, “which applications, and what versions are installed on those servers?”

In order to answer these questions, we developed a “WildflyMonitor” web application which is installed on each of our Wildfly instances.  The application will collect information about the local Wildfly instance that it’s running on, including the names and versions of the hosted apps, and publish that information to our messaging system.  This information eventually makes it into our Wildfly Registry DB, where it is collected and organized by Wildfly instance.

A rough diagram of the architecture appears below.

WildflyRegistry

In the example, there are 3 Wildfly instances, lisprod01, 02, and 03, which are reporting their applications to the registry. The table below the DB illustrates how the data is organized by server, and then by applications, with each Wildfly instance is running 2 applications.    The WildflyRegistry REST service then makes this information available to any client on the network, including Chef recipes.

 

The next step is then to modify the Chef script

The snippet above shows the script contacting the REST service, looping through all the servers that were returned until the desired server to clone is found.  Once the server is found, the script loops through that server’s list of applications and creates a list of hashes with the app name mapped to its version number.

Next, the script loops through each of the apps which were discovered in the previous snippet.

 

First the script constructs the URL to the web app in our Nexus repository. The script then downloads each web app to the tmp folder on the server.  The script then calls a shell script which deploys the applications to Wildfly utilizing the Wildfly command line interface.

The shell script which is called by Chef to perform the actual deployment to Wildfly is fairly straightforward and appears below.

 

That’s it, based on the data in our WildflyRegistry we are able to use this Chef script and shell script to create a clone of an existing Wildfly instance running on our network.

 

 

Developing Trading Applications with the SumZero Trading API

I have open sourced a Java trading library which I have been using to develop automated trading applications for many years.  The SumZero Trading API provides the ability to develop trading applications for the equity, futures, and currency markets, by utilizing the following sub APIs

  • Market Data API – Request real time Level 1 (NBBO) and Level 2 (Market Depth) market data
  • Broker API – Submit, execute, and monitor orders
  • Historical Data API – Request intraday and end-of-day historical market data.
  • Strategy API – Develop trading strategies to automatically place buy/sell orders based on user defined algorithms.

The library includes implementation of all of these APIs for Interactive Brokers, except for the Broker API, which also has an implementation for Quantitative Brokers.

The libraries are licensed under the MIT open source license and source code is available at:
https://github.com/rterp/SumZeroTrading

In future posts I will show how easy it is to connect to Interactive Brokers to request real-time market data and place a trade using the API.

twitter: @RobTerpilowski
twitter: @SumZeroTrading

Using Apache Camel and ActiveMQ to Implement Synchronous Request/Response.

Implementing a synchronous request/response pattern with Apache Camel and ActiveMQ is quite a bit easier than you may expected, and has allowed us to leverage our current messaging infrastructure to facilitate synchronous exchanges between applications where we otherwise may have needed to create a new web service.

Below is an example of setting up two Camel endpoints which will demonstrate the request/response pattern.

First, configure the connection to the JMS message broker.  In this case, an ActiveMQ broker is created in-process.

 

Next, set up the producer route.  First a processor is created, which will print the body of the message.  The route will be executed when a file is dropped into the /Users/RobTerpilowski/tmp/in directory, and routed to the robt.test.queue destination.  Once the route has completed, the processor will be executed.  What we are hoping to see is that the message has been modified by the consuming endpoint when this route has completed.  The important piece to note here is the url:
jms-broker:queue:robt.test.queue?exchangePattern=InOut
exchangePattern=InOut tells Camel that the route is a syncrhonous request/response

 

Next, set up the consumer endpoint.  Again, a processor is created which will be run when the route has completed.  This processor will first print the message that the producer sent.  It will then replace the message with a new message saying that the original message was seen.  This endpoint will listen on the robt.test.queue and route the result to the directory /Users/RobTerpilowski/tmp/out.  When the route has completed, the processor will update the message.  If everything works correctly, the producer endpoint should be able to see the modified message.

So now that the routes are set up, it’s time to send a message.  Saving a file in the specified input directory will kick things off.  The file will contain the text “HelloCamel”.

 

$ echo "HelloCamel" > /Users/RobTerpilowski/tmp/in/message.txt

The consumer listening on the robt.test.queue immediately sees the message arrives, and prints the message body.

CONSUMER received message: HelloCamel

 

The producer endpoint then receives the modified message back, with confirmation that the consumer endpoint did indeed see the message.

PRODUCER Received response: I Saw it!!! It contained: HelloCamel

 

Make Sure Your Server Clocks are in Sync!

As I finished my first test services that would utilize the Request/Response pattern I created an integration test where they connected to messaging broker that was running in-process.  Things looked great, and the services were communicating without any issues.  I deployed the services to a Wildfly instance running locally, which were pointing at a messaging broker on our staging server.  However, this time when I started my test, the requests were consistently timing out, and never making it back from the second service.  I literally spent the entire day deconstructing each service piece by piece to see what was going on.  I then remembered something about clock synchronization in the Camel JMS documentation.  I checked both the clock on the VM and the clock on the staging server and proceeded to do a face palm when I saw there was a four hour difference, lesson learned!

 

twitter: @RobTerpilowski

Using Dependency Injection in a Java SE Application

It would be nice to decouple components in client applications the way that we have become accustom to doing in server side applications and providing a way to use mock implementations for unit testing.

Fortunately it is fairly straightforward to configure a Java SE client application to use a dependency injection framework such as Weld.

The first step is to include the weld-se jar as a dependency in your project. The weld-se jar is basically the Weld framework repackaged along with its other dependencies as a single jar file which is about 4MB.

    <dependency>
        <groupId>org.jboss.weld.se</groupId>
        <artifactId>weld-se</artifactId>
        <version>2.2.11.Final</version>
    </dependency>

 

Implement a singleton which will create and initialize the Weld container and provide a method to access a bean from the container.

import org.jboss.weld.environment.se.Weld;
import org.jboss.weld.environment.se.WeldContainer;


public class CdiContext {

    public static final CdiContext INSTANCE = new CdiContext();

    private final Weld weld;
    private final WeldContainer container;

    private CdiContext() {
        this.weld = new Weld();
        this.container = weld.initialize();
        Runtime.getRuntime().addShutdownHook(new Thread() {
            @Override
            public void run() {
                weld.shutdown();
            }
        });
    }

    public <T> T getBean(Class<T> type) {
        return container.instance().select(type).get();
    }
}

 

Once you have the context you can then use it to instantiate a bean which in turn will inject any dependencies into the bean.

import java.util.HashMap;
import java.util.Map;


public class MainClass {
    protected String baseDir;
    protected String wldFileLocation;
    protected String dataFileDir;
    protected int timeInterval = 15;
    protected String outputFileDir;

    public void run() throws Exception {
        CdiContext context = CdiContext.INSTANCE;

        //Get an isntance of the bean from the context
        IMatcher matcher = context.getBean(IMatcher.class);

        matcher.setCommodityTradeTimeMap( getDateTranslations(1, "6:30:00 AM", "6:35:00 AM", "6:45:00 AM") );

        matcher.matchTrades(wldFileLocation, dataFileDir, timeInterval, outputFileDir);

    }

What is great is that there are no annotations required on the interfaces or their implementing classes. Weld will automatically find the implementation and inject it in the class where defined. ie. there were no annotations required on the IDataFileReader interface or its implementing classes in order to @Inject it into the Matcher class. Likewise neither the IMatcher interface nor the Matcher class require annotations in order to be instantiated by the CdiContext above.

public class Matcher implements IMatcher {

    //Framework will automatically find and inject
    //an implementation of IDataFileReader

    @Inject
    protected IDataFileReader dataFileReader;

twitter: @RobTerpilowski
@LimitUpTrading

JPA java.lang.IllegalArgumentException: No query defined for that name (Solved)

I’ve recently been working with the [Camel JPA] (http://camel.apache.org/jpa.html) component for moving data from one of our SQL servers to our messaging system.

We have a number of Entity POJOs defined that contain a named query which the JPA component uses in order to query the database to select the appropriate records that need to be processed. Everything was working great and I decided to move these beans to a separate library that could be shared with other applications. However, once I did this the original application started encountering the following error.

java.lang.IllegalArgumentException: No query defined for that name [AllinboundMessagesSqlBean.findByProcessed]

I checked the classpath and the beans were in fact being found, but the named queries on the beans were not being found. It took some research, but the solution to the problem ended up proving to be very simple.

The change was to explicitly add the entity classes to the application’s persistance.xml

com.lynden.json.beans.AllinboundLoopBean
com.lynden.json.beans.AllinboundMessagesSqlBean

Once the classes were defined in the file, the app was then able to find these entity beans that were in a separate .jar. Hopefully this will help others out there who may have run into a similar issue.

twitter: @RobTerpilowski

ClassCastException on Hibernate 4.3.x and Glassfish 4.x

I am attempting to utilize Hibernate 4.3.8 in a service that I am creating that will be running on Glassfish 4.1. When I attempt to

read an object from the db, such as the example below:

Product product = entityManager.find(Product.class, 980001);

The following exception is thrown

java.lang.ClassCastException: com.lynden.allin.service.Product cannot be cast to com.lynden.allin.service.Product

At first glance this may seem a bit strange since the 2 classes appear identical, and they are, but the issue is that there are 2 instances of the class being loaded by different classloaders. When the entityManager attempts to cast the class, it grabs a version that the service itself doesn’t know about since the service the reference that the service has was created from a different class loader.

After some searching it appears that this is a know issue with Hibernate 4.3.6 and newer:

https://hibernate.atlassian.net/browse/HHH-9446

The solution for the time being is to downgrade hibernate to 4.3.5 in order to avoid this issue in Glassfish.

twitter: @RobTerpilowski