Simple SOAP client

I needed a quick and simple SOAP client to verify some odd return values from a web service. I also wanted the client to be able to send and receive badly formatted XML for testing purposes, and preferably pretty print the xml. Didn't feel very appealing to try to do this in Java, so I decided to do it with JRuby.

Ruby already has the soap4r library, which given a wsdl url gives you convenient objects for performing requests. Unfortunately soap4r tries to retrieve the wsdl with a http GET, and the service is configured to only respond to POST (yes, even for wsdl request). So I decided to go directly to the HTTPClient. Here it is, including pretty printing the output, if someone else finds it useful.

require 'rubygems'
require 'httpclient'

soapbody = <<-eos
<soap:Envelope>
  ** your soapy stuff goes here **
</soap:Envelope>
eos

header = {'Content-Type' => 'application/soap+xml'}
endpoint = "http://localhost:8080/testService"

client = HTTPClient.new
#Uncomment next line if you want to see requests and responses as sent.
#client.debug_dev=STDOUT
result = client.post(endpoint, soapbody, header)

puts '== Content'
print result.body

There are some more sample usages of HTTPClient here, the class documentation is here.

If you have a valid xml reply and want to pretty print it, add:

require "jrexml"
require "rexml/document"
doc = REXML::Document.new result.body
doc.write( STDOUT, 3 )

Unconference at Mejsla

We've had another unconference day, this time at Mejsla and arranged by yours truly. Participants were me, Sven Johansson, Thomas Podal, Fredrik Rubensson, Mårten Gustafson and Henrik Engström.

Topics

Topics explored during the day included:

• the Erlang based key-value store Riak
• the cross-browser WebSocket Socket.IO
• the Scala based web framework Lift,
• JavaScript as a backend language, and especially its asynchronous node.js framework
• the graph database Neo4J
• Google's Guava libraries
• the JVM based Lisp-like Clojure
• getting started with Ruby + Rails + Heroku
• the experimental Mirah language
• JQuery.

Blocks

A slow wireless network and the time it took to get to a "Hello World" state usually meant that we only had time to do a quick spike for each topic. The day was still very interesting, and we learned enough to see what might be worth to pursue further, but for the next unconference I think we will create a Wiki or similar so we can collect topic suggestions in advance. Then we can see which topics have the most interest in, and people who are sure they are going to spend time on a topic can download tools and libraries in advance. That way they can make sure they have passed the 1st compile and "hello world" steps, and maybe even have prepared a few questions or demos if they feel like it.

Lift

Two topics I participated in but which were examples of these "high overhead" tasks, were node.js and Lift. For Lift, I started downloading and installing the Scala language, but then discovered that the easiest way to get started was instead to download the sbt - the Simple Build Tool jar from Google, create a shell script on path to execute the jar, and then run

sbt update ~jetty-run

on an empty Lift project. This will download Scala, Lift, and all the dependent jars from the nearest Maven repository, then start the small Jetty server. Lift detected file changes live, which enabled a quick cycle of modifications and then reloading pages to instantly see the changes. Unfortunately Lift crashed with an OutOfMemoryError:PermGen exception after just a few changes to a html page in the "static" directory. PermGen problems is an annoyance for most alternative languages on the JVM, but this didn't give a very stable impression. After a restart things were much more stable and we couldn't reproduce the error, so it may have been an effect of the large number of initial downloads to get the system started the first time.

node.js

A lot of the time was spent on building node.js itself with "make", so we didn't get very far. But at least we got the "asynchronous web server in 10 lines" up and running. Again I was impressed with IntelliJ though, which had nice support for JavaScript editing. If I'm going to continue using fully featured IDEs, I feel I really should learn to use IntelliJ, and purchase it.

Mirah

Mirah is a very interesting language. We got it up and running very quickly (though if we hadn't had a working JRuby and Git set up it might have been a different story). Syntactically the language resembles Ruby very much, but it turns out it is actually statically typed and compiled. The types are generally inferred (like in Scala) for uncluttered code and a minimal amount of typing. We only created a few Java classes which can be imported and used in Java projects, but apparently you should be able to plug in different compiler backends so that you can emit... anything you may like. Source for other languages, Dalvik bytecode, or whatever. The mappings to Java bytecode seemed to work fine, we only had trouble with creating a "void" method. I'm going to experiment a lot more with this language, expect blog posts and hopefully a lightning speech together with Peter Lind at JFokus 2011.

Guava

This was a revelation for me. I had heard it mentioned in passing before, on a presentation by Enno Runne for instance, but it was one in an ever growing list of "things I ought to spend some time on". It turned out that these libraries are something that will help me enormously in my day-to-day Java programming:

• A @VisibleForTesting annotation is helpful for documentation, as I very often relax visibilities on helper methods to enable unit testing on them, this will eliminate a lot of boilerplate javadocs.
• Classes that add functional style methods for manipulating collections (given the syntactical and semantical restrictions of the Java language and platform of course).
• Helper methods for primitives and arrays, which will be extremely helpful for me to clean up a library which is written in C style and does nothing but manipulate bytes and byte arrays.
• Additions to the java.util.concurrent package, including a ListenableFuture that you can chain together for asynchronous workflows.

These libraries every Java developer should be aware of. Big thanks to Henrik who had prepared some demos on this in advance. This was not expected of the participants, but it was very helpful.

Ruby, Rails and Heroku

A quick introduction to the Ruby language to those who were unfamiliar with it. We then generated a Rails project and I explained the structure of it, what could be done with the scripts, and so on. We then went on to what we thought would be the meat of the session, Heroku. Only, it took less than 5 minutes to integrate Git and Heroku. Pushing changes to Git made Heroku update, autodetect that it was a Rails project, download dependencies and start up an instance of our application in their cloud. Jaw-droppingly simple, as is often the case with Ruby. We then went back and did some more Ruby and Rails programming. Sven, whom I've had a hard time convincing about the excellence of Ruby, seemed to be on the way to becoming a convert.


The future, and books about it

Towards the end of the day we were mentally exhausted and instead of doing another programming session we sat together and talked about good books, fictional and not. The only titles I remember were Kent Beck's Implementation Patterns, The Human Body 2.0, Charles Stross' Accelerando, and Vernor Vinge's Rainbow's End.


What we didn't have time for

Topics that remained on the board, but which we hopefully will be able to do on another day:

• Testing Java with Ruby's excellent testing tools through JTestR
• Ruby web framework Sinatra + HAML markup.
  
• Getting started with Python
• Doing the Software Craftsmanship Ruby refactoring excercises from SC 2010
• Amazon's EC2 cloud
• Massive scaling with MOM, especially ActiveMQ
• Getting started with event-driven design and Actors with Akka
• HTML 5
• Devops and OpsCode Chef
• Java 2.0 discussion. (Presumably means - next Java version).
• Cucumber and Cuke4Duke

Summary

Big thanks to all the participants for making this an excellent day, it was a privilege to learn from and with you.

Unconference

Unconference day

A few weeks ago I was at a learning event arranged by Måns Sandström from Adaptiv together with Ola Ellnestam from Agical. It was called "Learning the learned". The people who arranged it had noticed that senior developers have difficulty finding worthwhile learning events. Courses are often very expensive and is often on such a basic level that any of the participants could have been in charge of the course themselves if given a day to prepare. Conference presentations, while inspiring, are often too short to go into any greater depth, and have to take into consideration that many in the audience may be juniors.

So the solution was to have the senior developers teach each others. The format was inspired by the "unconference" or "Open Spaces" concepts that you may have seen from conferences. In other words, instead of having one person in charge and many passive listeners, people are encouraged to self organize in groups around topics that interest them and then discuss, pair program, draw on whiteboards, or whatever they can come up with. Måns and Ola invited a number of people who they thought had valuable things to contribute to such a day. After a few unfortunate last minute cancellations we were 7 participants at the start of the day.

At the beginning of the day, each participant wrote down a short sentence on a sticky note for every topic he* found interesting and wanted to explore. We put the notes up on a whiteboard in turn and explained a little more in depth what each topic was about. We then wrote our names on every note that sounded interesting. Then we picked two notes with the highest number of votes and formed two groups. One group wanted to explore "Testing frameworks for functional languages", and my group wanted to discuss "Unit testing beyond 'assertEquals'". We discussed how it feels like there are often two sources of non-DRY code when writing tests in Java - test data setup, and secondly the assertion lines themselves (so yes, pretty much all of it, unfortunately). For test data setup we quickly discussed Object Mother and Data Builder patterns, and mock frameworks like Mockito. For the assertions parts, I learned for the first time about Matchers (hamcrest to be specific) and the assertThat method. The combination of the two creates very readable error messages and tests that can be quickly and cleanly written by combining predicating matchers. At the end of the session we did a quick recap of what we had learned for the other group. As people came up with new topic ideas, new sticky notes went up on the board during the day.

Second and third sessions I spent doing the first few exercises of the excellent 15 exercises to know a programming language using Clojure, together with Ola Ellnestam and Peter Hultgren who has since founded the Stockholm Clojure User Group. At the final session of the day people were pretty mentally exhausted, and instead of programming we all did a session together where we discussed "Books every developer should read" - programming related and not.

In between sessions Ola was in charge of short exercises that were either medition and focusing exercises, or little games designed to make the participants get to know each other better. These were great fun, and helped greately to keep concentration up.

For me this was a fantastic day. I learned much more than I have at the conferences I usually attend, plus I got to know new interesting people. Before going I was a bit nervous - the topics I had come up with felt "too easy" - stuff a senior programmer should already know. But considering how expansive our field is (and growing every day), no one can be an expert at everything, so this was really something I needn't have worried about (and neither should you, if you decide to go to something similar).

I think that if I could go to something like this once a month, it would accelerate my learning and my career enormously. Hopefully this will become a regular event, the participants all agreed we should try to do it at least once every couple of months. And there is nothing stopping you from arranging a similar event, the concept is not trademarked or anything (though the people who arranged this day say mentioning them as a source of inspiration is appreciated).



* Only guys present this time unfortunately. Sad that gender imbalance in our sector is actually growing! But that is a topic for another post...

Load log4j.properties from jar file in webapp

Having environment specific log4j.properties inside a jar file is hardly an ideal solution, but if you find yourself in a similar situation:

import java.net.URL;
import org.apache.log4j.PropertyConfigurator;
import javax.servlet.http.HttpServlet;

public class Log4jConfLoad extends HttpServlet {

  @Override
  public void init() {
    try {
      String prefix = "jar:file:" + getServletContext().getRealPath("/");
      String path = "WEB-INF/lib/the.jar!/" + System.getProperty("env") + "/log4j.properties";
      PropertyConfigurator.configure(new URL(prefix+path));
    catch (Exception ex) {
      //Because Servlet init can't be declared as throwing new exception.
      //Any thrown Exception here should show up in catalina.out (if Tomcat used)
      throw new RuntimeException(ex);
    }
  }
}

Also remember to add the servlet to web.xml with load-on-startup.

"That" Java example and preaching to the converted

There is one example of Java code that keeps coming up. I have seen variants of it at conference presentations several times, in Venkat Subramaniam's Programming Groovy and most recently in an essay by Neal Ford in The ThoughtWorks Anthology. This example compares Java with a scripting language, and goes usually something like this (here I combine all the worst practices from the books in one example):

JAVA VERSION:

import java.io.*;

public class ReadFile
{

public ReadFile(String path)
{
    BufferedReader reader = null;
     try
     {
         reader = new BufferedReader(
                                 new FileReader(path));
         String line = null;
         while((line = reader.readLine()) != null)
         {
               System.out.println(line);
         }
      }
     catch(FileNotFoundException ex)
     {
          ex.printStackTrace();
     }
     catch(IOException ex)
     {
          ex.printStackTrace();
     }
     finally
     {
          try
          {
            if (reader!=null)
            {
                reader.close();
            }
          }
          catch (IOException ex)
          {
           //Ignore
          }
    }
}

 public static void main(String[] args)
 {
  new ReadFile("thoreau.txt");
 }
}

GROOVY VERSION:

println new File('thoreau.txt').text

Wow, 1 line instead of almost 50! Java must really suck! The problem with this is that the Groovy version is obviously just a one-off, there is no attempt to make a reusable module or do robust error handling. If there is an error, the program just terminates with en error message. We can do exactly the same with the Java version and cut down the number of lines significantly if we just throw the exception. We can reduce the number of lines even more if we use a "modern" (1.5) API and the Java coding conventions for where to place brackets, etc:

import java.io.File;
import java.util.Scanner;

public class ReadFile {

    public static void main(String[] args) throws Exception {
        File in = new File("thoreu.txt");
        String contents = new Scanner(in).useDelimiter("\\Z").next();
        System.out.println(contents);
    }

}

It can be made a one or two lines shorter still, (at the price of a dependency on a big jar) if you are are willing to use an external library such as Apache Commons IO. Or I could have squeezed the three lines of logic into one, but I think this is a reasonable trade off between readability and verbosity. But this new and improved example is still much longer than the Ruby/Groovy/Scala/etc versions (especially if you count characters instead of lines); and arguably less clear (What's a scanner? What does "\\z" mean?).

The problem with using the original Java example is that will mainly convince those who are firmly in the "Yeah, Java totally sucks" camp. Even a very junior Java developer, who is curious about learning about a new language will see that the original example is either
a) Written by a rank amateur (unlikely considering the caliber of the authors) Or -
b) Wildly exaggerated to make Java look as bad as possible.
And that will make them less receptive to the text as a whole. With the number of active Java programmers probably (still) in the hundreds of thousands, alternative languages have a large number of possible recruits and allies. I would prefer to see a better example to convince them that it is worthwhile to look at other languages.

Pragmatic argumentation

I read Chad Fowler's The Passionate Programmer this weekend. All in all a pretty good book which I recommend. Remaining from the previous edition however is one of my least favorite paragraphs in the book.

In his essay "Great Hackers", Paul Graham annoyed the industry with the assertion that Java programmers aren't as smart as Python programmers. He made a lot of stupid Java programmers mad (did I say that?), causing a lot of them to write counterarguments on their websites. The violent reaction indicates that he touched a nerve.

So if, purely for the sake of argument mind, I claimed that "Chad Fowler is a racist" (he's not) and he got really upset about it, his reaction is an indication that I am on to something? No, only if you are on the level of a tabloid journalist. If you come with a hurtful accusation and people get really upset, that is an indication you are being a dick.

Notes on dataflow concurrency

At a conference, I heard someone say something to the effect of: "Declarative dataflow concurrency is awesome. You can't get deadlocks! It all just works, and it is so beautiful".

If you are unfamiliar with dataflow concurrency, it is a technique used in some languages, where a program, if it reaches a statement that requires the use of a variable which hasn't been assigned a value yet, pauses until the variable has been assigned by another thread and then continues the computation as if nothing had happened.

It is pretty neat, but I thought to myself - "Hang on, if we have two threads that each wait for the other to assign a variable, won't we get a circular dependency between them? Or if there is just one executing thread, won't that block forever? I don't understand, that doesn't sound like much different from what I'm used to...?"

Now a month later, I am currently reading the book "Concepts, Techniques and Models of Computer Programming" by Ray and Haridi. And indeed, on page 48 they state:

The computation models of this book use [dataflow programming]. This is unreasonable in a sequential system, since the program will wait forever. It is reasonable in a concurrent system, where it could be part of normal operation that some other thread binds the varialbe. [Dataflow programming] introduces a new kind of program error, namely a suspension that waits forever. For example, if a variable name is misspelled then it will never be bound. A good debugger should detect when this occurs.

So even if they wait on an infinitely unbound variable instead of lock that is never released, the experience for the user would appear to be much the same - a program that "hangs" without producing the desired result. I wrote this little snipped in Mozart to demonstrate:

declare

Y

X

thread X=Y+1 end

Y=X+1 end

{Browse X}

And indeed the browse window never appears. The main thread blocks on line 5, the spawned thread on line 4, each waiting for an unbound variable.

But even though it hasn't been stated outright in the book (at least not yet, I'm at page 350), I think there is an important difference compared to threaded programs as they are usually written in, for instance, Java. The Mozart program is deterministic (if written in a completely declarative style). It will *always* block, no matter how the threads are scheduled, which makes this programming error easier to detect and (hopefully) to locate and fix.

So what the person at the conference perhaps meant, was that the programs written in dataflow concurrency style don't have *race conditions*.