Apache Velocity

Apache Velocity is a template engine that has become a "legend in its own time" (LIOT). The criteria for a LIOT is when a software enters the common parlance of programmers not associated with its original implementation language and ports begin to appear in other languages.  Some of its application areas include templating automated emails and source code generation. There are a number of projects "powered by Velocity".

Hadoop, the MapReduce Paradigm and the Corresponding Mindset Change

The latest rage in the Java universe is Hadoop, a platform for "Big Data" processing. A skew of O'Reilly books have been published on the subject. It implements the MapReduce paradigm popularized by Google.

MapReduce is something inspired by LISP. A Map takes a function and a list of arguments and applies the function to each one of the arguments. A Reduce operation combines data points into one value using a binary operation.

So a "MapReduce" operation consists of 1) applying a function to a list, to generate a new list, 2) combine the elements of the list, using some binary operator (which could be simple addition, or something more complex, like XOR) to produce a single value.  Many algorithms can be expressed using this paradigm.

You can see how this can speedup parallelisable tasks. E.g. A word count on a huge file can be mapped onto different machines and results collated via Reduce.

It is a simple divide and conquer model for processing data in a parallel fashion.

It is the model used by Google to achieve massively parallel processing.

This type of computing, though, requires an altogether different mindset. Whereas previously we were designing programs for single machines, or rather single processor machines, we now need to create algorithms that work in the multiprocessor/multimachine context - parallel algorithms.

Awesome IO with Apache Commons

Awesome IO with Apache commons starts with org.apache.commons.io. Among the most useful aspects of this package are the static utility methods provided by IOUtils.

Hacking apache collections

Once you get the jar added to your project, you can start cruising the javadoc and seeing what you can make of the Apache collections classes (and interfaces, mind you).

This will take you a few steps beyond the humble plateau of plain-vanilla Maps and Lists, by introducing variations of Sets and different types of Maps, designed for special scenarios. But before you ascend to the Java data structure stratosphere, you might want to recap your knowledge on the basic interfaces in the Collections framework, such as the Map interface, and the List interface. An interesting point of comparison, is that while the Map interface has no superinterfaces, the List interface is superinterface'd by the Collection interface and the Iterable interface by extension, since Collections are by necessity Iterable.

Perhaps, you are interested in the numerous implementations of Bags (sets that allow repeat elements e.g. two pairs of socks in a gym bag), and wander over to TreeBag or HashBag (now deprecated). Also fascinating are the various implementations of the BidiMap interface such as TreeBidiMap, that allows lookup from keys to values and values to keys- with equal efficiency. It is more storage efficient than using two separate TreeMaps, although the DualTreeBidiMap does use this approach.

Remember always the distinction between interfaces and classes, do not try to create an object of type Bag by instantiating Bag, but rather create an instance of TreeBag (but not HashBag) i.e. instantiate the implementing classes (but not deprecated ones!)

Apache Commons - A Staple in the Java Open Source World

Apache Commons is a staple in the Java open source world.

commons-beanutils.jar (contains everything to do with beanies)
commons-collections-3.2.1 (contains apache commons collections)
commons-lang-2.4.jar (contains additional classes for java.lang)
commons-logging.jar
commons-CLI.jar (command-line parsing)

There is also a Commons sandbox of projects-in-the-making. Amongst these are:

CLI2 - redesign of Commons CLI
CSV - module for reading/writing CSV
OpenPGP - for signing and verifying data using OpenPGP
Javaflow - to capture the state of an application. Means of "checpointing" an application using the idea of "continuations" borrowed from Scheme.

Commons dormant contains stuff no longer maintained, like Clazz for introspection, and Cache for object caching services.