Clear your cache in NetBeans

There’s a problem I’ve run into in NetBeans once or twice where NetBeans places red underlines signalling compile errors in your projects, but the project actually builds and runs fine.  It’s very irritating, and I’m not sure what causes it, but from what I’ve been able to find online it’s caused by a corrupted cache (the editor view thinks the file cannot be compiled due to some stale information).  To fix this, execute the following command in the terminal

rm -rf /path/to/netbeans/6.8/var/cache/

(replacing 6.8 with the version of Netbeans you’re running)

Interview with Twitter dev explaining use of Scala

So what were our criteria for choosing Scala? Well first we asked, was it fast, and fun, and good for long-running process? Does it have advanced features? Can you be productive quickly? Developers of the language itself had to be accessible to us as we’d been burned by Ruby in that respect. Ruby’s developers had been clear about focusing it on fun, even sometimes at the expense of performance. They understood our concerns about enterprise-class support and sometimes had other priorities.

We wanted to be able to talk to the guys building the language, not to steer the language, but at least to have a conversation with them.

There’s a new, good post interviewing one of the lead back-end engineers of Twitter.  I knew that they had replaced some Ruby code with Scala due to performance concerns, but there’s new information in here to me, such as all the systems it’s used in.  It also gives a great summary of Scala’s benefits as a language. He makes a good point that you can easily hire Java programmers and train them to use Scala; it’s a lot harder to teach/hire a Haskell programmer, for instance.

Well worth a read.

TextMate usability flaws

I’ve posted previously about TextMate, a great text editor for Mac OSX.  While it is one of the best text editors I’ve used due to its simplicity, elegance, and power, there are a few UI features that drive me up the wall, and make it fall short of being perfect.

Tab key behavior

Here I have a bunch of text selected.  What will happen when I hit tab?

If you answered, delete your selection and insert a literal tab, you win the prize.  What happens in Microsoft Word, NetBeans Platform, and any number of other IDEs and text editors?  The text is indented.  This is clearly the better alternative in my mind because it is non-destructive, and you can get the destructive tab with a simple addition of any delete/backspace key before hitting tab.

Furthermore, it’s inconsistent – if I had selected that text and hit quote, I’d have the text surrounded by quotes, not a single quote replacing all my test.  I would argue that it does more harm than good to

To indent the region, you have to do ⌘-]; to unindent you do ⌘-[.  I’m sure there’s a way to remap tab and shift tab to indent the region, but it’s not immediately clear to me how to do that.  If you know how, please comment so people can see.  Every time I switch between Netbeans and TextMate I invariably delete a whole region of text I meant to indent by forgetting to context switch and use ⌘-].  Fortunately TextMate has extremely good undo support, so it’s not the end of the world, but it’s that little bit of friction that slows down work and adds up to frustration over multiple instances.

EDIT: The following sneaks up on me all the time and also is very irksome.  Here’s a rather typical situation:

Before backwards indenting

Here I want to move the block of text backward; I resist the urge to hit shift tab, and instead press ⌘-[.  What happens?  Not what you’d think.

After hitting un-indent

I really cannot think of a valid reason for this behavior; I have the first line selected, but it does not respond to the indentation command.  Very frustrating.

Syntax highlighting with unsaved files

If you are working on an unsaved file, there is no way to tell TextMate to treat it as a file of a certain programming language, and thus which syntax highlighting rules apply.

Why is this irksome?  When I’m writing a blog post, for instance, I have a lot of snippets I compose in TextMate.  In order to get the syntax highlighting to work correctly, I need to save each individual file.

Now, I’m not suggesting TextMate is magical and can read my mind as to what ruleset to apply to an unsaved file.  What I’m suggesting is the addition of a menu that allows you to apply the syntax rules of a given language to the current file, regardless of whether it’s saved or not.  Notepad++, a Windows text editor, has precisely this feature.

EDIT: Thanks to the comment by Marshall, I realize that this complaint is way off base – I didn’t realize where the menu was but it’s there.  Right where it says “Plain text” in the above screen shot, click that and you get a full list of languages to treat the file as.

Tab support

Tab support is abysmal.  There is no support for closing tabs with the middle mouse button, a convention followed by most major web browsers, as well as NetBeans and Eclipse.  Instead, you are forced to click a small close icon on each tab.  Fitts’s Law states that the smaller a button is, the longer it will take a user to navigate to; this is part of the reason that Apple has adopted menu bars at the top of the screen rather than floating with each window.  Doing so gives the menu effectively infinite height, as the user can slam the mouse up and not go past it.  Long story short, having the only means of closing tabs be a tiny button is bad UI design.

I could forgive the lack of middle button close IF the tabs supported sensible context menus to close other tabs.  Compare for instance the options of some other products that support tabs:

Firefox’s popup menu on tab

When you right click a tab in TextMate, no context menu appears – there is absolutely no menu option I can find to close all tabs.  Any system providing tabs should allow the user to make a blank slate for himself and focus on one (or zero) files at a time.  As it stands, you must click each tiny x button individually on all the tabs.

Finally, you cannot undock tabs.  Again, maybe I’ve just been spoiled by other products like Adium, Firefox, and NetBeans, but I consider it a very important feature to be able to undock tabs to show two windows side by side.  While tabs are certainly more space efficient than tiling windows side by side, sometimes you really need to compare two files side by side.  TextMate makes it very difficult to do that, especially when you have opened a project rather than individual files.

Here is the workflow in FireFox:

Before drag is initiated on the tab

While tab is being dragged; note that a translucent version of the page follows the cursor to indicate what is happening

After the drop is completed, the tab is split off from the original window and becomes a separate frame.

Conclusion

TextMate is an excellent text editor but not without some usability flaws.  I’ve detailed some features that irritate me about TextMate, due to their violation of the principle of least astonishment; I’ve used enough other similar systems to expect certain functionality, and this expectation is violated in a few ways.  These ways include the fact that hitting tab while text is selected replaces the contents with a literal tab rather than indenting the region, lack of a feature to syntax highlight unsaved files, an inability to close multiple tabs at once, and finally an inability to drag tabs out of the frame to become separate frames, so as to be able to compare documents side by side.  Software designers take note – if you are going to have tabs, you must build in these features or your users will feel seriously limited.

Excellent video detailing motivation behind open source

This has been all over Twitter, so I apologize if this isn’t new to you. I absolutely love the fact that this video had as a visual not a boring set of PowerPoint slides, but instead a beautifully illustrated backdrop that was both informative, clever, and funny (the BTTF reference made me laugh).

This comes to a slightly different conclusion than that of The Cathedral & the Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary, one of the more popular books explaining the open source movement.  In that work, the author Eric Raymond puts a lot more emphasis on the importance of recognition as a motivating factor for open source; it’s not just the fact that you are contributing to a worthwhile cause, but the fact that your name is attached to the work.

Obviously that book was written much earlier than this video was produced, but I wonder what the author would think of it.

Car Talk Puzzler #3: The Palindromic Odometer

The Palindromic Odometer

Driving along, Terry notices that the last four digits on the odometer are palindromic. A mile later, the last five digits are palindromic. [A mile later, the middle four digits are palindromic.] [One mile after that], all six are palindromic. What was the odometer reading when Terry first looked at it?

Full question
Answer

This is the third in my series of Car Talk puzzler solutions using whatever programming language strikes my fancy. Just like the last one I wrote, this one uses Scala. This is a much shorter post, as I am assuming some familiarity with Scala; go back and read the last post if you haven’t already.

object PalindromicPuzzlerSolver {
    // Pad out numbers to 6 places
    // Note that I can cal Java library functions seamlessly
    val df = new java.text.DecimalFormat("000000")

    // A string is a palindrome if it reversed equals itself
    def isPalindrome(word: String): Boolean = {
        // Reverse is a method of a RichString, which a String can be implicitly converted
        // into. Necessary to convert the RichString back into String after the reversal
        word == word.reverse.toString()
    }

    def fitsDescription(miles: Int): Boolean = {
        // Last 4 are palindome (dropping first 2)
        isPalindrome(df.format(miles).drop(2)) &&
        // Add one, last 5 are palindrome
        isPalindrome(df.format(miles + 1).drop(1)) &&
        // Add two more, all six are a palindrome
        isPalindrome(df.format(miles + 3))
    }

    def main(args: Array[String]) {
        val max = 1000000
        println( 0 until max filter fitsDescription )
        // Prints out
        // RangeF(198888, 199999)

        // This is the same as the java style:
        // println( (0.until(max)).filter(fitsDescription) )
        // This syntax only works when you are dealing with
        // methods that take one argument.
    }
}

Thus the solution is 198888, just as the answer proclaims. Another win for brute force.

There are three noteworthy things in this example, which didn’t come up in the last post (or weren’t explained fully)

1. Interoperability with Java
2. Use of drop
3. Convenience methods for specifying a range

Interoperability with Java

Any Java class can be imported and used with Scala; that is one of its strongest selling points. Scala runs in the same JVM as Java, so it makes sense that it should have access to Java classes. In fact many Scala classes are thin wrappers around Java objects, with syntactic sugar built in. For instance, Scala has a BigInt class which wraps a Java BigInteger, while providing overloaded methods to allow them to be used more like standard numbers. For instance,

scala> BigInt("103058235287305927350") + BigInt("288888203959230529352")
res16: BigInt = 391946439246536456702

scala> new java.math.BigInteger("103058235287305927350").add(new java.math.BigInteger("288888203959230529352"))
res21: java.math.BigInteger = 391946439246536456702

Note that the + is actually a method call; this is exactly the same thing as calling

scala> BigInt("103058235287305927350").+(BigInt("288888203959230529352"))
res22: BigInt = 391946439246536456702

In Scala, method names don’t have to be alphanumeric, which is how it gets away with what looks like operator overloading. This makes it a lot nicer to work with classes like BigDecimal or BigInteger, as you can use the more commonly used addition and subtraction symbols rather than explicitly calling “add” and “subtract” methods. Just realize that you are really calling methods like any other under the hood. When you understand this, you’ll also understand why Scala uses an underscore instead of * for wildcard – * can be used in method names, as it is used for multiplication!

Drop

Scala has full support for slicing iterable objects; see my previous discussion on slices if this term is unfamiliar.

scala> List(1,2,3,4).slice(1,3)
res27: List[Int] = List(2, 3)

scala> List(1,2,3,4).slice(1)
warning: there were deprecation warnings; re-run with -deprecation for details
res28: Seq[Int] = List(2, 3, 4)

scala> List(1,2,3,4).drop(1)
res29: List[Int] = List(2, 3, 4)

So drop(n) is exactly the same as slice(n) – it provides a new list with the first n elements removed. I use drop instead of slice due to the deprecation of the single argument form of slice.

Specifying a range
If you need to iterate over a range of numbers in Java, you’d do something like
for (int i = 0; i < max; i++) {
}

whereas in Python you can do something like
for x in range(max):

Scala similarly provides methods to create a range of numbers to iterate over.

scala> 0 until 10
res9: Range = Range(0, 1, 2, 3, 4, 5, 6, 7, 8, 9)

scala> 0 to 10
res10: Range.Inclusive = Range(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10)

Again, I want to stress that until is a method like any other; you are just free to leave off the . and the parens when it is unambiguous what you mean. Note that these commands are actually creating a sequence in memory. If you only need to access each element one at a time, and you are dealing with a large range, you might instead create an Iterator rather than a sequence. The Iterator will create each subsequent number as necessary, rather than building up the whole list ahead of time. This is much more space efficient.

Note that Python makes this distinction as well:

>>> range(10)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> xrange(10)
xrange(10)

xrange returns a generator object, as opposed to range which returns the actual list.

Returning to Scala, if I wanted an iterator instead I would do

scala> Iterator.range(0,10)
res11: Iterator[Int] = non-empty iterator

scala> Iterator.range(0,10).foreach(println)
0
1
2
3
4
5
6
7
8
9

Conclusion
I have shown another solution using Scala, while illustrating a few more features. In particular one needs to understand that Scala’s syntactic sugar is actually masking standard function calls under the hood; x + y is the same as x.+(y), where “+” is a valid function name.

Car Talk Puzzler #2: What do these words have in common? An imperative and functional approach using Java and Scala

Here’s the second in my series of Car Talk puzzler solutions. Today I’m going to be illustrating a solution first in Java, and then in Scala in order to show how functional programming features  can drastically reduce the number of lines of code, while still expressing the developer’s intent.

Puzzler:

What do these words have in common: pig, table, cab, real, yet, and ride?

Answer:

RAY: Here’s the answer. If you notice, the first letter in each of the words is from farther down in the alphabet than all the subsequent letters. And if we assigned a number to every letter in the alphabet, so that A is one, B is two, C is three and so on…

TOM: Yeah, yeah, yeah.

RAY: Then the numerical place in the alphabet of the first letter is equal to the sum of the numerical places of all the other letters. For example, Table: T is 20, A is 1, B is 2, L is 12, E is 5, so 1 plus 2 plus 12 plus 5 –

TOM: Is 20.

So, we have the answer.  But as the guys mention on the show, there are probably other words as well.  Let’s do a brute force search through the dictionary to find all such words.  The dictionary I”ll be using is available here, as a new line separated list of words, one per line.  This makes it very easy to deal with.

Here’s the rather straightforward solution in Java:

 import java.io.FileReader;
 import  java.io.BufferedReader;
 import java.io.FileNotFoundException;
 import java.io.IOException;

/**
* Determines all the words such that the first letter's ordinal position
* equals the sum of the remaining letters' ordinal positions.  For instance,
* "table"'s ordinal numbers are [20,1,2,12,5], and 20 = 1+2+12+5.  Thus
* "table" has this property.
*/
public class OrdinalPositionPuzzlerSolver {

    /**
    * Assumes the string consists just of A-Z, a-z.
    */
    public static boolean hasProperty(String s) {
        if (s.length() < 2) {
            return false;
        }
        char[] letters = s.toCharArray();
        int sum = 0;
        for (int i = 1; i < letters.length; i++) {
            int ordinal = ordinal(letters[i]) ;
            sum += ordinal;
        }
        return sum == ordinal(letters[0]);
    }

    /**
    * @return the 1 based ordinal position of the letter.  For instance,
    * a is 1, b is 2, etc.
    */
    public static int ordinal(char letter) {
        return Character.toLowerCase(letter) - 'a' + 1;
    }

    public static void main(String[] args) {
        String dictionaryPath = "dict.txt";
        /**
        http://www.javapractices.com/topic/TopicAction.do?Id=42
        */
        try {
            BufferedReader input = new BufferedReader(new FileReader(dictionaryPath));
            try {
                String line = null;
                while (( line = input.readLine()) != null){
                    if (hasProperty(line)) {
                        System.out.println(line);
                    }
                }
            }
            catch (FileNotFoundException e) {
                System.err.println("Error, file " + dictionaryPath + " didn't exist.");
            }
            finally {
                input.close();
            }
        }
        catch (IOException e) {
            System.err.println("IO Error:" + e);
        }

    }
}
 

The only place that bears some explanation might be the ordinal function:

/**
 * @return the 1 based ordinal position of the letter.  For instance,
 * a is 1, b is 2, etc.
 */
public static int ordinal(char letter) {
    return Character.toLowerCase(letter) - 'a' + 1;
}

Basically, letters (characters) in Java are really just numbers under the hood which get mapped back into letters when printed to the screen.  So, we can do arithmetic using letters.  ‘a’ + 1 = ‘b’, ‘b’ + 2 = ‘d’, and so on.  (This only works because the letters a-z and A-Z are mapped to a continuous range of integers).  So if we want to convert a->1, first we figure out a’s position from the start of the alphabet (‘a’ -‘a’ == 0, it’s the first), and add 1 to get the 1-based number.

The other thing to notice is how much junk code there is just to read the lines from our file:

try {
    BufferedReader input = new BufferedReader(new FileReader(dictionaryPath));
    try {
        String line = null;
        while (( line = input.readLine()) != null){
            if (hasProperty(line)) {
                System.out.println(line);
            }
        }
    }
    catch (FileNotFoundException e) {
        System.err.println("Error, file " + dictionaryPath + " didn't exist.");
    }
    finally {
        input.close();
    }
}
catch (IOException e) {
    System.err.println("IO Error:" + e);
}

Believe it or not, this is relatively idiomatic Java code to accomplish this task.  First we wrap a file reader in a buffered reader so that we do not continuously access the file; rather we read chunks into the file and buffer them for in-memory access.  That’s 20 lines of code just to iterate over the lines of a file, determine if they match a pattern, and print it to the terminal.

It really should not be this hard.

As I’ve said before, Java’s verbosity is one of my biggest complaints about it.  The program works, and it’s relatively straightforward, but it can’t really be described as elegant.  In that same post I mentioned my interest in Scala as an alternative to Java.  It’s a compiled language that runs in the JVM, but through its type inferencing and functional language aspects, it is much more concise; the creator Martin Odersky claims a typical Source Line of Code (SLOC) reduction of 2x between equivalent Scala and Java programs. Let’s reimplement the program in Scala, and see if this claim rings true.  (Note: I’m not an expert in scala so there’s probably even more concise, idiomatic ways to accomplish what I’m trying to do.  But let’s start somewhere).

The syntax of scala is very similar to that of Java.  One of the main differences is that type declarations occur after the variable rather than before.

 int x = 0; // Java
 val x: Int = 0 // Scala

Another syntactic difference is that generic type arguments occur within square brackets rather than angle brackets

 List<Integer> intList = ...; // java
 val intList: List[Int] =  ..// scala

Because of this, array indexing is done through use of parentheses rather than square brackets.

 // Java
 int[] x = {1,2,3,4};
 x[0] // 1

 // Scala
 val x: List[Int] = List(1,2,3,4)
 x(0) // 1

Due to type inferencing, you often don’t need to declare the type of your variables; the compiler can figure it out.  Thus the last declaration could be rewritten as

 val x = List(1,2,3,4)

I highly recommend you watch Martin Odersky’s introduction to Scala video; it’s about an hour long and goes over the design considerations and differences from Java.

There are no static functions in Scala; rather you create a singleton object through the keyword ‘object’ and access the method in the same way.

Finally, there is a read/eval/execute interpreter built-in, accessible by calling ‘scala‘ in the command line.  This allows you to experiment without having to explicitly compile any code; it allowed me to create the pieces iteratively before plunking them into a file for later compilation.

 scala
 Welcome to Scala version 2.7.3.final (Java HotSpot(TM) Client VM, Java  1.5.0_22).
 Type in expressions to have them evaluated.
 Type  :help for more information.

 scala> 1+1
 res0: Int = 2

 scala> print("hello world")
 hello world
 scala>
 

OK, enough beating around the bush.  Let’s solve this puzzler!

We’ll tackle this in 3 parts.
1) Summing a list of numbers
2) Completing the ordinal number function
3) Iterating over all the lines in the dictionary file

Summing a list of numbers

A first pass at a sum function might be as follows:

def sum(nums: Iterable[Int]): Int = {
    var sum = 0
    for (num <- nums) {
        sum += num
    }
    sum
}

Note the declaration of sum as a var as opposed to a val; a val is equivalent to a final value while a var can be reassigned.  Why did I declare the nums argument to be of type Iterable rather than List?  Well, a List is iterable.  So is an array.  So are any number of things.  We want to keep this function as generic as possible. Pasting this into our interpreter, we can test this out.

scala> def sum(nums: Iterable[Int]): Int = {
     |     var sum = 0
     |     for (num <- nums) {
     |         sum += num
     |     }
     |     sum
     | }
sum: (Iterable[Int])Int

scala> sum(List())
res4: Int = 0

scala> sum(List(1,2,3,4,5))
res5: Int = 15

scala> sum(Array(1,2,3,4))
res6: Int = 10

This works, but let’s look at another way to solve this problem – a functional rather than imperative solution.  Java is inherently imperative, meaning you’ll see a lot of explicit while and for loops.  Here’s just a quick excerpt of each the wikipedia articles to define each programming paradigm:

In computer science, functional programming is a programming paradigm that treats computation as the evaluation of mathematical functions and avoids state and mutable data. It emphasizes the application of functions, in contrast to the imperative programming style, which emphasizes changes in state.

In computer science, imperative programming is a programming paradigm that describes computation in terms of statements that change a program state. In much the same way that imperative mood in natural languages expresses commands to take action, imperative programs define sequences of commands for the computer to perform.

So whereas the imperative manner is completely explicit in how the sum is created, using a mutable state variable (sum), a more functional approach will eliminate the use of mutable state altogether.

The reduce function is absolutely crucial to understand in order to program in a functional manner; let’s examine Python’s definition of it before applying it to the problem at hand.

Here is the Python definition of reduce function:

reduce(function, iterable[, initializer])

Apply function of two arguments cumulatively to the items of iterable, from left to right, so as to reduce the iterable to a single value. For example, reduce(lambda x, y:x+y, [1, 2, 3, 4, 5]) calculates ((((1+2)+3)+4)+5). The left argument, x, is the accumulated value and the right argument, y, is the update value from the iterable. If the optional initializer is present, it is placed before the items of the iterable in the calculation, and serves as a default when the iterable is empty. If initializer is not given and iterable contains only one item, the first item is returned.

And the description of the built-in sum function is as follows:

sum(iterable[, start])¶

Sums start and the items of an iterable from left to right and returns the total. start defaults to 0. The iterable‘s items are normally numbers, and are not allowed to be strings. The fast, correct way to concatenate a sequence of strings is by calling ”.join(sequence). Note that sum(range(n), m) is equivalent to reduce(operator.add,range(n), m) To add floating point values with extended precision, see math.fsum().

While Scala doesn’t have a built-in sum function, it does have support for reducing a list through use of a function.  It has both a reduce left (left to right reduction) and reduce right (right to left reduction).  Here is Scala’s definition of reduceLeft:

def reduceLeft[B >: A](op : (B, A) => B) : B
Combines the elements of this iterable object together using the binary operator op, from left to right
Notes Will not terminate for infinite-sized collections.
Parameters op – The operator to apply
Returns op(… op(a₀,a₁), …, a_n) if the iterable object has elements a₀, a₁, …, a_n.
Throws Predef.UnsupportedOperationException – if the iterable object is empty.

This might be a little confusing but bear with me.

First, we need to write a binary function for our sum operator – after all, we want to pairwise reduce the list by using an add operator.  A first pass is as follows:

scala> def add(x: Int, y: Int): Int = { x + y }
add: (Int,Int)Int

scala> List(1,2,3,4).reduceLeft(add)
res12: Int = 10
scala> List().reduceLeft(add)
java.lang.UnsupportedOperationException: Nil.reduceLeft
at scala.List.reduceLeft(List.scala:1093)
at .(:6)
at .()
at RequestResult$.(:3)
at RequestResult$.()
at RequestResult$result()
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethod...
scala> List(-5,5).reduceLeft(add)
res14: Int = 0

Note that you cannot call sum on an empty list any more; doing so results in an UnsupportedOperationException.  This is arguably more correct than returning 0, but we will have to ensure that we don’t attempt to sum an empty iterable object.

So a working sum definition would be

def sum(nums: Iterable[Int]): Int = {
    def add(x: Int, y: Int): Int = { x + y }
    nums.reduceLeft(add)
}

Note that you can nest function definitions, as functions are objects in Scala, and thus can be defined wherever a variable could.  Note too that I never have to explicitly return a value; the last evaluated value is implicitly returned by the function.

Finally, if you want to be even more terse, you can replace the explicitly defined add function with an anonymous one:

def sum(nums: Iterable[Int]): Int = {
    nums.reduceLeft(_+_)
}

Read the Daniel Spiewak’s excellent blog post for a better explanation for what’s going on here.  Here’s a quick explanation:

(_+_) is actually a Scala shorthand for an anonymous function taking two parameters and returning the result obtained by invoking the + operator on the first passing the second.  Less precisely: it’s addition.  Likewise, (_*_) would define multiplication.

Completing the ordinal number functional

Recall the function that we had in Java for computing the ordinal position of each letter:

    public static boolean hasProperty(String s) {
        if (s.length() < 2) {
            return false;
        }
        char[] letters = s.toCharArray();
        int sum = 0;
        for (int i = 1; i < letters.length; i++) {
            int ordinal = ordinal(letters[i]) ;
            sum += ordinal;
        }
        return sum == ordinal(letters[0]);
    }

    /**
    * @return the 1 based ordinal position of the letter.  For instance,
    * a is 1, b is 2, etc.
    */
    public static int ordinal(char letter) {
        return Character.toLowerCase(letter) - 'a' + 1;
    }

Again, we could directly translate this code into Scala but doing so would not be making full use of Scala’s functional underpinnings.  We are converting from characters of the string to their ordinal positions in the alphabet.  While the Java code does this one letter at a time, keeping a sum, it’s just as correct to do the entire mapping process and then sum over the new list.  Pictorially:

Just as there is a reduce function to convert an iterable object into a single entry, so too is there a function to convert an Iterable of one type into an Interable of another; it is the map function.

// The underscore in an anonymous function like  this is a placeholder for the entry in the iterable collection currently  being processed
scala> List(1,3,4).map(_+5)
res19: List[Int] =  List(6, 8, 9)
scala> List("hi","there").map(_.toUpperCase())
res22:  List[java.lang.String] = List(HI, THERE)

The above examples map from Int-> Int and String->String but there’s nothing stopping us from converting types.

scala> List(100,300,400).map(_.toString())
res23:  List[java.lang.String] = List(100, 300, 400)

Finally, let’s apply the ordinal function to a string:

scala> "cat".map(Character.toLowerCase(_) - 'a' +  1)
res24: Seq[Int] = ArrayBufferRO(3, 1, 20)

Thus the entire function for determining if the word is a match is

[sourecode language=”scala”]
/**
* Determines if the word has the desired property of the ordinal position
* of first letter equalling the sum of the ordinal positions of the other
* letters
* Expects letters in the range [a-zA-Z];
*/
def matches(word: String): Boolean = {
if (word.length() < 2) {
false
}
else {
val ordinals = word.map(Character.toLowerCase(_) – 'a' + 1)
val ordinalSum = sum(ordinals.drop(1))
ordinals(0) == ordinalSum
}
}
[/sourecode]

Iterating over the lines in a file

Remember how it took 20 lines to iterate over all the lines in a file?  Fortunately Scala makes it incredibly easy:

val lines =  scala.io.Source.fromFile("dict.txt").getLines

The variable is actually not an array or a list; it is an iterator over the lines in the file.  But iterators are still iterable, so all of our previously mentioned functions for dealing with iterable objects still work.  Again, rather than looping over each element, explicitly testing, and printing whether it matched, we will use a functional approach, namely the ‘filter’ function.

scala>  List(1,2,3,4).filter(_%2==0)
res25: List[Int] = List(2, 4)

The function passed into filter must return Boolean; since our matches() function does that, we can insert it right in:

val lines =  scala.io.Source.fromFile("dict.txt").getLines
val matching =  lines.filter(matches)
matching.foreach(print(_))

When we run the complete code, we get the following list of matches:

• acknowledgements
  approximation
  bumptiously
  capitalizations
  circuitously
  conceptualizing
  consciousness
  consistently
  credulousness
  crystallizing
  disconcertingly
  discriminatory
  electrocutions
  excommunication
  existentialism
  experimentally
  exploitations
  expressively
  extraneously
  governmentally
  impersonations
  incomprehensibly
  indispensability
  initializations
  intercommunicated
  interpolations
  notwithstanding
  organizationally
  oversimplifying
  phenomenologically
  sportswriting

Wait a minute… what’s going on here?  These obviously don’t follow the pattern.  Let’s open a terminal and do some testing.

First we paste our two helper method definitions in

 Welcome to Scala version 2.7.3.final (Java  HotSpot(TM) Client VM, Java 1.5.0_22).
 Type in expressions to have  them evaluated.
 Type :help for more information.

 scala>  def sum(nums: Iterable[Int]): Int = {
 |          nums.reduceLeft[Int](_+_)
 |     }
 sum: (Iterable[Int])Int

 scala>

 scala>     /**
 |      * Determines  if the word has the desired property of the ordinal position
 |      * of first letter equalling the sum of the ordinal positions of  the other
 |      * letters
 |      * Expects letters in  the range [a-zA-Z];
 |      */

 scala>     def  matches(word: String): Boolean = {
 |         val ordinals =  word.map(Character.toLowerCase(_) - 'a' + 1)
 |         val  ordinalSum = sum(ordinals.drop(1))
 |         ordinals(0) ==  ordinalSum
 |     }
 matches: (String)Boolean

 scala> matches("table")
 res0: Boolean = true

 scala>  matches("chair")
 res1: Boolean = false

 scala>  matches("consistently")
 res2: Boolean = false
 

This is very strange.  “consistently” is one of the words on the list, but it doesn’t pass the test on its own.  In fact, none of them do.

What’s the problem?  After googling and debugging for a bit, I realized that the precondition I was assuming in my matches method was being violated; the letters were not in the range [A-Za-z].  How could that be?  Well, it turns out that the getLines() method returns the lines in the file – with new lines included!

scala> matches("consistently\r\n")
res5: Boolean = true

(The file had been created on Windows, and thus has a carriage feed as well as a new line marker).

Fortunately there is a method to eliminate whitespace (including control characters like these); it’s known as trim.

We can put the trim in one of two places, either in the matches() method itself,

def matches(word: String): Boolean = {
     val ordinals = word.trim().map(Character.toLowerCase(_) - 'a' + 1)
     val ordinalSum = sum(ordinals.drop(1))
     ordinals(0) == ordinalSum
}

scala> matches("consistently\r\n")
res6: Boolean = false

scala> matches("consistently")
res7: Boolean = false

Or we can introduce the trim while iterating over the lines in the collection.  I’ve opted for this second approach; I’m making it a precondition of calling the method matches() that the input is already sanitized.  We introduce an implicit mapping from the string to the trimmed version of the string:

// An iterator of lines in the file; note that they have newline control characters
val lines = scala.io.Source.fromFile("dict.txt").getLines
// Filter the lines, eliminating the new lines through trim mapping
val matching = lines.filter(i => matches(i.trim()))
matching.foreach(print(_))

After making the change, the result is

• cab
• hag
• jade
• leaf
• leg
• maced
• mica
• mid
• need
• pig
• real
• ride
• same
• sand
• seam
• sod
• table
• toad
• toe
• vial
• ward
• whack
• who
• wick
• win
• yeas
• yet
• zebra
• zinc

Tada!

The final version of the program is

object OrdinalPositionSolver {
    /**
     * Determines if the word has the desired property of the ordinal position
     * of first letter equalling the sum of the ordinal positions of the other
     * letters
     * Expects letters in the range [a-zA-Z];
     */
     def matches(word: String): Boolean = {
         if (word.length() < 2) {
            false
         }
         else {
             val ordinals = word.map(Character.toLowerCase(_) - 'a' + 1)
             val ordinalSum = sum(ordinals.drop(1))
             ordinals(0) == ordinalSum
         }
     }

    def sum(nums: Iterable[Int]): Int = {
        nums.reduceLeft[Int](_+_)
    }

    def main(args: Array[String]) {
        // An iterator of lines in the file; note that they have newline control characters
        val lines = scala.io.Source.fromFile("dict.txt").getLines
        // Filter the lines, eliminating the new lines through trim mapping
        val matching = lines.filter(i => matches(i.trim()))
        matching.foreach(print(_))
    }
}

Conclusion

I have solved the puzzler in two different ways, once in an imperative Java style and once in a more functional Scala style.  The functional way is extremely concise and expressive, doing in 30 lines of code what it takes the Java code to do in over 60, confirming the claim that a typical Scala program is at least twice as compact as the corresponding Java program.  I have introduced some of the salient syntactic differences between Java and Scala. I have also introduced some of the mainstays of the functional programming paradigm, namely map, reduce, and filter.  While there are efforts to make Java more functional, the fact that you cannot pass functions as objects severely inhibits the use of functional programming paradigms.

“Uncompilable source code” exception in NetBeans

If your project builds fine but then crashes with a runtime exception when it hits a block of code, the problem is probably with the “Compile on save” option. Right click on your project, choose Properties, Build -> Compiling. Uncheck Compile on Save. Manually build the project. When you re-run the code, the errors should go away.

Not sure what causes this, but it’s immensely frustrating and seemingly non-deterministic. Wish I had a better solution than this, but I don’t.

Edit:

I found perhaps a better solution on StackOverflow:

We finally got a solution, but still don’t quite know why the situation occurs. When you have Compile On Save activated, Netbeans generates a second set of class files for debugging etc. These are stored in $USER/.netbeans/var/cache/index/s*/java/*/classes

Somehow (not sure how) this directory can get corrupted or fail to update.

If you close netbeans, delete $USER/.netbeans/var/cache/index and all subdirectories and restart netbeans this clears the cache. If you have no compile errors, your problem ought to go away at this point.

NB: $USER is your user directory – on Windows 7 this is usually c:\Users\username, I guess on Unix it will be ~username.

If you get this problem please vote for, comment on, or add information to: http://netbeans.org/bugzilla/show_bug.cgi?id=182009

Thanks to Nick Fortescue for this information.

Maybe this is why people are afraid of the command line?

I think one of my favorite quotes sums this up nicely:

A wealth of information creates a poverty of attention – Herbert Simon

Args4j library for parsing Java command line arguments

args4j is a great tool for parsing command line arguments in Java.

Here’s the description from its homepage:

It makes the command line parsing very easy by using annotations.

You can generate the usage screen very easily.

You can generate HTML/XML that lists all options for your documentation.

Fully supports localization.

It is designed to parse javac like options (as opposed to GNU-style where ls -lR is considered to have two options l and R.)

It is licensed under the MIT license.

It is fairly simple to use:

1. Create a class holding all the options you wish to parse.
2. Annotate the fields of the class, telling args4j information about the command line arguments (what flags are necessary, what are the various short and long ways of specifying that argument)
3. In your main method, create an instance of your options holder class
4. Pass the instance from step 3 into an instance of CmdLineParser class, and then call parseArgument on the array of strings passed into the main method
5. If no parse exceptions are thrown, your options object has all of the required options fields filled in, which can then be queried via normal getters.

The sample main class provided by the args4j folks is especially useful; in this example they do not create a separate class to hold the options, but that is certainly an option.

I’m really not sure how this stacks up to other command line parsing tools for Java; suffice to say it’s small, fast, the annotations make it a breeze to specify a set of names for different named arguments… it just works.  Please use this instead of trying to roll your own options parsing code.

EventBus: Introduction and troubleshooting for annotation problems

Today I ran into a problem with a great project I use called EventBus, so I figured I’d use it as an opportunity to introduce the project, as well as give a solution to the problem I was facing.

EventBus, an open-source Java project, is described on its homepage as

Pub-sub Event broadcasting mechanism, Swing-oriented

In a nutshell, the publication/subscription mechanism is an alternative to explicitly defining listeners and adding them to objects of interest (the Observer/Observable pattern). This is beneficial because it allows for a more modular, late-binding approach to events. As the EventBus page describes,

“It replaces the tight coupling introduced by explicit listeners with a flexible loosely coupled design. ”

Here is the typical MVC type code for having objects notified of changes in state (from my post about creating a solar system model in Java):

public class SolarSystemModel extends Observable {

    // snip

    public void setDay(int day) {
        int oldDay = this.day;
        this.day = clampDay(day);
        if (oldDay != this.day) {
            setChanged();
            notifyObservers();
        }
    }

    public void setHour(int hour) {
        int oldHour = this.hour;
        this.hour = clampHour(hour);
        if (oldHour != this.hour) {
            setChanged();
            notifyObservers();
        }
    }
}

public class  SolarSystemView extends JComponent implements Observer {

    public void update(Observable o, Object arg) {
            repaint();
    }

}

This works fine but note a couple problems.
1) I am forced to either make my domain object extend Observable, in which case I lose out ability to subclass other more useful classes, or forced to roll my own Observer type interfaces, hold a list of listeners, add them all manually.

2) If we were to add another class that visualized the solar system, it would need to get a handle to the solar system model and be tightly coupled with it.

EventBus strives to get away from explicit listeners that the model objects need to keep track of. Instead, there are feeds of objects or topics that interested classes can register their interest in and receive the updated objects. The above example could be redone using EventBus as follows:

public class SolarSystemModel {

    // snip

    public void setDay(int day) {
        int oldDay = this.day;
        this.day = clampDay(day);
        if (oldDay != this.day) {
            EventBus.publish(this);
        }
    }

    public void setHour(int hour) {
        int oldHour = this.hour;
        this.hour = clampHour(hour);
        if (oldHour != this.hour) {
            EventBus.publish(this);
        }
    }
}

public class  SolarSystemView extends JComponent implements EventSubscriber<SolarSystemModel> {

    public SolarSystemView() {
        // Snip
        EventBus.subscribe(SolarSystemModel.class, this);
    }

    public void onEvent(SolarSystemModel m) {
        // Use the updated solar system model to repaint
        repaint();
    }
}

Again, note that the model now is free to extend whatever class it wants, rather than Observable.

This example is a bit too trivial to fully illustrate the power of EventBus; for a more real world example see the excellent article on refactoring a Swing financial app to use EventBus.

In addition to explicitly declaring that you are an EventSubscriber, you can annotate methods in your object to announce the same intent. For instance, rather than doing something like

public class View implements EventSubscriber<ModelClassA>, EventSubscriber<ModelClassB>, EventSubscriber<ModelClassC> {

    public View() {
        EventBus.subscribe(ModelClassA.class, this);
        EventBus.subscribe(ModelClassB.class, this);
        EventBus.subscribe(ModelClassC.class, this);
    }

    public void onEvent(ModelClassA) {
        // Handle this type of object
    }

    public void onEvent(ModelClassB) {
        // Handle this type of object
    }

    public void onEvent(ModelClassC) {
        // Handle this type of object
    }
}

you can instead name the methods however you want, using annotations:

public class View {

    public View() {
        // Important: without this line, the annotations will not be read
        AnnotationProcessor.process(this);
    }

    @EventSubscriber(eventClass=ModelClassA.class)
    public void handleTypeA(ModelClassA) {
        // Handle this type of object
    }

    @EventSubscriber(eventClass=ModelClassB.class)
    public void handleTypeB(ModelClassB) {
        // Handle this type of object
    }

    @EventSubscriber(eventClass=ModelClassC.class)
    public void handleTypeC(ModelClassC) {
        // Handle this type of object
    }
}

This is a good feature, since it allows you to customize the naming of your methods, providing more descriptive names than “onEvent”.

OK, after that rather lengthy introduction, here is the problem I was running into:

“incompatible types
required: java.lang.annotation.Annotation
found: org.bushe.swing.event.EventTopicSubscriber”

It was fairly inscrutable to me, so once I found out what was going on, I figured I had to write about it.

The problem is that there are identically named classes, one for the actually implementation, and one for the annotation. So instead of importing

org.bushe.swing.event.EventTopicSubscriber

you must instead import

org.bushe.swing.event.Annotation.EventSubscriber

If you use the “Fix all imports” in NetBeans, it will automatically choose the wrong one.