Archive for February, 2010

Touch screen interface considerations

February 24, 2010 5 comments
When moving to a new hardware platform such as a touch sensitive mobile device, it is important to design the UI with the constraints of the device in mind.  Early Windows tablets failed in large part due to the OS not being suited to touch interaction; icons were too tiny to press on without a stylus.  I have heard the same complaint about early generation Windows Mobile phones; the user in question said he had to literally press tiny squares on the screen where the up and down arrows were in the scroll bar.  The UI widgets from desktop programs were ported over to these new platforms without giving thought to the decreased screen resolution, screen size, and finger dexterity that a mobile platform provides.

The iPhone and its multitouch gestures provide very intuitive, natural ways of interacting with the reduced screen real estate.  Scroll bars are no longer controls that must be held and dragged; rather they are small, unobtrusive widgets that appear only while a page is being dragged.  The entire page becomes the draggable area, in keeping with the idea of maximizing the size of items that need to be interacted with.  Coupled with the pinch gestures for zooming in and out of webpages, the browsing experience on the iPhone is almost flawless.  There are two main problems stemming from the metaphor of dragging the entire webpage: the lack of hover and drag and drop.

Hover behavior

Given the fact that touch screens only pick up your inputs when you are touching the screen, there is no notion of hovering (the browser doesn’t display a cursor where you last touched).  Hovering over elements in a webpage usually has one of two effects:

1) Showing a tool tip

2) Displaying additional information in a section of the browser; e.g. mousing over an embedded link reveals the target of the link in the lower left corner while using firefox.

In very few cases, mousing over or hovering over elements actually acts like a click; see for an example.

Number 2 is not a big concern; when you long press on an embedded link, a screen pops up revealing the target of the link:

Number 1 is more of a concern.  If your site is designed to be viewed on a mobile browser, you should ensure that that you have a version of the CSS that does not require you to mouse over or hover over elements to reveal information.

Drag and drop

Drag and drop is a common desktop OS UI metaphor.  It can be slightly tricky for users to handle due to the dexterity required, especially if the drag and drop targets are small; most digital natives don’t have any problems with it.  It is less common on the web than traditional desktop fat clients, but there are certainly webpages that use it.  For instance, WordPress uses drag and drop to configure sidebar widgets:

The problem is that these movable elements cannot be selected and moved on the iPhone; they select when pressed but as soon as you move your finger, they deselect and instead the page moves.  While this would be a problem in and of itself, drag and drop is also used in common UI widgets such as sliders.


Sliders are an excellent UI controller for selecting an approximate value from a bounded range of continuous values.  Look at‘s elegant dual slider control for narrowing down price:
With a single control, we get two functions:
  1. Displaying the current range of values
  2. Allowing user to narrow down that range into a smaller subset.

For instance, if user only wants sneakers that cost between 50 and 100 dollars he just drags both ends of the slider bar:

This illustrates a drawback of the slider approach: it is imprecise when the range of available values is large and the control is small on the page; the slider would have to have subpixel precision in order to make user

The frustrating thing is that the iPhone has BRILLIANT slider support in its iPod app.  They get around the problem of precision by making use of the vertical dimension that normally is ignored.  When your finger is right on the slider, the slider follows your finger one to one.  When you move your finger down the screen, the sensitivity is turned down and it takes a greater horizontal change to elicit a change in the position of the slider (changing from “Hi-Speed Scrubbing” to half speed to quarter speed.)

White dot represents where the finger is held down

The drawback of this interaction is that it is novel; I discovered it by accident and I’m sure a lot of iPhone users don’t even know about it.  However, it is a wonderful solution to the problem of imprecise fingers; it could be used in traditional desktop apps as well.

Sites with interactive javascript graphs frequently use sliders; see for instance OKCupid’s blog post on why you should date older women:

On the iPhone, it is impossible to move the slider to interact with the graph.

The problem is that there is no standard HTML way of specifying a slider; instead sites use a mishmash of Javascript to get that effect.  (Read a tutorial how).  If there were a standard HTML component, it’s likely that the iPhone could handle it more gracefully.  For instance, the iPhone changes the standard way dropdown menus are implemented

Standard desktop view of a dropdown control

iPhone view of control – a finger-friendly wheel widget pops up, occupying half of the screen.

So one solution to the problem of lack of slider support would be to have a standard HTML widget for sliders that the iPhone could then interpret and render in a more useful way (e.g. with the vertical dimension added like on the iPod slider controls).

Another potential solution would be to make a gesture that indicates a mouse drag.  There is already a way to select text, which is done in a desktop environment via the same mechanism as drag and drop (press, move mouse, release mouse button).  I could see this working in one of two ways.

1) As a multitouch gesture.  One finger is held as still as possible indicating that the background should be anchored, while the other finger proceeds to press on the draggable component and move it around the screen.  The drawback of this is that the pinch gesture currently works even when one finger is held still; thus this gesture would overload that of pinch.  I know I personally pinch by moving both fingers, but some people might not.

2) When a draggable entity is long-pressed for long enough (1, 2 seconds?) browser enter different mode where motion events do not scroll the page but instead move the draggable item.

If you have a multitouch enabled browser, you can see how dragging might work in this proof of concept page. Unfortunately the pinch gesture does not work correctly – it’s much jerkier than a standard webpage.  Their implementation suggests a third alternative, immediately entering drag mode when a draggable item is touched.  The problem with this is that, if there were multiple draggable items on the screen it would be very difficult to actually grab a portion of the screen in order to pan around to new areas.

This way of handling drag and drop is used by games on iPhone, such as Words With Friends.  Note in the left picture the scroll bars indicating that the board is being moved.  On the right the finger is moving while over the E tile, so the E is dragged along with the finger.  Moving the finger off the screen is equivalent to releasing the mouse button in a standard drag and drop action; the tile is laid down in nearest square.


The iPhone has introduced some great UI metaphors such as pinch and zoom as well as making the entire area of the page draggable for easy scrolling of text.  In the realm of the web browser specifically, these advances prevent two common desktop OS UI metaphors from working, that of hovering over elements and dragging and dropping elements.  The problem of hovering is not insurmountable; web developers need to be aware of this drawback and code their websites in such a way that information is not lost if you cannot view tooltips or otherwise hover over elements.  The second problem, that of an inability to select an element, drag it, and drop it, is more pressing.  There are certain tasks that are absolutely impossible to complete while using an iPhone browser, such as anything requiring you to move a slider control.

Hopefully the new version of the iPhone OS can remedy the drag and drop problem.

Categories: UI Tags: , , ,

Netflix queue manager – a case study in UI design

February 16, 2010 Leave a comment

In this blog post, I examine some good and bad aspects of user interface design present in the queue management system on  By examining these issues we can see some concrete examples of more general purpose principles of UI design that are promulgated by books such as About Face by Alan Cooper

For those unfamiliar with Netflix or computer science data structures, a queue is a first-in-first-out data structure, just like a line at a grocery store.  By moving a movie to the top/front of the queue, you ensure that it is the next movie to be delivered.

Let’s examine some good and bad aspects of the UI design in the queue management system of Netflix.


The queue screen is very minimalist in its appearance, containing only relevant information without fluff.  For instance, the tabs immediately display how many movies you have both in your DVD queue as well as instant queues; you don’t have to look far.

One thing not immediately obvious to a casual observer is the consistent color coding scheme at play.  Notice in the tabs that the DVD section and any related text is red; the word Instant in the tab and all words related to the Instant queue are in blue.

This color scheme is present while searching for movies as well:

The queue screen does more than display information about the movies currently in your queue; if that were all it did, it could be a static HTML page.  Instead, this screen is where you can rearrange the order of the movies as well as delete them from the list.

Deleting a movie from the queue is entirely painless.  There is an unambiguous X link to the right of each movie.  Mercifully, clicking the X does not pop up a dialog box asking “Are you sure you want to delete the movie from the queue?” or any such nonsense.  Instead, the row becomes grayed out with an undo link replacing the previous X button.  This is absolutely the right way to do things, and I wish that more sites worked this way.  Throwing up a dialog box in front of destructive actions as a way of removing responsibility from the programmers/computer program is all too common and lazy.  It’s harder to implement the ability to undo destructive actions, but from an interaction design perspective it is far superior.  Users often click through dialogs without actually reading what they’re doing (I accidentally deleted a high-level Diablo II character this way.  I’m not bitter or anything).


After deletion.  Note that the movie underneath the removed one has its position moved up in the queue

In order to move movies around, you have three main options.  The first is dragging and dropping the movie to its new location.  The second is typing in a value in the text box and hitting enter.  The third, and probably most useful, is hitting the Top button which pushes the movie to the front of the queue.  Each works fine, though I will have more to say about the dragging and dropping option in the next section.


There is no mechanism to move multiple movies at once.  In general this would not be a problem, but one of the main reasons I use Netflix is to rent TV series on DVD that would otherwise be too expensive to own, and they often come in multiple discs.  If you want to move all six discs of a series up to the top of the queue, you have to move each disc individually – in reverse order.  If you start with disc 1 and move on through disc 6 pressing the TOP button each time (which is your natural inclination) you’ll end up with the series in reverse order, 6,5,4,3,2,1.

There are two solutions to this.
1) Allow multiple selection and movement.  This is executed to perfection in Animoto, an online tool for creating slideshows with music and animated transitions.

See the following screenshot:

As the image (or images) is moved around, all the other images move to make room.  The same could easily be done with rows of movies.  However, I do not feel that this is the best approach, since the most important function of the queue is to move movies to the front.  If you had a long queue and were trying to move something to the top via drag and drop, you’d have to move the items a long way.  Drag and drop is best when it occurs within the same screen so that no scrolling has to occur; it is a difficult task for some computer users to drag items to the very small sweet spot that causes the window to scroll up or down.

2) Automatically determine if the item you are attempting to move to the top of the list is part of a series.  If it is, either
  • ask if you want to move all of the DVDs up
  • automatically move all of the movies up and allow user to easily undo if this is not what he meant to do
I prefer the second option as I imagine the vast majority of time people intend to move all the DVDs in a series up to the top of a queue, and to have all those “yes” people have to answer a dialog box seems like a waste of time.

Another poor design choice for the queue screen is the lack of keyboard controls.  I like to do as much on my computer as possible without removing my hands from the keyboard; thoughtful UI design demands keyboard alternatives for the most frequently used operations.  Tab traversal works as expected through the movies, shifting the focus to the numerical position in the queue.  Unfortunately, this is the only operation available to you using only the keyboard; you cannot delete movies from the queue without using the mouse, as the tab traversal does not bring you through the fields of the individual movie.  Instead it skips from movie to movie.


Overall the queue management is painless, with the exception of the inability to move more than one movie at a time. I would give this aspect of the site a B+ for overall usability and design.

Categories: UI Tags: ,

R – Sorting a data frame by the contents of a column

February 12, 2010 3 comments

Let’s examine how to sort the contents of a data frame by the value of a column

> numPeople = 10
> sex=sample(c("male","female"),numPeople,replace=T)
> age = sample(14:102, numPeople, replace=T)
> income = sample(20:150, numPeople, replace=T)
> minor = age<18

This last statement might look surprising if you’re used to Java or a traditional programming language. Rather than becoming a single boolean/truth value, minor actually becomes a vector of truth values, one per row in the age column.  It’s equivalent to the much more verbose code in Java:

int[] age= ...;
for (int i = 0; i < income.length; i++) {
   minor[i] = age[i] < 18;

Just as expected, the value of minor is a vector:

> mode(minor)
[1] "logical"
> minor

Next we create a data frame, which groups together our various vectors into the columns of a data structure:

> population = data.frame(sex=sex, age=age, income=income, minor=minor)
> population
 sex age income minor
1    male  68    150 FALSE
2    male  48     21 FALSE
3  female  68     58 FALSE
4  female  27    124 FALSE
5  female  84    103 FALSE
6    male  92    112 FALSE
7    male  35     65 FALSE
8  female  15    117  TRUE
9    male  89     95 FALSE
10   male  26     54 FALSE

The arguments (sex=sex, age=age, income=income, minor=minor) assign the same names to the columns as I originally named the vectors; I could just as easily call them anything.  For instance,

> data.frame(a=sex, b=age, c=income, minor=minor)
 a  b   c minor
1    male 68 150 FALSE
2    male 48  21 FALSE
3  female 68  58 FALSE
4  female 27 124 FALSE
5  female 84 103 FALSE
6    male 92 112 FALSE
7    male 35  65 FALSE
8  female 15 117  TRUE
9    male 89  95 FALSE
10   male 26  54 FALSE

But I prefer the more descriptive labels I gave previously.

> population
     sex   age income minor
1    male  68    150 FALSE
2    male  48     21 FALSE
3  female  68     58 FALSE
4  female  27    124 FALSE
5  female  84    103 FALSE
6    male  92    112 FALSE
7    male  35     65 FALSE
8  female  15    117  TRUE
9    male  89     95 FALSE
10   male  26     54 FALSE

Now let’s say we want to order by the age of the people. To do that is a one liner:

> population[order(population$age),]
 sex age income minor
8  female  15    117  TRUE
10   male  26     54 FALSE
4  female  27    124 FALSE
7    male  35     65 FALSE
2    male  48     21 FALSE
1    male  68    150 FALSE
3  female  68     58 FALSE
5  female  84    103 FALSE
9    male  89     95 FALSE
6    male  92    112 FALSE

This is not magic; you can select arbitrary rows from any data frame  with the same syntax:

> population[c(1,2,3),]
 sex age income minor
1   male  68    150 FALSE
2   male  48     21 FALSE
3 female  68     58 FALSE

The order function merely returns the indices of the rows in sorted order.

> order(population$age)
 [1]  8 10  4  7  2  1  3  5  9  6

Note the $ syntax; you select columns of a data frame by using a dollar sign and the name of the column. You can retrieve the names of the columns of a data frame with the names function.

> names(population)
[1] "sex"    "age"    "income" "minor" 

> population$income
 [1] 150  21  58 124 103 112  65 117  95  54
> income
 [1] 150  21  58 124 103 112  65 117  95  54

As you can see, they are exactly the same.

So what we’re really doing with the command




Note the trailing comma; what this means is to take all the columns. If we only wanted certain columns, we could specify after this comma.

> population[order(population$age),c(1,2)]
 sex age
8  female  15
10   male  26
4  female  27
7    male  35
2    male  48
1    male  68
3  female  68
5  female  84
9    male  89
6    male  92
Categories: programming, R Tags: ,

Non-edible baby? Is there any other kind?

February 11, 2010 Leave a comment

If I had a dollar for every time I read a warning like that

Categories: photo

Running totals in R

February 11, 2010 Leave a comment

Let’s say we wanted to simulate flipping a coin 50 times using the statistical language R, where a 1 is a heads and 0 is tails.

> flips=sample(0:1, 50, replace=T)
> flips [1] 0 1 0 1 0 1 1 1 1 1 1 1 1 0 0 1 1 0 1 1 0 1 0 1 1 1 1 0 0 1 0 0 0 0 0 1 0 1
[39] 1 1 1 0 1 0 0 1 1 0 1 1

Now we can plot the values to see which were heads and which were tails:

> plot(flips, main="Coin flips",ylab="0 = tails, 1 = heads")

Raw values of heads and tails

What if we want to see a running total of the number of heads over time? I was faced with just this problem for a completely different domain; I’ve written the function myself multiple times in Java and other languages but I was hoping it would be built-in to a stats language like R.  Fortunately I was right; the command you want is cumsum (cumulative sum).  There are a total of four functions like this:

Cumulative Sums, Products, and Extremes


They work just as you’d expect.

> cumsum(flips)
 [1]  0  1  1  2  2  3  4  5  6  7  8  9 10 10 10 11 12 12 13 14 14 15 15 16 17
[26] 18 19 19 19 20 20 20 20 20 20 21 21 22 23 24 25 25 26 26 26 27 28 28 29 30
> plot(cumsum(flips), main="Number of heads flipped over time",ylab="Number of heads")

Running total of number of heads

This is a trivial example, but it certainly simplifies my life.

Categories: programming, R

Quick hit – antialiasing in Java Graphics2D

February 9, 2010 7 comments

If you’re trying to use antialiasing in Java, chances are you’ll do some searching and come to the conclusion that you have to set the rendering hints for the Graphics object in order to eliminate your jaggies. Probably like so:

Graphics2D g2 = (Graphics2D) g;

And you’ll probably wonder why your graphics are still jagged. That’s because you haven’t instructed Java what type of interpolation to use for the antialiasing; by default you’ll get Nearest Neighbor, which is fast but won’t get you far in the image department. You can instead choose to use bilinear interpolation (slower) or bicubic interpolation (slowest).

Here’s how to enable the alternate interpolation modes:

Graphics2D g2 = (Graphics2D) g;



You make a trade off between speed and image quality; to my eyes bilinear looks fine. Compare for yourself:

Thanks to Chris Cambpell’s blog for showing me this.

Human fallibility – static analysis tools

February 9, 2010 5 comments

The theme of this post is human fallibility: no one’s perfect and we’re bound to make mistakes while coding.  The following tools can help statically analyze source or markup and find mistakes early.


I am a huge fan of Python as a scripting language, as its syntax and language features allow you to code quickly and with minimal fuss. (I will definitely use it as a springboard for future blog discussion, especially with respect to its differences from Java).  I am used to statically compiled languages like the aforementioned Java, where the compiler will catch typos and uses of uninitialized variables; not having this ability in Python always makes me feel a bit hesitant to use it for more than quick scripts and small programming tasks.  (Clearly Python is well-suited to large scale production environments; this is more my hangup and lack of expertise in the language than anything else.)

Enter PyChecker, an open-source project that aims to detect some of the most common coding mistakes, including references to variables before assignment, passing wrong numbers of arguments to functions, and calling methods that don’t exist.   It won’t find all your mistakes, but if you have any long-running Python scripts, you’d much prefer to catch a typo before you start running than 90% through the computation.


JSON is an alternative to XML as a “lightweight data-interchange format”.  Unlike XML with its opening and closing angle brackets, JSON has a very clean syntax with few extraneous marks.  Here’s an example JSON file:

    "number": 1,
    "array": [
            "nested list"
    "birthdayMap": {
        "Nick": "3/24",
        "Andrew": "12/1"

I was working on a project using JSON as its means of representing data when I ran into problems with my hand-generated JSON – I had made mistakes, omitting brackets, not closing quotation marks, or other silly mistakes.  The Java library I was using to parse the JSON read in the whole file as a String, meaning all the contextual information was lost.  When an error ocurred, I was left with a cryptic error message like “syntax error, unexpected $end, expecting ‘}’ at character 2752” with no idea where in the file the error lay.

Thanks to my coworker Dave, I found the excellent tool JSONLint which not only highlights the exact line and location of your syntax error, it also reformats your JSON to a consistent amount of indentation for each nested level.  JSONLint is indispensible if you’re fat-fingering your JSON code.


PMD is a plugin for NetBeans, Eclipse, and a host of other Java IDEs and standard text editors that warns you of bad coding practices in your source code, as well as alerting you to potential errors.  There are a variety of rules specified in either XQuery notation or Java code that can be turned on and off at will; for instance if you are doing a lot of coding with interfaces to C and need to use shorts and bytes, you probably won’t want the “Don’t use shorts” warning popping up on every line you use a short in.

Some of the most useful rules I’ve found are the fall through in switch statements,
Not all of the rules are cut-and-dried, which the website acknowledges with the addition of a Controversial Rules section.  Some might be due to stylistic differences or just disagreements over whether or not certain constructs are bad practice.  For instance,


Since: PMD 1.0

A method should have only one exit point, and that should be the last statement in the method.

This rule is defined by the following Java class:


public class OneReturnOnly1 {
    public void foo(int x) {
        if (x > 0) {
            return "hey"; // oops, multiple exit points!
        return "hi";

Doing a search for “java multiple exit points” reveals that there is definitely a lot of discussion as to best practices in this regard, hence its inclusion in the Controversial Rules section.

Hopefully you’ve gained at least one new tool from this post; if not I’ll be posting more in days to come.