You don’t get big by writing tests

You don’t get big by writing a lot of tests (or checks). You get big by getting stuff done with competent people that pay attention to the changes they apply. YMMV – bradfordw

Source: http://news.ycombinator.com/item?id=3541317

An example of running unit tests – jetbrains.com Resharper

I vehemently disagree with this statement for three reasons.

First, it is extremely naive.  It implies that one can avoid the need for automated tests just by being an assiduous programmer.  If you are the sole developer on a project, this may be true (I doubt it).  Once you involve other programmers and the code is composed of different loosely coupled systems (as befits a good design), there is absolutely no way that a programmer can manually ensure that his changes are not breaking other code, even if his own code is perfect.

Second, it draws a false distinction between the behaviors of writing tests on the one hand and being competent and getting things done on the other.  Yes, writing tests slow down development in the short term, and in that sense are a hindrance to ‘getting things done’. In the long run they are absolutely crucial to the health of the code base.  Why?  Let me count just a few of the ways.

1. Well designed tests help you catch many common errors (fencepost errors, typos, mixed up conditionals, overflows, underflows, etc.)

2. Tests provide good documentation in the form of usages of your code to clients.

3. By coding tests which exercise the contract (external API) of your class, you have a safety net for refactoring and improving it.  You can swap algorithms, data structures, etc., while having some assurance that your code still performs correctly.

4. Tests allow you to prevent regressions.  If you fix a bug once, you can add the code that exercises that failure condition to the test suite and ensure that it does not creep back in with future maintenance.

While some of this may be possible to verify manually each time, it is incredibly wasteful of engineering time and talent.  Something as important as software testing and verification should not be left up to manual tests.

Third, and perhaps most importantly, it is patently false.  Large companies have many thousands (millions?) of tests.  The bigger the reach of software, the more potential cost a software error can have, and thus the more engineering effort is spent towards alleviating that risk.  The larger the software becomes, the less possible it is for a single person to understand the entire the system and to know all the possible repercussions of a change to his code.

I don’t know if the original poster was trolling or not, but it gave me a chance to collect some thoughts I’ve had about testing. When I was in college, I barely wrote tests of any kind unless they were explicitly required. The coding I did was mostly for projects that were completed in a week, submitted, and never seen again. After I joined the Northern Bites robotics team, I started working with a real, 100K+ SLOC code base that had evolved over years. There it was immediately drilled into my head the importance of testing. The fact that multiple people touched the same module, and that the same module might outlive your time on the team by years, made it absolutely crucial to test thoroughly.

It was also crucial to save time. We worked with Sony AIBO robots, and to put the new code on them involved cross compiling, putting the code on a memory stick, turning off the robot, inserting the stick, rebooting the robot, then waiting for the code to turn on. This easily took a minute or two each time. The more of the testing that could be performed automatically in software via unit tests and integration tests, the less time I had to spend in the painful compile/execute/debug cycle using actual robots.

Northern Bites in competition

Once I got to my first job, I mostly did software prototyping, meaning the quality of the code did not have to be incredibly high – they were proofs of concept, and were not intended for production. Nevertheless, I took with me the lessons I learned from the robotics team, and found that writing unit tests up front really did save a lot of time in the long run. Just as with the robots, it’s a lot faster to exercise the system via repeatable, automated tests rather than manually launching an app and verifying behavior.

I’ve since moved on and spent the last two years at a very large tech company, I am absolutely convinced of the efficacy and importance of automated tests. The smallest change can have unintended consequences, even when that code change is reviewed and signed off by other engineers. For instance, we recently had a case where someone changed a single flag to the empty string and it ended up breaking an entire pipeline in production. It was only configuration that was being changed, not code proper, yet it took down the system. Had there been a test that exercised the handling of the empty string, we would have prevented many hours of wasted effort. This sort of thing happens even with extremely smart, talented, conscientious people. I shudder to think how code bases devolve with only manual tests.

I’ve heard the expression ‘you play how you practice’, and it applies equally well to sports, music, and coding. The sooner you learn the importance of testing, the better. Even on my hobby projects, I will rarely write a line of code without starting with the tests. I encourage anyone reading this to do the same. The original poster claims that you don’t get big by writing tests. In my mind, you don’t get anywhere without writing tests.

JUnit’s clever assertEquals method

I ran into a feature of JUnit the other day that is very useful, and a neat touch for the developers to put in; it really shows the authors’ attention to detail.

JUnit, for those uninitiated, is a unit testing framework for Java.  JUnit provides a set of statically available assertions you can invoke within your test cases in order to ensure that your software is behaving as you think it should.  They are accessible from the org.junit.Assert package and can be statically imported as follows:

import static org.junit.Assert.*;

This allows you to reference the methods without prefixing them with Assert; in other words you can call assertEquals(Object first, Object second) as opposed to Assert.assertEquals(Object first, Object second).

Anyways, JUnit provides a whole raft of assertEquals methods, overloaded for different types. I was pleasantly surprised to see that JUnit is clever enough to provide a ‘diff’ed output when the assertions fail. This makes it much easier to see where the failure occurred, as the difference in output is highlighted and obvious. Here are some examples:

import org.junit.Test;
import static org.junit.Assert.*;

public class AbsoluteTest {

    @Test
    public void testStringAssertEquals() {
        assertEquals("Hello","hello");
    }

    @Test
    public void testArrayAssertDifferingLengthEquals() {
        assertArrayEquals(new int[]{5,7,8,10},new int[]{5,7,8,9,10});
    }

    @Test
    public void testArrayAssertSameLengthEquals() {
        assertArrayEquals(new int[]{5,7,8,10},new int[]{5,7,8,9});
    }

}

Here is the output:

Note that I used assertArrayEquals as opposed to assertEquals for the arrays; doing so compares arrays and their elements rather than just reference equality. Here is the same test with the method switched:

This is because the assertEquals method that takes two objects merely invokes the standard reference equality on the objects, and gives the toString representation of the arrays when they fail. Final little tip that relates to the previous point: if you ever need to print out the contents of an array in Java, Arrays.toString is your friend.

int[] array = {5,6,2,17,55};
System.out.println(array);
System.out.println(Arrays.toString(array));

Results:

[I@744a6cbf
[5, 6, 2, 17, 55]

The Arrays class has a lot of useful utility functions when working with arrays, including toString, and equals.