Archive for May, 2010

Some Thoughts on Readability

Monday, May 31st, 2010

Communication is everything in software development.

Unless you work on a team of one, it’s important that your code communicates easily to your team members. Even if you do work on a team of one, it’s important to communicate to your future self. Yeah, everyone has to read their own code once in a while, and unless your brain is a steel trap, you’ll forget stuff.

So do yourself and your team a favor. Choose names carefully. Here are some examples.

Example 1: Use enumerations instead of booleans

This example is C++. Let’s pretend you’re writing a parse() function that looks like this:

bool parse(const char *file_name, employee *e, bool validate_only)

That seems pretty clear, doesn’t it? Let’s break it down:

Inputs:

  • The path to the file to parse (called file_name)
  • The employee struct to populate with the parsing results
  • A flag to control whether to actually parse the file, or just dry-run (validate_only)

Outputs:

  • A true or false return value to indicate success or failure
  • The out structure, populated on success

So far so good? Yeah, probably.

Reading the code so far, the purpose of each parameter is very clear, but don’t make people read your header files if they don’t have to. Consider the code that actually calls this function. It would look something like this:

employee e;
bool ret = parse("/tmp/foo.txt", &e, true);
if (ret) {
   ...

Without looking back at the header file, can you tell me what this function does? For the most part, it’s obvious, but that true parameter is not clear. I’m also not happy with the boolean return value (though it is the lesser of the two offenders).

Don’t make me think so hard. If I stumble across the code above, I shouldn’t have to go hunt down a header file just to figure out what it does. We can make this easier on developers by changing those bools to enums, like so:

enum parse_result { success, failure };
enum parse_mode { validate_only, full_parse };
parse_result parse(const char *file_name, employee *e, parse_mode mode);

By making these changes, callers would see this instead:

employee e;
parse_result result = parse("/tmp/foo.txt", &e, validate_only);
if (result == parse_success) {
   ...

Instead of true, we read validate_only, and it’s immediately clear what this function is doing. The returned value is also very clear. There is no need to consult documentation or other header files to figure it out.

Example 2: Mutants

In Python, you can remove white space from strings with the strip() function. However, if I stumbled across code like this, I probably wouldn’t realize that it had a bug:

s = "   foobar   "
s.strip()
print s # What does this print?

That’s because the strip() function doesn’t actually strip whitespace from the string, but rather it returns a new string with the whitespace stripped.

In this case, I would recommend that strip() be renamed stripped() or tostripped() to make it more clear that it returns a new string.

For methods that mutate their instances, I recommend present-tense verbs like strip(), rename(), or clear(). But for methods that return modified copies of the instance, I recommend past-participle names like stripped(), renamed(), or cleared().

Qt4 got this right with their changes to QString. Under Qt3, QString had a method called stripWhiteSpace(), so bugs like this were common:

QString str = "   foobar   ";
str.stripWhiteSpace();
// BUG: Is str actually stripped now?

In version 4, they renamed this method to trimmed(), so now there’s no question:

QString str = "   foobar   ";
str.trimmed();
// The bug is more obvious now

Example 3: Be specific

When reading others’ code it is common to come across words like data or ret or node. While there are exceptions to this rule, these words usually aren’t specific enough to communicate your intent.

For example, let’s say I’m writing a Python program to run Linux commands and store their results in a map. Perhaps I make a function like this:

def run_commands(commands):
    data = {}
    for command in commands:
        output = subprocess.Popen(command,
            stdout=subprocess.PIPE).communicate()
        data[command] = output[0]
    return data

This just runs each command given to the function and puts the command’s output into a dictionary. Obviously, this is a tiny function, and as such, it may be an exception to the rule. However, if this were a more complicated function, perhaps one that executed each command in a separate thread and aggregated the results, data would not be very clear. Instead, I’d recommend renaming data to command_results. I might even go so far as to name it command_results_dict so it could be perfectly clear. Obviously, this rule can be taken too far. I would not recommend, for example, using a variable name like command_subprocess_stdout_result_dict. That would be insane.

Conclusion

So there you have it. Just a few pointers that are probably obvious to most of you already. We can all stand to clean up our communication now and then, so take this as a chance clean up the code you write today. I know I will.

Happy hacking!

If you like writing lots of code, TDD is for you

Sunday, May 23rd, 2010

If there’s one thing you can’t argue about Test Driven Development, it’s that it creates more code than software written without it. Not just unit test code, either. To make code testable, developers often introduce extra layers of abstraction to make their code accessible, in isolation, from within their unit test code. This can be a good thing, but can also lead to a code explosion.

On second thought, I imagine someone actually would argue that, over time, a TDD codebase will have less code, for various reasons, possibly because it is easier to maintain, possibly because it encourages less code duplication, and so on. That’s fine, but this article is about the initial set of code that gets you to a shipping state, from scratch. We’re talking about version one point oh. The first set of working code that gets your product out the door, for the first time.

Okay, all clear?

How about an example?

Brett L. Schuchert, who publishes on ObjectMentor’s blog wrote an interesting piece of using TDD with C++ to write “Hello, world!”.

Yes, seriously: “Hello world!”

You know, the program that’s so simple, you can write it practically from memory?

With your eyes closed.

With one hand tied behind your back.

Yeah, this one:

#include <iostream>
int main() {
   std::cout << "Hello, world!" << std::endl;
   return 0;
}

Mr. Schuchert decided to create “Hello, world!”, from scratch, by strictly following principles from Test Driven Development. An interesting idea.

The result?

Nearly 80 lines of code, counting a Makefile that he felt compelled to create because it got too bothersome to compile with a single command.

Let’s look at that visually. Here are a couple graphs comparing the number of lines of code and the number of bytes used by the two approaches, TDD vs. Regular:

After looking at this example, I’d like to consider a couple questions:

  • Where would a bug most likely appear in this code?
  • What are the odds of a 79-line code base containing a bug compared to a 5-line codebase?

Let’s consider these questions in order:

Where would a bug most likely appear in this code?

Of the 79 line of code in the TDD codebase, only 2 lines are actually being tested, namely:

std::cout << "Hello, world!" << std::endl;

And

return 0;

The other 77 lines of code are well represented by lines like this:

TEST(HelloWorld, CorrectOuputPutToCout) {
    mainImpl(0, 0);
    STRCMP_EQUAL("Hello World!\n", stream->str().c_str());
}

And this:

int main(int argc, char **argv) {
    return mainImpl(argc, argv);
}

In the added TDD code, we find function calls with multiple arguments, string conversions between std::string and const char* types, unit test macros, and so on.

In this example, the added TDD code is actually more complex than the code under test. If I were a gambling man, I would bet that, given a randomly introduced bug, it would pop up in the added TDD code and not the code under test. Sadly, a bug in TDD code can render your tests useless.

What are the odds of a 79-line code base containing a bug compared to a 5-line codebase?

Unfortunately, I don’t have any empirical evidence to support an answer to this question. However, I have read and written a lot of code over the past 10 years. One thing I can say with confidence: A larger codebase is generally likely to contain more bugs than a smaller codebase. Obviously, there are exceptions to this rule. Some code is, by its nature, more complicated and bug-prone than other code. But, generally speaking, more code is likely to have more bugs than less code. In this example, we are dealing with more than a 10-fold code explosion. While I wouldn’t argue that we should expect a 10-fold bug increase, I would suggest that there will be added bugs, when compared with the smaller codebase.

But what about over time?

Yes, overtime, your unit tests, if properly written, can prevent new bugs from being introduced to existing code, for certain kinds of codebases. For, “Hello, world!”, however, this is most certainly not true. I can’t imagine a case where even the least skilled developer would introduce a bug into this code. So, if you agree with me on that statement, then the argument that TDD reduces long-term maintenance costs is completely lost on this example.

Conclusion

Mr. Schuchert argues that using TDD for this example was the “right thing to do”, I have to respectfully disagree. Mr. Schchert concludes that TDD helped by removing business logic from main(), which is generally a good thing to do. In this example, however, I disagree. I feel that the business logic (i.e., printing “Hello, world!” and returning 0) is so simple and unlikely to change in the future, that the added complexity of unit tests only makes the implementation less tractable and the code more costly to maintain. He introduces a build-dependency (the unit test framework) and added a layer of indirection for future readers to follow when learning the code. Further, the probability of introducing bugs has increased by the sheer volume of added TDD code. I conclude that while this was an interesting exercise into the TDD practice, I don’t think that TDD adds value in this example.

What’s Wrong With This Dialog?

Sunday, May 16th, 2010

Sometimes I can’t help myself. This week, I read a blog post asking what was wrong with a dialog. I found plenty of stuff wrong, so I decided to write a response and create a replacement dialog. This is not intended as an attack on the dialog author. I hope it’s not received that way. Rather, I just felt an itch to answer the question posted with my own opinion on the subject.

Read on for the gory details.

Deciding what’s wrong

When I find myself asking if a feature is “wrong”, the first thing I do is ask if the feature should even exist. That is especially important with dialogs. Dialogs interrupt a user’s work flow, and for whatever non-deterministic-human-psychological reason, in-line GUI elements tend to interrupt humans less than pop-up dialogs. I try to avoid dialogs, and pop-ups, almost religiously. So when I find myself asking, “what’s wrong with this dialog?”, the next question I ask is “should this dialog even exist?”. Unfortunately, in this article, I can’t tell whether that’s the case, so I’ll assume it must exist.

The litany

In my view, this dialog contains many mistakes. Its core functionality is certainly accessible, but it’s not as easy as it should be. Here’s my list of mistakes:

Now I’ll try to explain myself in prose rather than pixels.

Mistake 1: Too many buttons

This dialog provides a simple CRUD user interface, but it has 6 buttons. That’s too many buttons for such a simple task

Mistake 2: Wasted space

Don’t get me wrong. I’m a fan of liberal white space between elements. I think it puts users at ease, when used properly. But in this case, there is too much wasted space.

Mistake 3: Misalignment

The 6 buttons all have icons, but none of the buttons or text align vertically. This forces the human eye to read more carefully, causing unnecessary stress.

Mistake 4: No way to revert

If I make a mistake with this dialog, my only option is to manually revert my changes. Sometimes this approach is okay, especially with simple dialogs like this one, but as a user, I find it comforting when there is a “cancel” button to rescue me from my own fat fingers. I realize that the GNOME project usually follows the “apply-as-you-type” paradigm, but I prefer the OK/Cancel/Apply approach.

Mistake 5: Editing, Deleting, and Re-ordering is too hard

Although the “focus-and-click” method for editing, deleting, and re-ordering has seen a lot of use over the years, I find it dated and confusing to users. I much prefer to inline the “edit” and “remove” buttons as close to their respective elements as possible, reducing confusion about exactly what will be deleted when I click a button. Also, clicking “Up” and “Down” to move the focused item in the list is old school. Much better to drag and drop in modern user interfaces.

The Proposal

If I woke up in the UI designer’s shoes tomorrow, and this dialog was my number one priority, I would probably refactor it as follows:

Improvement 1: Easy editing

The actions of the “Edit” and “Remove” buttons are super obvious. To edit an item, click the “Edit” button right on it. No ambiguity. You also don’t have to worry about graying out buttons when there is no selection.

Improvement 2: Drag and drop

To re-order items in the list, just drag ‘em.

Improvement 3: Cancel

If you don’t like your changes, click “Cancel”. Also, if you want to see the effect of your changes without closing the dialog, click “Apply”. Very handy if you want to try out lots of little changes without having to re-open the dialog between each change.

Improvement 4: No focusing

Now that the “Edit” and “Remove” buttons are in-line with the list items, there’s no need for a selection/focus model at all. That’s one less thing the user has to worry about.

Improvement 5: Simpler layout

The layout is essentially vertical. The user sees two areas, one above the other, and the two areas do only one thing each. The top area contains the list of editable items. The bottom area deals with saving your changes (the “add” button is a small exception). I think this design is more approachable and easier for a user to understand quickly.

Conclusion

So that’s what I propose. A simpler, cleaner approach using modern methods to make the editing task easier on users. It doesn’t make them think about the user interface, but rather they are freed to think about the task they want to accomplish with the user interface.

Smart Folder Synchronization with Python

Wednesday, May 12th, 2010

I have various files that download to my computer automatically. They arrive at different times of the day or night. When they do, I like to transfer them to a network drive on another computer automatically.

But there’s a wrinkle.

When the files auto-download to my computer, they go into one folder automatically, but when I transfer them to my network drive, I like them to go into different folders, based on their file name.

Python to the rescue.

I whipped out this little script that runs from cron every 5 minutes on my Mac, rsync’ing files from my “Downloads” folder into a specific subfolder of “/Volumes/Shared”, following a set of rules. In this script, any file with the string “dave” in its name gets copied to the “/Volumes/Shared/Dave Stuff” folder. Any file with “bob” in its name gets sent to “/Volumes/Shared/Files for Bob”, and so on.

Here it is for your enjoyment:

#!/usr/bin/python

# Filename filters, and which folders to send them to:
filters = {
    # Filename  :  Dest Folder
    'dave'      : 'Dave Stuff',
    'bob'       : 'Files For Bob',
    'frank'     : 'Franks Junk',
    }

src   = '/Users/Dave/Downloads'
dest  = '/Volumes/Shared'
rsync = 'rsync --times '

# ----------------------------------------------------------

import os;
import sys;
import subprocess;

# Only show progress when we're running in a terminal (and not cron):
if sys.stdout.isatty():
    rsync = rsync + '--progress '

for dir, dirs, files in os.walk(src):
    for filename in files:
        if filename.startswith(".") or filename.endswith(".part"):
            continue
        fullpath = os.path.join(dir, filename)
        for filter, destfolder in filters.iteritems():
            if filename.lower().find(filter) >= 0:
                fulldest = os.path.join(dest, destfolder)
                print "Copying '" + filename + "' to folder '" + destfolder + "'"
                cmd = rsync + ' "' + fullpath + '" "' + fulldest + '/."'
                process = subprocess.Popen(cmd, shell=True)
                try:
                    process.wait()
                except KeyboardInterrupt:
                    process.kill()
                    sys.exit(1)
                break
        else:
            print 'Could not find a home for file "' + filename + '"'

When this script runs, it just blindly tells rsync to transfer the files, but rsync will only transfer them if they are newer in the “src” folder than the “dest” folder. That’s thanks to rsync’s “–times” argument, which tells rsync to preserve the file times when it does the transfer.

So far it’s working great. I called it “sync-download-files” and created a crontab file called /Users/Dave/etc/crontab, that looks like this:

*/5 * * * * ~/bin/sync-download-files >/dev/null 2>&1

Then I ran this crontab command to install the job:

crontab /Users/Dave/etc/crontab

And voila! Files now auto-sync to /Volumes/Shared every 5 minutes. When there are no files to sync, the script completes in a couple seconds.

By the way, in my case, /Volumes/Shared is a Samba mounted network share to a WDTV Live box.