Inside F#

In my previous blog entry, I mentioned how pipelining interacts with type inference.  I wanted to find a natural example to talk about writing pipelines in F#, and Wikipedia helped out.  On this Wikipedia page describing unix command-line pipelining, there is an example of a unix pipeline that spell-checks a web page at a given URL.  So let’s write a similar tool in F#.  The logic of the individual steps in F# is a little different from the unix example, but it’s nevertheless still a great example of a long pipeline in action.

Let’s examine the pipeline (the last six lines of code) more closely.  We start by fetching a page from the web (a page I clearly picked at random, with absolutely no significance whatsoever :) ) using our HttpGet function, which returns the contents as a string:

Then we pipe this string into a function that produces a new string where all the non-word characters have been replaced by spaces:

Then we pipe that string into a function that splits the string on whitespace into an array of words:

Then we’d like to sort that array and get rid of duplicates, and we can do that simply by creating a Set of the words:

Then we want to filter down the set to only those words which fail to spell-check:

And finally, we print out each remaining (misspelled) word in the Set:

That’s a fine example of code that reads very naturally in a long pipeline. 

(Pipelining also demonstrates one of the benefits of well-authored functions that take arguments in curried form rather than tupled form.  However I’ll save those details for another blog entry.)

Of course, we could have written our web-spell-checker without using pipelining.  In one big expression, it looks like this:

Yuck.  This looks similar to how you’d write this code as one huge expression in any language (e.g. C#), and it reveals the problem with using huge expressions: they’re very hard to read.  This is true especially because expressions evaluate inside-out, which means the "first thing that happens" is nested in the middle of the expression, and then its result is passed as an argument to some function surrounding it, which is passed as argument to a function surrounding it… Code with deeply nested function calls just reads unnaturally to humans, since it’s easier to grok things in small bits in the sequence in which they occur.  As a result, the other main way to write such code without pipelining is to name each intermediate expression:

That code reads pretty well.  The functions now appear in the order they will be executed, and each line "does one thing".  Each intermediate result is named, and then referenced by name.  Depending on the exact nature of what you’re coding, these names can be either a good thing or a bad thing, I think, either helping or hindering the readability of the code.  In general, I feel you should name intermediate values only if the name you introduce adds readability-value to the code by explaining what’s happening.  In this example it’s actually a pretty close call, assuming the reader has a modest familiarity with the functions in the Regex class and the Set module.  So you’ll have to tune your personal coding aesthetic to decide whether to code like the example above with all the "let"s, or using the pipelining style:

Of course, it’s not all-or-nothing.  A mix of styles often produces the most readable code, and I am rather fond of:

for this example.  I think I like this form of the code because it enables the eye to scan down the left side of the code and see the major conceptual tasks happening: we’re going to get a page, we’re going to compute the words, and we’re going to do something with those words.

So there you go.  Hopefully now you have a better understanding of the pipeline operator "|>" – how it works and when you might choose to use it.  Given what a simple thing it is (recall that "x |> f" just means "f x"), this little operator sure provokes a lot of discussion!