TDD in type-checked FP languages


J. B. Rainsberger
 

Hi, folks. I'm curious about your experiences in test-driving in type-checked FP languages. I've worked a little in Elm and I'm playing around with Purescript. I have noticed a few things, but I'm curious about what you've noticed.

Does anyone here have industrial-strength experience doing evolutionary design in one of these languages? What did you notice about how you practised TDD?

Thanks.
--
J. B. (Joe) Rainsberger :: tdd.training :: jbrains.ca ::
blog.thecodewhisperer.com

--
J. B. (Joe) Rainsberger :: https://tdd.training :: https://blog.thecodewhisperer.com :: https://blog.jbrains.ca
Teaching evolutionary design and TDD since 2002


Arnaud Bailly
 

Hello J.B.,

This is a subject dear to my heart and I have had some experience doing various forms TDD in Haskell in the past years, after more than 10 years trying to practice in Java.
I even proposed a session on this very topic entitled "Type and Test Driven Development" at this year's DDDEurope but had to pull it back for personal reasons.

The first (and most) important thing I have noticed is that types play in part the role tests would have in other languages, in the sense that I use the compiler to provide feedback when I code, for example using so-called "holes" that can be filled somewhat automatically by the compiler. In essence, the compiler's output is just like another output of the dev environment, like tests, leading me to needed changes. This is particularly true when refactoring: changing a type, or a type's structure, leads to compiler errors to fix, then to test errors, then to more tests to introduce behavior.

The way I write code is usually very outside-in, starting from a high level function with some type, and then zooming in refining the type. I go back and forth between types and tests in this process. This might be supported in different ways depending on the language, and it's one of the reasons I have grown interest over the past few years with Idris (and dependently typed languages in general) as they promise to offer more opportunity to design code incrementally through a dialogue with the compiler.

Another types have changed the way I do TDD is that I try hard to write property-based tests instead of example based tests. Writing a property first, then letting the failing examples drive the implementation provides a great feedback loop that has the advantage of forcing you to cover a lot of cases you wouldn't usually care to cover.

Also, in Haskell especially, having a strict separation of pure and impure code has a strong effect on the way I design and write code: Because the compiler has a much easier task when working on pure functions, I try very hard to stay in the pure world which means pushing back effects to edges of the code, which of course leads naturally to port/adapters like "architecture".

Last but not least, working in an expressive type system makes it possible to have your code's language closer to your domain's, expressing "constraints" from the domain directly in the types rather than laboriously test-driving some encoding. Here is an example I have used recently in a conference which you might find interesting (unfortunately in French): https://github.com/abailly/xxi-century-typed/blob/master/atbordeaux2020/NIR.hs It's about representing a French SSN. I have tried to provide a strong case for better types by comparing a String-based representation with a strongly-typed one, with associated tests. 

-- 
Arnaud Bailly - @dr_c0d3


On Tue, Feb 9, 2021 at 12:44 PM J. B. Rainsberger <me@...> wrote:
Hi, folks. I'm curious about your experiences in test-driving in type-checked FP languages. I've worked a little in Elm and I'm playing around with Purescript. I have noticed a few things, but I'm curious about what you've noticed.

Does anyone here have industrial-strength experience doing evolutionary design in one of these languages? What did you notice about how you practised TDD?

Thanks.
--
J. B. (Joe) Rainsberger :: tdd.training :: jbrains.ca ::
blog.thecodewhisperer.com

--
J. B. (Joe) Rainsberger :: https://tdd.training :: https://blog.thecodewhisperer.com :: https://blog.jbrains.ca
Teaching evolutionary design and TDD since 2002


Brian Marick
 

Not what you asked, but when I was writing two (relatively simple) front ends in Elm, I deliberately didn’t write tests, relied on the compiler. I was disappointed in the results, as I found more problems after I was “done” than I’m used to. Most of them seemed to be faults of omission - “code not complex enough for the problem”. In theory, TDD isn’t about those, so shouldn’t cause me to realize them more quickly. In practice, it seems to, for me.

I worked a little on a Haskell project, mainly as a product owner and writing some tests for untested code. Not enough to have any conclusions, except that Haskell is a really nice language for writing tabular-style tests because of (1) relative lack of parentheses, (2) infix two-argument functions using backticks, and (3) precedence rules that didn’t seem to work against me. I forget the details, but I had a number of tests like:

f 1 2 3 `gives` 5

(Maybe that would require parens?)

Elixir is OK for tabular tests because of the pipe operator, but it’s not as good:

 


On Feb 9, 2021, at 5:43 AM, J. B. Rainsberger <me@...> wrote:

Hi, folks. I'm curious about your experiences in test-driving in type-checked FP languages. I've worked a little in Elm and I'm playing around with Purescript. I have noticed a few things, but I'm curious about what you've noticed.

Does anyone here have industrial-strength experience doing evolutionary design in one of these languages? What did you notice about how you practised TDD?

Thanks.
--
J. B. (Joe) Rainsberger :: tdd.training :: jbrains.ca ::
blog.thecodewhisperer.com

--
J. B. (Joe) Rainsberger :: https://tdd.training :: https://blog.thecodewhisperer.com :: https://blog.jbrains.ca
Teaching evolutionary design and TDD since 2002


Gregory Salvan
 

Hi,
My own experience isn't interesting as caml was one the first language I've learnt (25 years ago) and don't realise if changes I notice are due to the language itself or the paradigm.
So I'm curious too, what have you noticed JB ?


Le mar. 9 févr. 2021 à 12:44, J. B. Rainsberger <me@...> a écrit :
Hi, folks. I'm curious about your experiences in test-driving in type-checked FP languages. I've worked a little in Elm and I'm playing around with Purescript. I have noticed a few things, but I'm curious about what you've noticed.

Does anyone here have industrial-strength experience doing evolutionary design in one of these languages? What did you notice about how you practised TDD?

Thanks.
--
J. B. (Joe) Rainsberger :: tdd.training :: jbrains.ca ::
blog.thecodewhisperer.com

--
J. B. (Joe) Rainsberger :: https://tdd.training :: https://blog.thecodewhisperer.com :: https://blog.jbrains.ca
Teaching evolutionary design and TDD since 2002


J. B. Rainsberger
 

On Tue, Feb 9, 2021 at 11:03 AM Arnaud Bailly <arnaud.oqube@...> wrote:
 
This is a subject dear to my heart

I know. :)
 
[...] The first (and most) important thing I have noticed is that types play in part the role tests would have in other languages, in the sense that I use the compiler to provide feedback when I code, for example using so-called "holes" that can be filled somewhat automatically by the compiler. In essence, the compiler's output is just like another output of the dev environment, like tests, leading me to needed changes. This is particularly true when refactoring: changing a type, or a type's structure, leads to compiler errors to fix, then to test errors, then to more tests to introduce behavior.

This matches my experience. We had this discussion back in Rochegude about the desire to write tests for the pieces, but then wanting to trust composition to glue those pieces together.

The way I write code is usually very outside-in, starting from a high level function with some type, and then zooming in refining the type. I go back and forth between types and tests in this process. This might be supported in different ways depending on the language, and it's one of the reasons I have grown interest over the past few years with Idris (and dependently typed languages in general) as they promise to offer more opportunity to design code incrementally through a dialogue with the compiler.

This conforms to my mental model that says that the type checker runs mandatory microtests that I would otherwise probably not spend the time to write.

Another types have changed the way I do TDD is that I try hard to write property-based tests instead of example based tests. Writing a property first, then letting the failing examples drive the implementation provides a great feedback loop that has the advantage of forcing you to cover a lot of cases you wouldn't usually care to cover.

I noticed, when working in Elm, that I would begin with examples, then refactor towards property-based tests. It might be that I merely don't yet think "natively" in terms of properties.
 
Also, in Haskell especially, having a strict separation of pure and impure code has a strong effect on the way I design and write code: Because the compiler has a much easier task when working on pure functions, I try very hard to stay in the pure world which means pushing back effects to edges of the code, which of course leads naturally to port/adapters like "architecture".

Removing duplication from tests, especially of irrelevant details, tends to push the programmer to isolate pure code from impure code---at worst, they have pure core slightly contaminated with collecting parameters entirely in memory. When working in Elm or Purescript, I push myself to stay in pure code more ruthlessly.

Last but not least, working in an expressive type system makes it possible to have your code's language closer to your domain's, expressing "constraints" from the domain directly in the types rather than laboriously test-driving some encoding. Here is an example I have used recently in a conference which you might find interesting (unfortunately in French):

Pas grave, comme tu le sais déjà...

[...] https://github.com/abailly/xxi-century-typed/blob/master/atbordeaux2020/NIR.hs It's about representing a French SSN. I have tried to provide a strong case for better types by comparing a String-based representation with a strongly-typed one, with associated tests. 

I'll read this happily.

Over Christmas I spent some time trying to do the Advent of Code problems in Purescript. I learned a lot in only 2 weeks. I merely need more practice.
--
J. B. (Joe) Rainsberger :: tdd.training :: jbrains.ca ::
blog.thecodewhisperer.com

--
J. B. (Joe) Rainsberger :: https://tdd.training :: https://blog.thecodewhisperer.com :: https://blog.jbrains.ca
Teaching evolutionary design and TDD since 2002


J. B. Rainsberger
 

On Tue, Feb 9, 2021 at 3:48 PM Brian Marick <marick@...> wrote:

Not what you asked, but when I was writing two (relatively simple) front ends in Elm, I deliberately didn’t write tests, relied on the compiler. I was disappointed in the results, as I found more problems after I was “done” than I’m used to. Most of them seemed to be faults of omission - “code not complex enough for the problem”. In theory, TDD isn’t about those, so shouldn’t cause me to realize them more quickly. In practice, it seems to, for me.

I noticed that I wanted tests mostly to verify that I understood how Elm's MVC framework behaved and to check that my views rendered HTML correctly. Other than that, there wasn't much to test. Maybe that reflected the relative simplicity of my domain of choice.

I worked a little on a Haskell project, mainly as a product owner and writing some tests for untested code. Not enough to have any conclusions, except that Haskell is a really nice language for writing tabular-style tests because of (1) relative lack of parentheses, (2) infix two-argument functions using backticks, and (3) precedence rules that didn’t seem to work against me. I forget the details, but I had a number of tests like:

f 1 2 3 `gives` 5

(Maybe that would require parens?)

Elixir is OK for tabular tests because of the pipe operator, but it’s not as good:

 


These tidbits help. Thanks.
--
J. B. (Joe) Rainsberger :: tdd.training :: jbrains.ca ::
blog.thecodewhisperer.com

--
J. B. (Joe) Rainsberger :: https://tdd.training :: https://blog.thecodewhisperer.com :: https://blog.jbrains.ca
Teaching evolutionary design and TDD since 2002


J. B. Rainsberger
 

On Sun, Feb 14, 2021 at 11:27 AM Gregory Salvan <apieum@...> wrote:
 
My own experience isn't interesting as caml was one the first language I've learnt (25 years ago) and don't realise if changes I notice are due to the language itself or the paradigm.
So I'm curious too, what have you noticed JB ?

I notice that I want to test-drive the pieces, but not the glue. I trust the workflows, because they are mostly chains of compositions of functions.

I also notice that as I become more comfortable with the simpler monads (Maybe, Either, List), I find it even easier to glue pieces together using fmap and bind.

I never want test doubles, but I also rarely want end-to-end tests. Of course, there are no "stubs" in FP, but merely functions that return hardcoded values.
--
J. B. (Joe) Rainsberger :: tdd.training :: jbrains.ca ::
blog.thecodewhisperer.com

--
J. B. (Joe) Rainsberger :: https://tdd.training :: https://blog.thecodewhisperer.com :: https://blog.jbrains.ca
Teaching evolutionary design and TDD since 2002


Gregory Salvan
 


I never want test doubles, but I also rarely want end-to-end tests. Of course, there are no "stubs" in FP, but merely functions that return hardcoded values.

How do you test your edges ? your monads ?


J. B. Rainsberger
 

On Sat, Feb 20, 2021 at 1:12 PM Gregory Salvan <apieum@...> wrote:

I never want test doubles, but I also rarely want end-to-end tests. Of course, there are no "stubs" in FP, but merely functions that return hardcoded values.

How do you test your edges ? your monads ?

I don't write monads yet, at least not intentionally. I presume that if I write them, then I use the equivalent of stubs, except that they aren't really stubs, but rather just functions as parameters. I treat them in my mind like OOP interfaces.

As for edges, I presume you mean the point of integration with something outside of the system? I write Learning Tests for the thing outside the system, which are usually integrated tests, and then at the point of integration I choose whether to integrate with the real thing or with a function that invokes the real thing.
--
J. B. (Joe) Rainsberger :: tdd.training :: jbrains.ca ::
blog.thecodewhisperer.com

--
J. B. (Joe) Rainsberger :: https://tdd.training :: https://blog.thecodewhisperer.com :: https://blog.jbrains.ca
Teaching evolutionary design and TDD since 2002


Arnaud Bailly
 

Except for IO monads are values like anything else, at least in Haskell so “testing monads” usually amounts to testing functions.
For fine grained tests I try to avoid IO and use either the Handler pattern to group all IO actions together, or a custom implementation of a type class if I use mtl style.

Arnaud 

Envoyé de mon iPhone

Le 22 févr. 2021 à 00:53, J. B. Rainsberger <me@...> a écrit :


On Sat, Feb 20, 2021 at 1:12 PM Gregory Salvan <apieum@...> wrote:

I never want test doubles, but I also rarely want end-to-end tests. Of course, there are no "stubs" in FP, but merely functions that return hardcoded values.

How do you test your edges ? your monads ?

I don't write monads yet, at least not intentionally. I presume that if I write them, then I use the equivalent of stubs, except that they aren't really stubs, but rather just functions as parameters. I treat them in my mind like OOP interfaces.

As for edges, I presume you mean the point of integration with something outside of the system? I write Learning Tests for the thing outside the system, which are usually integrated tests, and then at the point of integration I choose whether to integrate with the real thing or with a function that invokes the real thing.
--
J. B. (Joe) Rainsberger :: tdd.training :: jbrains.ca ::
blog.thecodewhisperer.com

--
J. B. (Joe) Rainsberger :: https://tdd.training :: https://blog.thecodewhisperer.com :: https://blog.jbrains.ca
Teaching evolutionary design and TDD since 2002


Gregory Salvan
 


As for edges, I presume you mean the point of integration with something outside of the system? I write Learning Tests for the thing outside the system, which are usually integrated tests, and then at the point of integration I choose whether to integrate with the real thing or with a function that invokes the real thing.

If you write pure code you'd probably push your state changes, IO (monads in haskell)... to the edges of your program, you also probably inject your functions as arguments.

You asked about type-checked FP languages, it includes julia lang I've used as main language these last years (partially dynamic). I think it's a bit different because of multiple dispatch.
By injecting functions and not structures (immutable by default) you lose benefits of multiple dispatch.
So typically with a dummy ex. if you want to test function dosomething(res::AbstractIO, ...) where write(res::AbsractIO, ...) is called and your program use a structure of type IO <: AbstractIO.
I'd make a test double TestIO <: AbstractIO with a method write(res::TestIO, ...) and inject a structure TestIO instead of IO in dosomething function.
If domething calls open(res::AbstractIO) and close(res::AbstractIO) for ex. you don't need to override them if what you're testing is just write.

Honestly, I was not very comfortable with this at first, thinking of LSP... structures are immutable, you can't inherit concrete types (just abstract ones), so it's safe and efficient.
I still wonder if there's better solutions.




Gregory Salvan
 


Except for IO monads are values like anything else, at least in Haskell so “testing monads” usually amounts to testing functions.
For fine grained tests I try to avoid IO and use either the Handler pattern to group all IO actions together, or a custom implementation of a type class if I use mtl style.
What do you call Handler pattern ? do you have a dummy example ?


Arnaud Bailly
 

Envoyé de mon iPhone

Le 25 févr. 2021 à 20:29, Gregory Salvan <apieum@gmail.com> a écrit :

What do you call Handler pattern ? do you have a dummy example ?
It’s basically passing around a structure containing functions doing the IOs you need.

In Haskell, you can use various mechanisms to represent side effecting functions: Typeclasses (ie. interfaces which are resolved by the compiler or even dynamically depending on a type, think multiple dispatch), Free monads, or record-of-functions aka. Handler pattern.

Hth

Arnaud