Property based testing 101

Yet another testing talk?

This talk is different

People already said a lot about testing

Uncle Bob Sandi Metz

Why do we test?

Confidence
UTs are great for regression, quick to write
Non corner cases

Property Based Testing success stories

Riak
Level DB
Volvo

Riak - few calls to produce errors
Level DB - ~20 calls to make some errors
Volvo - errors in protocol, 200+ errors

How many tests are enough?

How many regular tests would we need to find LevelDB example?
LevelDB - only 2 functions - 2^n calls = 2^n unit tests

Unit Testing vs Property Based Testing

What does unit, integration, system etc tests have in common?
Example based vs Property based

What is property?

length(list) == length(reverse(list))
assert reverse(reverse(list)) equals list
assert reverse(list)[k] equals list[length(list)-k]

What else?

Generator
Shrinking

Generator

list <- generate_list

[1]
[-1, 3, 5]
[-50, -10, 3, 14, 25, 100]
etc

Shrinking

Let's assume reverse function that will remove element 42 if found

Failed: [-10, -8, ..., 42, ... 87, 100]
Shrinking...
Minimal failing test case: [42]

Sample test

list <- generate_list
k <- generate_int(0, length(list)-1)
assert length(reverse(list)) equals length(list)
assert reverse(reverse(list)) equals list
assert reverse(list)[k] equals list[length(list)-k]

Why should I care?

Even ruby, found some python stuff. Great for testing validation in oop

Bad stuff?

Time consuming
Learning curve

Takes time to write and run. Example with reverse - simple but need to find a lot of things.

Thank you & questions

Lukasz Skotarek

@dreat_ on Twitter

dreat on Elixir/3cityIt/HS Slack

mail: contact@dreat.info

Property based testing 101