1 * '1' = '1'

There are some days I wish Python is strongly typed. And today is one of them.

Background

I work on an algorithm trading desk. Our algorithm trading engine is in Python. For a few days my team has been working on a new feature which can assign a "weight" to each trading strategy, so for example if a strategy has a weight of 2, then each trading order it sends will automatically has its size doubled up.

Last night the development work was done. I run the tests, merged back to development branch, and deployed to our simulation environment.

This morning when I came into office, I was greeted by an error: basically it was shouting at me that the volume field in an order should be a number, not a string.

This has never happened before.

What happened

My first instinct is I could just add a simple int() wrapper around the volume field and back to my breakfast. Luckily I didn't and instead sat down to find the bug.

It turns out as we're reading the weights from a config file, but without properly converting it into an integer, so the weight stays as a string and everytime it's applied to order volume, Python won't complain about type mismatch, instead something like below happens:

1 * '1' = '1'

And an order with size '1', not number 1 is sent.

Why didn't my tests fail?

We have both unit tests and integration tests (where a dummy strategy will run throught the whole market data --> trading signal --> trading execution process).

In this case I didn't update the integration tests to include the weight. And in our code the default weight is 1 (NOTE: not '1'), so the tests happily passed.

Remediation

Just add an int() coersion when reading the weight from the config file.

I also included the weight field in my tests.

The real danger

It only occurred to me in the afternoon that if this has not been properly fixed, real disaster could happen:

Imagine a strategy with weight = 1 emits a trading signal with size = 3:

3 * '1' = '111'
int('111') = 111  #ahhhh fat finger

Now I don't even want to think about what 5 * '1' would turn out.

Takeaways

• NEVER blindly coerce types.
• Always update tests with new features.
• It's important to trace down to the root cause of a bug.