Test Driven Development, like a boss

Why, and how, to do TDD.

What is Test Driven Development, or TDD?

One way to write software is to write some code, then test it manually, make some more changes, test again, until you’re satisfied. Then, maybe, you write tests.

In TDD, on the other hand, first you write a test case (that fails), then write code to make it pass, make some more changes while making sure the test pass. Once you’re satisfied you run the whole test suite to make sure nothing else has broken.

Projects that follow TDD, end up having a robust test suite, and modularlized, testable code.

Why TDD?

There are a lot of reasons to practice TDD.

It forces you to write testable code

Have you ever found yourself making a small change and take like 5min to test it? Like, you need to “send a one-time SMS (text message) to new users on sign up”. So you do python manage.py runserver, delete the existing user using django admin, make an API call using postman to request an OTP, wait for the OTP, read the OTP from your phone, make another API to verify it, and then, finally, you get the “Welcome to my app” SMS. Just writing all these steps seems like such a task.

Instead, you could have just tested, "when a new user is created, call the SMS API with these params". It may sound trivial, but is surprisingly hard to do. You want to have loosely coupled pieces of code (or modules). One change here should not need changes (or worse, break anything) anywhere else.

Writing tests isn’t hard; its hard to write testable code.

Sounds silly, but:

TDD is useful even if you write the tests and don’t ever run them: you have modular and testable code.

A test suite gives confidence

Unsurprisingly, a codebase with high code coverage enables you to make changes to the code without worrying about regressions.

I take some time (5–30mins) to think about how I’m going to architect the piece of code I will be writing. I think of few test cases. Once this part is done, I become a robot: Write the test, check to make sure it fails, write the code, run the test.

My confidence level is so high that I only run tests. Sometimes, I use debug statements print or pprint but rarely do I use postman locally, or even python manage.py runserver.

Even when I write new celery tasks (celery is the background worker for the django web framework), I don’t even run celery locally.

I write a test to make sure the task is triggered appropriately (like when a user is created). Another test to call the task function directly and assert its side-effects. This practice probably deserves a post of its own.

I have face issues because of this lack of manual testing, but those incidents are few and far between; so I keep not doing it.

TDD in django/python

I started working on django since I joined apna.co and have been building the APIs for its android app. Been a real fun ride so far!

My typical workflow is that I use the atom text editor to write a test. I use either TestCase or APITestCase and almost specify some fixtures.

Then, in my bash terminal, I run something like:

python manage.py test --keepdb -- myapp.tests.RootViewTestCase.test_success

The above is to run the test_success test case in RootViewTestCase class.

I will talk about the few practices I find particularly useful in doing TDD with django.

Use the keepdb option

I always use the --keepdb option to run tests, like I have mentioned:

python manage.py test --keepdb -- myapp.tests.RootViewTestCase

This option reduces the test suite runtime by a huge margin- approximately by 10x in my case.

From the official docs:

--keepdb

Preserves the test database between test runs. This has the advantage of skipping both the create and destroy actions which can greatly decrease the time to run tests, especially those in a large test suite. If the test database does not exist, it will be created on the first run and then preserved for each subsequent run. Any unapplied migrations will also be applied to the test database before running the test suite.

When I first learnt how does keepdb preserve the database, my mind was blown! Every test case is wrapped inside an SQL transaction, which is rolled back at the end of the test. This means super fast setup and teardown’s of data (including fixtures). Later I learnt this is fairly common in most frameworks involving SQL databases, but nonetheless 🤯🔫!

Another subtle feature of keepdb is that it runs any new migrations. That is, if I create a new migration, and then run python manage.py test — keepdb, my tests still run very quickly.

Testing API response data

I don’t unit test the entire json response; just the parts that have some business logic in them. For example, a GET user API returns my (i.e. logged-in user’s) phone number when I request my user, but a masked version of the phone number for another user. Then I will write tests like:

def setUp(self):
    self.user = User.objects.get(id=1)
    self.client.force_authenticate(self.user)
def test_own_user(self):
    response = self.client.get('v1/user/1/')
    self.assertEqual(response.data['phone_number'], '+91' + self.user.phone_number)
def test_other_user(self, on_profile_viewed_delay):
    response = self.client.get('v1/user/2/')
    self.assertEqual(response.data['phone_number'], '+918709XXXXXX')

No manual testing?

No!

Sometimes, I just print the response data inside the test case, to be sure everything looks okay:

from pprint import pprint
pprint(response.data)

The above is the maximum extent of manual testing I do. Rarely do I use runserver to run my server locally.

Unit testing gives me enough confidence to deploy my code into the staging environment, without running the web or background processor server locally.

There is too much to cover about the specifics of how I do TDD. I will cover that in another post. Like testing when external API calls are involved, using celery, Djnago Rest Framework (DRF), etc. Do reply or tweet to @vedantsopinions on something you would like to know or are curious about.

Once I learnt TDD, I could not develop software any other way. I have tried but failed to not do TDD:

TDD is just f’ing amazing!

Still gives me goosebumps on how can a philosophy be so simple and counter-intuitive, yet so effective.

TDD made me look at software engineering from a different perspective