Testing private methods, TDD and Test-Driven Refactoring

While being a TDD--Test Driven Development--coach, I am always asked about testing private methods. I finally decided to write about my experience on testing private methods.

When really doing TDD (i.e. writing the test code before writing the functional code, and then refactoring the code) private methods are guaranteed to be test covered. When driving the functional code design and implementation from the unit test (aka Test Driven Development), no method is created private. Instead, the private methods are extracted—extract method refactoring step--from a public or package level method. The Example 1 below presents a scenario on applying TDD and how the private method is created.

New code development following TDD

Example 1: new code development following TDD

Consider the development example following a TDD sequence for creating methodA and methodB.

1. create testMethodA/ create methodA/ refactoring
2. create testMethodB/ create methodB/ refactoring: extract methodC

On the first sequence, the testMethodA is created for validating the functionality expected of methodA. The methodA is successfully created: All tests including the testMethodA are passing. Then you look for improvements in the code; refactoring.

On the second sequence, the testMethodB is created for validating the functionality expected of methodB. The methodB is successfully created. All tests including the testMethodB are passing. Then you look for improvements in the code; refactoring.

While looking for improvements in the TDD Refactoring step, you recognize that methodA and methodB have a common fragment of code (code duplication). You extract the common code fragment into a private method whose name explains the purpose of the method--methodC. Then methodA and methodB invoke methodC.

In this example testMethodA covers methodA, which invokes private methodC; therefore the private methodC is test covered.

Please keep in mind that this is a simplified example. Your methods and tests should not read like testMethodA / methodA, and testMethodB / methodB. Jeff Patton describes how test cases can describe design in his Test-Driven Development Isn’t Testing paper.

When doing TDD, the private methods emerge from your code. And because you test-drive your development, no functionality is added without a test. So, for code fully developed by following TDD you won’t have to test the private method separately: Private methods are already being tested by previously written tests.

Improving legacy code following Test-Driven Refactoring

Now let’s look into a more realistic example. The majority of the code I have been working on is not new code; therefore I am not doing pure TDD; instead, I am doing Test Driven Refactoring,

Test-Driven Refactoring

Test-Driven Refactoring is an evolutionary approach to improve legacy code which instructs you to have test-proven refactoring intent. Basically, you start by writing a passing test around the code to be improved, and then you refactor the code; improving its internals, yet still passing the test suite.

While doing Test Driven Refactoring, I am trying to perform small refactoring steps and, at times, I find myself attempting to test a private method. This typically happens when I am working on an existing code base with very low test coverage. And the public methods are too complex to write tests for.

Example 2: test driven refactoring for existing low test coverage codebase.

Consider that you want to improve the following code:

public void overstuffedMethodX(){

…

// very complex code

…

// invoke private method methodY()

someString = methodY();

…

}



private String methodY (){

…

}

In the scenario presented in Example 2, the public method does not have corresponding unit tests. And I don’t feel comfortable refactoring code which does not have tests around it. Therefore I will follow Test-Driven Refactoring for improving the code. Below I will explain two different approaches for doing Test-Driven Refactoring for improving the code in Example 2.

Top down Test-Driven Refactoring

First you create tests for the complex public method:

public void testOverstuffedMethodX(){…}

At this point there is test coverage around the overstuffedMethodX() functionality, so you are able to refactor the overstuffedMethodX() code, including refactoring for the private method methodY().

In the top down Test-Driven Refactoring approach, first the unit test for the public method is created, and then its internals (including the private methods) are refactored.

Let’s now look into another approach.

Bottom up Test-Driven Refactoring.

No test for the complex public method is created.

Instead you look for smaller pieces of improvement.

You change the access level for the private method to make it accessible from a unit test.

private String methodY (){}

becomes

String methodY (){} // package level access in Java

Then you write test for methodY()

public void testMethodY (){…}

Then you refactor methodY(), and verify that the improvement works as the testMethodY() test still passes.

In the bottom up Test-Driven Refactoring approach, you first improve the test coverage and the code for the private methods and the internals of the complex overloaded method. By doing so, the code becomes less complex; after that you can start moving up the chain, increasing and broaden the test coverage until you are able to take care of the public method.

When applying the bottom up Test-Driven Refactoring approach for Example 2, the private methodY() is made package level in order to be accessible by its corresponding unit test (consider the package access level for the Java language). Similarly to testMethodY(), other tests are added in a bottom up approach. After increasing the test coverage and improving the code internals, it becomes easier to create testOverstuffedMethodX(), and finally, refactor the overstuffed complex method.

Top down versus Bottom up Test-Driven Refactoring

Even I consider the top down approach to be more purist as it does not change the private methods access level, its implementation is not always straightforward. The public method might be almost “untestable” (e.g., static, dependencies, long, complex) in its current state. For such cases, a bottom-up approach might be more suitable. As I see it today, code refactoring activity is a combination of bottom-up and top-down approaches that enables you to have small proven steps towards a cleaner solution, which still keep the same interface (the external behavior which is verified by the test suite).

Bottom-line, while doing Test Driven Refactoring, I have provisionally added tests for the private methods. In Java, I deliberately remove the private namespace from the method declaration, changing the method access level to package access level. This way, the corresponding test class--located under the same package--can invoke the method and test it.

But I won’t stop the refactoring until I remove the test for the private methods. I have experienced two refactoring sequences which finish without explicit tests for private method.

In the first refactoring sequence, some later refactoring step moves the private method to a different class. Basically, you realize that the method’s functionality is beyond the original class’s purpose; you identify, create the new class, and move the method – which now is public access level.

In the second refactoring sequence, the method access level goes back to being private and you delete the test which was directly invoking it. Because of the increasing test coverage--as a result of the Test Driven Refactoring--new test scenarios are added for the public method which invokes the private method. Once you (perhaps assisted by test coverage and analysis tools) realize the extra test coverage on your private method, you perform my preferred refactoring step--unnecessary code deletion. The test for the private method can be safely deleted as its functionality is being tested by another test method.

Should I add test for private methods?

So, here is my answer to the test private method question: After successfully doing TDD or Test-Driven Refactoring, your code will not have specific tests for the private methods.

If you are developing new code, you are following TDD. You test-drive the development and no functionality is added without a test. Private methods are only created as the result of a refactoring step. And the path of code going through the private method is already being tested by previously written tests. So, after successfully doing TDD, your code will not have specific tests for the private methods.

If you are improving a legacy code, you should be following Test-Driven Refactoring. In this case, you may provisionally add tests for private methods. Gradually, with the increasing test coverage, the tests for the public methods will cover all the paths, including the paths going through the private methods. At this point, you don’t require the tests for private methods anymore. So, after successfully following Test-Driven Refactoring, your code will not have specific tests for the private methods.