Generated unit tests are a net negative

Posted by Joe Wilson on Tuesday, October 12, 2010 6:30 AM

One of the most common misunderstandings for developers new to unit testing is the value of code coverage.  We intuitively know that if coverage is low, we don't have enough tests, and if coverage is high, we probably do.  Some IT shops even set minimum code coverage percentages, so if you check in code that drops the coverage metric below 80%, you fail the build.

One way to get higher code coverage is to generate the tests.  Visual Studio has versions that can do this for you.  Point the code at a class, click OK a few times, and all your code in the class under test can be covered.  In fact, the generated test usually check for things you might not bother to, like what if there is a negative value passed in to a method, what if a zero passed in, what about a null, etc.  How can this be bad?  We're exercising the API of our class more fully, right?

What not to test

I think the focus needs to be on testing the right code in the right way.  There is no point in testing the .NET framework.  If you have a auto property that is an int, you don't need to test -1, 0, 1, and null.  If int stops working, everyone using .NET has a bug.  But int isn't going to fail, and even if it does, it's not your code anyway (unless you work at Microsoft on the .NET framework team).

So the first rule is be sure you're testing your code.  That means testing your methods that do real stuff.  If you have a method in your class that just wraps a call to another class, there really isn't anything worth testing.  All you can test is that the class can be created and the method can be accessed and has those parameters and they are of that type.  But you can do that better by testing the underlying class instead of the pass-through method.

Go faster when refactoring

In fact, testing these properties and pass-through methods and no arg constructors increases your code coverage numbers without adding anything meaningful to the test suite.  The net negative comes into play when it's time to refactor, whether it's adding or removing properties, renaming, moving things to different classes, etc.  Generated tests can really slow you down here as the new code stops building and the generated tests have to be removed and regenerated.

A better way is to write the tests you need first, then add the code to implement that business rule or feature.  Whether you're using TDD, BDD or something else, you'll end up with tests that express the intent of the code rather than its implementation.  Tests should emphasize the features of the application, not the coding artifacts.  These are the kind of tests that add real value to your project.

Tags: Unit Testing, Refactoring, TDD, BDD
Categories: Technical

Do we still need coding standards?

Posted by Joe Wilson on Tuesday, July 6, 2010 6:31 PM

The dark ages

I used to be really big on coding standards in the late 1990's.  It was so easy to slip into spaghetti code in classic ASP and VB6, and at the time, the best antidote was rigid standards and uncompromising code reviews. 

I was a hard ass about our coding standards.  Then, as now, the rationale was that coding standards would make the code easier to read and maintain in the inevitable and long maintenance phase.

The development process we used at the time was to stand around the white board and talk about the development tasks, then the developers would go off and separately code it up.  That was the other reason the coding standards mattered - we didn't want two code files to look totally different if written by different developers.

My teams used the naming conventions Microsoft came up with for Access/VBA/VB, including Hungarian notation prefixes, but a lot of the standards were about code comments, error handling code, whitespace and indenting, etc.

.NET comes on the scene

Fast forward maybe 5 or so years to the introduction of .NET and C#, and coding changed.  ASP.NET HTML didn't require <% %> tags anymore.  We could put the C# in code behind files and other project files.  We didn't need On Error anymore, we had Try-Catch.  XML comments seemed like a cool feature to show Intellisense drop downs for code you wrote in addition to framework Intellisense.

So we updated our coding standards to reflect the language and IDE changes, but they became more like guidelines instead of hard and fast rules.  By convention, we would avoid the spaghetti code just by keeping the right code in the right place.  Hungarian notation went away (slowly but surely).  The coding standards document was mostly reference code snippets and justifications for the conventions we used.  Code reviews became less common because it wasn't as big of an issue.

Today

A lot has changed since .NET first came out, and I code differently now.  I'm trying to make my code a lot more terse these days.  I'm a big fan of having clean, succinct, expressive code.  The fewer non-essential things on the screen to wade through, the better.  It's more readable and easier to refactor.

  • I could put Try-Catch blocks everywhere, but isn't it simpler to catch unhandled exceptions in one place.
  • I could add XML comments, but why?  It's either code in my same web project or a project that is owned by my solution.  I don't usually work on common library code where an assembly will be used across projects and Intellisense might be needed.
  • I don't put the variable type on the left side of the equal sign anymore and use "var" instead.
  • I've walked away from code regions and just order the items in the class file by convention.  ReSharper likes to put new fields under the last field declaration in the class, so region wrapping all the fields in a class with "#region Fields" just wastes time when refactoring to pull it back up into the region.

Not only has the way I code changed, but the tools have changed. ReSharper code cleanup templates and StyleCop warnings can be used to keep your team writing code in a style that you've all agreed on.  Just run the tool and have it clean up the code for you or flag code that doesn't meet the style guidelines.

Having tests around the code lets me refactor and use auto-code-cleanup tools freely.  I understand the project a little more fully every day, so I can rename my class, method, and variable names to be more descriptive and accurate without worrying about breaking code.

I'm also doing more pair programming these days, and this is a good way to enforce and socialize team coding styles.  It also means we don't have to refer to a coding standards document as often.  Since we're more story-focused, we're thinking more about the feature and less about coding style.  Taking ceremony code off the screen helps us keep that focus.

If I had to do it over

If I was going to write a coding standards document today, it would be more like a style guide to reflect my current less-is-more thinking, and I'd probably give my peers a ReSharper template to auto-clean their code as they go instead of a document to refer to.  StyleCop for ReSharper looks promising as a gentle background reminder of team coding styles.

What about your team?  Do you have a coding standards document people refer to?  Do you use tools to enforce standards/styles?

Tags: Coding Standards, Refactoring, Unit Testing
Categories: Technical

The case for TDD

Posted by Joe Wilson on Friday, July 17, 2009 7:39 PM

TDD is hard. It sounds like it wouldn't be hard.  It's just a little test.  The code is the hard part, right?  But it's really hard to do it well. I struggle with it plenty and I know my way around C# and OO pretty well.

There better be a payoff for this bass-ackwards way of coding.  It takes a while to get there, but I can see the light at the end of the tunnel in my own development.  Here's where I've seen my code improve as I've moved toward TDD:

Focusness

You focus on the behavior of your code instead of the implementation.  You don't start by thinking of tables, classes, and properties (or in my case, try hard not to).  You start off thinking of the business requirements and roles.  User stories are popular because they push you into restating the requirements in context: "As a (some role)...I want to (do something)...so that (some business value)".  The first two chunks of this can guide your test creation.  You can write methods that cover these scenarios.

TDD Logo If you're doing it the RED-GREEN-REFACTOR way, you'll write a test and just enough for the code to compile, get the test to pass with the minimum amount of code possible.  Finally, you can go back and clean up what you aren't proud of from your first effort, then run the test again.  You get to keep repeating that until you are happy with the code and that the requirement/story is met and the user accepts it.

Then you STOP.  That's the fourth step you don't see in the pretty RED-GREEN-REFACTOR icon.  If you don't stop, you loose your focus on the functionality and you start trying to gold-plate the code.  Which can lead to...

Dude, you know what would make this REALLY cool?

You're getting paid to write software to meet the requirements.  If you going beyond that, be careful who you're doing the coding for.  If it's the business, and you're glad to show them how much time you've spent on it, you might be justified in doing it.  But if you're coding it for yourself, do it on your own time.

Unfortunately, people who are drawn to programming are often tinkerers and bit-twiddlers.  I appreciate having a framework for my development that keeps me on task, because I am easily distracted by grand visions of what I might be able to code into the app.  There is no code we couldn't tweak a little and add this new cool pattern or just-learned technique to.  TDD forces you to go directly to the requirements, not the technology.

SOLID code

The more you move toward TDD, the more little classes you have with giant names.  Same thing with methods.  There are tons, they are very short (one screen or so max), and they have long names.  This is progress?

Yes, I think it is because of the SOLID principals.  SOLID is a horrible acronym of acronyms, but it can help you keep some object-oriented principals in mind.  I'll go over it and how I've tried to learn it in more depth in a future post, but the guys at Los Techies covered SOLID in depth already.  So the side effect of decoupled code is you have lots of little files that just do one little thing.  It took me a while to get over this and the file bloat in the project, but I'm at peace with it now. The payoff is that I am using more SOLID principals than I was before, and that's a good thing.

Oh yeah, you get some tests, too

It's been said before, but TDD is kind of misnamed.  Yes, you start with the test, but that's not the most important thing.  The most important thing is the requirement focus and the way it drives you toward a more loosely coupled design.  But it's pretty fun to see that screen light up with green checks or circles after you've made a change.

The real value of the tests themselves is that today's unit tests become tomorrow's regression tests.  You won't remember why you wrote the code last week, but you don't have to.  The tests can also serve as mini-documentation of the requirements if you write the test names in a consistent way.  There are BDD frameworks that let you spit out some HTML to do exactly this.

Ruthless Refactoring

All this adds up to something of a safety net for refactoring. Think of the confidence you can have changing the code, even if it's another team members code.  Or if (gasp!) they make changes to your code.  It's nice to know your code still works.

If you write the tests first, you know you've got the "happy path" covered on your code.  That's the one you started with.  You can write a few more tests to cover the likely deviations from that happy path, but when you get to the weird stuff (what if you order $999,999,999,999 in widgets?), call it good and move on.

Tags: TDD, SOLID, Refactoring
Categories: Technical

Blog links

  • Subscribe to this blogRSS feed
  • Archive of old posts

Popular posts

  • Autocomplete dropdown with jQuery UI and MVC
  • Handling Exceptions in ASP.NET MVC
  • Don't mock HttpContext
  • Review of Sharp Architecture
  • Evolution of a View in ASP.NET MVC
  • Comparison of Typemock Isolator and Rhino Mocks
  • Building a Windows 8 Live Tile with JavaScript
  • Setting Default Values for Multiple Value Parameters in Reporting Services
  • Buy, Build, or Both?
  • Why bother writing unit tests?

Tag cloud

  • AppHarbor
  • Areas
  • ASP.NET
  • ATDD
  • BDD
  • Castle Windsor
  • Coding Standards
  • Common Service Locator
  • continuous integration
  • Cookies
  • CRM
  • CSS
  • Custom Software
  • Data Annotations
  • DataTables
  • DDD
  • Dell
  • Dependency Injection
  • DTOs
  • ELMAH
  • git
  • GitHub
  • Html Helpers
  • HttpContext
  • IOC
  • iPad
  • iPhone
  • JavaScript
  • jQuery
  • jQuery Mobile
  • JSON
  • Kendo UI
  • Knockout
  • Microsoft Accounting
  • Moq
  • MVC
  • NHibernate
  • NuGet
  • NUnit
  • OData
  • optimizations
  • Patterns
  • POCOs
  • QuickBooks
  • Rails
  • Refactoring
  • Reporting Services
  • REST
  • Rhino Mocks
  • Session
  • Sharp Architecture
  • SOLID
  • SpecFlow
  • SQL Server
  • SSRS
  • TDD
  • TeamCity
  • TempData
  • Typemock
  • unit testing
  • Validation
  • Visual Studio
  • VMWare
  • WatiN
  • WCF
  • Web API
  • Web Essentials
  • Web Forms
  • Windows 7
  • Windows 8
  • WinJS

Archive

  • 2013
    • May (1)
    • April (1)
    • March (1)
    • February (3)
    • January (1)
  • 2012
    • December (1)
    • October (6)
    • September (3)
    • March (1)
  • 2011
    • October (1)
    • August (1)
    • June (3)
    • March (2)
    • February (2)
    • January (4)
  • 2010
    • December (2)
    • October (3)
    • September (1)
    • August (2)
    • July (1)
    • May (1)
    • April (2)
    • March (2)
    • February (3)
    • January (2)
  • 2009
    • November (3)
    • October (2)
    • September (5)
    • August (2)
    • July (3)

Blogroll

  • RSS feed for Dan WahlinDan Wahlin
  • RSS feed for Jimmy BogardJimmy Bogard
  • RSS feed for John PapaJohn Papa
  • RSS feed for Josh TwistJosh Twist
  • RSS feed for Los TechiesLos Techies
  • RSS feed for Phil HaackPhil Haack
  • RSS feed for Scott GuthrieScott Guthrie
  • RSS feed for Scott HanselmanScott Hanselman
  • RSS feed for Steve SandersonSteve Sanderson

Twitter

  • Twitter May 15, 7:20 PM

    At Denver .NET meetup to hear @rlacovara talk about SpecFlow

  • Twitter May 15, 9:26 AM

    @eriklane @extofer @greeleygeek That sounds weird to me, too. Values on the query string, sure, but not JSON.

  • Twitter May 15, 9:24 AM

    @kevinkrueger otherwise, people will not be as forthcoming about areas they hope the team can improve

  • Twitter May 15, 9:24 AM

    @kevinkrueger I think retros are best if they are for the team only, so they can have a frank discussion of how to get better.

  • Twitter May 14, 10:49 AM

    Terrific talk from @zekeli @html5denver last night "Cross domain Pong with window.postMessage" http://t.co/iO0AAlbJ7l http://t.co/5mnf2fZL0p

  • Follow me on TwitterFollow me on Twitter

Recognition

  • INETA Community Champions

Blog license

  • Creative Commons License
    Blog by Volare Systems is licensed under a Creative Commons Attribution 3.0 Unported License.
    Based on a work at http://volaresystems.com/blog/.