Frustration grows when unit testing with VS2005

I've always liked unit testing as it makes my job as a software developer much easier. But the unit testing features of Visual Studio 2005 makes me very frustrated and tired of writing unit tests.

There are two big key issues when using unit testing in VS2005.

• When I have executed a test and it fails; it will show up in a list with a red icon. When I double click on the failing row/test it will show a report stating what went wrong. Tell me something I don't know, and move the focus to the failed assertion.
  
• When I have fixed a bug in the project under test, I would like to re-run the test. In the "Test results" window I can choose "Rerun original test". When I click on it, it will re-run the test with the old code; and the test will fail again even if I fixed the bug. What happened, didn't I fix the code??
  
• What the h**l is "Assert.Fail failed."?

Visual Studio 2005:

Eclipse:

Issue 1
Since the test report window has already displayed that the test fail, I'm not interested in a report stating that the test failed, I know already that the test failed and (hopefully) the reason for it. What I am interested in is which assertion failed and where it is.

In VS2005 I have to do the following to get to the assertion:

1. Double click the failure in the "Test results" window.
2. Find the row number by reading the text in the stack trace section. Add row number to memory.
   
3. Open the unit testing code file.
4. Go to the row found in step 2
5. Fix the assertion

In Eclipse it is so much easier:

1. Click on the failure in the "Junit" window.
2. Double click on the "AssertionFailed" in the "Failure trace"
   
3. Fix the assertion

C'mon; 4 steps and many user interactions involving remembering trivial values to show the failed assertion. It is really frustrating to do it whenever something fails. If something is frustrating and takes time, the developer will stop doing it; thus stop unit testing the software. There is no context menu item to go to the failed assertion, only to go to the begining of the test method. With test cases that can be long, it is not so easy to find the failed assertion. Below is the test report that is shown when the user double clicks on the failure.

Issue 2
I have one project that contains the production code, and one test project next to it as I don't want to mix production code with test code. I've created a test project, added a reference to the production project, and verified that the test project depended on the main project. The test project can access the production classes, so far everything is fine.

But the problem comes when one of the tests fails because of a bug in a production class. When I update the code in the production project, and want to debug the last failed test I click on the "Debug original test". Sure the debugger starts, but the problem is that VS2005 forgot to build the whole solution including the production project. Instead of rebuilding the production project it will only rerun the test. The test will of course fail again, and I will be sitting there wondering if my fix didn't do anything. This has happen to many times, and I've wasted numerous minutes before realising that I must manually build the solution.

Perhaps I'm not using the features correctly, and MS has another view on old test results. Maybe test results should be saved every time a test is run. But for me test results has a very limited lifetime, as soon as I've fixed an error; I don't care about the results prior to the fix.

This does not happen in Eclipse, the minute that I've updated the code I can re-run the test by clicking on "Re-run Last Test - Failures first". No rebuild needed, no time spent wondering why a fix doesn't work.

Issue 3 (Minor)
I just saw this when writing this blog. How can an Assert.Fail() fail? The failure text states that the Assert.Fail() itself failed, to me that is not correct. The test failed itself, and should perhaps say "Failed due to an Assert.Fail() call". Unfortunately Eclipse is not much better as it states "AssertionFailedError" for a fail() call, but still that is better than VS2005.

Opinion
The VS2005 unit testing features are not user friendly, they don't help the user as good as they could. Sometimes a tool with a bad feature implementation is worse than a tool without the feature. Perhaps MS can improve the usability in VS2005, and make it easier for us developer that wants to utilise unit testing. It seems that the above design misstakes must be developed by people who doesn't practice unit testing.

Why no Equals() and GetHashCode() in System.Collection?

Today I had another brain freeze when I was trying to understand why two array lists would generate two different hash codes even if they contained the same data. But when I started my investigation in MSDN, I see that the ArrayList does not override the GetHashCode() nor the Equals() method. Ehhhhh?

So my question is how can I check if two lists are equal? NO, I don't want to go through all items. This should be handled by the framework classes and I don't want to do any unnecessary implementation.

If I look in Java, the ArrayList implements equals(), hashCode() as all good objects should. They have also a good explanation on how it is done here.

If I look through the other classes in System.Collection I see that no implements GetHashCode() or Equals(). If none of them do, then it must be a design choice. But what design choice would that be? If anyone have any explanation, please make a comment to this blog. I really want to know.

There is an interface in System.Collections.Generic that is named IEqualityComparer, which has two methods, Equals() and GetHashCode(). Why make an interface when every class implements those methods? And why doesn't the default list/collection classes implement this interface?

I did a little test application to test this out, because I can not believe it. Full listing here.
Pseudo code:

string a = "A";
string b = "B";
ArrayList list1 = new ArrayList();
list1.Add(a);
list1.Add(b);
ArrayList list2 = new ArrayList();
list2.Add(a);
list2.Add(b);

Console.WriteLine("list1.equals(list2) ? " + (list1.Equals(list2)));
Console.WriteLine("list1.equals(list1.clone) ? " + (list1.Equals(list1.Clone())));
Console.WriteLine("list1.hash == list2.hash ? " + (list1.GetHashCode() == list2.GetHashCode()));

This program would show:
list1.Equals(list2) ? False
list1.Equals(list1.Clone()) ? False
list1.hash == list2.hash ? False

If I create two array lists that contains the same items, the Equals() method will return FALSE. If I clone an array list and checks if they are equal the Equals() method returns FALSE. How can a CLONE not be an exact copy of the object, or how can two exact copies not be equal to each other?

The best part is that Hashtable doesn't implement GetHashCode() either. And that class require all other objects to implement that method in order to be used, but they refuse to do it themselves. "Do what I tell you, not what I do."

Is it because they know people will not implement those methods since developers are lazy? But why have they put them into the Object class if they can't be trusted? Or was it a copy-paste mistake from Java? To me this is so wrong.

Localisation with VS2005 will slow you down

In my previous blog about a localisation that went sour, I mentioned that the time for building a localised application takes longer and longer for every language that is added. I have now quantified my feeling and found out that adding a language adds 15-20 seconds for each language. Now that doesn't sound that bad right? Wrong, If you develop an application that takes one minute to build and adding languages almost triples that time, then it becomes tedious very quickly.

This is what I found out (table and graph):

• Default language took 69 seconds to build
• With two languages it took 85 seconds to build (16 extra seconds)
• With four languages it took 118 seconds to build (49 extra seconds)
• With six languages it took 144 seconds to build (75 extra seconds)

If a normal developer compiles 10 times an hour to test something, he/she will waste 12 minutes, just sitting to wait for VS2005 to build. I meanthat is an outrageous extra time, and I wonder if we can send a invoice to Microsoft for those hours that we just wait for the build complete? And I guess we have to make the work day longer as we get much less done.

For this project, the solution was to remove all language resource files from the project file. This means that the only language that is supported in mid-development is "default" and nothing else. So fixing those too narrow labels for localised texts has to wait for another day. The tool do add/remove all those languages to/from the project file is developed in house and can not be shared unfortunately. The tool is very simple to create, but in my view unnecessary.

Here we have a grave problem with the localisation in Visual Studio 2005. If it is done as MS wants, the build time to a project can (and probably will) go up for every language added. What is the outcome of that? Either the developer will stop adding localisations, constrain the development by only showing the default or sit and do nothing while the project builds. This is not a good solution. I would rather have a one-file seperate from the GUI, which reads on the fly when the application is started. That way the language files does not affect the build at all, and it will be a minor load time added for the user which probably will not be noticed.

Measurement notes:

• This is an observation of an application that has been under development for 3-4 years.
  
• They have tried to follow MS design guidelines.
• Clear case is used with dynamic views, which could have an impact on the result as it needs to access 200 new files for every added language.
• The application is not divided up into more than 2 GUI projects. Perhaps adding more sub projects could make the build quicker?
• I used a C# application to measure the time it took to execute a bat file containing "devenv app.sln /build debug"
• I measured each language addition at least 4 times
  

Localisation gone bad in C#

I'm doing maintenance on a huge C# application that was developed for a swedish customer. A while ago it was decided that the application should be sold outside Sweden, and therefore it had to be localised in english, romanian, chinese, etc. The application consists of roughly 200 controls/panels and forms. It was a huge task to translate everything, and from what I now can see MS Visual Studio 2005 did not help very much (as I'm maintaining it right now).

Apparently one new thing in VS, is that you can localise each and every control/form in the designer. That is nice when you have a few forms and controls that do not use the same words all the time. The developer creates a form, changes the language property and inserts all the translated texts. It is very easy to change a word in swedish to english, and it is all GUI based. But what happens if you have 200 controls/forms that share many texts? I have not figured out how to do that in VS2005 and GUI elements.

Developers are lazy (and they should be), so they will use the easiest way out. In this case it meant that nobody cared that the same words were used in many forms. Sometimes, people missed a label here and there. And for me maintaining it, it isn't nice.

I demand these things out of a localisation framework

• One file for each language. - VS2005 brings you one file for each language and GUI control. In the project it means 1,000 new XML files to handle. That is not good.
• One method call to get the texts for a language. - Yes, .NET helps partly with this (I will blog another rant about this later)
• Ability to change locale during runtime. - Yes, it is possible in .NET.

I need the following tools:

• Tool to find duplicates so they can be erased, and defined in one place. (Think OO but with languages) - VS2005 does not help you at all.
  
• Tool to spell check localisation files. Apparently there is no such tool out there. - I have not seen anything in VS2005.
• Tool to show missing translations for a language. - Nope, the company had to build their own software to extract missing translations and merging.

So what I got was to "translate" around 200 "missing" translation in an XML file, out of them there were 50 "OK", 5 "Cancel", 10 "Index", 15 "Close", etc. I can sure tell you that it could have been very tedious, but I built a very simple tool that would go through the file, so I didn't have to bother with copying/pasting. But once again, I have to develop tools to do stuff that the IDE should have helped me with.

Side note:
They discovered a side effect of adding new languages to the project files, for every new language you had to add 200 files, this made the project bigger and bigger. Now it takes eons to compile, and one of the core developers came up with the idea to remove all localisation files from the project files. When that was done, the time to compile was drastically cut which was good. Unfortunately we we're in integration testing and could not test any of the localisation bug fixes (D'oh).

So what did they end up with when using Visual Studio and .NET?

• Duplicated texts all over the resource files.
• 200 new files for each language
• Compilation time sky rocketed (I will bring stats for this later this week)
• Developed their own tools to handle the resource files (RESX) which require maintenance, etc.
• Forms that were not properly translated, which wasn't found out until the system test phase (or even later).
• And lets not forget the need to update the size of every button/label to cater for romanian texts. That is also tedious and really unnecessary work.
  

What is the proper way to do localisation in .NET and Visual Studio? What did they do wrong? What is the easier way out?

TabControl, ListView and checking list items

Today I found an odd feature with the ListView GUI control. The list view must be the king of Windows controls. I mean, what can't it do? It can display lists in 5 different ways, which is increasing as Windows evolves. It can even display a list with check boxes for each item, very similar to the CheckedListBox. But the ListView behaves very differently when it comes to checking and unchecking list items if it is placed in a TabControl.

The event ItemCheck should be called when the user or code checks an item in the list. But the oddity I found was that the ItemCheck event method would NOT be called when code/user checks a list item, but the event is first called when the user clicked on the tab that contained the control.

This is what I did (Full source code available at pastebin.com):

• Created a Form with a ListBox, TabControl and a ListView
• The TabControl contains 3 tab pages, the initial tab with a button, the second tab with a second ListView and the third tab page contained a CheckedListBox.
• Added event listeners for all three lists that would add text into the ListBox to show what happens and when.
• When pressing the button, it will check the first item in all lists (the two ListViews and the CheckedListBox).
  

This is what I found:

• When the button was clicked, it would call the ItemCheck method on the CheckedListBox and the ListView outside the tab control. It would NOT call the ItemCheck method on the ListView inside tab page 2. (Form screen shot)
• When I clicked on tab page 2, then it would call the ItemCheck method on the ListView inside that tab page. (Form screen shot). Somehow the ListView remembered that I wanted some of the items checked.
  

Now we have two differently behaviours depending if a ListView is inside a tab control or not.

1. If a ListView is outside a tab control it will call the ItemCheck method when the property is changed. But if the ListView is inside a tab control (that isn't visible) it will not call the ItemCheck method when the property is changed.
   
2. A CheckedListBox inside a tab control (not visible) will call the ItemCheck method when the property is changed; and the ListView will not call the ItemCheck method.
   

Workaround:
In our case I solved it by setting a boolean to false in the TabControl_SelectedIndexChanged() event, and then setting it to true in the TabControl_Selected() event. This way I can check from the boolean if the ItemCheck() comes from the user or is cached by the ListView. If the boolean is true, then I know the user checked a list item.



C'mon, which is the correct behaviour?
Why is there a need for a different behaviour?
Why do I have to bother?

Perhaps I didn't know that it is common GUI-OO design to have one implementation if a control is in a TabControl and one if it isn't.