IDE defaults

Defaults from several IDEs or more simple text-editors are already giving big help, e.g. in showing keywords, instance fields or comments. Still in my view they can be tweaked, most of editors give options to extend things. Either they work with a graphical interface for changing settings (IDEs like IntelliJ, Eclipse etc.) or are working themselves with plain text highlighting configuration files (vim, krusader etc.). Your syntax highlighting toolbox contains text-decorations (like italic, underscored, bold) and coloring (foreground, background).
For my tweaks I used my favorite IDE IntelliJ, which offers many syntax highlighting options. Just checkout your editor and see what is possible.

Example BEFORE

Following annoyed me on the default settings:

• Could not instantly see parameters and variables
• No difference between local-vars and parameters
• Non-javadoc comments were too grey. I write comments to explain the ‘Why’ or a important block of a code statement. So comments should be better visible.
• Todo comments were blue. Blue is a too “friendly” color to me, whereas looking at todos should wake me up!
• Instance and static vars were colored the same though they have different semantics.
• I tend to use more smaller methods as one monster method. The default highlighting does not separate between method declarations and calls.

The BEFORE snippet:

Example AFTER

I changed settings to:

• Non-instance + static variables are blue now. Parameters should be handled with more care. (changing them side-effect the callee), so they are bold.
• Static and instance vars have different colors now (pink vs. violet).
• Comments have slight green background now.
• Todo flags have the signal-color orange
• Methods are underscored. Declarations are bold, calls are non-bold.

The AFTER snippet:

Vimperator

Vimperator is a real gem! Adds some vim (editor) feeling to the browser. Makes you faster, because nearly all mouse action can be supplemented with keyboard shortcuts. Also automates more complicated flows with macros. At start using vimperator can be somewhat annoying because pressing some keys do unexpected things, but investing time to get used to it pays off definetely.

Xmarks

Xmarks saves your bookmarks to a server and makes synchronization possible between different machines. Very handy if you are working from different computers. Most likely it could be replaced by upcoming firefox 4 which offers this functionality in its core.

Web Developer

Web Developer is a nice webdeveloper testing kit. Numerous things can be done like style/CSS testing, gathering meta-information of the page, handling cookies, finding broken images.

Firebug

Firebug is a perfect accompany to Webdeveloper for testing/analyzing websites. Offers JavaScript debugging, analyzing DOM tree, viewing CSS styleor watching HTTP calls and request/response contents. It is also plugin aware (see below).

Firecookie

Firecookie is an addon for firebug. Makes cookies handling (reading, deleting, editing) much easier as with Webdeveloper plugin.

YSlow

YSlow is an addon for firebug, which offers performance test for webapplications. Gives a good overview how your site performs and gives a summary in grade style (A-F). If it gives you bad grade, still question whether they are appropriate in your special case (e.g. YSlow moans about missing CDNs, but an usage of a CDN doesn’t always makes sense or you don’t have any control over certain included components).

Live HTTP Headers

Firebug offers good HTTP traffic tracking. But sometimes I also use Live HTTP Headers because you can filter tracking HTTP calls by URL and content-type, for HTTP POST you can set your own defined payload.

JSONView

When testing webapps, instead of using curl sometimes it is handy to fire a HTTP request directly through firefox. If doing so by default firefox makes problems and prompts to save json (Content-type: application/json) as a file to instead of just displaying the content inside the browser window. JSONView bypasses this and displays json content appropriately.

This code has essential problems:

1. Because I want to avoid to write two more test-methods (essentialy the same production code is triggered), I am putting three test-cases into one test-method, which is bad practice. This is because test cases aren’t using the tearDown and setUp facilities and cannot guarantee isolation. That is also the reason that I could not use the excpectedException parameter inside the JUnit 4 @Test annotation. Alternative is to really use three test-methods, but code readability then suffers.
2. Even though above example is very simplified (only new Foo() is called) the test-code is not expressive. Surely you could improve this by extracting method and giving a good name. But still it is a bit blurry, why we do so and that it is just for using different test-data.

TestNG parameterized test-methods

TestNG does it better and builds the “test-case differs only in test-data” situation into its framework. This is done by building a DataProvider and passing parameters to the test-method. This way code gets more expressive and each different test-data set is executed as an isolated test-case also. Here an example of the TestNG version of above test-cases (I followed the code centric way, you can also configure your DataProvider through an XML file):

No atomic commits

Once you checked in files it is very hard to revert to a certain state, because atomic commits aren’t supported. When checking in multiple files, each file gets a new revision (similar to CVS) and not the check-in itself. I think this is a crucial feature, because you hardly want revert single files but complete commit actions (which should map tasks). With ClearCase you can only revert to certain states by using Labels. In practice using ClearCase Labels for each check-in is overkill and thus not done.

Crappy user interface

The GUI of ClearCase Explorer is just a big joke. Horrible in usability and ugly looking. Different and often necessary functions aren’t provided (e.g. recursively checking in worked on artifacts). Command line tool cleartool used with cygwin is much better, but still some things aren’t available like recursively adding new files/folders to source control. I have to laugh my head off if I read a 50 lines of code long script to workaround this.

High administration efforts

Administrating ClearCase beast is far from obvious or lightweight (in difference to other scm-systems like CVS, subversion or Git). Expect to put quite a few dedicated ClearCase experts to just keep it running.

Horrible performance

Nothing is worse as making your developers wait while interfacing with SCM-tool, it is like driving with hand brakes enabled. It slows down your brain and also your work. Getting fresh new files to your snapshot view takes around 30 minutes for 10K artifacts. An update (no artifacts were changed) for the same amount takes roughly 5 minutes. When experimenting a lot and jumping between different up-to-date views means a lot of waiting. It gets even worse, when you’re working on files and you want to check-in or update them. Check-out, check-in and adding to source control cycles take around 10-15 seconds which is obviously a nightmare. It gets very annoying when you’re refactoring renaming/moving types or methods (many files can be affected).

Lack of support of distributed development

Today software development is often a distributed thing (developers are spread around the world working on the same product/project). ClearCase definetely isn’t suitable for this, because it is badly suited for offline work. Doing a check-out (action before you can edit a file/folder) requires that you are network connected. Here you could use the hijack option but this is rather a workaround as a feature (you basically just unlock the file on the filesystem). If your development sites are far away from your ClearCase server the check-in/check-out latency can even increase so dramatically that it is not usable at all. There are workarounds for that like using ClearCase Multisite (scm DB replica technology), but you have to pay extra for it and is not trivial to adminstrate.

Git as alternative

Though being a big fan+supporter of Open Source I am still willing to pay money for good software. But looking at IBM-monster ClearCase I wouldn’t invest my money here, it has all these discussed shortcomings, and further more IBM doesn’t seem to invest money to improve their product significantly. Recently I had a look a Git scm which looks very good, especially for its branching+merging features, where ClearCase has its major strengths.

Code quality issue

Code gets written and as it changes (in most cases) it rottens also. In many projects/products changing and adding functionality means having a larger codebase. And the bigger it gets the more difficult it becomes to improve the overall code quality picture:

• Motivation of developers decreases to keep code clean. In the end code gets worse and worse, just have a look
  at the ‘Broken windows’ chapter of ‘The Pragmatic Programmer’ for details
• Efforts to clean up increases, more code needs to be maintained. Even if we are highly motivated to write ‘good’ code, it is often difficult to know where to start.

Solution: Improve what you change

There are two ways to tackle the problem: Big Bang or continous code improval. With BigBang you dedicate armies of developers who touch code just for the sake of making the code ‘better’. This approach has its drawbacks: You still don’t know where to start, it is difficult to sell to the product owners (feature stop) and further more it gets really boring (ever tried to tidy up your flat for a whole week without doing anything else?). An approach which worked better for team members and me was to introduce a kind of continous code improvement. It is important not to say that we get from 10000 warnings to 0, but to make a commitment that the warnings never ever get higher and must decrease after constantly. Here you improve your code quality in parallel to feature development and you only improve what you change. This much better sells to the product owner, you even don’t need to mention it, because it is part of your daily effort and should be transparent for non-techies.
Furhter more incremental work-items can be managed better and focusing on certain artifacts also reduces overall efforts. In my view the best “quality-gate” for such an improve-what-you-change workflow is the source control commit or checkin phase. You changed code and now you want to ensure that checked-in sources provide a certain code-quality level.

Source control dialog IntelliJ:

IntelliJ approach to continous code quality improvement

The real problem is the tooling support: Humans are very bad at automating repeatable tasks. For instance when I am using Eclipse IDE and commiting sources to for instance Subversion, I often forget to reformat source code, remove unused imports or am just overlooking bad code snippets. IntelliJ (tested on version 8.1) goes a much better way: It provides actions (auto-formatting, auto-importing) while commiting and besides that can run static code analysis. For this code analysis it includes the so called ‘Inspections’, which are numerous and sensible (). For some cases it even provides an automated correction of the code flaw and the corresponding code snippet (pressing Alt+Enter). All these actions and checks are performed for the items you are about to commit only. This way IntelliJ mandates the successful approach to improve what you change or touch.

Dialog to review inspections and inspection results after pressing ‘Review button’ :

Anticipation of incoming dependencies not possible

Officially Sun mentions that checked exceptions should be used in case of “expected” exceptions, where clients still could react. In contrast unchecked exception should be used if programming erros occur, where a reaction is hardly possible (e.g. NullPointerException, AssertionError).

In case of checked exception client code can decide whether to react to it directly (try/catch) or to delegate and pass it in the throws clause. In practice many checked exceptions aren’t interesting to client code, therefore you start to pass it to the throws clause. After a while spreading exceptions between different class-collaborations they are getting piled up and code-distraction is king and developers are constantly typing irrelevant code.

I’ve often seen code where developers got fed up with this pass-through of exceptions, workarounded this with a bad exception handling coding style. So, in theory checked exceptions should increase code quality but in practice I perceived the opposite. This is bad, because a programming language design should help you to write good code and not constrain your work. Following workaround code snippets are quite common:

Above is evil: Though exception should somehow be reported in lower area of method call stack, this try/catch just makes any error handling impossible. Such “nirvana” exceptions can make your life really hard when locating defects.

Problem is that above newly thrown RuntimeException is too general and analyzation of certain error-conditions is hard. Often the causing checked exception isn’t even passed to the RuntimeException constructor so finding the root cause of a failure is also difficult.

To summarize: If client code cannot handle the exception, in practice checked exceptions aren’t passed to throws clause of method declaration. Indeed you can get really crazy if you need to pass the exception the whole method call-stack. Especially when doing bigger refactorings such millions of checked exceptions in the throws declarations are extremely annoying. Looking at this shortcoming developers often go for an improper exception handling. The difficulty of api-designers is that they cannot anticipate the exception-interpretation of incoming dependencies and thus the use of a checked exception can lead to bad exception handling code. They just don’t know, if a handling of an exception makes sense or not, see following examples:

Example checked IOException

Trigger: IOException from java.io api gets thrown because a file under given pathname cannot be found.
Handling appropriate: My client-code represents a file system manager, where I am searching for a file. The information “File not found” is directly relevant and file system manager could proceed to look in other directories.
Handling not appropriate: I got a general app, where a property file should be loaded. Then the app should crash early/instantly, most likely I set a wrong path (Programming error).

Example checked RemoteException

Trigger: RemoteException gets thrown, because network connection for RMI communication is broken.
Handling appropriate: My client-code is a network monitoring tool and such information should be logged down for server downtime analyzation. Thus exception should be directly handled.
Handling not appropriate: I got two application components which are communicating with RMI. The defect is a misconfiguration of the firewall. Application itself cannot react on this, it is a plain network setup defect.

Now Sun’s exception convention gets into problems: They told that Programming-Errors should always be mapped to unchecked exceptions… As shown above this does not work out, in some contexts the root cause of IOException is a Programming error (property-file loading) in some not (file-manager).

Way out of checked exceptions

Most likely Java standard won’t change this exception design-flaw, especially for backward compatibility reasons. But there is still a solution: When you are forced to handle checked exceptions constantly you can wrap them in custom unchecked exceptions (-> extending an own custom exception hierachy from RuntimeException). This way other client code does not need to bother to look at non-interesting exception. The implication of mentioned points could also be that your api-design should only offer unchecked exceptions in method declarations.

Following this clients are not distracted by always doing this try/catch duet and code further more gets much better readable. Of course you should not just shove all checked exceptions directly into a RuntimeException (one of the workaround I explained above), you should be more concrete and build up a deeper exception type-hierachy to make further handling easier. Besides you should chain checked exception and pass it to the unchecked exception’s constructor to later find out root causes better.

Of course at one point such unchecked exceptions need to be handled at the outmost layer latest to make a proper report possible (e.g notifying monitoring/logging, sending back a user-friendly error HTML-message). You really don’t want to spill the whole stack-trace to a HTML or SOAP response or make your application to completely always shutdown.

Conclusion

Java introduced checked exceptions, which seemed to be a good idea in the first place to enforce clients to “have to know” about certain error conditions. But in practice developers do not pass “uninteresting” exceptions in throws clause, because code gets crowded and diffult to understand. They merely tend to workaround with inappropraite exception handling. Further more the view on exceptions, wheter they are worth the be catched or not, is context based and cannot be anticipated. To avoid this problem api-designers or client code generally should go for unchecked exception and a custom unchecked exception type-hierachy.

Annoyance of test-errors

Test-errors are very annoying because your tests should be the impartial authority saying your production works or not. And if you cannot rely on them and they are constantly lying to you, you quickly get the opinion that tests don’t help you but merely slow down your work tasks. In fact I noticed that many developers new to unit-tests are gettings frustrated by test-errors and thus are stopping to write them. This way they lose the advantages of a good test-suite and Test Driven Development which is a shame. Test-errors can roughly occur in two forms: false negatives and false positives.

False positive test

A false positive test gives you a failure, though the feature is behaving alright. You change or create some code, make everything compile but the corresponding test gives you a failure. After debugging for a while you see that the test made wrong expectations about the feature so that your test is inconsistent with the specified behaviour.

Very simplified example:

I experience false positive tests sometimes when using mock-frameworks (like EasyMock ), because when injecting mocks to class under test you expose some bits of implementation details to the test by recording behaviour to the mock. After changing implementation of class under test slightly, the calling of your mocks could change also. Without changing your mocks correspondly your test most likely will fail.

Another often occurring cause for a false positive is a wrong test setup, e.g. you could pass wrong or no instances at all to class under test, so it behaves different as expected from test case view. Example: You pass an implementation to class under test, which connects to a file-database and wants to read data. Database is not available and test fails. Here you made a test setup mistake: Actually you had to pass a persistence stub, which returns appropriate values for the test.

Further more your design could be too tightly coupled, where your class under test has many dependencies to other classes. Thus many other instances are called indirectly and it is difficult to isolate you “real” test case. A change of dependent classes makes the test fail, though class under test itself did not change and is still behaving fine.

False negative test

A false negative test reports you a success message that everything is alright but in fact it is broken. In this case all your test expectations (=asserts) are fine, but production code in fact has got a bug.

Same simple example:

What I see often in false negative tests, is that assert statements are formulated too weak and claim not enough from class under test. In many test cases you find a very non-saying assertNotNull(returnObject) and the properties of returnObject are not checked more detailed.

Preference of test-error

The lesser evil test-error are definitely the false positives. Here I get instantly notified that there IS a test-error. Looking at a false negative test I don’t get notified at all and the overall ‘good’ feeling that I got tests in place is very deceptive. You interpret more quality to your production code than there is.

What to do about test-errors

Awareness of both test-error categories is a good first step, but how can you avoid test-errors generally?

Suggestions to avoid false positives

An obvious cause for a false positive can be your test assert statements, maybe they are just plain wrong, requirements have been changed and class under test has been adjusted respectively. To avoid such assert statement mistakes you should develop test driven: When a requirement has been changed first look if feature has a corresponding test case and adjust it. After that change your production code. This way you avoid specification inconsistencies inside your test cases.

When looking at your production code, maybe your design is too tightly coupled and your tests are covering just to many classes? With this you should consider to introduce dependency-injection to make passings of alternative test-implementations possible. Or maybe you got a monster method and thus “feature-isolation” is not possible either? Here you should consider to use extract refactorings (method, interface, class), to achieve isolation and loose coupling and make stub/mock injections possible.

Suggestions to avoid false negatives

Most likely your assert statements are too weak and you don’t use enough test-data. Consider to add asserts an invoke class under test with alternative input values. The assertNotNull(returnObject) assert is often a good start to check whether the returnObject has been initialized at all, but if so, more “stronger” asserts should follow. Always remember: Apart from your application throwing an unexpected Exception inside test-run your test case will only fail if one of your asserts will evaluate to false!

Download/Installation IntelliJ

Though IntelliJ costs a bit you get a free evaluation license for a month. Download and installation is straightforward and easy. First noticed difference is the general workspace layout. In IntelliJ the eclipse-workspace is a project and a eclipse-project is a module. Further more IntelliJ configures things globally in your home directory and most settings are read from there no matter where you got your IntelliJ working directory with all your projects/modules. This is quite different to Eclipse where all workspaces are configured inside themselves in .metadata/ folder, and opening this workspace is completely uncoupled from any user-settings outside the workspace. It’s different, but is difficult to say which strategy to prefer, both got pros and cons, so I want to dig deeper what advantages exist inside IDE.

IntelliJ taking ahead

Following things I would emphasize when it comes to advantages of IntelliJ:

Clear IDE Layout

Generally the IntelliJ IDE layout and the look-and-feel is more tidied up and clear. You don’t have the millions of perspectives and views as in Eclipse. There is only one perspective with the main editor at center. On left, right and bottom you can find the so called ‘Tool-Windows’ (can be compared to Eclipse views). Also you can focus Tool-Windows better and discard them again: Alt+F1 jumps to respective window, pressing Alt+F1 closes it again. This way you tend to only have the Tool-Windows open, which you currently use. In eclipse I often end up with many opened up views, which I am not interested in. Further more you can change opened files in tab-bar very quickly by hitting Strg-Right/Left. When being in project view it is possible that a file gets opened in editor as soon as you focuse a file (in Eclipse this is only possible if file is already opened in a tab). To summarize I like the general IDE navigation much better in comparison to Eclipse.

Change History

This one is a real delight! In Eclipse I always found the Local-History, where you can track changes very clumsy and difficult to follow. In IntelliJ this is a big feature, you can follow your changes very well and can label them locally, so reverting is just a wink of an eye. This is very handy if you got a bigger refactoring or feature task, which includes certain steps. In Eclipse I tend to check-in many little changes to version control in a short (minutes-scale) time though I would like to commit only complete tasks (which other team members are interested in only). This is possible with IntelliJ: with the great change history feature I am still commiting in short iterations but at the same time only complete things. Last but not least the diff-view is much better as in Eclipse.

Dependency Matrix/General analyzations

IntelliJ 7.x version introduced the dependency matrix, where you can analyze dependencies between packages and thus can identify coupling problems of subsystems. Generally the analyzations of your project is better as in Eclipse. With Alt+F7 you can quickly see all incoming dependencies. The analyzation-syntax-highlighting is very helpful to concentrate on the essential code bits, too. For instance when pressing Strg+Shift+F7 over a variable, read (blue-color) and write (pink color) access are highligthed. This can come very hand when refactoring large methods. Eclipse got this grey highlighter when you got your caret over a text-bit, but it vanishes as soon as your cursor moved somewhere else and you can highlight only one thing at one time.

Intentions

Intentions are a way to save you work from doing things manually though a certain context should offer you this task done automatically. Example: When placing the caret over an interface just press Alt+Enter and you are offered to create an implementing class with method-skeletons. When just typing a method call which does not exist, you get offered to create a method-body with respective signature. Since Eclipse 3.3.2 this feature is partly also introduced but not in such a good way as with IntelliJ.

Auto-Save of changes

In Eclipse I am constantly pressing Strg+S to save my changes. This can be very annoying, because I do a save very often straight after editing before executing a test case or starting the application. IntelliJ takes off this burden and does an auto-save behind the scenes. I really wonder why Eclipse isn’t doing the same, maybe it could be a performance issue, because recompile is done after each save. IntelliJ seems to handle this auto-compile very good, if errors occur you get to know the compile problem instantly.

Plugin installation

Though standard IntelliJ comes with enough functionality, for some special things you still need to get some plugins (e.g. Jetbrains groovy/grails plugin). Never the less I found the search and installation much more straightforward as with Eclipse where you really can get headaches especially if there are some transitive plugin dependencies. Further more you are informed if an update got released.

Little things, big impact…

Generally little things make a big difference. In IntelliJ there are so many that I cannot list all of them, but here a some which quickly come into my mind:

• When searching for a type (Ctrl+N) you can add a shortcut name of a type: Entering VQTB would quickly pop up (VeryQuickTypeBrowsing), this saves a lot of typing. You can further more quickly decide whether to include only own project types or dependency projects/third-party libs, too. This way you don’t get overhelmed by the millions of classpath-artifacts.
• Unit-testing: If you are inside a JUnit test-class, pressing Strg+Shift+F10 outside a test method (or F9 for debug mode) executes all test-cases, if you are inside a test-method with your caret and press same shortcut only this single test-case is run. This is very handy because I often only work with one test-case if I debug or add a feature. For regression then I like to run the whole test-class or the whole package for subsystem-tests, so I step outside the test-method for class-run or go to package in project explorer to run a package-test. Shift+F10 opens a Run-dialog with last executions, so you can choose between last run test sets very quickly.
• Actions on commit: In Subversion commit-dialog you can choose ‘Organize imports’, ‘Format code’ or ‘Run inspections’ so you can do some actions on the code you would normally forget often (at least I do so). This way you get a better state in version control, where all imports and the code formatting is alright. Annoying diffs where you only see changes in different formatted code or unused imports should be vanished then.
• Tip of the day: If actions need some time (e.g. building project) you see a little dialog, where some nice shortcuts or general IDE tips are presented. This way without explicitly reading the online help, you learn a lot while just working normally with the IDE day by day.

Eclipse taking ahead

Of course there are some advantages of Eclipse, when compared to IntelliJ:

OSGi support

Eclipse migrated their former proprietary plugin architecture to standard OSGi platform to master the complexity of all these plugin dependencies and general plugin startup issues, which works great. Through the direct support of OSGi writing applications for the OSGi platform goes very well. OSGi Plugin for IntelliJ exists, but is far not as good as Eclipse with its equinox runtime and PDE environment.

Mylyn

I think Mylyn is a big plus for Eclipse, because it takes team collaboration with tracking-tools as a first child. With Mylyn you integrate common ticket-tools and can connect tickets with certain artifacts (packages, classes, other files etc.). There are many of mature Mylyn connectors (Bugzilla, trac, JIRA), which work very well. For IntelliJ you can choose from some ticket-tool integration plugins, but I did not perceive them as good as the Mylyn solution.

Eclipse Modeling Project

The Eclipse Modeling Project for doing MDSD looks very promising (though I must admit I only read articles about it, but haven’t tried it out yet). After searching on the web I could not find an equally mature MDSD platform or plugin (like the one from oaw) for IntelliJ.

IDE synchronization

IntelliJ offers a simple eclipse project files export (.project, .classpath meta-data), so checked in Intellij modules can be used by Eclipse. Still I must admit that I did not elaborate enough, whether the IDE bridge/export between Eclipse and IntelliJ could generally work in a bigger team. Especially the synchronization of plugin metadata (like maven2, AspectJ) should not work without problems.

Summary

There is no size-fits-all IDE. On top of it is often difficult to say which IDE is better, because the quality and functionality also depends on the released plugins (e.g. maven2 eclipse plugin is better with Eclipse, groovy/grails plugin is better with IntelliJ). The automatic and safe refactorings, which I heard was always a big plus of IntelliJ are in my perception equally mature to the ones of Eclipse. In my view IntelliJ has other very nice points. Apart from the ones I mentioned above, there are many other little things which added together make the real IDE difference, where you are always close to the keyboard got control over so many things. In my view in many circumstances (when not developing for OSGi platform, RCP-apps or because of other plugin-show stoppers) IntelliJ is definetely worth the money and can increase productivity if your are willing to invest some time to switch. The pricing is moderate and as an Open Source developer it is for free.

Tests give you invaluable feedback

In following situations the feedback of tests is very helpful or even essential:

• Existing tests tell you that you didn’t break existing behaviour after doing adaptions. These regression tests show you a green bar if a code change did not affect working features.
• Tests help you to introduce new features. Writings new tests is done in parallel when touching production code. This way you can concentrate on this single feature and you further more get a quick response, whether the feature is implemented correctly (green bar) or not (red bar).
• Tests help you to reproduce a bug. You write a new test case which shows you a the bug as a red bar. With this you have a very good starting point to debug and fix it. When fixed you get a positive response (green bar shows up).

The dilemma of weak test-suites

Of course one single test-case won’t do the catch. Your aim should be have many test-cases which summed up represent your whole strong test suite. Unfortunately test suites are often weak, especially when changing code they don’t give you a red bar if something has been broken. The reason why a test suite is weak can be manifold:

No practice of testing, too few tests

The view of tests as central “development-drivers” is not practiced. Things are implemented and after that roughly tested in a manual way. There is no coding of automatic tests which prove that bug-fixes are done correctly or features work like specified. Testcode is seen as not-so-important stuff, because it does not show up in the production code and this way does not give real additional value to the customer.

Tests aren’t executed regularly

Tests which are part of the test suite exist, but their execution is not part of the build. Thus they are hardly executed and are triggered manually. Often this is the case because the tests need human intervention and cannot be run automatically by a build server or similar. It also can be that in fact you have a completely automatic test suite but is just not common that it is executed often. If tests aren’t executed regularly (e.g. once a day, after each commit), you lose the advantage of quick feedback. With this running the tests after a week/month or so, you don’t know what code change caused the test failures or errors. After a while developers will get reluctant about tests, they will be getting used to red bars and simply put will just ignore them at some point. In the end tests won’t be maintained and your test code gets more a burden than a help. Finally your test-code decays to dead code.

Missing or very expensive tools

For test automation the proper tools are very important. Nowadays you get a lot of good open-source test tools (Fit, xUnit-frameworks). Never the less for some special areas it is not trivial how to automate some testing steps, especially in integration testing (e.g. setting up and validating smart-cards contents, or testing message-oriented/asynchronous systems in general). Some vendors sometimes provide some more advanced testing tools but these are either very expensive or/and cannot be extended for special problems.

The dilemma solution

Above mentioned constraints to get a good test-suite are often hard but can be tackled. First of all everyone in the team needs to understand that tests aren’t given to the customer but are inherent important to the development process. Well, you don’t include you favorite IDE to the customer, never the less you couldn’t create software without it, could you? Testing continously needs to be a new habit, so give the team some more time to dig into this, like planning trainings for JUnit or test automation in general. If the team starts to value tests there still could be the problem that there are hardly any automated tests, which can be very demotivating (’The where to start…?’ issue). Never the less you can start up by writing tests for all new features you want to implement or for areas you want to refactor. When knowing a bug always reproduce a bug with a test case initially. After doing this for a while you will see how your backing safety-net will grow bigger and bigger. Of course the test cases of the test division won’t be replaced by newly introduced test cases of the developers. There still are many tests which are difficult or cannot be automated at all (like user acceptance or GUI tests). So see developer-testing as supplements and not replacements for the “traditional” tests. Of course the testers could also help to improve the automated integration tests, because they are more black box focused. In a very mature test focused process a tester for instance would attach a reproducing integration test to a bug report.

As mentioned not executed tests some day will end up as dead code. To avoid this introduce certain triggers (e.g. nightly, after each commit) to run your test suite. See the test runnings as part of the software build itself. This view is very powerful and is also recommended by the Continous Integration concept. If run very regularly developers will soon start to appreciate the tests and will have “fun” with extending test-suite. Looking at the visible green bar or getting rid of a red one can be very motivating.

In former times test tools often had been something more for the people in the test division. The picture has changed a lot. Automated tests created directly from the developers is state of the art and is supported by many useful and open-source tools (Fitness, xUnit, db-unit, Mock-Frameworks, soap-ui etc.). They are extensible and can be combined to get an even more powerful testing tool. Further more Continous Integration servers for running and triggering tests are mature, free and can be extended (CruiseControl, Continuum, Hudson etc.).

Summary

Generally the view of tests from the developers side has changed a lot. Especially since the rise of Unit-testing, some new integration test-tools and the concept of Continous Integration, test-creation by developers is much more common. Even if your organization has not introduced this practice you can do so by building up your test-suite incrementally and executing it regularly. The harder part will be to get the developers to a testing routine/culture but this effort will far be outweighed by the benefits.

Emphasis on outside visible behaviour

Criticism:
Software enginners are paid to create or adapt a system which offers some direct business value, in this means you are claimed to fix a critical bug as quick as possible (hotfix), to add an eye opening feature or to eliminate a performance bottleneck. All these adaptions obviously focus on outside visible behaviour, and they are the real reason (well, lets ignore the political decisions for now…) why customers use software and pay for it. So the customer is right when not spending money directly on software maintenance.
Disproof:
Simply put: The customer is completely right! It should not be his business to know the insights of how we develop software. In the end the system should work as wanted and he must feel that the system is making his life easier. Never the less that does not mean that we should ignore maintainability. With keeping our design and code clean with refactorings we can react on customers needs much more quickly. If he wants a new feature or a quick bug-fix we are in the very good position to offer a new release after a short time, because the code and design is clean and thus code changes do not need ages. Here we got two advantages: The customer will be more pleased to get a new release soon and we software engineers get more resources to improve the functionality of the system.

The perfect all-solving upfront designed software

Criticism:
I do not need to refactor because my software is designed perfectly in the first place. It regards all extensions from the beginning on and thus can react quickly to changing business needs.
Disproof:
Well, I doubt that… Above assertion claims that you can look into the future where you know all market changes and upcoming requirements. This is quite unrealistic except you believe in crystal balls… The other way to make your software ready for the future is that you try to make your design ready for all possible changes. This won’t work because you are speculating and you are paying the speculation fee with an overly complex system. This harms maintainability because the system contains loads of code which will be never used/run (dead code) and the design is much more difficult to understand. Further more you are wasting resources to make your system pseudo-future ready.

Technology constraints

Criticism:
Refactoring is only doable with statically typed languages like Java or C#. The refactoring tools of respective IDEs (IntelliJ, Eclipse, VisualStudio etc.) are making refactorings possible.
Disproof:
Refactoring is a concept and not based on technologies! It is a way to keep software maintainable. Of course it is correct that especially static typed languages (like Java without use of reflection api) make refactorings easier as dynamic language constructs (Java reflection api, Python, PHP etc.). The meta-data inside your code makes refactorings safer and tool support can be more powerful. Never the less you can refactor with any technology as long as you know your tools well and can overlook the side-effects your refactorings steps can cause.

Blurry view of “good” design

Criticism:
In contrast to real world architecture where you are constrained by physic laws (otherwise your building will collapse somehow) the software-solution set for solving problems is much broader. There are innumerable ways to implement requirements and the judging whether it was done well or not is often quite subjective. Though you can find some objective metrics to measure software structure (lines of code per class/method) they still cannot express all aspects of maintainable software. How the heck could a computed metric tell you whether your namings are good, or if the relation between classes/packages well expresses the domain or requirements? So design generally is too subjective to get all developers under the hood.
Disproof:
Yes, software design and code quality in many ways is quite subjective, never the less there are common agreements what good design is and what principles should be followed: inversion of control, design for testability, open/close principle, encapsulation, maximize cohesion etc.. Some code quality bits can be subjectively measured and be integrated as quality metrics, like a method must not exceed a certain number of method lines of code (MLOC) or cyclomatic complexity should be under 10. But you’re right that metrics themselves will never express the complexity of software as a whole. That’s why design and manual code reviews should be carried out to spread design/code knowledge and to find out areas which should be improved. To summarize: You more or less can “measure” good design/code quality by both using subjective (metrics) and objective (manual review) practices.

System is too big to refactor

Criticism:
We realize that refactoring is a good practice but our system is just to big and we would need to spend just too much effort to make it noticeably better.
Disproof:
It is true that you can feel overhelmed by a big muddle of mud system but improving systems can be done in an incremental way. You could start with little code bits, for instance by refactoring code areas you are touching often. Think it like a desert: There is a lot to do but you can create some little oases which after a while getting bigger and one day will connect each other to form big green forests.

Refactorings are causing performance bottlenecks

Criticism:
When going towards refactoring you often end up with many little classes and methods with mostly little blocks (<15 MLOC). This is having an impact on performance because (to use Java as example) the JVM has to load many classes. The little methods are also causing performance problems because of the delegating calls through stack pop/push operations behind the scenes.
Disproof:
Performance is a very important non-functional criteria, but it is always tricky to speculate what can cause the bottlenecks and to generally write less maintainable code (monster classes, monster methods) for the performance sake. With powerful hardware nowadays in most cases performance problems have other reasons (like database or network calls). This is different compared to former times, where main memory was rare and performance could be hit on language level, too. Today we got better development environment and should focus on developing maintainable code in the first place. If we experience performance issues this should be surrounded by a profiler-test to be able to draw the right conclusions.

Untested legacy Code

Criticism:
In practice you hardly will find green-field projects, where you can follow refactoring practices from the beginning. Generally you start up with a bunch of existing code which you need to adapt. Often test coverage of the system (which is a symptom of legacy code) is not high enough to do manual refactoring steps. There is always the danger that you are unknowingly breaking existing behaviour. Especially refactoring beginners trying to dig into this topic often have to deal with the hardest untested legacy code issues, where usually you need well experienced software engineers to successful accomplish bigger refactorings. Thus the learning curve for refactoring “real world systems” is quite steep and is frustrating.
Disproof:
Well, this is a very hard one in my opinion quite true in reality (so the disproof is a bit misplaced here…). The only solution for that is to face the devil, practicing and reading a lot! For example start to refactor little private projects you coded a few years ago or just get some code being written by others (regarding this the open-source world is like paradise). After a while you get accustomed and refactoring gets a good habit in your daily job. I know the first steps will be hard, but give it some time and you will ask yourself how you could develop software without the refactoring concept. My favorite book talking about doing refactorings with untested legacy code is magnificent book by Michael Feathers. Apart from that one I highly recommend Refactoring in large projects and Refactoring to patterns.

Summary

I presented some issues why many people are sceptical against practicing refactoring. I discussed why respective arguments are non-issues and think that in every organization refactoring should be emphasized. I know getting accustomed to new habits is often uncomfy, never the less give it a chance and believe me that preserving and restructuring existing systems can be a very creative work, you feel much better about your software and after all it’s big fun.