Impact of Quality in Development

Click here to see the prior rambling tale.

Click here to see the next rambling tale.

Impact of Quality in Development

I was surprised by the high turnover rate amongst the developers at etransmedia. While I was there under a year, they had a greater than 100% turnover amongst their dozen or so developers and I was then the thrid most senior developer. For as long as I had worked as a computer programmer, turnover had been the bain of our development. If turnover ever got above 30%, we would suffer and consider what we could do to correct the problem. I had never seen anyone even attempt to do software development with such high turnover and that got me considering why it is so important the software be developed by a stable and well motivated team.

As I consider it, I believe that the reason is that any new development of technology, especially software, is a much more creative process than almost any other employment. If a programmer writes even a moderately complex function, there are countless possible solutions and the programmer needs to continuously make choices as the program is developed. Most of the choices have no impact whatsoever, but many have complex impact on the overall performance of the application as a whole.

The development process is further complicated by the fact that while it is possible to write quite detailed specifications of what the desired function should do, but as the programmer must make so many decisions and choices in the develpment process that the real quality of the function can not be controlled to any substantial degree.

For example, if you counted only substantial differences between possible functions, there might only be a million possible solutions of a simple function. All of the solutions meet the requirements of the specification, but perhaps only 1,000 would be great solutions. There might be 10,000 good solutions and 100,000 OK solutions. There could be 400,000 poor solutions and almost 500,000 awful solutions. Now all of these solutions meet the specifications but most of the solutions lead to a quite unsatisfactory user experience (as you can not specify a pleasant user experience as a part of the specification).

In order to get a great solution (which would make a technology a success rather than a failure), it is required that the programmer have the skills, knowledge, time and motivation to choose one of the great solutions. Please understand that it takes substantially greater effort to arrive at the great solutions and there is no way to force an unmotivated pogrammer to produce good quality code. Indeed, unhappy programmers can choose awful solutions which meet specifications out of spite.

Software development is further complicated by the fact that in almost all cases it is a team effort; no one programmer can complete all the sections that are required within the required time frame. Having a stable team is essential for the level of communication necessary for good quality solutions (part of the knowledge piece along with the necessary experience in the work domain) as well as developing the team standards for high quality code (other programmers can easily recognize junky code and part of the team ethos needs to be a desire to produce high quality code by each team member). Further, a stable team can develop methodologies so that adequate resources are provided to allow quality solutions. Lastly, it is only in a stable team that the motivation to produce great solutions can be developed.

The product developed at etranmedia which had such junky code can be understood in this light. The code was developed by adequate programmers (they had the required skills to do great code), but was developed for a demo which was later turned into a product without substantial revision. Demo code is flashy and developed quickly while production code is usable, efficient, scalable, reliable and servicable. As there is so little in common between the qualities of good production code and demo code, basically the code has to be rewritten from scratch for a production product (most of the decisions made in developing the demo code are wrong for a production environment). In essence, the programmers did not have the knowledge, time or motivation to do a quality production product.

The reason demo code is so much different from production code is that the targeted environment is so different. If code is intended to be installed just once and run only a few dozen times, then the correct choices will be to not handle all the odd special cases which inevitably crop up; just do the installation by hand (crafted by the programmer) and insure the demo does not encounter the odd cases. However, in a production environment all those odd cases need to be handled programmatically; it is a huge number of decisions concerning how to efficiently process all the odd cases, but it is also a great saving to handle those cases automatically rather than requiring manual intervention to correct the unhandled cases.

To move demo code into production without the complete revamping of all the choices made in the demo code will certainly incur huge support costs as all the odd cases come up and require manual correction with the inherent customer disatisfaction and ad hoc correction. The turn over of personnel greatly increases these costs as each new programmer needs to review all the thousands of choices made by the previous programmer to understand how to make the necessary choices to correct the problem. It is that loss of knowledge which makes high turn over so costly.

An analogy of moving demo code into production would be taking a movie set made of papier mache and making it your residence. It won't stand up to the weather or the wear and tear of day to day life. It will be woefully inadequate to the needs of real life and attempts to upgrade buildings made of papier mache will surely fail. Adding a roof that could withstand hail, snow, rain, and wind would be too heavy requiring a whole new frame and so on. Movie sets just aren't designed to actually live in them or to maintain.

In a development environment, turn over becomes particularly problematic. It not only indicates that the team members are not well integrated into the team, happy or highly motivated, but there is also a dramatic cost from turn over. In most professions there is moderate training required to become productive from one environment to the next, but in development, it is required that each team member be familiar with all the choices and design decisions that have been made previously by the team members. Of course a software developer can read the code written by a previous developer to learn the choices already made so that any updates and revisions (new choices) are compatible, but that is hugely time consuming. The reality is that the new programmer will only review a few of the choices made by the previous programmer (making new development slower, but not absurdly so) and the new choices won't fit well with the previous choices (leading to some gaps in coverage or bugs and some overlap of coverage making the code slow and cumbersome). All of which increases cost and decreases quality.

I believe that the culture of India had a big impact on why etransmedia could not effectively manage their development team as discussed in the next tale.

Click here to see the next rambling tale.


This page was last updated on May 3, 2012