Hi;

I think 90% of being confident and competent in reading other people's
code is probably just practice, practice and more practice.

However, I was wondering if anyone has any unique tips to share for
figuring out someone else's code, particularly legacy code.

I've had situations where I have had to read legacy code from
(bad,lazy) procedural programmers.   I've had to translate blocks of
code 100s of lines long, sometimes with dozens of nested conditionals.

Aside from resigning ( hah! ) how do you deal with a situation like
that?

Steve

Steve wrote:

> However, I was wondering if anyone has any unique tips to share for
> figuring out someone else's code, particularly legacy code.

Put it into a sandbox and add a feature to it.

-- 
  Phlip
  http://www.greencheese.org/ZeekLand <-- NOT a blog!!!

"Steve" <stevesusenet@yahoo.com> writes:

>I think 90% of being confident and competent in reading other people's
>code is probably just practice, practice and more practice.

>However, I was wondering if anyone has any unique tips to share for
>figuring out someone else's code, particularly legacy code.

>I've had situations where I have had to read legacy code from
>(bad,lazy) procedural programmers.   I've had to translate blocks of
>code 100s of lines long, sometimes with dozens of nested conditionals.

>Aside from resigning ( hah! ) how do you deal with a situation like
>that?


Hi Steve,

You may be interested in the book "Code Reading":

	http://www.spinellis.gr/codereading/
	http://www.osnews.com/story.php?news_id=5243
	http://www.techbookreport.com/tbr0040.html

and its newer companion "Code Quality":

	http://www.spinellis.gr/codequality/

-- 
Chris.

On 2006-04-19, Steve <stevesusenet@yahoo.com> wrote:
> Hi;
>
> I think 90% of being confident and competent in reading other people's
> code is probably just practice, practice and more practice.
>
> However, I was wondering if anyone has any unique tips to share for
> figuring out someone else's code, particularly legacy code.
>
> I've had situations where I have had to read legacy code from
> (bad,lazy) procedural programmers.   I've had to translate blocks of
> code 100s of lines long, sometimes with dozens of nested conditionals.
>
> Aside from resigning ( hah! ) how do you deal with a situation like
> that?

Tools like ctags, cscope and kscope help a lot; recent versions of
kscope in particular can plot callgraphs graphically which is very
useful.

KCachegrind is useful for this sometimes too-- sometimes only a small
amount of the code actually gets executed, so it lets you find your way
to the bits most worth looking at.

You can often be creative with other tools too-- judicious tracing and
counting of things can be used to build experiments that confirm or deny
hypotheses you form about how things might be working.

The other thing is understanding the domain. Really well-written code is
readable without knowing the domain (and you actually can learn the
domain from it), but bad code is only understandable if you already know
the domain. It's worth spending a bit of time working out how you'd
write a program to do the same thing yourself from scratch, and/or
looking at some readable open-source code that does a similar job. Then
you go back to the bad code again, and think, "ah now I see what this
moron was _trying_ to do..." and it starts to become a lot clearer. You
have to think a lot and try things out, you can't just passively read
the sources. This is a good thing, because reading bad code from top to
bottom as if it were a book quickly becomes very depressing.

Then eventually you have to just start hacking things in knowing they
will go wrong, but debugging the inevitable crashes eventually uncovers
how things were working. You have to find the point where you understand
enough of it to get things working reliably, because understanding the
entire system as thoroughly as you might like could take many years, and
much longer than a complete rewrite.

It always feels frustrating, but you always get there in the end. And
although you might feel like resigning, actually the sort of detective
work can turn out to be quite interesting in a strange way. You can
learn a lot from the mistakes that were made too-- some things are
fairly straightfoward, like dozens of nested conditionals or crazy
overcomplicated boolean expressions; but seeing where the design and
overall structure went wrong is often much more interesting and subtle.

Steve wrote:
> Hi;
>
> I think 90% of being confident and competent in reading other people's
> code is probably just practice, practice and more practice.
>
> However, I was wondering if anyone has any unique tips to share for
> figuring out someone else's code, particularly legacy code.
>
> I've had situations where I have had to read legacy code from
> (bad,lazy) procedural programmers.   I've had to translate blocks of
> code 100s of lines long, sometimes with dozens of nested conditionals.
>
> Aside from resigning ( hah! ) how do you deal with a situation like
> that?

Firstly, I'd agree with everything other people have said, especially
Philip.
Some other notes:-

#
I've dealt with quite a bit of this sort of code.  One thing I've
noticed is that there is a style of programming may appropriately
called "object unorientated".

The programmer writes a subroutine that takes a data structure (call it
X) as an argument along with several other things.  The routine then
figures out what the data structure X is from a choice of several using
a big switch-case statement.  Sometimes it finds out more about X as it
goes along using conditionals.

This type of code is really quite similar to OO code.  The routine
itself is similar to the virtual method dispatcher.  X is the data of
an object, the cases of switch statement are similar to virtual
methods.  The common code that is always executed is similar to a
method of a base object.

Quite often this method of coding is clearer than OO is because all the
code that does similar things is held together, rather than being
scattered all over the place in virtual methods as it often is in OO
code.  But sometimes it just goes crazy and huge functions with
hundreds of lines ensue.

#
Windows message handlers use large conditional structures.  There are
idioms among windows programmers for how they look. The code first
switches on the window message, then on wParam then on lParam (or vice
versa).  Windows programmers have got so used to reading these they
write really long ones without thinking about it or documenting them
heavily.

#
Tracing when conditionals are entered is often useful.  Especially the
first layer of conditionals.  Put in logging code detecting which of
the first layer of conditionals the code enters.  This cuts down the
amount of code you have to read to understand a single feature.

#
Try splitting the function further into subroutines.  Pick a big chunk
in the first level of conditionals, pull it out and make it a
subroutine.  Whether this works or not, and whether it's possible
without passing 10 different variables to it is a good test of how
nasty the code actually is.  If it's possible and easy then the code is
simpler than you think, and the change may not be needed. If it doesn't
work easily then the code is really nasty.

#
Sometimes boolean expressions are less clear than using a set of "if"
statements, even if the 'if' statements are longer.  Consider changing
any really complex boolean expressions.

#
Very occasionally there are functions that should not be modified
because they do such a complex task.  If there is a need to do a
related task then it should be done in a separate function because this
will result in clearer code.  This isn't always a sign of bad code,
sometimes its just a sign that something complex needs to be done.

"Steve" <stevesusenet@yahoo.com> wrote in message news:1145477839.263298.119320@u72g2000cwu.googlegroups.com...
> Hi;
> 
> I think 90% of being confident and competent in reading other people's
> code is probably just practice, practice and more practice.
> 
> However, I was wondering if anyone has any unique tips to share for
> figuring out someone else's code, particularly legacy code.
> 
> I've had situations where I have had to read legacy code from
> (bad,lazy) procedural programmers.   I've had to translate blocks of
> code 100s of lines long, sometimes with dozens of nested conditionals.
> 
> Aside from resigning ( hah! ) how do you deal with a situation like
> that?

Aside from resigning?  Don't get into it in the first place!

-- Bob Day
http://bobday.vze.com

Try looking at your own code a year later.

Bob Day wrote:

>> Aside from resigning ( hah! ) how do you deal with a situation like
>> that?
>
> Aside from resigning?  Don't get into it in the first place!

Oh Oh I'm too awesome to ever deal with legacy code! Oh, I refuse to read 
anyone's code except my own, and not even a year later!! Oh, send the 
corporate jet for me, daddy!!

;-)

-- 
  Phlip
  http://www.greencheese.org/ZeekLand  <-- NOT a blog!!! 

Print out some modules, preferably in a comfortably small font on big paper, 
spread them out on a conference table, and draw circles and arrows on top of 
them with highlighters.

-- 
  Phlip
  http://www.greencheese.org/ZeekLand  <-- NOT a blog!!! 

Ben C <spamspam@spam.eggs> wrote:
> On 2006-04-19, Steve <stevesusenet@yahoo.com> wrote:
>
>> I think 90% of being confident and competent in reading other people's
>> code is probably just practice, practice and more practice.
>>
>> However, I was wondering if anyone has any unique tips to share for
>> figuring out someone else's code, particularly legacy code.
>>
>> I've had situations where I have had to read legacy code from
>> (bad,lazy) procedural programmers.   I've had to translate blocks of
>> code 100s of lines long, sometimes with dozens of nested conditionals.
> 
> Tools like ctags, cscope and kscope help a lot; recent versions of
> kscope in particular can plot callgraphs graphically which is very
> useful.

I have once made a little perl script that converts the output of cflow
to a graphical representation of function calls using graphviz; mayby
this is of use to anyone. It's far from complete or perfect, but does
the job for me. It's available from http://zevv.nl/code/cflowviz/cflowviz, 
an example of it's output is on http://zevv.nl/code/cflowviz/flow.gif.  
I'm happy to receive comments or bugfixes.


-- 
:wq
^X^Cy^K^X^C^C^C^C

Ico wrote:
> Ben C <spamspam@spam.eggs> wrote:
>>On 2006-04-19, Steve <stevesusenet@yahoo.com> wrote:
>>
>>>I think 90% of being confident and competent in reading other people's
>>>code is probably just practice, practice and more practice.
>>>
>>>However, I was wondering if anyone has any unique tips to share for
>>>figuring out someone else's code, particularly legacy code.
>>>
>>>I've had situations where I have had to read legacy code from
>>>(bad,lazy) procedural programmers.   I've had to translate blocks of
>>>code 100s of lines long, sometimes with dozens of nested conditionals.
>>Tools like ctags, cscope and kscope help a lot; recent versions of
>>kscope in particular can plot callgraphs graphically which is very
>>useful.
> 
> I have once made a little perl script that converts the output of cflow
> to a graphical representation of function calls using graphviz; mayby
> this is of use to anyone. It's far from complete or perfect, but does
> the job for me. It's available from http://zevv.nl/code/cflowviz/cflowviz, 
> an example of it's output is on http://zevv.nl/code/cflowviz/flow.gif.  
> I'm happy to receive comments or bugfixes.

There are a few "bugs" that you may want to fix:

     23 if($output_dotfile) {
     24     open OUT, ">&STDOUT";
     25 } else {
     26     open(OUT, "| dot -Tps -Goverlap=none | convert - JPG:- | display -");
     27 }

You should verify that the OUT filehandle is valid or you will get no output
and have no idea what is wrong.

    133 close(OUT);

When you use a piped open you should also verify that the filehandle closed
correctly.

    145     open(IN, "CPP=cat cflow  $args $file 2>/dev/null |");

    222     close(IN);

See above.

    151         /^(\d+)+(\s+)\.*\s?(\S+)\s?\.*\s\{(.*)\}$/;
    152         ($n, $ws, $name, $location) = ($1, $2, $3, $4);

If the pattern on line 151 did not match then the contents of $1, $2, $3 and
$4 would retain their values from the last successful match.  You should only
use those variables if the match was successful.

      6 getopts('devnchI:');

Your usage message does not explain what the 'c' and 'I' options do.



John
-- 
use Perl;
program
fulfillment

John W. Krahn wrote:
> 
>     151         /^(\d+)+(\s+)\.*\s?(\S+)\s?\.*\s\{(.*)\}$/;
>     152         ($n, $ws, $name, $location) = ($1, $2, $3, $4);
> 
> If the pattern on line 151 did not match then the contents of $1, $2, $3 and
> $4 would retain their values from the last successful match.  You should only
> use those variables if the match was successful.

Also the expression (\d+)+ makes little sense because \d+ is greedy so there
is nothing left over for ()+ to match so you either want (\d)+ or (\d+) but
that is moot because you don't use the $n variable anywhere but here.


John
-- 
use Perl;
program
fulfillment

Rob Thorpe wrote:
> Steve wrote:
> > Hi;
> >
> > I think 90% of being confident and competent in reading other people's
> > code is probably just practice, practice and more practice.
> >
> > However, I was wondering if anyone has any unique tips to share for
> > figuring out someone else's code, particularly legacy code.
> >
> > I've had situations where I have had to read legacy code from
> > (bad,lazy) procedural programmers.   I've had to translate blocks of
> > code 100s of lines long, sometimes with dozens of nested conditionals.
> >
> > Aside from resigning ( hah! ) how do you deal with a situation like
> > that?
>
> Firstly, I'd agree with everything other people have said, especially
> Philip.
> Some other notes:-
>
> #
> I've dealt with quite a bit of this sort of code.  One thing I've
> noticed is that there is a style of programming may appropriately
> called "object unorientated".
>
> The programmer writes a subroutine that takes a data structure (call it
> X) as an argument along with several other things.  The routine then
> figures out what the data structure X is from a choice of several using
> a big switch-case statement.  Sometimes it finds out more about X as it
> goes along using conditionals.
>
> This type of code is really quite similar to OO code.  The routine
> itself is similar to the virtual method dispatcher.  X is the data of
> an object, the cases of switch statement are similar to virtual
> methods.  The common code that is always executed is similar to a
> method of a base object.
>
> Quite often this method of coding is clearer than OO is because all the
> code that does similar things is held together, rather than being
> scattered all over the place in virtual methods as it often is in OO
> code.  But sometimes it just goes crazy and huge functions with
> hundreds of lines ensue.
>
> #
> Windows message handlers use large conditional structures.  There are
> idioms among windows programmers for how they look. The code first
> switches on the window message, then on wParam then on lParam (or vice
> versa).  Windows programmers have got so used to reading these they
> write really long ones without thinking about it or documenting them
> heavily.
>
> #
> Tracing when conditionals are entered is often useful.  Especially the
> first layer of conditionals.  Put in logging code detecting which of
> the first layer of conditionals the code enters.  This cuts down the
> amount of code you have to read to understand a single feature.
>
> #
> Try splitting the function further into subroutines.  Pick a big chunk
> in the first level of conditionals, pull it out and make it a
> subroutine.  Whether this works or not, and whether it's possible
> without passing 10 different variables to it is a good test of how
> nasty the code actually is.  If it's possible and easy then the code is
> simpler than you think, and the change may not be needed. If it doesn't
> work easily then the code is really nasty.
>
> #
> Sometimes boolean expressions are less clear than using a set of "if"
> statements, even if the 'if' statements are longer.  Consider changing
> any really complex boolean expressions.

I disagree with this one point in an otherwise excellent essay.

if (a || b && ! c || ! d) bnargFlapdoodle();

is clearer than

boolean chamunkle = 0;
if (a)
    chamunkle = -1
else if (b && !c)
    chamunkle = -1
else if (d)
    chamunkle = -1;
if (chamunkle) bnargFlapdoodle();

even when I leave the inner Boolean expression b && !c in. Also, note
the unnecessary generation of a clumsy Boolean only for this test!

My take is that because an anti-intellectual and math-anxious MIS
management in America prefers not to hire mathematical sophisticates,
to whom the above complete Boolean expression is what the French would
call potage du canard  (duck soup), programmers only feel better when
they must trace through if thickets as opposed to making a truth table.

Another contributing factor is the fact that many of today's
programmers were Dazed and Confused by Pascal in High School in the
1980s, but today use a language which evaluates || and && quite
differently, rendering the programmer insecure as to whether the right
hand sides of and and or are even evaluated.

Still another problem with ifs is the different ways in which final
dangling else clauses are handled, and differing mental pictures. In
the above code, it's hard to tie the elses with the ifs even if the
code is formatted.

If Boolean business rules are truly overcomplex I suggest a radical
solution described in my </shamelessPlug>book, Build Your Own .Net
Language and Compiler (Apress 2004) </shamelessPlug>: design a little
language or graphical display in which the user enters the business
rules, and REPLACE the legacy code by a simpler interpreter of the
user's choices.

The political problem is that rats-nest Boolean code which nobunny
understands often indicate an underlying problem: that contrary to the
stated mission of the business or organization, some long-departed
users have made the business rules excessively complex in order to
conceal malfeasance or misfeasance. This seems for example to occur in
American health insurance applications, the mission of which has in
some cases become "deny the chump health care and let him die".

Transparency, in the form of a graphical user interface that would
allow the new user to see what rules actually apply when the system is
actually used (something of which CEOs and government figures today
seem proud to be ignorant, the rules per se being for the little
people) might actually be opening a can of worms.

>
> #
> Very occasionally there are functions that should not be modified
> because they do such a complex task.  If there is a need to do a
> related task then it should be done in a separate function because this
> will result in clearer code.  This isn't always a sign of bad code,
> sometimes its just a sign that something complex needs to be done.

Phlip wrote:
> Print out some modules, preferably in a comfortably small font on big paper,
> spread them out on a conference table, and draw circles and arrows on top of
> them with highlighters.

Yes, that's another good thing to do.

I once did this and afterwards stuck the listing to a big wall next to
my computer.  Occasionally afterwards, I kept thinking "now I've got
rid of most of the really nasty stuff" then I looked back at the wall
and saw loads of bits still highlighted.

Steve wrote:
> Hi;
> 
> I think 90% of being confident and competent in reading other people's
> code is probably just practice, practice and more practice.
> 
> However, I was wondering if anyone has any unique tips to share for
> figuring out someone else's code, particularly legacy code.
> 
> I've had situations where I have had to read legacy code from
> (bad,lazy) procedural programmers.   I've had to translate blocks of
> code 100s of lines long, sometimes with dozens of nested conditionals.
> 
> Aside from resigning ( hah! ) how do you deal with a situation like
> that?
> 
> Steve
> 

Sure, follow these simple steps:

1. Remove all comments from the code. There are programs available that
do this automatically.

2. Change all identifiers to meaningless or misleading ones. Again,
several automatic programs are available for this.

3. Discard all documentation.

4. Lose the phone number of the original programmer, so you can't
contact them.

Then see if you can read the program.

If, you find out that none of the above steps are necessary, as they
are already done, then congratulations, you already work at one of the
major corporations that make this country great.

scott moore wrote:
> Sure, follow these simple steps:
>
> 1. Remove all comments from the code....
> 2. Change all identifiers to meaningless or misleading ones....
> 3. Discard all documentation.
> 4. Lose the phone number of the original programmer, ...
> ...
> If, you find out that none of the above steps are necessary, as they
> are already done, then congratulations, you already work at one of the
> major corporations that make this country great.

(Or governments.)

I've never maintained a business application so my question is probably
very naive but ...

Why not just throw it all away and start from scratch?

My retired friend was called back into service to add Y2K capability
to tax assessment software.  Millions of dollars were spent just by
that
county to Y2Kize programs written in Cobol, with IBM's hierarchal DBMS.
I didn't tell my friend but I felt the money would be better spent
writing
from scratch with C/C++ and SQL.

Of course I frequently read in comp.risks about modern programming
projects coming in years late and $100M's over budget so I'm
probably wrong....

James

James Dow Allen said:

> Why not just throw it all away and start from scratch?

What makes you think the bunch of buffoons they'll hire this time will be 
any better at programming than the bunch of buffoons they hired last time?

-- 
Richard Heathfield
"Usenet is a strange place" - dmr 29/7/1999
http://www.cpax.org.uk
email: rjh at above domain (but drop the www, obviously)

James Dow Allen wrote:

> Why not just throw it all away and start from scratch?

Because all the debugging that went into the original code represented 
fine-tuning thousands of business decisions. These now only exist in two 
places - the scrambled mind of the program, and the demanding mind of the 
customers.

> I didn't tell my friend but I felt the money would be better spent
> writing from scratch with C/C++ and SQL.

Also because new development is incredibly high-risk for such projects. 
Politicians who expected to manage bribes and such are now thrust into the 
thankless role of managing developers.

> Of course I frequently read in comp.risks about modern programming
> projects coming in years late and $100M's over budget so I'm
> probably wrong....

http://mayford.ca/blog/?p=21

Dave Rooney writes, "I first learned about [highly incremental development 
with relentless testing] while working on a large federal government system 
called GCMS. We were starting to build the framework for an "army of Java 
developers" to use, but were stopped dead in our tracks after a few weeks 
when management was told they had to evaluate COTS products for the system. 
We warned them that the promises of 85% functionality off the self were BS, 
but they didn't listen..."

-- 
  Phlip
  http://c2.com/cgi/wiki?ZeekLand <-- NOT a blog!!! 

On Mon, 01 May 2006 12:15:37 GMT
"Phlip" <phlipcpp@yahoo.com> wrote:

> James Dow Allen wrote:
> 
> > Why not just throw it all away and start from scratch?
> 
> Because all the debugging that went into the original code represented 
> fine-tuning thousands of business decisions. These now only exist in two 
> places - the scrambled mind of the program, and the demanding mind of the 
> customers.

	Even worse the customers will not think of most of the edge cases
until they come up (again) they've long forgotten the first time they came
up and were worked out and the result enshrined in some obscure corner of
the software.

-- 
C:>WIN                                      |   Directable Mirror Arrays
The computer obeys and wins.                | A better way to focus the sun
You lose and Bill collects.                 |    licences available see
                                            |    http://www.sohara.org/

In article <IZm5g.545$zR3.209@newssvr33.news.prodigy.com>,
 "Phlip" <phlipcpp@yahoo.com> wrote:

> James Dow Allen wrote:
> 
> > Why not just throw it all away and start from scratch?
> 
> Because all the debugging that went into the original code represented 
> fine-tuning thousands of business decisions. These now only exist in two 
> places - the scrambled mind of the program, and the demanding mind of the 
> customers.
> 
> > I didn't tell my friend but I felt the money would be better spent
> > writing from scratch with C/C++ and SQL.
> 
> Also because new development is incredibly high-risk for such projects. 
> Politicians who expected to manage bribes and such are now thrust into the 
> thankless role of managing developers.
> 
> > Of course I frequently read in comp.risks about modern programming
> > projects coming in years late and $100M's over budget so I'm
> > probably wrong....
> 
> http://mayford.ca/blog/?p=21
> 
> Dave Rooney writes, "I first learned about [highly incremental development 
> with relentless testing] while working on a large federal government system 
> called GCMS. We were starting to build the framework for an "army of Java 
> developers" to use, but were stopped dead in our tracks after a few weeks 
> when management was told they had to evaluate COTS products for the system. 

You should have finished that last sentence:

   ...or they wouldn�t receive additional funding.

So now the question is really, was management wrong? Sure the COTS 
system wasn't going to work, they were warned, but they had a choice of 
evaluating them and getting the funding money, or not doing it and 
possibly having to shut the program down for lack of funds...

Given that and:

   ...I suppose we were seen as having a self-interest in the system 
   being built in-house.

That is a very real problem that management has to contend with...

In article <1145477839.263298.119320@u72g2000cwu.googlegroups.com>,
 "Steve" <stevesusenet@yahoo.com> wrote:

> Hi;
> 
> I think 90% of being confident and competent in reading other people's
> code is probably just practice, practice and more practice.
> 
> However, I was wondering if anyone has any unique tips to share for
> figuring out someone else's code, particularly legacy code.
> 
> I've had situations where I have had to read legacy code from
> (bad,lazy) procedural programmers.   I've had to translate blocks of
> code 100s of lines long, sometimes with dozens of nested conditionals.
> 
> Aside from resigning ( hah! ) how do you deal with a situation like
> that?

The best way I know of for understanding strange code (including my own 
some time later) is to change it. Either a change that I know can't 
possibly break anything (like adding whitespace) or a change that I know 
will refuse to compile. For example, I might change the name of a 
variable to get a bead on where it's used.

My goal is to find somewhere interesting to start, it's rarely the 
beginning. Some block of code which I don't understand but doesn't look 
like it will be too hard to figure out. I will try to work out what the 
preconditions and postconditions are to this block of code which leads 
me to an understanding of what the block does and how it does it, then I 
start looking at the blocks that use this block to try and figure out 
*why* the functionality is needed.

"scott moore" writes:

> 1. Remove all comments from the code. There are programs available that
> do this automatically.

Certainly treat all comments with a jaundiced eye.  IMO most comments are 
made at too fine a granularity, I want the big picture, not nitty-gritty, I 
can add  the latter myself.

> 2. Change all identifiers to meaningless or misleading ones. Again,
> several automatic programs are available for this.

First of all, I know sarcasm when I see it.  But at least for small student 
type programs One of the first steps I take is to change the identifiers to 
meaningless brief names, aa, bb,c  c .  So often the names chosen are simply 
misleading and one spends too much effort wondering what he meant by that - 
and guessing wrong with extremely high frequency.  The name encodes the 
programmers take on the situation (which was wrong), and leads to bad code. 
The compiler doesn't give a rat's ass about meaning and neither should I. 
If I *know* going in, that the name doesn't mean anything I can focus on the 
code itself.

I don not mean to suggest this works for the outer levels of huge programs, 
but it is might possible help when you get down to the level of a few 
selected functions.  I really feel sorry for people who have to do this for 
a living day after day; I suspect there are a great many programmers who do 
eaxactly that, year after year.  It has to be maddening.

Daniel T. wrote:

[snip]


> You should have finished that last sentence:
>
>    ...or they wouldn=B9t receive additional funding.
>
> So now the question is really, was management wrong? Sure the COTS
> system wasn't going to work, they were warned, but they had a choice of
> evaluating them and getting the funding money, or not doing it and
> possibly having to shut the program down for lack of funds...

No, management was directed to look into a COTS solution.  Further
funding was contingent on exploring that concept, not in choosing it.

>
> Given that and:
>
>    ...I suppose we were seen as having a self-interest in the system
>    being built in-house.
>
> That is a very real problem that management has to contend with...

Our (the team building the framework) track record in recent projects
was better than management's.

I can't remember off-hand if I mentioned that the COTS solution was the
3rd attempt to replace the legacy systems.  The first 2 attempts failed
outright, with absolutely nothing shipped for millions of dollars
spent.  Both of those attempts used cutting-edge technology for the
time, and there were no existing deployments of systems in that
organization with that technology.  Chalk up one huge project risk
right there.

In our case, we were building a Java framework that consolidated
features from half a dozen Java-based systems that had already been
delivered at that point.  We were also using the existing and proven
database and application server infrastructure.  The previous attempts
and the COTS solution all used different technologies, which introduced
another risk.

At any rate, we quite accurately predicted what would happen to GCMS.
I know people on that project who worked their asses off for what now
appears to be nothing.  People had vacations cancelled, worked 70 hour
weeks, and became quite burned out.  For nothing.

What a waste.

Personally, I'm tired of just tut-tutting such failures.  We know
better ways to build software now (and did back then as well).  People
have to their heads out of their butts and realize that traditional
development methods DON'T WORK.

Dave Rooney
Mayford Technologies
http://www.mayford.ca

dave.rooney wrote:

> No, management was directed to look into a COTS solution.  Further
> funding was contingent on exploring that concept, not in choosing it.

So in one corner are these airhead programmers wearing flip-flops and weird 
haircuts, and in the other corner stands the full weight of the marketing 
department of some big software outfit. Nice.

-- 
  Phlip
  http://c2.com/cgi/wiki?ZeekLand <-- NOT a blog!!! 

"Steve O'Hara-Smith" <steveo@eircom.net> wrote in message news:20060501231049.5dda6caf.steveo@eircom.net...
> On Mon, 01 May 2006 12:15:37 GMT
> "Phlip" <phlipcpp@yahoo.com> wrote:
> 
>> James Dow Allen wrote:
>> 
>> > Why not just throw it all away and start from scratch?
>> 
>> Because all the debugging that went into the original code represented 
>> fine-tuning thousands of business decisions. These now only exist in two 
>> places - the scrambled mind of the program, and the demanding mind of the 
>> customers.
> 
> Even worse the customers will not think of most of the edge cases
> until they come up (again) they've long forgotten the first time they came
> up and were worked out and the result enshrined in some obscure corner of
> the software.

And what do you do when, due to the code having been patched so many
times, or the language it was written in having become obsolete, or the
hardware it was written for having become obsolete, the program is
no longer maintainable?

-- Bob Day
http://bobday.vze.com

On Tue, 02 May 2006 14:31:51 GMT
"Bob Day" <xxxxxx@yyyyyy.com> wrote:

> "Steve O'Hara-Smith" <steveo@eircom.net> wrote in message news:20060501231049.5dda6caf.steveo@eircom.net...
> > On Mon, 01 May 2006 12:15:37 GMT
> > "Phlip" <phlipcpp@yahoo.com> wrote:
> > 
> >> James Dow Allen wrote:
> >> 
> >> > Why not just throw it all away and start from scratch?
> >> 
> >> Because all the debugging that went into the original code represented 
> >> fine-tuning thousands of business decisions. These now only exist in two 
> >> places - the scrambled mind of the program, and the demanding mind of the 
> >> customers.
> > 
> > Even worse the customers will not think of most of the edge cases
> > until they come up (again) they've long forgotten the first time they came
> > up and were worked out and the result enshrined in some obscure corner of
> > the software.
> 
> And what do you do when, due to the code having been patched so many
> times, or the language it was written in having become obsolete, or the
> hardware it was written for having become obsolete, the program is
> no longer maintainable?

	Call in a bunch of consultants who promise to make it all better
for a suitably large fee and cry when it doesn't all work.

-- 
C:>WIN                                      |   Directable Mirror Arrays
The computer obeys and wins.                | A better way to focus the sun
You lose and Bill collects.                 |    licences available see
                                            |    http://www.sohara.org/

>> No, management was directed to look into a COTS solution.  Further
>> funding was contingent on exploring that concept, not in choosing it.

> So in one corner are these airhead programmers wearing flip-flops and weird
> haircuts, and in the other corner stands the full weight of the marketing
> department of some big software outfit. Nice.

Hey, my haircut isn't weird! ;)

But seriously, this project wasn't farmed out to an integrator or big
software outfit but rather the work was performed in-house with
contractors from various (sometimes large) companies brought in.

However, the picture you paint isn't all that far from the truth.  As
far as management was concerned, we were "just programmers", and didn't
understand the nuances of building very large systems.  Um, that's
because we were busy actually delivering smaller systems!!

Dave Rooney
Mayford Technologies Inc.
http://www.mayford.ca

Bob Day said:

> And what do you do when, due to the code having been patched so many
> times, or the language it was written in having become obsolete, or the
> hardware it was written for having become obsolete, the program is
> no longer maintainable?

Scrap the annual bonus that would have been paid next month; put everybody 
on a minimum of 55 hours per week; expect daily, or preferably twice-daily, 
status reports; change your reporting procedures; re-organise the teams; 
hand out attaboys to the guys with the best hair; hire six contractors from 
a consultancy firm chosen randomly from your latest batch of corporate 
spam, at rates in inverse proportion to their knowledge and experience, and 
put them in charge of your in-house developers; and finally, announce in 
the trade press that Foo 2007 will be ready for release by the end of 
Spring quarter, with a whole new user-friendly paradigm and extra quux.

-- 
Richard Heathfield
"Usenet is a strange place" - dmr 29/7/1999
http://www.cpax.org.uk
email: rjh at above domain (but drop the www, obviously)

Bob Day wrote:

> And what do you do when, due to the code having been patched so many
> times, or the language it was written in having become obsolete, or the
> hardware it was written for having become obsolete, the program is
> no longer maintainable?

Google "sunken cost fallacy".

(I _like_ that term!!!)

-- 
  Phlip
  http://www.greencheese.us/ZeekLand <-- NOT a blog!!!

In article <1146578532.001205.193370@u72g2000cwu.googlegroups.com>,
 dave.rooney@mayford.ca wrote:

> Daniel T. wrote:
> 
> [snip]
> 
> 
> > You should have finished that last sentence:
> >
> >    ...or they wouldn�t receive additional funding.
> >
> > So now the question is really, was management wrong? Sure the COTS
> > system wasn't going to work, they were warned, but they had a choice of
> > evaluating them and getting the funding money, or not doing it and
> > possibly having to shut the program down for lack of funds...
> 
> No, management was directed to look into a COTS solution.  Further
> funding was contingent on exploring that concept, not in choosing it.
> 
> >
> > Given that and:
> >
> >    ...I suppose we were seen as having a self-interest in the system
> >    being built in-house.
> >
> > That is a very real problem that management has to contend with...
> 
> Our (the team building the framework) track record in recent projects
> was better than management's.
> 
> I can't remember off-hand if I mentioned that the COTS solution was the
> 3rd attempt to replace the legacy systems.  The first 2 attempts failed
> outright, with absolutely nothing shipped for millions of dollars
> spent.  Both of those attempts used cutting-edge technology for the
> time, and there were no existing deployments of systems in that
> organization with that technology.  Chalk up one huge project risk
> right there.
> 
> In our case, we were building a Java framework that consolidated
> features from half a dozen Java-based systems that had already been
> delivered at that point.  We were also using the existing and proven
> database and application server infrastructure.  The previous attempts
> and the COTS solution all used different technologies, which introduced
> another risk.
> 
> At any rate, we quite accurately predicted what would happen to GCMS.
> I know people on that project who worked their asses off for what now
> appears to be nothing.  People had vacations cancelled, worked 70 hour
> weeks, and became quite burned out.  For nothing.

How nice for you, but let's be fair here. The managers didn't have the 
luxury of waiting until the project failed and then saying, "I told you 
so." They were left with the choice (a) go with you and this new XP 
thing (a new technology with it's attendant risk,) or go with the other 
guy and their new technology.

The other guy won the contract but failed the managers, the managers 
failed the customers, and who did you fail? If only you could have 
convinced the managers, there would be many more happy people in the 
world today. All those people who worked all those hours on a doomed 
project because why?

Seriously though, I'm not literally saying you are at fault, the above 
comes from some time I did in the car sales industry. When it's your 
(self appointed) job to give your (potential) customer the information 
he needs to make the right decision (at least in your opinion,) and he 
fails to do so, then you haven't done your job...

As Phlip said, programmers vs professional sales people... When the job 
is to sell, I'd expect the latter to win.

Richard Heathfield wrote:
> James Dow Allen said:
> 
>> Why not just throw it all away and start from scratch?
> 
> What makes you think the bunch of buffoons they'll hire this time will be 
> any better at programming than the bunch of buffoons they hired last time?
> 

I'll ditto that. Everyone always says "this should be rewritten". That
rarely gets ok from the manager, who when the subject comes up, hardens
his or her jaw, looks wistfully at the horizon and talks sadly about
"project X", that went so very wrong, with programmers who no longer
work here.

However, I am old, so I have been around long enough to see rewrites
actually take place. The ones that succeed were the ones where the old
program was so much behind the times that there was no choice but to
move forward to the new program, which was invariably buggy and slow
beyond belief. As with all new programs, the bugs got fixed, and the
slowness was corrected or addressed with bigger faster CPUs, and life
went on.

Certainly %80 of the code out there should, indeed be thrown out.
However, rewriting it would simply create a new %80 trashpile.

I have a simple rule for rewrites.

Go ahead and rewrite, with the following ground rules:

1. The rewritten code must be compatible with the old code, module
by module. I.e., each module of the old code must mesh with the new
code, even if older modules are eventually discarded when replaced
by newer, better modules.

2. The old code, with the new code added, must continue to run and
be bug free while the rewrite occurs.

3. If the old code has no discernable modular structure (common), then
the old code must be reworked to be modular state before being replaced.

Unsurprisingly, I have yet, in my 30 year carrier as a programmer, to
be taken up on my simple method to rewrite programs. Instead (to be
fair), I shall list the methodology for rewrites that I have actually
observed in practice.

1. Create a new group to rewrite the code, consisting of few or no
people from the original code maintainers. They are busy, you see, and
most of them protested the rewrite as being unnecessary.

2. The code is written in isolation.

3. Several schedule deadlines are missed.

4. Active state of war exists between the old and new code groups.
The old group "has heard" the new code does not work, and does not
implement all the required functions.

5. The new group is placed under new management, since the old group
wants it killed off to reduce the budget.

6. The new code reaches a demonstration milestone.

7. Management recoils in shock and horror at how buggy and slow the
new code is.

8. The new code project is effectively killed.

9. The new code project is resurrected when its found that competitor
X is killing existing products with new features possible only in the
new code.

10. Several years later, most of the original code programmers have
left, and the units that use the old code are becoming unsaleable.

11. Management requires that new units use the new code.

12. New unit developers complain the new code is causing them to
be late in getting their product to market.

13. Finally, everything is working. Units are starting to sell.

14. The company is bought by competitor X, and most of its technology
is discarded in favor of the new companies products.

Or something like that.

scott moore wrote:

> Unsurprisingly, I have yet, in my 30 year carrier as a programmer, to
> be taken up on my simple method to rewrite programs. Instead (to be
> fair), I shall list the methodology for rewrites that I have actually
> observed in practice.
>
> 1. Create a new group to rewrite the code, consisting of few or no
> people from the original code maintainers. They are busy, you see, and
> most of them protested the rewrite as being unnecessary.
>
> 2. The code is written in isolation.

Oh, my. Waterfall doesn't work, even when the requirements are perfectly 
stable?

> 14. The company is bought by competitor X, and most of its technology
> is discarded in favor of the new companies products.

Great post, dude!

Rewrite incrementally, under test, targetting frequent releases, and retire 
bits of the old code early and often, guys!

-- 
  Phlip
  http://c2.com/cgi/wiki?ZeekLand <-- NOT a blog!!! 

On Fri, 05 May 2006 11:18:06 -0700
scott moore <nospam@nowhere.com> wrote:

> Go ahead and rewrite, with the following ground rules:
> 
> 1. The rewritten code must be compatible with the old code, module
> by module. I.e., each module of the old code must mesh with the new
> code, even if older modules are eventually discarded when replaced
> by newer, better modules.
> 
> 2. The old code, with the new code added, must continue to run and
> be bug free while the rewrite occurs.
> 
> 3. If the old code has no discernable modular structure (common), then
> the old code must be reworked to be modular state before being replaced.

	Of course one problem that causes the need to rewrite is that the
old code has a structure that doesn't fit the way requirements are (and
have been) changing. Radically restructuring code while still keeping it
working *and* tracking changing requirements is a tricky art (but great fun
to do).

-- 
C:>WIN                                      |   Directable Mirror Arrays
The computer obeys and wins.                | A better way to focus the sun
You lose and Bill collects.                 |    licences available see
                                            |    http://www.sohara.org/

Phlip said:

> scott moore wrote:
> 
>> 2. The code is written in isolation.
> 
> Oh, my. Waterfall doesn't work, even when the requirements are perfectly
> stable?

That isn't what he said.

Waterfall works fine, if you have the right people. I'm sure TDD works fine 
too, if you have the right people.

In fact, just about /any/ methodology works fine, if you have the right 
people.

The hard bit is finding the right people.

Most projects never manage that bit, so for most projects it doesn't matter 
which methodology they use, because none of them will work fine.


-- 
Richard Heathfield
"Usenet is a strange place" - dmr 29/7/1999
http://www.cpax.org.uk
email: rjh at above domain (but drop the www, obviously)

"Steve" <stevesusenet@yahoo.com> writes:
> However, I was wondering if anyone has any unique tips to share for
> figuring out someone else's code, particularly legacy code.

I am by no means an expert programmer, quite the opposite, but here are
my two cents:

With C programs and a few other languages, I find it helpful to run
the code in a debugger, such as gdb. I also find a literate programming
language, such as CWEB (there are other ones, such as noweb that aren't
oriented to a particular language) very helpful. Very simply, I commit
to converting the godawful program into a work of literature, which can
include my negative reviews of the original code and how it might be
done better. I don't have much experience with this approach, but I'm
presently taking someone else undocumented C program, just for practice,
and converting it to CWEB. One can do this without understanding the
program. The resulting file, which has a .w extension, can be handled in
the following ways:
(1) it can be run through a processor called cweave, which produces a TeX
    file, with extension .tex, which can then be run through TeX to produce
    a printable document that will be your first draft of an article on
    all of the details of the program. You write your second draft by
    marking up the first draft and then modifying the original .w file.
    This activity is rather pleasant, or at least I find it so, and removes
    a lot of pressure from the act of understanding the entire program all
    at once, in favor of dividing and conquering it.
(2) it can be run through a processor called ctangle, which produce a C file,
    with extension .c, which can then be compiled with a C compiler, e.g.
    with gcc. When compiled, the resulting executable is essentially what
    is supposed to be produced from the original C program.

Merely getting your .w file to run through cweave and then TeX, and also
through ctangle and then gcc, gives you a certain amount of passive knowledge
of the contents of the original C program, and doesn't really require you
to understand the program. And it lets you deal actively, rather than
passively, with the program. And marking up and editing your first draft
and planning your second draft, and subsequent drafts, are all active
and rewarding, or at least I find it so. And meanwhile, you get stronger
at reading and understanding the program and even learning things from
it, no matter how bad it looks.

Among the advantages of the document produced by cweave and TeX are:
(a) It has a table of contents.
(b) It has an index, listing every identifier in the C program and every
    section of the document in which that identifier is mentioned. So, you
    get for free something like the information provided by ctags.
(c) There is a certain amount of cross-referencing automatically generated
    by cweave that also makes it unnecessary in some cases to use the
    index and table of contents.

Thus, the first draft becomes both the book you are reading and the book
you are writing. I think it's pretty cool. There is a newsgroup devoted
to literate programming: comp.programming.literate

> Aside from resigning ( hah! ) how do you deal with a situation like that?

As long as no one is putting time pressure on you, it really doesn't matter
what the original code is like or how long it takes, especially if they
are paying you by the hour. At any rate, you need peace to do this and
really get anything out of it. So, you should have a clause in your
contract that says they aren't allowed to make you nuts when you do
stuff like this. If, on the other hand, the pressure comes from within,
based on your own priorities, do whatever you can to be nicer to yourself
and just think of this as something that will get done a little at a time
with a little bit of effort every day and, meanwhile, find other things to
do that you find easier to complete with the rest of your time.
-- 
Ignorantly,
Allan Adler <ara@zurich.csail.mit.edu>
* Disclaimer: I am a guest and *not* a member of the MIT CSAIL. My actions and
* comments do not reflect in any way on MIT. Also, I am nowhere near Boston.