Why does it take so long?

Why does it take so long to write really good software? Unfortunately, it seems that the devil is always in the detail, and things take longer than expected. I’m definitely not the only software developer where I have a bright idea, and I can conceptually see exactly what I want the software to do, but actually getting it to do that turns out to be a long and arduous battle.

Unfortunately, software is not a forgiving medium: for every situation where you can see roughly what you want, you then have to describe it in precise and excruciating detail to a CPU that has no intelligence whatsoever.

Another thing is writers block. If I’m enthused about something, then I can just code and code and code, but tricky details being what they are, sometimes, you have hours or days when the old grey cells don’t quite want to get themselves round the problem.

Another productivity issue is the extent to which you want to reuse code. Some people like being “hobby kit” programmers: they are very happy to take freely available bits of code off the web, and them sew them, patchwork quilt wise, into a product. The catch is of course that when it comes to code licensing, it’s not necessarily all theirs, and although it’s invisible to the end user, some of the hacks that get written to get unrelated modules of code to co-operate can be just plain ugly. A quicker way to get a product, but down the line, the developers start muttering, ewww. yuck!

Me, I tend to be a bit of a perfectionist, and I can write the code myself, I will. The downside with that is that after a decent software engineering degree, with the exception of things requiring specialist knowledge, I should be able to write the code for most things. Which means of course, that my default attitude seems to be “oh, I’ll just write a few more thousand lines, and it’ll be there.” Streaming systems, no problem, Object hierarchies, no problem, parsers, no problem, vector and matrix math, no problem, database persistence systems, no problem, interpreters, no problem…….

The problem is that when you want to write everything exactly the way you like it to be, it takes absolutely ages. It used to be that by the time I’d written five or six thousand lines of code, I couldn’t help but end up with some sort of a working program. Nowadays, I can write twenty or thirty thousand lines of internal structure, and still end up with a simple test program that has nothing but a text box, and two buttons on it! Ahhh…. such is the path of progress!

All comments welcome!

OO software: Corn is not beef is not hash.

My favourite bugbear. Appropriate delegation / compartmentalisation / structure in OO software. There are some excellent books concerning design patterns and anti-patterns.  The general rule of thumb is: stuff in a class deals with that class, and delegates to neighbours as appropriate. So:

  • The code that formats the message, is not
  • The code that retrieves the data that the message needs, which is not
  • The code that trawls through some structure looking for a target which might know how to deal with the message.

For those that don’t have time to read through the official textbooks, my rules of thumb:

Do one thing and one thing only in a function, examples:

  • Formatting a message.
  • Updating internal state on the basis of a message.
  • Delegating things elsewhere.
  • Performing some algorithmic calculation.
  • Converting from one format to another.

OO software works well on the basis of ignorance. Yes, it’s important that the code does stuff, but really good OO code works well when each bit of code knows only about a relatively limited subset of its own class, ancestor classes, and a minimal subset of stuff around it. Code should be as ignorant as possible. i.e. Keep your design contracts simple.

That means good OO software should be generic. Yes, some bits of code have to know the knitty-gritty of how some algorithm works, or how some data is structured, but in general, if your code does not know how to do X, and X is not locally or closely related to your class, then just delegate it elsewhere.

Joel-esque rant over.

 

PageMapper

To showcase the parsing code I’ve written, I now have a demo app which allows you to view the structure of pages, not unlike the debuggers in MS Edge, Chrome etc. For example:

It’s not perfect, and doesn’t parse all pages perfectly (Google being a good example….), but it does pretty well on most pages, and I am still improving the underlying parsers.

If you’d like to get hold of the source code or like the parsers as a library, then feel free to contact me, and we can discuss your educational or research use, or negotiate licensing terms.

Here are the 32 bit MSI and 64 bit MSI installers.

An alternate Streaming System.

Delphi has fairly good support for RTTI (reflection). Quite a few of my programs need to stream sets of classes to disk, and read those classes back. This library leverages the RTTI system to provide automatic streaming of classes and published properties, with an extensible API to perform custom streaming.

The source is here. No documentation I’m afraid, you will have to read code and think.

MFract – a simple fractal generator.

A couple of years back I wrote a simple fractal generator to test some of my threading libraries, and a custom streaming system that I had written. It generates very pretty pictures like this:

I have MSI installers for both 32 Bit Windows and 64 Bit Windows. The application comes with HTML help, and lots of sample files.

For those wanting to get techy, and as it contains plenty of good sample code, the source can be found here.

A multi-index container.

Delphi offers some fairly simple containers: list, queue, hash table, but does not have trees or tries.

I present here a multi-index container which maintains a doubly linked list, with multiple (sorted) indexes into it. The indexes are implemented as AVL trees. You can create custom indexes and perform high speed efficient sorting, searching and iteration.

Unlike the STL, no knowledge of templates is required.

 

 

Multithreading, The Delphi way, Part 2.

This tutorial is a comprehensive update to my original multithreading tutorial, which takes into account changes in Delphi compiler technology, changes in underlying hardware technology, improvements to the Win32(64)API, Delphi extensions for OSX and Android, and the result of 15 odd years of learning in industry how threading technologies actually work.

At the moment, it is incomplete, (I only have so many hours in my life), but offers many newer threading techniques, and scalable implementations thereof.

Multithreading, The Delphi way, Part 1.

Originally written in 2000, just a couple of years after I left University, this tutorial gives a reasonable introduction to Multithreading in Delphi.

Some of the material is a touch dated, and more recent versions of Delphi have built-in functionality which was manually written. All the basic concepts still apply, and most of the functions and API calls are still relevant and function in the same way.