Why programmers have trouble implementing scientific/engineering software

If it was that easy, we wouldn't need the scientists

In 2000, Professor Koutsoyiannis published a paper with a method for creating good synthetic time series. It doesn’t matter what “synthetic time series” are or why it is important to make them. The thing is that it is a hard problem and it involves fairly advanced math and concepts.

Soon after, Professor Efstratiadis (then a PhD student of Koutsoyiannis) implemented the method with Delphi, creating the software that they called “Castalia”. He maintained it for a few years, evolving it according to new research.

About ten years later, I asked them how long it would take me to understand the method (I don’t remember why we had this discussion, but it was probably because we were looking into what resources we’d need to maintain Castalia). They told me that if I did heavy reading and worked closely with them so that they could help me, it would take me about one month to understand the whole thing.

While this doesn’t seem much, it’s important to see that they were talking specifically about me, not just any programmer. I had been working with them for 20 years, I had done a little bit of research with them, and I already had some understanding of the concepts. Another programmer could need six months instead. In addition, I have the feeling that this research is relatively easy to understand; research based on more advanced math could require years to learn.

The naïve solution to the problem of scientists writing bad code is to let them do science and leave the programming to programmers. But it isn’t that simple. If the programmer needs years to understand the method, such a solution would be too expensive.