IHME has a high-performance computing with around 15,000 cores (as of June 2020). Our statistical applications are composed of many individual jobs spread across the cluster. It is feasible to manually control an application that has fewer that a hundred individual jobs. Manual control becomes burdensome when there are thousands of jobs, and simply impossible with hundreds of thousands of jobs. As part of my work at IHME I created Jobmon, a scientific workflow manager (similar to Nextflow and Cromwell) that manages distributed applications with up to a million separate jobs. Jobmon has a Python API, with a central restful control service and database. Jobmon enabled IHME to includes results for every year in the the GLobal Burden Disease study, rather than just for every fifth year. The final GBD application had 350,000 jobs, took 30 days, and completed without errors. Jobmon is described in this paper.
While working as a C++ developer in the 1990's I took a course on the Personal Software Process by Watts Humphrey. I was particularly impressed by the experimental verification that was possible. Unlike most methodologies, PSP includes a way of testing if it really works. The key ideas of PSP are:
In other words PSP is nothing more than scientific observation. I used PSP for five months on a C++ project, and then another four months on some Java projects. After two iterations I carefully went over all my recorded defects and discovered some patterns in my defects. My major problem was caused by copy-and-pasting code, and then not changing all the variable names etc. Having discovered that I paid more attention to those activities, and on my next component it actually ran the first time on the test harness without errors, something I had never managed before (or since I might add). My estimation for small projects also improved dramatically, although it still does not solve the problem of estimating large projects. It was not only clear that the methodology worked "in the small," but I had hard evidence (rather than opinion) that did work.
I performed all the work using paper forms.
As an interesting side effect of my PSP usage I realised that I had performed a very interesting study in the effectiveness of Java and C++. I had PSP data for several months in both languages, and the two data sets had the same programmer, similar applications etc - the only difference was the programming language. Just looking at the raw numbers it appeared that when I was writing in Java I wrote twice as many lines of code per minute, and produced half as many defects per line of code. If anything these numbers were conservative because I at that time I had seven years C++ experience, but was only just learning Java. However, when the numbers are analysed using a proper statistical test (Student's t-test) then it is only possible to say that Java is 50% better in either category, which is still a huge difference. I wrote the results up for Software Practise and Experience, a copy of the paper is here.
For Gallipoli and Mons I created counter design software, which is available in my GitHub account. There are also several fantasy language generators that I use in role-playing games.
Text and Images Copyright Geoffrey Phipps 2007-2020. Unauthorised copying or reuse prohibited.