Blog Viewer

SSPA Blog: Walking down the 640 kb memory lane …

  

This post is the first in what I hope to be a series of posts regarding our profession of “statistical programming.” As I pondered what to write about first, I did a mental map of the journey I’ve taken from formal education to the current job – and what programming tools I’ve used along the way. I thought I’d share a few memories of these tools. I’m interested in comments from readers – what statistical software have you used then or now that you’ve found particularly helpful, unique, or interesting?

In my college career, I recall three main statistical programming tools: SPSS, Minitab, and BMDP.

I can’t remember all the details – it’s been a really, really long time – but I think it was a PDP-11 that was the computing workhorse. I DO remember being truly wowed by SPSS! Being able to do a simple linear regression with a handful of statements was remarkable. Up until that time, I think I did all the arithmetic with a calculator and many sheets of scratch paper – with some stat book always nearby for reference.

I also recall being particularly impressed that Minitab could do the inverse of a matrix in … one line? Is that right? [ I should look this up ] Previously, I was forced to do the work by hand, and it was painful. That’s my Minitab college memory – working with vectors and arrays as objects themselves (quite a bit different than dealing with FORTRAN-based approaches).

BMDP came into the picture – I think I remember this right – because it offered some analysis of variance routines (maybe repeated measures?) that SPSS or Minitab couldn’t handle at that time. It was a pretty big deal!

SAS didn’t enter my world (or vice versa) until my first job – I remember being told to learn to program in SAS on my own – with the help of the manuals (SAS version 82.3, I think – again, is that right?). I was so proud of myself after mastering PROC UNIVARIATE.

Since then, it’s frankly mostly been SAS – with a brief interruption for work with RS/1 ( a statistical programming language – also vector/array based – which seems to have fallen by the wayside). Not surprising, I suppose – I worked for the RS/1 software vendor!

Lately, I’ve been trying to teach myself R – which is heavily used in the academic area and seems to be making a lot of headway in commercial/industrial work.

In our profession, there’s always a frontier – always something to learn.

By the way, bonus points to readers who get the reference in the post’s title! J

3 comments
7 views

Permalink

Tag

Comments

01-25-2011 23:29

That's an interesting note about R - all the more reason for all stat programmers to get familiar with how it works, etc.
Also, you make an interesting point about SAS and "programmable!" Learning SAS/Macro was (and continues to be) an interesting intellectual exercise - it's sort of "meta programming" - writing code which generates code. A very weird concept when I first encountered - and still not horribly obvious to someone who learned BASIC and FORTRAN as first programming languages.
I have to think a bit more about if SAS is truly "programmable." I suppose SAS/IML is more to what you mean - sort of more functional in some sense. Your point also makes me think about SAS in the software engineering paradigm - many SAS programmers, I think, came into SAS indirectly - not necessarily formally trained in software development.
I'd like to think about this a bit more - and I'm curious as to what other statistical programmers think about this ... Paul, thanks for posting this comment - you've started me on a path of some self-examination!

01-25-2011 18:01

Thanks for starting this thread, Mike.
I can't speak for members of the pharmaceutical world, but in finance, R is now the preferred tool for statistical programming. Really, nothing else comes close. The annual R/Finance Conference held here in Chicago draws participants from around the globe, and people applying for quantitative finance jobs need to know R.
Your posts make me wonder: what do you consider to be "programmable"? For instance, I would say that basic SAS is not programmable but SAS/Macro is programmable (albeit clumsily). To be programmable, I want to see variables, conditional execution, loops, and functions. What would you look for?
BTW, your bonus points are pretty easy: 640KB was the memory limit on the original IBM PC. Those days are long gone.
Paul

01-21-2011 01:19

After reviewing your blog, looking back at my journey during school days when I was introduced to one other computational software besides the once you outlined above was WinBUGS to perform Bayesian Analysis. But just last year I was introduced to an interface developed using SAS macros which performs the remote execution of WinBUGS system to do the Monte Carlo simulations and extract the estimates from it and pull them back in SAS for reporting purpose, which I was fascinated to know about.
It’s an interesting fact that we are finding ways to integrate and standardize such statistical programming tools seamlessly into each other to meet our reporting needs.