Climate model on a mac project: #12 In Production!

At last, the new machine is now doing production work and I've run almost 24 hours so far.  First, it's running a bit slower than anticipated, about 13.5 model years per day. I'm not sure what's causing the slower speeds, but it's still an acceptable speed considering my original estimate was only 7 model years per day.

Temperature remains a worry for me. The CPU temps seemed reasonable, but the RAM temps seemed rather high for long-term processing.  So, I ordered some fans to cool the RAM boards.

Tomorrow, I'll take a look at the output and start planning a possible Cocoa front-end for running the model.

 

Climate model on a mac project: #11 sysctl.conf and mpich

A key component for running the climate model I'm using is MPICH. MPICH is an implementation of the MPI (Message Passing Interface) library for supercomputing maintained at Argonne National Laboratory (disclosure: I am a former employee of the lab). The climate model uses MPICH to break down the world into smaller parts and distribute those parts to each CPU. However, these parts must also talk to each other because climate information must move from processor to processor so that each processor knows what's going on around it.

MPICH comes installed in Leopard (I'm not sure if it's installed with the developer installation or the standard client installation) and is meant for use with GCC only. It is also likely that it's intended for XGrid as well. So, for this project, the pre-installed MPICH does not have the required fortran to run my simulations.

As it turns out, the default Leopard configuration is lacking a critical configuration for running MPICH on a single mac. It does not allow for shared memory. Using shared memory can speed up computation when you're using multi-processor or multi-core machines because it helps limit how much ethernet bandwidth the processes will use. Ethernet bandwidth is a major bottleneck for processing speed, which is why many high performance clusters use optical networking such as Myranet.

To allow shared memory on Leopard, /etc/sysctl.conf must be created. The contents of the file include something like:

kern.sysv.shmmax=536870912 kern.sysv.shmmin=1
kern.sysv.shmmni=128
kern.sysv.shmseg=32
kern.sysv.shmall=131072

Unfortunately, I don't know the optimal settings for these parameters, I just found these settings off the net. However, they do work.

And now, MPICH can be compiled and installed on the system.  Here is the contents of my script to build MPICH:

make distclean
export CC=pgcc
export CXX=pgcpp
export FC=pgf90
export F90=pgf90
export F77=pgf77

export CFLAGS='-Msignextend '
export FC=pgf77
export FFLAGS=
export CCFLAGS=

./configure -prefix=/opt/mpich/pgi/7.2/ --with-device=ch_shmem --enable-g --enable-sharedlib

These instructions are for the Portland compilers and for a custom install location.

Unix, Microcomputers, 1985 on the Computer Chronicles

Hindsight is 20/20, so they say. I was a big fan of the PBS series "Computer Chronicles" during my university days. I suppose that was partially due to the fact I didn't have cable and it was a inexpensive way of learning about what was going on in the computer world. In any case, I recently discovered that many of the episodes of this show were online. When I have a chance, I flip through the archive and watch a bit of the show.

Today, I found an extremely interesting episode from 1985 on Unix. Since Linux and Macs are both derived from Unix, I thought I'd give this one a shot. At the time this show aired, I was still in high school and I owned a commodore 64 and loved using it. I don't think I had any knowledge of Unix at the time. I think my sister and cousins had encountered Unix by then at universities, but hated it.

"Will unix become the next MS-DOS?" asked Stuart, the host. Apparently, at this time "microcomputers" were becoming powerful enough to run Unix. Speculating that Unix may become THE DOS of the future, the show goes into a brief description of Unix.

Interestingly, some of the terminology of the show is fundamentally different than today. Specifically, what's the difference between mini and micro computers? I don't know, actually, but I can guess there is certainly a price difference!

Also interestingly, some of the criticisms of Unix brought up in the show are still relevant today. Specifically user friendliness. The demo someone gives of Unix is so user unfriendly that even the presenter pounds on the keys a bit and then just talks instead. Later, he even says that not everyone needs Unix...

Another big plus was C, the programming language (I don't think I touched C until the late 80's early 90's).

Another surprise to me was that Unix was widely considered unreliable at the time.

Various forms of unix were discussed or mentioned, including a GUI-driven HP (HP-UX) version.

In an opinion piece later in the show, Unix is panned as a microcomputer OS... "a user-hostile operating system"

In the news bit at the end, and the Pentagon's biggest threat at the time was a "Mentally unbalanced 16-year-old".

And for Gruber at Daring Fireball, there's a news item on keyboards!

Watch the episode

Climate model on a mac project: #10 Rebuilding

Now that the model clearly works using the test license from Portland, it's time to rebuild the system. So far, I've reinstalled MacOS onto the system and installed the developer tools.

Next, I installed required version control software. At this point, subversion and cvs are already on the system so I only needed to install GIT.

The next step in the process is to migrate my subversion and git repositories to the new system and get the servers going.

The version control systems will handle a few things. First, one of the systems will be used to handle the /etc directory. I found that I end up many different versions of mpich and other libraries and not only to I want to prevent messing up this directory, but possibly use branches to determine which versions of these software versions.

Model configurations will also be under version control. Right now, subversion is used to maintain these files. However, git will probably take over this role since a distributed model (since I use more than one cluster) make more sense for configuration management.

Next up: Modifying the system for shared memory runs.

GIT vs SVN

Over the last few years, I've really taken to using version control for as many of my projects as possible. This includes more than just source code. Climate model configurations, unix /etc directories, writing projects, presentation work, etc have benefitted from using version control. When I haven't used version control, I've lost track of files and I made serious mistakes.

These days, I'm suffering from a wealth of options for version control. Originally, I did all my work in CVS. When subversion (SVN) was released, I quickly shifted my repositories and never looked back. However, many more version control systems have appeared and use radically different models for source control. CVS and SVN use a central repository from which everyone obtains their files and commits their code. The new thing is distributed repositories. That is, there might be a canonical repository, but the repository people use on a day to day basis may be somewhere else, like their own machine. The practical upshot is that you're not dependent the network for committing; you can do it locally. So, if you work on an airplane for example, you can still have the benefits of version control while you work.

I've begun using GIT for some of my work. Many of the concepts of version control are very different than for CVS or SVN, so the transition isn't the most straightforward. However, to give it a go, I used GIT to do versioning for my 2008 AAPG poster session. For this kind of work, I don't have to use most of the features of version control, but it helps keep me from generating dozens of versions of the poster on my system. As such, GIT worked great. The files are large and the commits are very quick. The overall experience was quite good. After a little more testing, I'll likely transition all of my writing/presentation work over to GIT.

GIT seems to have potential for climate model configurations as well since quick branching, and distributed nature could be quite helpful. However, using SVN, I keep on my model configurations together as one repository. In GIT, I'd probably would have to have separate repositories for model configuration families.

GIT also will likely replace SVN for my source code. But that will take a bit more testing before I start using it widely.

SVN, however, will remain in place for many of my projects. I don't see any version control system as perfect and SVN is still a very good system. Furthermore, git support on windows (when I do windows work) is limited.

Climate model on a mac project: #9 Getting in running in MacOS X

One misgiving about running a climate model on the mac is that I don't trust linux to handle the Mac's fans to keep the mac cool. Apparently, there is even more to it. The mac also will throttle memory bandwidth if the ram gets too hot (see technote TN2156). Whether or not a Linux install not targeted to a Mac would handle these issues, I don't know.

All those worries seem to be moot, however. I managed to get the model running within Leopard. The next few posts will concern some of the details of how this was accomplished.


Tom