Disaster Recovery

My big fear is losing all my data in some sort of system crash. Well, I just had a system crash last week (bad hard drive). As it turned out, it was marginally painful. My backup system was designed for managing my data. Fortunately, my data were on the drive that survived. The dead drive housed the system OS.

After some false starts, I managed to find an old copy of the system and restored it onto a new disk. Within a few hours, the system was back up to normal, more or less.

Although I didn't backup as religiously as I should have, even a marginal backup scheme prevented a total loss. Any backup is better than none.

SATA vs. Firewire vs. USB 2.0

’ve been needing a serious evaluation of my backup habits. One area is mixing my storage media for longer-term backup. Up until now, I’ve been using DVD’s, but given the hundreds of gigabytes of data I can create in a month or two, and questions regarding long-term storage of writable DVD media, I’ve begun delving into the possibility of using hard drives as part of an overall strategy. But the world of external drives has become exceedingly complicated. Firewire (IEEE-1394), USB 2.0, and more recently SATA/ESATA.

The performance rating of these interfaces are 400 mb/s for 1394a and 800 mb/s 1394b, 480 mb/s for USB 2.0, and 1500 mb/s for SATA 1 and 3000 mb/s for SATA 2. But what are their real-world performances like? The following is the first part of my exeriments on performance

The equipment:
1. The computer - Macbook pro - USB 2, 1394a, Express card for external SATA, internal SATA drive

2. External Firewire/USB 2.0 drive with a 250 GB IDE drive.

3. External SATA 1/USB 2.0 drive case with a 320 GB SATA drive.

Method:

I used a typical directory for me, 17.35 GBs, and copied from one drive to the other several times using a shell script. Although I’m mac based, none of these files required resource forks so no special handling was required.

Here are the results of copying back and forth to the mac’s internal drive. The figure below shows the copy time in seconds (best results are SHORTER).



Not surprisingly, the SATA interface performed the best. USB 2.0 performed the worst. Generally, the hardware limitations of the mac’s internal workings are a major factor here, but the results are inline with other results out there on the web. The other limitation is the external drives themselves. Keep in mind that most drives can only write about 30-80 MB/s and the best performer here is only about 230 mb/s, or about 30 MB/s (note that this speed is far far slower that each of the interface’s ratings).


The second experiment is copying external drive to external drive. Since I only have two cases and limited USB 2.0 ports, I was limited in the experiment. Here are the copy times in seconds (best results are SHORTER):




Again, USB 2.0 is the slower of the interfaces. Interestingly, the copy speed to the SATA drive from the Firewire drive is relatively fast: 293 mb/s, or about 37 MB/s; this is the fastest copy speed of the experiment, and it was approaching the 400 mb/s for the firewire interface. I consider that a very good result for Firewire along with SATA.


This first experiment show that my purchase of a SATA1/USB combo case was probably not the best solution - a case that had SATA1 or 2 and firewire would have been far better for my needs. USB 2.0 as a solution is clearly not good enough for me.

Okay, I understand - XCode, Universals, and Version Control

Originally published: Monday, August 21, 2006

Will Shipley of Delicious Monster argued at a group discussion at Adler Planetarium that developers should avoid using their own custom frameworks, and harness the power of version control instead. To some extent, this didn’t go down all that well. As someone who was saw the discussion online after the fact, I was left a little perplexed, but then I do consider myself an amateur. After all, frameworks are handy ways on reusing code, and how would a version control system going to help me reuse code effectively?

Enter my work to port my applications to Universal binary (ppc/intel binaries). I’ve had to go back and retouch all my frameworks, and figure out how to build universal binaries of the open source code I use. The universal open source code began to show me the Will Shipley light...

Usually, open source code compiles easily; run ./configure, and make. With universal code, you have to do a bit more than that, depending on the project. So, it was time to get the open source in a local version control repository. The problem is, of course, at this point I now had about five separate version controlled projects. What I typically did before now is check out and compile each project manually. The problem with this is the code can easily go stale and I can make mistakes easily. Furthermore, it becomes a pain if I need to switch computers, which happens more often when I travel, and I have to go through that entire process again.

Enter subversion. Subversion is my preferred version control system. Of the many things in subversion I don’t know, the svn:externals property is one of those things that would have made life much easier. By setting the svn:externals properties, subversion can go out to other repositories or other places within the same repository and bring them into your project hierarchy. So, all those frameworks and open source projects can become a part of my main project fairly easily. Then it’s just a matter of making my main project dependent on those sub projects. No tedious copying, and no working builds going out of date.

So, now that I can do all this with libraries and frameworks, can I get rid of my frameworks?? As it turns out, I can, in theory, get rid of one of them (well, actually all of them, but let’s not get greedy), although I haven’t as of yet. All it takes is reorganizing my existing framework repository to just include the files, not xcode project, and check them out within another project. Then, they act just like regular files I can add to a project - and updating as needed!

So, now that I see Mr. Shipley’s point and how it can be done, will I take his advice? Time will tell.

Universal Mac App: My first shot at Universal compiles

Originally published on Monday, August 14, 2006

With a good deal of trepidation, I grudgingly converted one of my apps to Universal format for Mac. That is, it runs on native PPC or Intel code. Although there has been a great deal of comment out there that building Universal apps is easier than it sounds, I felt my app was one of those which would be a difficult battle. The app itself uses GUI, open source, a custom framework, and byte order specific code. Worse still, my code style isn’t what I’d call “good”. Uhg!

For the most part, it was actually very painless. Fortunately, the open source code handled byte order issues internally, so that turned out a little easier than expected. In the areas in which I specificially swapped bytes, well, I mostly had that code right anyway, so with a little work, it’s just fine too.

Of course, I haven’t fully tested the result. As far as I can tell, it works almost exactly as bad on a intel mac as it does on a PPC mac. When I say bad, I mean that it’s not well or fully implemented. It’s just a GUI app I’ve been working on for years that I never bothered to polish (I plan on making a polished version, but will I???).

So, rating my first experience for 1-10, I’d say 8.5 (1 point off for having to do it in the first place).

XML and InDesign

Originally posted Saturday, July 8, 2006 on another blog site;

One of the issues I’ve been wondering is how well does InDesign really deal with XML and can Docbook be used directly? So, what’s wrong with docbook? For HTML creation, it’s great. For PDFs, it’s good. Compared to InDesign, the PDFs are not higher in quality, but the fact docbook is a free system is nothing to sneeze at. What’s forced me to move, in part, from Docbook to InDesign is because InDesign allows placement of postscript images, where Docbook-FOP does not.

One of the surprising differences, particularly in XML handling, is that InDesign doesn’t handle Docbook out of the box. It can be done, however. But for my purposes, I’m not really THAT tied into Docbook because almost all of my Docbook files that I’m currently using are generated by software. With some changes in the software, a little setup time in InDesign, and a javascript for InDesign, I also automatically layout the same information (with postscript images instead of lower quality jpegs) in a much nicer form and, I think, an easier to control environment.

Where InDesign falls flat in comparison, at least so far for me, is the exporting to GoLive and generating web pages. Not happy with that workflow, at least not yet. Docbook handles this much better. So, I’ll still use Docbook for html layout.