Is Garbage Collection Needed For NSOperationQueue?

I ran into a brick wall over the last few days.  I've been processing my climate model images using two NSOperation subclasses and NSOperationQueue in MacOSX Leopard.  In my case, the wall was the fact that the Queue simply refused to release my operations until the queue was empty.  If I only had a few operations, this would be okay.  However, I generate about 2000-2200 operations in a fairly short amount of time, and they should take about an hour or so total.  However, when I tried submit them all, the system was completely bogged down by the end of the run and would crash.  It was so bad, I'd have to restart if I wanted to get back to work.

If you're familiar with these objects, you'd think this is because I didn't release the objects after submitting to the queue.   I tripled checked; everything was properly released.  I stuck a NSLog statement in the -(void)dealloc method to see when it was called.  Absolutely none of my operations were dealloced until the job queue was empty.  There are dependencies, however, but eliminating those did not solve the problem and only added complications.

I tried a number of approaches to get the queues to empty.  I created multiple queues, each one to be destroyed after running a number of jobs, but this set up failed.  I created a queue manager that would feed  a single queue after all jobs were submitted - failed.  Frustration!

I then recognized that the problem was likely removing the completed object from an array was an autorelease issue.  That is, the object isn't deleted right away when it's removed from the queue.  Instead, it hangs around for a while until an autorelease pool removes the object.  In my own code, this problem is easily solved by wrapping the code in a pool, between

NSAutoreleasePool *aPool = [[NSAutoreleasePool alloc] init];

   and  

[aPool release];

That usually solves the problem.

In this case, however, the queue is an opaque library class, I can't get in there and add a pool!  In desperation, I started looking up the relatively new garbage collection in Objective-C 2.0.  My hope here would be that garbage collection could be my way of injecting an autorelease pool action into the queue.

My first try was a failure, although some objects were released.  I needed a mechanism to trigger collection.   As it turned out, there was such a mechanism: 

[collector collectIfNeeded];

 I called this method when a change occurred in the number of operations held by the queue changed using KVO.  This worked!  I watched as the jobs ran and finalized!

As it turned out, however, if an object is a dependency for another object, it is still not released after it runs, only after the depended object is released, which makes sense.  However, what this means is that a large number of objects still lingered far longer than they should.  It appears that the queue takes a first come, first serve attitude that skips an operation if it isn't ready and doesn't come back to that object until every other object gets a chance.  The solution for this (which failed without garbage collection) is a second queue that handled all the operations that were dependent on other operations.  Thus, the second queue ended up running these operations far sooner and thus releasing both objects more quickly.   

Finally, I'm ready to move on to the next problem!