I realized the other day that on OS X, the Spotlight indexing process is started using launchd. This makes it very easy to modify the launchd configuration script to insure that background indexing uses the least amount of CPU and I/O bandwidth possible.

Edit the configuration script by running this command as root:

# open /System/Library/LaunchDaemons/com.apple.metadata.mds.plist

You should find yourself in the Property List Editor application. Now add two keys at the top-level, one named LowPriorityIO, which is a boolean set to true, and another named Nice which should be an integer set to 20.

Now whenever the mds spawns mdworker processes to index recent changes to the file system, it won’t get in your way quite as much as before. (Without this change, mdworker processes run at the same priority as user processes, according to output from the ps axl command).

Tags: Optimization

It's been a while since I've posted a script; life has been distracting lately. I also wanted to let this current script mature a lot more before sharing it, as it has the potential to be destructive. Use wisely!

It's name is linkdups, and it's a Python program to recursively walk through a directory tree and hard-links any files together whose contents match exactly. That means that if you have two files, each taking up 10 Kb, afterwards they will be linked to the same contents for a total savings of 10 Kb. Read More...

Tags: Python, Optimization, Filesystems, Shell

There has been a small debate among some Mac users about whether defragmenting your disks is necessary for the smooth operation of OS X. I’ve always been in the camp of those who do it regularly, because I’ve seen what my disk ends up looking like after a few weeks of not doing it (and how pretty the graph looks afterwards). It could all be psychological, but I find the progress bar rather hypnotizing, and my wife has been known to find me staring at it for hours on end. Ok, very psychological.

Well, I’ve found one circumstance where defragmentation is definitively helpful: compressing sparse disk images.

I have a habit of creating encrypted disk images for every client I work for. A lot happens in those images, which tend to grow and shrink quite a bit. OS X has a command to squeeze out the unused space, which looks like this:

$ hdiutil compact DiskImage.sparseimage

I just ran this command on one of my work images, and it reported a savings of 786 Mb out of 3 Gb. Just for interest’s sake, I ran iDefrag on the same volume, which eliminated the small amount of fragmentation that had built up, and compacted the files down toward the beginning of the image.

Running hdiutil compact on the same disk image again resulted in a further savings of 518 Mb! Given that the image itself is now 1.4 Gb, that’s about a 30% further reduction.

So, if you’re like me and you use lots of virtual disk images — and you’ve always wondered if defragmentation tools were worth anything at all — here’s one reason to consider it.

Tags: Optimization, Virtualization, Filesystems

Common Lisp has often been called a "multi-paradigm" language, in that it allows you to program in many different styles, sometimes simultaneously: imperative, object-oriented, functional, statically typed, etc. It depends on what style you want to adopt, how your code will look.

Recently I've been porting a C++ accounting system to Common Lisp. And after only six weeks, the port is nearly complete — a feat I credit to the power of the Lisp language and the facilities it offers I'd been forced to replicate in C++.

But as the port nears completion, I find myself questioning some of the design decisions. Did C++ force me down a path where Lisp can offer a better alternative? Read More...

Tags: Common Lisp, Programming, Optimization, FP

I’ve found a very easy way to use local DNS caching on the Mac: simply setup Internet sharing from one device you don’t use, to another you don’t use. Since I almost always get my Internet access through wireless, I’ve setup my system to share my Fireware port to anyone connected to my Ethernet port.

Although this sharing setup doesn’t do any sharing, what it does do is to cause OS/X to run a local DHCP server and a local DNS server. This local DNS server takes its nameserver addresses from your current Internet configuration (in my case, wireless), so everything is automatically setup to cache DNS from the nameservers you’re actually using.

The only thing necessary to do is to change /etc/resolv.conf to point at your new local nameserver. Using the Network Preferences Pane, find out what your local Ethernet address is (to use my example; you’ll have to find the IP of the interface you’re sharing from). Now edit /etc/resolv.conf so it looks something like this:

nameserver 192.168.2.1

In my case, my local Ethernet interface gets set to 192.168.2.1 when I’m using sharing. You may need to setup a cron job (check out the utility CronniX) in order for your resolv.conf to get overwritten each time with this setting.

Now site back and enjoy the added speed of cached DNS! This is especially helpful on connections that drop packets a lot, since I find a great number of the “pauses” in my Internet usage all relate to lagged out DNS lookups.

Tags: DNS, Optimization