The Freedom to Somnambulate

Week of May 28, 2000

Not too long after X-file made it through the finish line at the 26KB advertised in the London Daily Telegraph, a status window was added with code that had gathered dust waiting for a semblance of relevance. Being able to see things like free disk space on a logical volume or partition, the number of objects in a directory, how many were selected, etc. was trivial, and so the code was not used, but when the type of pending file operation was suggested, a third status window pane even Exploder doesn't have was added and the code was worked over and a status window was finally put in.

The story behind finding the free disk space is a grievous one. The Win32 API has a function called GetDiskFreeSpace for this task (that sounds natural, doesn't it?) and the documentation for GetDiskFreeSpace starts out as follows:

BOOL GetDiskFreeSpace(
    LPCTSTR lpRootPathName,         // address of root path
    LPDWORD lpSectorsPerCluster,    // address of sectors per cluster
    LPDWORD lpBytesPerSector,       // address of bytes per sector
    LPDWORD lpNumberOfFreeClusters, // address of number of free clusters
    LPDWORD lpTotalNumberOfClusters // address of total number of clusters
);

All very straightforward. You don't fill in lpRootPathName - if you did, you'd win the Bart Simpson Kwijibo Award. Left blank, the function returns the free space on the drive you're currently on, which is what you're after. You don't need the last value, but you do need the other three, and you have to multiply them together to figure out how many bytes are free. And that is potentially a very big number, more than the native Win32 DWORD can handle.

Double precision floating point is a good way to go for such a calculation, as it will give you decimal points in the result. But 64-bit integer math can also be used, and will generally result in significant code savings. How well Windows 9x supported 64-bit integer math was not known, so an executable image was sent to a likely guinea pig for testing (thank you, pig).

The guinea pig reported back that the pig puter only got 2GB on a volume, no matter that the volume in question was significantly larger. Woah, what's up? The rest of the documentation on GetDiskFreeSpace is what's up. As you will see, Win32 also has a new super pooper function called GetDiskFreeSpaceEx which always reports the correct result - but wouldn't you know it? This function is not supported on all versions of Windows 9x. Again, what's up? Here is the relevant passage in GetDiskFreeSpace's documentation - get ready with your flight bag:

Remarks

Windows 95:

The GetDiskFreeSpace function returns incorrect values for volumes that are larger than 2 gigabytes. The function caps the values stored into *lpNumberOfFreeClusters and *lpTotalNumberOfClusters so as to never report volume sizes that are greater than 2 gigabytes.

Even on volumes that are smaller than 2 gigabytes, the values stored into *lpSectorsPerCluster, *lpNumberOfFreeClusters, and *lpTotalNumberOfClusters values may be incorrect. That is because the operating system manipulates the values so that computations with them yield the correct volume size.

Great, isn't it? You have to read that several times to get the full import of what's going on here. And note the officious declaration that some values are intentionally incorrect so that others intentionally are not - magnificent DoubleSpeak, and some magnificent system Redmond is administering there, isn't it? But again, what exactly is 'up'? To find that out, let's go way way back to...

The Dawn of Civilization

Once upon a time, before even Stanley Kubrick's monkeys drooled on the monolith, there was something called a DOS Volume Header. The DOS Volume Header was a great header, but it did have its limitations, designed as it was in Redmond Washington. Among other things, it severely limited the number of clusters on a DOS volume. The cluster count was stored in the header in two bytes, 16 bits, so the maximum number of clusters on any volume was 65536. Well almost. Actually it was 65534, because the first two clusters were used as a kind of 'convenience placeholders' by the file allocation table head programmers (the 'FAT Head Programmers').

This was great. As volumes and disk capacity increased, opportunities for people using DOS machines actually decreased: they lost more and more disk space to the ever bloating volume clusters. A shortcut of a mere 300 bytes (another Microsoft design gem) consumed as much as 32 kilobytes on disk. Go Microsoft!

And DIRENTRY records had this limitation too: the starting cluster was again only 16 bits. (It would have to be, because the FAT head programmers didn't have bigger FATs.)

Yet it never needed to be a problem at all: there have always been ten completely totally imperviously unabashedly unused bytes in the DIRENTRY record, begging, screaming for use, for relevance. After the file name, its extension, and the attributes follow ten whole bytes of pure nothing until we get into the modification date and time, file size, and starting cluster number. They're just begging to be put to use (and some hackers did use them - 'nuff said). And yes, at least once earlier Redmond had increased the capacity of DOS drives, and very well could have done it again. But Microsoft needs to retain its freedom to somnambulate: in their dubiously intellectual world, the rule is that nothing needs to be improved, no code need be tweaked, and as long as there is no competition, things are just hunky dory. Major news flash.

A quick perusal of Ray Noorda's documents provides good supplementary reading in this matter: MS-DOS stagnated for years while ISVs and OEMs everywhere yelped out their annoyance and frustration at its increasingly embarrassing limitations, but their pleas fell on deaf ears (and some would say decidedly numb minds) in Redmond. So finally they all went to the UK in search of someone who didn't mind doing something about it - a single someone, one person, who completely re-wrote MS-DOS and made it sensible for the first time in a long time. And the result, later known as NDOS or Novell DOS, was very very good, extremely good in fact.

This caused a wave of panic in Redmond, with the somnambulists walking into walls and Billg sending angry memos asking for what was to become the infamous 'AARD' code, a shameless hack with a single purpose: to frighten NDOS users off. And Windows 95, then under development, was bound irrevocably to DOS, not for the sake of integration as Redmond would have us believe, but to force the OEMs through product bundling away from NDOS and keep it out of the market.

(So the next time you hear Microsoft representatives or dupes nag about that precious 'freedom to innovate', you know now where you can tell them to get off and what they can do about it.)

None of this would have been necessary if Redmond had fixed Tim Paterson's MS-DOS from the start, if volume control blocks and the directory entries and lots of other things had been properly attended to, and DOS users everywhere would have benefited immeasurably, with zigabytes more disk space at their disposal. And if Microsoft had not exercised its freedom to somnambulate, they would not have ever needed to cripple the function GetFreeDiskSpace either.

For ISV applications of today the dilemma is obvious: the original Windows 9x cannot get at GetFreeDiskSpaceEx because it isn't there, while GetFreeDiskSpace returns deliberately incorrect values.

But didn't Microsoft correct GetFreeDiskSpace for updates to Windows 9x? Think again - this is Microsoft code! They left the bad function as is, deliberately left the bad code in there: instead of fixing it, they invented a completely new one. (Ok, grab the flight bag again if you need it.)

And so how in hell's name are ISVs to deal with this mess, this typical Microsoft Mess? What Microsoft recommends is a classic for sure, and it's completely typical of the Microsoft approach to ISV application development: 'please fix what we did not feel like fixing ourselves'.

Isn't it great? Here we see the kind of hoops Microsoft expects ISVs to jump through because they themselves are too lazy. (If you're not acquainted with this very Microsoft thing, then this is your first lesson - this is typical Microsoft.)

Using LoadLibrary(Ex) obviates the whole point of using import libraries. We're not supposed to have to use these except in 'hacks', and goodness but doing something as innocent as finding out free disk space is not supposed to be a hack - but it's what the proposed Microsoft snippet becomes. LoadLibrary(Ex) is probably the wrong function to call anyway, as almost every process running in any machine is dependent on KERNEL32.DLL. Whatever. Having all that 'shitecode' in a serious application is simply not acceptable.

What people don't seem to grab in this so-called 'browser war' that TP is on the verge of putting an end to is that the victims are not just AOL newbies or Internet Explorer casualties: the ISVs have been hit, and hit hard.

If Microsoft had devoted less time to Nixonian kamikaze missions against middleware and more time to cleaning up their own act we'd all be a lot happier today. Perhaps Windows 9x wouldn't leak like a sieve and crash like a piece of junk. Perhaps Dave Cutler would have stayed around and we'd have a decent successor to WinNT 4 instead of that blight, Windows 2000.

And perhaps there'd be a single GetDiskFreeSpace function and even old die hard MS-DOS users would have considerably more free space on their disks than they have today.