The Registry

A sightseeing tour, with a few tips and suggestions for good maintenance programs.

Playing it Safe

Get It Try It

[Disclaimer: don't come crying to us if you do something to your system and things screw up. We assume no responsibility. Is that clear? Thank you. Now on to the good stuff. Ed.]

It is really important that you get the wherewithal and the guts to get into this thing. You should never go where you are unsure of yourself, and yet being a bit unsure is always par for the course here, and so you should know how to play it safe and practice it well too.

You need to know how to keep things backed up on your system, and know how to restore things when something bad does happen. On 9x this is easy - you have only two files to worry about: SYSTEM.DAT and USER.DAT. If you can't copy these while online, then boot twice into 9x without doing anything in between, check that the backup files SYSTEM.DA0 and USER.DA0 have in fact been re-written each time, and then copy them. Maybe create a special directory tree where you keep various backups of these files, together with various annotations, such as 'this was before I installed Office 2K', etc.

On NT it's a different matter and requires a bit more hop skip and jumping. The easiest way is to have mirrored NT setups. As you're probably going to use NTFS if you've got any sense, it will only be NT that can read into your file system, so you need something to start with. With mirrored setups you can go into either boot and manipulate the Registry files from the other boot.

Saved NT Registry files are compressed with the LZ algorithm, meaning the command line utility EXPAND will expand them back again. You have to specify the name of the target file in each case, but this is a no-brainer: the compressed saved files merely add an underscore as an extension and the uncompressed files have normally no extension at all.

Your saved compressed NT Registry files are (mostly) in the repair sub-directory to WINNT, and your online files are in the config subdirectory to system32. The files you want are default, sam, security, software, and system. system.alt is a copy of system, and you don't want the event files and log files. These will all be compressed and saved into your repair directory when you run the utility RDISK. It is important to back up your Registry often, so you have to copy these files in repair somewhere, and date them and annotate them so you remember what they mean.

You will also need your personal settings Registry file. This is always called ntuser.dat, but its location depends on who you are when you log on. RDISK will normally not touch this file, so you have to copy it all on your own. If you log on as Administrator, then it will be in your default Administrators profile directory tree, normally under Profiles which in turn is under WINNT. Just copy this file to a neutral location and put it with the other repair files you've assembled there.

But while you can manipulate the repair files at any time, you can't touch ntuser.dat while you're logged on. This file - and the other files in config - is locked down so tight even Bill Gates couldn't get at it. So for this exercise you have to have another boot to go into.

Likewise when you need to restore, you need a second boot to work from. Now you could in theory always work from your three diskette startup, but this is a very invisible way of working, and it takes a lot of time and grief to go through, and there is no flexibility at all.

When you want to restore a Registry on a mirrored partition, boot to its mirror and then expand the repair files. Use the proper names, and then move them into the target boot's config directory. If you're using NTFS compression, these files might come up as compressed, and you don't want that. It's no catastrophe if you forget, but it makes it easier on the system to de-compress them before you try a re-boot. And don't forget to put your own personal ntuser.dat in its proper place. The ntuser.dat that you found in your repair directory is another user file - this is the file for the default settings for the 'default user'. Don't mix the two up.

How Often?

How often should you back things up in your Registry? You should back it up every time you are about to attempt a major software install, and every time you are fairly sure you've succeeded with a major software install. If you have to backtrack later, you don't want to have to re-install a lot of stuff which is already working satisfactorily, and if you are unsure about a coming install (and who isn't), then you want a clean and surgically sound way of removing all the junk that the new install puts in your system - and simply restoring a Registry that was in use right before the suspect install is the best and most fail safe way.

Getting Around

It's time to take a look around in there. No matter what platform you're on, use the default Registry Editor (REGEDIT.EXE) for this one. Fire it up. If you are running NT and were smart and kept your NT 4 Beta 2 Registry Editor around, then you will have a key fewer than everyone else. The Registry Editor for the final release was the wrong one, and it was never corrected; also, this one is almost twice the size of its predecessor, and thus wastes a cool 50KB on your disk.

In either case, you should see a number of folders on the left, and they are:

HKEY_CLASSES_ROOT
HKEY_CURRENT_USER
HKEY_LOCAL_MACHINE
HKEY_USERS
HKEY_CURRENT_CONFIG

And, if you're running 9x or have the bugged REGEDIT for NT, you will see another key, HKEY_DYN_DATA, which we can forget for now.

The first five keys above are what we are going to concentrate on. And the first lesson about them is that three of them are highly unreal. In fact, the Registry has only two keys, and this division of labor makes a lot of sense.

The real keys are HKEY_LOCAL_MACHINE and HKEY_USERS, commonly abbreviated HKLM and HKU. The remaining three keys are shortcuts to sub-folders of these two. We'll start by expanding the two real keys and then looking at what the other three have to offer.

The division of labor in the Registry is between what is germane to the local machine and what is germane to the currently logged on user. Thus HKLM contains a bunch of machine-specific settings and HKU contains user-specific settings. You will never see anyone else's settings up there under HKU aside from your own and the default settings, but when it comes to HKLM you will see everything that applies to the entire machine and thus to any logged on user.

Keys and Values

Get It Try It

First a word on the structure of this Registry: it's very similar to an ordinary file system, and in fact follows a number of rules that Windows file systems use. For example, case is preserved but case-sensitive names are not supported. If you create something called sOfTwArE, that's the way it will look in the Registry; but if you try to create something else on the same level with the name SoFtWaRe, the system will burp.

In the context of the Registry we speak of keys, values, and data. Keys are the equivalent of file system directories; values are the equivalent of files; and data are the equivalent of the contents of files. Further, data stored in the Registry is always stored according to one of a set of pre-defined types. The Registry can be quite clever in this regard.

So the directories on the left are actually called keys, and what you see on the right at any one given level are the values. And when you double click on any value, you see that value's data. Easy peasy.

The system is hierarchical, and because it exists as a set of finite files on disk and not as separate files and directories for each level you encounter, it's going to get bloated from time to time. The analogy is OLE storages and streams: when data becomes too large for the block it is located in, the Registry must allocate a new block to store it. The old block may go unused for some time. This fragmentation is reflected in the ever increasing size of your Registry files on disk. One of the easiest ways to reduce disk storage is to simply save a Registry and restore it again. If the Registry was fragmented, this operation will cut it back down to size.

Anyway, you use the Registry Editor just like you would the Explorer. Directories/keys on the left, files/values on the right.

It is important to understand that anything you do here is changed immediately and there is no 'undo'. Curiously, the Windows 3 Registry Editor could accomplish this. But no matter. None of the current Registry editors can, so be careful. When in doubt, do not save, but hit Cancel instead.

Data is stored in a myriad of formats. Yes, you can save data as ordinary character strings, but you can also save data as multi-zero terminated strings (like argv, envp, and the common file dialog filter strings) and you can save strings as expandable strings as well. The Registry admits of a number of macros which, if you want, will be expanded when you read back the data.

Then there is the little endian DWORD format, the big endian DWORD format, the binary format, and a multitude of others which we don't have go to into right now. There's even a non-format format (REG_NONE), which can be used interchangeably with the binary format.

Peeking Closer

Get It Try It

Let's open HKEY_LOCAL_MACHINE. What we see are the following sub-keys:

HARDWARE
SAM
SECURITY
SOFTWARE
SYSTEM

Some of these keys will not be seen on 9x; no matter. HARDWARE is a so-called volatile key: its contents are not written to disk. On NT, this is the result of the system polls by NTDETECT at system boot. SAM and SECURITY are security keys. SAM stands for Security Account Manager. Normally these keys are closed to even Administrators, but there are ways around everything. SOFTWARE is the root level for all machine-specific software settings. If you take a peek inside, you'll see the names of several vendors. Finally, SYSTEM is where at least NT keeps its service and driver information.

Associations

Get It Try It

Open the SOFTWARE key and when there, look for the subkey Classes and open it. You should see a long list of extensions. If your Registry is not in shape, merely opening this key can take considerable time and race your CPU - meaning it's probably best to start cleaning as soon as possible.

Extensions are half the way to determining what application runs a file. When you double click on a file, the shell will start here first. It will try to find the extension that your file has. Let's take a universal extension to work with here: go down to the subkey .hlp. This is the key the shell would search for if you'd just double clicked a WinHelp file.

Now notice the value on the right. It has no name, it's called only 'default', and its data should be 'hlpfile'. Fine. Make a note of this and prepare for the next step.

What the shell would do now if it was attempting to open a file with the extension .hlp would be search for the key 'hlpfile'. File extensions can thus be grouped according to 'file type': several different file extensions can all resolve to the same file type and associated application. From where you are right now in the Registry, invoke the Find dialog and search for hlpfile.

There should be at least one, perhaps two subkeys to hlpfile. In addition, you will see that the key hlpfile itself has a default value, probably 'Help File': this is the 'pretty name' given files of this type in the Explorer. Under hlpfile you might see the subkey DefaultIcon: this is the icon used for display in the shell. The number at the end, after the comma, is an absolute index into the designated file, but if it is a negative value, then the number is an absolute identifier instead. You can generally leave these values alone.

The other subkey should be a directory tree: shell/open/command. You might have further branches on your system, such as shell/print/command and so forth. The shell subkey denotes that this is something the shell needs to know to activate files; the open or print subkey is the particular action requested; and the command key is one of a number of (at least two) alternatives for opening the file in question. This command key will have a complete command line for launching the associated application from scratch and getting at the file you double clicked. A comparative key, ddeexec, is used for opening files in applications that have already been launched.

The default value for command will be something like '%SystemRoot%\System32\winhlp32.exe %1'. The % signs enclose Registry macros. In this case, %SystemRoot% refers of course to the OS setup directory. The curious %1 at the end denotes 'the command line' or whatever it was you double clicked.

So that's the procedure when you double click something: the shell goes into the Registry, searches for the file's extension, gets the default value at that key, then searches for that default value, then climbs down the shell/open/command sub-tree, and plucks out the command line found there. Like taking candy from a baby.

Lesson one: if you have file types which lead to programs on disk which you don't have or no longer have, these file types are simply cluttering your Registry. Lesson Two: if you have extensions which resolve to file types which no longer exist - you are cluttering your Registry.

ShellNew

What happens when you right click on your desktop? You get a little menu with the 'New' submenu on it. How big is this submenu? Do you really want all that junk on your 'New' menu? If you don't, here's how to get rid of it: you simply go to the top of HKLM/Software/Classes and start searching for 'ShellNew'. You'll find these little monsters right under extensions or maybe file types. Check out what they mean, and if you don't want them around anymore, just delete them. One of the most important lessons you're going to learn as you use a PC is that everybody and his brother wants to get onto your desktop, onto your Start menu, and even on this menu. Really, it's mostly a big pain. So get rid of the entries you don't like. The effect is instantaneous; this means yes, every time you right click on your desktop and select New, the shell is actually going through your entire Registry and looking for these little suckers. It also means that as soon as they are deleted from the Registry, they disappear from your desktop context menu: you don't have to log in again or re-boot to have the changes take effect.

Globally Unique Identifiers

Get It Try It

If you want a real treat, go back up to HKLM/Classes and find the CLSID subkey. Now carefully open it, and have a brown paper bag handy. This mess is the OLE/ActiveX sub-tree. This is where all those wonderful re-usable components reside. They are identified by a 128 bit globally unique identifier (outside Windows they're more often called universally unique identifiers, so for once Microsoft has been humble) which is expressed in text form, complete with cute dashes here and there, and enclosed in curly braces. This subkey is almost bottomless. Every time you have a bit of shoddy software that makes its way into your system, the odds are that it's planted something here, something you don't need and something which will only slow down your system. Getting rid of bad CLSID's is not that simple, but fortunately there are tools which can take care of this (and they don't include RegClean - just hold on).

Think you've seen enough? Go from CLSID down to Interface and open this one up the same way. On most systems, this one is even bigger. Again, it's a lot of OLE/ActiveX junk, and it can't be cleaned manually, not unless you have great stamina. Again, it's best left to dedicated tools.

It's very important to not be overwhelmed by 'holy wrath' when you gaze upon this bloat: a single indiscreet move, although understandable, might render your system useless. Contain your anger, if you feel any, and wait for the presentation of the right tools to clean unnecessary junk out.

But it is important to realize just what a burden these two keys put on your system: firstly, they of course are a major cause of Registry bloat; but secondly, and almost more importantly, they are a reason your system might come to a grinding halt.

The wonderful Windows shell watches everything you do while you're logged on and is continually monitoring your activities on disk and in the Registry. So all that processor power is getting eaten up by something after all. Just for the heck of it, you might try temporarily moving a file mentioned under CLSID or Interface to a new directory - or you don't have to if you don't want to, you can take our word for what happens: the shell should automatically change the data in the Registry for the location of the file.

Meaning it is important to cut the size of these keys down as much as is possible. You cannot cut corners here: what is there and being used must remain; but you can use the proper tools to get rid of the stuff that is not being used, that is just lying around and clogging and bloating things. When you've gutted your Registry in this fashion, and if you thereafter back up and restore your Registry files, you should note not only that the files are smaller, but that your system runs a bit faster.

The Tools

Get It Try It

The tool to use is not RegClean, but RegMaid, and it too comes from Microsoft. Unlike its cousin, it is not a megabyte download, and it works more than satisfactorily on all platforms. The difference here is that it's a lot more effective and thorough, that you have to know what you are doing, and that you have to read this first so you know how to clean up after it - even RegMaid is incomplete and flawed.

Start by going to Microsoft's site and finding it. It's always there, but they change the download URL all the time. When you open up the ZIP file (it's a ZIP even if it has the EXE extension) you will notice that the source code is included. You don't need that for this exercise. All you need is the EXE itself and its help file.

Read the help file thoroughly first so you have a good idea what you're up to. The help file explains in detail not only how to use the program, but how the program works - a very good move on the authors' part. There are four buttons on the toolbar, corresponding to 'levels of cleanliness'. As you click through them, carefully, from left to right, you eliminate 'junk' at each step - isolating and singling out even more junk at the next step, etc.

RegMaid will attempt to hook up to all the registered components in your system - if it fails for any CLSID or Interface etc., it will list the orphan in its output window. It is here you must be very careful. Some programs don't supply complete paths for their components, because they know already where to look; but RegMaid will not know of this secret location. Thus it makes good sense to not delete anything which does not have a fully qualified path. For those entries that do have fully qualified paths and are declared by RegMaid as missing - delete them. Hit Refresh and then move on to the next button. Then do the same thing. As you move farther right, more and more junk will be isolated. Repeat the process as many times as you want, always beginning from the far left and moving to the right, until you are satisfied you've got the lot. Then exit RegMaid.

Now go into your Registry with your Registry Editor and put your selection in the left (key) pane on HKLM and search for the string 'Maid'. Have the editor look at keys, values and data, and not search for whole string only. Every instance you find of the string 'Maid' should refer to the program you just ran. Yes, RegMaid is far from perfect. So just delete the RegMaid entries you find. Note: you cannot search for 'RegMaid', you must search for 'Maid', and if you want to know why, just do as you're told and you'll find out on your own in not too long.

Oh - hopefully you've backed up your Registry before running this program?

RegMaid will not eradicate all junk in your Registry - but it will get at a lot. If you open HKLM/Software and detect subkeys there that refer to software you used to but no longer have installed, you might delete these keys as well. Rather than backing up your entire Registry just for this, you might export the key you are about to delete before deleting it. That way if something goes wrong you can just double click the REG file to get everything back in there again.

HKEY_USERS

Get It Try It

You'll also find vendor-specific information under HKEY_USERS, but to get to it you'll have to pass through two other subkeys on the way. One is always called .DEFAULT, and that is for 'default users', and the other one has an unpronounceable name: that's you. That's your SID, or security identifier. Open it up.

The first subkey you find is probably AppEvents. This is a directory tree for sound schemes. We can pass over this one.

The next subkey is probably Console. This key contains settings for your consoles - non-GUI apps and what is often called by that horrible epithet 'DOS Prompt'.

The next subkey is larger and more significant. This is Control Panel. This is where you'll find all your settings from Control Panel. Normally for any changes you make here to take effect, you'll have to either log on again or re-boot the system.

Accessibility is for the hard of seeing; Appearance is just what it sounds like; Colors are the all-important color settings for your desktop components, stored in RGB values; Current refers to your current desktop color scheme; Custom Colors are used in the Color dialogs down on the lower left; International has your 'regional' settings for date/time and currency format; Mouse has mouse settings; Patterns are the desktop background patterns you see in your desktop property sheet (not a good idea to use); Sound turns sounds on and off in your system; and Sounds defines which sound scheme, if any, is in effect.

We jumped over the subkey Desktop because it's bigger and more important. Most of the values here, and elsewhere under Control Panel, can be adjusted directly from the corresponding applet. One that cannot is MenuShowDelay. This is the delay, in milliseconds, between when you select something on the Start menu with a submenu of its own and when the system will actually display that submenu. You can try it if you like, but you'll see why a delay is necessary. 400 milliseconds is the normal default here: you might want to speed it up a bit, but anything lower than 200 milliseconds will make you dizzy and annoyed in no time flat.

The ScreenSaveTimeOut value is the delay in seconds before an idle system launches its screen saver. You can have a bit of fun here: if you ever come across an unattended machine (hopefully belonging to someone who is a friend and will appreciate the joke) you can go in here in their Registry and change this value to something very very low - like 10 or less. For even more 'fun' you can change the corresponding value under .DEFAULT. That way even ten seconds of idleness will launch the screen saver - making it all but impossible to use the system. As for how you correct this 'joke' - we'll leave that up to you!

HKU/You/Software

Get It Try It

This is where your application specific settings will be found. The standard is to start with the vendor name at the next level, immediately followed by the product name, followed perhaps by the version number, and finally the settings themselves.

Unfortunately many ISVs don't do a good job of saving settings here, and here's a bit of the historical background:

Many applications are actually legacy software from the days of Win16 when the only opportunity for saving settings was with INI files. Of course, a vendor could have been creative and used their own format with their own files, but that would have been perhaps too much to ask. Most used the INI file system because Microsoft told them to do so.

INI file technology is severely limited. For one, no INI file can be larger than 64KB, as the BBC found out to its dismay when they'd put their entire teleprompter system through this technology. For another, INI files are pure text files. You cannot store null bytes. So everything is either a string or something that can represent a number but in string form. Not highly versatile.

And to get at all this data (they were called entries and the entries had values - a highly confusing terminology today) you had dedicated functions provided by the operating system (provided by Microsoft) to get or set only one of these at a time. The amount of code needed to do something simple like save the desktop coordinates of an application window was a tad ridiculous. You had to get the coordinates of course, then you had to use a formatting function to turn them into a string, and when you got them back on application startup you had even more work cut out for you: you had to actually parse the string. Or maybe you saved each one of the four coordinates as a unique value: less work for you now (in a way) but so much more later.

With the advent of the Registry one would assume all these vendors would heave a sigh of relief and spend at least one afternoon to work over their code and get this lugubrious stuff out of there. But no. They simply replaced each INI file I/O call with a corresponding Registry I/O call, which is about as wasteful as you can get. Creating a 'section' in an INI file has almost no overhead at all; but creating a new key in the Registry has an astounding overhead.

To make matters worse (or better, depending on who you are), the Microsoft Foundation Classes framework supports (even today, yes, more than five years after INI files were supposed to be gone forever) the wrapping of archaic INI file I/O calls to the Registry. Note that these only work if you only use the string format in the Registry and only if you only store one data bit per value. And you can't get much more inefficient than that.

Finally, the golden rule is that one should never store more than 2KB at any one key in the Registry, but just check and see how many vendors break this rule. And some of the settings are close to the ridiculous: most applications should be able to run without anything stored there. In such case they will use their 'factory defaults'. And these default settings will be adequate to get the application running smoothly. Why then, if you the user don't explicitly change any of these settings, do these applications insist on storing what is only their factory defaults in there? Any number of Microsoft system applets (Notepad, Paint, et al.) are guilty of this crime. It not only results in an unnecessarily bloated Registry and more sluggish system, but application startups will also suffer. Why then do they do this? The only reason (if it can be called that) is that the programmers were too lazy to keep track of whether you actually changed something or not. And as these apps still use INI file technology, they could save only those values you had changed. The overhead would be more than compensated by the savings on disk and the more smooth running of your apps and your system in general. But they don't do this, they don't do what is common sense and desirable, so 'that's just that'.

The Other Three

Get It Try It

There were at least five keys at the top level in the Registry, yet we've only covered two. What are the others?

They're shortcuts. Open your Registry Editor again, go down to HKLM/Software/Classes, take a long last look, and then go up to HKEY_CLASSES_ROOT (HKCR) and open that one. Look familiar? HKCR is just a shortcut into HKLM/Software/Classes. It's not the same data twice; it's the same data once.

Now go down to HKU/You and open that one and take a good look, then come up and open HKEY_CURRENT_USER (HKCU). Look familiar again? Yes, the real key for your personal logon settings is at HKU/You, but HKCU is a shortcut to those settings. Again, it's not the same data twice; it's just a shortcut.

What does this mean? Well, to a programmer it means that the character strings needed to access certain parts of the Registry don't have to be as long as one would initially believe; to a historian it means that this is why we only saw HKEY_CLASSES_ROOT with Windows 3; and to a user it means that you should never start a search above HKEY_LOCAL_MACHINE: if you do, you'll just be searching through the Registry twice and taking much more time to do it.

As to where HKEY_CURRENT_CONFIG leads, we'll leave that as an exercise. Hint: it's under HKLM.