About | Buy Stuff | News | Products | Rants | Search | Security
Home » Resources » Software » Reviews » The Evidence Eliminator Documents

About Shredding

Vital to the application domain of a computer security utility are the existence of sensitive data you knew nothing about, and the ability to not only delete this data but shred it. Deleting data is not the same thing as shredding it, just as tossing a hardcopy document into the dust bin is not the same thing as destroying it. This article will look at what shredding is and how it works.

It's Still There


Get It

Try It

Deleting files and directories from your hard drive does not remove their contents. Deleting them only makes them inaccessible. With time new files and directories will fill the disk areas they occupied, but even then it's possible to get at 'what's underneath'. Even wiping a disk completely clean by submitting it to a hard format will not protect it.

Data storage on disk is all a series of 0's and 1's. Disk controllers use a bit of lenience when interpreting what is there. A 0 represents no magnetic charge at all; a 1 represents a full magnetic charge.

Over time, as the disk is used again and again, the proximity of the 0's and 1's to absolute values of 0 and full magnetic charge will decrease accordingly. If a 0 bit is overwritten with a 1 the new magnetic charge will not be full but only a portion thereof, e.g. 95%. This is still good enough for the disk controller to understand, but it simultaneously reveals what the previous value was.

If the bit in question is supposed to be a 1 but only has 90% of the full magnetic charge, then it might be assumed that there were two 0 bits there and two 1 bits as well. And so forth.

Forensic analysis software uses these facts to construct complete previous layers on a disk - and successfully as well it must be pointed out. What seems to be a completely clean disk can in fact hide a wealth of information.

It's Almost Too Easy

Yet such software need not always dig so deep: Utilities such as EnCase from Guidance Software merely look at everything on a disk - and construct 'cases' from this alone. For there are thousands of places where compromising data can be found - even after a file is deleted or even shredded:

  • The file name itself. This can often be a dead giveaway.
  • Slack in OLE2 storages. OLE2 disk writes are more efficient if they do not have to reconstruct their storages and streams for each update, but this can lead to fragmentation and thereby slack.
  • Cluster slack. Everything is allocated in clusters - a number of disk sectors. A single 300 byte shortcut will consume at least 512 bytes under NTFS and quite a bit more (32KB was not uncommon not too long ago) under FAT16. The driver won't bother wiping the rest of the cluster, as the file size gets stored in the directory entry and the driver won't read beyond it.
  • MBR slack. Yes, naive hackers sometimes use minimal storage opportunities here - in the very first sector on the drive.
  • Boot track slack. Wintel boxes don't use but one sector on the first track - for the MBR - and then skip to the first sector on the next track for the first partition sector. Again, naive hackers often try to hide things here as well.
  • Journal files. Sent by the OS to the printer. Deleted immediately the print operation is complete.
  • Paging files. On NT, everything that is in memory was first on disk - and if it was edited before being flushed back, it might have ended up in a dedicated swap file. Likewise, data sections in executables will 'copy on write' to a swap file once their contents need be changed. On 9x, anything can happen.
  • Undo files. Some editors - text, graphics, the scope is unlimited - create temporary files to store undo and redo information - or just a copy of the file currently being edited. Once the session is over, these files will be deleted.
  • The Windows Registry. URLs, MRU lists et al.

How To Make It Go Away

The objectives of computer security software are therefore twofold:

  1. Delete compromising data not only in files, but on the disk itself.

  2. Shred what was previously on the disk to the point that forensic analysis software and even more advanced data recovery utilities will not be able to recover it.

NISPOM

The National Industrial Security Program Operating Manual from January 1995 contains guidelines which are often quoted in this context. NISPOM takes all possible media into account, even seeing the danger in disposing of used cathode ray tubes in an insecure fashion. Of particular interest is Chapter 8 - Automated Information System Security. Section 8-306 ('Maintenance') contains a 'Clearing and Sanitization Matrix'. This matrix deals with magnetic tape, magnetic disk, optical disk, memory, printers and cathode ray tubes. The sanitization methods range from overwriting to degaussing and complete destruction.

For example, NISPOM recommends performing an ultra-violet erase on EPROM; cleaning platens then destroying ribbons and finally removing all power including batteries for printers; and as long as the medium has not housed anything above the 'secret' level, a file shredding technique for hard drives.

d. Overwrite all addressable locations with a character, its complement, then a random character and verify. THIS METHOD IS NOT APPROVED FOR SANITIZING MEDIA THAT CONTAINS TOP SECRET INFORMATION.

If the drive has been used for more highly classified data storage, Type I or Type II degaussing or 'total destruction' must be used.

m. Destroy - Disintegrate, incinerate, pulverize, shred, or melt.

As such, NISPOM does not attempt any advanced shredding for hard drives, the most accepted method still being 'total destruction'.

GUTMANN

Dr. Peter Gutmann's 'Secure Deletion of Data from Magnetic and Solid-State Memory' was published a year and a half after NISPOM and takes a dim view of that US government document:

'There are two main problems with these official guidelines for sanitizing media. The first is that they are often somewhat old and may predate newer techniques for both recording data on the media and for recovering the recorded data. For example most of the current guidelines on sanitizing magnetic media predate the early-90's jump in recording densities, the adoption of sophisticated channel coding techniques such as PRML, the use of magnetic force microscopy for the analysis of magnetic media, and recent studies of certain properties of magnetic media recording such as the behaviour of erase bands.

'The second problem with official data destruction standards is that the information in them may be partially inaccurate in an attempt to fool opposing intelligence agencies (which is probably why a great many guidelines on sanitizing media are classified). By deliberately under-stating the requirements for media sanitization in publicly-available guides, intelligence agencies can preserve their information-gathering capabilities while at the same time protecting their own data using classified techniques.'

According to Dr. Gutmann, shredding must necessarily take into account the media encoding scheme used. At present there are three different encoding schemes in use: run length limited (RLL) (1,7) and (2,7) coding, and modified FM (MFM). Understanding how these schemes work may be beyond the scope of this article, but understanding that shredding must take them into account is not.

Dr. Gutmann has come up with a series of 35 (THIRTY FIVE) steps which should be performed if data is to be truly secured. Shredding both begins and ends with four completely separate overwrites with random data which is randomised anew for each step. The innermost 27 steps all target one or more of the three encoding schemes.

Also, the order of these innermost 27 steps must be randomised, and the randomisation methods used to create both this sequence and also the data used in the other eight steps must be adequately obfuscated.

Getting It To Disk

Further, vital to any approach is the ability to actually flush data to disk after each and every overwrite. Modern operating systems such as Microsoft Windows will hold written data in cache and delay file writes until an appropriate time. Shredding must circumvent this 'enhancement' and ensure that all writes are in fact flushed to disk for each step of the shred. The risk is otherwise that only the very last write performed will actually be placed on the media - thus defeating the whole purpose of the operation.

Is It Really Gone?

The EE approach to deleting is spelled out in its progress report window and in its log file (see below). It is very thorough.

The EE approach to shredding is spelled out in numerous places:

  1. Write all 0's to disk.
  2. Write all 1's to disk.
  3. Write 'random data' to disk - at least once and at most nine times.

This is indeed a DoD method and does come from NISPOM, but it is intended for use only with electronically erasable programmable ROM (EEPROM) - and not with hard drives.

Dr. Gutmann comments:

The recovery of at least one or two layers of overwritten data isn't too hard to perform by reading the signal from the analog head electronics with a high-quality digital sampling oscilloscope, downloading the sampled waveform to a PC, and analysing it in software to recover the previously recorded signal. What the software does is generate an 'ideal' read signal and subtract it from what was actually read, leaving as the difference the remnant of the previous signal.

The 'EEPROM' method might defeat simpler forensic analysis utilities such as EnCase, but it is not in accordance with the DoD recommendations for sanitization of hard drives - much less those of Dr. Gutmann - and it will not defeat the more advanced data recovery utilities in use today.

Prev | TOC | Next

About | Buy | News | Products | Rants | Search | Security
Copyright © Radsoft. All rights reserved.