Disk over writing software

Subba Rao · External Since: Oct 14, 2004 Posts: 6

Hi,

I am looking for some software that will over write the files on the HD
several times over. This software should meet the DOD security
requirement of cleaning up the data on the hard disk.

Thank you in advance for any help.
--
SR
castellan2004-mail.DeleteThis@SPAMBUSTER.yahoo.com
Please remove SPAMBUSTER to reply via email.

Unruh · External Since: May 27, 2005 Posts: 2213

Subba Rao writes:

>Hi,

>I am looking for some software that will over write the files on the HD
>several times over. This software should meet the DOD security
>requirement of cleaning up the data on the hard disk.

There is no such requirement.
HOwever,
dd if=/dev/urandom of=/dev/hdc
will do it
Repeat 3 times.
If that is not enough, open the disk and take a blow torch to the surfaces,
heating all platter to red hot.
The above assumes you want to overwrite the whole disk. If you onlyuwant a
partition, put in the partition number instead
/dev/hdc5
say
A single file is too dangerous as the file location can jump around and the
file can have stuff at the end of it.
But if you want
dd if=/dev/urandom of=/name/of/the/file
This will overwrite the current file and then keep growing the file until
it covers the whole freee space on the disk.
>Thank you in advance for any help.
>--
>SR
>castellan2004-mail@SPAMBUSTER.yahoo.com
>Please remove SPAMBUSTER to reply via email.

Ertugrul Soeylemez · External Since: Nov 03, 2005 Posts: 271

Subba Rao (06-03-05 19:04:16):

> I am looking for some software that will over write the files on the
> HD several times over. This software should meet the DOD security
> requirement of cleaning up the data on the hard disk.

Important: Don't use any programs related to security, which are not
widely known. Don't use any programs related to security, for which
there is no source code publicly available. Especially forget GUI
programs, as they tend to substitute security by comfort.

There is a (not so) little command line program especially designed for
this purpose. It's called 'shred' and is preinstalled on almost any
non-Windows system. Its main intention is to wipe out entire data
storage devices. By default it overwrites the device 25 times, which
should be enough to make data effectively physically unrecoverable. If
you believe that this is not enough, you may increase this amount. But
be prepared to wait quite a while for it to finish.

To make data unrecoverable on magnetic storage, it's not enough to
overwrite it with purely random data (e.g. /dev/urandom output). This
does not only take forever, it's not even secure. Shred uses special
patterns to overwrite the storage. By calling it with the -v option,
you can see those patterns.

If you want to securely delete your entire /dev/hda, it's enough to
type:

# shred /dev/hda

To overwrite 45 times instead of 25, do this:

# shred -n45 /dev/hda

Making data unrecoverable is not enough in all cases. Sometimes it's
good to make the hard-disk look like it never had any previous
filesystem, which might be interesting. You may want to add a final
overwrite with zeros. You can do this with the -z option:

# shred -z /dev/hda

This is intended to trick people inspecting your hard-disk. But
personally I would classify a hard-disk filled with zeros as very
suspicious. Don't do this. Instead, read on.

After the shred command returns, I really recommend creating a new
partition table and install a fresh operating system. This confuses
possible attackers, as they expected an 'empty' hard-disk.

Sometimes you even need to be more smart. Sometimes it isn't even good
enough to make the hard-disk look innocuous. If you really want to
trick most people, then I would even create an encrypted filesystem with
some random key, and fill it with one large file generated by /dev/zero
or /dev/urandom (which one really doesn't matter for an encrypted
filesystem -- use /dev/zero, it's a lot faster).

An attacker will find that encrypted filesystem and try to break the
encryption, possibly making the 'real' data harder to recover with each
try.

Regards.

Chris Cox · External Since: Apr 05, 2004 Posts: 408

Subba Rao wrote:
> Hi,
>
> I am looking for some software that will over write the files on the HD
> several times over. This software should meet the DOD security
> requirement of cleaning up the data on the hard disk.
>
> Thank you in advance for any help.

See others for some reasonable answers to you question...

I just want to suggest a different strategy. Either wipe (with whatever
software) and vault the drives (the fact that you are trying to
reuse the drives means they have value to you) or physically
destroy the drive (if the drive is going away from your organization).

My point is that data forensics may get better and media changes
might make the "current" overwriting process void for future media types.
If the drive is going into a place where you feel it might
be examined... destroy the drive (so it goes nowhere). If you feel
you "need" to get the value out of the drive, then wipe it and put it
into your own protected storage for use later.

Moe Trin · External Since: Aug 12, 2004 Posts: 1733

On Sun, 05 Mar 2006, in the Usenet newsgroup comp.os.linux.security, in article
, Subba Rao wrote:

[Followup set - if it belongs on comp.os.linux.security, it does NOT belong
in comp.os.linux.misc]

>I am looking for some software that will over write the files on the HD
>several times over. This software should meet the DOD security
>requirement of cleaning up the data on the hard disk.

If this is an official requirement, talk to the Contracting Officer's
Representative and find out EXACTLY what the requirements are and what
approved software may be available. Don't do ANYTHING without getting
that approval.

If you merely wish to get something that sounds great, google for '5220.22-M'
which is the (US) DoD number for the "National Industrial Security Program
Operating Manual" (NISPOM) ISBN 0-16-045560-X. Briefly, to sanitize a
non-removable rigid drive, that the choices of degaussing, destruction of the
drive, or a 3 pass wipe are acceptable methods for disk sanitation. Note that
the 3 pass wipe method is NOT acceptable for drives that contained Top Secret
information.

It should be noted that this issue has been done to death on bugtraq several
times.

As for actual software needed to overwrite the disk (overwriting files
ALONE does not satisfy the DoD requirement), you likely have all that is
needed in '/bin/dd', '/dev/zero' and '/dev/urandom'.

Old guy

Lawrence D'Oliveiro · External Since: Oct 29, 2004 Posts: 669

In article ,
Ertugrul Soeylemez wrote:

>To make data unrecoverable on magnetic storage, it's not enough to
>overwrite it with purely random data (e.g. /dev/urandom output). This
>does not only take forever, it's not even secure. Shred uses special
>patterns to overwrite the storage.

This doesn't agree with the Peter Gutmann paper, if I recall rightly.

Moe Trin · External Since: Aug 12, 2004 Posts: 1733

Followup-To: set to comp.os.linux.security.

On Mon, 17 Apr 2006, in the Usenet newsgroup comp.os.linux.security, in article
, Lawrence D'Oliveiro wrote:

>>To make data unrecoverable on magnetic storage, it's not enough to
>>overwrite it with purely random data (e.g. /dev/urandom output). This
>>does not only take forever, it's not even secure. Shred uses special
>>patterns to overwrite the storage.
>
>This doesn't agree with the Peter Gutmann paper, if I recall rightly.

The Gutmann paper is nearly ten years old - think there might have been
some changes in technology in that time? See 'Putting the "NSA Data
Overwrite Standard" Legend to Death...' a thread on Bugtraq on 4/5 Feb 2003.

Old guy

Matthias Kirchhart · External Since: Apr 10, 2006 Posts: 7

-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160

Lawrence D'Oliveiro wrote:

> In article ,
> Ertugrul Soeylemez wrote:
>
>>To make data unrecoverable on magnetic storage, it's not enough to
>>overwrite it with purely random data (e.g. /dev/urandom output). This
>>does not only take forever, it's not even secure. Shred uses special
>>patterns to overwrite the storage.
>
> This doesn't agree with the Peter Gutmann paper, if I recall rightly.

If you *really* need to destroy the data on a tape, wouldn't it be easier
just to throw it into a fire until it is just a small pile of fluid
plastic? Smile

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iQIVAwUBREPlqdqtd8S+cgRiAQOYew/+KevAtU3S0pdzZ4Ib4uL9Fa4Er7WG6wUU
oRuxWtgyjXdA9D1uvrq1LmN9l6/M7//nD615Pg6ea0OfNcNT/uhghquAWR5uxuk4
FHinABP13VKEemCgGfvoZmZ7CekULkLClQj7CxX8B0hbS3RpzagA/m5ne1FANvpi
zMmRaPcKuddYPYLWp4KtDlJSx0Og2dT9XcDoqrvt6sJ3wszPJ+CPP2wP1yi/ok9M
P86z+uBSQ0XHQhIbXPhm2AO6NOv5qTLaLRJ77E8XLg8LpO9xawU/c3mDbMT1Sxjx
aEY0s0Ws68EmcIugoO7C/NRmkTYgMEK/Hb3nQEN82Dz2A7kdCOChICfBj8xM9bhN
J5JvM/b9vL/lqjZpDvfNCIvI7SVQGTskMRmD8TmC4F3pmtrmlojF28j3kqXjocm+
eFC3+NZxkwRYOjg4abfZ3Pn4hTxbfarzPjKfnEKBlk7ns5Fiu462RWseAAl+V96j
1QKxEOn6RwGeZBDDYjbqi12xGxf6yodb9hyNwauePCHrcWgDOl4cmq1gmP9bXIEM
7G2YeCiHiyT+oM//pGAmpNb1TruWOwD82uAvNqkkPNPdePDZjvlVVUeSfvSWU8dS
udSKB9e1PxY53Oy/Q+DvAYhwHUOaG28SCA0DJoDKcDAikwLyR9gyras14IxNIyjF
BSz9Vd1ByyU=
=UHaT
-----END PGP SIGNATURE-----

Unruh · External Since: May 27, 2005 Posts: 2213

Matthias Kirchhart writes:

>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: RIPEMD160

>Lawrence D'Oliveiro wrote:

>> In article ,
>> Ertugrul Soeylemez wrote:
>>
>>>To make data unrecoverable on magnetic storage, it's not enough to
>>>overwrite it with purely random data (e.g. /dev/urandom output). This
>>>does not only take forever, it's not even secure. Shred uses special
>>>patterns to overwrite the storage.
>>
>> This doesn't agree with the Peter Gutmann paper, if I recall rightly.

>If you *really* need to destroy the data on a tape, wouldn't it be easier
>just to throw it into a fire until it is just a small pile of fluid

Agreed. Or hard disks.

Bill Davidsen · External Since: Jan 07, 2005 Posts: 321

Moe Trin wrote:
> Followup-To: set to comp.os.linux.security.
>
> On Mon, 17 Apr 2006, in the Usenet newsgroup comp.os.linux.security, in article
> , Lawrence D'Oliveiro wrote:
>
>>> To make data unrecoverable on magnetic storage, it's not enough to
>>> overwrite it with purely random data (e.g. /dev/urandom output). This
>>> does not only take forever, it's not even secure. Shred uses special
>>> patterns to overwrite the storage.
>> This doesn't agree with the Peter Gutmann paper, if I recall rightly.
>
> The Gutmann paper is nearly ten years old - think there might have been
> some changes in technology in that time? See 'Putting the "NSA Data
> Overwrite Standard" Legend to Death...' a thread on Bugtraq on 4/5 Feb 2003.
>
> Old guy

A lot depends on who you need to exclude. It takes a lot to stop a TLA
(three letter agency) but a few passes will prevent the local salvage
guy from getting at the data. I use DBAN (ask Google) in one of the
medium security modes, that pretty much blocks anyone who doesn't have
high tech.

--
-bill davidsen (davidsen@tmr.com)
"The secret to procrastination is to put things off until the
last possible moment - but no longer" -me

Moe Trin · External Since: Aug 12, 2004 Posts: 1733

On Fri, 21 Apr 2006, in the Usenet newsgroup comp.os.linux.security, in article
, Bill Davidsen wrote:

>Moe Trin wrote:

>> The Gutmann paper is nearly ten years old - think there might have been
>> some changes in technology in that time? See 'Putting the "NSA Data
>> Overwrite Standard" Legend to Death...' a thread on Bugtraq on 4/5 Feb 2003.

>A lot depends on who you need to exclude. It takes a lot to stop a TLA
>(three letter agency) but a few passes will prevent the local salvage
>guy from getting at the data.

One thing people often forget is defective blocks on the media that modern
drives "auto-fix". What they do is copy the data that had been on the block
(if at all possible) to a "spare" block reserved for this purpose. The
substitution is transparent - done by the drive firmware, and not under
O/S control. This means that some blocks may still exist with old data
even after a thousand writes with random data. For common users, the data
is inaccessible, as you need the manufacturers software to reset the bad
and spare block table to access those blocks (which may or may not still
be usable). This may or may not be an acceptable level of risk. This is
the reason that NISPOM requires physical destruction of media that contained
"Top Secret" data.

None the less, do a search on the Bugtraq archive for a thread titled
"Peter Gutmann data deletion theaory?" (typo is in original) from July 2005.
"Simple Nomad" (a name that may ring bells if you are in security) wrote

"The quote is from 1996. I spoke with Guttman about this at AusCERT a
few years ago and even *he* doesn't believe it anymore. Drive technology
has changed substantially since then."

>I use DBAN (ask Google) in one of the medium security modes, that pretty
>much blocks anyone who doesn't have high tech.

The program has been mentioned several times. Multiple pass overwrites with
semi-random data are usually adequate - people forget that the ones and
zeros are not written directly to disk, but a density improvement and data
management scheme called Run Length Limited writes a different pattern to
allow packing those bits in tightly.

Old guy

Ertugrul Soeylemez · External Since: Nov 03, 2005 Posts: 271

ibuprofin RemoveThis @painkiller.example.tld (Moe Trin) (06-04-22 17:26:37):

> One thing people often forget is defective blocks on the media that
> modern drives "auto-fix". What they do is copy the data that had been
> on the block (if at all possible) to a "spare" block reserved for this
> purpose. The substitution is transparent - done by the drive firmware,
> and not under O/S control. This means that some blocks may still exist
> with old data even after a thousand writes with random data. For
> common users, the data is inaccessible, as you need the manufacturers
> software to reset the bad and spare block table to access those blocks
> (which may or may not still be usable). This may or may not be an
> acceptable level of risk. This is the reason that NISPOM requires
> physical destruction of media that contained "Top Secret" data.

That's why I prefer encrypting my hard-disk instead of being concerned
about proper disposition. Using a secure key, I could lose it or throw
it away without having concerns.

Luckily proper encryption cannot be distinguished from random noise. So
an attacker might even think that I've overwritten it that way, trying
to hide data stored on the disk previously. My current hard-disk has
never received a single bit of clear-text.

By the way, is there any way to query the number of such defective
blocks for IDE drives? I don't seem to find a suitable hdparm-option.

Regards.

Grant · External Since: Jan 04, 2006 Posts: 622

On Tue, 25 Apr 2006 06:36:50 +0200, Ertugrul Soeylemez wrote:

>By the way, is there any way to query the number of such defective
>blocks for IDE drives? I don't seem to find a suitable hdparm-option.

smartctl ( Cool

- Control and Monitor Utility for SMART Disks

Grant.
--
Memory fault -- brain fried

Moe Trin · External Since: Aug 12, 2004 Posts: 1733

On Tue, 25 Apr 2006, in the Usenet newsgroup comp.os.linux.security, in article
, Ertugrul Soeylemez wrote:

>That's why I prefer encrypting my hard-disk instead of being concerned
>about proper disposition. Using a secure key, I could lose it or throw
>it away without having concerns.
>
>Luckily proper encryption cannot be distinguished from random noise. So
>an attacker might even think that I've overwritten it that way, trying
>to hide data stored on the disk previously. My current hard-disk has
>never received a single bit of clear-text.

Not using a swap partition?

Old guy

Loki Harfagr · External Since: Oct 14, 2004 Posts: 286

Le Tue, 25 Apr 2006 06:36:50 +0200, Ertugrul Soeylemez a écrit :

> ibuprofin DeleteThis @painkiller.example.tld (Moe Trin) (06-04-22 17:26:37):
>
>> One thing people often forget is defective blocks on the media that
>> modern drives "auto-fix". What they do is copy the data that had been
>> on the block (if at all possible) to a "spare" block reserved for this
>> purpose. The substitution is transparent - done by the drive firmware,
>> and not under O/S control. This means that some blocks may still exist
>> with old data even after a thousand writes with random data. For
>> common users, the data is inaccessible, as you need the manufacturers
>> software to reset the bad and spare block table to access those blocks
>> (which may or may not still be usable). This may or may not be an
>> acceptable level of risk. This is the reason that NISPOM requires
>> physical destruction of media that contained "Top Secret" data.
>
> That's why I prefer encrypting my hard-disk instead of being concerned
> about proper disposition. Using a secure key, I could lose it or throw
> it away without having concerns.
>
> Luckily proper encryption cannot be distinguished from random noise. So
> an attacker might even think that I've overwritten it that way, trying
> to hide data stored on the disk previously. My current hard-disk has
> never received a single bit of clear-text.
>
> By the way, is there any way to query the number of such defective
> blocks for IDE drives? I don't seem to find a suitable hdparm-option.

Just for the dubious case in 45 days you're still searching Wink

here's an addup to your last question.

For SMART drives you've got smart tools, as posted elsewhere.
For e2fs and family you'll find 'badblocks' tool in e2fsprogs.

PS:
As for the the encrypting of HDs, do not omit your to encrypt
your swap partition (or use an overall encrypt in RAM too).

Loki Harfagr · External Since: Oct 14, 2004 Posts: 286

Le Mon, 19 Jun 2006 05:27:39 +0200, Ertugrul Soeylemez a écrit :

> Loki Harfagr (06-06-18 18:50:40):
>
>> As for the the encrypting of HDs, do not omit your to encrypt your
>> swap partition (or use an overall encrypt in RAM too).
>
> Encrypting the whole hard-disk is going to be overkill for most
> configurations.

Right, but wasn't it reasonably on topic for this group to think
that this was not the question Wink

Besides, you were the one to say :
"That's why I prefer encrypting my hard-disk instead of being concerned
about proper disposition."

to which I absolutely agree Smile

I was just reminding the occasional reader (obviously not you) a
small point about swap that would defeat all previous efforts if not controlled.

> It's useless to encrypt */bin directories, but they
> will cause a noticable speed impact when encrypted -- at least, if you
> don't have much RAM or a slow CPU.

Agreed too.

>
> However, one should encrypt his home directory, or better yet, the
> entire /home tree. Encrypted swap is an obligation in my opinion,

If swap is *supposed* to be used then *YES* do encrypt it.
Or don't use swap, *or* use an "overall encrypt in RAM" for data,
so even if some gets in the swap reposit at some time it'll just be
a part of the noise.

> and
> modern distributions should provide an easy way to do all this
> automatically on installation, if desired (using PAM for home directory
> encryption or similar). Unfortunately I don't know of any distribution,
> which does that.

Er ... You seem concerned with securing a system and then
you install PAM on it !? That'd be quite as good by only rot13'ing
the filenames Wink

> I don't know exactly, what you mean by "overall encrypt in RAM", but if
> you mean encrypting all RAM contents,

Right, I wasn't very clear! I just meant about using encrypted data
for data, the main hole in a system isn't the RAM (or if it'd be that'd
just mean you're already assaulted beyond help) but in the possibility of
some data written clear on some disk, use a RAM /tmp and encrypting pipes
then you may have proper data encryption for the parts that counts.

> this is either useless or
> technically difficult. Remember that you have to save the key
> somewhere, and you will save it in RAM.

Not only, but I'll agree on that point too, at least there'll be
a moment when the keys would be in the RAM.

> To keep this short: Using RAM
> encryption is going to be more difficult than breaking it,

That's a part of the game Wink

> thus it's
> useless.

Well, difficult or painful aren't necessarily synonyms of useless, but
I'll agree on that point too if you meant something like "users that use
heavy passwords" will end up to print them or forget them" ;D)

> By the way, for current RAM chips, it is hard to read the RAM contents,
> without having access to the operating system running, and/or without
> disturbing its operation.

And on this you have my total reddition, sorry if my poor english
writing could make you feel I was criticizing your sayings.

Cheers.