As daunting as securing your Linux system might seem, every extra step makes a difference. It’s almost always better to make a modest stride than let uncertainty keep you from starting. Fortunately, there are a few basic techniques that greatly benefit users at all levels, and knowing how to securely wipe your hard drive in Linux is one of them. Because I adopted Linux primarily with security in mind, this is one of the first things I learned. Once you have absorbed this lesson, you will be able to part with your hard drives safely.
I beg to differ with the nay-sayers. Overwriting every addressable byte on your disk with '0' will definitely add a significant barrier to anyone trying to recover that data.
Can some magnetic residue of the original data be remaining on the disk? Maybe. But the cost of the equipment and expertise to recover it is going to steep. Maybe if your of interest to a well funded spy or criminal organization will you have a real concern.
Flash is a little different story. Flash cells should erase cleaner. But there is the reserve sectors and the question of whether they are rotated through with the usual wear leveling algorithms. But this can be addressed with multiple passes. I would probably use 0 (/dev/zero) on the first pass and then 0xff on the second and repeat the cycle again. If the spare sectors are part of the wear leveling cycle 4x the storage capacity will be more than enough to hit them. I leave off with the 0xff because that leaves the flash in the native "erased" state, ready for writing new data.
I don't know of a readily available source of 0xff bytes but the code to generate them with a compiled language is stupid simple. I wrote such a tool a long time back, which I still use. I would use a compiled language because we are talking about generating a *LOT* of bytes and interpreted languages will take much longer.
Some things to consider:
'dd' writes to the kernel's cache, like most programs. So you need to make sure the cache gets flushed to the device. Using the "sync" command as root can do that or shutdown/reboot will force the flush.
You can pipe any source of data into dd if you want to use a different erase pattern. Just use the standard pipe (|) arrangement. You could also use /dev/random or /dev/urandom to write random data. The first is probably completely useless as it would take a very long time for your system to generate the random data. The latter is faster because the random data is of lesser quality. It will still take a long time.
I would contend with the authors remark that this procedure is CPU bound. Although his formula is not as efficient as it could be its mostly an I/O bound operation. On a decent SATA or SCSI system I have routinely kept a drive saturated with I/O, this way, while not observing a noticeable impact on the rest of the system. The main thing you have to do is tell dd to use a larger buffer with the "bs", block size, option. I use 1M and it seems to provide real good throughput... and yes it will write a partial block if needed.
So something like:
dd if=/dev/zero of=/dev/sda bs=1M
And, yes, a tool like "shred" will do a more thorough job at trying to prevent someone scrounging magnetic residue.
The title is misleading. This is not a way to "secure your Linux system", rather to "securely delete your files". And it fails at that since the usage of dd for security wipe is not considered 100% safe, other tools are available (shred for example).
A while back I did some research on wiping data. While I don't have time now to refer to my notes, (someone correct me if I'm wrong) the impression I got was that:
1.) The DD command is an unreliable way to wipe data on today's more technologically complex hard drives.
2.) Thumbdrives are entirely different from disk drives and cannot be securely deleted using conventional means, if at all. As far as I know, a readily available solution does not exist for this.
3. The safest way to wipe a hard disk drive is to download a data wiping program from the drive manufacture's website designed for that specific drive. They know how their drive is built. (This is what I did before donating an old personal computer to a business.)
Other points worth consideration:
DOES IT MATTER WHAT FILE SYSTEM YOU USE?
Before manually wiping a drive using DD or formatting, have you ever changed the file system (NTFS, Ext4...)? There may be isolated sections of the older filesystem that are untouchable by the new filesystem, even with formatting.
A filesystem does not perfectly fill all the space on a drive, and you might even had set up multiple partitions.
IS DRIVE ENCRYPTION A SOLUTION?
I've heard encryption recommended as a sure way to wipe drives... just encrypt the entire drive using an impossibly long password, then forget the password. However, I think even this may be a misrepresentation.
One of the concerns of safeguarding data is that a recovery expert might disassemble the drive, place it in a expensive scanner, and recover residual information, (such as between the tracks on a disk drive) or otherwise piece together enough leftover data to recover some partial files.
This is why, when wiping data, a drive is formatted several times over and/or is overwritten with random data instead of zeros.
If you are concerned that data might be recovered in this manner, (by accessing underlying/residual data or data fragments) then any encrypted data would be bypassed. You might just as well have written random data over your drive.
IS RESIDUAL DATA RECOVERY REALLY POSSIBLE?
It's been suggested that on modern hard disk drives, with their densely compacted data tracks, such recovery techniques are not practical or at least unrealistically expensive to attempt.
But I wouldn't toss such concerns entirely out the window. And there may be other, simpler techniques such as scanning the sectors that have been marked as bad or unreliable by the drive's firmware.
You may have old data on sections of the drive that are now entirely unaccessible to you and your formatting attempts, but which could be easily read if the drive's firmware chip was replaced. This may apply to thumdrives as well as disk drives.
WILL IT REALLY TAKE 1,000 YEARS TO CRACK A GOOD ENCRYPTION?
I've already mentioned how relying on encryption to wipe data may not be a perfect solution. Besides this, there is another factor which *nobody* seems to consider.
While the premise of "1,000 years to crack" might be correct if you used the same computer for that 1,000 years, does such analysis account for the ever-increasing speed of computers... and can it?
What speeds might the next breakthrough in processor technology achieve? Who can say what the next discovery in mathematically decryption will be? Yesterday's encryption could be rendered obsolete tomorrow.
Encryption is good for information in transit at that time. But if the same information might be of value in the future, a sinister character may simply hold on to it until such time it can be readily decrypted.
This applies to stolen data drives or intercepted internet transactions. And such risk factors *must* be assessed for any important data protection strategy.
THE CONCLUSION?
To keep from going bonkers, it helps to remember that data protection is really about risk assessment.
For your own personal or small business computer that you want to give away, a DD or full format on the existing filesystem followed by a data wipe using manufacturer software should be sufficient. And, really, if it's not, the drive should be incinerated and you should save the receipt (smiles).
We're taking disk drives here, not thumbdrives... or email servers or ISP's or Google, which is a whole different subject.
Before manually wiping a drive using DD or formatting, have you ever changed the file system (NTFS, Ext4...)? There may be isolated sections of the older filesystem that are untouchable by the new filesystem, even with formatting.
A filesystem does not perfectly fill all the space on a drive, and you might even had set up multiple partitions.
I've heard encryption recommended as a sure way to wipe drives... just encrypt the entire drive using an impossibly long password, then forget the password. However, I think even this may be a misrepresentation.
One of the concerns of safeguarding data is that a recovery expert might disassemble the drive, place it in a expensive scanner, and recover residual information, (such as between the tracks on a disk drive) or otherwise piece together enough leftover data to recover some partial files.
This is why, when wiping data, a drive is formatted several times over and/or is overwritten with random data instead of zeros.
If you are concerned that data might be recovered in this manner, (by accessing underlying/residual data or data fragments) then any encrypted data would be bypassed. You might just as well have written random data over your drive.
Before manually wiping a drive using DD or formatting, have you ever changed the file system (NTFS, Ext4...)? There may be isolated sections of the older filesystem that are untouchable by the new filesystem, even with formatting.
A filesystem does not perfectly fill all the space on a drive, and you might even had set up multiple partitions.
IS DRIVE ENCRYPTION A SOLUTION?
I've heard encryption recommended as a sure way to wipe drives... just encrypt the entire drive using an impossibly long password, then forget the password. However, I think even this may be a misrepresentation.
One of the concerns of safeguarding data is that a recovery expert might disassemble the drive, place it in a expensive scanner, and recover residual information, (such as between the tracks on a disk drive) or otherwise piece together enough leftover data to recover some partial files.
This is why, when wiping data, a drive is formatted several times over and/or is overwritten with random data instead of zeros.
If you are concerned that data might be recovered in this manner, (by accessing underlying/residual data or data fragments) then any encrypted data would be bypassed. You might just as well have written random data over your drive.
IS RESIDUAL DATA RECOVERY REALLY POSSIBLE?
It's been suggested that on modern hard disk drives, with their densely compacted data tracks, such recovery techniques are not practical or at least unrealistically expensive to attempt.
But I wouldn't toss such concerns entirely out the window. And there may be other, simpler techniques such as scanning the sectors that have been marked as bad or unreliable by the drive's firmware.
You may have old data on sections of the drive that are now entirely unaccessible to you and your formatting attempts, but which could be easily read if the drive's firmware chip was replaced. This may apply to thumdrives as well as disk drives.
WILL IT REALLY TAKE 1,000 YEARS TO CRACK A GOOD ENCRYPTION?
I've already mentioned how relying on encryption to wipe data may not be a perfect solution. Besides this, there is another factor which *nobody* seems to consider.
While the premise of "1,000 years to crack" might be correct if you used the same computer for that 1,000 years, does such analysis account for the ever-increasing speed of computers... and can it?
What speeds might the next breakthrough in processor technology achieve? Who can say what the next discovery in mathematically decryption will be? Yesterday's encryption could be rendered obsolete tomorrow.
Encryption is good for information in transit at that time. But if the same information might be of value in the future, a sinister character may simply hold on to it until such time it can be readily decrypted.
This applies to stolen data drives or intercepted internet transactions. And such risk factors *must* be assessed for any important data protection strategy.
THE CONCLUSION?
To keep from going bonkers, it helps to remember that data protection is really about risk assessment.
For your own personal or small business computer that you want to give away, a DD or full format on the existing filesystem followed by a data wipe using manufacturer software should be sufficient. And, really, if it's not, the drive should be incinerated and you should save the receipt (smiles).
We're taking disk drives here, not thumb drives... or email servers or ISP's or Google, which is a whole different subject.
Loading ...
Securing Your Linux System Bit by Bit
Posted by: Jonathan Terrasi June 13, 2017 10:35 AMAs daunting as securing your Linux system might seem, every extra step makes a difference. It’s almost always better to make a modest stride than let uncertainty keep you from starting. Fortunately, there are a few basic techniques that greatly benefit users at all levels, and knowing how to securely wipe your hard drive in Linux is one of them. Because I adopted Linux primarily with security in mind, this is one of the first things I learned. Once you have absorbed this lesson, you will be able to part with your hard drives safely.
Can some magnetic residue of the original data be remaining on the disk? Maybe. But the cost of the equipment and expertise to recover it is going to steep. Maybe if your of interest to a well funded spy or criminal organization will you have a real concern.
Flash is a little different story. Flash cells should erase cleaner. But there is the reserve sectors and the question of whether they are rotated through with the usual wear leveling algorithms. But this can be addressed with multiple passes. I would probably use 0 (/dev/zero) on the first pass and then 0xff on the second and repeat the cycle again. If the spare sectors are part of the wear leveling cycle 4x the storage capacity will be more than enough to hit them. I leave off with the 0xff because that leaves the flash in the native "erased" state, ready for writing new data.
I don't know of a readily available source of 0xff bytes but the code to generate them with a compiled language is stupid simple. I wrote such a tool a long time back, which I still use. I would use a compiled language because we are talking about generating a *LOT* of bytes and interpreted languages will take much longer.
Some things to consider:
'dd' writes to the kernel's cache, like most programs. So you need to make sure the cache gets flushed to the device. Using the "sync" command as root can do that or shutdown/reboot will force the flush.
You can pipe any source of data into dd if you want to use a different erase pattern. Just use the standard pipe (|) arrangement. You could also use /dev/random or /dev/urandom to write random data. The first is probably completely useless as it would take a very long time for your system to generate the random data. The latter is faster because the random data is of lesser quality. It will still take a long time.
I would contend with the authors remark that this procedure is CPU bound. Although his formula is not as efficient as it could be its mostly an I/O bound operation. On a decent SATA or SCSI system I have routinely kept a drive saturated with I/O, this way, while not observing a noticeable impact on the rest of the system. The main thing you have to do is tell dd to use a larger buffer with the "bs", block size, option. I use 1M and it seems to provide real good throughput... and yes it will write a partial block if needed.
So something like:
dd if=/dev/zero of=/dev/sda bs=1M
And, yes, a tool like "shred" will do a more thorough job at trying to prevent someone scrounging magnetic residue.
A while back I did some research on wiping data. While I don't have time now to refer to my notes, (someone correct me if I'm wrong) the impression I got was that:
1.) The DD command is an unreliable way to wipe data on today's more technologically complex hard drives.
2.) Thumbdrives are entirely different from disk drives and cannot be securely deleted using conventional means, if at all. As far as I know, a readily available solution does not exist for this.
3. The safest way to wipe a hard disk drive is to download a data wiping program from the drive manufacture's website designed for that specific drive. They know how their drive is built. (This is what I did before donating an old personal computer to a business.)
Other points worth consideration:
DOES IT MATTER WHAT FILE SYSTEM YOU USE?
Before manually wiping a drive using DD or formatting, have you ever changed the file system (NTFS, Ext4...)? There may be isolated sections of the older filesystem that are untouchable by the new filesystem, even with formatting.
A filesystem does not perfectly fill all the space on a drive, and you might even had set up multiple partitions.
IS DRIVE ENCRYPTION A SOLUTION?
I've heard encryption recommended as a sure way to wipe drives... just encrypt the entire drive using an impossibly long password, then forget the password. However, I think even this may be a misrepresentation.
One of the concerns of safeguarding data is that a recovery expert might disassemble the drive, place it in a expensive scanner, and recover residual information, (such as between the tracks on a disk drive) or otherwise piece together enough leftover data to recover some partial files.
This is why, when wiping data, a drive is formatted several times over and/or is overwritten with random data instead of zeros.
If you are concerned that data might be recovered in this manner, (by accessing underlying/residual data or data fragments) then any encrypted data would be bypassed. You might just as well have written random data over your drive.
IS RESIDUAL DATA RECOVERY REALLY POSSIBLE?
It's been suggested that on modern hard disk drives, with their densely compacted data tracks, such recovery techniques are not practical or at least unrealistically expensive to attempt.
But I wouldn't toss such concerns entirely out the window. And there may be other, simpler techniques such as scanning the sectors that have been marked as bad or unreliable by the drive's firmware.
You may have old data on sections of the drive that are now entirely unaccessible to you and your formatting attempts, but which could be easily read if the drive's firmware chip was replaced. This may apply to thumdrives as well as disk drives.
WILL IT REALLY TAKE 1,000 YEARS TO CRACK A GOOD ENCRYPTION?
I've already mentioned how relying on encryption to wipe data may not be a perfect solution. Besides this, there is another factor which *nobody* seems to consider.
While the premise of "1,000 years to crack" might be correct if you used the same computer for that 1,000 years, does such analysis account for the ever-increasing speed of computers... and can it?
What speeds might the next breakthrough in processor technology achieve? Who can say what the next discovery in mathematically decryption will be? Yesterday's encryption could be rendered obsolete tomorrow.
Encryption is good for information in transit at that time. But if the same information might be of value in the future, a sinister character may simply hold on to it until such time it can be readily decrypted.
This applies to stolen data drives or intercepted internet transactions. And such risk factors *must* be assessed for any important data protection strategy.
THE CONCLUSION?
To keep from going bonkers, it helps to remember that data protection is really about risk assessment.
For your own personal or small business computer that you want to give away, a DD or full format on the existing filesystem followed by a data wipe using manufacturer software should be sufficient. And, really, if it's not, the drive should be incinerated and you should save the receipt (smiles).
We're taking disk drives here, not thumbdrives... or email servers or ISP's or Google, which is a whole different subject.
Regards,
Other points worth consideration:
DOES IT MATTER WHAT FILE SYSTEM YOU USE?
Before manually wiping a drive using DD or formatting, have you ever changed the file system (NTFS, Ext4...)? There may be isolated sections of the older filesystem that are untouchable by the new filesystem, even with formatting.
A filesystem does not perfectly fill all the space on a drive, and you might even had set up multiple partitions.
IS DRIVE ENCRYPTION A SOLUTION?
I've heard encryption recommended as a sure way to wipe drives... just encrypt the entire drive using an impossibly long password, then forget the password. However, I think even this may be a misrepresentation.
One of the concerns of safeguarding data is that a recovery expert might disassemble the drive, place it in a expensive scanner, and recover residual information, (such as between the tracks on a disk drive) or otherwise piece together enough leftover data to recover some partial files.
This is why, when wiping data, a drive is formatted several times over and/or is overwritten with random data instead of zeros.
If you are concerned that data might be recovered in this manner, (by accessing underlying/residual data or data fragments) then any encrypted data would be bypassed. You might just as well have written random data over your drive.
DOES IT MATTER WHAT FILE SYSTEM YOU USE?
Before manually wiping a drive using DD or formatting, have you ever changed the file system (NTFS, Ext4...)? There may be isolated sections of the older filesystem that are untouchable by the new filesystem, even with formatting.
A filesystem does not perfectly fill all the space on a drive, and you might even had set up multiple partitions.
IS DRIVE ENCRYPTION A SOLUTION?
I've heard encryption recommended as a sure way to wipe drives... just encrypt the entire drive using an impossibly long password, then forget the password. However, I think even this may be a misrepresentation.
One of the concerns of safeguarding data is that a recovery expert might disassemble the drive, place it in a expensive scanner, and recover residual information, (such as between the tracks on a disk drive) or otherwise piece together enough leftover data to recover some partial files.
This is why, when wiping data, a drive is formatted several times over and/or is overwritten with random data instead of zeros.
If you are concerned that data might be recovered in this manner, (by accessing underlying/residual data or data fragments) then any encrypted data would be bypassed. You might just as well have written random data over your drive.
IS RESIDUAL DATA RECOVERY REALLY POSSIBLE?
It's been suggested that on modern hard disk drives, with their densely compacted data tracks, such recovery techniques are not practical or at least unrealistically expensive to attempt.
But I wouldn't toss such concerns entirely out the window. And there may be other, simpler techniques such as scanning the sectors that have been marked as bad or unreliable by the drive's firmware.
You may have old data on sections of the drive that are now entirely unaccessible to you and your formatting attempts, but which could be easily read if the drive's firmware chip was replaced. This may apply to thumdrives as well as disk drives.
WILL IT REALLY TAKE 1,000 YEARS TO CRACK A GOOD ENCRYPTION?
I've already mentioned how relying on encryption to wipe data may not be a perfect solution. Besides this, there is another factor which *nobody* seems to consider.
While the premise of "1,000 years to crack" might be correct if you used the same computer for that 1,000 years, does such analysis account for the ever-increasing speed of computers... and can it?
What speeds might the next breakthrough in processor technology achieve? Who can say what the next discovery in mathematically decryption will be? Yesterday's encryption could be rendered obsolete tomorrow.
Encryption is good for information in transit at that time. But if the same information might be of value in the future, a sinister character may simply hold on to it until such time it can be readily decrypted.
This applies to stolen data drives or intercepted internet transactions. And such risk factors *must* be assessed for any important data protection strategy.
THE CONCLUSION?
To keep from going bonkers, it helps to remember that data protection is really about risk assessment.
For your own personal or small business computer that you want to give away, a DD or full format on the existing filesystem followed by a data wipe using manufacturer software should be sufficient. And, really, if it's not, the drive should be incinerated and you should save the receipt (smiles).
We're taking disk drives here, not thumb drives... or email servers or ISP's or Google, which is a whole different subject.
Regards,