Wednesday, January 5, 2011

forensics: is Shred securely deleting the files?

This article from the SANS Computer Forensics Blog analyzes how the tool shred behaves when wiping files from the file system.

If fact, like any other 'userland' tool, shred opens the file/s by using  the available system calls. Therefore the application cannot access some metadata that remains in the file system.

In the Linux/Unix realm we have tools like shred for securely overwriting files before deleting them in order to prevent recovery of the deleted file.  If your adversary is sufficiently advanced (or just not lazy), they can obviously use these tools to frustrate your forensic investigation.  Previously, I had thought that shred removed all traces of the file from disk.  But in the course of some other file system research I was doing, I've realized that there may be a few lingering artifacts.
 If you look at the shred source code, it simply opens the file like any normal user process, jumps to the beginning of the file, and proceeds to overwrite the file from beginning to end.  As a normal user process, it has access to the data blocks that make up the file content, but not the indirect block metadata.  The file system drivers in the OS "hide" the indirect blocks from the shred program.  A program that wanted to clobber the indirect block data would have to have superuser privileges so that it could open the raw disk device and attack the necessary blocks directly.  Normal user space programs like shred aren't going to have this level of access.

The author of the post verifies the behavior with Sleuthkit . As expected, shred is not able to access  an indirect block that contains metadata  and it does not get deleted. This only can be achieved by opening the raw device and overwriting the needed blocks, that needs admin. privileges.

UPDATE: the file can still be completely recovered from the journal with ext3grep,  if it was recently deleted. We can find some examples in this site.
 Every time a file is accessed, it's Access Time is changed, and it's inode is written to disk, along with 31 other inodes that reside in the same block. When that happens, a copy of that block is written to the journal. Therefore, if your partition isn't too large compared to your journal, and if you at least recently accessed the files you want to recover, you might be able to recover the block pointers from the journal.

1 comment:

  1. Hello Xavier Garcia,
    I know this was posted quite a while back but I just read it.

    It seems to me you have misinterpreted the articles you have referenced and have drawn an incorrect conclusion - that shred does not work.

    The article by Hal Pomerantz shows that shred does not access indirect blocks where metadata is stored.
    Note that he does not state that the erased data can be reconstructed.
    Mr. Pomerantz concludes his article by stating it is better to use dd to zero out the file's blocks as root, which indicates Hals' ignorance in the matter of data recovery.

    Shred was written to make real data recovery difficult. Real data recovery that is performed in labs with specialized equipment that reads back into the "echoes" of previous writes onto the same disk locations.
    For more information refer to the article written by Peter Guttman @

    As for the file recovery example given by carlo the author of ext3grep @ []:
    If ever I've seen taking something out of its context - this would be it.
    There is no connection between a scenario of recovering data deleted by rm (as in carlos's case) where what's missing is the metadata, and the scenario in question - where the actual file data is overwritten (and where metadata is insignificant)


    NOTE, that the shred manual and code state specifically that they DO NOT work where file systems journal DATA as well as metadata (e.g where ext3 is mounted with data=journal, which is not the default)