A Survey On Deduplication Methods

There is an increasing demand for systems that can provide secure data storage in a cost-effective manner. Having duplicate records occupies more space and even increases the access time. Thus there is a need to eliminate duplicate records. This sounds to be simple but requires an tedious work since duplicate records do not share a common key and/or they contain errors that make duplicate matching a difficult task. Errors are also introduced as the result of transcription errors, incomplete information, lack of standard formats, or any combination of these factors. Several approaches areproposed to eliminate duplicate data first at the file level and then at the chunk level to reduce the duplicatelookup complexity. In this paper,few of the methods are discussed with its advantages and disadvantages. And also a better solution is proposed.