logo (C) Bas Schuiling

BlackHole

please do not use: I can't guarantee if it keeps your data safe. Feel free to fetch the sourcecode and work on that; I happily accept patches.

BlackHole is an opensource data deduplicating network block device (a NBD server).
As BlackHole works on the block-level, it is filesystem agnostic. So you can create ext3, xfs, jfs, any filesystem you like on it. Tested with ext2, ext3, ntfs and reiserfs.
NBD is short for 'network block device'. It is comparable to iSCSI and FCoE. For allmost all platforms NBD-servers are available. NBD-clients are in the Linux, HURD and Solaris kernel.
BlackHole can use MD5, SHA-1 and SHA-256/384/512 to generate block-hashes.
Since version 0.3 it also includes synchronous and a-synchronous replication (other products with this functionality are called metro mirror and global mirror).
Since version 0.8 it can create snapshots in sparse files.
Since version 1.7 BlackHole can encrypt the datastore.
Since version 1.8 BlackHole can use Kyoto Cabinet (a NoSQL database).
Since version 2.0 BlackHole supports multiple LUNs, compression and encryption.
Version 2.1 brings LDAP authentication, 2.2 adds zoning and IMDisk support.

Warning

This is a beta version: don't use it with data you don't have a backup of.

Download

binarysourcecomment
BlackHole-2.5.jarMajor overhal: kyotocabinet backend should now be safe for sudden interrupts. You are strongly afviced to use the kyotocabinet backend. Note: the KC datastore in this release is NOT COMPATIBLE with older versions!
bh-2.5-src.tar.bz2Source code of above jar file.
BlackHole-2.3.jarbh-2.3.tgzFixed memory leak in MySQL backend

How to use it

See manual.pdf for a short manual.

Please note: the files in the datastore directory are sparse files so even they look like large files, they only use diskspace that is really used.


This program was developed with the help from .

License

In short: it is released under GPLv2.
BlackHole is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 2.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

Links

What is data de-duplication (wikipedia)
LessFS (open source) - LessFS is on filesystem level
Opendedup (open source)
ZFS is capable of automatic data de-duplication
DataDomain
NetApp
IBM

Check out my united states Mega Millions lottery winning help page