Logo

Docs

Create NSRL BoltDB

FileTrove needs a BoltDB that holds SHA1 hashes from the NSRL. You can download a prepared database from within FileTrove while installing it. This database is created from the “Modern Minimal” RDS. If you want to create your own database or if you want a specific database version you can follow this steps. You will need the tool admftrove from the repository.

  1. Download a RDS set from the NSRL webpage, check the hash sums of the file and unzip the archive file: NSRL (Note: The uncompressed files for the minimal set are ~150 GB).
  2. Change into the directory of the unzipped archive, e.g. cd RDS_2024.03.1_modern_minimal.
  3. Get the SHA1 hashes from the SQLite database and write one per line into a text file: sqlite3 RDS_2024.03.1_modern_minimal.db “select distinct sha1 from file” > sha1_uniq_2024.03.01_modern_minimal.txt.
  4. Feed the text file to admftrove, this creates the file nsrl.db. You also have to provide the –nsrlversion flag. This string is used by ftrove for the session information, e.g. admftrove –creatensrl sha1_uniq_2024.03.01_modern_minimal.txt –nsrlversion RDS_2024.03.1_modern_minimal.
  5. Optional: As Bolt creates files with a huge overhead, you should optimize the size with bbolt compact (it is really bbolt, see BBolt. bbolt compact -o OUTPUTFILE SOURCEFILE
  6. Copy the compacted nsrl.db into the db directory of ftrove.

Note: The creation of the database file will take several hours.