Hello everyone, I’m in the process of setting up long term storage of a large amount of media files for my family. Since this is stuff that’s rarely accessed, I don’t need it to be always fully available, and I want to limit the actual transfer of files to a specific schedule.

Since knowing what’s already in storage is kinda non-trivial, are there any tools out there that allow me to browse which files are in cold storage, without the actual drives being attached / mounted? So only info like file location, name, metadata, perhaps icon, etc.

The transfer part I can manage to automate, as I can think of a few ways of allowing my users to “stage” pulling and pushing info from and to cold storage, and having the transfers be done on a fixed schedule like every two weeks or something.

Obviously self-hosted, open source stuff that’s not vibecoded is preferred.

Thanks in advance!

  • grue@lemmy.worldEnglish
    24·
    6 hours ago

    Before unmounting:

    $ tree -afx /PATH/TO/VOLUME > file_listing.txt

    Later, when you want to search:

    $ grep PATTERN file_listing.txt

  • aksdb@lemmy.worldEnglish
    5·
    4 hours ago

    What kind of media files? For a typical media collection something like Emby or Jellyfin is quite nice. Then you have your collection and it would just refuse to play media if the file in question isn’t mounted.

  • wilo108@lemmy.mlEnglish
    3·
    3 hours ago

    Lots of good ideas here. For other reasons I use:

    find /path/to/mount -xdev -printf '%p\t%y\t%s\t%T@\n' | zstd > /path/to/output_fs.tsv.zst

    on a regular basis. For convenience I also munge the output into json so that it can be piped into gdu -f - for a very convenient tui interface; but you could also just pass -o to gdu and save the resulting json. Very convenient.

  • Azura The Spellkissed@lemmy.blahaj.zoneEnglish
    6·
    4 hours ago

    Not a GUI/webapp, but I like to use ncdu snapshots (“exports”). ncdu is an interactive file browser / disk usage analyzer with a terminal UI, which allow for convenient browsing but also, y’know, figuring out what to delete in case of storage cleanups.

    creating a snapshot (“exporting”):
    ncdu /pathyouwanttoscan -2 --one-file-system --compress -o snapshot_ncdu.json.zstd

    (-2 enables progress bar and error messages during the scan)

    reading (“importing”):
    ncdu -f snapshot_ncdu.json.zstd

  • jj2026@lemmy.zipEnglish
    2·
    3 hours ago

    Haven’t used it yet, but I think git annex is built for this. Plan to do the same eventually. It lets you keep the index on hot storage.

  • MonkderVierte@lemmy.zipEnglish
    2·
    3 hours ago

    Like, find /your/mount -print |sort > files.list? (sort is just so it looks nice)

    And then the same for the new dir and using any diff viewer for differences or grep -f for automation.

    You mean like that?

  • InnerScientist@lemmy.worldEnglish
    2·
    5 hours ago

    I’d probably write a bash script that recursively goes through the disk and creates any folders it sees on another path, and when it finds a file it creates a corresponding empty file with the same name. If you want more info you could script some metadata tools that extract the info and create a dummy media file with the same metadata.

  • lime!@feddit.nuEnglish
    2·
    6 hours ago

    need it to be a hierarchical thing? otherwise just cp -vur src/ dst/ >> files.txt && sort -bdufo files.txt files.txt and look in the file.