I remember there was an issue with Baloo and BTRFS where Baloo keeps reindexing snapshots so if I have one unique file with one unique word and 5 snapshots, searching for the word in Dolphin, Baloo would return 6 results (the original plus 5 snapshots) and all pointing to the same file.
Has this issue been sorted out?
I would like to hear from users using Baloo, BTRFS and snapshotting with snapper.
Thank you
There are fixes here, for KF6:
https://invent.kde.org/frameworks/baloo/-/merge_requests/131
and cherrypicked for KF5:
https://invent.kde.org/frameworks/baloo/-/merge_requests/169
I think that was about a year ago…
They do seem to stop the “if you have BTRFS, you can get a new device number when you reboot and if you get a new device number, Baloo reindexes everything”. OpenSUSE suffered the most and I’m assuming if you are using snapper…
You specifically say “snapshots” so if you are doing snapshots into a folder under your home, you would need to exclude it… (not personally met this but there was a bug about it…)
I am not sure I get it correct.
I understand the issue is still there and it will still reindex snapshots and give multiple search results all pointing to the same file. Right?
I understand the issue is still there and it will still reindex snapshots and give multiple search results all pointing to the same file. Right?
I think the issue is fixed, although whether you see the fix depends on your distro. A year is a fair time though…
It is quite easy to test, set up indexing on a BTRFS disk, try a “baloosearch -i one-of-your-files” (maybe baloosearch6), reboot and repeat. You get to see Baloo’s internal “docID” listed with a “-i”, if you see the same file appearing with a different DocID (and additional copies appearing after reboots) then you might not have the patch. (See Bug 475919).
For the snapshots, if you are saving snapshots to a .snapshot folder, then you should exclude this (see Bug 481101)
With all my due respect “I Think” means you are not sure!
Nobody tested on a real machine?
I am on EndeavourOS, KDE.
Generally EndeavourOS should have the latest software.
I wonder if anybody actually tested it to give a precise answer, if It is sorted out actually or not.
Nobody tested on a real machine?
There will have been many tests on various machines. I know Fedora adopted the patch early and deployed it before it was released on Neon. When looking just at Neon, the patch will have been installed on the “Neon Unstable” and “Neon Testing” editions before reaching “Neon User”.
I wonder if anybody actually tested it to give a precise answer, if It is sorted out actually or not.
I’ve just downloaded EndeavourOS to try it out. The default install seems to be on ext4, I’ve checked that and it is OK, there are no duplicate results. I repeated the install, manually partitioning the disk to use BTRFS and there are also no duplicate results. I see the device number sometimes change on reboot (as might be expected) but this does not affect Baloo.
If you are running on a system “with history” and have stepwise upgraded rather than reinstalled over the years, it may be good advice to purge and reindex to get a clean index.
To test it properly:
-the system and /home should be BTRFS
-Snapper should be installed
-a single unique file should be there say file xyz.txt with one word say “abracadabra”
-a few snapshots should be taken by snapper
-maybe we need a boot snapshot to be sure.
Then do the search by content for the word “abracadabra”
If it returns only one result then it is OK.
Before that if you had 3 snapshots then it would return 4 results all pointing to the same file (3 snapshotted plus the original one)
If you test ti this way and it gives only one result then it is great and problem is sorted.
Thank you.
I really appreciate your help.
KDE is my “only” favorite.
When you snapshot @home, the files appear under /home/.snapshots. A standard Baloo config (of indexing your home folder and below) won’t index them so you are safe…
Trying it out: if I have a “testfile.txt” and shapshot home, the file appears in “/home/.snapshots/1/snapshot/test/testfile.txt”. This is not accessible with a normal user account and, as Baloo runs as you, it also cannot explore the structure.
I’ve tried some manual snapshots of “home” and baloosearch6 doesn’t find duplicates. I’ve also tried a manual index command:
balooctl6 index /home/.snapshots/1/snapshot/test/testfile.txt
and get a file access error.
As above if you are creating snapshots under your $HOME (a /home/test/.snapshots), it is best to exclude that from indexing.
Thank you @tagwerk . So, I understand that I can install my EndeavourOS with BTRFS, Snapper and just exclude /home/.snapshots/ from being indexed and I won’t have the issue of multiple results pointing to the same file.
Did I get it right?
Did I get it right?
I’ve learnt that there are always surprises 8-] but I think from my test install and simple setup you will be OK.
If you see something odd, open a bug report. I don’t think there’s much experience with EndeavourOS and troubleshooting involves a lot of “Can we make the same happen on Neon?”.
This is good news.
I hope Baloo would be configured by default to exclude ALL hidden files and folders by default. Simple logic, who needs really to search for a hidden file or its content?! It is hidden for a purpose!
In the config file all hidden files and folders should be excluded by default.
I hope you pass this suggestion to Baloo developers. Sorry, I am no expert to see how to do it and an old enough man!
I will reinstall again with BTRFS, Snapper, BTRFS Assitant… etc and modify the config file to exclude the hidden snapshot folder and all commonly hidden folders and files.
I will report back how it goes.
Yes, that’s normally what happens.
People do enable hidden file and folder indexing and get surprised by what happens (you probably don’t want to index your .cache or .local/share/Trash folders)
Sorry, I just checked my Baloo config file after default installation. Nothing was excluded. I did it manually. As I told you before that I had the issue of multiple result with BTRFS. You may check the EndeavourOS forums. My user name there is limotux
I have the config file excluding the snapshot folder.
Everything is working fine.
Thank you.