Cannot properly use Baloo

jamal-ali1989 · November 17, 2024, 1:04pm

So, I have switched from Linux mint a few weeks ago to Fedora KDE 41.
While I have found most things to be more pleasant and customizable compared to Mint, I cannot get to configure Baloo properly to conduct a normal file search. I set baloo to index my entire home folder and its contents, including hidden files. I can monitor baloo using the command “balooctl6 monitor” to see what it is doing.
This is the problem I am having: I have a folder with some .docx files, which I confirmed to be indexed. I do a content search for a certain keyword and expect roughly a few files to pop up. To my surprise, only one of them shows up. I do another search with another keywords and those files do show up, indicating that they have been indexed.
I installed Nemo file manager to test this, and the same keyword that produces a single file in Dolphin, produces all of them in Nemo file manager.
Has anyone had this issue? I really like Dolphin but I cannot get over this inconsistency.
To make matters worse, I cannot find a single search app that can also perform content search.

I am curious, why did KDE go with this method instead of “regular” search, or whatever method it is Nemo uses? I don’t have to wait for indexing to complete and it looks for all keywords, without missing any.
Would it be possible for Dolphin to work like this in the future?

skyfishgoo · November 18, 2024, 1:15am

what is the output of these commands?

balooctl6 failed
balooctl6 check
balooctl6 status
balooctl6 config list includeFolders

if the last one only shows /home/user/, consider adding folders individually

if your baloo index needs to be rebuilt for any reason, use the disable, purge, and enable commands to start over.

reconsider indexing hidden files and folders as it just tends to balooN the index… you can exclude specific directories like ~/.cache for instance.

ben2talk · November 18, 2024, 3:51am

I have tried this in the past - I think this is definitely a mistake and will vastly bloat Baloo…
I did this mostly because I was (am) lazy and didn’t really bother much with learning how to search properly.

It led me down a rabbit hole believing (and finding lots of agreement) that Baloo is a buggy bloated mess… however, the opposite is true if used with care.

There are exceptions to this rule, however.

For example, I do include my .config folder (so that I can pull up conky configs easily) and a couple of other folders… and it doesn’t take long to add what’s important to you without just trying to include everything:

My Baloo...

❯ balooctl6 failed
All Files were indexed successfully

💡 balooctl => balooctl6
                                                        
❯ balooctl6 check
Started search for unindexed files
                                                           
❯ balooctl6 status
Baloo File Indexer is running
Indexer state: Checking for unindexed files
Total files indexed: 34,848
Files waiting for content indexing: 4
Files failed to index: 0
Current size of index is 605.34 MiB
                                                          
❯ balooctl6 config ls includeFolders
/home/ben/Dropbox/Admin/Scripts/
/home/ben/.local/share/plasma/
/home/ben/.config/
/mnt/T3/T3-Server/
/mnt/T4/T4-Server/
/mnt/T4/ebooks/
/mnt/W2/Movies/
/mnt/W2/TV/
/home/ben/

You should also not be keeping .docx files in a hidden folder, it’s clearly a USER file (and not a cache, config or other ‘hidden’ file).

Baloo should make all indexed files available via KRunner, and if that’s not finding what you need then I generally open up Dolphin and use KFind:

You can use wildcards too - here *

jamal-ali1989 · November 18, 2024, 11:07am

I am sorry but I did not understand how to proceed from here. If baloo indexed my entire home directory why can it not find a simple query from a .docx file? It is not hidden, by the way, but should that even matter?
Trying to index that specific file returns that it has already been indexed.
What is the reason for this behaviour of Baloo?

jamal-ali1989 · November 18, 2024, 11:12am


balooctl6 failed
All Files were indexed successfully

balooctl6 check
Indexed random files

balooctl6 status
Baloo File Indexer is running
Indexer state: Indexing file content
Total files indexed: 274.048
Files waiting for content indexing: 106.091
Files failed to index: 0
Current size of index is 7,36 GiB

What does this mean? Keep in mind that those files that are supposed to show up do show up when I search another query, but baloo seems to “discriminate” against certain words, which makes no sense.

I ran balooctl6 status again and it was not running. Then I enabled it and it said: Enabling and starting the File Indexer
fadmin@fedora:~$ kf.baloo: Failed to add exclude folder config entry for "/home/fadmin
What does this even mean?
In the “File search” menu, home directory is shown as indexed. with contents and hidden files/folders.

ben2talk · November 18, 2024, 1:06pm

What does this mean? Files waiting for content indexing: 106.091 means that well over a hundred thousand more files are waiting for content indexing.

Baloo is quite a passive indexer - mostly using idle time to index files. It will not be busy indexing while you’re working on your computer, that’s why it reports to you that it is not running.

If it ‘failed to add config entry’ for ‘/home/fadmin’ I am not surprised - as I am sure you already tried to use baloo to index your home folder.

Surely home/fadmin must be indexed - assuming that’s your home folder. How can you exclude it?

It works well for me, but I don’t need the entire contents of all my files indexing - neither do I need all my hidden files to be indexed.

You have told Baloo not only to index the entire contents of /home, including all the trashy and cache files, and it needs a lot more time to finish the massive task you set for it.

Now I must confess, I did include file contents - so let’s see if I can find a script I wrote last year, something to do with setting volume directly with a mouse gesture:

I don’t know if a docx file can be indexed, as I don’t use them… but for sure you have to wait for indexing to finish before you expect it to work - and you should try not to make your indexer index huge numbers of files which it has no point in indexing.

skyfishgoo · November 18, 2024, 3:01pm

yeah, that’s a lot of files, and a huge index… it does seem to be working tho.

this brings me back to my use the disable and purge commands to start over.

i would not recommend hidden files be turned on and i would ignore the undeletable entry about your home folder.

then i would add each individual folder that you want to have indexed before you enable baloo again.

maybe just focus on the folder where your .docx files are located and baloo will index the contents of those files as well if you have that box checked.

then when those files are complete according to status then try adding additional folders.

jamal-ali1989 · November 18, 2024, 10:05pm

Thank you for your responses. I think I have explained the issue wrongly though. The reason it says 100.000 files is because I purged the indexing to try again. All of my files had earlier been indexed.
Here are the problems:

In the file search menu. When I add my home directory, this is what I see, even after EVERY single file has been indexed:

image1145×313 27.1 KB

I think that “failed to add exclude folder” refers to the unindexed “home” folder?
2) I have a small folder with 10-20 .docx files. Now all of these are indexed as I can confirm through terminal. They have certain words in their content. For example, “fuse” and “circuit”. If I search for “fuse”, only 1 file shows up, while it should be 10 or so. If I search for “circuit”, the file that should have popped up earlier does show up. Now, all of these are content searches.
I tried this with Nemo’s content search and searching either “fuse” or “circuit” works exactly as expected. So, Baloo misses certain words for an unknown reason, which brings me to my next point
3) Would KDE add an option to dolphin for “regular” but slower search, just like in Nemo? Unfortunately, Baloo is the only problem so far that I cannot navigate with Fedora KDE. I’d have been much happier with normal search functions that works for even newly added files without any delay. It is driving me insane when I search for a keyword in contents of a file that I know for sure contains it and it is still missing it.
Also, every time I let baloo index and use “balooctl6 monitor” it gives me an error about “recursivecounter=2” or something, even if I just leave my PC alone…

skyfishgoo · November 18, 2024, 11:11pm

from the File Search settings GUI screen under Data to index:

check the filenames and contents box
uncheck the hidden files and folders box

go thru and delete all the locations you have added and set any you cannot delete to “indexed”, be sure to hit apply, then exit the GUI

from a command line window balooctl6 disable, then balooctl6 purge

run balooclt6 status to verify that it disabled.

reboot your PC

run balooclt6 status again to verify that it still disabled.

enter the GUI and verify that you only have one entry for you /home/user directory, that it says “Indexed” and that the delete button is greyed out.

if you have more than one entry something is wrong… if you cannot delete it from the GUI then close the GUI and edit the ~/.confitg/balloofilerc file manually in a text editor to remove the extra directory.

when the GUI looks good and is in agreement with these command line outputs
balooctl6 config list hidden
balooctl6 config list contentIndexing
balooctl6 config list includeFolders
balooctl6 config list excludeFolders

then, and only then, is it time to issue the command
balooctl6 enable
then close the window.

from there you should see the GUI flashing each item being indexed, or you can use balooclt6 monitor in a console window to follow the progress.

depending on how many files it has to access this can take a while to finish, which is why i recommend you ONLY index the folder with the documents you are having issue with and wait for it complete.

it’s easy to add more folders but you cannot remove items from the index unless you explicitly tell it to clear each one, one at a time… it’s easier to just purge and let it start over with a smaller list of folders.

if this is too much effort then just use nemo since that seems to be working the way you like.

trnqullea · November 19, 2024, 5:22pm

… If I search for “fuse”, only 1 file shows up, while it should be 10 or so …

Baloo ought to give you files containing “fuse”, won’t give you hits for substrings, “infuse” for example…

… Would KDE add an option to dolphin for “regular” but slower search …

If you disable Baloo, Dolphin falls back to its internal search. Not fast, maybe you’ll have trouble with searching your “/mnt” folders, but it’s there…

jamal-ali1989 · November 19, 2024, 8:00pm

Alright, can I recreate these steps but also allow indexing of hidden files?
EVERY time I index my home folder, “apply” the settings and reopen the “File search” menu. The “not indexed” home folder comes back. And this is after I deleted the line in baloofileirc showing my home folder in the excluded list. Is this a bug?

Those 4 commands you mentioned.
balooctl6 config list hidden returns no
balooctl6 config list contentIndexing returns yes
balooctl6 config list includeFolders returns:

/home/fadmin/Documents/
/home/fadmin/Pictures/
/home/fadmin/Videos/
/home/fadmin/Music/

Since you said I cannot remove files from the index, does it mean I have to purge it periodically? Isn’t there a command to just purge the deleted files?
Honestly, I’d have been much happier with unindexed search. Does KDE have plans to introduce it? I don’t think I’d be the only person that would find this problematic.
Also, this doesn’t explain why it cannot find a simple keyword in a .docx file.

jamal-ali1989 · November 19, 2024, 8:10pm

Baloo ought to give you files containing “fuse”, won’t give you hits for substrings, “infuse” for example…

I am aware of that but it does not find “fuse” itself. On the other hand, Nemo even finds substrings, so even the word “fuse” attached to other words won’t be an issue. I think this is a bug, but I wanna confirm it for sure before I go on to report it. I can deal with Baloo, as long as content search works normally. On the other hand, would it be possible for Baloo index file to get rid of deleted files, whether manually or automatically, in the future?

If you disable Baloo, Dolphin falls back to its internal search. Not fast, maybe you’ll have trouble with searching your “/mnt” folders, but it’s there…

So, Dolphin does have its own internal search? I tried it, but:

It cannot find even non-hidden files in hidden folders. So I have to go inside that folder and moving one level up removes all those results
No content search

The problem is that I wanna use Dolphin since it has “Show in groups” option that I really like. That’s why I don’t wanna return back to Nemo.

skyfishgoo · November 19, 2024, 9:40pm

if you want to add indexing of the hidden files, you only need to check the box and hit apply … adding things to the index is far easier than removing them.

the stray home folder entry could be a bug, esp since you deleted it from the rc file and it still shows in the GUI… does it get put back into the rc file after interacting with the GUI?

you didn’t show the output from the excludeFolders command so i assume it’s not longer on the list after your change to the rc file… i would report the GUI bug as a stray reference that cannot be removed.

to remove files from the index, unless i’m blind, the only ways are clear and purge, the former requires a filename and the latter is all or nothing.

happy to be wrong about that, but that’s all i see from the help menu.

jamal-ali1989 · November 19, 2024, 9:56pm

My bad, so I reset everything back to defaults on “File search” GUI. I apply it and the only entry for my home folder shows as “indexed”. However, if I reopen and close it, the “non-indexed” version comes back and stays along with the “indexed” version.
When there are two of these, balooctl6 config list excludeFolders returns kf.baloo: Failed to add exclude folder config entry for "/home/fadmin
However, if I get rid of them by either using “defaults” button or deleting it off the “baloofilerc”, this error goes away and that commands returns literally nothing. As a result, my home folder stays non-indexed. Just as a note, I installed Fedora KDE a few weeks ago and the only “tinkering” I have done was installing packages, flatpaks and changing some settings for my personal preferences.
Regarding baloo, while I would ideally prefer search like Nemo, I can still live with it, if its content search works normally, so I don’t have to use Nemo to verify it works properly every single time. On the other hand, there is no way to “clear” deleted files automatically? So, the index file will keep getting larger and larger unless I purge it from time to time?
Also, If I index a location and then index its subfolder specifically, will this “mess” Baloo up somehow, as in making it index files twice or something. For example, /home/documents are indexed and I index home/documents/newfolder on top of it. Would that be fine?

dzon · November 19, 2024, 9:59pm

No. As far as I know, any type of search of this kind fails. Why install/use an entire file manager like Nemo just for search? That’s why I sometimes use Catfish. Have a look at this post ( scroll to bottom) https://discuss.kde.org/t/trouble-with-dolphin-search/12322

That same “…ony” search will not pop up anything. Not in Dolphin, Milou…whatever.

jamal-ali1989 · November 19, 2024, 10:59pm

I think I am misexplaining the problem. I wanna reiterate that these are all content searches, not filename searches. I am not expecting Dolphin to do something that no other file manager does. There is a .docx file with certain words in it. Using Nemo works exactly exactly like one would expect it to. It finds words whether it is part of a substring or it is a whole word.
I created a new .docx and .odt file for testing. After I confirmed with terminal that they work, I tested it. First, the search seemed to be “case sensitive”, which was weird. I had to use the correct case of a letter for Dolphin to produce the needed result. However, after a few more minutes it stopped being case sensitive. Maybe it was a fluke, I’m willing to ignore that.
On the other hand, Dolphin cannot find words as substrings. Searching “ducer” will not return a file with “Producer” in it. Nemo on the other hand, does support this.
However, I still have .docx files that terminal claims to be indexed that still do not produce certain words that newly created documents work with. I cleared them, re-indexed them, but this makes no sense.

It would be great if Dolphin offered an option to let users choose substring search and also hopefully, normal intuitive search like in Nemo. I don’t wanna have my file manager be separate from my search tool. That’s why I’d like this to work.
Would I be taken seriously if I submitted this as a bug?
I feel like I am expecting something that no one else does. It is surprising to me that most people are fine with baloo compared to regular search. I don’t think I’d be the first one to try content searching.

dzon · November 19, 2024, 11:46pm

Again, dolphin doesn’t do it. Yes, it would be nice if it could. Hence why me, personally, use catfish for that sort of thing and why I created a servicemenu for it. Because it does find it. It doesn’t matter what type of file it is. I guess this type of search is wanted by many but… It’s not a bug of course.

skyfishgoo · November 19, 2024, 11:53pm

using the “defaults” button is the way to go to avoid the “notindexed” entry from coming back with your /home folder… so i would set it to the default, purge your index, then add your choice folder and let it rebuild.

in my experience that default setting indexes exactly nothing… i don’t think this is a bug, but a double entry is.

so if you want to index a folder in your /home, then you add that folder (ignoring the default entry that you cannot delete and apparently does nothing).

indexing does not take forever and runs in the background so if you deleted a lot of content from one of your indexed folders, then just purge the index it and let it rebuild to reduce the size.

i don’t think there is any harm in having an overly large index as long as it’s giving you the search results you want.

jamal-ali1989 · November 20, 2024, 12:09am

The problem is that defaults button makes my home folder “not indexed”. When I change it back to “indexed” and reopen it, the second “not indexed” entry comes back.
I tried to ignore it and let Baloo keep indexing while monitoring it, it goes on for a while and gives me dumb errors like:

corrupted double-linked list
KCrash: Attempting to start /usr/libexec/kf6/baloo_file
Waiting for file indexer to start
Press Ctrl+C to stop monitoring

And regarding purging. Is KDE planning to fix this, so when there are deleted files, it removes them from the indexing list periodically? Or is it just gonna stay there, unless I recreate the index file again?

jamal-ali1989 · November 20, 2024, 12:14am

Unfortunately, I cannot find Catfish for Fedora.
In the meantime, is it worth suggesting this feature? Honestly, it would be so great if they simply copied Nemo’s way of search and integrated it to Dolphin.