Dolphin struggles to load directories which contain many files

yeah i originally installed only the dolphin symbols but made a better one with most stuff in it. the gdb stuff doesn’t seem to apply to hotspot for me but the symbol packages do the thing, just gotta figure out what they all are. looks like a lot of time is spent probing file mime types. i think most file managers just assume by extension as its faster than actually scanning the files for what they are. gotta go sleep soon but i’ll check any replies later

1 Like

@meow, no need to worry about that. I don’t think anyone on this forum expects conversations to be synchronous.

1 Like

You mentioned you are using debian.
Can you specify the versions you are using ?

If dolphin 24.02+ does not reproduce the issue, this is unlikely to get fixed for older versions as those in debian. Then report the bug to debian as per your distro policy (Debian -- Debian BTS - reporting bugs).

Looking at the flamegraph, it seems a lot of time is spend fetching files mimetypes. Do you have some mimetype filtering on ? Do you have dolphin 23.08 and hide hidden folders on ?

yes i have had this issue for as long as i have used dolphin.

Is it dolphin window freezes with the group column visible when loading a 7000 image files directory with Dolphin 24.02 ?

Probably not.

Then please define Precisely your own issue.

it seems that gnome/gtk has optimized something much better and doesn’t scrape all the directories every time you open them like kde does. i can literally hear it trash through my hdds every single time i open something with couple thousands of files but gnome has the file lists cached so that never happens and the listing opens instantly.

Dolphin/KIO does cache its file data fetched and share it with other dolphin views within the same window, caches previews… Then it is about how much data is read by nautilus vs dolphin, are they both displaying previews ? are they the exact same directory ? Are you in a GNOME session ? Is nautilus or dolphin computing directory size ? Which options are enabled ?
I know nautilus does not have directory size feature. That and previews are the heaviest data to load from disk.

Nice of you to make a flamegraph, that’s the best input users/testers can provide to devs to resolve performance issues.

1 Like

oh damn it is 23.08.1. i thought sid would not lag behind much. now that i know its old i got 24.02.2 from flatpak but its still very slow.

and yes hidden files are hidden to keep the home directory nice. not sure what you mean with group column but if usergroups, that is not enabled. only things enabled are name, size and modified and from there its set to show latest files on top.

i also noticed that it does indeed cache stuff but it wont take very long before it wants to do the whole expensive process again. i could also confirm that it is storing these in ram as running the drop_caches 1 thing caused it to do the slow thing again immediately. also i always turn off all previews because i use the details view mode which would make them so small that generating them would only be a massive waste of resources. and no im not running a gnome session but in the past i have tried other things. i moved to kde so i could try out wayland and kde was the only environment that supported it and i could get to look almost like my xfce setup.

also since you mentioned directory size being a very heavy thing, i reinstalled pcmanfm because i remember it being pretty fast and it still is but i never paid attention to this little detail that it will not show directory sizes, only the sizes of the files in the current directory. if thats what causes the issue is there any way to make dolphin not show directory sizes but still show file sizes?

1 Like

@meow, have you tried decreasing the value of

or (if that doesn’t work) using

if you aren’t, currently? Unfortunately, the preference isn’t directory or file-specific.

And in the next version of dolphin 24.05 that comes out end of month, you will be able to turn off any directory size calculation.

2 Likes

@rokejulianlockhart
I realize with your latest flamegraph (https://bugsfiles.kde.org/attachment.cgi?id=169499) you have the same issue as @meow additionally: a lot of time spent determining the file mimetype in the wrong place that causes dolphin ui to be unresponsive even with my last patch.

I think it is worse treating this case as a separate bug.

kfileitemmodel: let worker fetch mimetype (!776) · Merge requests · System / Dolphin · GitLab should help that.

2 Likes

Thanks, @meven. However, I certainly don’t notice the issue if I do experience it (albeit probably because I’m using an PCIe 4.0 NVMe SSD). Always glad to see more asynchronicity, though.

that would definitely help speed up the checks. my files are on sata hdds with a filesystem setup that would give someone nightmares so it seems to take almost a minute for it to have a fully ready listing of those files.

also i notice someone in the bug above mentioned gwenview which seems to be an image viewer. i switched my image viewer to eog for similar reasons. no idea what they do but i can open a massive image directory on it and the image will be visible instantly and the entire directory browsable through the image viewers next/previous image feature without any disk thrashing. i tried many viewers before settling on eog and none of them could load the entire directory instantly and without tons of disk activity as if it was preloading the entire directory

1 Like

@meow, that’s definitely worth asking them about. I’d be interested in a response by the developer - it might be that they interact with your FS directly for something special, like Everything does on NTFS.