Dolphin struggles to load directories which contain many files

When accessing my directory containing most of my photographs (which mostly aren’t in subdirectories, because I use DigiKam to tag them instead)

I can wait almost 5 seconds to access the directory, during which Dolphin entirely freezes (the STOP kind of hang). Is anyone else with a similarly large directory experiencing this? If so, have you any potential solutions for me?


Relevantly, is there a reason that it doesn’t seem to load them dynamically? (To ensure that the user doesn’t accidentally freak about a directory or file being absent, perhaps?)

7k files does not seem that many. Maybe try to temporarily disable previews when you’re in a different folder.

1 Like

@inv-er5kar,

I didn’t think it was many, considering how many a photographer must take during their work. I know some who just leave them completely unorganized…

Anyway, I’m embarassed to say that I didn’t consider disabling Preview, but relieved to say that it’s unfortunately not the cause. It didn’t change anything at all.

Maybe a stupid question, it’s not the folder name, what is it “(titl…” in your preview? Several folders? Looks very different here!

1 Like

@inv-er5kar,

It’s just Pandoc-formatted JSON5[1], like Revision - Stack Overflow demonstrates:

`{key: "name"}`{JSON5}.fileExtension

with a prefixed tab so that I can use checkboxes and hiding dots without making the file list unreadable:

. [x]	`{name: "English Name",	id: "UNIXEpochTimeToBase36"}`{JSON5}.fileExtension

I’ve duplicated the content into a directory with mere ASCII characters as its name, and it made no noticeable difference:

  1. Screenshot_20240512_224302

Thanks for the idea, though.

It has occasionally been a problem (like in KRename’s plugin window):

XD


  1. json5/README.md at de344f0619bda1465a6e25c76f1c0c3dda8108d9 · json5/json5 · GitHub ↩︎

Please report a bug with details so that somebody else could reproduce your situation.

Number of files (7k), total size(32GB), numbers of directly average file size(~4.5MB), what kind of drive (SSD or HDD) and filesystem (ext4, smb, nfs…?).
And your distro + versions.

If you have the time and some technical skill I would suggest you to try to observe the 5s freeze with hotspot and extract a flamegraph matching the freeze time-window. That method will for sure diagnose the issue origin.

1 Like

@meven, many thanks. I’ve reported 486956 – Dolphin takes ≥5s to load ≥52 GiB, 7x10^3-file directory. and, using https://koji.fedoraproject.org/koji/rpminfo?rpmID=37327082, completed hotspot/README.md at 35871adeec5adc882f2134804058b1071886bcc5 · KDAB/hotspot · GitHub.

However, I’m a little confused by how to identify the relevant section of the performance record, and consequently how to extract it:

Are you able to assist? You can grab the perf.data from 486956 – Dolphin takes ≥5s to load ≥52 GiB, 7x10^3-file directory., if that’s of use.

Are you able to assist? You can grab the perf.data from 486956 – Dolphin takes ≥5s to load ≥52 GiB, 7x10^3-file directory., if that’s of use.

The perf.data is specific to your distro can’t be exported unfortunately only to the same distro with the same exact packages.

I am interested in the “Flame Graph” tab mostly. It will tell what was dolphin doing.
It does require debug symbols, it seems you have them or at least a bunch.
Since you are using fedora 40, you can simply once launch gdb dolphin and it will download them for you: type once c, wait a bunch during download and once dolphin opens you can close it, the debug symbols are downloaded.

At least we can see the dolphin main-thread very busy, that would make it unresponsive in the Time line".

1 Like

Awesome you took the effort to use hotspot !

This isn’t very easy, I am aware.

1 Like

@meven, thanks for that. Needed to do sudo dnf debuginfo-install dolphin-24.02.2-2.fc40.x86_64 - it was indeed absent. I’ve consequently recreated the performance data. However, unfortunately, I can’t provide the flamegraph image due to "Failed to export flamegraph: image is empty". · Issue #637 · KDAB/hotspot · GitHub.

is the perf.perfparser file of any use?


:smiley: You put in effort to assist me. It’s fair. Regardless, it’s not going to be solved unless I put some work in, and acquiring the data was also surprisingly intuitive. You’ve recommended a good tool.

Then don’t export it, if you can have it display take a screenshot when displayed, or multiple so i can see more or less the whole thing.

1 Like

@meven, I’ve managed to get it to work using "Failed to export flamegraph: image is empty". · Issue #637 · KDAB/hotspot · GitHub. I’ll post that here and link it in the bug report, since non-Harmony Bugzilla isn’t a great place to display images:

1 Like

Nice from the flamegraph we can see a lot of time is spend checking the group() of the files and making syscalls to get files’ group.

Thanks, it shows a regression for an evolution I did.

6 Likes

yes i have had this issue for as long as i have used dolphin. it seems that gnome/gtk has optimized something much better and doesn’t scrape all the directories every time you open them like kde does. i can literally hear it trash through my hdds every single time i open something with couple thousands of files but gnome has the file lists cached so that never happens and the listing opens instantly.
these programs really need to promote their support sections better so people can actually report issues like this

how? i saw this recommended elsewhere but you literally can’t, at least not in dolphin settings and no scrolling the 5 mib limit wont turn it off but instead removes the limit completely and generates previews for everything. i also read that each file gets probed for things like type which is part of what makes the process so god damn slow
Screenshot_20240514_001542

@meow,

Performance

Considering that @meven correctly pinpointed my issue to a specific attribute column’s visibility, and the fact that it was due to a recent commit, I expect that your issue is different. However, using the information from the previous discussion, would you be able to provide a flame graph for your issue, as I have? I ask because that would demonstrate, without doubt, whether it differs.

Assistance

Are you aware of every KDE application’s “Help” pages?

These display ways to report bugs. However, I agree that they should link to their relevant application’s section on this Discourse forum. I’ll propose it, if you don’t want to first (if you do, please link that here).

Window Screenshots

If you utilize spectacle’s “Active Window” mode, you shan’t need to crop screenshots of specific windows:

uhh turns out that getting the debug symbols for debian is a bit more complicated but i’ll try figure it out as this has been driving me insane a long time

1 Like

@meow, yeah, that’s one of the reasons I went for Fedora rather than Debian. Does gdb’s debuginfo option work? That might at least supplement it if you can’t install the relevant *-debuginfo packages:

RokeJulianLockhart@sayw4i:~$ gdb dolphin
GNU gdb (Fedora Linux) 14.2-1.fc40
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from dolphin...
Reading symbols from /usr/lib/debug/usr/bin/dolphin-24.02.2-2.fc40.x86_64.debug...
(gdb) run
Starting program: /usr/bin/dolphin 

This GDB supports auto-downloading debuginfo from the following URLs:
  <https://debuginfod.fedoraproject.org/>
Enable debuginfod for this session? (y or [n]) y

Guidelines and HOWTOs/Debugging - KDE Community Wiki might be of use, if you’ve not read it.

i figured out that there was a different repository that needed to be added to get the symbol packages so thats sorted now. is this graph any good or do i need to install more stuff?

oh also this is from a hdd with ~8000 files of which most are pretty small like less than 1 mb but few can be around 30. running on debian unstable

1 Like

@meow, the “??” prefixes on some of the stacks would demonstrate to me that some symbols are absent, but I’m novice at this. You might want to check that all relevant packages are installed (do try running gdb with debuginfo enabled, if you haven’t - it works well). Except that, I’ll defer to someone else with more experience.

Regardless, thanks for the report. If your issue is similar to mine, the more corroboration, the better!