Making KIO copy many files fast

Community Blogs
1

There is a bug in KDE's bug tracker that is almost as old as some of our contributors: bug 342056, "Ridiculously slow file copy (multiple small files)", reported by Alexander Nestorov back in 2014. The report is blunt: copying a 15 GB folder of roughly 3 million small files took 5 to 10 hours in KDE, versus about 20 minutes with rsync. One commenter summed up the mood:

This is a companion discussion topic for the original entry at https://blogs.kde.org/2026/07/28/making-kio-copy-many-files-fast/
15 Likes
2

Excellent work Meven!!

5 Likes
3

OMG yes, I’m so glad. It always felt so silly to have to resort to cp for doing small-file transfers. thank you so much @meven :purple_heart:

1 Like
4

Why FAT/NTFS are out of the fast path?

5

Mostly out of caution, and because they require special handling for file names.

It could change.

The concerned MR is still under review file: add a batch-copy command to the file worker (!2282) · Merge requests · Frameworks / KIO · GitLab

Quoting the code

    // The batch command runs synchronously in the worker and is not killable mid-flight, so reserve
    // it for a local, responsive destination. Skip FAT/NTFS, whose name/symlink quirks the per-file
    // path handles.
3 Likes
6

For me a killer feature.

Thank you!

1 Like
7

This is so sweet. You the man Méven!

1 Like
8

Did you not pursue assembling the list of files in the worker to reduce the size of commands being passed?

9

That list was already being built by the copyJob to have initially the total size to copy, then the files are sent in batch of 512 files, might be tweaked to account for total size.

The worker here is already the backend/protocol implementation, as opposed to the job/event-loop/application calling it.

All is there to see:

Still lacks review, a goal of my blog post was to gather some reviewer attention.