So, I have a USB drive in exfat file format that has been working reasonably well up until now.
Today, I have attempted to copy some files into it. More specifically, it is a combination of epub, mp3 and zip files (compressed mp3 files) inside it. There doesn’t seem to be anything wrong with these individual files. In total, there are around 950 files in 16 GiB.
When I try to copy it, it copies around half of them at super fast speeds, but it becomes extremely slow around the middle and the speed dips to 50-100 KiB / second.
If I let it continue, it would take over 2 hours just to copy such a small amount, which is ridiculous. I tried to copy the file that it struggled to copy on its own and it happened super fast.
What could cause this? I compressed that entire folder into a zip file and it copied it under 10 minutes. This is really annoying and doesn’t make sense to me at all.
I doubt that this is a Dolphin error - to verify it, try copying using cp -r or rsync and compare speeds.
Also check the drive for errors, and you could also test with NTFS or FAT32 on another filesystem or USB drive.
16GB is definitely going to obliterate any caching - and that explains the initial ‘fast’ copying, and the drive can be fragmented too.
USB drives can slow down dramatically with many small writes, because of wear leveling and ‘write amplification’ - zipping reduces the number of writes, which is far more efficient.
I learned with Windows 98 it’s best to zip files up before moving them to/from your USB.
Hi!
Sometimes this can happen due to hidden files that are not visible by default. You can show this by pressing ctrl + h. Also you can try to copy your files manualy using cp in terminal.
How can I check the state of the drive and how much read/write speed I should expect of it? One of the system tools says that it “passed” the health check, whatever that means.
Thank you for the suggestion, but this is definitely not it. It is a set of personal folders and I always show hidden folders by default.
It just seems insulting to me that a 50 MB zip/mp3 file is transferred with a speed of 110 KiB/s
Why would you be personally insulted by a classic USB file transfer performance issue?
Fast initial transfer slowing down dramatically mid-process, individual file transfers working fine, and zipped files copying quickly all point to file system overhead and potential USB drive issues.
It’s about 950 individual files. For EVERY file, an entry in the exFAT file allocation table is made, disk space must be found, metadata must be updated, and then your Write Cache will be exhausted and so now you must wait for each file to be physically written to the slow USB drive before accepting the next file.
That’s the WALL you’re hitting, where speeds go down from MB/s to KB/s sometimes.
If the drive is anywhere near full, then writing will get even slower with many more random writes.
It’s just the limitations of low-cost storage… The same happens with fast NVMe drives also (depending on how you use them) where many cheaper drives can write at ridiculously fast speeds of 7000 MB/s, but then the ‘sustained write’ speed after that can vary dramatically.
Please try this, before blaming dolphin. This will tell whether the issue is lower in the stack.
Normally dolphin is only slightly slower than those.
Indeed.
exfat (the most USB stick filesystem) inheriting for some of the poor design of FAT32 is notoriously slow, and the Linux implementation may not be optimal.
IIRC it can have quadratic complexity just writing metadata. That is, the more files to copy, the square longer it takes…
You can try to format the stick to ext4 and compare. IIRC you will get much better speed. This won’t be compatible with other OS though. So you will need to reformat probably.