Writing to USB stick - (Caja) non-sync'ed?

When doing a copy+paste from hard drive to USB stick, Caja is reporting the operation as complete, even though the system is still dumping buffers onto “chip”.

I discovered this because of my habit of using “sync” manually, and it was taking forever … when I did not expect it to do that.

I expected the copy operation onto USB stick to be “sync’ed” and that Caja would not report until it was fully “stored” on that stick.

QUESTION:

Is there a system setting to force Caja to wait for that copy to physically complete before reporting the operation as complete?

5 Likes

I have had the same problem, especially when using the exfat format on the USB stick. I went back to fat32 because of that problem. I have had it happen occasionally on fat with larger data files.

4 Likes

I think the important distinction here is between the copy operation finishing and the data being fully flushed to the USB device.

Linux can keep written data in the filesystem/page cache, so Caja can finish the copy while the kernel is still writing dirty buffers to the USB stick. sync is basically exposing that difference — which explains why it can sit there for a while afterward.

I’m not aware of a Caja setting that simply makes every copy operation wait for a full fsync()/flush before reporting completion. For critical transfers, manually running:

sync

before unplugging the device is still a good safety step.

You can also target the filesystem rather than flushing everything with:

sync --file-system /path/to/file

And definitely don’t unplug the stick just because Caja’s progress window disappeared. The safest sequence is: copy → sync → unmount/eject → remove.

The exFAT/FAT32 observation is interesting too, but I’d be careful about assuming the filesystem itself is the root cause — the write-cache behavior and USB device/controller can also make a big difference.

3 Likes

caja → buffer → usb

caja says I’m done when file left caja and is in the buffer.

usb write is not completed until buffer is empty.

sync seems to be the tool to verify buffer is empty.

4 Likes

Here is where I have a problem with the disconnect between Caja and writing to the USB stick, or anything USB (including offline backup).

If the device is internal, there is no risk of a user removing the drive before the buffers or journals have been flushed.

For “plug-n-play” (a.k.a. USB-attached), that risk is as great as the human urge to reach for the plug and pull it out! When Caja tells you the copy is done, the casual user’s expectation is that it implies there are no pending buffer writes that need to be completed! For me, however much IT experience and knowledge I have, I still consider that a bug on the part of Caja, no matter what all the designers of Linux might say.

So, is there some kind of configuration file that can be defined such that any USB-attached storage device must also include the specification outlined in that configuration file, above and beyond what might be embedded configurations for device mount defaults?

If I have a glance at the man page for mount, there are the following flags:

  • dirsync
  • sync

Is there a way to configure the mount service (for /etc/fstab and for automounter) such that these two options would apply to any USB storage device, in a complementary way, to any existing device-specific specifications that might be included in the /etc/fstab file?

Is that possible?

3 Likes

I learned long ago to watch the LED on the USB stick and not remove the stick until the LED indicates data exchange is finished.

3 Likes

I agree with everything everyone has previously said about what I need to continue doing before pulling out my USB sticks/drives, but I still feel that Caja should be doing differently.

Hoping that Caja “evolves”.

:slight_smile:

4 Likes

You can permanently disable write-caching and make Caja show the true physical write speed. Tell Linux to use synchronous mounts (sync) for external USB drives instead of asynchronous ones. That way, caja will always be in sync with the usb drive. Issues:

  1. need to modify udev rules
  2. copy will be slower
  3. usb lifespan may be shortened
5 Likes

Thank you, Pavlos. I appreciate your suggestion about the sync part.

I especially appreciate that you underlined the “slow-down” because the sync would be on each block-write, and that is not what I was looking for.

I was only looking for Caja to only report at completion of the actual disk write, not the OS buffer write.

If that sync function were only applied to the last block of every file being transferred, that might be one approach, but my gut tells me that even that might be excessive.

So … I would like that function to operate such that the sync only occurs for the very last block of the very last file being transferred, if that is possible.

Anybody know if that is possible? or if the Caja code is self-contained enough that it could be modified to achieve that functionality?

3 Likes

This is a really interesting observation, I’ve never realised this. I agree with you that the file manager (user-facing) should show the practical state of the copy, i.e., when it’s actually there on the disk. Just wondering now what Thunar would show… Should try that out!

4 Likes

Reported as issue on MATE Desktop GitHub.

5 Likes

I always have Thunar installed. I use both and like Caja better. Thunar’s search function sucks in my opinion. I started using Thunar years ago when Caja wouldn’t show album art unless I opened it in Thunar first

I like if I use a program like Kid3 it moves it to the top of the context menu which Caja does not. Also when you transfer it asks if you want to copy or move. If you answer move it wipes the files at the same time it writes to the new disk. I often move from Caja to Thunar. I use it mostly for moving large music files. For regular everyday work I only use Caja. I like it’s layout, options and search functions better.

I also made my own Icon for Thunar because I don’t like the default icon.

4 Likes

Short answer ‘NO’

Long answer:
As far as Caja is concerned, when it writes the last bit, copying is finished. Caja is totally oblivious to the existence of a whole OS bufferstack that is doing the real writing in the background.

3 Likes

What happened to me was half the files looked like a blank sheet of paper. The names and extension mp3 was there, but the data was not. Some I left in 15-30 minutes. I figured it out with gnome-disks, it would show a spinning arrow even though Caja had said the file transfer was complete.

I used three different laptops which made no differnce. Two were Ubuntu Mate and one was Sparky Linux Mate, so all used Caja. Stlll I had the same results when I tried Thunar.

3 Likes