File permissions change when transfering between external drives and laptop
I noticed a few years ago that when I transfer files back and forth between my laptop and my external drive all the files that I have transfered have changed permissions.
I format all my external drives as exFAT so I can use larger files.
Why does this happen?
Is there a better way to keep the file permissions intact when transfering files back and forth between external drives?
The test file: Fantastic Fungi (2019).mkv
+++++++++++++++++++++++++++++++++++++++++++++++++++
This is what the file permssions looks like before I transfer it to my external hard drive
ls -l
-rw-r–r-- 1 user user 577761580 May 2 2024 ‘Fantastic Fungi (2019).mkv’
+++++++++++++++++++++++++++++++++++++++++++++++++++
This is what the file permssions looks like after I transfer it back to my laptop
ls -l
-rwxr-xr-x 1 user user 577761580 May 2 2024 ‘Fantastic Fungi (2019).mkv’
When I right click file permissions dialogue box. The “Allow this file to run as a program” is ticked.
+++++++++++++++++++++++++++++++++++++++++++++++++++
The way have overcome this is to run a simple one liner to reset the permissions for directories and files.
Open a terminal in the directory of the folders and files you want to change
All directories will be 775. All files will be 664
find . -type d -exec chmod 0755 {} ;
find . -type f -exec chmod 0644 {} ;
Directory permission 0755 is similar to “drwxr-xr-x”
File permission 0644 is equal to “-rw-r–-r–-“.
-type d = directories
-type f = files
+++++++++++++++++++++++++++++++++++++++++++++++++++
Exfat does not support permissions, so when it gets moved to the drive, that information is lost.
If permission information is important to you and compatibility with non-linux devices isn’t, you can reformat the device as ext4 to support all linux features.
Thank you SavvyWolf
This is particularly annoying when I have to upgrade my distro and all my files have to be moved to an external drive.
Unfortunately some of my files are up to 10Gb. thats why I stayed with exFAT.
I will certainly try Ext4 on my external drives.
10Gb is not a big file relatively speaking - both ext4 and btrfs (for example) can handle 16TiB and larger. If this was your only reason for choosing exFAT then you can definately migrate.
Hey IanTwenty
Thank you
ext4 seems the way to go for me
Try compressing it in to a tar, this will save permissions.
you can also use backup tools like “Pika backup” (borg backup).
Thank you Eideen
I have never used any back up programs, Maybe I should consider it.
Both Borg and Pike-backup are in the offical repos (extra).
I shall check them out
If it’s an external SSD, I like to format my drives as f2fs, which is a filesystem designed for flash memory, so it might be a bit faster and last longer than ext4. But that’s just personal preference and ext4 should always work fine.
Hey SteveTech
I dont use internal or external SSD’s.
It maybe because I am an old greybeard, but I prefer to suffer the slight loss in speed of a HDD, so I can dd/erase/wipe them and reuse them again and again.
Though I have been using “disktest” to erase SSD’s and HDD’s recently and it has been working great. Much faster than zeroing with dd /dev/zero, shred or wipe.
https://crates.io/crates/disktest
https://github.com/mbuesch/disktest
However, I digress, I had not heard of f2fs before.
Ive been having an intersting read online.
It appears that it is a default for android phones.
I do have a few SSD’s laying around from when I replace them with HDD’s. So I will test f2fs on one of those.
Thank you
This is absolutely normal. FAT/exFAT do not support unix permissions (let alone Linux ext4’s any special flags etc). So each time you copy files there, the permissions and all other flags are lost or get bad in general.
To save your permissions you have two options:
-
Zip/targzip or xz your linux files before you copy them on your fat drives. Preferably on files that overall aren’t larger than 1 gb, just to avoid other weird problems.
-
Use ext4 on your external drives.
Thank you Eugenia
Compressisng the files sounds a great idea.
I have in fact compressed a 12Gb file that I split into 10Gb chunks that still decompresses without problems.
This is particularly annoying when I have to upgrade my distro and all my files have to be moved to an external drive.
Unfortunately some of my files are up to 10Gb. thats why I stayed with exFAT.
I will certainly try Ext4 on my external drives. I will test it this week
-
exfat or fat32 is great for interoperability between linux and windows but has limited functionality under linux.
If you’re using your external drive only under linux, I suggest switching to a filesystem that works better with unix like permissions and special bits.
Also, like others, depending on your use-case I would suggest something with journaling like ext3 or ext4. If you happen to power of your system while writing something to that drive, the fs does not get corrupted/can automatically recover.
For backups with rollback maybe a FS with copy on write and automatic compression like btrfs or zfs would be better.
With btrfs borg backups allows you to create incremental backups of btrfs subvolumes. I use it to backup my home, etc and /subvolumes on my “backup server” (old pc with two raid1 hdds).
I have a friend who administeres backups for his company (afaik ~100-200GB delta per week) and he swears by zfs. I found btrfs simpler though.
Hey pitiable_sandwich540
Thank you
I dont know why I have been so focussed and stuck on exFAT for all these years.
It must have been something I read somewhere that led me to it.
from all the decent feedback i have gotten on here, ext4 seems the best way to go.
I should have known this being a linux user for over 20 years.
exFAT is a Microsoft creation that (unsurprisingly) doesn’t understand or preserve Linux-style file permissions. Neither did any of the FAT varieties before it. So the permissions on the files when you get them back relate to the mount options you pass to the exFAT drive (in this case, you probably want to set
dmaskandfmask), or the permissions on the directory it’s mounted to.If you don’t want to twiddle with mount options, you could reformat the external disks using Linux-native filesystems like ext4, but you’ll lose the ability to mount them on Windows if you do that.
Thank you nyan
I will look into the dmask and fmask mount options.
Definitely a no! in regards to attaching my drives to any windows machine.
The last windows machine I turned on was a Windows 95 machine when they first came out.
I thank god, That I wouldnt even know how to turn on a windows 10 or 11 surveillance machine.
but you’ll lose the ability to mount them on Windows if you do that.
One viable solution to that, is to mount that drive as SMB share, best of the two worlds !
ExFat doesn’t support file permissions so a default value is used, I believe you can only change the permissions at mount time
Thank you Hack3900
Thunars “allow this file to run as a program” is generally bugged. Always set to on for me on files like xml, md, json, so they execute on single-click instead of opening the default action.
chmod -xdoesn’t change that checkmark either or only sometimes.Btw, any file manager that is similiar in features (custom actions) and look but less crashy.
