My NAS/File Server has gone through multiple iterations after the years so it’s interesting to know the reasons for each iteration and the reason why I’m migrating it once again.
First File Server (Ubuntu 11.10)
My first NAS implementation was an old desktop PC running Ubuntu 11.10 that was running samba. I didn’t know about using RAID/mdadm at that time so it used a single 2TB hard drive that would mirror itself with rsync to another 2TB hard drive nightly.
I remember having to tune samba to get gigabit speeds (~110MB/s read/write) but this was a smaller issue when compared to the permissions. The biggest issue I had was that new files and folders that were created did not respect the sticky guid (group id). At times I would need to correct all the group permissions as you would randomly find files that did not have the right permissions.
For the most part this worked but I grew tired of having to fix the permissions. At this same time I was working in a IT department supporting Windows Servers/domain. I couldn’t help but notice that Windows Servers didn’t have issues as file servers (excluding Windows Server 2003/2003 R2 Cut + Paste not using destination permissions)
Enter Windows 2012 R2 File Server
Whilst I was studding my university degree we had access to Microsoft Dreamspark (I think it was called), which gave me access to free licences for Microsoft Software. This was so we could have access to Visual Studio as this was pre Visual Studio Code. However I downloaded ISOs and keys for Windows File Servers and Microsoft SQL Server.
Also at this same time the underlying file server was also upgraded and all major services were converted to Virtual Machines using KVM. So this meant that the file server was transitioned to Windows 2012 R2 using virtual disks for the storage.
This has worked exceptionally well as a NAS and also sharing over SMB my now Plex media library. However Windows Server 2012 R2 has an EOL (end of life) of October 23 2023.
This lead me to think of what to do. Build a new Windows Server or change up to something else. Client wise there is a mixture of Windows, Mac and Linux clients connecting over SMB and NFS protocols (However Windows 2012 R2 was only handling SMB).
Move to Samba Shares
I decided to move to all the data from a Virtual machine to the KVM host for a multitude of reasons.
- Could share same data over SMB/NFS
- Raw files on ZFS array instead of having to use large virtual disk files (10TB)
- Don’t have to pay for new Windows Licence ($689AUD) + Cals (honestly was not going to buy CALs anyway 😆)
Copying this required a multitude of steps as all of the Windows data is backed up every night to a Synology NAS. The steps are listed below:
- Perform offline backup of all data (the most important step!!!)
- Create new ZFS file system for data
- Disable Windows Server scheduled backup of data to Synology NAS
- Copy Plex data (10TB) from Windows Server to KVM host
- Update backup scripts to backup from KVM host to Synology NAS
- Perform full backup (10TB) to Synology NAS
Although simple this took many days to complete due to the amount of data to move over a gigabit network.
Surprise File Modification Time Issue
I thought all was well and I turned on the automatic backups again. These are ran via scheduled GitHub Actions however this failed after 6 hours. What I didn’t realise is that when running the backups over SMB via rsync the modification timestamps were not preserved. This meant that the entire 10TB was trying to be updated every night.
So if the file had a last modified timestamp of 2023-05-06T19:57 but the backup was ran at 2023-05-07T00:00:00 the resulting timestamp on the backup server was 2023-05-07T00:00:00.
A quick google showed other people having this same issue on stackoverflow but trying their suggestions, prompting ChatGPT and the Samba docs I could not get it to work correctly.
A quick test using NFS instead of SMB for the backup resolved this issue. It was probably the right decision to move this but it meant that I then had to move my backups for all my servers to using NFS instead of SMB as they are defined in Ansible script’s (so the mount is part of a role).
As you can guess this meant deleting all the backup data from the Synology NAS and re-copying all 10TB again. However after this the entire backup for this server was completed in minutes when the automatic backup was re-enabled.
Job Done…Right?
For all intensive purposes this was all completed and running as expected. Backups were running, I could add new files over Samba (over ethernet…hint) and Plex could mount and access the files over NFS instead of SMB.
However a serious issue was about to pop up that would change my entire architecture. Stay tuned for my next blog article about Samba over WIFI for a full breakdown.
Leave a Reply