I'm trying to demo vranger so I can push it through to get the licensing approved as our primary backup solution. We're running 3 esxi5.1 hosts in a DRS cluster hosting around 45 VM, and another 5.1 host being used as a dev environment before we move the server into production on the cluster. Anytime I try to back anything up on host 4 it fails with Error 2129. Yes I know this usually points to a space issue, but that is not the issue I assure you. The nfs share I'm backing up to has 13TB free on it. There are a few servers on the cluster that are giving the same issue, but the rest so far are backing up just fine with no errors to this same nfs share. The settings for the nfs share are exported as follows
/mnt/data *(rw,sync,no_subtree_check,insecure,all_squash,anonuid=0,anongid=0)
host 4 was originally still on esx5.0 so I updated it to 5.1 to match the others, still no luck.
I migrated the servers off of 4 over to the cluster, no change.
I changed the datastore they were located on...no change.
I spun up a brand new server 2012 from scratch and installed vranger on it....no change.
I shut down the servers and rand the update vmware hardware...no change.
I've tried both synchronous and asynchronous on the nfs share...still fails
I created a new nfs share on a separate server, and no dice...
There is absolutely nothing I can see different between a server that succeeds, and one that fails. It creates the folders for the server, creates the initial .var file for the backup snapshot. It completes the snapshot, but when it goes to commit the backup it fails
Below is the output from one of the failed backup attempts
[2013-11-15 13:07:23.815]: vRanger Backup & Replication - v6.1.0.35402
[2013-11-15 13:07:23.815]: Selected Options: Check destination for free space. | Compress backed up files. | Enable Cataloging. | Selected SpaceSavingTech: None | Included Disks [30, 11, 15, 29, 50, 28, 24, 38, 9, 8, 6, 41, 1, 17, 22, 45, 3, 34, 40, 64, 58, 23, 52, 54, 32, 26, 57, 44, 2, 10, 55, 62, 5, 16, 47, 19, 53, 18, 43, 63, 35, 42, 27, 60, 31, 12, 14, 39, 36, 7, 33, 51, 48, 37, 61, 49, 59, 13, 25, 56, 46, 4, 21, 20].
[2013-11-15 13:07:23.815]: Task for virtual machine Internal was queued.
[2013-11-15 13:07:43.252]: SourceVm:Internal | Uuid:42052cb9-9192-630b-7fb9-d25fcccd01e0 | VC:Kronos, Host:esx03 [ESXi 5.1.0]
[2013-11-15 13:07:43.252]: Beginning backup task for Internal
[2013-11-15 13:07:43.252]: Starting task validation.
[2013-11-15 13:07:43.252]: Connection to esx03 was properly validated.
[2013-11-15 13:07:43.252]: esx03 is properly licensed.
[2013-11-15 13:07:43.252]: Test connection to repository OSB2 starting...
[2013-11-15 13:07:44.514]: Test connection to repository OSB2 successful!
[2013-11-15 13:07:44.951]: Ending task validation... success!
[2013-11-15 13:07:44.951]: Beginning initialization of backup information.
[2013-11-15 13:07:44.951]: Retrieving the tasks parent information.
[2013-11-15 13:07:45.184]: Retrieving save points for any full backups associated with this job.
[2013-11-15 13:07:45.184]: Finished initialization of backup information successfully.
[2013-11-15 13:07:45.184]: Determined backup type for task: Full
[2013-11-15 13:07:45.184]: Gathering credentials for Internal completed.
[2013-11-15 13:07:45.434]: Checking for VMotion completed.
[2013-11-15 13:07:45.714]: Gathering data for Internal from API completed.
[2013-11-15 13:07:46.805]: Gathering data for esx03 from API completed.
[2013-11-15 13:07:49.206]: Retrieving the VM BIOS configuration completed.
[2013-11-15 13:07:53.024]: Creating a snapshot for vRanger completed.
[2013-11-15 13:07:54.084]: Gathering data for Internal from API completed.
[2013-11-15 13:07:55.393]: Local machine is a virtual machine.
[2013-11-15 13:07:55.393]: Backup task will be attempted using a Machine-based VDDK network connection.
[2013-11-15 14:05:43.344]: Backing up disk 'vix:r:kronos:902:[Isilon_Node_06] Internal/Internal.vmdk:moref=vm-350:snapshot-384:6' completed.
[2013-11-15 14:05:43.360]: Backup failed: Error: 2129 - can't write nfs:/mnt/data/Backups/Internal_42052cb9-9192-630b-7fb9-d25fcccd01e0/Internal_20131115_130743_0825d12e-ebb7-4a4f-bfcf-d252697de99d/.9f2c79a6-146a-4a73-9586-af7c2ea84cbc/.9f2c79a6-146a-4a73-9586-af7c2ea84cbc.var
[2013-11-15 14:05:43.360]: The failed backup task will be retried.