NFC connectivity troubleshooting steps

KB ID: 1198
Product: Veeam Backup & Replication
Version: All
Published: 2011-08-26
Last Modified: 2022-03-07
Languages: DE | IT | FR | ES
Get weekly article updates
By subscribing, you are agreeing to have your personal information managed in accordance with the terms of Veeam's Privacy Policy.

Cheers for trusting us with the spot in your mailbox!

Now you’re less likely to miss what’s been brewing in our knowledge base with this weekly digest

error icon

Oops! Something went wrong.

Please try again later.

Challenge

A job fails with an error related to NFC (Network File Copy) connectivity. For example:
Error: NFC storage connection is unavailable.
Failed to create NFC download stream.
Failed to create NFC upload stream.
1198NFCErrorExample

Cause

The cause of most NFC errors fall into 4 primary categories:

  • DNS
       (The proxy or Veeam server cannot resolve the ESXi host)
  • Port (902)
       (The proxy can resolve the IP, but port 902 is blocked)
  • Permissions*
       (The account specified in [Backup Infrastructure] for the vCenter does not have permissions)
  • File Locks
       (The file Veeam is trying to read is locked within the vSphere environment)

*If the account that Veeam Backup & Replication is using to communicate with the VMware Environment has granular permissions set, please confirm all permissions are set according to the Granular Permissions Guide.
 
Due to the nature of NFC as a client<->server type connection, there is also potential that all things may be correct on the Veeam client-side, and there exists an issue within the VMware Infrastructure itself.

Solution

Troubleshooting Tip
When investigating this issue, it is advisable to reconfigure the job to use a specific proxy. This is done to isolate the issue to a single proxy being the culprit. This also makes it easier to identify which server the logs will be found on.

Logs to Investigate

To investigate the potential of DNS, Ports, Permissions, or File Lock issues, you will need to review the Agent process logs.

  1. Navigate to the following location on the VMware Backup Proxy:
    %programdata%\Veeam\Backup
    
  2. Open the folder that matches the name of the job that is having an issue.
  3. Within this folder, you will need to find the agent log for the specific VM.
    • For a Backup Job:
      Agent.<JobName>.Source.<VMName>.log
      
    • For a Replication Job:
      Agent.<JobName>.Source.<VMName>.log
      
      Agent.<JobName>.Target.<VMName>.log
      
  4. Search for the following:
    Creating NFC download stream
    
  5. Here you will want to look for the following items that could point to the cause of the error.
    1. DNS
      If there is an issue with DNS, the following error will be found.
      nfc| Resolving host name (esx1) to IP address...
      nfc| Resolving host name (esx1) to IP address... Failed.
      
    2. Port
      If there is an issue with ports, there will be an error like this:
      nfc| Establishing connection with the host [esx1]. Port: [902]. Failed.
      
    3. Permissions
      If there is an issue with permissions, there will be an error after the following:
      nfc| Sending authd message: [SESSION ID NUMBER].
      nfc| Sending authd message: [Name of Host].
      nfc| Waiting for the authd reply message...
      
    4. File Locks
      If the file is locked, there will be an error like this:
      nfc | Starting file download. File: [[datastore] Test/Test.vmx]. Failed.
      ERR |Failed to start file downloading from host [esx1]. File path: [[datastore] Test/Test.vmx].
      ERR |DiskLib error: [13].The file is locked or in use – File open failed: File not open
      --tr:Failed to start file downloading. VMFS path: [[datastore] Test/Test.vmx].
      

Troubleshooting

DNS Issues (Most Common)

The proxy must be able to resolve the hostname or FQDN of the ESXi hosts in the Virtual Infrastructure.

Port Issues

Further testing can be done using the following PowerShell command. The destination should be the hostname/FQDN/IP shown in the logs for the host.

Test-NetConnection -ComputerName "<destination_host>" -Port 902
If the Proxy cannot reach port 902, check all applicable Firewalls between the server and the ESXi host.

Permissions Issues

When a vCenter Server or standalone ESXi host is added to the [Backup Infrastructure] section of Veeam Backup & Replication, the user assigns an account to be used. That account should have sufficient privileges to all objects related to the VM being backed up.

If the account that Veeam Backup & Replication is using to communicate with the VMware Environment has granular permissions set please confirm all permissions are set according to the Granular Permissions Guide.

File Lock Issue

In certain circumstances, the ESXi host may establish a lock on VM files, preventing Veeam from accessing the file.

For more information, review the following VMware KB article regarding file locks:
Investigating virtual machine file locks on ESXi hosts (10051)

In some cases, Restarting the Management Agents on the ESXi host may correct NFC connection issues.

Isolation Tips

  • Try a different proxy. (Isolates issue to specific Veeam Proxy)
  • Try vMotioning the VM to a different host. (Isolates issue to specific ESXi host)
  • Try Storage vMotioning the VM to a different datastore. (Isolates possible missing permission at datastore level)
  • Try adding the ESXi host the VM is on as a standalone host to Veeam, then create a backup job to pull the VM from the standalone host.
    (This will help isolate if the Management Agents need to be restarted on the ESXi host)
    Remember to remove the job and standalone host from Veeam's configuration after the issue is resolved.

More information

The proxy being used to process a VM can be identified by opening the “Task” log specific to the VM from inside the folder named after the job in %programdata%\Veeam\Backup. Searching the Task log for the words “starting agent” will allow for the identification of which server performed the task.

Further information can be found in this forum post: http://forums.veeam.com/veeam-backup-replication-f2/failed-to-create-nfc-download-stream-t2991-15.html#p69806

The NFC connection requires the following:

  • DNS Resolution of the target host
  • Port 902 is open to/from Backup Server/Proxy to ESX(i) host
  • Permissions to download files via ESX(i) host and/or vCenter

 
An issue with Port 902 may represent an issue with a firewall on the ESXi host, Veeam Proxy, or the connection between the two.

Click here to send feedback regarding this KB, or suggest content for a new KB.
To report a typo on this page, highlight the typo with your mouse and press CTRL + Enter.

Spelling error in text

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Thank you!

Thank you!

Your feedback has been received and will be reviewed.

Oops! Something went wrong.

Please try again later.

KB Feedback/Suggestion

This form is only for KB Feedback/Suggestions, if you need help with the software open a support case

By submitting, you are agreeing to have your personal information managed in accordance with the terms of Veeam's Privacy Policy.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Verify your email to continue your product download
We've sent a verification code to:
  • Incorrect verification code. Please try again.
An email with a verification code was just sent to
Didn't receive the code? Click to resend in sec
Didn't receive the code? Click to resend
Thank you!

Thank you!

Your feedback has been received and will be reviewed.

error icon

Oops! Something went wrong.

Please try again later.