Challenge
Advanced transport modes (Virtual Appliance (HotAdd) or Direct Storage access) may not work, failing over to the NBD transport mode instead.
If you are seeing error "Cannot use advanced transport modes, failing over to NBD" you should check the HotAdd attacher agent log (default location - C:\ProgramData\Veeam\Backup\Job_name\Agent.Job_name.VM_name.Hotadd.Attacher.log) on affected backup proxy/proxies, you may encounter following “SSL exception” error:
[08.04.2020 8:26:46.680] < 724> vdl| [vddk] 2020-04-08T20:26:46.680-07:00 error -[00724] [Originator@6876 sub=Default] SSL Async Handshake Timeout : Read timeout after approximately 25000ms. Closing stream <SSL(<io_obj p:0x000001ef0b15eb68, h:1100, <TCP 'x.x.x.x : 49746'>, <TCP 'x.x.x.x : 443'>>)> [08.04.2020 8:26:51.668] < 5412> vdl| [vddk] 2020-04-08T20:26:51.668-07:00 error -[05412] [Originator@6876 sub=IO] HandshakeCb; <SSL(<io_obj p:0x000001ef0b15eb68, h:-1, <TCP 'x.x.x.x : 49746'>, <TCP 'x.x.x.x : 443'> FD Closed>)>; error: class Vmacore::Ssl::SSLHandshakeTimeoutException(SSL Exception: The file handle supplied is not valid)
Cause
The SSL certificate and thumbprint verification is a mandatory process starting from VMware VDDK 6.0. Advanced transport modes (Virtual Appliance (HotAdd) or Direct Storage access) are utilizing TLS protocol unlike the NBD mode.
Solution
If your vCenter has a self-signed certificate tree, install all certificates on all proxies.
https://kb.vmware.com/s/article/2108294