Adding an on-premises SharePoint organization with the Use SSL parameter enabled fails with the following error:
“Connecting to remote server <servername> failed with the following error message: The server certificate on the destination computer (<servername>:<port>) has the following errors: Encountered an internal error in the SSL library.”
WinRM service is not configured to use HTTPS.
To make WinRM use HTTPS, please follow Microsoft article How To: Configure WINRM for HTTPS on each machine in your SharePoint farm.
Once done, ensure that a specified port matches the one configured as per the article above, and try adding organization again.
This form is only for KB Feedback/Suggestions, if you need help with the software open a support case