Mutexer

Appearance

Custom CA Certificates ​

Overview ​

When a Horizon circuit is configured with the HTTPS protocol, the agent device forwards traffic to the backend service over a TLS-encrypted connection. As part of the TLS handshake, the agent's operating system must verify the backend service's certificate against its local trust store. Certificates issued by well-known public Certificate Authorities (such as Let's Encrypt, DigiCert, or GlobalSign) are trusted by default on most Linux distributions, as their root certificates are included in the system's CA bundle. However, backend services in industrial environments frequently use certificates issued by a private or internal Certificate Authority - for example, an organization's own PKI infrastructure, a self-signed root CA, or a vendor-specific CA used to secure device management interfaces. In these cases, the agent device's operating system will reject the TLS connection because it cannot verify the certificate chain, and requests forwarded through the Horizon circuit will fail.

To resolve this, the private CA's root certificate must be installed in the agent device's trust store. This page provides instructions for Ubuntu, which is the primary supported operating system for Mutexer Agents.

Prerequisites ​

  • Root or sudo access on the agent device
  • The CA root certificate file in PEM format (typically a .crt or .pem file)
  • The certificate must be the root CA certificate (or intermediate CA certificate) that issued the backend service's TLS certificate

If the certificate is in DER (binary) format rather than PEM (Base64-encoded text), convert it first:

bash
openssl x509 -inform DER -in certificate.der -out certificate.crt

Installing a Custom CA on Ubuntu ​

Step 1: Copy the certificate to the system CA directory ​

Copy the PEM-formatted certificate file to /usr/local/share/ca-certificates/. The file must have a .crt extension.

bash
sudo cp your-custom-ca.crt /usr/local/share/ca-certificates/

Step 2: Update the CA trust store ​

Run the update-ca-certificates utility to regenerate the system's consolidated CA bundle. This scans /usr/local/share/ca-certificates/ for new certificates and adds them to the trust store at /etc/ssl/certs/ca-certificates.crt.

bash
sudo update-ca-certificates

The command outputs the number of certificates added. Verify that the output includes 1 added:

Updating certificates in /etc/ssl/certs...
1 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.

Step 3: Verify the installation ​

Confirm that the certificate has been added to the trust store by checking for it in the consolidated bundle:

bash
awk -v cmd='openssl x509 -noout -subject' '/BEGIN/{close(cmd)};{print | cmd}' < /etc/ssl/certs/ca-certificates.crt | grep "your-ca-name"

Alternatively, test the TLS connection directly against the backend service:

bash
openssl s_client -connect <backend-ip>:<backend-port> -CAfile /etc/ssl/certs/ca-certificates.crt

A successful verification displays Verify return code: 0 (ok) in the output.

Step 4: Restart the Mutexer Agent ​

After updating the trust store, restart the Mutexer Agent to ensure it picks up the new CA certificates:

bash
sudo service mutexer restart

Removing a Custom CA ​

To remove a previously installed custom CA certificate:

  1. Delete the certificate file from /usr/local/share/ca-certificates/:
bash
sudo rm /usr/local/share/ca-certificates/your-custom-ca.crt
  1. Regenerate the trust store:
bash
sudo update-ca-certificates --fresh
  1. Restart the Mutexer Agent:
bash
sudo service mutexer restart

Troubleshooting ​

Certificate chain errors ​

If the backend service presents an intermediate certificate that is not in the trust store, both the intermediate and root CA certificates must be installed. Each certificate should be placed as a separate .crt file in /usr/local/share/ca-certificates/, followed by running update-ca-certificates.

Certificate format issues ​

The update-ca-certificates utility requires certificates in PEM format with a .crt extension. Files with .pem, .cer, or .der extensions are not processed. Rename PEM-formatted files to use .crt, or convert DER-formatted files using the openssl command shown in the Prerequisites section.

Agent does not recognize the new certificate ​

If the Horizon circuit continues to fail after installing the CA certificate, verify the following:

  • The certificate file is present in /usr/local/share/ca-certificates/
  • The update-ca-certificates command reported 1 added
  • The Mutexer Agent has been restarted after the trust store update
  • The installed certificate is the correct CA that issued the backend service's TLS certificate (not the service's leaf certificate)

WARNING

Installing a CA certificate into the system trust store causes the operating system to trust all certificates issued by that CA. Only install CA certificates from trusted sources. Rogue or compromised CA certificates can enable man-in-the-middle attacks against any TLS connection on the device.

Copyright © 2024 Mutexer Pty Ltd