How To Avoid An SSL Certificate Error For An Old OS Caused By Expired LetsEncrypt's Root Certificate

Updated: Apr 29

If you are using a root certification from LetsEncrypt, you might discover that an SSL certificate error occurring on access from an old OS. LetsEncrypt's root certificate was changed to a cross-root certificate with a certification authority "ISRG Root X1", which is valid until 2035, due to the expiration of "DST Root CA X3" whose expiration date was on September 30th, 2021. After "DST Root CA X3" was expired, SSL certificate errors have been occurring in some environments when accessing sites using LetsEncrypt certificates.

Causes and solutions by different OS

For CentOS7, Amazon Linux, Amazon Linux2

This is most likely due to the fact that the root certificate installed in the OS is outdated and the "ISRG Root X1" root certificate does not exist in the root certificate tree. You can update the ROOT certificate with the following command.

yum -y update ca-certificates

The problem also occurs when the OpenSSL version is old (1.0.1 or older). In that case, you need to upgrade OpenSSL to the latest version as well.

For CentOS6 and common errors

Cross root certificate validation requires OpenSSL 1.0.2 or higher version, but CentOS6 is no longer supported and only versions up to 1.0.1 are available. OpenSSL 1.0.1 or older versions do not have the partial chain function for SSL certificates, so the certificate cannot be verified correctly so that an SSL communication error will occur.

The following are examples of common error patterns.

Example of wget

wget https://aWebsiteThatUsesACirtificateFromLetsEncrypt

Execution result

--2021-10-12 12:06:18--  https://aWebsiteThatUsesACirtificateFromLetsEncrypt/
Resolving aWebsiteThatUsesACirtificateFromLetsEncrypt... xx.xx.xx.xx
Connecting to aWebsiteThatUsesACirtificateFromLetsEncrypt|xx.xx.xx.xx|:443... connected.
ERROR: cannot verify aWebsiteThatUsesACirtificateFromLetsEncrypt’s certificate, issued by “/C=US/O=Let's Encrypt/CN=R3:
  Issued certificate has expired.
To connect to aWebsiteThatUsesACirtificateFromLetsEncrypt insecurely, use ‘--no-check-certificate’.

Example of OpenSSL

openssl s_client -connect aWebsiteThatUsesACirtificateFromLetsEncrypt:443 -servername aWebsiteThatUsesACirtificateFromLetsEncrypt < /dev/null

Execution result

    Start Time: 1633378075
    Timeout   : 300 (sec)
    Verify return code: 10 (certificate has expired)

Example of a PHP program that uses the OpenSSL library (e.g. to communicate

If a program uses the get_file_contents function as shown below example, it will fail to validate the certificate.

$url = "https://aWebsiteThatUsesACirtificateFromLetsEncrypt/";
$output = file_get_contents($url);

Execution result

PHP Warning: file_get_contents(): SSL operation failed with code 1. OpenSSL Error messages:
error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed in "executedProgram" on line 3
PHP Warning: file_get_contents(): Failed to enable crypto in "executedProgram" on line 3
PHP Warning: file_get_contents(https://aWebsiteThatUsesACirtificateFromLetsEncrypt/): failed to open stream: operation failed in "executedProgram" on line 3

The problem does not occur if you are using the curl command or the PHP curl function. This is probably because the curl command uses not only the OpenSSL library, but also the NSS library.

ldd /usr/bin/curl | grep ssl => /usr/lib64/ (0x00007f43fc44b000) => /usr/lib64/ (0x00007f43fb1c5000)

ldd /usr/bin/wget | grep ssl => /usr/lib64/ (0x00007f79a04c6000)

ldd /usr/bin/openssl | grep ssl => /usr/lib64/ (0x00007f2872a3a000)

rpm -qf /usr/lib64/


rpm -qf /usr/lib64/


How to solve the problem You will either need to modify your program to use the curl command or a curl-based library, or you will need OpenSSL version 1.0.2 or higher. However, as mentioned earlier, CentOS6 does not provide versions higher than 1.0.1, so you need to get SRPM from the CentOS7 repository, modify it for CentOS6, then build and install the right version if it is difficult to modify the program.

Reference: RHEL/CentOS 6 OpenSSL client compatibility after DST Root CA X3 expiration

*This is a procedure that has been proven on our servers, but please note that we have not been able to confirm the impact and operation of all OpenSSL functions.

*Fundamentally, it is recommended that you update your OS.

How to forcefully install OpenSSL 1.0.2 on CentOS6

Create an rpm package from srpm

  • Obtain the OpenSSL 1.0.2 rpm package from the CentOS7 repository

cd /usr/local/src
wget --no-check-certificate
rpm -i openssl-1.0.2k-21.el7_9.src.rpm

*Move to a directory named rpmbuild which is created under the home directory of the user who executed the rpm command. The source code will be extracted in the directory.

cd ~/rpmbuild

  • Fix SPEC file

Comment out "%patch68" as it is not necessary.

sed -i -e "s/%patch68/#%patch68/g" SPECS/openssl.spec

  • Fix patch file

The glibc function "secure_getenv" is not available in CentOS6, so replace it with "getenv".

sed -i -e "s/secure_getenv/getenv/g" SOURCES/openssl-1.0.2a-env-zlib.patch
sed -i -e "s/secure_getenv/getenv/g" SOURCES/openssl-1.0.2j-deprecate-algos.patch
sed -i -e "s/secure_getenv/getenv/g" SOURCES/openssl-1.0.2a-fips-ctor.patch

  • Build the rpm package

If you don't have the necessary tools for the build installed, install them and then rpmbuild.

yum install rpm-build krb5-devel zkib-devel lksctp-tools-devel zlib-devel gcc
rpmbuild -ba SPECS/openssl.spec

Install the rpm package

  • Install the packages required for the dependency (perl-WWW-Curl)

yum install perl-WWW-Curl

  • Go to the directory where the rpm package was built, and install it with the rpm command

cd RPMS/
rpm -Uvh openssl-*

  • Check if the version has been upgraded

openssl version

*You are good to go if the version is 1.0.2k

Deactivate the DST Root CA X3 certificate

The problem will not go away if the DST Root CA X3 certificate is still in the root certificate of the OS. It is an expired certificate anyways so let's make it invalidated.

  • Update the root certificate of the OS

yum update ca-certificates

  • Enable the update-ca-trust feature and update the root certificate with the update-ca-trust command

update-ca-trust enable

Right after updating the root certificate with the "yum update ca-certificates" command, the trusted root certificate (/pki/tls/certs/ca-bundle.crt) file in the OS will contain not only the contents of the certificate but also comments such as Issuer and expiration date.

However, if you run the "update-ca-trust" command without arguments, the trusted root certificate of the OS will be reorganized into a file with only the contents of the certificate and comments.

Be sure to run the command in the next step to ensure that only the certificate part is extracted.

  • Extract only the "DST Root CA X3" certificate and save it as a file under /etc/pki/ca-trust/source/blacklist/

perl -e 'while(<>){last if $_ =~ m/DST Root CA X3/;}print $_;while(<>){last if length($_)==1;print $_}' </etc/pki/tls/certs/ca-bundle.crt > /etc/pki/ca-trust/source/blacklist/DST_Root_CA_X3.pem

If the file named /etc/pki/ca-trust/source/blacklist/DST_Root_CA_X3.pem is created and the content looks like the following, you are good to go. You can also create the file manually if it does not work.

#DST Root CA X3

  • Disable "DST Root CA X3" certificate

update-ca-trust extract
grep "DST Root CA X3" /etc/pki/tls/certs/ca-bundle.crt

You are good to go if you don't see anything.

grep "ISRG Root X1" /etc/pki/tls/certs/ca-bundle.crt

You should see "# ISRG Root X1" as a result.

After the update, run the program that previously provided the error to see if the problem has been resolved. It is recommended to restart the service that provides SSL such as Apache if the server is running the service.

This blog post is translated from a blog post by Seiken Sato on our Japanese website Beyond Co..