Question

Unable to connect to the server: x509: certificate has expired or is not yet valid: current time

  • 3 January 2024
  • 6 replies
  • 651 views

Userlevel 1
Badge +7

Hi

From management server we are giving Kubectl command giving this below error shown in screenshot.

We are using IFS Cloud 23R2. Please provide suggestions.

 

 


6 replies

Badge +3

Hello,
This is probably due to Timezone or Server time difference between your Management server and Middleware server. This wasnt a major issue before 23R2 but after updating to 23R2, I have noticed the same error.

In order to fix this, try to set the same time on both the servers and regenerate the kube config (.\main.ps1 -resource 'GETKUBECONFIG')

Hope this helps!
/Nishanth

Userlevel 7
Badge +31

This issue was reported via a case to IFS. The error occurred because the K8s certificates have expired. The solution was to recreate the Kubernetes cluster and re-install IFS Cloud by following the documentation:

https://docs.ifs.com/techdocs/23r2/070_remote_deploy/010_installing_fresh_system/030_preparing_server/50_windows_managementserver/#parameters_required_for_the_main_powershell_script_action

As per the above documentation, running the following command will re-install the K8s cluster, given that you have already specified the correct values in the main_config.json file.

ps> .\main.ps1

After this, you can re-install IFS Cloud using the IFS Cloud installer, by using the "mtinstaller" options as mentioned in the documentation:
https://docs.ifs.com/techdocs/23r2/070_remote_deploy/010_installing_fresh_system/200_installing_ifs_cloud/035_ifs_cloud_ifsinstaller/020_installer_actions/

After this, ran into another issue which was caused by Ingress not being installed correctly. It was resolved by re-installing the ifs-ingress namespace. 

Badge +1

Hello,

I just had the same issue happen to our (on prem) DEV environment. Just Note, running main.ps1 will turn on the firewall on the middleware server which blocked all https traffic. Turning the firewall off fixed that for me,

 

This issue brings up a couple questions for me:

  1. Is there a way to know when a certificate is going to expire for an environment?
  2. Is there a way to renew the certificate before it expires without reinstalling the k8 cluster and IFS middleware?

Having to take the whole system down for 45+- minutes just to renew a certificate is very inconvenient for a 24/7 company even if it is only once a year.

 

Thank you.

Badge +1

I came up with my own script as to how to get the certificate expiration date:

 

Powershell:

kubectl exec -it --namespace=ifs-ingress ((kubectl get pods -A) -match 'nginx-controller' -split '\s+')[1] -- /bin/bash -c "openssl x509 -in /etc/ingress-controller/ssl/default-fake-certificate.pem -text"

 

Bash:

kubectl exec -it --namespace=ifs-ingress $(kubectl get pods -A | grep nginx-controller | awk '{ print $2}') -- /bin/bash -c "openssl x509 -in /etc/ingress-controller/ssl/default-fake-certificate.pem -text"

 

I would still like to know if there a way to renew the certificate before it expires without reinstalling the k8 cluster and IFS middleware?

Userlevel 7
Badge +31

 Hi @VenatorInc,

Certificates used in Microk8s usually have a 1 year lifetime. There will be an IFS Cloud major release twice every year. With the evergreen approach, it is advised to be no more than 2-3 SUs behind to make sure the environments are patched for latest security vulnerabilities, bugs etc. and also to get the latest features released with every Release Update.

With every RU, a new set of binaries will also be released, which can be downloaded from IFS Cloud Build Place. Ideally, you are supposed to re-install the cluster using these latest binaries. So if you adhere to this evergreen approach, you are not supposed to run into this certificate expiration issue.

However, if you still need to renew certificates, there is a cert-manager add-on in Microk8s which can be used to update certificates. I am not sure what kind of an impact it would have with IFS Cloud Middle Tier as it is not mentioned in the documentation or anywhere else, so I wouldn’t recommend using it.

Hope this helps!

Userlevel 3
Badge +8

Does anyone know if this is for Remote deployments only? Or does it also affect Managed Cloud customers? Trying to find out if customers in the cloud need to be worried and addressing this proactively.

 

Thanks,

Gui

Reply