Hello,
This is probably due to Timezone or Server time difference between your Management server and Middleware server. This wasnt a major issue before 23R2 but after updating to 23R2, I have noticed the same error.

In order to fix this, try to set the same time on both the servers and regenerate the kube config (.\main.ps1 -resource 'GETKUBECONFIG')

Hope this helps!
/Nishanth

This issue was reported via a case to IFS. The error occurred because the K8s certificates have expired. The solution was to recreate the Kubernetes cluster and re-install IFS Cloud by following the documentation:

https://docs.ifs.com/techdocs/23r2/070_remote_deploy/010_installing_fresh_system/030_preparing_server/50_windows_managementserver/#parameters_required_for_the_main_powershell_script_action

As per the above documentation, running the following command will re-install the K8s cluster, given that you have already specified the correct values in the main_config.json file.

ps> .\main.ps1

After this, you can re-install IFS Cloud using the IFS Cloud installer, by using the "mtinstaller" options as mentioned in the documentation:
https://docs.ifs.com/techdocs/23r2/070_remote_deploy/010_installing_fresh_system/200_installing_ifs_cloud/035_ifs_cloud_ifsinstaller/020_installer_actions/

After this, ran into another issue which was caused by Ingress not being installed correctly. It was resolved by re-installing the ifs-ingress namespace. 

Hello,

I just had the same issue happen to our (on prem) DEV environment. Just Note, running main.ps1 will turn on the firewall on the middleware server which blocked all https traffic. Turning the firewall off fixed that for me,

This issue brings up a couple questions for me:

1. Is there a way to know when a certificate is going to expire for an environment?
2. Is there a way to renew the certificate before it expires without reinstalling the k8 cluster and IFS middleware?

Having to take the whole system down for 45+- minutes just to renew a certificate is very inconvenient for a 24/7 company even if it is only once a year.

Thank you.

I came up with my own script as to how to get the certificate expiration date:

Powershell:

kubectl exec -it --namespace=ifs-ingress ((kubectl get pods -A) -match 'nginx-controller' -split '\s+')'1] -- /bin/bash -c "openssl x509 -in /etc/ingress-controller/ssl/default-fake-certificate.pem -text"

Bash:

kubectl exec -it --namespace=ifs-ingress $(kubectl get pods -A | grep nginx-controller | awk '{ print $2}') -- /bin/bash -c "openssl x509 -in /etc/ingress-controller/ssl/default-fake-certificate.pem -text"

I would still like to know if there a way to renew the certificate before it expires without reinstalling the k8 cluster and IFS middleware?

 Hi @VenatorInc,

Certificates used in Microk8s usually have a 1 year lifetime. There will be an IFS Cloud major release twice every year. With the evergreen approach, it is advised to be no more than 2-3 SUs behind to make sure the environments are patched for latest security vulnerabilities, bugs etc. and also to get the latest features released with every Release Update.

With every RU, a new set of binaries will also be released, which can be downloaded from IFS Cloud Build Place. Ideally, you are supposed to re-install the cluster using these latest binaries. So if you adhere to this evergreen approach, you are not supposed to run into this certificate expiration issue.

However, if you still need to renew certificates, there is a cert-manager add-on in Microk8s which can be used to update certificates. I am not sure what kind of an impact it would have with IFS Cloud Middle Tier as it is not mentioned in the documentation or anywhere else, so I wouldn’t recommend using it.

Hope this helps!

Does anyone know if this is for Remote deployments only? Or does it also affect Managed Cloud customers? Trying to find out if customers in the cloud need to be worried and addressing this proactively.

Thanks,

Gui

Hello ​@Charith Epitawatta / all,

This morning, we experienced a production down issue, which we've identified was caused by the internal ingress certificate expiry.

However, when I checked around two months ago, the certificate was set to expire on 15th November 2025. Yet, today — 23rd May 2025 — it appears to have expired unexpectedly.

There have been no recent deliveries or known changes to the production environment that could have influenced this.

Could you please help clarify:

• What factors could cause a certificate to expire earlier than its visible expiry date?

• Is there any mechanism (e.g. auto-rotation, misconfiguration, clock issues) that might explain this?

• Are there any known issues or previous cases where the ingress controller or cert-manager mismanaged a certificate?

Looking forward to your insights so we can avoid this in future.

Thanks,
Hardik

error we got:-


PS D:\ifsremote\ifsroot> kubectl exec -it --namespace=ifs-ingress ((kubectl get pods -A) -match 'nginx-controller' -split '\s+')[1] -- /bin/bash -c "openssl x509 -in /etc/ingress-controller/ssl/default-fake-certificate.pem -text"
E0523 06:27:45.856566    4736 memcache.go:265] couldn't get current server API group list: Get "https://<middelware-IP>:16443/api?timeout=32s": x509: certificate has expired or is not yet valid: current time 2025-05-23T06:27:45Z is after 2025-05-22T18:33:24Z
E0523 06:27:45.862380    4736 memcache.go:265] couldn't get current server API group list: Get "https://<middelware-IP>:16443/api?timeout=32s": x509: certificate has expired or is not yet valid: current time 2025-05-23T06:27:45Z is after 2025-05-22T18:33:24Z
E0523 06:27:45.867595    4736 memcache.go:265] couldn't get current server API group list: Get "https://<middelware-IP>:16443/api?timeout=32s": x509: certificate has expired or is not yet valid: current time 2025-05-23T06:27:45Z is after 2025-05-22T18:33:24Z
E0523 06:27:45.872803    4736 memcache.go:265] couldn't get current server API group list: Get "https://<middelware-IP>:16443/api?timeout=32s": x509: certificate has expired or is not yet valid: current time 2025-05-23T06:27:45Z is after 2025-05-22T18:33:24Z
E0523 06:27:45.879065    4736 memcache.go:265] couldn't get current server API group list: Get "https://<middelware-IP>:16443/api?timeout=32s": x509: certificate has expired or is not yet valid: current time 2025-05-23T06:27:45Z is after 2025-05-22T18:33:24Z
Unable to connect to the server: x509: certificate has expired or is not yet valid: current time 2025-05-23T06:27:45Z is after 2025-05-22T18:33:24Z
error: pod, type/name or --filename must be specified

Hi ​@hardik,

How did you determine the expire date of the certificates? Is it using Microk8s commands? I don’t think a clock skew would cause this kind of an issue, since the dates are too far apart. 

Hello ​@Charith Epitawatta 
i used below command:-

 

Powershell:

kubectl exec -it --namespace=ifs-ingress ((kubectl get pods -A) -match 'nginx-controller' -split '\s+')s1] -- /bin/bash -c "openssl x509 -in /etc/ingress-controller/ssl/default-fake-certificate.pem -text"

and this gave the dates as 15th nov 2 months ago.