Question

Upgrade microk8s fails with certificate error

  • 11 December 2023
  • 2 replies
  • 173 views

Userlevel 6
Badge +23
  • Superhero (Employee)
  • 600 replies

Hi Community,

I’m having issues updating an internal microk8s cluster. 

The command I’m trying to run is:
 

sudo snap refresh microk8s --channel=1.23/stable

It prompts following error:
 

error: cannot refresh "microk8s": Post https://api.snapcraft.io/v2/snaps/refresh: x509: certificate
       is valid for ingress.local, not api.snapcraft.io

 

When issuing the following command to check on the certificate I’m also getting a strange response back:

openssl s_client -connect api.snapcraft.io:443

CONNECTED(00000003)
depth=0 O = Acme Co, CN = Kubernetes Ingress Controller Fake Certificate
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 O = Acme Co, CN = Kubernetes Ingress Controller Fake Certificate
verify error:num=21:unable to verify the first certificate
verify return:1
---
Certificate chain
 0 s:O = Acme Co, CN = Kubernetes Ingress Controller Fake Certificate
   i:O = Acme Co, CN = Kubernetes Ingress Controller Fake Certificate
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIDbzCCAlegAwIBAgIQINrsNaTDS8eAPt8YOke7PDANBgkqhkiG9w0BAQsFADBL
MRAwDgYDVQQKEwdBY21lIENvMTcwNQYDVQQDEy5LdWJlcm5ldGVzIEluZ3Jlc3Mg
Q29udHJvbGxlciBGYWtlIENlcnRpZmljYXRlMB4XDTIzMTIwODA4MzIzOFoXDTI0
MTIwNzA4MzIzOFowSzEQMA4GA1UEChMHQWNtZSBDbzE3MDUGA1UEAxMuS3ViZXJu
ZXRlcyBJbmdyZXNzIENvbnRyb2xsZXIgRmFrZSBDZXJ0aWZpY2F0ZTCCASIwDQYJ
KoZIhvcNAQEBBQADggEPADCCAQoCggEBAJsNJUM3DRNlnceWvV1oRDjohPpAmo1+
2u9OSRLfvfwNHKp+F5b53ZlXbf5ME+iAEkw0U3ng9UroryGZT5wyuDdjgl/ecbJv
1p9LwkqNfVMqwQrZ2yXk1FpoikSUjOyJg7BTGhnJITHe0A/OqQjsnSVFsuqxjx/0
PeMLNrsPm9hS9McThT9+luEg5eyS8mFHuMmjC4NQSIubdVFoUJ6736qE8qKszhOd
nt9oF2V0WHnjXe0ERuonFPsmpTVU0HfphOav0JxQHc5KBVJEpns54jdYX22A7F6n
yEduWrVqQTFPC345yzaLc2Y/TxvNzs46+BobUYHSLK8fXvUk6QCimIMCAwEAAaNP
ME0wDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQMMAoGCCsGAQUFBwMBMAwGA1UdEwEB
/wQCMAAwGAYDVR0RBBEwD4INaW5ncmVzcy5sb2NhbDANBgkqhkiG9w0BAQsFAAOC
AQEAjs6Gn0HiWissUT2OPCvRSSafuZeh6xlyKp46GGatlG61cepdpYP/8zAEANsG
rco/GLpwExFcql2IOXAXpZ5BPMulR6sfEY03vnxuXvhPFnfWUN+o4m4dBokST8Lb
UD/Jn9z6HLmceBLo4cgrGV2D0JEaVv/PsNCoKzvYPtCi7KL4yMSW1cAsDHt7VFr3
USvVSQayXCc2rNHpgBpvsu3/XxMIRPmaKbgOst7nTbeCvWNnyEPG5YhZQ3gl0LsF
kRtHFZdcb5cz/BQNZJPR/PJ70WTsdI9Bi0FURuu6CbE2L4fna4SIgsuSjWkbkV6e
7W76oiVFAvIgHT3+OYp2akd9GA==
-----END CERTIFICATE-----
subject=O = Acme Co, CN = Kubernetes Ingress Controller Fake Certificate

issuer=O = Acme Co, CN = Kubernetes Ingress Controller Fake Certificate

---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 1365 bytes and written 401 bytes
Verification error: unable to verify the first certificate
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES128-GCM-SHA256
    Session-ID: 1D8EF098FFCB4F45AE54FE92960114F4207D4633A67B1D942CF7F4D50195B389
    Session-ID-ctx:
    Master-Key: B5073B8FEAD9ECB42330B63E1230A478673422ACB0B76A208591CC60D5FBDD6BD854E75ACAECFDD0F38A319AFC61FEA1
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1702288150
    Timeout   : 7200 (sec)
    Verify return code: 21 (unable to verify the first certificate)
    Extended master secret: yes
---
closed


Obviously, this looks like a certificate issue.

I tried all sort of things to troubleshoot this issue but haven’t found a solution yet. Did someone else had those kind of issues when upgrading microk8s?


OS: Ubuntu 20.04.6 LTS (GNU/Linux 6.6.4-060604-generic x86_64)


Best regards
Roman


2 replies

Userlevel 5
Badge +10

Have you tried a complete reinstall 😁
If e.g. a cert expires i have not found a way of renewing them… !?
I guess the same goes for “corrupted” certs.


You better wipe out any traces of the old mk8s before reinstalling on a server.

sudo snap remove --purge microk8s
sudo snap remove --purge kubectl
sudo rm -R /root/snap/microk8s
sudo rm -R /home/ifs/microk8s
sudo rm -R /home/ifs/snap
sudo rm -R /home/ifs/remote-scripts
sudo rm /home/ifs/install-k8s.sh
sudo reboot

Userlevel 6
Badge +23

Thanks for the response, Henrik. No, I didn’t try to completely uninstall and reinstall yet. This would be the “last restort”! ...yeah, I also tried to update certs with some commands but without luck.

 

Best regards
Roman

Reply