After KCC is set up, the validationwebhook intercepts deletion of even non-kcc CRDs

[koustubh25]

Checklist

• I did not find a related open issue.
• I did not find a solution in the troubleshooting guide: (https://cloud.google.com/config-connector/docs/troubleshooting)
• If this issue is time-sensitive, I have submitted a corresponding issue with GCP support.

Bug Description

Config Connector Version

> kubectl get ns cnrm-system -o jsonpath='{.metadata.annotations.cnrm\.cloud\.google\.com/version}' 
1.99.0

Kubernetes Version

GKE v1.23.14-gke.1800

I have KCC set up in namespaced mode on GKE cluster. I also have other CRDs (non KCC) installed in the same cluster. When I try to delete the other CRDs (non KCC), the validation webhook configuration intercepts it.
This is not desirable to me for reasons described in the Log Output section.

I found this issue #202 which is somewhat related, but it focuses more on the KCC uninstallation process.

Additional Diagnostic Information

I see the config for abandon-on-uninstall.cnrm.cloud.google.com validation webhook is

rules:  
 - apiGroups:                                                                                                            
     - apiextensions.k8s.io                                                                                                
     apiVersions:                                                                                                          
     - v1                                                                                                                  
     operations:                                                                                                           
     - DELETE                                                                                                              
     resources:                                                                                                            
     - customresourcedefinitions                                                                                           
     scope: '*'

Would it be better if it has selector so that it only intercepts kcc CRDs? e.g. adding this to the validation webhook

  objectSelector:
    matchLabels:
      cnrm.cloud.google.com/managed-by-kcc: "true"

Or is there any way I can do this already?

Kubernetes Cluster Version

GKE v1.23.14-gke.1800

Config Connector Version

1.99.0

Config Connector Mode

namespaced mode (default)

Log Output

In my case, I have a number of CRDs (some are namespace scoped) in the cluster and the default netpols is deny all, so I end up with below, when I try to delete them

I get this error when I try to delete the CRD certificaterequests.cert-manager.io

CustomResourceDefinition/certificaterequests.cert-manager.io immutable field detected, failed to delete object, error: Internal error occurred: failed calling webhook "abandon-on-uninstall.cnrm.cloud.google.com": failed to call webhook: Post "[https://abandon-on-uninstall.cnrm-system.svc:443/abandon-on-uninstall?timeout=10s](https://abandon-on-uninstall.cnrm-system.svc/abandon-on-uninstall?timeout=10s)": context deadline exceeded

Steps to reproduce the issue

Pre-requisite:

1. Have other non KCC CRDs (namespace scoped) in the cluster

2. Have the default netpols as deny all

3. Set up config connector in k8s cluster in namespaced mode https://cloud.google.com/config-connector/docs/how-to/advanced-install#manual

4. Delete any non KCC CRD in the cluster

You will see that the webhook call times out.

YAML snippets

No response

[diviner524]

@koustubh25 could you please share more details on step 2? Do you have a sample YAML which shows how you set the default netpols?

[koustubh25]

thanks for the reply @diviner524
Sure, here is my default deny-all netpol

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-policy
  namespace: my-ns
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress

But, my question really is - is it necessary for the the validationwebhookconfiguration abandon-on-uninstall.cnrm.cloud.google.com to intercept all CRD deletions?

or would it be better if the webhook has an objectSelector like this

  objectSelector:
    matchLabels:
      cnrm.cloud.google.com/managed-by-kcc: "true"

[diviner524]

@koustubh25 Thanks for providing the YAML! Yes your are correct. An objectSelector can be introduced to specify this webhook only affects KCC CRDs. We will add that.

[mbzomowski]

This was fixed in KCC v1.100.0.