Deleting k8s objects stuck in finalizers

Mon 05 February 2018

tags: useful

While playing around with https://rook.io, teardown/cleanup did not work as smoothly as I had hoped.

Specifically, I had a stuck namespace rook and customresourcedefinition clusters.rook.io. This is probably due to blindly copying teardown instructions ;)

It seemed as if only these two objects were stuck, due to finalizers watching over them that were waiting on some sort of confirmation that was never going to come... This is what I ended up doing, adapted from a comment on a similar GitHub issue:

k get ns rook -o json > rook-ns.json
<edit to remove the spec.finalizers entries, leaving an empty list>
kubectl proxy --port=8080  # in another terminal
curl -H "Content-Type: application/json" -X PUT --data-binary \
  @rook-ns.json http://localhost:8080/api/v1/namespaces/rook/finalize

For the CRD, I had to GET the API until I found the right endpoint, ending up with:

k get crd clusters.rook.io -o json > crd-clusters.rook.io.json
<remove the metadata.finalizers entries this time>
curl -H "Content-Type: application/json" -X PUT --data-binary \
    @crd-clusters.rook.io.json
    http://localhost:8080/apis/apiextensions.k8s.io/v1beta1/customresourcedefinitions/clusters.rook.io

And then of course clean up the dataDirHostPath defined in rook-cluster.yaml's Rook Cluster object on the nodes.

Oh... and Ahmet's kubectl aliases are amazingly useful!

Blogroll

Social