service account issuer needs to be the same on all servers

[hi117]

As per hasCorrectIssuer in jwt.go:

// hasCorrectIssuer returns true if tokenData is a valid JWT in compact
// serialization format and the "iss" claim matches the iss field of this token
// authenticator, and otherwise returns false.

Setting the issuer to the ip address means that having more than one apiserver will cause unpredictable auth failures. If requests happen to get directed to the apiserver that issued their token then it will auth the request. But if the request hits any other apiserver, then it won't auth the request.

I fixed this by setting it to the same string for all apiservers in my cluster. There can be additional rules but for the use case presented here, just setting it to any string works as long as its the same on all apiservers. See https://kubernetes.io/docs/reference/command-line-tools-reference/kube-apiserver/ and --service-account-issuer for more info.