Consul stanza should allow different tags for nomad and nomad-client

[scalp42]

Hi folks,

This is a feature request related to the consul stanza: https://www.nomadproject.io/docs/configuration/consul/#tags

We'd like to be register Nomad HTTP API in Fabio but unfortunately specifying Fabio tags in the tags field of the Consul stanza will have Fabio pick up http, serf and RPC ports:

There's the nomad-client service which gets registered and always points to 4646 but unfortunately because we can't separate the tags between client and server, we'll sometimes inherit 4647 or 4648` for example (see screenshot) when we really just want the HTTP API.

This is our consul stanza:

{
  "consul": {
    "address": "127.0.0.1:8500",
    "auth": null,
    "auto_advertise": true,
    "checks_use_advertise": true,
    "client_auto_join": true,
    "client_service_name": "nomad-client",
    "server_auto_join": true,
    "server_service_name": "nomad",
    "tags": ["fabio-/nomad-client strip=/nomad-client"]
  }
}

Cheers!

[notnoop]

Hi @scalp42 . Thanks for raising the issue. If I understand correctly, this is relevant when an agent is both a server and a client? Otherwise, you can control each node independently.

Makes sense. To help us understand the problem and motivate prioritizing it, mind if you share the issues you face with fabio having more ports than necessary?

[scalp42]

@notnoop that's correct, only an issue when both server and client (our case).

The issue is that we want to load balance requests going to the Nomad HTTP API on port 4646 which is why we can't have rpc and serf ports in it.

[cPanelRicky]

@notnoop Just thought I'd mention in this issue since it's the most recent that has a response. I went through the open issues and this specific request seems to have been made at least 3 different times dating back to 2019.

Please don't take this as me being pushy at all (I looked into what would be needed to implement this and it's definitely not a simple change), I just wanted to make sure that the frequency with which this request is made in hopes that it may jump onto someones TODO list at some point!

#8081
#6016

[spuder]

To clarify, this isn't just a problem when trying to add nomad as a service (though that appears to be a popular pit fall). This also applies to any service that includes a consul connect sidecar proxy

Here 50% of the traffic is going to the proxy which is undesirable.

[apollo13]

@spuder you can configure custom tags for the sidecar though: https://www.nomadproject.io/docs/job-specification/sidecar_service#tags -- sadly you have to configure them at least to be [dummy] because I think even an empty array (not just nil) will copy the tags from the parent.

[the-maldridge]

I just ran into this again after having to increment the "it was DNS" counter over with Void Linux. We use traefik which finds almost everything via consul, but finds the nomad server via DNS. I figured I'd be smart and have it find everything via consul, and smacked right into this config-shaped wall.

[baodrate]

Relevant, since the motivation for this ticket is for configuring fabio to route to the nomad webUI: fabiolb/fabio#757