Re: publish/subscribe data between services

From: Buck Evan <buck_at_yelp.com>
Date: Tue, 8 Mar 2016 09:50:55 -0800

On Mon, Mar 7, 2016 at 6:49 PM, Colin Booth <cathexis_at_gmail.com> wrote:

> On Mar 7, 2016 6:41 PM, "Patrick Mahoney" <pat_at_polycrystal.org> wrote:
> >
> > On 2016-03-07 7:41 pm, Buck Evan wrote:
> >
> >> How do services become aware of each others' port numbers?
> >>
> >
> > Not sure if this would be simpler than IP-per-playground, but you
> > could look into Linux's network namespaces [1] to assign
> > netns-per-playground. Within the netns, you can use statically known
> > ports, and you can use `ip netns exec $namespace command...` to run
> > arbitrary commands within the namespace. There's a fair amount of
> > config to properly create the loopback address within each netns, and
> > possibly allow routing through the primary netns (the one attached to
> > the LAN) if that's needed.
> >
> I was thinking something similar but instead of using a network namespace
> instead simply binding against a subdomain of a wildcard dns record. As
> long as your naming scheme was reasonably stable
> ($svc.$playgroundname.$wildcarddomain), a service that restarted would be
> able to bind to it's well-known port, only within the context of that
> particular subdomain.
>
> I think that might be less work than setting up individual network
> namespaces, depending on your access to change internal dns.
>

We've gone through several schemes that resemble what you've proposed.
What exactly did you have in mind? An A record or CNAME?
We used to use A records, but we then had to guard against playgrounds
racing to acquire the same IP.
Our engineer years ago solved this via pip-delimited data in DNS TXT
records... it wasn't resilient.

We now use CNAMEs and so can disregard races for the same IP, but now have
the problem of colliding ports.
The current solution is a per-box nginx proxy that redirects CNAMES to
link-local IPs (eg 127.127.3.2).
We then use static ports for each service on that IP.

What's more, I'm not sure this problem is strictly about ports.
I've had situations where a dummy password is generated for a service on
startup, and another service must know that dummy password to connect.
I believe the general problem is sharing configuration among services.
Received on Tue Mar 08 2016 - 17:50:55 UTC