Re: A better method than daisy-chaining logging files?

From: Dewayne Geraghty <dewayne.geraghty_at_heuristicsystems.com.au>
Date: Mon, 17 Jun 2019 16:25:15 +1000

On 31/05/2019 10:52 pm, Brett Neumeier wrote:
> On Fri, May 31, 2019 at 4:21 AM Laurent Bercot <ska-supervision_at_skarnet.org>
> wrote:
>
>>> I just attempted to link an apache24 instance to its log files via a
>>> bundle, which isn't acceptable to s6-rc-compile.
>> My advice is to use s6-rc's producer/consumer mechanism for one
>> of the log streams, and use a named pipe for the other one, without
>> cramming it into the s6-rc mechanism. That would typically mean:
>>
>> - configure apache24 to output its access log to stdout
>> - declare apache24 as a producer for apache24-access-log and
>> apache24-access-log as a consumer for apache24
>> - apache24-access-log is a simple s6-log invocation, reading
>> from its stdin
>> - mkfifo /var/run/apache24/error-fifo (with appropriate rights)
>> - declare that apache24 outputs its error log to
>> /var/run/apache24/error-fifo
>> - apache24-error-log has its run script doing something like:
>> redirfd -r 0 /var/run/apache24/error-fifo s6-log your-logging-script
>> - manually list apache24-error-log in apache24's dependencies, so
>> apache24 doesn't start before apache24-error-log. (The pipeline
>> mechanism automatically adds apache24-access-log to apache24's deps.)
>> - manually define any bundles you want.
>>
>
> For what it's worth, I use approximately this setup on my s6- and
> s6-rc-managed nginx server. The only difference is that I have nginx using
> /dev/stdout as its _error_ stream; and then I have a service that creates a
> separate fifo for each site defined in the nginx configuration. Nginx
> writes each access log to the appropriate fifo, and there's a separate
> s6-log process consuming from each of the fifos. I have had no problems
> whatever with that setup, it works like a charm and was really pretty
> straightforward to set up.
>
> In fact, I find that there are a lot of services I want to run that can
> either log to syslog or write to a specific filesystem location, and the
> same "service writes to a fifo, s6-log reads from the fifo" mechanism works
> fine for all of them. Since I use that pattern so frequently, I create a
> `/run/log-fifos` directory to contain all the fifos. I think that makes the
> entire mechanism pretty obvious and transparent, which is my general goal
> with system administration.
>
> Cheers,
>
> Brett
>

Thank-you both for your sound advise. I did in fact implement Laurent's
suggestions, unfortunately I was a flu early-adopter here in Australia.

Brett, I think I'm more on the same page now and upon reflection, my
question was pretty much a newbie as I'd failed to fully grasp that that
s6-rc is not independent of s6 and that s6 dependencies are my friend
which they now are.

My setup is a little more complicated. I have FreeBSD jails running the
service, and create a shared mount point where the service runs within
the jail, communicates to a nullfs device where the fifo queue resides.
 It all works nicely until there is a rotation, which I induce with
# s6-svc -a /run/scan/apache24-error-log

The result is a directory containing
-rw-r--r-- 1 mylogger www 0B Jun 17 15:34 state
-rw-r--r-- 1 mylogger www 0B Jun 17 15:34 lock
-rwxr--r-- 1 mylogger www 329B Jun 17 15:34 previous
-rw-r--r-- 1 mylogger www 0B Jun 17 15:34 current

and an error message
s6-log: warning: unable to finish previous .s to logdir
/var/log/httpd/error: Operation not permitted

I've su'ed into the /var/log/httpd/error as "logger" and I'm able to
create and compress files within the directory; so there are no
permission issues. And both execlineb and s6-log are installed with 766
privs.

Does s6-log require root:wheel privs to perform functions within the log
directory?


FYI: and largely for those trying to use *BSD and slightly challenging
setup:
The final logger is

#!/usr/local/bin/execlineb -P
s6-setuidgid mylogger
redirfd -r 0 /m/jail3/fifo/apache24-error
s6-log -b n14 r7000 s100000 S3000000 /var/log/httpd/error
# /m is specially mounted and accessible. :)

And the apache24 httpd.conf contains
ErrorLog "/fifo/apache24-error"

For *BSD folks
mkdir -p /m/jails3/fifo /jails/jail3/fifo
mount -t nullfs /m/jail3/fifo /jails/jail3/fifo

Kind regards, Dewayne
Received on Mon Jun 17 2019 - 06:25:15 UTC