Re: Trouble starting services on boot with s6-rc

From: <dark.pen9108_at_fastmail.com>
Date: Sat, 20 Aug 2022 10:30:30 -0400

>
>> root 34 0.0 0.0 192 52 pts/0 S+ 18:53 0:00 /package/admin/s6-2.11.1.0/command/s6-sudoc -e -t 30000 -T 4999 -- up 3
>
> The "-T 4999" part means there's a 5 second timeout on running your
> sleeponstart service. And since this service is waiting for 30 seconds
> before succeeding, and 30 > 5... it means that it times out after 5
> seconds and gets killed, so the transition fails, and longrun_test
> (which depends on it) is never brought up.
>
> s6-overlay-specific section:
>
> - You should see all that happening in your container's logs - s6-rc
> prints informative messages when something fails. Unless you have
> changed the S6_VERBOSITY value from the default 2 to something lower.
>
> - The 5 seconds timeout likely comes from the fact that you have not
> modified the S6_CMD_WAIT_FOR_SERVICES_MAXTIME value, which is 5000 by
> default. As is written in both the README[1] and the migration guide[2]
> :P
> If your real service needs it, you can disable the timeout by adding to
> your Dockerfile:
> ENV S6_CMD_WAIT_FOR_SERVICES_MAXTIME=0
> or you can adjust it to a bigger value than 5000.

ahh, very obvious. Sorry, I did read the docs(quite a few times) but starting from zero it was quite a lot to grok and I overlooked this(and I'm sure a number of other things). After adjusting S6_CMD_WAIT_FOR_SERVICES_MAXTIME, things are working as expected.

I am a bit ashamed to admit I cannot find the logs. From reading https://wiki.gentoo.org/wiki/S6_and_s6-rc-based_init_system#logger I thought maybe I should be looking for file /run/uncaught-logs but could not find any such file in my docker instance(I understand, docker is not Gentoo).

While the docs did speak a lot to the directory structure used by s6, I still am finding it quite hard to figure out what the default directories are for some things. (e.g. I was clear on where my uncompiled s6-rc service directories should go but they seemed to "magically" get complied on boot and show up in a scan_dir)

One additional item. As seems like not a great idea to smash the timeout for all services. Is there any way to adjust it on a per service basis? If not consider me a +1 to kindly add it to a wishlist somewhere.

Thanx so much for the quick reply, help, and great free software!

n.b. You should put up a BTC lightning "tip bucket" somewhere :)

-Benjamin

On Sat, Aug 20, 2022, at 8:57 AM, Laurent Bercot wrote:
> >Hoping this is the right place to ask for some help as I am very new to s6 and not well versed on any init system.
>
> s6-overlay questions are normally asked in the "Issues" section of the
> s6-overlay GitHub repository, but since yours are really s6-rc
> questions,
> it's fine :) (even though the answers are related to s6-overlay!)
>
>
> >I can see that my services are registered with s6-rc
> >[Aug 19 18:43:04 root_at_s6 ~]# s6-rc-db list all | grep -e longrun_test -e sleeponstart
> >sleeponstart
> >longrun_test
>
> That proves your services are *in the database*, not that they're going
> to get started. To make sure your services are started on boot, you
> should also check that they're declared in the "user" bundle (that's
> the bundle where users declare what services are part of the final
> machine state, by s6-overlay policy).
>
> $ s6-rc-db contents user | grep -e longrun_test -e sleeponstart
>
> But since they *are* started when you boot your container, it means
> they're indeed declared there, so that's not the issue. The real issue
> is here:
>
> >root 34 0.0 0.0 192 52 pts/0 S+ 18:53 0:00 /package/admin/s6-2.11.1.0/command/s6-sudoc -e -t 30000 -T 4999 -- up 3
>
> The "-T 4999" part means there's a 5 second timeout on running your
> sleeponstart service. And since this service is waiting for 30 seconds
> before succeeding, and 30 > 5... it means that it times out after 5
> seconds and gets killed, so the transition fails, and longrun_test
> (which depends on it) is never brought up.
>
> s6-overlay-specific section:
>
> - You should see all that happening in your container's logs - s6-rc
> prints informative messages when something fails. Unless you have
> changed the S6_VERBOSITY value from the default 2 to something lower.
>
> - The 5 seconds timeout likely comes from the fact that you have not
> modified the S6_CMD_WAIT_FOR_SERVICES_MAXTIME value, which is 5000 by
> default. As is written in both the README[1] and the migration guide[2]
> :P
> If your real service needs it, you can disable the timeout by adding to
> your Dockerfile:
> ENV S6_CMD_WAIT_FOR_SERVICES_MAXTIME=0
> or you can adjust it to a bigger value than 5000.
>
> Good luck,
>
> --
> Laurent
>
>
> [1]: https://github.com/just-containers/s6-overlay/blob/master/README.md
> [2]:
> https://github.com/just-containers/s6-overlay/blob/master/MOVING-TO-V3.md
>
>
Received on Sat Aug 20 2022 - 16:30:30 CEST

This archive was generated by hypermail 2.4.0 : Sat Aug 20 2022 - 16:32:10 CEST