Re: cross-compile skalibs

From: Laurent Bercot <ska-skaware_at_skarnet.org>
Date: Tue, 17 Sep 2019 20:11:16 +0200

On 9/14/2019 2:11 PM, Jens Rehsack wrote:
> I don't have such a database and such a database is imposible to have.

  That's a real shame, because it's the *only way* cross-compilation
can ever work.

  You praise autotools for supporting ac_cv_$check variables, that can
be supported from the outside. This is the correct mechanism indeed,
and the _exact same mechanism_ skalibs uses, albeit with a different
syntax. skalibs calls those variables "sysdeps".
  However, the difference is that skalibs forces you to provide the
sysdeps for your target architecture, whereas autotools does not.

  If there is no sysdeps database for various architectures, how does
autotools work when you do not provide appropriate ac_cv_$check
variables?

  It guesses. That's how.
  And guesses can be wrong.
  In the absence of a sysdeps database, it is likely that a project
that uses autotools will be *broken* when you cross-compile for an
exotic system, because autotools will take guesses, and some of those
guesses will be wrong.

  You *cannot* avoid this issue, no matter what build system you use.
It's an inherent problem with cross-compiling. Autotools does not have
divination powers, it cannot magically know the quirks of the target
architecture if you don't provide them.
  Projects that use autotools make the choice of being buildable
without much effort from the packager, at the cost of probably being
buggy on lesser known systems.
  skarnet.org projects make the choice of being correct everywhere,
even if it requires more effort from the packager's part.

  I want my software to be included in distributions, in Yocto,
everywhere possible. I really do. But I will *not* compromise on
correctness. I don't care what everyone else does, when you build
skalibs, you'll have something that works, period.


> This is a more or less hopeless attempt to ride a dead horse.
  This horse is very much alive; and the problem of knowing the
quirks of a target architecture when cross-compiling is, as of
today in 2019, unsolved. You may think autotools has solved it,
but that does not mean that it has. And I don't think the problem
can really be solved without a sysdeps database.

  Now, let's be constructive and go into details. Ccing √Čric Le Bihan
who has done some similar work to integrate s6 to Buildroot, and
I'd like his input.

  There are 3 ways skalibs computes sysdeps:
  A. a compilation test,
  B. a linking test,
  C. a running test.

  You can see, more or less, all the tests that skalibs does by
grepping for 'choose' in the configure script. (It's a bit more
complex than that because some symbols are defined in different
libraries depending on the OS and the libc, and the related tests
don't use the 'choose' function, but unless I'm mistaken they all
fall in category B.) Case A is 'choose c', case B is 'choose cl',
and case C is 'choose clr'.

  Case A is not a real problem for cross-compilation: it's mainly
about testing whether macros are defined. All those sysdeps could
arguably be replaced with #ifdef forests in the skalibs headers.
I just chose to make them sysdeps instead because I dislike #ifdef
forests - they slow down compilation time for *every* TU that
includes headers, and they're ugly. But if case A is the only kind
of sysdeps that remains, it's possible to do away with them.

  Case B is the majority. Unlike case A, those sysdeps need to be
addressed at build time (instead of just adding logic to headers),
but they're not really problematic because they only require an
executable to be linked, and the result is just success/failure of
the compilation+linking phase: that can be done when cross-compiling.

  Case C is the real issue, where testing requires building an
executable *and running it* in order to know the behaviour of the
target system. And that, obviously, is what can't be done when you
cross-compile.
  skalibs doesn't have a lot of those, but unfortunately, those few
are really necessary, because they're about actual behaviour of a
function, and not just whether the function exists or not. For
instance, how do you test whether malloc(0) returns a null pointer
or not, without actually running malloc(0) on your target?

  With some effort on my part, the amount of sysdeps that really
*need* to be manually provided could be reduced to 8 or 9; and
everything else could be automatically tested. However, there will
always be a few inveterates that have to be there, and we need to
decide what to do with those. Guessing is not an option; but I'm
open to other solutions.

-- 
  Laurent
Received on Tue Sep 17 2019 - 18:11:16 UTC

This archive was generated by hypermail 2.3.0 : Sun May 09 2021 - 19:38:49 UTC