-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
On Fri, Jun 23, 2017 at 11:38 PM, A. Wilcox
ELFv1 can operate as little endian (and did for a brief period as
a transition), ELFv2 can operate as big endian. PowerPC64 Linux
only will be 64 bit little endian going forward, although the
existing big endian, ELFv1 Linux distributions will continue to be
supported. There is no infrastructure or distribution into which
a PPC64BE ELFv2 libc can be installed.
Is there a technical reason why PowerPC64 Linux will only be
little-endian? Do the Power8 / Power9 chips not support BE mode for
Linux officially?
The processors support operation in either endian mode.
I have an IRC log where I was chatting with an IBM engineer and they
said there is no reason P8/P9 can't run Linux in BE mode. The only
reason to run LE mode is for better GPU support since most of the
Radeons run their framebuffers in that mode (so you'd have much better
performance, esp when using them for GPGPU). But that could have been
then, and not now, since that conversation was almost a year ago.
A PPC64 big endian ELFv2 port is an interesting exercise, but does
not match or interact with any other Linux distributions or
toolchains. All of the PPC64 BE Linux ports are based on ELFv1 and
have no intention of changing.
Except those based on musl? I mean, we at Adélie haven't /shipped/
anything PPC64 yet, but I have very good reasons for that (and will
get to them later in this email).
Because the PowerPC software ecosystem is based on and designed to
those assumptions. All of the JITs are based on that. All of the
optimized libraries are based on that. All of the hand-written
assembly code is based on that.
Some test ABI and endianness separately, some don't. It definitely is
less well tested, if at all.
You can do whatever you want, but it has been difficult enough fixing
all of the poor assumptions in the entire Open Source and proprietary
source ecosystem for the change to PPC64LE ELFv2. If you and Adelie
want to take on that challenge for PPC64BE ELFv2, great. The
OpenPower Foundation and its members are not going to fight that
battle.
I am not exactly certain what FreeBSD is planning.
https://svnweb.freebsd.org/base?view=revision&revision=291668
FreeBSD supports either ABI in either endianness, but defaults to
ELFv2 for both BE and LE with any compiler that supports it.
And this is why FreeBSD remains challenged. As I wrote above, all of
the PPC64BE software ecosystem is written for ELFv1, so this cuts off
FreeBSD from that entire ecosystem. Even packages in the FreeBSD
ports won't build and work correctly because the code for PPC assumes
the endianness and the ABI.
Post by A. WilcoxLet us also not forget that LoPAPR[1] defines (at R1-2.7-1 in my
copy, version 1.1 dated 24 March 2016) that Power Architecture
platforms "must by default operate with Big-Endian addressing".
I think that you're inferring too much into this statement. The
platform has to interoperate with big-endian addressing, especially
for firmware that assumes big endian, but that does not mean that
operating systems must support big endian user space applications.
To me, the standard is completely clear that the operating system
(this being the kernel, i.e. Linux itself, and base userland, i.e.
musl) needs to support both to be compliant with the LoPAPR standard;
at least §2.7, §4.2.3, §B.5.1.
The sections refer to the processor and the platform, not the OSes and
user applications. Boot and RTAS (runtime abstraction services) is
big endian. The hypervisor may be big endian. The OS must be able to
interact with the system -- the underlying system -- in big endian
order. It says nothing about presenting a big endian environment to
user programs.
Of course, each distro can pick what it wants to support, what tool
chain to use, and so on. But the base system needs to support both.
Post by A. WilcoxAre you aware of any little-endian specific code in
musl/powerpc64? I assume that libc-test would probably catch
most of it when I am able to run it, but until then, it would be
nice to know if there is anything I need to work on in the
meantime.
The PPC64 port of Musl does not assume little endian addressing,
but Musl currently only supports ELFv2. All of the toolchains and
operating systems that support ELFv2 are little endian. All of
the big endian toolchains and operating systems are designed for
ELFv1. There is no overlap.
Except Adélie, Sabotage, and anyone who is creating their own
environment without using a distribution. Or are you saying that GCC
assumes LE with ELFv2?
GCC doesn't assume LE = ELFv2, but not all code generation for PPC is
that flexible.
That is the primary reason I haven't shipped any PPC64 image yet. In
addition to the usual badness of porting an entire distro worth of
packages to a platform nobody has really used yet (had a similar time
with musl on MIPS64 and 32-bit PowerPC), I'm a bit uneasy on the
toolchain itself being able to understand what Rich has said. Since
ELFv2 says that Power8 is the minimum ISA, gcc can do whatever it
wants, and I'm not sure if -mcpu=power6 (specific lower ISA) or
- -mcpu=powerpc64 (generic) will affect its code output when it sees
- -mabi=elfv2. So I'm going to need to put any PPC64 image through a
much more rigorous test than I did any other platform.
I added the macro tests for portability and completeness.
The only ports of Musl that will function on existing, supported,
big-endian PowerPC systems are the 32 bit "powerpc" port and an
unimplemented PPC64 BE ELFv1 port.
Except Rich specifically said that he did not want an ELFv1 port for
64-bit PowerPC when I asked him, so I don't think that's going to happen
.
Again, do you have a _technical_ reason that I cannot spend next
weekend making a PPC64 BE image using musl + ELFv2 ABI? Or is this
political / community in nature?
I apologise if my words seem strong, but I do not take this lightly.
We have a number of users clamouring for us to save their older PPC64
hardware from unmaintained AIX, unmaintained Debian, or in some cases
ten-or-more year old fruity OSes. I obviously do not expect ABI
compatibility with decades-old non-Linux Unixes. However, if there
needs to be an ELFv1 port for a technical reason, I may have to
investigate maintaining the port myself.
As I wrote above, the entire external ecosystem makes the endianness /
ABI assumption. Golang assumes this. OpenJDK assumes this. ATLAS
BLAS and OpenBLAS assume this. GMP assumes this. PyPy assumes this.
Mono assumes this. libffi assumes this. Erlang probably assumes this.
FFMPEG, x264, libvpx assume this. MongoDB may assume this. NVIDIA
nvcc assumes this. Etc., etc., etc.
It's not that the packages fundamentally cannot be fixed, but the
FLOSS ecosystem is much larger, richer, complex and more
interdependent. If one wants to create an embedded system, one can
exert control over the entire software ecosystem. For a
Linux-compatible system, one cannot.
If you accept that some parts of the software ecosystem simply won't
build or function correctly for your system and configuration, or some
packages randomly will stop building or stop functioning correctly
after a package is updated, fine. If you want to track it all down,
fine. I hope that you are incorporating all of that into the price
that you are charging your customers and that they understand the
additional risk. When you and your customers rely on the larger Linux
ecosystem, you are relying on a lot of assets and resources outside of
your control.
Thanks, David