Discussion:
[PATCH] add definition of max_align_t to stddef.h
(too old to reply)
Pawel Dziepak
2014-04-30 20:23:01 UTC
Permalink
max_align_t is an object type with the strictest alignment supported by
the implementation in all contexts. An union is used to "choose" the greater
of the two greatest scalar types: long long and long double.

Signed-off-by: Pawel Dziepak <***@quarnos.org>
---
include/alltypes.h.in | 2 ++
include/stddef.h | 1 +
2 files changed, 3 insertions(+)

diff --git a/include/alltypes.h.in b/include/alltypes.h.in
index c4ca5d5..ec3873f 100644
--- a/include/alltypes.h.in
+++ b/include/alltypes.h.in
@@ -18,6 +18,8 @@ TYPEDEF unsigned _Int64 uint64_t;
TYPEDEF unsigned _Int64 u_int64_t;
TYPEDEF unsigned _Int64 uintmax_t;

+TYPEDEF union { long double ld; long long ll; } max_align_t;
+
TYPEDEF unsigned mode_t;
TYPEDEF unsigned _Reg nlink_t;
TYPEDEF _Int64 off_t;
diff --git a/include/stddef.h b/include/stddef.h
index 0a32919..cab34d9 100644
--- a/include/stddef.h
+++ b/include/stddef.h
@@ -7,6 +7,7 @@
#define NULL ((void*)0)
#endif

+#define __NEED_max_align_t
#define __NEED_ptrdiff_t
#define __NEED_size_t
#define __NEED_wchar_t
--
1.9.0
Szabolcs Nagy
2014-04-30 21:42:51 UTC
Permalink
Post by Pawel Dziepak
+TYPEDEF union { long double ld; long long ll; } max_align_t;
this is wrong

- ld and ll identifiers are not reserved for the implementation
(you could name them _ld, _ll or __ld, __ll etc)

and see previous max_align_t discussion
http://www.openwall.com/lists/musl/2014/04/28/8

- compiler implementations are non-conforming on some platforms
(_Alignof returns inconsistent results for the same object type so
reasoning about alignments is problematic, there are exceptions
where this is allowed in c++11 but not in c11)

- max_align_t is part of the abi and your solution is incompatible
with gcc and clang (your definition gives 4 byte _Alignof(max_align_t)
on i386 instead of 8)

there is probably not much choice and musl will have to copy the
silly definition used in gcc/clang making max_align_t not very
useful (it does not reflect malloc alignment supported by the libc
nor the object alignments supported by the compiler)
Rich Felker
2014-04-30 22:43:11 UTC
Permalink
Post by Szabolcs Nagy
Post by Pawel Dziepak
+TYPEDEF union { long double ld; long long ll; } max_align_t;
this is wrong
- ld and ll identifiers are not reserved for the implementation
(you could name them _ld, _ll or __ld, __ll etc)
It's also not clear to me why this should go in alltypes.h. Unless it
needs to be arch-specific, it could go directly in stddef.h.
Post by Szabolcs Nagy
and see previous max_align_t discussion
http://www.openwall.com/lists/musl/2014/04/28/8
- compiler implementations are non-conforming on some platforms
(_Alignof returns inconsistent results for the same object type so
reasoning about alignments is problematic, there are exceptions
where this is allowed in c++11 but not in c11)
Is there a bug filed against gcc yet?
Post by Szabolcs Nagy
- max_align_t is part of the abi and your solution is incompatible
with gcc and clang (your definition gives 4 byte _Alignof(max_align_t)
on i386 instead of 8)
This is probably not very important, but I agree it's desirable to be
consistent.
Post by Szabolcs Nagy
there is probably not much choice and musl will have to copy the
silly definition used in gcc/clang making max_align_t not very
useful (it does not reflect malloc alignment supported by the libc
nor the object alignments supported by the compiler)
The definition of max_align_t is very confusing:

"an object type whose alignment is as great as is supported by the
implementation in all contexts"

But as far as I can tell, malloc has nothing to do with max_align_t;
the latter only pertains to observable behavior, and the alignment
malloc produces is not observable. The fact that our malloc produces
addresses which are multiples of 16 (or 32) as an implementation
detail does not mean that alignments up to 16 (or 32) are "supported
by the implementation" in this context; they just happen to work. As
its public interface contract, malloc only guarantees sufficient
alignment for types not produced with _Alignas (or similar GNU C
attribute usage).

Morally, max_align_t should be an object whose alignment requirement
is equal to the max alignment requirement for all types that don't
involve _Alignas.

Rich
Paweł Dziepak
2014-05-04 02:52:28 UTC
Permalink
Post by Rich Felker
Post by Szabolcs Nagy
Post by Pawel Dziepak
+TYPEDEF union { long double ld; long long ll; } max_align_t;
this is wrong
- ld and ll identifiers are not reserved for the implementation
(you could name them _ld, _ll or __ld, __ll etc)
It's also not clear to me why this should go in alltypes.h. Unless it
needs to be arch-specific, it could go directly in stddef.h.
I thought it would be better if types required by the standard are
defined in one file. It doesn't really look like alltypes.h is
reserved for arch-specific types. On the other hand, there indeed is
not any reason not to put that definition directly in stddef.h.

Paweł
Paweł Dziepak
2014-05-04 11:42:05 UTC
Permalink
Post by Rich Felker
Post by Szabolcs Nagy
Post by Pawel Dziepak
+TYPEDEF union { long double ld; long long ll; } max_align_t;
this is wrong
- ld and ll identifiers are not reserved for the implementation
(you could name them _ld, _ll or __ld, __ll etc)
It's also not clear to me why this should go in alltypes.h. Unless it
needs to be arch-specific, it could go directly in stddef.h.
Post by Szabolcs Nagy
and see previous max_align_t discussion
http://www.openwall.com/lists/musl/2014/04/28/8
- compiler implementations are non-conforming on some platforms
(_Alignof returns inconsistent results for the same object type so
reasoning about alignments is problematic, there are exceptions
where this is allowed in c++11 but not in c11)
Is there a bug filed against gcc yet?
The behavior in GCC 4.9 has changed. _Alignof(long long) now is always
4. _Alignof(max_align_t) remains 8 though. Because of this, the
solution I proposed in earlier post doesn't work anymore
(_Alignof(max_align_t) would be 4 if GCC 4.9 didn't complain that
_Alignas(long long) reduces alignment of long long which is weird but
probably doesn't matters much in this discussion) and I looks like the
only option is to use __attribute__((__aligned__(...))). I don't think
there is reason for me to send another version of this patch since
there has already been sent a patch which defines max_align_t in such
way.

Paweł
Jens Gustedt
2014-05-07 05:02:15 UTC
Permalink
Hello,
Post by Paweł Dziepak
The behavior in GCC 4.9 has changed. _Alignof(long long) now is always
4. _Alignof(max_align_t) remains 8 though.
just to be clear, is it that _Alignof(max_align_t) is 8 for their
version of max_align_t or for your version.

Rich is correct, if it would be for your version, this would be a
bug.

But if it would be for their version, this would be just a unilateral
decision about the API they have taken, and musl should then mimic
that behavior. The idea behind that might be that they consider that
some special type(s) that are C extensions have "usual allignment".

Jens
--
:: INRIA Nancy Grand Est :: http://www.loria.fr/~gustedt/ ::
:: AlGorille ::::::::::::::: office Nancy : +33 383593090 ::
:: ICube :::::::::::::: office Strasbourg : +33 368854536 ::
:: ::::::::::::::::::::::::::: gsm France : +33 651400183 ::
:: :::::::::::::::::::: gsm international : +49 15737185122 ::
Paweł Dziepak
2014-05-04 02:36:03 UTC
Permalink
Post by Szabolcs Nagy
Post by Pawel Dziepak
+TYPEDEF union { long double ld; long long ll; } max_align_t;
this is wrong
- ld and ll identifiers are not reserved for the implementation
(you could name them _ld, _ll or __ld, __ll etc)
I will fix that. However, I must admit I don't see why members of the
union (or struct) have to use identifiers reserved for the
implementation. It's not like they can conflict with anything, isn't
it?
Post by Szabolcs Nagy
and see previous max_align_t discussion
http://www.openwall.com/lists/musl/2014/04/28/8
- compiler implementations are non-conforming on some platforms
(_Alignof returns inconsistent results for the same object type so
reasoning about alignments is problematic, there are exceptions
where this is allowed in c++11 but not in c11)
- max_align_t is part of the abi and your solution is incompatible
with gcc and clang (your definition gives 4 byte _Alignof(max_align_t)
on i386 instead of 8)
The behavior of _Alignof on x86 is indeed quite surprising. I actually
don't see why 8 is the right value and 4 isn't - System V ABI for x86
doesn't mention any type with alignment 8. Anyway, I agree that it
would be a good thing to mach the definition gcc and clang use, i.e.
something like that:

union max_align_t {
alignas(long long) long long _ll;
alignas(long double) long double _ld;
};
Post by Szabolcs Nagy
there is probably not much choice and musl will have to copy the
silly definition used in gcc/clang making max_align_t not very
useful (it does not reflect malloc alignment supported by the libc
nor the object alignments supported by the compiler)
Well, alignments supported by the compiler may be different from the
alignments supported by the libc and that depends on how the
implementation supports extended alignments. max_align_t specifies the
greatest fundamental alignment which is guaranteed to be supported in
all contexts (i.e. it's at least as strict as the strictest alignment
required by fundamental types).

Paweł
Rich Felker
2014-05-04 05:02:13 UTC
Permalink
Post by Paweł Dziepak
Post by Szabolcs Nagy
Post by Pawel Dziepak
+TYPEDEF union { long double ld; long long ll; } max_align_t;
this is wrong
- ld and ll identifiers are not reserved for the implementation
(you could name them _ld, _ll or __ld, __ll etc)
I will fix that. However, I must admit I don't see why members of the
union (or struct) have to use identifiers reserved for the
implementation. It's not like they can conflict with anything, isn't
it?
#define ll 0
#include <stddef.h>
Post by Paweł Dziepak
Post by Szabolcs Nagy
and see previous max_align_t discussion
http://www.openwall.com/lists/musl/2014/04/28/8
- compiler implementations are non-conforming on some platforms
(_Alignof returns inconsistent results for the same object type so
reasoning about alignments is problematic, there are exceptions
where this is allowed in c++11 but not in c11)
- max_align_t is part of the abi and your solution is incompatible
with gcc and clang (your definition gives 4 byte _Alignof(max_align_t)
on i386 instead of 8)
The behavior of _Alignof on x86 is indeed quite surprising. I actually
It's also wrong. The correct alignment for max_align_t on i386 is 4,
not 8. It's a bug that GCC ever returns 8 for alignof on i386. We
really need to file a bug against GCC and explain this clearly,
because I have a feeling they're going to be opposed to fixing it...
Post by Paweł Dziepak
don't see why 8 is the right value and 4 isn't - System V ABI for x86
doesn't mention any type with alignment 8. Anyway, I agree that it
You're completely right; GCC is wrong.
Post by Paweł Dziepak
would be a good thing to mach the definition gcc and clang use, i.e.
union max_align_t {
alignas(long long) long long _ll;
alignas(long double) long double _ld;
};
This should not give results different from omitting the "alignas".
The only reason it does give different results is a bug in GCC, so we
should not be copying this confusing mess that's a no-op for a correct
compiler. (Applying alignas(T) to type T is always a no-op.)

Rich
Paweł Dziepak
2014-05-06 10:35:55 UTC
Permalink
Post by Rich Felker
Post by Paweł Dziepak
Post by Szabolcs Nagy
Post by Pawel Dziepak
+TYPEDEF union { long double ld; long long ll; } max_align_t;
this is wrong
- ld and ll identifiers are not reserved for the implementation
(you could name them _ld, _ll or __ld, __ll etc)
I will fix that. However, I must admit I don't see why members of the
union (or struct) have to use identifiers reserved for the
implementation. It's not like they can conflict with anything, isn't
it?
#define ll 0
#include <stddef.h>
Ah, I didn't thought about that. Thanks for clarification.
Post by Rich Felker
Post by Paweł Dziepak
Post by Szabolcs Nagy
and see previous max_align_t discussion
http://www.openwall.com/lists/musl/2014/04/28/8
- compiler implementations are non-conforming on some platforms
(_Alignof returns inconsistent results for the same object type so
reasoning about alignments is problematic, there are exceptions
where this is allowed in c++11 but not in c11)
- max_align_t is part of the abi and your solution is incompatible
with gcc and clang (your definition gives 4 byte _Alignof(max_align_t)
on i386 instead of 8)
The behavior of _Alignof on x86 is indeed quite surprising. I actually
It's also wrong. The correct alignment for max_align_t on i386 is 4,
not 8. It's a bug that GCC ever returns 8 for alignof on i386. We
really need to file a bug against GCC and explain this clearly,
because I have a feeling they're going to be opposed to fixing it...
Post by Paweł Dziepak
don't see why 8 is the right value and 4 isn't - System V ABI for x86
doesn't mention any type with alignment 8. Anyway, I agree that it
You're completely right; GCC is wrong.
Post by Paweł Dziepak
would be a good thing to mach the definition gcc and clang use, i.e.
union max_align_t {
alignas(long long) long long _ll;
alignas(long double) long double _ld;
};
This should not give results different from omitting the "alignas".
The only reason it does give different results is a bug in GCC, so we
should not be copying this confusing mess that's a no-op for a correct
compiler. (Applying alignas(T) to type T is always a no-op.)
I should have checked whether GCC 4.9 has changed before sending that.
As I said earlier, alignof in 4.9 seems to be fixed and on i386 for
fundamental types values <=4 are returned. alignof(max_align_t)
remains 8, though.

However, while 4, undobtedly, is the expected value of
alignof(max_align_t) I don't think that 8 is really wrong (well, from
the C11 point of view). The standard is not very specific about what
max_align_t really should be and if the compiler supports larger
alignment in all contexts there is no reason that alignof(max_align_t)
cannot be larger than alignof() of the type with the strictest
alignment requirements.
Obviously, since max_align_t is the part of ABI it is not like the
implementation can set alignof(max_align_t) to any value or it would
risk compatibility problems with binaries compiled with different
max_align_t. Since both GCC and Clang already define max_align_t so
that its alignment is 8 on i386 I think that Musl should do the same.

Paweł

PS Please, do not remove me from CC.
Rich Felker
2014-05-07 03:13:06 UTC
Permalink
Post by Paweł Dziepak
Post by Rich Felker
Post by Paweł Dziepak
would be a good thing to mach the definition gcc and clang use, i.e.
union max_align_t {
alignas(long long) long long _ll;
alignas(long double) long double _ld;
};
This should not give results different from omitting the "alignas".
The only reason it does give different results is a bug in GCC, so we
should not be copying this confusing mess that's a no-op for a correct
compiler. (Applying alignas(T) to type T is always a no-op.)
I should have checked whether GCC 4.9 has changed before sending that.
As I said earlier, alignof in 4.9 seems to be fixed and on i386 for
fundamental types values <=4 are returned. alignof(max_align_t)
remains 8, though.
Then GCC still has a bug. The above definition should give an
alignment of 4, not 8. Neither alignas(long long) nor alignas(long
double) should impose 8-byte alignment.
Post by Paweł Dziepak
However, while 4, undobtedly, is the expected value of
alignof(max_align_t) I don't think that 8 is really wrong (well, from
the C11 point of view). The standard is not very specific about what
max_align_t really should be and if the compiler supports larger
alignment in all contexts there is no reason that alignof(max_align_t)
cannot be larger than alignof() of the type with the strictest
alignment requirements.
Obviously, since max_align_t is the part of ABI it is not like the
implementation can set alignof(max_align_t) to any value or it would
risk compatibility problems with binaries compiled with different
max_align_t. Since both GCC and Clang already define max_align_t so
that its alignment is 8 on i386 I think that Musl should do the same.
If we want to achieve an alignment of 8, the above definition is
wrong; it will no longer have alignment 8 once the bug is fixed.
However I'm not convinced it's the right thing to do. Defining it as 8
is tightening malloc's contract to always return 8-byte-aligned memory
(note that it presently returns at least 16-byte alignment anyway, but
this is an implementation detail that's not meant to be observable,
not part of the interface contract).

Rich
Luca Barbato
2014-05-07 04:14:38 UTC
Permalink
Post by Rich Felker
If we want to achieve an alignment of 8, the above definition is
wrong; it will no longer have alignment 8 once the bug is fixed.
However I'm not convinced it's the right thing to do. Defining it as 8
is tightening malloc's contract to always return 8-byte-aligned memory
(note that it presently returns at least 16-byte alignment anyway, but
this is an implementation detail that's not meant to be observable,
not part of the interface contract).
The current natural alignment shouldn't be 32 for AVX and 16 for SSE ?

Not sure how wasteful would be but it would be surely a boon for the
applications I'm mostly involved.

lu
Rich Felker
2014-05-07 04:29:12 UTC
Permalink
Post by Luca Barbato
Post by Rich Felker
If we want to achieve an alignment of 8, the above definition is
wrong; it will no longer have alignment 8 once the bug is fixed.
However I'm not convinced it's the right thing to do. Defining it as 8
is tightening malloc's contract to always return 8-byte-aligned memory
(note that it presently returns at least 16-byte alignment anyway, but
this is an implementation detail that's not meant to be observable,
not part of the interface contract).
The current natural alignment shouldn't be 32 for AVX and 16 for SSE ?
Not sure how wasteful would be but it would be surely a boon for the
applications I'm mostly involved.
If you're working with data that needs additional alignment, you have
to use aligned_alloc (C11), posix_memalign (POSIX), or memalign
(legacy). Just assuming the result of malloc will be aligned beyond
the alignment requirements of any standard type is unsafe.

Rich
Luca Barbato
2014-05-07 05:12:39 UTC
Permalink
Post by Rich Felker
Post by Luca Barbato
Post by Rich Felker
If we want to achieve an alignment of 8, the above definition is
wrong; it will no longer have alignment 8 once the bug is fixed.
However I'm not convinced it's the right thing to do. Defining it as 8
is tightening malloc's contract to always return 8-byte-aligned memory
(note that it presently returns at least 16-byte alignment anyway, but
this is an implementation detail that's not meant to be observable,
not part of the interface contract).
The current natural alignment shouldn't be 32 for AVX and 16 for SSE ?
Not sure how wasteful would be but it would be surely a boon for the
applications I'm mostly involved.
If you're working with data that needs additional alignment, you have
That's the part that is annoying, the larger register is 32byte in those
platforms.
Post by Rich Felker
to use aligned_alloc (C11), posix_memalign (POSIX), or memalign
(legacy). Just assuming the result of malloc will be aligned beyond
the alignment requirements of any standard type is unsafe.
That we do already obviously, with the additional fun of not having a
realloc matching the mentioned functions in most platforms.

Having the memory functions 32-byte aligned and a mean to probe for it
would simplify a lot of code.

lu
Rich Felker
2014-05-07 22:48:41 UTC
Permalink
Post by Luca Barbato
Post by Rich Felker
Post by Luca Barbato
Post by Rich Felker
If we want to achieve an alignment of 8, the above definition is
wrong; it will no longer have alignment 8 once the bug is fixed.
However I'm not convinced it's the right thing to do. Defining it as 8
is tightening malloc's contract to always return 8-byte-aligned memory
(note that it presently returns at least 16-byte alignment anyway, but
this is an implementation detail that's not meant to be observable,
not part of the interface contract).
The current natural alignment shouldn't be 32 for AVX and 16 for SSE ?
Not sure how wasteful would be but it would be surely a boon for the
applications I'm mostly involved.
If you're working with data that needs additional alignment, you have
That's the part that is annoying, the larger register is 32byte in those
platforms.
And it will keep getting larger. Obviously changing the definition of
types and/or the ABI again and again is not the solution. The solution
is requesting the alignment you want.

BTW note that in the case of audio and video, depending on which
sample you start at, your data will not be aligned even if the start
of the buffer is aligned (think video filters working on a sub-image,
for example). So in cases like that your code should just handle the
misaligned head/tail parts separately. Note that GCC (and AFAIK
clang/llvm) already do this for you with -O3 and a -march that
supports vector ops.
Post by Luca Barbato
Post by Rich Felker
to use aligned_alloc (C11), posix_memalign (POSIX), or memalign
(legacy). Just assuming the result of malloc will be aligned beyond
the alignment requirements of any standard type is unsafe.
That we do already obviously, with the additional fun of not having a
realloc matching the mentioned functions in most platforms.
This is really a minor limitation. realloc cannot realistically be
expected to grow an object in-place in most cases; the only common
exception is when you're working with new memory at the top of the
heap and there's nothing else making small allocations that might get
placed right after your buffer that needs to grow.

In particular, if you're using a sane growth stratgey (geometric),
almost all calls to realloc are likely to move the buffer, so you're
just as well off calling malloc (or aligned_alloc) and free yourself.
The main exception to this might be for HUGE buffers where realloc can
use mremap with MREMAP_MAYMOVE.
Post by Luca Barbato
Having the memory functions 32-byte aligned and a mean to probe for it
would simplify a lot of code.
I think it would complicate the code because now you'd have two cases
to maintain, the case for implementations that always give excessive
alignment and the case for ones that don't. And one code path would
likely bitrot and have bugs.

In any case, the overhead would be undesirable. If/when I make some
improvements to malloc and its strategy for returning memory for use
by other processes (freeing commit charge), I'm also hoping to drop
the granularity on 64-bit platforms from 32 down to 16 or maybe even
smaller. There's really no need to store a size_t in the headers for
chunks which are only used for allocation sizes up to 128k/256k. This
kind of thing potentially makes a big difference for bloated OO/C++
apps that are allocating tons of tiny objects like linked list nodes
that are just 3 pointers (next/prev/data) and short strings.

Rich
Luca Barbato
2014-05-08 12:07:20 UTC
Permalink
Post by Rich Felker
Post by Luca Barbato
Post by Rich Felker
Post by Luca Barbato
Post by Rich Felker
If we want to achieve an alignment of 8, the above definition is
wrong; it will no longer have alignment 8 once the bug is fixed.
However I'm not convinced it's the right thing to do. Defining it as 8
is tightening malloc's contract to always return 8-byte-aligned memory
(note that it presently returns at least 16-byte alignment anyway, but
this is an implementation detail that's not meant to be observable,
not part of the interface contract).
The current natural alignment shouldn't be 32 for AVX and 16 for SSE ?
Not sure how wasteful would be but it would be surely a boon for the
applications I'm mostly involved.
If you're working with data that needs additional alignment, you have
That's the part that is annoying, the larger register is 32byte in those
platforms.
And it will keep getting larger. Obviously changing the definition of
types and/or the ABI again and again is not the solution. The solution
is requesting the alignment you want.
No, but having the correct value for new architectures would be sort of
more correct. Is not that those won't be used.
Post by Rich Felker
BTW note that in the case of audio and video, depending on which
sample you start at, your data will not be aligned even if the start
of the buffer is aligned (think video filters working on a sub-image,
for example).
I know quite well =)
Post by Rich Felker
So in cases like that your code should just handle the
misaligned head/tail parts separately. Note that GCC (and AFAIK
clang/llvm) already do this for you with -O3 and a -march that
supports vector ops.
That isn't the concern at all =)
Post by Rich Felker
Post by Luca Barbato
Post by Rich Felker
to use aligned_alloc (C11), posix_memalign (POSIX), or memalign
(legacy). Just assuming the result of malloc will be aligned beyond
the alignment requirements of any standard type is unsafe.
That we do already obviously, with the additional fun of not having a
realloc matching the mentioned functions in most platforms.
This is really a minor limitation. realloc cannot realistically be
expected to grow an object in-place in most cases; the only common
exception is when you're working with new memory at the top of the
heap and there's nothing else making small allocations that might get
placed right after your buffer that needs to grow.
In particular, if you're using a sane growth stratgey (geometric),
almost all calls to realloc are likely to move the buffer, so you're
just as well off calling malloc (or aligned_alloc) and free yourself.
The main exception to this might be for HUGE buffers where realloc can
use mremap with MREMAP_MAYMOVE.
That's what we are doing but as you know quite well, less custom code
the better.
Post by Rich Felker
I think it would complicate the code because now you'd have two cases
to maintain, the case for implementations that always give excessive
alignment and the case for ones that don't. And one code path would
likely bitrot and have bugs.
Not really, we already have platforms doing that so the code is there
and exercised on those.
Post by Rich Felker
In any case, the overhead would be undesirable. If/when I make some
improvements to malloc and its strategy for returning memory for use
by other processes (freeing commit charge), I'm also hoping to drop
the granularity on 64-bit platforms from 32 down to 16 or maybe even
smaller. There's really no need to store a size_t in the headers for
chunks which are only used for allocation sizes up to 128k/256k.
I see, nice to know that's the plan =) As said it would had been a nice
to have if it comes for free.

lu
Rich Felker
2014-05-08 14:25:08 UTC
Permalink
Post by Luca Barbato
Post by Rich Felker
Post by Luca Barbato
That's the part that is annoying, the larger register is 32byte in those
platforms.
And it will keep getting larger. Obviously changing the definition of
types and/or the ABI again and again is not the solution. The solution
is requesting the alignment you want.
No, but having the correct value for new architectures would be sort of
more correct. Is not that those won't be used.
I'm not sure what you mean by "the correct value". The definition with
the union should give the correct value (max alignment requirement of
any standard type) for all archs.
Post by Luca Barbato
Post by Rich Felker
In any case, the overhead would be undesirable. If/when I make some
improvements to malloc and its strategy for returning memory for use
by other processes (freeing commit charge), I'm also hoping to drop
the granularity on 64-bit platforms from 32 down to 16 or maybe even
smaller. There's really no need to store a size_t in the headers for
chunks which are only used for allocation sizes up to 128k/256k.
I see, nice to know that's the plan =) As said it would had been a nice
to have if it comes for free.
Yeah, not much comes for free though...

Rich
Paweł Dziepak
2014-05-07 09:28:58 UTC
Permalink
Post by Rich Felker
Post by Paweł Dziepak
Post by Rich Felker
Post by Paweł Dziepak
would be a good thing to mach the definition gcc and clang use, i.e.
union max_align_t {
alignas(long long) long long _ll;
alignas(long double) long double _ld;
};
This should not give results different from omitting the "alignas".
The only reason it does give different results is a bug in GCC, so we
should not be copying this confusing mess that's a no-op for a correct
compiler. (Applying alignas(T) to type T is always a no-op.)
I should have checked whether GCC 4.9 has changed before sending that.
As I said earlier, alignof in 4.9 seems to be fixed and on i386 for
fundamental types values <=4 are returned. alignof(max_align_t)
remains 8, though.
Then GCC still has a bug. The above definition should give an
alignment of 4, not 8. Neither alignas(long long) nor alignas(long
double) should impose 8-byte alignment.
To clarify: GCC defines max_align_t so its alignment is 8. My original
definition (without alignas) makes max_align_t 4-byte-aligned (both
GCC 4.8.2 and 4.9). My second definition (with alignas) results in
8-byte-aligned max_align_t on GCC 4.8.2 and bug in GCC 4.9
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=61053

GCC uses its own __alignof__ to define max_align_t. __alignof__
returns the recommended alignment (as oposed to minimal in case of
alignof), which in case of long long is 8.
Post by Rich Felker
Post by Paweł Dziepak
However, while 4, undobtedly, is the expected value of
alignof(max_align_t) I don't think that 8 is really wrong (well, from
the C11 point of view). The standard is not very specific about what
max_align_t really should be and if the compiler supports larger
alignment in all contexts there is no reason that alignof(max_align_t)
cannot be larger than alignof() of the type with the strictest
alignment requirements.
Obviously, since max_align_t is the part of ABI it is not like the
implementation can set alignof(max_align_t) to any value or it would
risk compatibility problems with binaries compiled with different
max_align_t. Since both GCC and Clang already define max_align_t so
that its alignment is 8 on i386 I think that Musl should do the same.
If we want to achieve an alignment of 8, the above definition is
wrong; it will no longer have alignment 8 once the bug is fixed.
However I'm not convinced it's the right thing to do. Defining it as 8
is tightening malloc's contract to always return 8-byte-aligned memory
(note that it presently returns at least 16-byte alignment anyway, but
this is an implementation detail that's not meant to be observable,
not part of the interface contract).
I've mentioned earlier that it seems that the only option is to use
GCC extensions (i.e. __alignof__) to match their definition of
max_align_t, just like it is done in this patch:
http://www.openwall.com/lists/musl/2014/04/28/3
It is not nice that GCC forces malloc to support "extended" alignment
but I don't think there is much that can be done about it.

Paweł
Rich Felker
2014-05-07 23:07:29 UTC
Permalink
Post by Paweł Dziepak
Post by Rich Felker
Post by Paweł Dziepak
Post by Rich Felker
Post by Paweł Dziepak
would be a good thing to mach the definition gcc and clang use, i.e.
union max_align_t {
alignas(long long) long long _ll;
alignas(long double) long double _ld;
};
This should not give results different from omitting the "alignas".
The only reason it does give different results is a bug in GCC, so we
should not be copying this confusing mess that's a no-op for a correct
compiler. (Applying alignas(T) to type T is always a no-op.)
I should have checked whether GCC 4.9 has changed before sending that.
As I said earlier, alignof in 4.9 seems to be fixed and on i386 for
fundamental types values <=4 are returned. alignof(max_align_t)
remains 8, though.
Then GCC still has a bug. The above definition should give an
alignment of 4, not 8. Neither alignas(long long) nor alignas(long
double) should impose 8-byte alignment.
To clarify: GCC defines max_align_t so its alignment is 8. My original
definition (without alignas) makes max_align_t 4-byte-aligned (both
GCC 4.8.2 and 4.9). My second definition (with alignas) results in
8-byte-aligned max_align_t on GCC 4.8.2 and bug in GCC 4.9
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=61053
GCC uses its own __alignof__ to define max_align_t. __alignof__
returns the recommended alignment (as oposed to minimal in case of
alignof), which in case of long long is 8.
That's very confusing, so thanks for clarifying.

BTW, is __alignof__(long double) really giving 8? If so that's utter
nonsense. sizeof(long double) is 12, and alignment must always divide
the size of the type...
Post by Paweł Dziepak
Post by Rich Felker
Post by Paweł Dziepak
However, while 4, undobtedly, is the expected value of
alignof(max_align_t) I don't think that 8 is really wrong (well, from
the C11 point of view). The standard is not very specific about what
max_align_t really should be and if the compiler supports larger
alignment in all contexts there is no reason that alignof(max_align_t)
cannot be larger than alignof() of the type with the strictest
alignment requirements.
Obviously, since max_align_t is the part of ABI it is not like the
implementation can set alignof(max_align_t) to any value or it would
risk compatibility problems with binaries compiled with different
max_align_t. Since both GCC and Clang already define max_align_t so
that its alignment is 8 on i386 I think that Musl should do the same.
If we want to achieve an alignment of 8, the above definition is
wrong; it will no longer have alignment 8 once the bug is fixed.
However I'm not convinced it's the right thing to do. Defining it as 8
is tightening malloc's contract to always return 8-byte-aligned memory
(note that it presently returns at least 16-byte alignment anyway, but
this is an implementation detail that's not meant to be observable,
not part of the interface contract).
I've mentioned earlier that it seems that the only option is to use
GCC extensions (i.e. __alignof__) to match their definition of
http://www.openwall.com/lists/musl/2014/04/28/3
It is not nice that GCC forces malloc to support "extended" alignment
but I don't think there is much that can be done about it.
I don't see how GCC "forces" this. The definition of max_align_t from
glibc's stddef.h, which we do not use, is what forces it.

As I see it, we have a choice whether to use the "8" definition on
i386 or use the natural definition, which would yield "4" on i386.
This is not an ABI issue (it does not affect the ability to use
glibc-built object files/binaries/shared libraries with musl, nor the
C++ name mangling ABI) but an API issue.

Moreover, I can't see any easy way that code assuming GCC's definition
could cause breakage on musl if we went with the other definition. The
x86 instructions that require additional alignment require at least
16-byte alignment, not 8-byte, so aligning as max_align_t will not
produce objects that can be used with sse instructions directly.

Rich
Szabolcs Nagy
2014-05-08 10:57:22 UTC
Permalink
Post by Rich Felker
BTW, is __alignof__(long double) really giving 8? If so that's utter
nonsense. sizeof(long double) is 12, and alignment must always divide
the size of the type...
no, long long is 8 though

#include <stdio.h>
#define p(x) printf("%s == %d\n", #x, (int)(x))
int main()
{
p(__alignof__(long long));
p(__alignof__(long double));
p(__alignof__(union{long long a; long double b;}));
p(_Alignof(long long));
p(_Alignof(long double));
p(_Alignof(union{long long a; long double b;}));
}

gcc 4.8 on i386:
__alignof__(long long) == 8
__alignof__(long double) == 4
__alignof__(union{long long a; long double b;}) == 4
_Alignof(long long) == 8
_Alignof(long double) == 4
_Alignof(union{long long a; long double b;}) == 4

gcc 4.9 on i386:
__alignof__(long long) == 8
__alignof__(long double) == 4
__alignof__(union{long long a; long double b;}) == 4
_Alignof(long long) == 4
_Alignof(long double) == 4
_Alignof(union{long long a; long double b;}) == 4

gcc 4.9 on arm:
__alignof__(long long) == 8
__alignof__(long double) == 8
__alignof__(union{long long a; long double b;}) == 8
_Alignof(long long) == 8
_Alignof(long double) == 8
_Alignof(union{long long a; long double b;}) == 8
Post by Rich Felker
As I see it, we have a choice whether to use the "8" definition on
i386 or use the natural definition, which would yield "4" on i386.
This is not an ABI issue (it does not affect the ability to use
glibc-built object files/binaries/shared libraries with musl, nor the
C++ name mangling ABI) but an API issue.
assuming max_align_t does not appear in a function prototype
Rich Felker
2014-05-08 14:11:58 UTC
Permalink
Post by Szabolcs Nagy
Post by Rich Felker
As I see it, we have a choice whether to use the "8" definition on
i386 or use the natural definition, which would yield "4" on i386.
This is not an ABI issue (it does not affect the ability to use
glibc-built object files/binaries/shared libraries with musl, nor the
C++ name mangling ABI) but an API issue.
assuming max_align_t does not appear in a function prototype
As long as the union tag and size are the same, it should not matter.
I don't think alignments affect i386 argument passing convention at
all. Certainly the traditional va_arg macro does not account for them;
it just assumes every argument is aligned to 4 bytes.

Rich
Paweł Dziepak
2014-05-08 16:41:19 UTC
Permalink
Post by Rich Felker
Post by Paweł Dziepak
Post by Rich Felker
Post by Paweł Dziepak
However, while 4, undobtedly, is the expected value of
alignof(max_align_t) I don't think that 8 is really wrong (well, from
the C11 point of view). The standard is not very specific about what
max_align_t really should be and if the compiler supports larger
alignment in all contexts there is no reason that alignof(max_align_t)
cannot be larger than alignof() of the type with the strictest
alignment requirements.
Obviously, since max_align_t is the part of ABI it is not like the
implementation can set alignof(max_align_t) to any value or it would
risk compatibility problems with binaries compiled with different
max_align_t. Since both GCC and Clang already define max_align_t so
that its alignment is 8 on i386 I think that Musl should do the same.
If we want to achieve an alignment of 8, the above definition is
wrong; it will no longer have alignment 8 once the bug is fixed.
However I'm not convinced it's the right thing to do. Defining it as 8
is tightening malloc's contract to always return 8-byte-aligned memory
(note that it presently returns at least 16-byte alignment anyway, but
this is an implementation detail that's not meant to be observable,
not part of the interface contract).
I've mentioned earlier that it seems that the only option is to use
GCC extensions (i.e. __alignof__) to match their definition of
http://www.openwall.com/lists/musl/2014/04/28/3
It is not nice that GCC forces malloc to support "extended" alignment
but I don't think there is much that can be done about it.
I don't see how GCC "forces" this. The definition of max_align_t from
glibc's stddef.h, which we do not use, is what forces it.
It is GCC that provides stddef.h, not glibc.
Post by Rich Felker
As I see it, we have a choice whether to use the "8" definition on
i386 or use the natural definition, which would yield "4" on i386.
This is not an ABI issue (it does not affect the ability to use
glibc-built object files/binaries/shared libraries with musl, nor the
C++ name mangling ABI) but an API issue.
What about something like this?

struct foobar {
char foo;
alignas(max_align_t) char bar;
};

The binary representation of this structure depends on the definition
of max_align_t and binaries compiled with different
alignof(max_align_t) won't be compatible.

Paweł
Rich Felker
2014-05-08 17:41:38 UTC
Permalink
Post by Paweł Dziepak
Post by Rich Felker
As I see it, we have a choice whether to use the "8" definition on
i386 or use the natural definition, which would yield "4" on i386.
This is not an ABI issue (it does not affect the ability to use
glibc-built object files/binaries/shared libraries with musl, nor the
C++ name mangling ABI) but an API issue.
What about something like this?
struct foobar {
char foo;
alignas(max_align_t) char bar;
};
The binary representation of this structure depends on the definition
of max_align_t and binaries compiled with different
alignof(max_align_t) won't be compatible.
Indeed. This is not an ABI issue with libc.so, but it's an API
difference that translates into an ABI difference between third-party
translation units if they use max_align_t as you've described. Whether
we care, I'm not sure. At least historically there were other cases
like this in musl where type sizes differed in ways that didn't affect
libc ABI but did affect ABI between third-party programs (jmp_buf
comes to mind) but most if not all of these were changed.

I'm not strongly opposed to imposing the 8-byte alignment requirement
on malloc (it would be hard to make malloc work on smaller granularity
anyway, and most archs need 8-byte alignment anyway for long long and
for double) but I generally dislike the inconsistency of defining
max_align_t in a semi-absurd way on i386, as well as the issue of
having to use non-portable definitions and/or arch-specific ones.

BTW in your above example, it's not even clear to me if that use of
alignas is valid. It makes no sense for an object to have an alignment
that does not divide its type (imagine if you added [2] to the end of
bar) and in other places (like the contract of aligned_alloc)
alignments that do not divide the size are explicitly illegal. I'd
like to understand this before making a decision.

Rich
Jens Gustedt
2014-05-08 18:45:05 UTC
Permalink
Hello,
Post by Rich Felker
BTW in your above example, it's not even clear to me if that use of
alignas is valid.
Besides the question of such a thing makes sense or noţ with the
current version of the standard it isn't syntactically valid. alignas
can't be applied to struct fields. So for the moment the whole
discussion in the standard about types with extended alignment is
pointless.

This issue has been raised as a defect report, and it seems that the
committee agrees to change this in a corrigendum.

http://www.open-std.org/jtc1/sc22/wg14/www/docs/dr_444.htm

Jens
--
:: INRIA Nancy Grand Est :: http://www.loria.fr/~gustedt/ ::
:: AlGorille ::::::::::::::: office Nancy : +33 383593090 ::
:: ICube :::::::::::::: office Strasbourg : +33 368854536 ::
:: ::::::::::::::::::::::::::: gsm France : +33 651400183 ::
:: :::::::::::::::::::: gsm international : +49 15737185122 ::
Paweł Dziepak
2014-05-08 19:11:24 UTC
Permalink
Post by Jens Gustedt
Hello,
Post by Rich Felker
BTW in your above example, it's not even clear to me if that use of
alignas is valid.
Besides the question of such a thing makes sense or noţ with the
current version of the standard it isn't syntactically valid. alignas
can't be applied to struct fields. So for the moment the whole
discussion in the standard about types with extended alignment is
pointless.
This issue has been raised as a defect report, and it seems that the
committee agrees to change this in a corrigendum.
http://www.open-std.org/jtc1/sc22/wg14/www/docs/dr_444.htm
My bad, it is legal in C++11 though.

Paweł
Jens Gustedt
2014-05-08 19:22:06 UTC
Permalink
Post by Paweł Dziepak
Post by Jens Gustedt
Hello,
Post by Rich Felker
BTW in your above example, it's not even clear to me if that use of
alignas is valid.
Besides the question of such a thing makes sense or noţ with the
current version of the standard it isn't syntactically valid. alignas
can't be applied to struct fields. So for the moment the whole
discussion in the standard about types with extended alignment is
pointless.
This issue has been raised as a defect report, and it seems that the
committee agrees to change this in a corrigendum.
http://www.open-std.org/jtc1/sc22/wg14/www/docs/dr_444.htm
My bad,
no, no, the "bad" is clearly in the standard, this is by no means easy
to read nor to understand

all that max_align_t discussion shows that the alignment stuff in the
standard is underspecified and is missing semantics.

Jens
--
:: INRIA Nancy Grand Est :: http://www.loria.fr/~gustedt/ ::
:: AlGorille ::::::::::::::: office Nancy : +33 383593090 ::
:: ICube :::::::::::::: office Strasbourg : +33 368854536 ::
:: ::::::::::::::::::::::::::: gsm France : +33 651400183 ::
:: :::::::::::::::::::: gsm international : +49 15737185122 ::
Paweł Dziepak
2014-05-08 19:45:39 UTC
Permalink
Post by Rich Felker
Post by Paweł Dziepak
Post by Rich Felker
As I see it, we have a choice whether to use the "8" definition on
i386 or use the natural definition, which would yield "4" on i386.
This is not an ABI issue (it does not affect the ability to use
glibc-built object files/binaries/shared libraries with musl, nor the
C++ name mangling ABI) but an API issue.
What about something like this?
struct foobar {
char foo;
alignas(max_align_t) char bar;
};
The binary representation of this structure depends on the definition
of max_align_t and binaries compiled with different
alignof(max_align_t) won't be compatible.
Indeed. This is not an ABI issue with libc.so, but it's an API
difference that translates into an ABI difference between third-party
translation units if they use max_align_t as you've described. Whether
we care, I'm not sure. At least historically there were other cases
like this in musl where type sizes differed in ways that didn't affect
libc ABI but did affect ABI between third-party programs (jmp_buf
comes to mind) but most if not all of these were changed.
I'm not strongly opposed to imposing the 8-byte alignment requirement
on malloc (it would be hard to make malloc work on smaller granularity
anyway, and most archs need 8-byte alignment anyway for long long and
for double) but I generally dislike the inconsistency of defining
max_align_t in a semi-absurd way on i386, as well as the issue of
having to use non-portable definitions and/or arch-specific ones.
I agree. The question remains, though, whether the consequences of
defining max_align_t differently are acceptable.
Post by Rich Felker
BTW in your above example, it's not even clear to me if that use of
alignas is valid. It makes no sense for an object to have an alignment
that does not divide its type (imagine if you added [2] to the end of
bar) and in other places (like the contract of aligned_alloc)
alignments that do not divide the size are explicitly illegal. I'd
like to understand this before making a decision.
6.7.5 doesn't mention such requirement. _Alignas, obviously, cannot
reduce the alignment requirement and the specified alignment has to
has to be either a valid fundamental alignment or valid extended
alignment supported by the implementation. Moreover, 6.2.8 requires
that valid alignment is a nonnegative integral power of two. As for
the additional requirement in contract of aligned_alloc 7.22.3.1
states that the requested alignment has to be valid and divide size of
the requested memory block. I don't see how that would disallow using
in alignas alignment larger than the size of the object.

Paweł
Rich Felker
2014-05-08 20:02:30 UTC
Permalink
Post by Paweł Dziepak
6.7.5 doesn't mention such requirement. _Alignas, obviously, cannot
reduce the alignment requirement and the specified alignment has to
has to be either a valid fundamental alignment or valid extended
alignment supported by the implementation. Moreover, 6.2.8 requires
that valid alignment is a nonnegative integral power of two. As for
the additional requirement in contract of aligned_alloc 7.22.3.1
states that the requested alignment has to be valid and divide size of
the requested memory block. I don't see how that would disallow using
in alignas alignment larger than the size of the object.
The alignment of a type must divide its size; this is fundamental to
the existence of arrays. It's possible that, for an ugly definition of
"alignment of an object" independent of an alignment associated with
the type, some objects could be aligned with more alignment than their
size, but I'm not convinced that the standard intends to allow such
nonsense. My point about aligned_alloc was that its interface
requirements reflect the notion that alignment always divides size.

Rich
Paweł Dziepak
2014-05-08 20:45:20 UTC
Permalink
Post by Rich Felker
Post by Paweł Dziepak
6.7.5 doesn't mention such requirement. _Alignas, obviously, cannot
reduce the alignment requirement and the specified alignment has to
has to be either a valid fundamental alignment or valid extended
alignment supported by the implementation. Moreover, 6.2.8 requires
that valid alignment is a nonnegative integral power of two. As for
the additional requirement in contract of aligned_alloc 7.22.3.1
states that the requested alignment has to be valid and divide size of
the requested memory block. I don't see how that would disallow using
in alignas alignment larger than the size of the object.
The alignment of a type must divide its size; this is fundamental to
the existence of arrays. It's possible that, for an ugly definition of
"alignment of an object" independent of an alignment associated with
the type, some objects could be aligned with more alignment than their
size, but I'm not convinced that the standard intends to allow such
nonsense. My point about aligned_alloc was that its interface
requirements reflect the notion that alignment always divides size.
I'm not sure how alignas() could cause problems with arrays. It cannot
be used in declarations of typedefs. When applied to structure member
or the whole structure (like struct alignas(N) foo { ... }, C++11
alows it) it changes the size of the structure itself so the alignment
remains less than or equal size of the object and in declarations like
this:

alignas(N) T foo[K];

alignas() applies to the array foo, not each individual element of that array.

Paweł

Continue reading on narkive:
Loading...