Discussion:
musl and android
(too old to reply)
Рысь
2015-01-15 09:13:22 UTC
Permalink
Hello all!

Are there any efforts or even a project which aims to port musl to
android platform?

For a year I slowly had built and running a couple of C tty only
programs such as iptables, tcpdump and of course busybox and many
others including my own with musl libc linked statically. They work
perfectly (no much differences between an ARM phone and, for example,
raspberry pi SoC) but a few items I missed:

+ Proper DNS resolving is not available
+ Translation of android special user names is not done

As an advanced Linux user I know that android is not friendly enough to
plain C stuff and it's libc is even not finished now so I aimed to port
at least listed things to musl.

Because I am not going beyond listed items, a patch can be developed
just to support these inside musl-linked binaries.

I am first here or there is already someone who done this before?
Rich Felker
2015-01-15 11:01:58 UTC
Permalink
Post by Рысь
Hello all!
Are there any efforts or even a project which aims to port musl to
android platform?
For a year I slowly had built and running a couple of C tty only
programs such as iptables, tcpdump and of course busybox and many
others including my own with musl libc linked statically. They work
perfectly (no much differences between an ARM phone and, for example,
+ Proper DNS resolving is not available
+ Translation of android special user names is not done
As an advanced Linux user I know that android is not friendly enough to
plain C stuff and it's libc is even not finished now so I aimed to port
at least listed things to musl.
Because I am not going beyond listed items, a patch can be developed
just to support these inside musl-linked binaries.
I am first here or there is already someone who done this before?
Hi,

The above issues are complicated by the fact that musl aims to produce
binaries that work anywhere and that follow existing practice rather
than inventing new mechanisms -- a principle which Android ignored or
even treated with hostility.

Supporting alternate passwd/group backends is an open agenda item, but
the proposed methods we're looking at are still not really "Android
compatible". Same goes for DNS -- the recommended way to do alternate
name resolver backends is a local nameserver, but that's not very
friendly to the Android setup either.

I'm not sure what the best approach here is. Perhaps it's to treat
Android as a new body of existing practice and also support the
gratuitously-different Android things inline in the same binaries.
This seems ugly but perhaps preferable to having multiple incompatible
targets.

For now, the easiest way to deploy musl on Android (by this, I mean as
part of an Android app) seems to be including your own, possibly
patched, copy of the dynamic linker (libc.so/ld-musl) in the package
and executing the program via a wrapper script that manually invokes
the dynamic linker (so the hard-coded PT_INTERP pathname isn't
needed). Or of course static linking works (with similar patching if
necessary). But these are not the approaches I'd like to be
recommending in the long term... :(

Rich
u***@aetey.se
2015-01-15 12:00:05 UTC
Permalink
Post by Rich Felker
copy of the dynamic linker (libc.so/ld-musl) in the package
and executing the program via a wrapper script that manually invokes
the dynamic linker (so the hard-coded PT_INTERP pathname isn't
needed).
But these are not the approaches I'd like to be
recommending in the long term... :(
Actually I believe (and know from long time experience) this to be
the only "sane"/robust/general way to run dynamically linked executables.

I don't think that the implications of hardcoding the interpreter
path were well understood when dynamic linking was first deployed,
the hardcoding merely became percepted as the only/natural approach
when the purpose was to cheaply imitate the behaviour of statically
linked programs. (This mimics the #!/... which is similarly
limited/broken. The plain text scripts are though relatively easy
to modify to hack around the limitation, according to local curcumstances)

Rune
Rich Felker
2015-01-15 12:15:36 UTC
Permalink
Post by u***@aetey.se
Post by Rich Felker
copy of the dynamic linker (libc.so/ld-musl) in the package
and executing the program via a wrapper script that manually invokes
the dynamic linker (so the hard-coded PT_INTERP pathname isn't
needed).
But these are not the approaches I'd like to be
recommending in the long term... :(
Actually I believe (and know from long time experience) this to be
the only "sane"/robust/general way to run dynamically linked executables.
It depends on your perspective. If you're viewing them as
self-contained entities, then yes, but if you're viewing them as
something running in an existing library ecosystem, there's no
problem.
Post by u***@aetey.se
I don't think that the implications of hardcoding the interpreter
path were well understood when dynamic linking was first deployed,
the hardcoding merely became percepted as the only/natural approach
when the purpose was to cheaply imitate the behaviour of statically
linked programs. (This mimics the #!/... which is similarly
limited/broken. The plain text scripts are though relatively easy
to modify to hack around the limitation, according to local curcumstances)
I think this could be fixed easily by having the kernel support
$ORIGIN in PT_INTERP.

Rich
u***@aetey.se
2015-01-15 13:04:32 UTC
Permalink
Post by Rich Felker
Post by u***@aetey.se
Post by Rich Felker
and executing the program via a wrapper script that manually invokes
the dynamic linker (so the hard-coded PT_INTERP pathname isn't
needed).
Actually I believe (and know from long time experience) this to be
the only "sane"/robust/general way to run dynamically linked executables.
It depends on your perspective. If you're viewing them as
self-contained entities, then yes, but if you're viewing them as
something running in an existing library ecosystem, there's no
problem.
Right, it depends. For the second perspective you seem to imply that
an "ecosystem" is to be managed in a certain way.

Binaries belonging to my "library ecosystems" still can be subject to
the C library tests and upgrades on a per-binary basis, not only "all
binaries belonging to the same ecosystem at once", which is of course
possible as well.

This would be impossible if I'd rely on the hardcoded loader path.
Post by Rich Felker
Post by u***@aetey.se
I don't think that the implications of hardcoding the interpreter
path were well understood when dynamic linking was first deployed,
the hardcoding merely became percepted as the only/natural approach
when the purpose was to cheaply imitate the behaviour of statically
linked programs. (This mimics the #!/... which is similarly
limited/broken. The plain text scripts are though relatively easy
to modify to hack around the limitation, according to local curcumstances)
I think this could be fixed easily by having the kernel support
$ORIGIN in PT_INTERP.
Unfortunately, no. $ORIGIN does not and can not replace a run time
choice of the dynamic loader. As a simple example, consider a binary on
a readonly media. How would you convince the kernel to run a different
loader than assumed (among others) by the path to the mount point of
the media? In my eyes the mounting of the media (possibly with lots of
binaries on it) and running a certain loader for a certain binary are
very different things and do not have to / should not depend on each other.

Rune
Rich Felker
2015-01-15 13:34:45 UTC
Permalink
Post by u***@aetey.se
Post by Rich Felker
Post by u***@aetey.se
Post by Rich Felker
and executing the program via a wrapper script that manually invokes
the dynamic linker (so the hard-coded PT_INTERP pathname isn't
needed).
Actually I believe (and know from long time experience) this to be
the only "sane"/robust/general way to run dynamically linked executables.
It depends on your perspective. If you're viewing them as
self-contained entities, then yes, but if you're viewing them as
something running in an existing library ecosystem, there's no
problem.
Right, it depends. For the second perspective you seem to imply that
an "ecosystem" is to be managed in a certain way.
Binaries belonging to my "library ecosystems" still can be subject to
the C library tests and upgrades on a per-binary basis, not only "all
binaries belonging to the same ecosystem at once", which is of course
possible as well.
This would be impossible if I'd rely on the hardcoded loader path.
That's why I said i depends on your perspective. From your perspective
this does not work. From most traditional distributions' perspectives,
it does.
Post by u***@aetey.se
Post by Rich Felker
Post by u***@aetey.se
I don't think that the implications of hardcoding the interpreter
path were well understood when dynamic linking was first deployed,
the hardcoding merely became percepted as the only/natural approach
when the purpose was to cheaply imitate the behaviour of statically
linked programs. (This mimics the #!/... which is similarly
limited/broken. The plain text scripts are though relatively easy
to modify to hack around the limitation, according to local curcumstances)
I think this could be fixed easily by having the kernel support
$ORIGIN in PT_INTERP.
Unfortunately, no. $ORIGIN does not and can not replace a run time
choice of the dynamic loader. As a simple example, consider a binary on
a readonly media. How would you convince the kernel to run a different
loader than assumed (among others) by the path to the mount point of
the media? In my eyes the mounting of the media (possibly with lots of
binaries on it) and running a certain loader for a certain binary are
very different things and do not have to / should not depend on each other.
If you're distributing the binary and dynamic loader/libc and all
libraries it needs together, I'd assume they'd all be in the same
directory, or else in ../lib/ relative to the binary. In that case
$ORIGIN works perfectly fine. Note that $ORIGIN is _not_ an
environment variable; it's a dynamic-linker feature for locating
libraries relative to the ELF file (main executable or other library)
that needs (via DT_NEEDED) them, and the same concept would work for
PT_INTERP.

Rich
u***@aetey.se
2015-01-15 15:56:38 UTC
Permalink
Actually I do not suggest any change, musl suits my usage model well
(kudos for the support of running the loader explicitly)!
Post by Rich Felker
That's why I said i depends on your perspective. From your perspective
this does not work. From most traditional distributions' perspectives,
it does.
Well said.

The traditional distributions choose to live with the far reaching
consequences of their technical choices. I argue that some constraints
(iow sacrifices) are not necessary and only exist as a consequence
of the unfortunate choices.
Post by Rich Felker
Post by u***@aetey.se
Unfortunately, no. $ORIGIN does not and can not replace a run time
choice of the dynamic loader.
If you're distributing the binary and dynamic loader/libc and all
libraries it needs together, I'd assume they'd all be in the same
directory, or else in ../lib/ relative to the binary. In that case
You suggest a somewhat more relaxed set of assumptions than the
traditional ones. Yes this would be useful, but unfortunately still
setting some more or less arbitrary boundaries.
Post by Rich Felker
$ORIGIN works perfectly fine. Note that $ORIGIN is _not_ an
environment variable; it's a dynamic-linker feature for locating
libraries relative to the ELF file (main executable or other library)
that needs (via DT_NEEDED) them, and the same concept would work for
PT_INTERP.
Indeed, good to point this out for a casual reader.

Using an environment variable would be otherwise a quite general solution
but I guess this would be very hard to convince the kernel developers to
implement (presumably hardly possible to implement without redesigning
the kernel security model).

Regards,
Rune
Рысь
2015-01-15 13:53:21 UTC
Permalink
В Thu, 15 Jan 2015 06:01:58 -0500
Post by Rich Felker
Post by Рысь
Hello all!
Are there any efforts or even a project which aims to port musl to
android platform?
For a year I slowly had built and running a couple of C tty only
programs such as iptables, tcpdump and of course busybox and many
others including my own with musl libc linked statically. They work
perfectly (no much differences between an ARM phone and, for
+ Proper DNS resolving is not available
+ Translation of android special user names is not done
As an advanced Linux user I know that android is not friendly
enough to plain C stuff and it's libc is even not finished now so I
aimed to port at least listed things to musl.
Because I am not going beyond listed items, a patch can be developed
just to support these inside musl-linked binaries.
I am first here or there is already someone who done this before?
Hi,
The above issues are complicated by the fact that musl aims to produce
binaries that work anywhere and that follow existing practice rather
than inventing new mechanisms -- a principle which Android ignored or
even treated with hostility.
As for now, I do not aim to make portable binaries since I do not
redistribute them (okay, I distribute them to my friends. Does that
count?)
Post by Rich Felker
Supporting alternate passwd/group backends is an open agenda item, but
the proposed methods we're looking at are still not really "Android
compatible". Same goes for DNS -- the recommended way to do alternate
name resolver backends is a local nameserver, but that's not very
friendly to the Android setup either.
Do not forget about timezones in android (I forgot to mention that -
sorry)
Post by Rich Felker
I'm not sure what the best approach here is. Perhaps it's to treat
Android as a new body of existing practice and also support the
gratuitously-different Android things inline in the same binaries.
This seems ugly but perhaps preferable to having multiple incompatible
targets.
For now, the easiest way to deploy musl on Android (by this, I mean as
part of an Android app) seems to be including your own, possibly
patched, copy of the dynamic linker (libc.so/ld-musl) in the package
and executing the program via a wrapper script that manually invokes
the dynamic linker (so the hard-coded PT_INTERP pathname isn't
needed). Or of course static linking works (with similar patching if
necessary). But these are not the approaches I'd like to be
recommending in the long term... :(
Rich
No I am not using musl as a part of android app. I use musl in low
level C programs (relative to Java user apps) such as busybox, strace,
tcpdump, iptables, iproute2... That's why I do seek for a quick
solution now.

Android is not friendly at all, but I need to do something with it. I
want musl on it working fine, because I do want get rid of ugly android
development toolchain and it's partial libc. I completely tired of that
environment.

So, if there are no any other attempts, I can try?

Thanks.
Рысь
2015-01-18 06:32:37 UTC
Permalink
В Thu, 15 Jan 2015 16:13:22 +0700
Post by Рысь
Hello all!
Are there any efforts or even a project which aims to port musl to
android platform?
For a year I slowly had built and running a couple of C tty only
programs such as iptables, tcpdump and of course busybox and many
others including my own with musl libc linked statically. They work
perfectly (no much differences between an ARM phone and, for example,
+ Proper DNS resolving is not available
+ Translation of android special user names is not done
As an advanced Linux user I know that android is not friendly enough
to plain C stuff and it's libc is even not finished now so I aimed to
port at least listed things to musl.
Because I am not going beyond listed items, a patch can be developed
just to support these inside musl-linked binaries.
I am first here or there is already someone who done this before?
I made a patch to implement all three things for musl on android.

I do not post it here as attachment to this mail because it normally
should not be integrated with musl. The patch itself is here:
http://lynxlynx.tk/prg/patches/musl-1.1.4_android.patch, notes about
it: http://lynxlynx.tk/eng/musl_android/.

Patch is invasive enough I think and not optimized and I agree with Rich
that it must go as a special treat. I tried however move away all
translating code to src/android and headers to include/android.

Patch is still in it's alpha stage, and probably will be changed often.

Modifications made for musl 1.1.4.

If anyone interested to audit it - welcome!
Rich Felker
2015-01-18 06:44:40 UTC
Permalink
Post by Рысь
В Thu, 15 Jan 2015 16:13:22 +0700
Post by Рысь
Hello all!
Are there any efforts or even a project which aims to port musl to
android platform?
For a year I slowly had built and running a couple of C tty only
programs such as iptables, tcpdump and of course busybox and many
others including my own with musl libc linked statically. They work
perfectly (no much differences between an ARM phone and, for example,
+ Proper DNS resolving is not available
+ Translation of android special user names is not done
As an advanced Linux user I know that android is not friendly enough
to plain C stuff and it's libc is even not finished now so I aimed to
port at least listed things to musl.
Because I am not going beyond listed items, a patch can be developed
just to support these inside musl-linked binaries.
I am first here or there is already someone who done this before?
I made a patch to implement all three things for musl on android.
I do not post it here as attachment to this mail because it normally
http://lynxlynx.tk/prg/patches/musl-1.1.4_android.patch, notes about
it: http://lynxlynx.tk/eng/musl_android/.
Patch is invasive enough I think and not optimized and I agree with Rich
that it must go as a special treat. I tried however move away all
translating code to src/android and headers to include/android.
Patch is still in it's alpha stage, and probably will be changed often.
Modifications made for musl 1.1.4.
If anyone interested to audit it - welcome!
Thanks. If nothing else, this is very informative as to how Android
does things. I noticed a couple things that could be a lot less
invasive, I think, particularly the builtin users/groups. Couldn't you
just fmemopen() a constant string inside libc in place of /etc/passwd
or /etc/group and use the existing code to parse it?

One thing I noticed -- you seem to have some memory leaks in the code
that produces the passwd/group output -- it's malloc'ing space for
names each time and I don't see anywhere it gets freed. Maybe I'm
missing something though; I just read it briefly.

Another general question I have is why can't the /system issue be
fixed? Is there any good reason Android doesn't have symlinks like
/bin->/system/bin, /etc->/system/etc, /lib->/system/lib,
/tmp->/data/tmp, etc.? The gratuitous incompatibility is really
disgusting.

Rich
Рысь
2015-01-18 08:01:19 UTC
Permalink
В Sun, 18 Jan 2015 01:44:40 -0500
Post by Rich Felker
Post by Рысь
В Thu, 15 Jan 2015 16:13:22 +0700
Post by Рысь
Hello all!
Are there any efforts or even a project which aims to port musl to
android platform?
For a year I slowly had built and running a couple of C tty only
programs such as iptables, tcpdump and of course busybox and many
others including my own with musl libc linked statically. They
work perfectly (no much differences between an ARM phone and, for
+ Proper DNS resolving is not available
+ Translation of android special user names is not done
As an advanced Linux user I know that android is not friendly
enough to plain C stuff and it's libc is even not finished now so
I aimed to port at least listed things to musl.
Because I am not going beyond listed items, a patch can be
developed just to support these inside musl-linked binaries.
I am first here or there is already someone who done this before?
I made a patch to implement all three things for musl on android.
I do not post it here as attachment to this mail because it normally
http://lynxlynx.tk/prg/patches/musl-1.1.4_android.patch, notes about
it: http://lynxlynx.tk/eng/musl_android/.
Patch is invasive enough I think and not optimized and I agree with
Rich that it must go as a special treat. I tried however move away
all translating code to src/android and headers to include/android.
Patch is still in it's alpha stage, and probably will be changed often.
Modifications made for musl 1.1.4.
If anyone interested to audit it - welcome!
Thanks. If nothing else, this is very informative as to how Android
does things. I noticed a couple things that could be a lot less
invasive, I think, particularly the builtin users/groups. Couldn't you
just fmemopen() a constant string inside libc in place of /etc/passwd
or /etc/group and use the existing code to parse it?
This sounds more interesting idea, I forgot that get*ent() open fd and
there is fmemopen() function. But app_/ux_ax names still need a
dedicated translation function, probably the same code which will form
a passwd-like string and feed it into get*ent().
Post by Rich Felker
One thing I noticed -- you seem to have some memory leaks in the code
that produces the passwd/group output -- it's malloc'ing space for
names each time and I don't see anywhere it gets freed. Maybe I'm
missing something though; I just read it briefly.
Yes the patch is alpha and the reason why there is so much code is that
I did not know how get*ent() were implemented in musl from beginning,
and learned them only when translation code was ready. This patch is
just to show how a translation layer can be implemented on musl to
(completely) port it to android, and I am probably going to completely
rewrite uid/name translation code based on your suggestion in next days
(or at least try to). Memory leaks are expected now :(
Post by Rich Felker
Another general question I have is why can't the /system issue be
fixed? Is there any good reason Android doesn't have symlinks like
/bin->/system/bin, /etc->/system/etc, /lib->/system/lib,
/tmp->/data/tmp, etc.? The gratuitous incompatibility is really
disgusting.
First, android has /etc -> /system/etc in place, but, unfortunately, no
more :(

I made /etc -> /system/etc change just because I distrust symlinks when
doing security things. It is usually stable across devices, but it's
just symlink.

It even does not have usual Unix /tmp directory or it's surrogate
somewhere in filesystem. /data/tmp is my adoption, though it's
non-standard on android.

Second, android has a very different bootup process, and all those
links in / are created during bootup (on a tmpfs?). Sadly only /etc is
defined by android developers to be symlink to /system/etc.

Third, there are probably many differences between many android device
vendors, android OS types (different android versions including ancient
ones, different android forks that exist today, vendor customisations
of fs layout and in future many others). In fact, android is _very_
fragmented OS.

All this is based on my experience with some android devices I working
(and worked) with.
Post by Rich Felker
Rich
Your additional input on this will be welcome, thanks!
Rich Felker
2015-01-18 16:40:10 UTC
Permalink
Post by Рысь
В Sun, 18 Jan 2015 01:44:40 -0500
Post by Rich Felker
Post by Рысь
В Thu, 15 Jan 2015 16:13:22 +0700
Post by Рысь
Hello all!
Are there any efforts or even a project which aims to port musl to
android platform?
For a year I slowly had built and running a couple of C tty only
programs such as iptables, tcpdump and of course busybox and many
others including my own with musl libc linked statically. They
work perfectly (no much differences between an ARM phone and, for
+ Proper DNS resolving is not available
+ Translation of android special user names is not done
As an advanced Linux user I know that android is not friendly
enough to plain C stuff and it's libc is even not finished now so
I aimed to port at least listed things to musl.
Because I am not going beyond listed items, a patch can be
developed just to support these inside musl-linked binaries.
I am first here or there is already someone who done this before?
I made a patch to implement all three things for musl on android.
I do not post it here as attachment to this mail because it normally
http://lynxlynx.tk/prg/patches/musl-1.1.4_android.patch, notes about
it: http://lynxlynx.tk/eng/musl_android/.
Patch is invasive enough I think and not optimized and I agree with
Rich that it must go as a special treat. I tried however move away
all translating code to src/android and headers to include/android.
Patch is still in it's alpha stage, and probably will be changed often.
Modifications made for musl 1.1.4.
If anyone interested to audit it - welcome!
Thanks. If nothing else, this is very informative as to how Android
does things. I noticed a couple things that could be a lot less
invasive, I think, particularly the builtin users/groups. Couldn't you
just fmemopen() a constant string inside libc in place of /etc/passwd
or /etc/group and use the existing code to parse it?
This sounds more interesting idea, I forgot that get*ent() open fd and
there is fmemopen() function. But app_/ux_ax names still need a
dedicated translation function, probably the same code which will form
a passwd-like string and feed it into get*ent().
The intent is that get{gr,pw}{nam,uid} will eventually support
alternate backends via sending a request and receiving a response in
passwd/group line format. On normal systems this would take place via
a unix socket in some reasonable standard directory, but for Android
you could in principle build a reply mechanism into the binary in
place of it. Note that this would probably not support get{pw,gr}ent
enumeration, but I don't see that as a problem. Traditionally
enumeration does not work with backends like nis or ldap, as far as I
know, and it would be an unreasonable/abusive idiom on such large user
databases.
Post by Рысь
Post by Rich Felker
Another general question I have is why can't the /system issue be
fixed? Is there any good reason Android doesn't have symlinks like
/bin->/system/bin, /etc->/system/etc, /lib->/system/lib,
/tmp->/data/tmp, etc.? The gratuitous incompatibility is really
disgusting.
First, android has /etc -> /system/etc in place, but, unfortunately, no
more :(
I made /etc -> /system/etc change just because I distrust symlinks when
doing security things. It is usually stable across devices, but it's
just symlink.
It even does not have usual Unix /tmp directory or it's surrogate
somewhere in filesystem. /data/tmp is my adoption, though it's
non-standard on android.
Second, android has a very different bootup process, and all those
links in / are created during bootup (on a tmpfs?). Sadly only /etc is
defined by android developers to be symlink to /system/etc.
Third, there are probably many differences between many android device
vendors, android OS types (different android versions including ancient
ones, different android forks that exist today, vendor customisations
of fs layout and in future many others). In fact, android is _very_
fragmented OS.
All this is based on my experience with some android devices I working
(and worked) with.
If / is a tmpfs, I don't see why proper symlinks can't be created, or
why /tmp can't just be created as a directory on / or as another tmpfs
mount on /tmp. Obviously you can't do these kinds of things as a
normal application installing on Android, but if you're doing your own
system programming on Android or if you have root, this seems like the
obvious correct fix...

I'd really like to hear from the Android side what their motive is for
breaking this by not providing the proper symlinks.

Rich
Рысь
2015-01-19 18:00:33 UTC
Permalink
В Sun, 18 Jan 2015 11:40:10 -0500
Post by Rich Felker
Post by Рысь
В Sun, 18 Jan 2015 01:44:40 -0500
Post by Rich Felker
Post by Рысь
В Thu, 15 Jan 2015 16:13:22 +0700
Post by Рысь
Hello all!
Are there any efforts or even a project which aims to port
musl to android platform?
For a year I slowly had built and running a couple of C tty
only programs such as iptables, tcpdump and of course busybox
and many others including my own with musl libc linked
statically. They work perfectly (no much differences between
an ARM phone and, for example, raspberry pi SoC) but a few
+ Proper DNS resolving is not available
+ Translation of android special user names is not done
As an advanced Linux user I know that android is not friendly
enough to plain C stuff and it's libc is even not finished
now so I aimed to port at least listed things to musl.
Because I am not going beyond listed items, a patch can be
developed just to support these inside musl-linked binaries.
I am first here or there is already someone who done this before?
I made a patch to implement all three things for musl on
android.
I do not post it here as attachment to this mail because it
normally should not be integrated with musl. The patch itself
http://lynxlynx.tk/prg/patches/musl-1.1.4_android.patch, notes
about it: http://lynxlynx.tk/eng/musl_android/.
Patch is invasive enough I think and not optimized and I agree
with Rich that it must go as a special treat. I tried however
move away all translating code to src/android and headers to
include/android.
Patch is still in it's alpha stage, and probably will be changed often.
Modifications made for musl 1.1.4.
If anyone interested to audit it - welcome!
Thanks. If nothing else, this is very informative as to how
Android does things. I noticed a couple things that could be a
lot less invasive, I think, particularly the builtin
users/groups. Couldn't you just fmemopen() a constant string
inside libc in place of /etc/passwd or /etc/group and use the
existing code to parse it?
This sounds more interesting idea, I forgot that get*ent() open fd
and there is fmemopen() function. But app_/ux_ax names still need a
dedicated translation function, probably the same code which will
form a passwd-like string and feed it into get*ent().
The intent is that get{gr,pw}{nam,uid} will eventually support
alternate backends via sending a request and receiving a response in
passwd/group line format. On normal systems this would take place via
a unix socket in some reasonable standard directory, but for Android
you could in principle build a reply mechanism into the binary in
place of it. Note that this would probably not support get{pw,gr}ent
enumeration, but I don't see that as a problem. Traditionally
enumeration does not work with backends like nis or ldap, as far as I
know, and it would be an unreasonable/abusive idiom on such large user
databases.
Post by Рысь
Post by Rich Felker
Another general question I have is why can't the /system issue be
fixed? Is there any good reason Android doesn't have symlinks like
/bin->/system/bin, /etc->/system/etc, /lib->/system/lib,
/tmp->/data/tmp, etc.? The gratuitous incompatibility is really
disgusting.
First, android has /etc -> /system/etc in place, but,
unfortunately, no more :(
I made /etc -> /system/etc change just because I distrust symlinks
when doing security things. It is usually stable across devices,
but it's just symlink.
It even does not have usual Unix /tmp directory or it's surrogate
somewhere in filesystem. /data/tmp is my adoption, though it's
non-standard on android.
Second, android has a very different bootup process, and all those
links in / are created during bootup (on a tmpfs?). Sadly only /etc
is defined by android developers to be symlink to /system/etc.
Third, there are probably many differences between many android
device vendors, android OS types (different android versions
including ancient ones, different android forks that exist today,
vendor customisations of fs layout and in future many others). In
fact, android is _very_ fragmented OS.
All this is based on my experience with some android devices I
working (and worked) with.
If / is a tmpfs, I don't see why proper symlinks can't be created, or
why /tmp can't just be created as a directory on / or as another tmpfs
mount on /tmp. Obviously you can't do these kinds of things as a
normal application installing on Android, but if you're doing your own
system programming on Android or if you have root, this seems like the
obvious correct fix...
I'd really like to hear from the Android side what their motive is for
breaking this by not providing the proper symlinks.
Rich
I talked about android as by default an unprivileged user, i.e. most
android devices are shipped so that owner of device cannot modify
certain firmware parts and cannot get root privilege.

Android developers decided to hide script that creates such a layout
into initramfs which is hard to modify on most devices (certain vendor
customisations apply, most do that via undescribed proprietary boot
formats which are flashed directly onto boot partition). Of course an
ordinary user is waaay far away from a right to modify it without dirty
hacks, and / tmpfs has permissions not to create any inodes by
unprivileged users and in fact is mounted readonly.

The "correct fix" is another hardish thing: every such device must be
hacked, and there must be a point from where you can start your own
rc.local. initramfs is not a friend, and on most devices rooted with
supersu (a common proprietary su(1) variant for android with graphical
asking interface) there is /system/etc/install-recovery.sh from where
you can invoke (or even have it as symlink) your rc.local. It's not
portable though (I had seen device which cannot be rooted at all
because vendor locked bootloader completely, or devices which are not
reacted on this file presence in any form)

------------

I still see android issue as a separate one, so patch code in _any_ form
must not go into main musl. If it go, then you must detect somehow are
you running on android or on regular system. One possible way I'm
seeing now is that environment variable ANDROID_PROPERTY_WORKSPACE,
it's presence is that you're likely on android. In any other things,
android is very fragmented. And there is another completely unrelated
code for any normal systems, like this code querying mmaped file.

I think this code or even whole musl should be ported to android as a
separate project, or even musl should replace bionic in future android
(or at least in it's forks). Reason is that android changes are too
invasive.

This is output of "ls -la /" from CyanogenMod 10.1 (no much
differences from original android):

# ls -la /
drwxr-xr-x root root 2015-01-15 02:08 acct
drwxrwx--x system cache 2013-09-27 15:00 cache
-rwxr-x--- root root 268176 1970-01-01 07:00 charger
dr-x------ root root 2015-01-15 02:08 config
lrwxrwxrwx root root 2015-01-15 02:08 d -> /sys/kernel/debug
drwxrwx--x system system 2014-11-12 14:20 data
-rw-r--r-- root root 136 1970-01-01 07:00 default.prop
drwxr-xr-x root root 2015-01-15 02:08 dev
drwxrwxr-x radio system 2013-01-08 22:44 efs
lrwxrwxrwx root root 2015-01-15 02:08 etc -> /system/etc
lrwxrwxrwx root root 2015-01-15 02:08 extSdCard -> /storage/sdcard1
-rw-r----- root root 1178 1970-01-01 07:00 fstab.espresso10
-rwxr-x--- root root 113592 1970-01-01 07:00 init
-rwxr-x--- root root 1288 1970-01-01 07:00 init.cm.rc
-rwxr-x--- root root 13512 1970-01-01 07:00 init.espresso10.rc
-rwxr-x--- root root 3570 1970-01-01 07:00 init.espresso10.usb.rc
-rwxr-x--- root root 2770 1970-01-01 07:00 init.goldfish.rc
-rwxr-x--- root root 21827 1970-01-01 07:00 init.rc
-rwxr-x--- root root 301 1970-01-01 07:00 init.superuser.rc
-rwxr-x--- root root 1795 1970-01-01 07:00 init.trace.rc
-rwxr-x--- root root 3947 1970-01-01 07:00 init.usb.rc
drwxrwxr-x root system 2015-01-15 02:08 mnt
dr-xr-xr-x root root 1970-01-01 07:00 proc
drwxr-xr-x root root 1970-01-01 07:00 res
drwx------ root root 2013-09-24 09:37 root
drwxr-x--- root root 1970-01-01 07:00 sbin
lrwxrwxrwx root root 2015-01-15 02:08 sdcard -> /storage/emulated/legacy
dr-xr-x--- system sdcard_r 2015-01-15 02:08 storage
drwxr-xr-x root root 2015-01-15 02:08 sys
drwxr-xr-x root root 2014-02-20 09:47 system
-rw-r--r-- root root 1993 1970-01-01 07:00 ueventd.espresso10.rc
-rw-r--r-- root root 272 1970-01-01 07:00 ueventd.goldfish.rc
-rw-r--r-- root root 5897 1970-01-01 07:00 ueventd.rc
lrwxrwxrwx root root 2015-01-15 02:08 usbdisk0 -> /storage/usbdisk0
lrwxrwxrwx root root 2015-01-15 02:08 vendor -> /system/vendor

Note files with epoch time - they're copied into this space when system
boots.
Рысь
2015-01-21 10:34:03 UTC
Permalink
В Sun, 18 Jan 2015 11:40:10 -0500
Post by Rich Felker
Post by Рысь
В Sun, 18 Jan 2015 01:44:40 -0500
Post by Rich Felker
Post by Рысь
В Thu, 15 Jan 2015 16:13:22 +0700
Post by Рысь
Hello all!
Are there any efforts or even a project which aims to port
musl to android platform?
For a year I slowly had built and running a couple of C tty
only programs such as iptables, tcpdump and of course busybox
and many others including my own with musl libc linked
statically. They work perfectly (no much differences between
an ARM phone and, for example, raspberry pi SoC) but a few
+ Proper DNS resolving is not available
+ Translation of android special user names is not done
As an advanced Linux user I know that android is not friendly
enough to plain C stuff and it's libc is even not finished
now so I aimed to port at least listed things to musl.
Because I am not going beyond listed items, a patch can be
developed just to support these inside musl-linked binaries.
I am first here or there is already someone who done this before?
I made a patch to implement all three things for musl on
android.
I do not post it here as attachment to this mail because it
normally should not be integrated with musl. The patch itself
http://lynxlynx.tk/prg/patches/musl-1.1.4_android.patch, notes
about it: http://lynxlynx.tk/eng/musl_android/.
Patch is invasive enough I think and not optimized and I agree
with Rich that it must go as a special treat. I tried however
move away all translating code to src/android and headers to
include/android.
Patch is still in it's alpha stage, and probably will be changed often.
Modifications made for musl 1.1.4.
If anyone interested to audit it - welcome!
Thanks. If nothing else, this is very informative as to how
Android does things. I noticed a couple things that could be a
lot less invasive, I think, particularly the builtin
users/groups. Couldn't you just fmemopen() a constant string
inside libc in place of /etc/passwd or /etc/group and use the
existing code to parse it?
This sounds more interesting idea, I forgot that get*ent() open fd
and there is fmemopen() function. But app_/ux_ax names still need a
dedicated translation function, probably the same code which will
form a passwd-like string and feed it into get*ent().
The intent is that get{gr,pw}{nam,uid} will eventually support
alternate backends via sending a request and receiving a response in
passwd/group line format. On normal systems this would take place via
a unix socket in some reasonable standard directory, but for Android
you could in principle build a reply mechanism into the binary in
place of it. Note that this would probably not support get{pw,gr}ent
enumeration, but I don't see that as a problem. Traditionally
enumeration does not work with backends like nis or ldap, as far as I
know, and it would be an unreasonable/abusive idiom on such large user
databases.
Post by Рысь
Post by Rich Felker
Another general question I have is why can't the /system issue be
fixed? Is there any good reason Android doesn't have symlinks like
/bin->/system/bin, /etc->/system/etc, /lib->/system/lib,
/tmp->/data/tmp, etc.? The gratuitous incompatibility is really
disgusting.
First, android has /etc -> /system/etc in place, but,
unfortunately, no more :(
I made /etc -> /system/etc change just because I distrust symlinks
when doing security things. It is usually stable across devices,
but it's just symlink.
It even does not have usual Unix /tmp directory or it's surrogate
somewhere in filesystem. /data/tmp is my adoption, though it's
non-standard on android.
Second, android has a very different bootup process, and all those
links in / are created during bootup (on a tmpfs?). Sadly only /etc
is defined by android developers to be symlink to /system/etc.
Third, there are probably many differences between many android
device vendors, android OS types (different android versions
including ancient ones, different android forks that exist today,
vendor customisations of fs layout and in future many others). In
fact, android is _very_ fragmented OS.
All this is based on my experience with some android devices I
working (and worked) with.
If / is a tmpfs, I don't see why proper symlinks can't be created, or
why /tmp can't just be created as a directory on / or as another tmpfs
mount on /tmp. Obviously you can't do these kinds of things as a
normal application installing on Android, but if you're doing your own
system programming on Android or if you have root, this seems like the
obvious correct fix...
I'd really like to hear from the Android side what their motive is for
breaking this by not providing the proper symlinks.
Rich
Ok, I posted a version that is completely new in uid/gid translation,
which follows your suggestions.

It was really easy to implement it (although somewhat tricky with _r
functions at start) and no more malloc/free dances! It should be more
sane now.

Please see it (link is same):
http://lynxlynx.tk/prg/patches/musl-1.1.4_android.patch

As of previous version, I do not recommend it to be included in musl.
Rich Felker
2015-01-21 18:36:37 UTC
Permalink
Post by Рысь
В Sun, 18 Jan 2015 11:40:10 -0500
Post by Rich Felker
Post by Рысь
В Sun, 18 Jan 2015 01:44:40 -0500
Post by Rich Felker
Post by Рысь
В Thu, 15 Jan 2015 16:13:22 +0700
Post by Рысь
Hello all!
Are there any efforts or even a project which aims to port
musl to android platform?
For a year I slowly had built and running a couple of C tty
only programs such as iptables, tcpdump and of course busybox
and many others including my own with musl libc linked
statically. They work perfectly (no much differences between
an ARM phone and, for example, raspberry pi SoC) but a few
+ Proper DNS resolving is not available
+ Translation of android special user names is not done
As an advanced Linux user I know that android is not friendly
enough to plain C stuff and it's libc is even not finished
now so I aimed to port at least listed things to musl.
Because I am not going beyond listed items, a patch can be
developed just to support these inside musl-linked binaries.
I am first here or there is already someone who done this before?
I made a patch to implement all three things for musl on android.
I do not post it here as attachment to this mail because it
normally should not be integrated with musl. The patch itself
http://lynxlynx.tk/prg/patches/musl-1.1.4_android.patch, notes
about it: http://lynxlynx.tk/eng/musl_android/.
Patch is invasive enough I think and not optimized and I agree
with Rich that it must go as a special treat. I tried however
move away all translating code to src/android and headers to
include/android.
Patch is still in it's alpha stage, and probably will be changed often.
Modifications made for musl 1.1.4.
If anyone interested to audit it - welcome!
Thanks. If nothing else, this is very informative as to how
Android does things. I noticed a couple things that could be a
lot less invasive, I think, particularly the builtin
users/groups. Couldn't you just fmemopen() a constant string
inside libc in place of /etc/passwd or /etc/group and use the
existing code to parse it?
This sounds more interesting idea, I forgot that get*ent() open fd
and there is fmemopen() function. But app_/ux_ax names still need a
dedicated translation function, probably the same code which will
form a passwd-like string and feed it into get*ent().
The intent is that get{gr,pw}{nam,uid} will eventually support
alternate backends via sending a request and receiving a response in
passwd/group line format. On normal systems this would take place via
a unix socket in some reasonable standard directory, but for Android
you could in principle build a reply mechanism into the binary in
place of it. Note that this would probably not support get{pw,gr}ent
enumeration, but I don't see that as a problem. Traditionally
enumeration does not work with backends like nis or ldap, as far as I
know, and it would be an unreasonable/abusive idiom on such large user
databases.
Post by Рысь
Post by Rich Felker
Another general question I have is why can't the /system issue be
fixed? Is there any good reason Android doesn't have symlinks like
/bin->/system/bin, /etc->/system/etc, /lib->/system/lib,
/tmp->/data/tmp, etc.? The gratuitous incompatibility is really
disgusting.
First, android has /etc -> /system/etc in place, but,
unfortunately, no more :(
I made /etc -> /system/etc change just because I distrust symlinks
when doing security things. It is usually stable across devices,
but it's just symlink.
It even does not have usual Unix /tmp directory or it's surrogate
somewhere in filesystem. /data/tmp is my adoption, though it's
non-standard on android.
Second, android has a very different bootup process, and all those
links in / are created during bootup (on a tmpfs?). Sadly only /etc
is defined by android developers to be symlink to /system/etc.
Third, there are probably many differences between many android
device vendors, android OS types (different android versions
including ancient ones, different android forks that exist today,
vendor customisations of fs layout and in future many others). In
fact, android is _very_ fragmented OS.
All this is based on my experience with some android devices I
working (and worked) with.
If / is a tmpfs, I don't see why proper symlinks can't be created, or
why /tmp can't just be created as a directory on / or as another tmpfs
mount on /tmp. Obviously you can't do these kinds of things as a
normal application installing on Android, but if you're doing your own
system programming on Android or if you have root, this seems like the
obvious correct fix...
I'd really like to hear from the Android side what their motive is for
breaking this by not providing the proper symlinks.
Rich
Ok, I posted a version that is completely new in uid/gid translation,
which follows your suggestions.
It was really easy to implement it (although somewhat tricky with _r
functions at start) and no more malloc/free dances! It should be more
sane now.
http://lynxlynx.tk/prg/patches/musl-1.1.4_android.patch
As of previous version, I do not recommend it to be included in musl.
It would be very much appreciated to send patches as attachments
rather than links. There's no way to have a well-documented ongoing
discussion of the patch if it's not in a permanent place for review,
and attached to the email/archives is the best way to achieve that.

Rich
Рысь
2015-01-22 02:37:52 UTC
Permalink
В Wed, 21 Jan 2015 13:36:37 -0500
Post by Rich Felker
It would be very much appreciated to send patches as attachments
rather than links. There's no way to have a well-documented ongoing
discussion of the patch if it's not in a permanent place for review,
and attached to the email/archives is the best way to achieve that.
Rich
Sorry.

Now patch is attached (I think it is more sane now to include it here)
Рысь
2015-01-31 15:08:56 UTC
Permalink
В Thu, 22 Jan 2015 09:37:52 +0700
В Wed, 21 Jan 2015 13:36:37 -0500
Post by Rich Felker
It would be very much appreciated to send patches as attachments
rather than links. There's no way to have a well-documented ongoing
discussion of the patch if it's not in a permanent place for review,
and attached to the email/archives is the best way to achieve that.
Rich
Sorry.
Now patch is attached (I think it is more sane now to include it here)
No comments about code correctness and security?

That patch I sent to list contains a typo in android_isappusr(): return
(p != 0) instead of (p == 0).

The code works fine on my phone for now, I moved all bionic depend
binaries to patched musl without pain. Thanks for your useful hints.
Rich Felker
2015-02-03 05:52:51 UTC
Permalink
Post by Рысь
В Thu, 22 Jan 2015 09:37:52 +0700
В Wed, 21 Jan 2015 13:36:37 -0500
Post by Rich Felker
It would be very much appreciated to send patches as attachments
rather than links. There's no way to have a well-documented ongoing
discussion of the patch if it's not in a permanent place for review,
and attached to the email/archives is the best way to achieve that.
Rich
Sorry.
Now patch is attached (I think it is more sane now to include it here)
No comments about code correctness and security?
I haven't yet had a chance to look in detail. I just read over it
briefly again and one thing that popped out is that some aspects are
not namespace clean. In particular there are standard functions
depending on symbols with names like android_* rather than something
in the reserved namespace like __android_*.

One other thing that struck me was all the __istonum functions. The
need for separate functions for all those things seems to be an
artificial creation due to a bad calling convention -- using a
pointer-to-result as the argument. If you instead returned the value
and used a pointer-to-error-flag as the argument, one function would
suffice for all integer types.

In general I suspect there's still more that could be streamlined or
where the size of the diff versus upstream could be reduced.

I didn't see any other glaring security or correctness issues but I'll
let you know if I notice another when I get a chance to read it in
more detail.
Post by Рысь
That patch I sent to list contains a typo in android_isappusr(): return
(p != 0) instead of (p == 0).
The code works fine on my phone for now, I moved all bionic depend
binaries to patched musl without pain. Thanks for your useful hints.
Nice!

Rich

Continue reading on narkive:
Loading...