Discussion:
[musl] Problems with pthreads from a shared object?
Barry Flartus
2018-11-07 23:00:14 UTC
Permalink
Hello,

I am encountering some issues that I believe are musl related when trying to
use pthreads from a shared object. Below is a minimal example (2 files) of
the problem.

==================== test.c ====================

// Compile with
// musl-gcc -Wall -lpthread -shared -fPIC test.c -o test.so

#include <pthread.h>
#include <stdio.h>

static void *launch(void *arg);
int start_thread(void);
void entry_point(void) __attribute__((constructor));

static pthread_t gthread;

static void *launch(void *arg)
{
for (int i = 0; i < 4; i++){
printf("[%d] Hello from the thread\n", i);
}
return NULL;
}

int start_thread(void)
{
printf("Starting the thread\n");
return pthread_create(&gthread, NULL, launch, NULL);
}

void entry_point(void){
puts("Starting Execution");
int res = start_thread();
printf("start_thread returned %d\n", res);
if (res == 0){
pthread_join(gthread, NULL);
} else {
printf("pthread_create() returned an error. Aborting.\n");
}
}

==================== launch.c ====================

// Compile with:
// gcc launch.c -ldl -o launch

#include <dlfcn.h>
#include <stdio.h>
#include <stdlib.h>

int main(){
void *handle;

puts("\nLaunching test...");
handle = dlopen("./test.so", RTLD_NOW);
if (!handle) {
fprintf(stderr, "%s\n", dlerror());
exit(EXIT_FAILURE);
}
dlerror();

return EXIT_SUCCESS;
}

When the above examples are compiled with gcc, everything works correctly
and
produces the following output:

Launching test...
Starting Execution
Starting the threadstart_thread returned 0
[0] Hello from the thread
[1] Hello from the thread
[2] Hello from the thread
[3] Hello from the thread

When I switch to musl pthread_create() returns the integer '38', which is an
undocumented return value. I assume this corresponds with ENOSYS from
errno.h
but I can't seem to figure out why this is happening.

Output:

Launching test...
Starting Execution
Starting the thread
start_thread returned 38
pthread_create() returned an error. Aborting.

Is this even supported by musl? If so, where have I gone wrong?

Thanks in advance!
Rich Felker
2018-11-07 23:31:42 UTC
Permalink
Post by Barry Flartus
Hello,
I am encountering some issues that I believe are musl related when trying to
use pthreads from a shared object. Below is a minimal example (2 files) of
the problem.
==================== test.c ====================
// Compile with
// musl-gcc -Wall -lpthread -shared -fPIC test.c -o test.so
#include <pthread.h>
#include <stdio.h>
static void *launch(void *arg);
int start_thread(void);
void entry_point(void) __attribute__((constructor));
static pthread_t gthread;
static void *launch(void *arg)
{
for (int i = 0; i < 4; i++){
printf("[%d] Hello from the thread\n", i);
}
return NULL;
}
int start_thread(void)
{
printf("Starting the thread\n");
return pthread_create(&gthread, NULL, launch, NULL);
}
void entry_point(void){
puts("Starting Execution");
int res = start_thread();
printf("start_thread returned %d\n", res);
if (res == 0){
pthread_join(gthread, NULL);
} else {
printf("pthread_create() returned an error. Aborting.\n");
}
}
==================== launch.c ====================
// gcc launch.c -ldl -o launch
#include <dlfcn.h>
#include <stdio.h>
#include <stdlib.h>
int main(){
void *handle;
puts("\nLaunching test...");
handle = dlopen("./test.so", RTLD_NOW);
if (!handle) {
fprintf(stderr, "%s\n", dlerror());
exit(EXIT_FAILURE);
}
dlerror();
return EXIT_SUCCESS;
}
When the above examples are compiled with gcc, everything works correctly
and
Launching test...
Starting Execution
Starting the threadstart_thread returned 0
[0] Hello from the thread
[1] Hello from the thread
[2] Hello from the thread
[3] Hello from the thread
When I switch to musl pthread_create() returns the integer '38', which is an
undocumented return value. I assume this corresponds with ENOSYS from
errno.h
but I can't seem to figure out why this is happening.
Launching test...
Starting Execution
Starting the thread
start_thread returned 38
pthread_create() returned an error. Aborting.
Is this even supported by musl? If so, where have I gone wrong?
Thanks in advance!
Are you running on an ancient kernel or a kernel built with a lot of
default functionality configured out? These should be the only
conditions under which pthread_create can return ENOSYS.

Rich
Barry Flartus
2018-11-07 23:40:35 UTC
Permalink
Nope! Linux ubuntu-x64 4.4.0-116-generic - this is just the kernel from a
default Ubuntu 16.04 install.
Post by Barry Flartus
Post by Barry Flartus
Hello,
I am encountering some issues that I believe are musl related when
trying to
Post by Barry Flartus
use pthreads from a shared object. Below is a minimal example (2 files)
of
Post by Barry Flartus
the problem.
==================== test.c ====================
// Compile with
// musl-gcc -Wall -lpthread -shared -fPIC test.c -o test.so
#include <pthread.h>
#include <stdio.h>
static void *launch(void *arg);
int start_thread(void);
void entry_point(void) __attribute__((constructor));
static pthread_t gthread;
static void *launch(void *arg)
{
for (int i = 0; i < 4; i++){
printf("[%d] Hello from the thread\n", i);
}
return NULL;
}
int start_thread(void)
{
printf("Starting the thread\n");
return pthread_create(&gthread, NULL, launch, NULL);
}
void entry_point(void){
puts("Starting Execution");
int res = start_thread();
printf("start_thread returned %d\n", res);
if (res == 0){
pthread_join(gthread, NULL);
} else {
printf("pthread_create() returned an error. Aborting.\n");
}
}
==================== launch.c ====================
// gcc launch.c -ldl -o launch
#include <dlfcn.h>
#include <stdio.h>
#include <stdlib.h>
int main(){
void *handle;
puts("\nLaunching test...");
handle = dlopen("./test.so", RTLD_NOW);
if (!handle) {
fprintf(stderr, "%s\n", dlerror());
exit(EXIT_FAILURE);
}
dlerror();
return EXIT_SUCCESS;
}
When the above examples are compiled with gcc, everything works correctly
and
Launching test...
Starting Execution
Starting the threadstart_thread returned 0
[0] Hello from the thread
[1] Hello from the thread
[2] Hello from the thread
[3] Hello from the thread
When I switch to musl pthread_create() returns the integer '38', which
is an
Post by Barry Flartus
undocumented return value. I assume this corresponds with ENOSYS from
errno.h
but I can't seem to figure out why this is happening.
Launching test...
Starting Execution
Starting the thread
start_thread returned 38
pthread_create() returned an error. Aborting.
Is this even supported by musl? If so, where have I gone wrong?
Thanks in advance!
Are you running on an ancient kernel or a kernel built with a lot of
default functionality configured out? These should be the only
conditions under which pthread_create can return ENOSYS.
Rich
Rich Felker
2018-11-07 23:45:26 UTC
Permalink
Post by Barry Flartus
Nope! Linux ubuntu-x64 4.4.0-116-generic - this is just the kernel from a
default Ubuntu 16.04 install.
OK. Can you run strace -f ./launch (or whatever the launch program
executable is called) and post the output?

Rich
Post by Barry Flartus
Post by Barry Flartus
Post by Barry Flartus
Hello,
I am encountering some issues that I believe are musl related when
trying to
Post by Barry Flartus
use pthreads from a shared object. Below is a minimal example (2 files)
of
Post by Barry Flartus
the problem.
==================== test.c ====================
// Compile with
// musl-gcc -Wall -lpthread -shared -fPIC test.c -o test.so
#include <pthread.h>
#include <stdio.h>
static void *launch(void *arg);
int start_thread(void);
void entry_point(void) __attribute__((constructor));
static pthread_t gthread;
static void *launch(void *arg)
{
for (int i = 0; i < 4; i++){
printf("[%d] Hello from the thread\n", i);
}
return NULL;
}
int start_thread(void)
{
printf("Starting the thread\n");
return pthread_create(&gthread, NULL, launch, NULL);
}
void entry_point(void){
puts("Starting Execution");
int res = start_thread();
printf("start_thread returned %d\n", res);
if (res == 0){
pthread_join(gthread, NULL);
} else {
printf("pthread_create() returned an error. Aborting.\n");
}
}
==================== launch.c ====================
// gcc launch.c -ldl -o launch
#include <dlfcn.h>
#include <stdio.h>
#include <stdlib.h>
int main(){
void *handle;
puts("\nLaunching test...");
handle = dlopen("./test.so", RTLD_NOW);
if (!handle) {
fprintf(stderr, "%s\n", dlerror());
exit(EXIT_FAILURE);
}
dlerror();
return EXIT_SUCCESS;
}
When the above examples are compiled with gcc, everything works correctly
and
Launching test...
Starting Execution
Starting the threadstart_thread returned 0
[0] Hello from the thread
[1] Hello from the thread
[2] Hello from the thread
[3] Hello from the thread
When I switch to musl pthread_create() returns the integer '38', which
is an
Post by Barry Flartus
undocumented return value. I assume this corresponds with ENOSYS from
errno.h
but I can't seem to figure out why this is happening.
Launching test...
Starting Execution
Starting the thread
start_thread returned 38
pthread_create() returned an error. Aborting.
Is this even supported by musl? If so, where have I gone wrong?
Thanks in advance!
Are you running on an ancient kernel or a kernel built with a lot of
default functionality configured out? These should be the only
conditions under which pthread_create can return ENOSYS.
Rich
Barry Flartus
2018-11-07 23:53:33 UTC
Permalink
Post by Rich Felker
OK. Can you run strace -f ./launch (or whatever the launch program
executable is called) and post the output?
Rich
Here you go!

$ strace -f ./launch
execve("./launch", ["./launch"], [/* 22 vars */]) = 0
brk(NULL) = 0xe44000
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or
directory)
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or
directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=42286, ...}) = 0
mmap(NULL, 42286, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fcda3aa9000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or
directory)
open("/lib/x86_64-linux-gnu/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3,
"\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\240\r\0\0\0\0\0\0"..., 832)
= 832
fstat(3, {st_mode=S_IFREG|0644, st_size=14608, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7fcda3aa8000
mmap(NULL, 2109680, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) =
0x7fcda368b000
mprotect(0x7fcda368e000, 2093056, PROT_NONE) = 0
mmap(0x7fcda388d000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7fcda388d000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or
directory)
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\t\2\0\0\0\0\0"...,
832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1868984, ...}) = 0
mmap(NULL, 3971488, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) =
0x7fcda32c1000
mprotect(0x7fcda3481000, 2097152, PROT_NONE) = 0
mmap(0x7fcda3681000, 24576, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1c0000) = 0x7fcda3681000
mmap(0x7fcda3687000, 14752, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fcda3687000
close(3) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7fcda3aa7000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7fcda3aa6000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7fcda3aa5000
arch_prctl(ARCH_SET_FS, 0x7fcda3aa6700) = 0
mprotect(0x7fcda3681000, 16384, PROT_READ) = 0
mprotect(0x7fcda388d000, 4096, PROT_READ) = 0
mprotect(0x600000, 4096, PROT_READ) = 0
mprotect(0x7fcda3ab4000, 4096, PROT_READ) = 0
munmap(0x7fcda3aa9000, 42286) = 0
fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 2), ...}) = 0
brk(NULL) = 0xe44000
brk(0xe65000) = 0xe65000
write(1, "\n", 1
) = 1
write(1, "Launching test...\n", 18Launching test...
) = 18
open("./test.so", O_RDONLY|O_CLOEXEC) = 3
read(3,
"\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\240!\0\0\0\0\0\0"..., 832)
= 832
fstat(3, {st_mode=S_IFREG|0755, st_size=40152, ...}) = 0
getcwd("/vagrant/test/wtf/public", 128) = 25
mmap(NULL, 2128504, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) =
0x7fcda30b9000
mprotect(0x7fcda30c0000, 2093056, PROT_NONE) = 0
mmap(0x7fcda32bf000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6000) = 0x7fcda32bf000
close(3) = 0
mprotect(0x7fcda32bf000, 4096, PROT_READ) = 0
write(1, "Starting Execution\n", 19Starting Execution
) = 19
write(1, "Starting the thread\n", 20Starting the thread
) = 20
write(1, "start_thread returned 38\n", 25start_thread returned 38
) = 25
write(1, "pthread_create() returned an err"..., 46pthread_create() returned
an error. Aborting.
) = 46
exit_group(0) = ?
+++ exited with 0 +++

$ ltrace -f ./launch
[pid 13508] __libc_start_main(0x400746, 1, 0x7ffff030bbc8, 0x4007b0
<unfinished ...>
[pid 13508] puts("\nLaunching test..."
Launching test...
)
= 19
[pid 13508] dlopen("./test.so", 2Starting Execution
Starting the thread
start_thread returned 38
pthread_create() returned an error. Aborting.
)
= 0x12d6440
[pid 13508] dlerror()

= nil
[pid 13508] +++ exited (status 0) +++
Rich Felker
2018-11-08 00:43:12 UTC
Permalink
Post by Barry Flartus
Post by Rich Felker
OK. Can you run strace -f ./launch (or whatever the launch program
executable is called) and post the output?
Rich
Here you go!
$ strace -f ./launch
execve("./launch", ["./launch"], [/* 22 vars */]) = 0
brk(NULL) = 0xe44000
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or
OK, this is a glibc program, not a musl one. That's your problem. How
did you compile/link it?

Rich
Barry Flartus
2018-11-08 01:02:01 UTC
Permalink
Post by Rich Felker
OK. Can you run strace -f ./launch (or whatever the launch program
executable is called) and post the output?
Rich
The "launching" program was compiled with gcc:
gcc launch.c -ldl -o launch

When I attempt to compile the "launching" program with musl-gcc I get the
following output:
***@ubuntu-x64:~$ /usr/local/musl/x86_64/bin/musl-gcc launch.c -ldl -o
launch
***@ubuntu-x64:~$ ./launch

Launching test...
Dynamic loading not supported

The shared object was compiled with musl-gcc:
/usr/local/musl/x86_64/bin/musl-gcc -Wall -lpthread -shared -fPIC test.c -o
test.so
Rich Felker
2018-11-08 01:28:39 UTC
Permalink
Post by Barry Flartus
Post by Rich Felker
OK. Can you run strace -f ./launch (or whatever the launch program
executable is called) and post the output?
Rich
gcc launch.c -ldl -o launch
You can't use shared libraries built for musl with glibc, so it's
expected that this doesn't work.
Post by Barry Flartus
When I attempt to compile the "launching" program with musl-gcc I get the
launch
Launching test...
Dynamic loading not supported
Apparently it's static-linked. Did you omit shared libraries when
building musl?
Post by Barry Flartus
/usr/local/musl/x86_64/bin/musl-gcc -Wall -lpthread -shared -fPIC test.c -o
test.so
If so (if there's no libc.so, only libc.a, present) then this command
will also be producing a broken .so file. You can't make shared
libraries without a shared libc existing.

Rich
Jeffrey Walton
2018-11-08 08:50:00 UTC
Permalink
Post by Rich Felker
Post by Barry Flartus
Post by Rich Felker
OK. Can you run strace -f ./launch (or whatever the launch program
executable is called) and post the output?
...
$ strace -f ./launch
execve("./launch", ["./launch"], [/* 22 vars */]) = 0
brk(NULL) = 0xe44000
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or
OK, this is a glibc program, not a musl one. That's your problem. How
did you compile/link it?
Sorry to butt-in.

Alpine Linux uses musl. BF won't need anything special to use or
exercise musl (and certainly won't need to worry about
cross-pollination).

Jeff
Barry Flartus
2018-11-08 14:53:38 UTC
Permalink
Post by Jeffrey Walton
Sorry to butt-in.
Alpine Linux uses musl. BF won't need anything special to use or
exercise musl (and certainly won't need to worry about
cross-pollination).
Jeff
Jeff, no worries about butting in. Could you clarify your statement?
Im not quite sure I understand your cross-pollination analogy :)
Rich Felker
2018-11-08 15:17:15 UTC
Permalink
Post by Barry Flartus
Post by Jeffrey Walton
Sorry to butt-in.
Alpine Linux uses musl. BF won't need anything special to use or
exercise musl (and certainly won't need to worry about
cross-pollination).
Jeff, no worries about butting in. Could you clarify your statement?
Im not quite sure I understand your cross-pollination analogy :)
I think it would be helpful to understand what you're trying to do.

If your goal is just ensuring your application is compatible with musl
so that it can be built and used on musl-based systems/distros, using
one (such as Alpine, or Adelie, or Void with musl, or Sabotage, etc.),
possibly in a container, is going to be the easiest way to achieve
that.

If your goal is building an application with musl and dlopen'd modules
that you can deploy to systems that might or might not be musl-based,
this can be done, but requires using dynamic linking and some glue for
explicitly running the dynamic linker, since it can't be found via a
fixed absolute path the way 'native' dynamic linking works.

If your goal is to get around the glibc limitation that loading
libpthread dynamically doesn't work, trying to use musl to produce .so
files that use threads that glibc-linked programs can load, this
simply does not, and cannot, work. A much better simpler solution is
just ensuring the main program always links with -lpthread when using
glibc.

Rich
Barry Flartus
2018-11-08 17:14:39 UTC
Permalink
Post by Rich Felker
I think it would be helpful to understand what you're trying to do.
Thanks for your help so far. Let me try and explain what I'm trying to
accomplish. I have a program that runs as an executable and uses pthreads.
I've compiled this program with musl statically with the end goal of it
being portable across older and newer systems. I want to also be able to
compile this program as a shared object so it may be loaded via dlopen()
inside of a glibc program. As mentioned previously, if I compile my shared
object with glibc, it loads via dlopen().

My lack of understanding is this: if I directly compile in musl's libc.a
(which contains its implementation of pthreads) into my shared object,
shouldn't it have the relevant pthreads functions compiled in, without
having runtime issues? That's what seems to work for my musl-compiled
static executable, so I'm trying to wrap my head around why it wouldn't
work for a shared object. In my case, I'm considering the shared object
just a different form-factor for the same program.
Rich Felker
2018-11-08 17:37:29 UTC
Permalink
Post by Barry Flartus
Post by Rich Felker
I think it would be helpful to understand what you're trying to do.
Thanks for your help so far. Let me try and explain what I'm trying to
accomplish. I have a program that runs as an executable and uses pthreads.
I've compiled this program with musl statically with the end goal of it
being portable across older and newer systems. I want to also be able to
compile this program as a shared object so it may be loaded via dlopen()
inside of a glibc program. As mentioned previously, if I compile my shared
object with glibc, it loads via dlopen().
My lack of understanding is this: if I directly compile in musl's libc.a
(which contains its implementation of pthreads) into my shared object,
shouldn't it have the relevant pthreads functions compiled in, without
having runtime issues? That's what seems to work for my musl-compiled
It has the right code for being able to run in a process where the
(part of) libc inside your .so is the only libc in the process. It
does not have the right code for interoperating with a different
arbitrary libc in the same process where both of them rightly believe
themselves the sole owner of various process-global or thread-global
state.
Post by Barry Flartus
static executable, so I'm trying to wrap my head around why it wouldn't
work for a shared object. In my case, I'm considering the shared object
just a different form-factor for the same program.
Conceptually, the reasons it won't work are unbounded; any particular
list could become incomplete with newer versions of musl and/or glibc.

In practice, issues you would definitely face include but are not
limited to:

- Anything that depends on initialization at startup time will fail,
as the musl __libc_start_main was never called.

- Aside from a few fields which are compiler ABI, the content/layout
of the thread structure is an implementation detail subject to
change even between versions, and will not match between musl and
glibc. This matters even if the glibc code is not starting threads
since the main thread will have a glibc-format thread structure
pointed to by its thread pointer (%fs:0 on x86_64) and any glibc
functions called from your library will expect glibc-format thread
structures (even if the thread was started by the musl
pthread_create).

- Assuming you didn't link with -Bsymbolic[-functions], calls to libc
functions that were linked in your library may (will) get interposed
by glibc functions already present, and the musl functions won't
actually get called, except for the ones that don't exist in glibc
(because they're in its librt or libpthread or something). And when
some functions from musl which do end up getting called execute,
they'll call a mix of glibc functions (when they reference something
bu its public symbol) and musl functions (when they call
musl-internal interfaces). These will be expecting/operaring on
different and incompatible data structures.

- If you did link with -Bsymbolic[-functions], you'd end up calling
musl's malloc in the same process as glibc's malloc and they'd be
fighting over ownership of the brk. Things would also blow up badly
if a pointer returned by one's malloc was passed to the other's
realloc/free at some point.

- Many more things...

Ultimately, it's *always* unsafe to have two libraries with
conflicting interfaces linked into the same program, and even moreso
when they're something as central (with ownership of central global
state) as libc.

Rich
Barry Flartus
2018-11-08 17:56:48 UTC
Permalink
Post by Barry Flartus
Post by Barry Flartus
Post by Rich Felker
I think it would be helpful to understand what you're trying to do.
Thanks for your help so far. Let me try and explain what I'm trying to
accomplish. I have a program that runs as an executable and uses
pthreads.
Post by Barry Flartus
I've compiled this program with musl statically with the end goal of it
being portable across older and newer systems. I want to also be able to
compile this program as a shared object so it may be loaded via dlopen()
inside of a glibc program. As mentioned previously, if I compile my
shared
Post by Barry Flartus
object with glibc, it loads via dlopen().
My lack of understanding is this: if I directly compile in musl's libc.a
(which contains its implementation of pthreads) into my shared object,
shouldn't it have the relevant pthreads functions compiled in, without
having runtime issues? That's what seems to work for my musl-compiled
It has the right code for being able to run in a process where the
(part of) libc inside your .so is the only libc in the process. It
does not have the right code for interoperating with a different
arbitrary libc in the same process where both of them rightly believe
themselves the sole owner of various process-global or thread-global
state.
Post by Barry Flartus
static executable, so I'm trying to wrap my head around why it wouldn't
work for a shared object. In my case, I'm considering the shared object
just a different form-factor for the same program.
Conceptually, the reasons it won't work are unbounded; any particular
list could become incomplete with newer versions of musl and/or glibc.
In practice, issues you would definitely face include but are not
- Anything that depends on initialization at startup time will fail,
as the musl __libc_start_main was never called.
- Aside from a few fields which are compiler ABI, the content/layout
of the thread structure is an implementation detail subject to
change even between versions, and will not match between musl and
glibc. This matters even if the glibc code is not starting threads
since the main thread will have a glibc-format thread structure
pointed to by its thread pointer (%fs:0 on x86_64) and any glibc
functions called from your library will expect glibc-format thread
structures (even if the thread was started by the musl
pthread_create).
- Assuming you didn't link with -Bsymbolic[-functions], calls to libc
functions that were linked in your library may (will) get interposed
by glibc functions already present, and the musl functions won't
actually get called, except for the ones that don't exist in glibc
(because they're in its librt or libpthread or something). And when
some functions from musl which do end up getting called execute,
they'll call a mix of glibc functions (when they reference something
bu its public symbol) and musl functions (when they call
musl-internal interfaces). These will be expecting/operaring on
different and incompatible data structures.
- If you did link with -Bsymbolic[-functions], you'd end up calling
musl's malloc in the same process as glibc's malloc and they'd be
fighting over ownership of the brk. Things would also blow up badly
if a pointer returned by one's malloc was passed to the other's
realloc/free at some point.
- Many more things...
Ultimately, it's *always* unsafe to have two libraries with
conflicting interfaces linked into the same program, and even moreso
when they're something as central (with ownership of central global
state) as libc.
Rich
Rich, thank you so much for taking the time to explain this to me!
Ultimately
my problems stem from my lack knowledge in this area, so you really have
taught me so much and I am very appreciative of that.

Additionally, thank you for your work on musl, it really is fantastic! I
will
continue to use it in the correct manner.

Best,
Barry

Continue reading on narkive:
Search results for '[musl] Problems with pthreads from a shared object?' (Questions and Answers)
10
replies
What is AIX Box?
started 2006-05-08 15:58:44 UTC
hardware
Loading...