undefined symbols prevent linking on OSX / homebrew setup

etuttle commented

2020-01-22 02:34:21 +01:00

(Migrated from github.com)

As mentioned here I got this working by adding LDFLAGS="-Wl,-undefined -Wl,dynamic_lookup" to my env. Here's a more complete report of the problem.

I've attached a terminal session with only a few minor edits for clarity. Also attached two of CMake's logs.

cmake-shell-log.txt
CMakeError.log
CMakeOutput.log

As mentioned [here](https://github.com/babelouest/glewlwyd/pull/94#issuecomment-576575074) I got this working by adding `LDFLAGS="-Wl,-undefined -Wl,dynamic_lookup"` to my env. Here's a more complete report of the problem. I've attached a terminal session with only a few minor edits for clarity. Also attached two of CMake's logs. [cmake-shell-log.txt](https://github.com/babelouest/glewlwyd/files/4094828/cmake-shell-log.txt) [CMakeError.log](https://github.com/babelouest/glewlwyd/files/4094829/CMakeError.log) [CMakeOutput.log](https://github.com/babelouest/glewlwyd/files/4094830/CMakeOutput.log)

babelouest commented

2020-01-22 05:12:37 +01:00

(Migrated from github.com)

I'll look around to check if the LDFLAGS is the only way for you to build or if there are improvements to make to the cmake script.

Thanks for the bug report!

I'll look around to check if the `LDFLAGS` is the only way for you to build or if there are improvements to make to the cmake script. Thanks for the bug report!

babelouest commented

2020-01-22 19:49:57 +01:00

(Migrated from github.com)

Hello @etuttle ,

Can you try the last commit github.com/babelouest/glewlwyd@92009b4190 and see if you still have the same error? I'm not sure this will fix everything but at least the cmake script is more clean to continue to work on this problem.

Hello @etuttle , Can you try the last commit https://github.com/babelouest/glewlwyd/commit/92009b4190bd60e38d5226698e1fac8d07946bd6 and see if you still have the same error? I'm not sure this will fix everything but at least the cmake script is more clean to continue to work on this problem.

etuttle commented

2020-01-23 06:07:45 +01:00

(Migrated from github.com)

Here's what I get with make VERBOSE=1.
92009b4.txt

Here's what I get with `make VERBOSE=1`. [92009b4.txt](https://github.com/babelouest/glewlwyd/files/4101329/92009b4.txt)

babelouest commented

2020-01-23 15:41:30 +01:00

(Migrated from github.com)

I made another change in the cmake script to order the projects build: github.com/babelouest/glewlwyd@bdb1dc4735

Can you retry it?

I made another change in the cmake script to order the projects build: https://github.com/babelouest/glewlwyd/commit/bdb1dc47351647180c557e4702ed8443c14cb764 Can you retry it?

etuttle commented

2020-01-24 05:56:33 +01:00

(Migrated from github.com)

I tried bdb1dc4. It still fails with undefined symbol errors :-/.

I've attached two logs this time. So far in this issue, I've had -DCMAKE_POLICY_DEFAULT_CMP0079=NEW in my $cmake_opts. The first log (bdb1dc4.txt) has -DCMAKE_POLICY_DEFAULT_CMP0079=NEW, the second (bdb1dc4-2.txt) does not. The result is pretty similar, but cmake chooses a different build order. (It fails to link something different)

bdb1dc4.txt
bdb1dc4-2.txt

I looked it up to remind myself why I had added that cmake setting, and it has to do with the "linking targets from other directories" error you might remember from #94. Wish I had remembered this detail earlier. CMP0079 was introduced in cmake 3.13. I have 3.15.5 on my Mac, and Travis currently has 3.12.4. Aha! I installed cmake 3.15.5 in my travis simulator container and tried to build #94 on linux again. It failed initially with:

CMake Error at CMakeLists.txt:189 (target_link_libraries):
  Attempt to add link library "orcania" to target "ulfius" which is not built
  in this directory.

  This is allowed only when policy CMP0079 is set to NEW.

Then adding the option makes the build succeed. So that's one solution that we know works, but it requires a cmake upgrade.

I tried bdb1dc4. It still fails with undefined symbol errors :-/. I've attached two logs this time. So far in this issue, I've had `-DCMAKE_POLICY_DEFAULT_CMP0079=NEW` in my $cmake_opts. The first log (bdb1dc4.txt) has `-DCMAKE_POLICY_DEFAULT_CMP0079=NEW`, the second (bdb1dc4-2.txt) does not. The result is pretty similar, but cmake chooses a different build order. (It fails to link something different) [bdb1dc4.txt](https://github.com/babelouest/glewlwyd/files/4106801/bdb1dc4.txt) [bdb1dc4-2.txt](https://github.com/babelouest/glewlwyd/files/4106804/bdb1dc4-2.txt) I looked it up to remind myself why I had added that cmake setting, and it has to do with the "linking targets from other directories" error you might remember from #94. Wish I had remembered this detail earlier. `CMP0079` was introduced in cmake 3.13. I have 3.15.5 on my Mac, and Travis currently has 3.12.4. Aha! I installed cmake 3.15.5 in my travis simulator container and tried to build #94 on linux again. It failed initially with: ``` CMake Error at CMakeLists.txt:189 (target_link_libraries): Attempt to add link library "orcania" to target "ulfius" which is not built in this directory. This is allowed only when policy CMP0079 is set to NEW. ``` Then adding the option makes the build succeed. So that's one solution that we know works, but it requires a cmake upgrade.

babelouest commented

2020-01-28 17:18:52 +01:00

(Migrated from github.com)

What is the full command line you're using to build with cmake?

babelouest commented

2020-02-09 20:17:17 +01:00

(Migrated from github.com)

@etuttle , any update?

etuttle commented

2020-02-23 07:24:22 +01:00

(Migrated from github.com)

Strange, somehow I missed the notifications for the last couple updates to this issue...

The full command line I used to call cmake can be seen in the first few lines of the .txt files I posted (https://github.com/babelouest/glewlwyd/issues/97#issuecomment-577991174). I tend to use a $cmake_opts shell variable, but you can see where I define that variable on the previous line in the .txt file. Did you need more than that?

It seems likely that to solve this problem in clean way may require a cmake upgrade, but we also have the LDFLAGS work-around I mentioned in the issue report above.

Strange, somehow I missed the notifications for the last couple updates to this issue... The full command line I used to call cmake can be seen in the first few lines of the .txt files I posted (https://github.com/babelouest/glewlwyd/issues/97#issuecomment-577991174). I tend to use a $cmake_opts shell variable, but you can see where I define that variable on the previous line in the .txt file. Did you need more than that? It seems likely that to solve this problem in clean way may require a cmake upgrade, but we also have the LDFLAGS work-around I mentioned in the issue report above.

babelouest commented

2020-02-23 15:38:43 +01:00

(Migrated from github.com)

So if I update the documentation for OSX users to use CMake 3.15.5 and add the flag LDFLAGS="-Wl,-undefined -Wl,dynamic_lookup", this will do the trick?

So if I update the documentation for OSX users to use CMake 3.15.5 and add the flag `LDFLAGS="-Wl,-undefined -Wl,dynamic_lookup"`, this will do the trick?

etuttle commented

2020-02-23 23:33:46 +01:00

(Migrated from github.com)

There are two paths I'm aware of:

Upgrade travis and all users to cmake 3.13+. Also require all builds use -DCMAKE_POLICY_DEFAULT_CMP0079=NEW. Keep going down this path of adding target_link_libraries or add_dependencies until we get a clean build. I suspect the changes will end up looking like my branch in PR #94. This is the "clean" option because all platforms are the same, and there will be a minimum of undefined symbols / dangling links.
Alternatively, suggest MacOS users add LDFLAGS="-Wl,-undefined -Wl,dynamic_lookup". This works with old cmake and the existing cmake code in Glewlwyd. But it is unsatisfying because clang will ignore missing symbols when linking, and there are more of them than there has to be on OSX, because cmake isn't building in the ideal order.

There are two paths I'm aware of: - Upgrade travis and all users to cmake 3.13+. Also require all builds use `-DCMAKE_POLICY_DEFAULT_CMP0079=NEW`. Keep going down this path of adding `target_link_libraries` or `add_dependencies` until we get a clean build. I suspect the changes will end up looking like my branch in PR #94. This is the "clean" option because all platforms are the same, and there will be a minimum of undefined symbols / dangling links. - Alternatively, suggest MacOS users add `LDFLAGS="-Wl,-undefined -Wl,dynamic_lookup"`. This works with old cmake and the existing cmake code in Glewlwyd. But it is unsatisfying because clang will ignore missing symbols when linking, and there are more of them than there has to be on OSX, because cmake isn't building in the ideal order.

babelouest commented

2020-02-23 23:57:13 +01:00

(Migrated from github.com)

Upgrade travis and all users to cmake 3.13

Requiring cmake to version 3.13 for all platforms isn't possible because this version is too recent and not present in some of the supported Linux distributions:

Ubuntu Bionic: 3.10
Debian Stretch: 3.7

And Glewlwyd builds and runs fine on those versions

But if we can restrict to 3.13 for OSX only I have no problem with that:

if (APPLE)
  cmake_minimum_required(VERSION 3.13)
else ()
  cmake_minimum_required(VERSION 3.5)
endif ()

Also, I'm totally ok with fixing and cleaning the order of target_link_libraries and add_dependencies but I don't have any OSX device nor the expertise to do that on my own, any help would be much appreciated.

Concerning the option -DCMAKE_POLICY_DEFAULT_CMP0079=NEW, since it's specific for OSX build, I can mention it in the paragraph Install for OSX but not for all platforms.

> Upgrade travis and all users to cmake 3.13 Requiring cmake to version 3.13 for all platforms isn't possible because this version is too recent and not present in some of the supported Linux distributions: - Ubuntu Bionic: [3.10](https://packages.ubuntu.com/bionic/cmake) - Debian Stretch: [3.7](https://packages.debian.org/stretch/cmake) And Glewlwyd builds and runs fine on those versions But if we can restrict to 3.13 for OSX only I have no problem with that: ``` if (APPLE) cmake_minimum_required(VERSION 3.13) else () cmake_minimum_required(VERSION 3.5) endif () ``` Also, I'm totally ok with fixing and cleaning the order of `target_link_libraries` and `add_dependencies` but I don't have any OSX device nor the expertise to do that on my own, any help would be much appreciated. Concerning the option `-DCMAKE_POLICY_DEFAULT_CMP0079=NEW`, since it's specific for OSX build, I can mention it in the paragraph `Install for OSX` but not for all platforms.

etuttle commented

2020-02-24 02:59:28 +01:00

(Migrated from github.com)

Got it, so we're aiming to fix this so OSX builds get the ideal build order. But retain backwards compatibility with older cmake. At the cost of if/else statement(s) in CMakeLists.txt.

Let me take a shot at that. It may end up looking like PR #94, except with if/else statements around some of the target_link_libraries calls:

	if (APPLE)
		target_link_libraries(usermoddatabase ${MOD_LIBS} ${GNUTLS_LIBRARIES} ${NETTLE_LIBRARIES} ${ULFIUS_LIBRARIES} ${HOEL_LIBRARIES} ${JANSSON_LIBRARIES} ${ORCANIA_LIBRARIES} ${YDER_LIBRARIES})
	else ()
		target_link_libraries(usermoddatabase ${MOD_LIBS})
	endif ()

... because the cross-directory target_link_libraries are required for newer cmake (and osx) but cause older cmake to fail with the error you mention in https://github.com/babelouest/glewlwyd/pull/94#issuecomment-576075808.

Got it, so we're aiming to fix this so OSX builds get the ideal build order. But retain backwards compatibility with older cmake. At the cost of if/else statement(s) in CMakeLists.txt. Let me take a shot at that. It may end up looking like PR #94, except with if/else statements around some of the `target_link_libraries` calls: ``` if (APPLE) target_link_libraries(usermoddatabase ${MOD_LIBS} ${GNUTLS_LIBRARIES} ${NETTLE_LIBRARIES} ${ULFIUS_LIBRARIES} ${HOEL_LIBRARIES} ${JANSSON_LIBRARIES} ${ORCANIA_LIBRARIES} ${YDER_LIBRARIES}) else () target_link_libraries(usermoddatabase ${MOD_LIBS}) endif () ``` ... because the cross-directory `target_link_libraries` are required for newer cmake (and osx) but cause older cmake to fail with the error you mention in https://github.com/babelouest/glewlwyd/pull/94#issuecomment-576075808.

babelouest commented

2020-12-16 15:27:31 +01:00

(Migrated from github.com)

Hello @etuttle , I'm wondering if you still plan to complete the build for OSX?

etuttle commented

2020-12-17 03:57:12 +01:00

(Migrated from github.com)

Oh I forgot about this one... sorry. Also I have a feeling that the solution we came up with is going to work everywhere but at the cost of ugly code in CMakeLists.txt, which we won't like either...

I'll try to spend some time coming up with an even better solution, but of course I have other things keeping me busy. If anyone else tries to build with homebrew libs, they should find this ticket which has hints and links to different ways to do it.

Oh I forgot about this one... sorry. Also I have a feeling that [the solution we came up with](https://github.com/babelouest/glewlwyd/issues/97#issuecomment-590145174) is going to work everywhere but at the cost of ugly code in `CMakeLists.txt`, which we won't like either... I'll try to spend some time coming up with an even better solution, but of course I have other things keeping me busy. If anyone else tries to build with homebrew libs, they should find this ticket which has hints and links to different ways to do it.

Rows
Columns

undefined symbols prevent linking on OSX / homebrew setup #97