Building outside source dir fails on MinGW/MSYS

[vknecht]

With 1.4 branch (still have to test with trunk/1.9), trying to build CS outside the source directory with MinGW/MSYS fails due to 1st level Jamfiles (like $TOPDIR/include/Jamfile, $TOPDIR/data/Jamfile, ...) not being found. No problem like that on Linux.

[sunshine]

Maybe post the actual error message(s) along with Jamconfig, the generated Jamfile, config.log, and csconfig.h.

[vknecht]

Error message is as follows:

[...]
checking whether to emulate makefiles... yes
configure: creating ./config.status
config.status: creating Jamfile
config.status: executing include/csconfig.h commands
config.status: executing Jamconfig commands
config.status: executing Makefile commands
config.status: WARNING: failed to create makefile emulation layer
/home/USERNAME/14/CS/mk/Jamfile: No such file or directory
/home/USERNAME/14/CS/data/Jamfile: No such file or directory
/home/USERNAME/14/CS/libs/Jamfile: No such file or directory
/home/USERNAME/14/CS/plugins/Jamfile: No such file or directory
/home/USERNAME/14/CS/apps/Jamfile: No such file or directory
/home/USERNAME/14/CS/docs/Jamfile: No such file or directory
/home/USERNAME/14/CS/include/Jamfile: No such file or directory
/home/USERNAME/14/CS/scripts/Jamfile: No such file or directory
warning: unknown rule IncludeDir
warning: unknown rule IncludeDir
warning: unknown rule IncludeDir
warning: unknown rule CleanDir
warning: unknown rule CleanDir
warning: unknown rule Help
warning: unknown rule InstallDoc
warning: unknown rule Description
warning: unknown rule Description
warning: unknown rule Description
warning: unknown rule Description
warning: unknown rule Description
warning: unknown rule Description
warning: unknown rule Description
warning: unknown rule Description
warning: unknown rule Description
warning: unknown rule Description
warning: unknown rule Description
warning: unknown rule Description
warning: unknown rule Description
warning: unknown rule Description
warning: unknown rule RegisterCompileGroups
warning: unknown rule Property
don't know how to make staticplugins

[sunshine]

Replying to vknecht:

/home/USERNAME/14/CS/mk/Jamfile: No such file or directory

Silly question: Did you verify that these files are present?

[vknecht]

Replying to sunshine:

Replying to vknecht:

/home/USERNAME/14/CS/mk/Jamfile: No such file or directory

Silly question: Did you verify that these files are present?

Indeed, they're here, 'ls'ing or 'vi'ing them works.

[sunshine]

Is the /home/USERNAME/14 a Windows path or an MSYS path?

[vknecht]

It's a MSYS path I guess. /home is mapped to the home directory under MSYS install directory.

[sunshine]

Replying to vknecht:

It's a MSYS path I guess. /home is mapped to the home directory under MSYS install directory.

That's probably the problem. Jam won't know anything about MSYS paths, so it can't convert it to a Windows path. As a test, edit the TOP variable in the generated Jamconfig so that it mentions the actual Windows path corresponding to that MSYS path, and see if Jam commands work after that. Ideally, the correct value for TOP needs to be set automatically, but it is not yet clear how to do so. Does MSYS have a command for performing path transformations like Cygwin's cygpath?

[sunshine]

Replying to sunshine:

As a test, edit the TOP variable in the generated Jamconfig

I meant the TOP variable in the generated Jamfile. It is the very first line.

[vknecht]

Replying to sunshine:

Replying to sunshine:

As a test, edit the TOP variable in the generated Jamconfig

I meant the TOP variable in the generated Jamfile. It is the very first line.

I did that, using the form /drive-letter/path/to/home/USERNAME/14/CS, and still the same problem (error message reflects that new path). I also tried with CS source and build dirs outside MSYS home, the /drive-letter/path/to/CS path-type was then used automatically, but errror still arises. I could also reproduce the problem on another machine with fresh MinGW/MSYS installed.

There's the mount command, which gives something like that:

$ mount
C:\DOCUME~1\vince\LOCALS~1\Temp on /tmp type user (binmode,noumount)
C:\msys\1.0 on / type user (binmode,noumount)
C:\msys\1.0 on /usr type user (binmode,noumount)
c:\mingw on /mingw type user (binmode)
a: on /a type user (binmode,noumount)
c: on /c type user (binmode,noumount)
h: on /h type user (binmode,noumount)
r: on /r type user (binmode,noumount)

I see nothing else obvious in /bin or /usr/bin which deals with paths...

[sunshine]

Replying to vknecht:

I did that, using the form /drive-letter/path/to/home/USERNAME/14/CS, and still the same problem (error message reflects that new path).

I think you may have misread my instructions. As a test, you should set TOP to the actual Windows-style path (i.e. c:\foo\bar or c:/foo/bar), not an MSYS-style /drive/foo/bar path.

[vknecht]

Oh, right. I had tried c:\foo\bar but it didn't turn out well (takes \ as escape char). Now, trying c:/foo/bar/ it works.

[sunshine]

Replying to vknecht:

Oh, right. I had tried c:\foo\bar but it didn't turn out well (takes \ as escape char). Now, trying c:/foo/bar/ it works.

Backslash can be normalized to forward slashes via CS_PATH_NORMALIZE() in CS/mk/autoconf/path.m4.

I suppose it would be possible to augment CS_PATH_NORMALIZE() so that it also knows how to translate /c/foo/bar paths under MSYS to corresponding c:/foo/bar paths since it is a straight-forward transformation.

Dealing with other MSYS paths, however, is more difficult. It could be possible to parse the output of 'mount' and perform conversion of MSYS path to Windows path based upon that information, but the approach seems quite ugly and unnecessarily difficult and fragile.

[sunshine]

Replying to sunshine:

Dealing with other MSYS paths, however, is more difficult.

We may be able to take advantage of the MSYS_AC_CANONICAL_PATH() Autoconf macro posted here: http://article.gmane.org/gmane.comp.gnu.mingw.msys/2785

It can be invoked behind-the-scenes by CS_PATH_NORMALIZE(), thus hiding the ugliness.

[sunshine]

Replying to sunshine:

It can be invoked behind-the-scenes by CS_PATH_NORMALIZE(), thus hiding the ugliness.

Unfortunately, this is complicated by the fact that CS_PATH_NORMALIZE() generically normalizes all paths in its input argument. It does not require that its argument is solely a path. For instance, 'foo-config --cflags' might emit several -I directives with corresponding paths, and CS_PATH_NORMALIZE() will correctly normalize all of them. This feature is used heavily by the CS/mk/autoconf/*.m4 macros.

[sunshine]

Resolved by r31050 and r31051.

[vknecht]

Merged in 1.4 with r31105 and r31106.
CEL seems to need the same treatment. Will make some tests tomorrow if I have time.

[sunshine]

Replying to vknecht:

CEL seems to need the same treatment.

The changes were applied to CEL trunk in cel:r3672 and cel:r3673.

[vknecht]

Tested with 1.4, it's basically, except for a glitch with CS static plugins when building static CEL apps:

[...]
WriteDepFlags2 ./out/msysx86/optimize/libs/cel_staticplugins.jam 
WriteRelayDeps ./out/msysx86/optimize/libs/cel_staticplugins.jam 
...updated 320 target(s)...
h:ome/USERNAME/14/CSbuild/./out/msysx86/optimize/libs/crystalspace_staticplugins.jam: No such file or directory
celtst_static: Warning: Static plugin "bezier" from package "CRYSTAL" not available. 
celtst_static: Warning: Static plugin "bezierldr" from package "CRYSTAL" not available. 
[...]

Looking in CEL's Jamconfig, I see:

CRYSTAL.STATICDEPS ?= h:ome/USERNAME/14/CSbuild/./out/msysx86/optimize/libs/crystalspace_staticplugins.jam ;

[sunshine]

Replying to vknecht:

Tested with 1.4, it's basically, except for a glitch with CS static plugins when building static CEL apps:

Moved this problem to its own ticket: #541