Lemmings Forums
Lix => Lix Main => Topic started by: Simon on April 01, 2018, 09:07:15 AM
-
The homepage's Lix download for Windows (http://www.lixgame.com) is a 32-bit executable. Building 64-bit Windows executables from D source is possible, but hard. (This was true in 2018, it's now wrong.) I'll continue to ship 32-bit builds.
Since ~2020, I've shipped 64-bit Windows builds and 32-bit Windows builds.
Problems of 32-bit: Windows D Lix 32-bit may crash on huge maps (width x height >= 2.8 million pixels). Our theory is that it allocates so much VRAM that it begins swapping the VRAM to RAM, and 32-bit Lix cannot address more than 2 GB of RAM. My workaround is to print a warning over the level preview on huge maps, but still allow to play these maps. Post on the D forums about trouble with the 32-bit GC (https://forum.dlang.org/thread/pmunyfdozzumkbqgxptd@forum.dlang.org).
64-bit Lix would still be a chubby glutton who belongs into a diet camp, but at least 64-bit Lix won't crash on large maps, and overall RAM consumption is less. No idea why. And any Windows computer younger than 12 years can run 64-bit apps.
Reqires fat MSVS: While 32-bit builds with dmd and its included optlink, 64-bit need a different linker. It looks like the only working 64-bit linker comes with Microsoft Visual Studio (MSVS), a proprietary IDE + SDK + toolchain that requires 5 GB to even install. Maybe you don't need the IDE itself, only the toolchain and the SDK, but it's still a sizable chunk. The Community edition may be gratis, but you're subjected to licensing terms that seem crazy when you're accustomed to free-and-open-source.
MinGW + D = crash: A possible alternative for MSVS is the MinGW toolchain. I've used that in the past to compile C++ Lix and am moderately familiar with it. It offers gcc, g++, make, and ld for Windows, and probably some other useful tools. This wiki entry explains installing dmd (https://wiki.dlang.org/Installing_DMD), but it doesn't explain the MinGW toolchain, or how it interacts with dmd.
One direct test with MinGW and the MS build tools made Windows 64-bit Lix crash at runtime. No stacktracke, but we didn't enter it with a debugger. The offered MinGW variant still uses VC++ 2010 redistributable, maybe that's the problem for running the app.
NuGet DLLs/LIBs are good: To run Lix, you need Allegro 5 DLLs (I ship them with the binary Windows download), and to build Lix, you must create LIBs from the DLLs. Recently, I've tried to update the DLLs. Suddenly, the Allgero DLLs required extra DLLs (not from Allgero) that weren't needed before, therefore I've postponed updating Allgero. But Allegro offers NuGet packages both for 32-bit and 64-bit, you can extract DLLs with readily-made LIBs from those, and they don't require any extra DLLs. I should use these in the binary download.
All of this research is not by me, but by Forestidia. Thanks for your research! Please nudge me to fix any mistakes above.
-- Simon
-
Since version 1.13.0 ldc self-sufficiently creates 64bit builds on Win, user only needs a particular Visual C++ runtime, which most should have.
Apparently even dmd is self-sufficient since version 2.079 (https://dlang.org/blog/2018/03/03/dmd-2-079-0-released/). For an successful build you need to put 64bit Allegro libs in the lib64 folder. For suppression of console window change in dub.json the line "lflags-windows-dmd": ["-L/subsystem:windows"], to two lines"lflags-windows-x86-dmd": ["-L/subsystem:windows"],
"lflags-windows-x86_64-dmd": ["-subsystem:windows", "-entry:mainCRTStartup"],
as of version 2.093.1
-
Since version 1.13.0 ldc self-sufficiently creates 64bit builds on Win, user only needs a particular Visual C++ runtime, which most should have.
Apparently even dmd is self-sufficient since version 2.079 (https://dlang.org/blog/2018/03/03/dmd-2-079-0-released/). For an successful build you need to put 64bit Allegro libs in the lib64 folder. For suppression of console window change in dub.json the line "lflags-windows-dmd": ["-L/subsystem:windows"], to two lines"lflags-windows-x86-dmd": ["-L/subsystem:windows"],
"lflags-windows-x86_64-dmd": ["-subsystem:windows", "-entry:mainCRTStartup"],
as of version 2.093.1
It is possible that dmd at some future time updates the linker so that there as with ldc the wmain C entry point is then the correct one. Then the linker flag for dmd 64bit becomes the same as the ldc one ('wmain' instead of 'main').
Versions of ldc older than 1.21.0 had still the main C entry point as above. Change was through that pull request (https://github.com/ldc-developers/ldc/pull/3351) for v. 1.21.0.
-
Since version 1.13.0 ldc self-sufficiently creates 64bit builds on Win, user only needs a particular Visual C++ runtime, which most should have.
Apparently even dmd is self-sufficient since version 2.079 (https://dlang.org/blog/2018/03/03/dmd-2-079-0-released/). For an successful build you need to put 64bit Allegro libs in the lib64 folder. For suppression of console window change in dub.json the line "lflags-windows-dmd": ["-L/subsystem:windows"], to two lines"lflags-windows-x86-dmd": ["-L/subsystem:windows"],
"lflags-windows-x86_64-dmd": ["-subsystem:windows", "-entry:mainCRTStartup"],
as of version 2.093.1
It is possible that dmd at some future time updates the linker so that there as with ldc the wmain C entry point is then the correct one. Then the linker flag for dmd 64bit becomes the same as the ldc one ('wmain' instead of 'main').
Versions of ldc older than 1.21.0 had still the main C entry point as above. Change was through that pull request (https://github.com/ldc-developers/ldc/pull/3351) for v. 1.21.0.
The wmain C entry point seems to be part of the ldc variant of druntime, so I don't know if it ever gets a thing for dmd. As a non-programmer I can't fully assess that, but I think I got led astray by the fact that it manifests itself in the linker flag. So probably unrelated to the linker but rather to the library. (?)
-
Thanks for the investigation and the theory!
The entry point will likely depend on the runtime. It's possible that LDC and DMD come each with their own, different, C and D runtimes.
When the program starts, execution begins not in Lix; the D runtime first sets up things like the garbage collector, then runs Lix's main(). Now, I haven't researched enough what (w)mainCRTStartup is; it sounds like the entry point for the C runtime, not the D runtime, which then in turn jumps into the D runtime.
For the future, I'll keep possible changes of the runtime entry point names in mind.
-- Simon
-
ldc 1.25 has upgraded the linker lld (to v11.0.1) and the upgraded linker lets even Hello World programs fail on Win 7 ('lld-link: error: failed to write the output file: permission denied') and Wine 6.4 ('lld-link: error: failed to write the output file: Path is invalid.'). Win 8.1 seems still fine for building with ldc/lld-link.
See issue at ldc (https://github.com/ldc-developers/ldc/issues/3675)
See issue at LLVM (https://bugs.llvm.org/show_bug.cgi?id=49503)
See possible fix at LLVM(?) (https://reviews.llvm.org/rG64ab2b6825c5aeae6e4afa7ef0829b89a6828102)
See the upper possible fix at github (https://github.com/llvm/llvm-project/commit/64ab2b6825c5aeae6e4afa7ef0829b89a6828102)
See similiar bug at LLVM (https://bugs.llvm.org/show_bug.cgi?id=48378)
-
(One of) the maintainer(s) of ldc doesn't think that the possible fix works for the Win7/Wine issue. See new issue to integrate fix (https://github.com/ldc-developers/ldc/issues/3691).
I'm not convinced but don't know either. Basically I read that as a wontfix but at some time LLVM will get the upgrade with the possible fix and later it will be so probably integrated in ldc. No clue how long that will take.
I personally haven't gotten the workaround to work with dub. Maybe it could be helpful with dmd in the future in case there the problem occurs as well.
Until version 1.24 ldc should still work for Win 7 and Wine users.
-
It seems that the possible fix will be part of 12.0.0 of LLVM.
-
Good news: 12.0.0 of LLVM is released, unclear when it gets part of ldc.
If I have suitable bindings built, I could get Lix to build with a variant of the workaround by integrating in the dub.json file:
"dflags-ldc2": ["--linker=lld-link"],probably better:
"dflags-windows-x86_64-ldc2": ["--linker=lld-link"],
Unfortunately not any built bindings seem to work. It is tedious even with suitable bindings built.
Each binding that has to be built has its own dub.json.
Building the bindings with ldc 1.25.x went into the problem that the called argument '-lib' seems to override or similiar effect the dflag.
If I copied the command line that 'dub -v' provided and deleted '-lib' it complained about "subsystem must be defined". Maybe the mixing of compiler and linker, maybe sth. else.
I didn't get to work a direct integration of lld-link in the config file of ldc.
-
ldc 1.26 is released and --version still unfortunately says that it is LLVM 11.0.1 based, although they have in their LLVM fork already 1.12.0. Maybe for the next version 1.27.
-
Wine v. 6.7 seems to build with LDC 1.25.1, so the LLVM 11.0.1 linker problem only applies to Win 7 in the end.
-
Good news: ldc 1.27.0-beta1 is out and based on LLVM 12.0.0.
It works on Win7 and I assume that will apply to older versions of Wine as well.
Edit: It doesn't work for older versions of Wine unfortunately.
-
Only verified for Arch-Linux, but very well possible that it affects Win as well:
- Cutting edge DMD v2.098.1 doesn't build a release build anymore:
Error:
'function file.filename.base.IFilename.isChildOf
FAIL .dub/build/application-release-linux.posix-x86_64-dmd_v2.098.1-B60974BF1C37AB9E479B0D5164D1DB7B/ lix executable
dmd failed with exit code -4.
Full exception: object.Exception@source/dub/compilers/compiler.d(116): dmd failed with exit code -4.
----------------
??:? [0x55bc83f0c226]'
- Cutting edge ldc 1.28.1 doesn't build anymore without increasing the stack size (on Linux possible with e.g. 'ulimit -s 16384')
Error could be related to [Windows] ldc2 fails to compile project with various sudden halts seen in -vv output #3913 (https://github.com/ldc-developers/ldc/issues/3913) (stack overflow)
-
Thanks for documenting these!
When the compilers get new releases, I'll look into these again. I hope these problems will fix themselves.
The DMD release build is particularly odd, DMD does not crash, but doesn't print an error message either; it just exits with code -4, the normal error code that it also gives on any normal static error in your D code.
Note to myself: I should Dustmite Lix for this -4 with no other error message. Dustmite will produce the minimal example that still brings DMD to this behavior. Merely Dustmiting for -4 is not enough, a few lines with one undeclared variable will also give -4; I'll have to explicitly Dustmite for -4 and no printed compiler error message.
-- Simon
-
Building Lix with cutting edge ldc on Win doesn't seem to be affected by the stack overflow.
The release build with dmd is fixed in Lix 0.9.42.
-
The win-build.bat specifies the compiler to ldc. The question is if this is necessary or should at least mentioned in the file name since somebody could just have dmd set up and get an error.
-
In the Windows 64-bit build instructions, we recommend to install LDC and then to double-click win-build.bat. This works well and I recommend this to everybody for release builds. LDC builds the fastest-running executables.
The win-build.bat specifies the compiler to ldc. The question is if this is necessary or should at least mentioned in the file name since somebody could just have dmd set up and get an error.
Now, if somebody installs only DMD and not LDC, he has diverged from the release notes early. I'll assume that he knows what he's doing. The build instructions even cover DMD and tell him to type the command manually. If he still double-clicks win-build.bat, he'll get the following error: Error Failed to spawn process "ldc2" (The system cannot find the given file.) Given that he see this only after he has willfully diverged from the build instructions and installed DMD instead of LDC, I'd say it's fine as it is. He can edit the script or type the build command manually.
I could also write win-build.bat to be compiler-agnostic, but then it'll pick DMD over LDC when both are installed. I encourage everybody to build release builds with LDC. DMD is better for active development, when you need the fastest-possible feedback and care less about producing fast-running binaries.
It should be possible to test in the script for installed compilers ahead of the dub call, e.g., with a Windows equivalent of Linux's which. Then you build with LDC explicitly only if LDC is found, otherwise you build in the compiler-agnostic way. If you want to investigate this, I'll be happy to merge it.
Readers may wonder: Why different compilers for different builds, aren't there build types? Yes, there are build types; you can instruct either compiler to build debug binaries or optimize for release. Still, you'll see the fastest build times with DMD without optimization, and you'll get the fastest-running binaries from LDC in release mode.
-- Simon