Standardize Lix to LF line endings, no CRLF even on Windows

Started by Simon, September 17, 2026, 03:02:56 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Simon

Hi,

I want to move Lix to LF line endings consistently across all operating systems.
Explanation: What are line endings (LF/CRLF)?

Situation

Documentation in doc/ is CRLF. The levels in the levels/ have a mish-mash between CRLF and LF endings. At least it's consistent within one file. Windows Lix generates levels/replays/options/... with CRLF endings. Lix on other OSes generates files with LF endings. Lix, in every version and on every OS, accepts and understands both CRLF and LF.

When I edit levels or receive levels, I run dos2unix over everything before checking it into git, so that newer stuff is standardized to LF.

Plan

Convert to LF all documentation, levels, ..., all text files in the repository. Ship only LF on all systems. This will be one big commit in the git repository that touches a lot of files at once: 507 files changed, 171,715 insertions(+), 171,715 deletions(-)

Make Windows Lix generate levels/replays/options/... with LF, not CRLF.

Reasons

LF have been the default everywhere for over 20 years, except on Windows.

Notepad, Windows's default text editor, finally works with LF in Windows 10 and Windows 11. It shows each of LF and CRLF as a linebreak.

Before, in Windows 8 and earlier, Notepad linebroke only at CRLF. Such old Notepad would display a file with LF endings in a long single line. This is the reason why the Lix documentation in doc/ has CRLF, and why I've kept CRLF in Lix's level/replay output. I like it if users can discover the formats and hack Lix at file level. But the Notepad problem doesn't exist anymore because Notepad has been fixed in Windows 10/11.

All Lix versions (forever) have understood both CRLF and LF. If newer Lix outputs LF on every system, it doesn't break backward compatibility.

I like LF more than CRLF.

Nobody should notice any new problem in the first place, unless he uses old Notepad or makes a git diff of newer levels against the mish-mash level tree. (git diff -w will ignore the line endings.) I have a soft spot for the Windows 7 users even though I've never used 7 myself. Those users should install a good text editor: Notepad++, Geany, ...

The UTF-8 Everywhere Manifesto has a paragraph about line endings:

Quote from: https://utf8everywhere.org/#faq.crlfAlways use \n (0x0a) line endings, even on Windows. Files should be read and written in binary mode, which guarantees interoperability—a program will always give the same output on any system.

The UTF-8 Everywhere Manifesto is lovely in general. If you have some technical background, read it in full. :cute:

Advice from that explanation I linked at the start:

Quote from: https://thetexttool.com/compare/crlf-vs-lfStandardize on LF in your repository and data pipelines. Only convert to CRLF at the very end if a Windows tool requires it.

And there is no such requirement because Windows Lix has accepted LF forever.

-- Simon

Silken Healer

Quote from: Simon on September 17, 2026, 03:02:56 PMI like it if users can discover the formats and hack Lix at file level.

I also like it if users can discover formats and hack Lix at the file level too.

Quote from: Simon on September 17, 2026, 03:02:56 PMI have a soft spot for the Windows 7 users even though I've never used 7 myself. Those users should install a good text editor: Notepad++, Geany, ...

I also have a soft spot for Windows 7. I also have another soft spot for everything to just work too on the default text editor without having to install a new one, which would incline me to want CRLF to be used. Though this is just my personal soft spot, maybe not a good idea to force on everyone if LF has all these other reasons for being better? Sorry, I don't have the technical background to understand what LF or CRLF means and I haven't had the motivation to understand it lately but I wanted to add my thoughts here before you just decide to go ahead with it as no one else replies. Would there be any way for Windows 7 users still edit Lix levels on Notepad and it would just be trickier or it would just not work entirely or...? Again as I said sorry I don't have the technical background to understand what LF or CRLF means and I haven't had the motivation to read the website explaining but I wanted to put my thoughts here before you just go ahead with it due to no replies.

Forestidia86

#2
Quote from: Silken Healer on October 22, 2026, 05:01:49 PM
Quote from: Simon on September 17, 2026, 03:02:56 PMI like it if users can discover the formats and hack Lix at file level.

I also like it if users can discover formats and hack Lix at the file level too.

Quote from: Simon on September 17, 2026, 03:02:56 PMI have a soft spot for the Windows 7 users even though I've never used 7 myself. Those users should install a good text editor: Notepad++, Geany, ...

I also have a soft spot for Windows 7. I also have another soft spot for everything to just work too on the default text editor without having to install a new one, which would incline me to want CRLF to be used. Though this is just my personal soft spot, maybe not a good idea to force on everyone if LF has all these other reasons for being better? Sorry, I don't have the technical background to understand what LF or CRLF means and I haven't had the motivation to understand it lately but I wanted to add my thoughts here before you just decide to go ahead with it as no one else replies. Would there be any way for Windows 7 users still edit Lix levels on Notepad and it would just be trickier or it would just not work entirely or...? Again as I said sorry I don't have the technical background to understand what LF or CRLF means and I haven't had the motivation to read the website explaining but I wanted to put my thoughts here before you just go ahead with it due to no replies.


The problem is that Notepad on systems older than Win10 usually doesn't understand the LF line ending. This means that the text in the file appears to be in one line without line breaks (apart from maybe random line breaks if you have that option on). You can open for example anyway.txt in the Lovely folder with Notepad to see how it looks in Notepad to you (assuming you are on Win7).
The current Lix version creates CRLF endings on Windows when saving a level in the Lix editor. So you can transfrom the line ending to CRLF with "legacy" Lix. Just open Any Way You Want in the Lix editor and save. Then look at the file anyway.txt again, it should show properly then.

Simon

Quote from: Forestidia86 on October 22, 2026, 06:16:08 PMAny Way You Want

Hmm, Lovely/anyway.txt ships with LF, and Lovely/escapethepit.txt ships with CRLF. This kind of mish-mash has been in all Lix releases (current is 0.10.36) for all operating systems. That's annoying. I'm going to normalize this in git independently of what we're eventually going to ship in the releases.

Quote from: Silken Healer on October 22, 2026, 05:01:49 PMjust work too on the default text editor
Quote from: Forestidia86 on October 22, 2026, 06:16:08 PMNotepad on systems older than Win10 usually doesn't understand the LF line ending. This means that the text in the file appears to be in one line without line breaks

Right, Windows Notepad in Windows 8/7/earlier ignores lone LF characters, and instead shows all LF-separated lines text together as a single long line. I consider this an outright bug in Notepad.

One idea is to automatically convert all levels/docs/... to CRLF when I create the Windows release. I.e.: Keep LF in git, ship LF for Linux, and ship CRLF for Windows. Then the files will look different on different systems, have different sizes, different checksums, ... but it's probably the strongest contender to shipping LF everywhere.

Still I'd prefer to ship LF everywhere. Everybody will get the same files, everybody will see shared semantically-identical levels as file-identical, all regardless of tooling. Among only the Windows users (global OS share, not specific to Lix users), the ratio of Win 8/7/earlier dropped from 3 % in January 2025 to 1 % in October 2026, source. I expect this downward trend to continue. And in that small ratio, you're only hit if you use the broken default text editor. I am willing to work around some bugs, that's why I've shipped the documentation in CRLF for years, but the benefit is getting too low.

Quote from: Silken Healer on October 22, 2026, 05:01:49 PMWould there be any way for Windows 7 users still edit Lix levels on Notepad and it would just be trickier

On Windows 7, mass-convert everything manually to CRLF, then edit in Notepad. A common way to convert on Linux is unix2dos. I don't know about the easiest way in Windows. Stack overflow question: Anything like dos2unix for Windows? says: Install Notepad++ and use its mass conversion feature. Or use a Powershell one-liner.

-- Simon