Enable DECCOLM support via a private mode escape sequence

[j4james]

Summary of the Pull Request

While the DECCOLM escape sequence is technically implemented, it's disabled by a flag that is not user-accessible. This PR adds support for an escape sequence (XTerm's private mode 40) which essentially enables that flag, so users can turn on DECCOLM support in an XTerm-compatible way. Tested manually, with Vttest, and with unit tests.

PR Checklist

• Closes DECCOLM escape sequence doesn't work #171
• CLA signed. If not, go over here and sign the CLA
• Tests added/passed
• Requires documentation to be updated
• I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: DECCOLM escape sequence doesn't work #171

Detailed Description of the Pull Request / Additional comments

My initial plan was to build this on top of the existing _fIsSetColumnsEnabled flag, but after further investigation that proved not to be practical. In order to be compatible with XTerm, the flag needs to be checked in the _DoDECCOLMHelper method rather than in SetColumns, since it needs to prevent all facets of the DECCOLM operation taking effect - not just the column sizing.

And in the future, if the SetColumns method is also going to be used to support the DECSCPP escape sequence, that will need a flag of its own - it's not controlled by the same option as DECCOLMN.

For those reasons, I've removed the existing _fIsSetColumnsEnabled flag and added a new _fIsDECCOLMAllowed flag which is checked in the _DoDECCOLMHelper method. I've then added a EnableDECCOLMSupport method to toggle that flag, and added a case in _PrivateModeParamsHelper to call that method in response to the private mode 40 sequence.

While I did mention my initial plans on issue #171, I never actually got feedback from core contributors, so I completely understand if this PR is rejected in favour of a different approach. I'm just offering it as one possible solution.

Validation Steps Performed

I've added an output engine unit test which validates that the private mode escape sequences trigger the EnableDECCOLMSupport call, and some screen buffer unit tests which check that the DECCOLM sequence does take effect when allowed, and doesn't do anything when disallowed.

I've also done a fair amount of manual testing, and run several of the Vttest suites which toggle between 80 and 132 column modes, and which now work (at least in terms of column width) where previously they didn't.

[zadjii-msft]

This is great. Thanks!

[DHowett-MSFT]

Curious: What happens in xterm if you DECCOLM and then enter the alternate screen buffer? Or, enable private mode 40, then enter the alt buffer? Or enter the alt buffer, then enable DECCOLM?

[j4james]

Curious: What happens in xterm if you DECCOLM and then enter the alternate screen buffer? Or, enable private mode 40, then enter the alt buffer? Or enter the alt buffer, then enable DECCOLM?

Good questions. First off, private mode 40 doesn't seem to be affected by anything other than the SM/RM mode sequences themselves. Not even an RIS hard reset will change that.

As for DECCOLM itself, changing back and forth to the alt buffer doesn't seem to affect it. If you made the screen size smaller while you're in the alt buffer (by whatever means), once you restore the original buffer, you can end up losing content that is now cut off by the smaller screen size. That makes sense when you consider that the resize could be a manual user choice, which they're unlikely to want reset when the buffer is restored.

In the Windows terminal it seems the full original buffer is restored, but not the original size, so you'll get a horizontal scroll bar. This doesn't seem unreasonable either, but it's not exactly compatible with XTerm. So that may well be something you want to fix, but I think it's probably orthogonal to the DECCOLM functionality - it's already an issue for the other resizing escape sequence (CSI 8 ; h ; w t).

Another difference I noticed in testing now is that XTerm resets DECCOLM (i.e. the screen width goes back to 80) after an RIS hard reset (it doesn't reset other resizing operations though). If we wanted to match that behaviour, I think we'd have to add an additional state variable to track the fact that DECCOLM had been set, so we know to reset it. I'm not sure if that is really worth the effort.

[DHowett-MSFT]

This is awesome. Thanks for working on it!

[miniksa]

I'm good with this.

[ghost]

🎉Windows Terminal Preview v0.3.2142.0 has been released which incorporates this pull request.:tada:

Handy links:

• Release Notes
• Store Download