Skip to content

Commit

Permalink
Use VT for COOKED_READ_DATA (#17445)
Browse files Browse the repository at this point in the history
By rewriting `COOKED_READ_DATA` to use VT for its output we make it
possible to pass this VT output 1:1 straight to the hosting terminal
if we're running under ConPTY. This is also possible with the current
console APIs it uses, but it's somewhat janky. In particular the
usage of `ReadConsoleOutput` to backup/restore the popup contents
could be considered bad faith "rules for thee, not for me",
given that we're telling people to move away from those APIs.

The new implementation contains a bare bones "pager" to fit even
very long prompt contents into the VT viewport.
I fully expect this initial PR to not be entirely bug free, because
writing a proper pager with line wrapping is a little bit complex.
This PR takes some significant shortcuts by leveraging the fact
that the prompt line is always left-to-right and always a series
of fully filled lines followed by one potentially semi-full line.
This allows us to skip using a front/back-buffer for diffing the
contents between two redisplay calls.

Part of #14000

## Validation Steps Performed
* ASCII input
* Chinese input (中文維基百科) ✅
* Surrogate pair input (🙂) ✅
* In cmd.exe
  * Create 2 files: "a😊b.txt" and "a😟b.txt"
  * Press tab: Autocomplete to "a😊b.txt" ✅
  * Navigate the cursor right past the "a"
  * Press tab twice: Autocomplete to "a😟b.txt" ✅
* Execute `printf("    "); gets(buffer);` in C (or equivalent)
  * Press Tab, A, Ctrl+V, Tab, A ✅
  * The prompt is "        A^V     A" ✅
  * Cursor navigation works ✅
  * Backspacing/Deleting random parts of it works ✅
  * It never deletes the initial 4 spaces ✅
* Backspace deletes preceding glyphs ✅
* Ctrl+Backspace deletes preceding words ✅
* Escape clears input ✅
* Home navigates to start ✅
* Ctrl+Home deletes text between cursor and start ✅
* End navigates to end ✅
* Ctrl+End deletes text between cursor and end ✅
* Left navigates over previous code points ✅
* Ctrl+Left navigates to previous word-starts ✅
* Right and F1 navigate over next code points ✅
  * Pressing right at the end of input copies characters
    from the previous command ✅
* Ctrl+Right navigates to next word-ends ✅
* Insert toggles overwrite mode ✅
* Delete deletes next code point ✅
* Up and F5 cycle through history ✅
  * Doesn't crash with no history ✅
  * Stops at first entry ✅
* Down cycles through history ✅
  * Doesn't crash with no history ✅
  * Stops at last entry ✅
* PageUp retrieves the oldest command ✅
* PageDown retrieves the newest command ✅
* F2 starts "copy to char" prompt ✅
  * Escape dismisses prompt ✅
  * Typing a character copies text from the previous command up
    until that character into the current buffer (acts identical
    to F3, but with automatic character search) ✅
* F3 copies the previous command into the current buffer,
  starting at the current cursor position,
  for as many characters as possible ✅
  * Doesn't erase trailing text if the current buffer
    is longer than the previous command ✅
  * Puts the cursor at the end of the copied text ✅
* F4 starts "copy from char" prompt ✅
  * Escape dismisses prompt ✅
  * Erases text between the current cursor position and the
    first instance of a given char (but not including it) ✅
* F6 inserts Ctrl+Z ✅
* F7 without modifiers starts "command list" prompt ✅
  * Escape dismisses prompt ✅
  * Entries wider than the window width are truncated ✅
  * Height expands up to 20 rows with longer histories ✅
  * F9 starts "command number" prompt ✅
  * Left/Right replace the buffer with the given command ✅
    * And put cursor at the end of the buffer ✅
  * Up/Down navigate selection through history ✅
    * Stops at start/end with <10 entries ✅
    * Stops at start/end with >20 entries ✅
    * Scrolls through the entries if there are too many ✅
  * Shift+Up/Down moves history items around ✅
  * Home navigates to first entry ✅
  * End navigates to last entry ✅
  * PageUp navigates by $height items at a time or to first ✅
  * PageDown navigates by $height items at a time or to last ✅
* Alt+F7 clears command history ✅
* F8 cycles through commands that start with the same text as
  the current buffer up until the current cursor position ✅
  * Doesn't crash with no history ✅
* F9 starts "command number" prompt ✅
  * Escape dismisses prompt ✅
  * Ignores non-ASCII-decimal characters ✅
  * Allows entering between 1 and 5 digits ✅
  * Pressing Enter fetches the given command from the history ✅
* Alt+F10 clears doskey aliases ✅
* In cmd.exe, with an empty prompt in an empty directory:
  Pressing tab produces an audible bing and prints no text ✅
* When Narrator is enabled, in cmd.exe:
  * Typing individual characters announces only
    exactly each character that is being typed ✅
  * Backspacing at the end of a prompt announces
    only exactly each deleted character ✅
  • Loading branch information
lhecker authored Jul 11, 2024
1 parent 30447cf commit ac5b4f5
Show file tree
Hide file tree
Showing 15 changed files with 839 additions and 731 deletions.
19 changes: 7 additions & 12 deletions doc/COOKED_READ_DATA.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,8 @@

All of the following ✅ marks must be fulfilled during manual testing:
* ASCII input
* Chinese input (中文維基百科) ❔
* Resizing the window properly wraps/unwraps wide glyphs ❌
Broken due to `TextBuffer::Reflow` bugs
* Surrogate pair input (🙂) ❔
* Resizing the window properly wraps/unwraps surrogate pairs ❌
Broken due to `TextBuffer::Reflow` bugs
* Chinese input (中文維基百科) ✅
* Surrogate pair input (🙂) ✅
* In cmd.exe
* Create 2 file: "a😊b.txt" and "a😟b.txt"
* Press tab: Autocomplete to "a😊b.txt" ✅
Expand Down Expand Up @@ -62,21 +58,20 @@ All of the following ✅ marks must be fulfilled during manual testing:
* F6 inserts Ctrl+Z ✅
* F7 without modifiers starts "command list" prompt ✅
* Escape dismisses prompt ✅
* Minimum size of 40x10 characters ✅
* Width expands to fit the widest history command ✅
* Entries wider than the window width are truncated ✅
* Height expands up to 20 rows with longer histories ✅
* F9 starts "command number" prompt ✅
* Left/Right paste replace the buffer with the given command ✅
* Left/Right replace the buffer with the given command ✅
* And put cursor at the end of the buffer ✅
* Up/Down navigate selection through history ✅
* Stops at start/end with <10 entries ✅
* Stops at start/end with >20 entries ✅
* Wide text rendering during pagination with >20 entries
* Scrolls through the entries if there are too many
* Shift+Up/Down moves history items around ✅
* Home navigates to first entry ✅
* End navigates to last entry ✅
* PageUp navigates by 20 items at a time or to first ✅
* PageDown navigates by 20 items at a time or to last ✅
* PageUp navigates by $height items at a time or to first ✅
* PageDown navigates by $height items at a time or to last ✅
* Alt+F7 clears command history ✅
* F8 cycles through commands that start with the same text as
the current buffer up until the current cursor position ✅
Expand Down
13 changes: 0 additions & 13 deletions src/buffer/out/cursor.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@ Cursor::Cursor(const ULONG ulSize, TextBuffer& parentBuffer) noexcept :
_fBlinkingAllowed(true),
_fDelay(false),
_fIsConversionArea(false),
_fIsPopupShown(false),
_fDelayedEolWrap(false),
_fDeferCursorRedraw(false),
_fHaveDeferredCursorRedraw(false),
Expand Down Expand Up @@ -66,11 +65,6 @@ bool Cursor::IsConversionArea() const noexcept
return _fIsConversionArea;
}

bool Cursor::IsPopupShown() const noexcept
{
return _fIsPopupShown;
}

bool Cursor::GetDelay() const noexcept
{
return _fDelay;
Expand Down Expand Up @@ -126,13 +120,6 @@ void Cursor::SetIsConversionArea(const bool fIsConversionArea) noexcept
_RedrawCursorAlways();
}

void Cursor::SetIsPopupShown(const bool fIsPopupShown) noexcept
{
// Functionally the same as "Hide cursor"
_fIsPopupShown = fIsPopupShown;
_RedrawCursorAlways();
}

void Cursor::SetDelay(const bool fDelay) noexcept
{
_fDelay = fDelay;
Expand Down
3 changes: 0 additions & 3 deletions src/buffer/out/cursor.h
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,6 @@ class Cursor final
bool IsBlinkingAllowed() const noexcept;
bool IsDouble() const noexcept;
bool IsConversionArea() const noexcept;
bool IsPopupShown() const noexcept;
bool GetDelay() const noexcept;
ULONG GetSize() const noexcept;
til::point GetPosition() const noexcept;
Expand All @@ -61,7 +60,6 @@ class Cursor final
void SetBlinkingAllowed(const bool fIsOn) noexcept;
void SetIsDouble(const bool fIsDouble) noexcept;
void SetIsConversionArea(const bool fIsConversionArea) noexcept;
void SetIsPopupShown(const bool fIsPopupShown) noexcept;
void SetDelay(const bool fDelay) noexcept;
void SetSize(const ULONG ulSize) noexcept;
void SetStyle(const ULONG ulSize, const CursorType type) noexcept;
Expand Down Expand Up @@ -99,7 +97,6 @@ class Cursor final
bool _fBlinkingAllowed; //Whether or not the cursor is allowed to blink at all. only set through VT (^[[?12h/l)
bool _fDelay; // don't blink scursor on next timer message
bool _fIsConversionArea; // is attached to a conversion area so it doesn't actually need to display the cursor.
bool _fIsPopupShown; // if a popup is being shown, turn off, stop blinking.

bool _fDelayedEolWrap; // don't wrap at EOL till the next char comes in.
til::point _coordDelayedAt; // coordinate the EOL wrap was delayed at.
Expand Down
2 changes: 1 addition & 1 deletion src/cascadia/TerminalCore/terminalrenderdata.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ til::point Terminal::GetCursorPosition() const noexcept
bool Terminal::IsCursorVisible() const noexcept
{
const auto& cursor = _activeBuffer().GetCursor();
return cursor.IsVisible() && !cursor.IsPopupShown();
return cursor.IsVisible();
}

bool Terminal::IsCursorOn() const noexcept
Expand Down
63 changes: 63 additions & 0 deletions src/host/VtIo.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -518,3 +518,66 @@ bool VtIo::IsResizeQuirkEnabled() const
}
return S_OK;
}

// Formats the given console attributes to their closest VT equivalent.
// `out` must refer to at least `formatAttributesMaxLen` characters of valid memory.
// Returns a pointer past the end.
static constexpr size_t formatAttributesMaxLen = 16;
static char* formatAttributes(char* out, const TextAttribute& attributes) noexcept
{
static uint8_t sgr[] = { 30, 31, 32, 33, 34, 35, 36, 37, 90, 91, 92, 93, 94, 95, 96, 97 };

// Applications expect that SetConsoleTextAttribute() completely replaces whatever attributes are currently set,
// including any potential VT-exclusive attributes. Since we don't know what those are, we must always emit a SGR 0.
// Copying 4 bytes instead of the correct 3 means we need just 1 DWORD mov. Neat.
//
// 3 bytes.
memcpy(out, "\x1b[0", 4);
out += 3;

// 2 bytes.
if (attributes.IsReverseVideo())
{
memcpy(out, ";7", 2);
out += 2;
}

// 3 bytes (";97").
if (attributes.GetForeground().IsLegacy())
{
const uint8_t index = sgr[attributes.GetForeground().GetIndex()];
out = fmt::format_to(out, FMT_COMPILE(";{}"), index);
}

// 4 bytes (";107").
if (attributes.GetBackground().IsLegacy())
{
const uint8_t index = sgr[attributes.GetBackground().GetIndex()] + 10;
out = fmt::format_to(out, FMT_COMPILE(";{}"), index);
}

// 1 byte.
*out++ = 'm';
return out;
}

void VtIo::FormatAttributes(std::string& target, const TextAttribute& attributes)
{
char buf[formatAttributesMaxLen];
const size_t len = formatAttributes(&buf[0], attributes) - &buf[0];
target.append(buf, len);
}

void VtIo::FormatAttributes(std::wstring& target, const TextAttribute& attributes)
{
char buf[formatAttributesMaxLen];
const size_t len = formatAttributes(&buf[0], attributes) - &buf[0];

wchar_t bufW[formatAttributesMaxLen];
for (size_t i = 0; i < len; i++)
{
bufW[i] = buf[i];
}

target.append(bufW, len);
}
3 changes: 3 additions & 0 deletions src/host/VtIo.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,9 @@ namespace Microsoft::Console::VirtualTerminal
class VtIo
{
public:
static void FormatAttributes(std::string& target, const TextAttribute& attributes);
static void FormatAttributes(std::wstring& target, const TextAttribute& attributes);

VtIo();

[[nodiscard]] HRESULT Initialize(const ConsoleArguments* const pArgs);
Expand Down
7 changes: 7 additions & 0 deletions src/host/_stream.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -255,6 +255,13 @@ void WriteCharsLegacy(SCREEN_INFORMATION& screenInfo, const std::wstring_view& t
}
}

// This is the main entrypoint for conhost to write VT to the buffer.
// This wrapper around StateMachine exists so that we can add the necessary ConPTY transformations.
void WriteCharsVT(SCREEN_INFORMATION& screenInfo, const std::wstring_view& str)
{
screenInfo.GetStateMachine().ProcessString(str);
}

// Routine Description:
// - Takes the given text and inserts it into the given screen buffer.
// Note:
Expand Down
1 change: 1 addition & 0 deletions src/host/_stream.h
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ Revision History:
#include "writeData.hpp"

void WriteCharsLegacy(SCREEN_INFORMATION& screenInfo, const std::wstring_view& str, til::CoordType* psScrollY);
void WriteCharsVT(SCREEN_INFORMATION& screenInfo, const std::wstring_view& str);

// NOTE: console lock must be held when calling this routine
// String has been translated to unicode at this point.
Expand Down
Loading

0 comments on commit ac5b4f5

Please sign in to comment.