Rendering by frame diffs rather than complete redraws

[tombh]

Firstly I really like this project, I've been using it as a monitor dashboard in a production app and it's helped surface some significant bugs in my own code, so thank you.

One minor problem I have is flickering in tmux. I'm using rich.live and have tried various refresh rates and sleeps, etc. There's zero flickering outside of tmux (in a plain Alacritty terminal) so I'm assuming that tmux is just not as efficient at rendering. From a cursory look at rich's code it seems that Live (or would it be Layout's responsibility?) redraws the entire frame/screen, rather than the perhaps more convoluted method of just updating what's changed.

In theory, rendering changes between 2 TTY states could be achieved by a standalone program, eg; ./my_rich_app.py | tty_updater. I would actually be interested in writing that program as it would be useful for Browsh (a terminal-based web browser) I maintain. Oh I just realised, that's not so straight forward because you'd have to detect both the moment of screen clearing and the end of the frame rendering, so it would only work in the narrow case of apps that redraw the entire TTY, but then that's also the only use case you'd want diffed rendering 🤔

Or maybe I'm just mistaken about the source of the flickering. Or maybe it wouldn't be too hard to implement in rich, if so I might be able to PR it.

[willmcgugan]

Hi @tombh

Glad you like Rich.

I have had a report that here is flickering in tmux, but I haven't looked in to it. Live does write the whole screen each time. It would be possible to calculate the deltas, but that's probably easier done at the Segment level in Rich (an intermediate data structure before it generates ansi codes).

I'm working on another project which is more aimed at dynamic interfaces, which is capable of only updating changed portions of the screen. But that's not ready yet.

Browsh looks awesome, BTW. Going to have a play with that.

[tombh]

Thanks about Browsh 🤓

I've actually found the flickering problem seems to have more to do with my terminal, Alacritty, as I've found that Kitty has no problem rendering my rich.live dashboard in tmux. Great news about the textual project, it'll have/has good input support as well I suppose, so I'll definitely want to use that.

[kovidgoyal]

No clue about tmux, I dont touch terminal multiplexers with a ten foot bargepole. If you want to see it in action do the following in kitty

printf 'hello, \033P=1s\033\\world!'; sleep 0.5s; printf '\033P=2s\033\\'

You will see hello printed first and world printed after half a second.

[kovidgoyal]

Oh and according to its man page tmux supports it, search for sync in https://man7.org/linux/man-pages/man1/tmux.1.html

[tombh]

OMG I fixed it!!

def _check_buffer(self) -> None:
    """Check if the buffer may be rendered."""
    if self.quiet:
        del self._buffer[:]
        return
    with self._lock:
        if self._buffer_index == 0:
            if self.is_jupyter:  # pragma: no cover
                from .jupyter import display

                display(self._buffer, self._render_buffer(self._buffer[:]))
                del self._buffer[:]
            else:
                text = self._render_buffer(self._buffer[:])
                del self._buffer[:]
                if text:
                    try:
                        if WINDOWS:  # pragma: no cover
                            # https://bugs.python.org/issue37871
                            write = self.file.write
                            for line in text.splitlines(True):
                                write(line)
                        else:
                            if os.getenv("TMUX") is not None:

                                def tmux_passthrough(message: str):
                                    return f"\033Ptmux;\033{message}\033\\"

                                def sequence(command: str):
                                    return f"\033{command}\033\\"

                                bsu = tmux_passthrough(sequence("P=1s"))
                                esu = tmux_passthrough(sequence("P=2s"))
                                text = f"{bsu}{text}{esu}"
                            self.file.write(text)
                        self.file.flush()
                    except UnicodeEncodeError as error:
                        error.reason = f"{error.reason}\n*** You may need to add PYTHONIOENCODING=utf-8 to your environment ***"
                        raise

The key was to just completely ignore tmux by using extra escape sequences to "smuggle" the sync update sequences through tmux and directly to the underlying terminal emulator.

If this seems like a reasonable addition I can submit it as a PR.

[willmcgugan]

PR would be great 👍

This synchronisation code is a new one on me. I've never seen a great deal of flickering probably because Rich outputs a screenful of text in a single write. But there can be some flicker, so I'm keen to give it a try.

[kovidgoyal]

Even if you do only a single write per call, there is no guarantee that the data will be read() in a single call by the terminal emulator. In fact, if it is over a network connection, and the amount of data is larger than a single packet it wont be. Some terminal emulators, such as kitty, have a so called input delay so that input received from the client is batched up even if it comes in multiple packets. That is why you wont see flicker in such terminals. Others like alacritty just write it as it is received with no batching, for some misguided belief in performance.

It was precisely to address this that egmont, george and I created the synchronize mechanism. And on an implementation note, as I said tmux actually does support it so you shouldn't use the passthrough (am guess @tombh is testing with an older version of tmux that doesnt have the support). Just always emit the code, it will be ignored by things that dont support it which is harmless. There is one caveat to be aware of, the Apple terminal doesn't parse DCS escape codes correctly, so on that terminal you will get garbage printed to the screen when using this escape code. This is in violation of the ECMA 48 spec, so hopefully Apple will fix it, but then this Apple so who knows.