Orca can't be stopped in native dialogs

[alex19EP]

Issue Type: Bug

open any native dialog about or check for updates.
press ctrl to stop speech.
orca should stop talking.

this issue similar to #93056

VS Code version: Code - Insiders 1.56.0-insider (b4816cf, 2021-04-20T07:24:33.853Z)
OS version: Linux x64 5.11.15-arch1-2

System Info

Item	Value
CPUs	Intel(R) Celeron(R) CPU 1005M @ 1.90GHz (2 x 1900)
GPU Status	2d_canvas: enabled gpu_compositing: enabled multiple_raster_threads: disabled_off oop_rasterization: disabled_off opengl: enabled_on rasterization: disabled_software skia_renderer: enabled_on video_decode: disabled_software vulkan: disabled_off webgl: enabled webgl2: enabled
Load (avg)	2, 1, 1
Memory (System)	3.70GB (0.64GB free)
Process Argv	--no-sandbox --crash-reporter-id 9fe453d7-430e-46b5-bfd9-60af52319e71
Screen Reader	yes
VM	0%
DESKTOP_SESSION	mate
XDG_CURRENT_DESKTOP	MATE
XDG_SESSION_DESKTOP	mate
XDG_SESSION_TYPE	x11

Extensions (40)

Extension	Author (truncated)	Version
meson	asa	1.3.0
github-markdown-preview	bie	0.0.2
markdown-checkbox	bie	0.1.3
markdown-emoji	bie	0.1.0
markdown-preview-github-styles	bie	0.2.0
markdown-yaml-preamble	bie	0.0.4
vscode-astyle	chi	0.9.0
systemd-unit-file	coo	1.0.6
vscode-github-actions	csc	0.17.0
vscode-markdownlint	Dav	0.40.3
githistory	don	0.6.16
gitlens	eam	11.4.1
EditorConfig	Edi	0.16.4
vscode-pull-request-github-insiders	Git	2021.4.29343
markdown-extended	jeb	1.0.19
better-cpp-syntax	jef	1.15.5
restructuredtext	lex	148.0.0
kconfig	luv	1.0.3
vscode-docker	ms-	1.12.1
vscode-language-pack-ru	MS-	1.55.2
python	ms-	2021.5.746022950-dev
vscode-pylance	ms-	2021.4.1
jupyter	ms-	2021.6.751425733
remote-containers	ms-	0.169.0
remote-ssh	ms-	0.65.4
remote-ssh-edit	ms-	0.65.4
remote-wsl	ms-	0.56.0
cmake-tools	ms-	1.7.0
cpptools	ms-	1.3.1
cpptools-extension-pack	ms-	1.0.0
vsliveshare	ms-	1.0.4131
vsliveshare-audio	ms-	0.1.91
vsliveshare-pack	ms-	0.4.0
vscode-yaml	red	0.18.0
platform-settings	run	0.0.2
rewrap	stk	1.14.0
code-spell-checker	str	1.10.2
code-spell-checker-russian	str	0.2.9
cmake	twx	0.0.17
grammarly	znc	0.14.0

A/B Experiments

vsliv695:30137379
vsins829:30139715
vsliv368:30146709
vsreu685:30147344
python383cf:30185419
pythonvspyt602:30291494
vspor879:30202332
vspor708:30202333
vspor363:30204092
pythonvsdeb440:30224570
pythonvsded773:30223139
pythonvspyt875:30259475
pythonvspyt639:30291487
pythontb:30258533
vspre833cf:30267465
pythonptprofiler:30281269
vsdfh931cf:30280410
vscnewfiletextcf:30288478
vshan820:30276952
pythondataviewer:30285072
vscus158:30286553
vscgsv2:30286872

[alex19EP]

ping @isidorn @joanmarie

[joanmarie]

I can take a look later, but if Orca doesn't interrupt speech and Orca commands fail to work then that means Orca is not being notified of input events.

[isidorn]

@alex19EP was this working before?
Asking because we just updated to Electron12 and that might have broken this

[jvesouza]

@isidorn The problem manifested itself after upgrading to electron version 12.04.
Another side effect is that I am now unable to use the orca's flat review mode to explore the dialogue.

[joanmarie]

If flat review mode doesn't work at all (as opposed to presenting the wrong object), that's the same problem: Orca doesn't know you pressed any key (e.g. the key to review the current word/line/char/object/etc.).

[isidorn]

Also assigning to @deepak1556 for the electron update

[webczat]

looks to me like chrome regression as this probably also doesn't work in newest unstable. Was like that for some time, but because I haven't updated for few weeks, not sure.

[jvesouza]

Any news regarding this issue? It is still present in the version that uses electron 12.0.12.

Thanks.

[webczat]

but, newer chrome-dev doesn't seem to have it, so I think it's gonna be fixed soon.

[isidorn]

I tried to search for this issue in the Chromium repository but I failed.
@joanmarie might you maybe know when and how Chrome fixed this? Asking because if it is a simple fix we could backport it to our Electron version.

[jvesouza]

Except for errors on my part, this problem is still present even after updating to electron 16.

[isidorn]

@webczat you mentioned that newer chrome versions have this fixed? Can you confirm that in the latest stable chrome this is working?

[webczat]

hmm that is kinda weird because this bug seems to disappear and reappear. my current test after update few days ago is:
seems fixed in 98.0.4758.102
however, seems not fixed in 100.0.4892.0

[isidorn]

@webczat would you be so kind to file an issue https://bugs.chromium.org/p/chromium/issues/list
And then we can start talking to Googlers and @joanmarie so they investigate.

[jvesouza]

Without wanting to be annoying, the problem still exists for me in the vSCode version that uses electron 17.1.1.

[joanmarie]

The bug (fixed in 2020) is https://bugs.chromium.org/p/chromium/issues/detail?id=1042864 which landed in 86.0.4199.0.

Regarding what @webczat stated about it became unfixed, newer versions of Chrome/Chromium for some reason do not autofocus the open/save-as dialogs. This is true for all users rather than an accessibility thing. I have to press Alt+F6 or click on it before I can interact with it. But once that dialog is focused, Orca's commands work for me.

[jvesouza]

Just in order to help, I'm posting here the steps I use to reproduce the problem.

1. Make sure that orca is running.
2. Launch VSCode.
3. Press alt+h and locate the about menu using the up arow.
4. Press the enter key.
5. When the orca starts to announce the information contained in the about dialog, try to silence it by pressing the ctrl key.

The orca is not silenced even if I press other keys. I need to wait for it to announce all the text so I can go back to interacting with the keyboard.

[joanmarie]

@jvesouza: Thanks! I'm afraid the TL;DR version is "Yup. I know. Did the Chromium bug fix I mentioned make it into Electron yet?"

Here's the long version:

I can reproduce it, just as you described. I'm afraid it's still https://bugs.chromium.org/p/chromium/issues/detail?id=1042864. The title/summary of that bug is "Key events are not emitted for native dialogs (Open, Save As, VSCode's About)".

Here are some Orca-free steps to reproduce the problem described by @jvesouza:

1. Download key-events.py, the accessible event listener attached to the Chromium bug I mentioned above.
2. Launch that listener in a terminal
3. Launch VSCode
4. Press Alt+h and locate the about menu using Up Arrow.
5. Press enter/return
6. In the resulting dialog, press Tab a bunch of times to switch focus between the buttons
7. Press Escape to close the dialog

Expected results: All key presses would show up in the terminal
Actual results: As soon as the dialog has focus, no key presses show up in the terminal. Here's what I see:

Alt_L pressed
h pressed
Up pressed
Return pressed

That's it. No Tab (which I pressed a ton) and no Escape. Nothing gets logged until I exit that "About" dialog.

In other words, Orca has no idea that you pressed any keys as soon as you are in the About dialog. Ditto for Open and Save As.

Why that impacts this VSCode bug: Orca shuts up speech in response to your pressing keys. If it doesn't know you pressed a key, it cannot do the shutting up. See my earlier comment.

Similarly, Orca commands work by being notified of key presses and then determining a given key press matches an Orca command. For instance, if Orca knows you pressed KeyPad 8, it reviews the current line. If the app doesn't tell Orca you pressed KeyPad 8, crickets. See this comment.

Having said all that, the fix for the Chromium issue can be found on the bug itself. Here is a direct link to the commit. Again, it landed in 2020 and Orca commands work for me in native dialogs in Chrome/Chromium. So either the fix is not yet included in the versions of Electron being used, or we have an unaddressed variant of the same bug.

Make sense?

[isidorn]

@joanmarie latest VS Code Insiders uses Electron 17 that uses Chrome 98 so that fix is in.
The latest VS Code stable uses Electron 13 which uses Chrome 91 so that fix should also be in. fyi next stable will also jump to Electron 17.

[jvesouza]

Interestingly, a similar issue was reported at #93056 and was fixed in electron version 11.

[joanmarie]

That is interesting. I wonder what the difference is....

In the meantime I filed https://bugs.chromium.org/p/chromium/issues/detail?id=1306015.

[isidorn]

@joanmarie thanks a lot for filling this issue. If it does not get traction I can also try to ping Google people.

[jvesouza]

Once again, forgive me for being annoying, but now that VSCode is using Electron 22, is there any possibility of fixing this bug?

[isidorn]

@jvesouza thanks for the ping. I know @deepak1556 has this on the plan and plans to investigate more this or next milestone. I expect us to have more details soon. Thank you for your understanding.

[jvesouza]

I don't have an explanation but it looks like this problem is gone. from the moment I started using the 45-alpha version of orca.
To confirm I downgraded to version 44.1 of orca and the problem returned.
Could anyone else confirm?

[joanmarie]

@jvesouza: That would make sense. Due to issues related to Gtk4, Wayland, etc., we can no longer count on getting input events. 😞

We use (or rather used to use) input events to interrupt speech. Now, sadly, we have to do this heuristically. It sounds like we're getting it right for VS Code with native dialogs. Yay!

You may find other instances where speech gets incorrectly cut off, which should be reported on the Orca list and/or GitLab.

@isidorn et al.: With respect to this issue here, please note: I do NOT intend to backport the heuristics into earlier versions of Orca because I expect there will be regressions (plural). Heuristics are hard. Thus having the input events from the native dialogs used by VS Code will still be a requirement for Orca v44.x and earlier. Please and thank you. 😄

[isidorn]

@jveszoua thanks for letting us know.
@joanmarie makes sense :) Thanks a lot for fixing this in newer versions of Orca 👏

@deepak1556 not sure if there is an easy fix on our side? If not, I am fine that we post-pone this issue since it is fixed with latest Orca.

[jvesouza]

@jvesouza: That would make sense. Due to issues related to Gtk4, Wayland, etc., we can no longer count on getting input events. disappointed

We use (or rather used to use) input events to interrupt speech. Now, sadly, we have to do this heuristically. It sounds like we're getting it right for VS Code with native dialogs. Yay!

I ended up not mentioning it in my previous comment, but the flat review mode is also working in this case.

[joanmarie]

@isidorn: You forgot the fingerquotes around "fixing". 😛 (The history of accessible input events in Linux is a long and contentious one.)

Also, you obviously should do what makes the most sense for your project. However, please be aware that Orca v45 (not released) requires the very latest version of AT-SPI2. As a result, Orca users not running the very latest version of their distro -- including those running LTS versions -- might not see this Orca "fix" for a while (years in the LTS case).

[isidorn]

@joanmarie thanks for clarifying 😄
Let's see what @deepak1556 says.

[tristan957]

@isidorn we essentially need 1 of the following 2 things for a fix.

Implement a libportal-based file chooser dialog in Electron or inherit the file chooser dialogs from Chromium. I brought this up in the linked Electron issue somewhere above, but there has been no movement from their side.

The other option is to just revert the offending Electron commits.

[jvesouza]

I don't know what changed, but the fact is that this problem also disappeared when I use version 44 of orca.
For me the change seems to have been in vscode because if I use version 1.80 of vscode together with orca 44 the problem returns, but if I use the insiders version of vscode together with orca 44 the problem does not happen.

[isidorn]

I will optimistically go ahead and close this issue. Since we have updated electron multiple times.
If this issue still persists please let me know and I will reopen it. Thank you