Topics

Braille routing keys, which API?

mohammad suliman
 

Hello all,
It has been a long time since I have posted here, so hope all of you are well, and doing well also!
I have a question regarding the braille routing keys. The question is as follows: Which API NVDA calls to set the cursor position when the user presses a routing key on a braille display on the web?
To clarify, here is what we are trying to achieve: we have an <input> element, and we want to track the cursor position when the user moves the cursor mainly by the braille display routing keys, and by arrows also, but this is not the issue here. So, we want to know when the user presses a routing key, and handle it appropriately in our system. We tried to listen to click and keypress events, but none of them is triggered as far as we have observed. So, in short, my question: how NVDA behaves when the user presses a routing key, while the focus is in an element with a caret, and this element is a web element.
Thanks in advance!
Mohammad

 

Dear Mohammad,


NVDA asks the browser to change the caret position, so you probably have to listen to caret or selection events if that's possible. Communication to the browser is performed with the accessibility API of use, UIA in Edge and IE and IAccessible2 in Firefox and Chrome.


Regards,

Leonard

Op 20-11-2019 om 18:48 schreef mohammad suliman:

Hello all,
It has been a long time since I have posted here, so hope all of you are well, and doing well also!
I have a question regarding the braille routing keys. The question is as follows: Which API NVDA calls to set the cursor position when the user presses a routing key on a braille display on the web?
To clarify, here is what we are trying to achieve: we have an <input> element, and we want to track the cursor position when the user moves the cursor mainly by the braille display routing keys, and by arrows also, but this is not the issue here. So, we want to know when the user presses a routing key, and handle it appropriately in our system. We tried to listen to click and keypress events, but none of them is triggered as far as we have observed. So, in short, my question: how NVDA behaves when the user presses a routing key, while the focus is in an element with a caret, and this element is a web element.
Thanks in advance!
Mohammad

mohammad suliman
 

Hi Leonard,
Thanks for that info brother. So, we tried to listen for the selection change event. In Google chrome, this event is fired when the user navigates the textarea with arrow keys, or with the braille display routing keys. This is brilliant, and, behaves as one expects from the browser in those situations. However, for our surprise, Firefox doesn't do this at all!! Which is very strange actually. The only event that Firefox fires is a mousedown event when you click the routing key where the cursor is found. I am guessing that NVDA triggers this event? Not sure though.
So, if you, or someone can share a suggestion how to overcome this, it will be much appreciated!
have a grate time!
Mohammad


On Thu, 21 Nov 2019 at 12:47, Leonard de Ruijter <alderuijter@...> wrote:

Dear Mohammad,


NVDA asks the browser to change the caret position, so you probably have to listen to caret or selection events if that's possible. Communication to the browser is performed with the accessibility API of use, UIA in Edge and IE and IAccessible2 in Firefox and Chrome.


Regards,

Leonard

Op 20-11-2019 om 18:48 schreef mohammad suliman:
Hello all,
It has been a long time since I have posted here, so hope all of you are well, and doing well also!
I have a question regarding the braille routing keys. The question is as follows: Which API NVDA calls to set the cursor position when the user presses a routing key on a braille display on the web?
To clarify, here is what we are trying to achieve: we have an <input> element, and we want to track the cursor position when the user moves the cursor mainly by the braille display routing keys, and by arrows also, but this is not the issue here. So, we want to know when the user presses a routing key, and handle it appropriately in our system. We tried to listen to click and keypress events, but none of them is triggered as far as we have observed. So, in short, my question: how NVDA behaves when the user presses a routing key, while the focus is in an element with a caret, and this element is a web element.
Thanks in advance!
Mohammad

 

Hey Mohammad,


This is interesting. Relevant code to look at in NVDA is NVDAObjects.IAccessible.IA2TextTextInfo._setCaretOffset and _setSelectionOffsets, I believe.

I assume the selection event isn't fired either when changing the caret position with arrow keys?


Regards,

Leonard

Op 21-11-2019 om 20:28 schreef mohammad suliman:

Hi Leonard,
Thanks for that info brother. So, we tried to listen for the selection change event. In Google chrome, this event is fired when the user navigates the textarea with arrow keys, or with the braille display routing keys. This is brilliant, and, behaves as one expects from the browser in those situations. However, for our surprise, Firefox doesn't do this at all!! Which is very strange actually. The only event that Firefox fires is a mousedown event when you click the routing key where the cursor is found. I am guessing that NVDA triggers this event? Not sure though.
So, if you, or someone can share a suggestion how to overcome this, it will be much appreciated!
have a grate time!
Mohammad

On Thu, 21 Nov 2019 at 12:47, Leonard de Ruijter <alderuijter@...> wrote:

Dear Mohammad,


NVDA asks the browser to change the caret position, so you probably have to listen to caret or selection events if that's possible. Communication to the browser is performed with the accessibility API of use, UIA in Edge and IE and IAccessible2 in Firefox and Chrome.


Regards,

Leonard

Op 20-11-2019 om 18:48 schreef mohammad suliman:
Hello all,
It has been a long time since I have posted here, so hope all of you are well, and doing well also!
I have a question regarding the braille routing keys. The question is as follows: Which API NVDA calls to set the cursor position when the user presses a routing key on a braille display on the web?
To clarify, here is what we are trying to achieve: we have an <input> element, and we want to track the cursor position when the user moves the cursor mainly by the braille display routing keys, and by arrows also, but this is not the issue here. So, we want to know when the user presses a routing key, and handle it appropriately in our system. We tried to listen to click and keypress events, but none of them is triggered as far as we have observed. So, in short, my question: how NVDA behaves when the user presses a routing key, while the focus is in an element with a caret, and this element is a web element.
Thanks in advance!
Mohammad

mohammad suliman
 

Hello Leonard,
Thanks for this valuable info bro!! According to our observation with Firefox, no selection event is fired when the user changes the cursor position neither with arrow keys, nor with braille routing keys.
I hope that we are mistaking with this regard, and I hope to hear from someone from the browser side of things. Is Jamie still subscribed to this list, and can offer us his insight on this? This will be much appreciated!!
thanks and have a nice weekend!!
Mohammad

On Fri, 22 Nov 2019 at 08:49, Leonard de Ruijter <alderuijter@...> wrote:

Hey Mohammad,


This is interesting. Relevant code to look at in NVDA is NVDAObjects.IAccessible.IA2TextTextInfo._setCaretOffset and _setSelectionOffsets, I believe.

I assume the selection event isn't fired either when changing the caret position with arrow keys?


Regards,

Leonard

Op 21-11-2019 om 20:28 schreef mohammad suliman:
Hi Leonard,
Thanks for that info brother. So, we tried to listen for the selection change event. In Google chrome, this event is fired when the user navigates the textarea with arrow keys, or with the braille display routing keys. This is brilliant, and, behaves as one expects from the browser in those situations. However, for our surprise, Firefox doesn't do this at all!! Which is very strange actually. The only event that Firefox fires is a mousedown event when you click the routing key where the cursor is found. I am guessing that NVDA triggers this event? Not sure though.
So, if you, or someone can share a suggestion how to overcome this, it will be much appreciated!
have a grate time!
Mohammad

On Thu, 21 Nov 2019 at 12:47, Leonard de Ruijter <alderuijter@...> wrote:

Dear Mohammad,


NVDA asks the browser to change the caret position, so you probably have to listen to caret or selection events if that's possible. Communication to the browser is performed with the accessibility API of use, UIA in Edge and IE and IAccessible2 in Firefox and Chrome.


Regards,

Leonard

Op 20-11-2019 om 18:48 schreef mohammad suliman:
Hello all,
It has been a long time since I have posted here, so hope all of you are well, and doing well also!
I have a question regarding the braille routing keys. The question is as follows: Which API NVDA calls to set the cursor position when the user presses a routing key on a braille display on the web?
To clarify, here is what we are trying to achieve: we have an <input> element, and we want to track the cursor position when the user moves the cursor mainly by the braille display routing keys, and by arrows also, but this is not the issue here. So, we want to know when the user presses a routing key, and handle it appropriately in our system. We tried to listen to click and keypress events, but none of them is triggered as far as we have observed. So, in short, my question: how NVDA behaves when the user presses a routing key, while the focus is in an element with a caret, and this element is a web element.
Thanks in advance!
Mohammad

derek riemer
 

Does it fire event_caret?
Also, take a look at script_routing, and the gestures assigned to it. The gestures system creates a map dynamically by consulting a lot of different subsystems in NVDA. The gesture map handles the mapping from a routing key press to a script. when a user presses a routing key, NVDA looks up any scripts associated with that key, discovers the key in the globalCommands system, finds a script, and fires the relevant script, which does the magic. I think that script is in source/globalCommands.py and just fires braille.handler.routeTo with the correct braille cell. (gesture.routingIndex)
After a complicated set of abstractions, NVDA finally calls routeTo on the braille region associated with that key (The internals of the braille system are being glossed over because I don't want to explain all of the components for you when it isn't relevant to your question). For browsers, you probably are dealing with a TextInfoRegion or CursorManagerRegion. Can you type brlRegions in the NVDA python console after snapshotting the console on the element you're interested in inspecting? This will show you the regions. I don't remember exactly how regions work, or in which order the regions appear, but consulting the code for braille.Handler will hopefully help.

On Fri, Nov 22, 2019 at 11:33 PM mohammad suliman <Mohmad.s93@...> wrote:
Hello Leonard,
Thanks for this valuable info bro!! According to our observation with Firefox, no selection event is fired when the user changes the cursor position neither with arrow keys, nor with braille routing keys.
I hope that we are mistaking with this regard, and I hope to hear from someone from the browser side of things. Is Jamie still subscribed to this list, and can offer us his insight on this? This will be much appreciated!!
thanks and have a nice weekend!!
Mohammad

On Fri, 22 Nov 2019 at 08:49, Leonard de Ruijter <alderuijter@...> wrote:

Hey Mohammad,


This is interesting. Relevant code to look at in NVDA is NVDAObjects.IAccessible.IA2TextTextInfo._setCaretOffset and _setSelectionOffsets, I believe.

I assume the selection event isn't fired either when changing the caret position with arrow keys?


Regards,

Leonard

Op 21-11-2019 om 20:28 schreef mohammad suliman:
Hi Leonard,
Thanks for that info brother. So, we tried to listen for the selection change event. In Google chrome, this event is fired when the user navigates the textarea with arrow keys, or with the braille display routing keys. This is brilliant, and, behaves as one expects from the browser in those situations. However, for our surprise, Firefox doesn't do this at all!! Which is very strange actually. The only event that Firefox fires is a mousedown event when you click the routing key where the cursor is found. I am guessing that NVDA triggers this event? Not sure though.
So, if you, or someone can share a suggestion how to overcome this, it will be much appreciated!
have a grate time!
Mohammad

On Thu, 21 Nov 2019 at 12:47, Leonard de Ruijter <alderuijter@...> wrote:

Dear Mohammad,


NVDA asks the browser to change the caret position, so you probably have to listen to caret or selection events if that's possible. Communication to the browser is performed with the accessibility API of use, UIA in Edge and IE and IAccessible2 in Firefox and Chrome.


Regards,

Leonard

Op 20-11-2019 om 18:48 schreef mohammad suliman:
Hello all,
It has been a long time since I have posted here, so hope all of you are well, and doing well also!
I have a question regarding the braille routing keys. The question is as follows: Which API NVDA calls to set the cursor position when the user presses a routing key on a braille display on the web?
To clarify, here is what we are trying to achieve: we have an <input> element, and we want to track the cursor position when the user moves the cursor mainly by the braille display routing keys, and by arrows also, but this is not the issue here. So, we want to know when the user presses a routing key, and handle it appropriately in our system. We tried to listen to click and keypress events, but none of them is triggered as far as we have observed. So, in short, my question: how NVDA behaves when the user presses a routing key, while the focus is in an element with a caret, and this element is a web element.
Thanks in advance!
Mohammad



--
Derek Riemer
Improving the world one byte at a time!        ⠠⠊⠍⠏⠗⠕⠧⠬ ⠮ ⠸⠺ ⠐⠕ ⠃⠽⠞⠑ ⠁⠞ ⠁ ⠐⠞⠖
•    Accessibility enthusiast.
•    Proud user of the NVDA screen reader.
•    Open source enthusiast.
•    Skier.

•    Personal website: https://derekriemer.com