Topics

UIAutomation and the EditableTextWithAutoSelectDetection class

Ian Reed
 

I'm writing an edit control that uses UIAutomation to communicate with NVDA.
It exposes the text pattern, and that seems to work fine to get NVDA to read and braille the current char, word, or line, and to navigate by char, word and line.

I fire UIA text changed and text selection changed events at appropriate times, but NVDA seems to ignore these.
I can tell because when I insert a character into the edit box, NVDA does not update the braille on my display, nor does it read the current word correctly when I use the read word at review cursor command (Numpad 5 on desktop layout).
If I use an arrow key the text on the braille display will update immediately, but not when I type characters.
When looking at the NVDA log viewer, I can see my edit box is classified as an EditableTextWithoutAutoSelectDetection object.
Looking at \NVDAObjects/behaviors.py I can see that I want my edit box to be classified as an EditableTextWithAutoSelectDetection object instead so my text changed events will not be ignored.

Can anyone tell me how to get my edit box classified as WithAutoSelection?

Thanks,
Ian Reed

 

Hi,
In order to change this, NVDA must listen to text change event, which is only available in alpha builds at the moment.
Cheers,
Joseph

-----Original Message-----
From: nvda-devel@groups.io <nvda-devel@groups.io> On Behalf Of Ian Reed
Sent: Sunday, June 9, 2019 7:05 PM
To: nvda-devel@groups.io
Subject: [nvda-devel] UIAutomation and the EditableTextWithAutoSelectDetection class

I'm writing an edit control that uses UIAutomation to communicate with NVDA.
It exposes the text pattern, and that seems to work fine to get NVDA to read and braille the current char, word, or line, and to navigate by char, word and line.

I fire UIA text changed and text selection changed events at appropriate times, but NVDA seems to ignore these.
I can tell because when I insert a character into the edit box, NVDA does not update the braille on my display, nor does it read the current word correctly when I use the read word at review cursor command (Numpad
5 on desktop layout).
If I use an arrow key the text on the braille display will update immediately, but not when I type characters.
When looking at the NVDA log viewer, I can see my edit box is classified as an EditableTextWithoutAutoSelectDetection object.
Looking at \NVDAObjects/behaviors.py I can see that I want my edit box to be classified as an EditableTextWithAutoSelectDetection object instead so my text changed events will not be ignored.

Can anyone tell me how to get my edit box classified as WithAutoSelection?

Thanks,
Ian Reed

Ian Reed
 

Thanks for the quick response.
Is the alpha support for the text change event likely to make it to a public release at some point?
In the meantime, can you give me any guidance on making an app module that forces NVDA to listen to text change events for my specific edit box, or for each specific app written with it?

Thanks.

On 6/9/2019 8:12 PM, Joseph Lee wrote:
Hi,
In order to change this, NVDA must listen to text change event, which is only available in alpha builds at the moment.
Cheers,
Joseph

-----Original Message-----
From: nvda-devel@groups.io <nvda-devel@groups.io> On Behalf Of Ian Reed
Sent: Sunday, June 9, 2019 7:05 PM
To: nvda-devel@groups.io
Subject: [nvda-devel] UIAutomation and the EditableTextWithAutoSelectDetection class

I'm writing an edit control that uses UIAutomation to communicate with NVDA.
It exposes the text pattern, and that seems to work fine to get NVDA to read and braille the current char, word, or line, and to navigate by char, word and line.

I fire UIA text changed and text selection changed events at appropriate times, but NVDA seems to ignore these.
I can tell because when I insert a character into the edit box, NVDA does not update the braille on my display, nor does it read the current word correctly when I use the read word at review cursor command (Numpad
5 on desktop layout).
If I use an arrow key the text on the braille display will update immediately, but not when I type characters.
When looking at the NVDA log viewer, I can see my edit box is classified as an EditableTextWithoutAutoSelectDetection object.
Looking at \NVDAObjects/behaviors.py I can see that I want my edit box to be classified as an EditableTextWithAutoSelectDetection object instead so my text changed events will not be ignored.

Can anyone tell me how to get my edit box classified as WithAutoSelection?

Thanks,
Ian Reed





 

Hi,
Alpha to public; that depends on how the text change event tracking will be in the future.
As for temporarily enabling text change event: possibly, which will require telling UIA handler that you wish to let an event handler handle text change event (this will work properly in recent Windows 10 releases, not on 8.x from what I've been told). Note that this solution is based on the source code in front of me - both NVDA master branch and my own Windows 10 App Essentials add-on, both of which can listen to text change events and perform appropriate action (in case of my add-on, to log things about this event).
Cheers,
Joseph

-----Original Message-----
From: nvda-devel@groups.io <nvda-devel@groups.io> On Behalf Of Ian Reed
Sent: Sunday, June 9, 2019 9:49 PM
To: nvda-devel@groups.io
Subject: Re: [nvda-devel] UIAutomation and the EditableTextWithAutoSelectDetection class

Thanks for the quick response.
Is the alpha support for the text change event likely to make it to a public release at some point?
In the meantime, can you give me any guidance on making an app module that forces NVDA to listen to text change events for my specific edit box, or for each specific app written with it?

Thanks.

On 6/9/2019 8:12 PM, Joseph Lee wrote:
Hi,
In order to change this, NVDA must listen to text change event, which is only available in alpha builds at the moment.
Cheers,
Joseph

-----Original Message-----
From: nvda-devel@groups.io <nvda-devel@groups.io> On Behalf Of Ian
Reed
Sent: Sunday, June 9, 2019 7:05 PM
To: nvda-devel@groups.io
Subject: [nvda-devel] UIAutomation and the
EditableTextWithAutoSelectDetection class

I'm writing an edit control that uses UIAutomation to communicate with NVDA.
It exposes the text pattern, and that seems to work fine to get NVDA to read and braille the current char, word, or line, and to navigate by char, word and line.

I fire UIA text changed and text selection changed events at appropriate times, but NVDA seems to ignore these.
I can tell because when I insert a character into the edit box, NVDA
does not update the braille on my display, nor does it read the
current word correctly when I use the read word at review cursor
command (Numpad
5 on desktop layout).
If I use an arrow key the text on the braille display will update immediately, but not when I type characters.
When looking at the NVDA log viewer, I can see my edit box is classified as an EditableTextWithoutAutoSelectDetection object.
Looking at \NVDAObjects/behaviors.py I can see that I want my edit box to be classified as an EditableTextWithAutoSelectDetection object instead so my text changed events will not be ignored.

Can anyone tell me how to get my edit box classified as WithAutoSelection?

Thanks,
Ian Reed






Ian Reed
 

Thanks.
Unfortunately I'm currently still on Windows 7.

Perhaps UIAutomation was the wrong choice. Could I get better integration with NVDA if I switched to MSAA/IAccessible?
I can see that the Scintilla notepad++ edit box does not have this problem, nor does plain old notepad.
How can I duplicate that behavior?

On 6/9/2019 10:55 PM, Joseph Lee wrote:
Hi,
Alpha to public; that depends on how the text change event tracking will be in the future.
As for temporarily enabling text change event: possibly, which will require telling UIA handler that you wish to let an event handler handle text change event (this will work properly in recent Windows 10 releases, not on 8.x from what I've been told). Note that this solution is based on the source code in front of me - both NVDA master branch and my own Windows 10 App Essentials add-on, both of which can listen to text change events and perform appropriate action (in case of my add-on, to log things about this event).
Cheers,
Joseph

-----Original Message-----
From: nvda-devel@groups.io <nvda-devel@groups.io> On Behalf Of Ian Reed
Sent: Sunday, June 9, 2019 9:49 PM
To: nvda-devel@groups.io
Subject: Re: [nvda-devel] UIAutomation and the EditableTextWithAutoSelectDetection class

Thanks for the quick response.
Is the alpha support for the text change event likely to make it to a public release at some point?
In the meantime, can you give me any guidance on making an app module that forces NVDA to listen to text change events for my specific edit box, or for each specific app written with it?

Thanks.

On 6/9/2019 8:12 PM, Joseph Lee wrote:
Hi,
In order to change this, NVDA must listen to text change event, which is only available in alpha builds at the moment.
Cheers,
Joseph

-----Original Message-----
From: nvda-devel@groups.io <nvda-devel@groups.io> On Behalf Of Ian
Reed
Sent: Sunday, June 9, 2019 7:05 PM
To: nvda-devel@groups.io
Subject: [nvda-devel] UIAutomation and the
EditableTextWithAutoSelectDetection class

I'm writing an edit control that uses UIAutomation to communicate with NVDA.
It exposes the text pattern, and that seems to work fine to get NVDA to read and braille the current char, word, or line, and to navigate by char, word and line.

I fire UIA text changed and text selection changed events at appropriate times, but NVDA seems to ignore these.
I can tell because when I insert a character into the edit box, NVDA
does not update the braille on my display, nor does it read the
current word correctly when I use the read word at review cursor
command (Numpad
5 on desktop layout).
If I use an arrow key the text on the braille display will update immediately, but not when I type characters.
When looking at the NVDA log viewer, I can see my edit box is classified as an EditableTextWithoutAutoSelectDetection object.
Looking at \NVDAObjects/behaviors.py I can see that I want my edit box to be classified as an EditableTextWithAutoSelectDetection object instead so my text changed events will not be ignored.

Can anyone tell me how to get my edit box classified as WithAutoSelection?

Thanks,
Ian Reed









 

Hi,
I think it'd be best to use MSAA/IAccessible for now if you wish to target older Windows releases and/or older NVDA versions. UI Automation has interesting advantages, but it is known that Microsoft sometimes introduces oddities here and there - I myself have experienced it many times as a Windows Insider and as an NVDA code contributor (that's part of the reason why I keep Windows 10 App Essentials add-on around in order to provide timely fixes for UIA oddities in Windows 10 Insider Preview builds).
Cheers,
Joseph

-----Original Message-----
From: nvda-devel@groups.io <nvda-devel@groups.io> On Behalf Of Ian Reed
Sent: Sunday, June 9, 2019 10:03 PM
To: nvda-devel@groups.io
Subject: Re: [nvda-devel] UIAutomation and the EditableTextWithAutoSelectDetection class

Thanks.
Unfortunately I'm currently still on Windows 7.

Perhaps UIAutomation was the wrong choice. Could I get better integration with NVDA if I switched to MSAA/IAccessible?
I can see that the Scintilla notepad++ edit box does not have this problem, nor does plain old notepad.
How can I duplicate that behavior?


On 6/9/2019 10:55 PM, Joseph Lee wrote:
Hi,
Alpha to public; that depends on how the text change event tracking will be in the future.
As for temporarily enabling text change event: possibly, which will require telling UIA handler that you wish to let an event handler handle text change event (this will work properly in recent Windows 10 releases, not on 8.x from what I've been told). Note that this solution is based on the source code in front of me - both NVDA master branch and my own Windows 10 App Essentials add-on, both of which can listen to text change events and perform appropriate action (in case of my add-on, to log things about this event).
Cheers,
Joseph

-----Original Message-----
From: nvda-devel@groups.io <nvda-devel@groups.io> On Behalf Of Ian
Reed
Sent: Sunday, June 9, 2019 9:49 PM
To: nvda-devel@groups.io
Subject: Re: [nvda-devel] UIAutomation and the
EditableTextWithAutoSelectDetection class

Thanks for the quick response.
Is the alpha support for the text change event likely to make it to a public release at some point?
In the meantime, can you give me any guidance on making an app module that forces NVDA to listen to text change events for my specific edit box, or for each specific app written with it?

Thanks.

On 6/9/2019 8:12 PM, Joseph Lee wrote:
Hi,
In order to change this, NVDA must listen to text change event, which is only available in alpha builds at the moment.
Cheers,
Joseph

-----Original Message-----
From: nvda-devel@groups.io <nvda-devel@groups.io> On Behalf Of Ian
Reed
Sent: Sunday, June 9, 2019 7:05 PM
To: nvda-devel@groups.io
Subject: [nvda-devel] UIAutomation and the
EditableTextWithAutoSelectDetection class

I'm writing an edit control that uses UIAutomation to communicate with NVDA.
It exposes the text pattern, and that seems to work fine to get NVDA to read and braille the current char, word, or line, and to navigate by char, word and line.

I fire UIA text changed and text selection changed events at appropriate times, but NVDA seems to ignore these.
I can tell because when I insert a character into the edit box, NVDA
does not update the braille on my display, nor does it read the
current word correctly when I use the read word at review cursor
command (Numpad
5 on desktop layout).
If I use an arrow key the text on the braille display will update immediately, but not when I type characters.
When looking at the NVDA log viewer, I can see my edit box is classified as an EditableTextWithoutAutoSelectDetection object.
Looking at \NVDAObjects/behaviors.py I can see that I want my edit box to be classified as an EditableTextWithAutoSelectDetection object instead so my text changed events will not be ignored.

Can anyone tell me how to get my edit box classified as WithAutoSelection?

Thanks,
Ian Reed










Bill Dengler
 

textChange for UIA should land with UIA in consoles (2019.3/2019.4) for Win10 users (Maybe limited to later builds, but should be definitely available on at least 1809).

Bill

On Jun 10, 2019, at 00:55, Joseph Lee <@joslee> wrote:

Hi,
Alpha to public; that depends on how the text change event tracking will be in the future.
As for temporarily enabling text change event: possibly, which will require telling UIA handler that you wish to let an event handler handle text change event (this will work properly in recent Windows 10 releases, not on 8.x from what I've been told). Note that this solution is based on the source code in front of me - both NVDA master branch and my own Windows 10 App Essentials add-on, both of which can listen to text change events and perform appropriate action (in case of my add-on, to log things about this event).
Cheers,
Joseph

-----Original Message-----
From: nvda-devel@groups.io <nvda-devel@groups.io> On Behalf Of Ian Reed
Sent: Sunday, June 9, 2019 9:49 PM
To: nvda-devel@groups.io
Subject: Re: [nvda-devel] UIAutomation and the EditableTextWithAutoSelectDetection class

Thanks for the quick response.
Is the alpha support for the text change event likely to make it to a public release at some point?
In the meantime, can you give me any guidance on making an app module that forces NVDA to listen to text change events for my specific edit box, or for each specific app written with it?

Thanks.

On 6/9/2019 8:12 PM, Joseph Lee wrote:
Hi,
In order to change this, NVDA must listen to text change event, which is only available in alpha builds at the moment.
Cheers,
Joseph

-----Original Message-----
From: nvda-devel@groups.io <nvda-devel@groups.io> On Behalf Of Ian
Reed
Sent: Sunday, June 9, 2019 7:05 PM
To: nvda-devel@groups.io
Subject: [nvda-devel] UIAutomation and the
EditableTextWithAutoSelectDetection class

I'm writing an edit control that uses UIAutomation to communicate with NVDA.
It exposes the text pattern, and that seems to work fine to get NVDA to read and braille the current char, word, or line, and to navigate by char, word and line.

I fire UIA text changed and text selection changed events at appropriate times, but NVDA seems to ignore these.
I can tell because when I insert a character into the edit box, NVDA
does not update the braille on my display, nor does it read the
current word correctly when I use the read word at review cursor
command (Numpad
5 on desktop layout).
If I use an arrow key the text on the braille display will update immediately, but not when I type characters.
When looking at the NVDA log viewer, I can see my edit box is classified as an EditableTextWithoutAutoSelectDetection object.
Looking at \NVDAObjects/behaviors.py I can see that I want my edit box to be classified as an EditableTextWithAutoSelectDetection object instead so my text changed events will not be ignored.

Can anyone tell me how to get my edit box classified as WithAutoSelection?

Thanks,
Ian Reed











Ian Reed
 

That is too bad.

How about another approach.
Instead of trying to detect the text change events, is there a way I can write an app module that for my app, and my edit box, will ask UIA for the updated text info every time a key is pressed?
Since almost every key press in an edit box will trigger text changes (or at least caret movement), it seems reasonable to check for the updated text after each one.
In fact it seems NVDA already does this for every press of an arrow key. I just want it to work for all keys.

How can I detect any key down/up/press within an app module?

Thanks for continueing to help with my questions.

On 6/9/2019 11:11 PM, Joseph Lee wrote:
Hi,
I think it'd be best to use MSAA/IAccessible for now if you wish to target older Windows releases and/or older NVDA versions. UI Automation has interesting advantages, but it is known that Microsoft sometimes introduces oddities here and there - I myself have experienced it many times as a Windows Insider and as an NVDA code contributor (that's part of the reason why I keep Windows 10 App Essentials add-on around in order to provide timely fixes for UIA oddities in Windows 10 Insider Preview builds).
Cheers,
Joseph

-----Original Message-----
From: nvda-devel@groups.io <nvda-devel@groups.io> On Behalf Of Ian Reed
Sent: Sunday, June 9, 2019 10:03 PM
To: nvda-devel@groups.io
Subject: Re: [nvda-devel] UIAutomation and the EditableTextWithAutoSelectDetection class

Thanks.
Unfortunately I'm currently still on Windows 7.

Perhaps UIAutomation was the wrong choice. Could I get better integration with NVDA if I switched to MSAA/IAccessible?
I can see that the Scintilla notepad++ edit box does not have this problem, nor does plain old notepad.
How can I duplicate that behavior?


On 6/9/2019 10:55 PM, Joseph Lee wrote:
Hi,
Alpha to public; that depends on how the text change event tracking will be in the future.
As for temporarily enabling text change event: possibly, which will require telling UIA handler that you wish to let an event handler handle text change event (this will work properly in recent Windows 10 releases, not on 8.x from what I've been told). Note that this solution is based on the source code in front of me - both NVDA master branch and my own Windows 10 App Essentials add-on, both of which can listen to text change events and perform appropriate action (in case of my add-on, to log things about this event).
Cheers,
Joseph

-----Original Message-----
From: nvda-devel@groups.io <nvda-devel@groups.io> On Behalf Of Ian
Reed
Sent: Sunday, June 9, 2019 9:49 PM
To: nvda-devel@groups.io
Subject: Re: [nvda-devel] UIAutomation and the
EditableTextWithAutoSelectDetection class

Thanks for the quick response.
Is the alpha support for the text change event likely to make it to a public release at some point?
In the meantime, can you give me any guidance on making an app module that forces NVDA to listen to text change events for my specific edit box, or for each specific app written with it?

Thanks.

On 6/9/2019 8:12 PM, Joseph Lee wrote:
Hi,
In order to change this, NVDA must listen to text change event, which is only available in alpha builds at the moment.
Cheers,
Joseph

-----Original Message-----
From: nvda-devel@groups.io <nvda-devel@groups.io> On Behalf Of Ian
Reed
Sent: Sunday, June 9, 2019 7:05 PM
To: nvda-devel@groups.io
Subject: [nvda-devel] UIAutomation and the
EditableTextWithAutoSelectDetection class

I'm writing an edit control that uses UIAutomation to communicate with NVDA.
It exposes the text pattern, and that seems to work fine to get NVDA to read and braille the current char, word, or line, and to navigate by char, word and line.

I fire UIA text changed and text selection changed events at appropriate times, but NVDA seems to ignore these.
I can tell because when I insert a character into the edit box, NVDA
does not update the braille on my display, nor does it read the
current word correctly when I use the read word at review cursor
command (Numpad
5 on desktop layout).
If I use an arrow key the text on the braille display will update immediately, but not when I type characters.
When looking at the NVDA log viewer, I can see my edit box is classified as an EditableTextWithoutAutoSelectDetection object.
Looking at \NVDAObjects/behaviors.py I can see that I want my edit box to be classified as an EditableTextWithAutoSelectDetection object instead so my text changed events will not be ignored.

Can anyone tell me how to get my edit box classified as WithAutoSelection?

Thanks,
Ian Reed













Ian Reed
 

I got it working.
Thought I'd share the code snippet for anyone else who might benefit from this.
Also, am I supposed to call nextHandler? It seems like some app modules do not, but others do.
This actually seems like reasonable behavior for all UIA edit boxes, since they don't have AutoSelectDetection, is there a reason it isn't used for them?

class AppModule(appModuleHandler.AppModule):
    def event_typedCharacter(self, focused, nextHandler, ch):
        # we only get the updated text if it matches our specific UIA class name.
        if focused.UIAElement and focused.UIAElement.cachedClassName == "NonVisual.Edit":
oldInfo=focused.makeTextInfo(textInfos.POSITION_SELECTION)
            focused.reportSelectionChange(oldInfo)
        nextHandler()

On 6/10/2019 1:46 AM, Ian Reed via Groups.Io wrote:
That is too bad.

How about another approach.
Instead of trying to detect the text change events, is there a way I can write an app module that for my app, and my edit box, will ask UIA for the updated text info every time a key is pressed?
Since almost every key press in an edit box will trigger text changes (or at least caret movement), it seems reasonable to check for the updated text after each one.
In fact it seems NVDA already does this for every press of an arrow key. I just want it to work for all keys.

How can I detect any key down/up/press within an app module?

Thanks for continueing to help with my questions.


On 6/9/2019 11:11 PM, Joseph Lee wrote:
Hi,
I think it'd be best to use MSAA/IAccessible for now if you wish to target older Windows releases and/or older NVDA versions. UI Automation has interesting advantages, but it is known that Microsoft sometimes introduces oddities here and there - I myself have experienced it many times as a Windows Insider and as an NVDA code contributor (that's part of the reason why I keep Windows 10 App Essentials add-on around in order to provide timely fixes for UIA oddities in Windows 10 Insider Preview builds).
Cheers,
Joseph

-----Original Message-----
From: nvda-devel@groups.io <nvda-devel@groups.io> On Behalf Of Ian Reed
Sent: Sunday, June 9, 2019 10:03 PM
To: nvda-devel@groups.io
Subject: Re: [nvda-devel] UIAutomation and the EditableTextWithAutoSelectDetection class

Thanks.
Unfortunately I'm currently still on Windows 7.

Perhaps UIAutomation was the wrong choice. Could I get better integration with NVDA if I switched to MSAA/IAccessible?
I can see that the Scintilla notepad++ edit box does not have this problem, nor does plain old notepad.
How can I duplicate that behavior?


On 6/9/2019 10:55 PM, Joseph Lee wrote:
Hi,
Alpha to public; that depends on how the text change event tracking will be in the future.
As for temporarily enabling text change event: possibly, which will require telling UIA handler that you wish to let an event handler handle text change event (this will work properly in recent Windows 10 releases, not on 8.x from what I've been told). Note that this solution is based on the source code in front of me - both NVDA master branch and my own Windows 10 App Essentials add-on, both of which can listen to text change events and perform appropriate action (in case of my add-on, to log things about this event).
Cheers,
Joseph

-----Original Message-----
From: nvda-devel@groups.io <nvda-devel@groups.io> On Behalf Of Ian
Reed
Sent: Sunday, June 9, 2019 9:49 PM
To: nvda-devel@groups.io
Subject: Re: [nvda-devel] UIAutomation and the
EditableTextWithAutoSelectDetection class

Thanks for the quick response.
Is the alpha support for the text change event likely to make it to a public release at some point?
In the meantime, can you give me any guidance on making an app module that forces NVDA to listen to text change events for my specific edit box, or for each specific app written with it?

Thanks.

On 6/9/2019 8:12 PM, Joseph Lee wrote:
Hi,
In order to change this, NVDA must listen to text change event, which is only available in alpha builds at the moment.
Cheers,
Joseph

-----Original Message-----
From: nvda-devel@groups.io <nvda-devel@groups.io> On Behalf Of Ian
Reed
Sent: Sunday, June 9, 2019 7:05 PM
To: nvda-devel@groups.io
Subject: [nvda-devel] UIAutomation and the
EditableTextWithAutoSelectDetection class

I'm writing an edit control that uses UIAutomation to communicate with NVDA.
It exposes the text pattern, and that seems to work fine to get NVDA to read and braille the current char, word, or line, and to navigate by char, word and line.

I fire UIA text changed and text selection changed events at appropriate times, but NVDA seems to ignore these.
I can tell because when I insert a character into the edit box, NVDA
does not update the braille on my display, nor does it read the
current word correctly when I use the read word at review cursor
command (Numpad
5 on desktop layout).
If I use an arrow key the text on the braille display will update immediately, but not when I type characters.
When looking at the NVDA log viewer, I can see my edit box is classified as an EditableTextWithoutAutoSelectDetection object.
Looking at \NVDAObjects/behaviors.py I can see that I want my edit box to be classified as an EditableTextWithAutoSelectDetection object instead so my text changed events will not be ignored.

Can anyone tell me how to get my edit box classified as WithAutoSelection?

Thanks,
Ian Reed