Re: Problem with fast browse mode - multiple NVDAObjects are created

Tony Malykh
 

Hi Michael,

Thanks for your response. I really appreciate that you would like to prioritize this, but as I mentioned in my previous email, I won't have time anymore to work on this, at least in the nearby future. I have a day job and it's consuming more and more of my time. Besides, my investigation on this PR take me deeper and deeper into NVDA internals and it looks like polishing this PR until it's production-ready might require more time than I initially estimated. I wish someone else could finish this job some day...

Best regards

Tony


On 4/15/2019 4:50 AM, Michael Curran wrote:

Hi Tony,


I would like to publicly apologize for not replying to your email around a month ago. I should have at very least explained that I did not have time to look at this at that point in time.


I still stand by my statement that I do think this work is important, and that I would like to see this in 2019.2. I also want to get this work in before speech refactor / Python 3. Therefore, I will in deed prioritize this now.


I need to clear my head enough to give this full attention and try and reconcile the issues I previously had with the PR. I am currently on personal leave but will be back next week and will start looking into it again then.


Thanks again for your contributions.


Mick










On 4/14/2019 5:06 AM, Tony Malykh wrote:

Hi Reef,
I appreciate your explanation.
1. I haven't heard from Michael for over a month. At this point I believe he saw my email and decided not to reply. I understand he was busy, but I can't believe he didn't have time to reply a few lines to me for over a month.
2. I believe I explained my problem clearly enough - duplicate  NVDA objects are created. The rest are details, test case, etc - you can find that in my previous emails in this thread.
3. I understand that you guys have your own set of priorities. However when you're saying that (and correct me if I'm wrong), it creates an impression that fast browse mode is not important or not worthy of Michael's reply. Here I pulled the list of 22 NVDA issues that I was trying to fix by fast browse mode (some of them are closed as duplicates):
    * nvda jumping the text box to enter the text to translate in google translator #8992
    * nvda 2018.4 beta 2 stuck in sum elements in web page #8999
    * Navigating quickly in edit fields on GMail causes NVDA to change from focus to browse mode unexpectedly #8211
    * When navigating the gmail page gsuite corporate inbox using the default view, nvda gets stuck in the navigation of the chat tabs #8111
    * NVDA cuts words and reports parts of different elements together on one key press; first occurance since nvda 2018.4 #9129
    * Internal focus handling seems quirky #5831
    * Browse mode problems with slow/inappropriate focus changes in browsers #2039
    * Losing Focus When Using Browse Mode #1544
    * Long pressing arror key will force to enter edit mode in firefox. #4719
    * In Firefox 37,when I move focus to a new line or control NVDA sometimes sets focus back to the previous line or control which had focus #5032
    * focus mode is activated due to caret movement sometimes randomly even when configured not to do so #5359
    * Firefox: Performance Degradation since merging #2039 Fix #5411
    * NVDA jumps when navigating on the web #2211
    * NVDA jumps to the bottom of the page when reaching the main landmark or heading level 1 on certain pages #4797
    * cursor moving backwards in navigation with the keyboard #2312
    * firefox edit fields: duplicate llines and words #3156
    * Sometimes focus jumps back to a previous web element in Firefox when arrowing down a webpage #4522
    * Internet Explorer 11 64 bit on windows 7 #6450
    * Timing issues in virtual buffers #6586
    * The user is unexpectedly dropped into focus mode on Twit.tv #6625
    * the cursor jumps backwards when browsing certain websites in Firefox, Chrome and Edge #7956
    * in chrome, sometimes focus is jumping #8013
So apparently NVDA users are bothered by bugs in NVDA browse mode as they keep creating new issues for years.
4. My attempts to fix all the bugs of fast browse mode take me deeper and deeper into NVDA internals. This particular problem with duplicate NVDA objects is very hard to fix. I could fix it for Chrome and Firefox by rewriting speech cache,but then in Internet explorer duplicate NVDA objects even have different IA2UniqueIDs, which makes it impossible for me to relate them. In edge there is another problem with focus event not being received, which I havn't started to look at. I was hoping to at least ask someone to explain to me what was the logic of creating NVDAObjects, when a new one is created as at a glance there doesn't seem to be any cache in NVDA code. But again, I received no reply. I was just trying to help NVDA, but it looks like my help is not very welcome here, Michael is only willing to be a passive code reviewer. Well, he's the boss here, but this attitude will make much harder for contributors to submit any pull requests longer than a few lines of code. Given all of that and given lack of my own free time, I'll not be working on fast browse mode anymore. I wish someone else could one day fix NVDA browse mode.
--Tony


On 4/10/2019 2:37 AM, Reef Turner wrote:
Hi Tony,

I can't speak for Michael, or the previous conversations that you have had. However, I can point out that this has been a very busy month for us at NV Access. We have had a week at CSUN AT conference (and got over the resulting jetlag), worked hard to get 2019.1 out the door and are pursuing a 2019.1.1 release. Michael and the rest of the team are taking some leave before we get into the work for the next quarter. I'm telling you this just to give some context.

Internal to NV Access, I'm sure you can appreciate that we have our own priorities and quite a busy schedule. The best advice I can give you for getting help on your project, is to try to distil the problem down to it's core, explain it is simply and succinctly as you can. You may find that there are other people on this list who can help.

On Tue, 2 Apr 2019 at 18:15, Tony Malykh <anton.malykh@...> wrote:
Hello Michael,

Did you see my previous emails?

--Tony



On 3/25/2019 3:12 PM, Tony Malykh via Groups.Io wrote:
> Hello Michael,
>
> I was wondering if I could have any assistance with this issue? I was
> under impression that you offered to help me with debugging fast
> browse mode PR... I haven't heard from you for more than two weeks, so
> should I keep waiting? Or did I misunderstand you and I'm own my own
> with this PR?
>
> Thanks
>
> --Tony
>
> On 3/20/2019 9:20 AM, Tony Malykh via Groups.Io wrote:
>> Moving this thread to new mailing list.
>>
>> [BCC old mailing list]
>>
>>
>> --Tony
>>
>> On 3/9/2019 2:49 PM, Tony Malykh wrote:
>>> Hello NVDA devs and especially Michael,
>>>
>>> I am trying to work on fast browse mode PR (a.k.a. focus follows
>>> browse mode off).
>>> Just a quick recap: In v1 of this PR there were some  bugs related
>>> to the fact that some logic was asynchronous and it relied on
>>> gainFocus event being received from the operating system. However in
>>> some cases for unknown reason operating system wouldn't send this
>>> event, causing bugs that are hard to deal with, such as inability to
>>> activate form elements on web pages.
>>> I am now working on v2:
>>> https://github.com/nvaccess/nvda/pull/9360
>>> The code is by no means ready to be reviewed, but it's just for the
>>> reference.
>>> The main change in v2 is replacing asynchronous logic with
>>> synchronous logic there, and in general it works, but there is one
>>> new bugI'm facing that I would like to discuss and maybe ask for
>>> some help.
>>>
>>> Basic description of the bug:
>>> When unchecking a checkbox on a web page, its new state is not being
>>> announced. The bug only reproduces for unchecking - that is checking
>>> previously blank checkbox works as expected.
>>> Here are the concrete steps to reproduce the bug.
>>> 1. Enter Fast browse mode by pressing NVDA+num row 8.
>>> 2. In Chrome or Firefox navigate to
>>> https://www.ironspider.ca/forms/checkradio.htm
>>> 3. Check the first 3 checkboxes on the page
>>> 4. Go back to the first check box
>>> 5. Press space to uncheck it. The check box will be successfully
>>> unchecked, but the new state will not be announced immediately.
>>>
>>> I spent some time debugging this , and here are my findings so far.
>>> Announcing state change works like this. First
>>> speech.speakObject(obj,controlTypes.REASON_ONLYCACHE) is called to
>>> cache the previous state. In traditional browse mode this function
>>> is called when user navigates to that checkbox, and therefore system
>>> focus is switched to that checkbox as well. This happens in
>>> BrowseModeDocumentTreeInterceptor.event_gainFocus.
>>> Then, assuming that user presses space bar on a checkbox,
>>> BrowseModeTreeInterceptor._activatePosition is triggered, which
>>> activates this checkbox. Shortly, in response to that we receive a
>>> state changed event, that is processed in
>>> NVDAObject.event_stateChange(), which in turn calls
>>> speech.speakObjectProperties(self,states=True,
>>> reason=controlTypes.REASON_CHANGE).
>>> So we make two calls to speech module: the first one to cache object
>>> properties, the second one to speak whatever properties have
>>> changed. Cached properties are stored inside NVDAObject, so it is
>>> very important that the same object is passed to both these calls.
>>>
>>> Now back to fast browse mode v2. Since we don't update system focus
>>> on every browse mode focus change, we need to update system focus
>>> right before activating the checkbox. This is accomplished in
>>> BrowseModeTreeInterceptor.maybeSyncFocus(). And in the same
>>> function, right after I call obj.setFocus(), I also call
>>> speech.speakObject(obj,controlTypes.REASON_ONLYCACHE).
>>> However, here is the problem. When we receive state changed event,
>>> the function NVDAObject.event_stateChange is called on a different
>>> NVDA object instance. So we completely miss the cached state.
>>> When I say a different NVDAObject instance, I only mean a different
>>> python object. So there are apparently (at least) two NVDAObjects,
>>> I'll call them NVDAObject1 and NVDAObject2. Both represent the same
>>> checkbox in the browser, both have the same IA2UniqueID and
>>> windowHandle. Yet in the first call (cacheonly) I see NVDAObject1,
>>> and the state changed event comes in with NVDAObject2.
>>> To illustrate this, here is the example from my logs (slightly
>>> formatted):
>>> cacheonly from maybeSyncFocus():
>>>     <NVDAObjects.IAccessible.mozilla.Mozilla object at 0x07A6A370>
>>>     uniqueID = (67548, -33555216)
>>> speakObject on stateChanged
>>>     <NVDAObjects.IAccessible.mozilla.Mozilla object at 0x07A6A1F0>
>>>     uniqueID = (67548L, -33555216)
>>>
>>> So the big question I have is why there are two separate NVDAObjects
>>> created for the same checkbox in my scenario? And why they are not
>>> created Without my PR? What is logic behind creating new NVDA
>>> Objects? I did some cursory debugging of that and I saw that most of
>>> NVDAObjects are created using function getNVDAObjectFromEvent()
>>> defined in IAccessible2\__init__.py, and there doesn't seem to be
>>> any caching logic there.
>>> It seems there are two potential approaches of tackling this issue.
>>> 1. We need to figure out why two separate NVDA objects are created.
>>> 2. Potentially another approach is to allow creating multiple python
>>> NVDAObjects for the same Checkbox (or any other real IAccessible2
>>> object), and have a properties cache that would take this into
>>> account. Some kind of weakKeyDictionary might work. Property caching
>>> logic in speech.py might have to be redesigned though.
>>>
>>> And also another thing worth noticing in the logs that I pasted
>>> above. UniqueID is a tuple of WindowHandle and IA2UniqueID . If you
>>> looked at both entries carefully, you would notice that WindowHandle
>>> is slightly different: even though the value is the same, in the
>>> first entry it is int, and in the second entry it is long. This
>>> looks very suspicious to me. Do you think this might be the reason
>>> why two copies of NVDAObjects are created? On the other hand it
>>> doesn't explain why two objects are not created in stock NVDA.
>>>
>>> Anyway, this is current state of my second attempt to implement fast
>>> browse mode. Any suggestions will be appreciated!
>>> Best
>>> --Tony
>>>
>>
>>
>>
>
>
>





--
Reef Turner
Software Developer 
 
Facebook: https://www.facebook.com/NVAccess 
Twitter: @NVAccess 


Join nvda-devel@groups.io to automatically receive all group messages.