Topics

Relationship between screen point and location

Alberto Buffolino
 

Hi all,
I'm playing with getNVDAObjectFromPoint in NVDAObjects.IAccessible, and there are some concepts I never understood exactly.
If I run in NVDA Python console:
obj=getNVDAObjectFromPoint(0,0)
obj.name
I see that it's the system bar, and it's ok. Then:
obj.location
returns RectLTWH(left=0, top=0, width=33, height=33), so:
obj=getNVDAObjectFromPoint(0,32)
returns again same object, because I moved across the height increasing the y, I think. But:
obj=getNVDAObjectFromPoint(32,32)
moving across the width, returns a different object. I get the first object increasing x only up to 23.
Why?
Sorry if this is not so simple to explain... and thanks in advance for replies. :)
Alberto

Rui Fontes
 

For me, that happens, because in position 32,32 you are already in the first object inside the system bar.

Verify if the second object is not the first children of system bar...


Rui Fontes


Às 17:36 de 16/12/2019, Alberto Buffolino escreveu:

Hi all,
I'm playing with getNVDAObjectFromPoint in NVDAObjects.IAccessible, and there are some concepts I never understood exactly.
If I run in NVDA Python console:
obj=getNVDAObjectFromPoint(0,0)
obj.name
I see that it's the system bar, and it's ok. Then:
obj.location
returns RectLTWH(left=0, top=0, width=33, height=33), so:
obj=getNVDAObjectFromPoint(0,32)
returns again same object, because I moved across the height increasing the y, I think. But:
obj=getNVDAObjectFromPoint(32,32)
moving across the width, returns a different object. I get the first object increasing x only up to 23.
Why?
Sorry if this is not so simple to explain... and thanks in advance for replies. :)
Alberto

Alberto Buffolino
 

Rui Fontes, il 16/12/2019 19.13, ha scritto:
For me, that happens, because in position 32,32 you are already in the first object inside the system bar.
Alberto:
apparently, no. The first child has location RectLTWH(left=0, top=0, width=33, height=33).
But surely there is something I don't understand in coordinate system.
Alberto

Karl-Otto Rosenqvist
 

Hi Alberto!
You write that the rectangle has the following coordinates:

Left: 0
Top: 0
Width: 33
Height: 33

This means that all values for X that’s between 0 (left) and 33 (left + width) and all values for Y that’s between 0 (top) and 33 (top + height) will be within the borders of the rectangle.

If X = 32 then it’s between Left and Left + Width. The same goes for Y.

Did I make it understandable?

If you have a rectangle that’s:

Left: 20
Top 20
Width 25
Height 25

The rectangle also can be defined as:

Left: 20
Top: 20
Right :45
Bottom: 45


The point X: 32 and Y: 32 is well inside the boundries of the rectangle.


Cheers
Karl-Otto
MAWINGU
0701-75 98 56
https://mawingu.se
Orgnr: 750804-3937

16 dec. 2019 kl. 19:32 skrev Alberto Buffolino <a.buffolino@...>:

Rui Fontes, il 16/12/2019 19.13, ha scritto:
For me, that happens, because in position 32,32 you are already in the first object inside the system bar.
Alberto:
apparently, no. The first child has location RectLTWH(left=0, top=0, width=33, height=33).
But surely there is something I don't understand in coordinate system.
Alberto


Alberto Buffolino
 

Karl-Otto Rosenqvist, il 16/12/2019 21.50, ha scritto:
This means that all values for X that’s between 0 (left) and 33 (left + width) and all values for Y that’s between 0 (top) and 33 (top + height) will be within the borders of the rectangle.
Alberto:
Hi Karl,
ok, all as I guessed, thanks for your explanation.
But, as I said, the object change when x>23 (becoming the titlebar), I don't understand why.
Another linked question: what are the maximum values of x and y?
I imagine 1920 and 1080 respectively, that is, my screen resolution. Right?
Thanks again for replies, I hope this research brings an interesting feature in next days ;)
Alberto

Karl-Otto Rosenqvist
 

Yes, you’re spot on. However, I’m not sure how the X and Y behaves when you have multiple screens and the mouse is on one of the non-main screens.


Regards

Karl-Otto
MAWINGU
0701-75 98 56
https://mawingu.se
Orgnr: 750804-3937

17 dec. 2019 kl. 09:41 skrev Alberto Buffolino <a.buffolino@...>:

Karl-Otto Rosenqvist, il 16/12/2019 21.50, ha scritto:
This means that all values for X that’s between 0 (left) and 33 (left + width) and all values for Y that’s between 0 (top) and 33 (top + height) will be within the borders of the rectangle.
Alberto:
Hi Karl,
ok, all as I guessed, thanks for your explanation.
But, as I said, the object change when x>23 (becoming the titlebar), I don't understand why.
Another linked question: what are the maximum values of x and y?
I imagine 1920 and 1080 respectively, that is, my screen resolution. Right?
Thanks again for replies, I hope this research brings an interesting feature in next days ;)
Alberto


Javi Domínguez
 

Hello, Alberto.

Some things you should keep in mind:

1. What the object navigator shows you is different from what is actually drawn on the screen. Objects navigator moves logically through the hierarchy of objects but not fisically by  the geography of the screen. An object can be seen in the navigator but actually were covered by another overlapping object, hidden beyond the edge of the window, etc..

obj.location returns object position (left and top) and size (width and height). But that is their logical position. This may have the invisible or hidden state; it can also have objects on top that hide it, often their own children.

getNVDAObjectFromPoint returns the object effectively drawn on the screen, in the foreground, at that particular point.

An example:

You have a listBox object with location RectLTWH(left=0, top=0, width=500, height=500)

getNVDAObjectFromPoint(0,0) maybe returns the listBox object but getNVDAObjectFromPoint(1,1) will probably return a listItem object.

2. The relationship between the drawn object and location info of thst same object can change in several ways as the interface is designed.
In some, for example VLC, several objects may have the same location, which is fixed.
They are shown or hidden by the STATE_INVISIBLE status attribute or by overlapping some panels over others. In other, location info changes dynamically. Calibre, for example, hide objects by giving them width = 0 and height = 0.

3. obj.location is the information provided by the accessibility API. It is not usual, but it may not be updated correctly.
I stumbled upon this problem in some version of Calibre.
When you move right through the table a horizontal scroll occurs on the screenbut tableColumnHeader location info was not updated. When I was going to look for it at the position that obj.location gave me to click, I wasn't really there.

Greetings

Javi Dominguez

El 16/12/2019 a las 18:36, Alberto Buffolino escribió:
Hi all,
I'm playing with getNVDAObjectFromPoint in NVDAObjects.IAccessible, and there are some concepts I never understood exactly.
If I run in NVDA Python console:
obj=getNVDAObjectFromPoint(0,0)
obj.name
I see that it's the system bar, and it's ok. Then:
obj.location
returns RectLTWH(left=0, top=0, width=33, height=33), so:
obj=getNVDAObjectFromPoint(0,32)
returns again same object, because I moved across the height increasing the y, I think. But:
obj=getNVDAObjectFromPoint(32,32)
moving across the width, returns a different object. I get the first object increasing x only up to 23.
Why?
Sorry if this is not so simple to explain... and thanks in advance for replies. :)
Alberto

Javi Domínguez
 

Hello again.

I've been trying one thing and I return to the same subject.

* In NVDA console move cursor to 7,16. You can use Golden Cursor Addon or winUser.setCursorPos.

* In that place is the object
<NVDAObjects.IAccessible.MenuItem object at 0x075A2770>
with location = RectLTWH(left=0, top=0, width=22, height=22)

* If you now move the cursor a little to the right, until 17,16, you will find the object
<NVDAObjects.IAccessible.Titlebar object at 0x07AB1B30>
with location = RectLTWH(left=16, top=-5, width=1264, height=28)

As you can see, the system menu reaches position 22 but is partially hidden by the title bar that begins at position 16.

It's another example of what I explained before. When there is more than one object in an same area obj.location is not enough to know which one is actually on the screen. That's what it is for obj.objectFromPoint.

Greetings

Javi Dominguez

P.S. This is on my screen, maybe in another one with another resolution the values ​​are different but something similar will happen.

def objInPos(x, y):
...     winUser.setCursorPos(x, y)
...     return NVDAObjects.IAccessible.getNVDAObjectFromPoint(x,y)
...
objInPos(7,16)
<NVDAObjects.IAccessible.MenuItem object at 0x05125430>
objInPos(7,16).location
RectLTWH(left=0, top=0, width=22, height=22)
objInPos(17,16)
<NVDAObjects.IAccessible.Titlebar object at 0x04F1D810>
objInPos(17,16).location
RectLTWH(left=16, top=-5, width=1264, height=28)


El 22/12/2019 a las 14:16, Javi Domínguez escribió:
Hello, Alberto.

Some things you should keep in mind:

1. What the object navigator shows you is different from what is actually drawn on the screen. Objects navigator moves logically through the hierarchy of objects but not fisically by  the geography of the screen. An object can be seen in the navigator but actually were covered by another overlapping object, hidden beyond the edge of the window, etc..

obj.location returns object position (left and top) and size (width and height). But that is their logical position. This may have the invisible or hidden state; it can also have objects on top that hide it, often their own children.

getNVDAObjectFromPoint returns the object effectively drawn on the screen, in the foreground, at that particular point.

An example:

You have a listBox object with location RectLTWH(left=0, top=0, width=500, height=500)

getNVDAObjectFromPoint(0,0) maybe returns the listBox object but getNVDAObjectFromPoint(1,1) will probably return a listItem object.

2. The relationship between the drawn object and location info of thst same object can change in several ways as the interface is designed.
In some, for example VLC, several objects may have the same location, which is fixed.
They are shown or hidden by the STATE_INVISIBLE status attribute or by overlapping some panels over others. In other, location info changes dynamically. Calibre, for example, hide objects by giving them width = 0 and height = 0.

3. obj.location is the information provided by the accessibility API. It is not usual, but it may not be updated correctly.
I stumbled upon this problem in some version of Calibre.
When you move right through the table a horizontal scroll occurs on the screenbut tableColumnHeader location info was not updated. When I was going to look for it at the position that obj.location gave me to click, I wasn't really there.

Greetings

Javi Dominguez


El 16/12/2019 a las 18:36, Alberto Buffolino escribió:
Hi all,
I'm playing with getNVDAObjectFromPoint in NVDAObjects.IAccessible, and there are some concepts I never understood exactly.
If I run in NVDA Python console:
obj=getNVDAObjectFromPoint(0,0)
obj.name
I see that it's the system bar, and it's ok. Then:
obj.location
returns RectLTWH(left=0, top=0, width=33, height=33), so:
obj=getNVDAObjectFromPoint(0,32)
returns again same object, because I moved across the height increasing the y, I think. But:
obj=getNVDAObjectFromPoint(32,32)
moving across the width, returns a different object. I get the first object increasing x only up to 23.
Why?
Sorry if this is not so simple to explain... and thanks in advance for replies. :)
Alberto

Alberto Buffolino
 

Javi Domínguez, il 22/12/2019 14.16, ha scritto:
Some things you should keep in mind:
Alberto:
Hi Javi,
I don't remember if I already replied, so...
Thanks to have summarized location considerations in this and subsequent mail, I know them at this point, but so they are well explained :)
Alberto