Topics

Finding components on an application from addon - asking for stategies and insights to do it

Marlon Brandão de Sousa
 

Hello,

I have a need for an addom. Basically, in VSCode, I many times need to know the line / column I am at the moment.

I know that performing now a read status line will give me this information, but there are many other things that I don't need on the status bar when quickly in needs to know the line / collumn positions.

Further more, this is a specific case for a generic question so that others can also learn the strategy to achieve the goals I am trying to reach at the moment.

Basically, when scripting an addon, we are trying to do one of three things:
- Modify the way a given component is handled by NVDA so that it is read correctly.
- Query information from parts of the application and present them in an easier way for the addon user, so that they can gater needed information that is visually easy to get but that without sight would require the user to perform object navigation and to visit several parts of the UI to get.
- Similarly to the above, perform clicks or other manipulations on the UI to achieve actions that are common to the work flow of the application but that are either inaccessible via keyboard or very hard to perform in a productive way, many times making an efficient use of that application not viable at all.

My need with VS Code can be classified at the item 2 of the above list.

Unfortunately I get pretty lost when achieving second and third items. The first is achievable by using an overlay class thus overwiting the component NVDA uses to handle a given component. About The second and third would involve navigating through the window structure and searching components by some kind of id and then querying information or sending actions too that component.

In the old times when NVDA didn't exist other screen readers offered higher level API's to do just this kind of thing. Speaking of JAWS, the home row utility helped a lot when exploring an application internal structure so that we could try to find an id, a class, a Window name and such and then search for that identifier to get access to the wanted component state. How can I do something similar via NVDA addon? Is there any addon, any piece of the source code I should look into to start figuring it out?


Thanks,

Marlon

Pawel Urbanski
 

I've worked on an VS Code add-on. VSC is an Electron application. I
guess you could read though the code of Development Kit by Andy who
recently used Beautiful Soup library to parse HTML. The line number is
in teh status bar, which itself is hosted in some DIV container.
I will try to figure out if I could add line announcement to my add-on
by using textInfos offsets.

On 12/06/2020, Marlon Brandão de Sousa <splyt.lists@...> wrote:
Hello,

I have a need for an addom. Basically, in VSCode, I many times need to
know the line / column I am at the moment.

I know that performing now a read status line will give me this
information, but there are many other things that I don't need on the
status bar when quickly in needs to know the line / collumn positions.

Further more, this is a specific case for a generic question so that
others can also learn the strategy to achieve the goals I am trying to
reach at the moment.

Basically, when scripting an addon, we are trying to do one of three
things:
- Modify the way a given component is handled by NVDA so that it is read
correctly.
- Query information from parts of the application and present them in an
easier way for the addon user, so that they can gater needed information
that is visually easy to get but that without sight would require the
user to perform object navigation and to visit several parts of the UI
to get.
- Similarly to the above, perform clicks or other manipulations on the
UI to achieve actions that are common to the work flow of the
application but that are either inaccessible via keyboard or very hard
to perform in a productive way, many times making an efficient use of
that application not viable at all.

My need with VS Code can be classified at the item 2 of the above list.

Unfortunately I get pretty lost when achieving second and third items.
The first is achievable by using an overlay class thus overwiting the
component NVDA uses to handle a given component. About The second and
third would involve navigating through the window structure and
searching components by some kind of id and then querying information or
sending actions too that component.

In the old times when NVDA didn't exist other screen readers offered
higher level API's to do just this kind of thing. Speaking of JAWS, the
home row utility helped a lot when exploring an application internal
structure so that we could try to find an id, a class, a Window name and
such and then search for that identifier to get access to the wanted
component state. How can I do something similar via NVDA addon? Is there
any addon, any piece of the source code I should look into to start
figuring it out?


Thanks,

Marlon





Marlon Brandão de Sousa
 

I use and appreciate your addon. This question was exactly because I wanted to send a pull request to it. Glad your are here.


I know VS Code is an electron application. What I do not know is how to, via NVDA, navigate through the accessibility tree to find and read what I need. I also do not know how to access the accessibility tree so that I know beforehand what I am looking for. This is what I am trying to figure out so that I can collaborate with addons that will work on vscode, discord and postman and other apps.


Do you have any insight?

On 12/06/2020 08:28, Pawel Urbanski wrote:
I've worked on an VS Code add-on. VSC is an Electron application. I
guess you could read though the code of Development Kit by Andy who
recently used Beautiful Soup library to parse HTML. The line number is
in teh status bar, which itself is hosted in some DIV container.
I will try to figure out if I could add line announcement to my add-on
by using textInfos offsets.

On 12/06/2020, Marlon Brandão de Sousa <splyt.lists@...> wrote:
Hello,

I have a need for an addom. Basically, in VSCode, I many times need to
know the line / column I am at the moment.

I know that performing now a read status line will give me this
information, but there are many other things that I don't need on the
status bar when quickly in needs to know the line / collumn positions.

Further more, this is a specific case for a generic question so that
others can also learn the strategy to achieve the goals I am trying to
reach at the moment.

Basically, when scripting an addon, we are trying to do one of three
things:
- Modify the way a given component is handled by NVDA so that it is read
correctly.
- Query information from parts of the application and present them in an
easier way for the addon user, so that they can gater needed information
that is visually easy to get but that without sight would require the
user to perform object navigation and to visit several parts of the UI
to get.
- Similarly to the above, perform clicks or other manipulations on the
UI to achieve actions that are common to the work flow of the
application but that are either inaccessible via keyboard or very hard
to perform in a productive way, many times making an efficient use of
that application not viable at all.

My need with VS Code can be classified at the item 2 of the above list.

Unfortunately I get pretty lost when achieving second and third items.
The first is achievable by using an overlay class thus overwiting the
component NVDA uses to handle a given component. About The second and
third would involve navigating through the window structure and
searching components by some kind of id and then querying information or
sending actions too that component.

In the old times when NVDA didn't exist other screen readers offered
higher level API's to do just this kind of thing. Speaking of JAWS, the
home row utility helped a lot when exploring an application internal
structure so that we could try to find an id, a class, a Window name and
such and then search for that identifier to get access to the wanted
component state. How can I do something similar via NVDA addon? Is there
any addon, any piece of the source code I should look into to start
figuring it out?


Thanks,

Marlon





Pawel Urbanski
 

Give me this weekend to investigate. ;)

On 12/06/2020, Marlon Brandão de Sousa <splyt.lists@...> wrote:
I use and appreciate your addon. This question was exactly because I
wanted to send a pull request to it. Glad your are here.


I know VS Code is an electron application. What I do not know is how to,
via NVDA, navigate through the accessibility tree to find and read what
I need. I also do not know how to access the accessibility tree so that
I know beforehand what I am looking for. This is what I am trying to
figure out so that I can collaborate with addons that will work on
vscode, discord and postman and other apps.


Do you have any insight?


On 12/06/2020 08:28, Pawel Urbanski wrote:
I've worked on an VS Code add-on. VSC is an Electron application. I
guess you could read though the code of Development Kit by Andy who
recently used Beautiful Soup library to parse HTML. The line number is
in teh status bar, which itself is hosted in some DIV container.
I will try to figure out if I could add line announcement to my add-on
by using textInfos offsets.

On 12/06/2020, Marlon Brandão de Sousa <splyt.lists@...> wrote:
Hello,

I have a need for an addom. Basically, in VSCode, I many times need to
know the line / column I am at the moment.

I know that performing now a read status line will give me this
information, but there are many other things that I don't need on the
status bar when quickly in needs to know the line / collumn positions.

Further more, this is a specific case for a generic question so that
others can also learn the strategy to achieve the goals I am trying to
reach at the moment.

Basically, when scripting an addon, we are trying to do one of three
things:
- Modify the way a given component is handled by NVDA so that it is read
correctly.
- Query information from parts of the application and present them in an
easier way for the addon user, so that they can gater needed information
that is visually easy to get but that without sight would require the
user to perform object navigation and to visit several parts of the UI
to get.
- Similarly to the above, perform clicks or other manipulations on the
UI to achieve actions that are common to the work flow of the
application but that are either inaccessible via keyboard or very hard
to perform in a productive way, many times making an efficient use of
that application not viable at all.

My need with VS Code can be classified at the item 2 of the above list.

Unfortunately I get pretty lost when achieving second and third items.
The first is achievable by using an overlay class thus overwiting the
component NVDA uses to handle a given component. About The second and
third would involve navigating through the window structure and
searching components by some kind of id and then querying information or
sending actions too that component.

In the old times when NVDA didn't exist other screen readers offered
higher level API's to do just this kind of thing. Speaking of JAWS, the
home row utility helped a lot when exploring an application internal
structure so that we could try to find an id, a class, a Window name and
such and then search for that identifier to get access to the wanted
component state. How can I do something similar via NVDA addon? Is there
any addon, any piece of the source code I should look into to start
figuring it out?


Thanks,

Marlon







Pawel Urbanski
 

Marlon - would you mind having a Hangout call to talk about use cases
on the line of VS Code and NVDA? On a side note: I am porting my
add-on to some other IDEs. ;)

On 12/06/2020, Pawel Urbanski via groups.io
<pawel=e-urbanski.com@groups.io> wrote:
Give me this weekend to investigate. ;)

On 12/06/2020, Marlon Brandão de Sousa <splyt.lists@...> wrote:
I use and appreciate your addon. This question was exactly because I
wanted to send a pull request to it. Glad your are here.


I know VS Code is an electron application. What I do not know is how to,
via NVDA, navigate through the accessibility tree to find and read what
I need. I also do not know how to access the accessibility tree so that
I know beforehand what I am looking for. This is what I am trying to
figure out so that I can collaborate with addons that will work on
vscode, discord and postman and other apps.


Do you have any insight?


On 12/06/2020 08:28, Pawel Urbanski wrote:
I've worked on an VS Code add-on. VSC is an Electron application. I
guess you could read though the code of Development Kit by Andy who
recently used Beautiful Soup library to parse HTML. The line number is
in teh status bar, which itself is hosted in some DIV container.
I will try to figure out if I could add line announcement to my add-on
by using textInfos offsets.

On 12/06/2020, Marlon Brandão de Sousa <splyt.lists@...> wrote:
Hello,

I have a need for an addom. Basically, in VSCode, I many times need to
know the line / column I am at the moment.

I know that performing now a read status line will give me this
information, but there are many other things that I don't need on the
status bar when quickly in needs to know the line / collumn positions.

Further more, this is a specific case for a generic question so that
others can also learn the strategy to achieve the goals I am trying to
reach at the moment.

Basically, when scripting an addon, we are trying to do one of three
things:
- Modify the way a given component is handled by NVDA so that it is
read
correctly.
- Query information from parts of the application and present them in
an
easier way for the addon user, so that they can gater needed
information
that is visually easy to get but that without sight would require the
user to perform object navigation and to visit several parts of the UI
to get.
- Similarly to the above, perform clicks or other manipulations on the
UI to achieve actions that are common to the work flow of the
application but that are either inaccessible via keyboard or very hard
to perform in a productive way, many times making an efficient use of
that application not viable at all.

My need with VS Code can be classified at the item 2 of the above list.

Unfortunately I get pretty lost when achieving second and third items.
The first is achievable by using an overlay class thus overwiting the
component NVDA uses to handle a given component. About The second and
third would involve navigating through the window structure and
searching components by some kind of id and then querying information
or
sending actions too that component.

In the old times when NVDA didn't exist other screen readers offered
higher level API's to do just this kind of thing. Speaking of JAWS, the
home row utility helped a lot when exploring an application internal
structure so that we could try to find an id, a class, a Window name
and
such and then search for that identifier to get access to the wanted
component state. How can I do something similar via NVDA addon? Is
there
any addon, any piece of the source code I should look into to start
figuring it out?


Thanks,

Marlon









Marlon Brandão de Sousa
 

Definitely. Tomorrow or sundae what utc are you?

Obrigado,
Marlon

Em 12 de jun de 2020, à(s) 09:26, Pawel Urbanski <pawel@...> escreveu:

Marlon - would you mind having a Hangout call to talk about use cases
on the line of VS Code and NVDA? On a side note: I am porting my
add-on to some other IDEs. ;)

On 12/06/2020, Pawel Urbanski via groups.io
<pawel=e-urbanski.com@groups.io> wrote:
Give me this weekend to investigate. ;)

On 12/06/2020, Marlon Brandão de Sousa <splyt.lists@...> wrote:
I use and appreciate your addon. This question was exactly because I
wanted to send a pull request to it. Glad your are here.


I know VS Code is an electron application. What I do not know is how to,
via NVDA, navigate through the accessibility tree to find and read what
I need. I also do not know how to access the accessibility tree so that
I know beforehand what I am looking for. This is what I am trying to
figure out so that I can collaborate with addons that will work on
vscode, discord and postman and other apps.


Do you have any insight?


On 12/06/2020 08:28, Pawel Urbanski wrote:
I've worked on an VS Code add-on. VSC is an Electron application. I
guess you could read though the code of Development Kit by Andy who
recently used Beautiful Soup library to parse HTML. The line number is
in teh status bar, which itself is hosted in some DIV container.
I will try to figure out if I could add line announcement to my add-on
by using textInfos offsets.

On 12/06/2020, Marlon Brandão de Sousa <splyt.lists@...> wrote:
Hello,

I have a need for an addom. Basically, in VSCode, I many times need to
know the line / column I am at the moment.

I know that performing now a read status line will give me this
information, but there are many other things that I don't need on the
status bar when quickly in needs to know the line / collumn positions.

Further more, this is a specific case for a generic question so that
others can also learn the strategy to achieve the goals I am trying to
reach at the moment.

Basically, when scripting an addon, we are trying to do one of three
things:
- Modify the way a given component is handled by NVDA so that it is
read
correctly.
- Query information from parts of the application and present them in
an
easier way for the addon user, so that they can gater needed
information
that is visually easy to get but that without sight would require the
user to perform object navigation and to visit several parts of the UI
to get.
- Similarly to the above, perform clicks or other manipulations on the
UI to achieve actions that are common to the work flow of the
application but that are either inaccessible via keyboard or very hard
to perform in a productive way, many times making an efficient use of
that application not viable at all.

My need with VS Code can be classified at the item 2 of the above list.

Unfortunately I get pretty lost when achieving second and third items.
The first is achievable by using an overlay class thus overwiting the
component NVDA uses to handle a given component. About The second and
third would involve navigating through the window structure and
searching components by some kind of id and then querying information
or
sending actions too that component.

In the old times when NVDA didn't exist other screen readers offered
higher level API's to do just this kind of thing. Speaking of JAWS, the
home row utility helped a lot when exploring an application internal
structure so that we could try to find an id, a class, a Window name
and
such and then search for that identifier to get access to the wanted
component state. How can I do something similar via NVDA addon? Is
there
any addon, any piece of the source code I should look into to start
figuring it out?


Thanks,

Marlon










Pawel Urbanski
 

Unfortunately, this weekend can be tricky, but almost any day
throughout the week. Just drop me a line at: pawel@... so
we take it of this thread and do not polute the list with messages.

On 12/06/2020, Marlon Brandão de Sousa <splyt.lists@...> wrote:
Definitely. Tomorrow or sundae what utc are you?

Obrigado,
Marlon

Em 12 de jun de 2020, à(s) 09:26, Pawel Urbanski <pawel@...>
escreveu:

Marlon - would you mind having a Hangout call to talk about use cases
on the line of VS Code and NVDA? On a side note: I am porting my
add-on to some other IDEs. ;)

On 12/06/2020, Pawel Urbanski via groups.io
<pawel=e-urbanski.com@groups.io> wrote:
Give me this weekend to investigate. ;)

On 12/06/2020, Marlon Brandão de Sousa <splyt.lists@...> wrote:
I use and appreciate your addon. This question was exactly because I
wanted to send a pull request to it. Glad your are here.


I know VS Code is an electron application. What I do not know is how
to,
via NVDA, navigate through the accessibility tree to find and read what
I need. I also do not know how to access the accessibility tree so that
I know beforehand what I am looking for. This is what I am trying to
figure out so that I can collaborate with addons that will work on
vscode, discord and postman and other apps.


Do you have any insight?


On 12/06/2020 08:28, Pawel Urbanski wrote:
I've worked on an VS Code add-on. VSC is an Electron application. I
guess you could read though the code of Development Kit by Andy who
recently used Beautiful Soup library to parse HTML. The line number is
in teh status bar, which itself is hosted in some DIV container.
I will try to figure out if I could add line announcement to my add-on
by using textInfos offsets.

On 12/06/2020, Marlon Brandão de Sousa <splyt.lists@...> wrote:
Hello,

I have a need for an addom. Basically, in VSCode, I many times need
to
know the line / column I am at the moment.

I know that performing now a read status line will give me this
information, but there are many other things that I don't need on the
status bar when quickly in needs to know the line / collumn
positions.

Further more, this is a specific case for a generic question so that
others can also learn the strategy to achieve the goals I am trying
to
reach at the moment.

Basically, when scripting an addon, we are trying to do one of three
things:
- Modify the way a given component is handled by NVDA so that it is
read
correctly.
- Query information from parts of the application and present them in
an
easier way for the addon user, so that they can gater needed
information
that is visually easy to get but that without sight would require the
user to perform object navigation and to visit several parts of the
UI
to get.
- Similarly to the above, perform clicks or other manipulations on
the
UI to achieve actions that are common to the work flow of the
application but that are either inaccessible via keyboard or very
hard
to perform in a productive way, many times making an efficient use of
that application not viable at all.

My need with VS Code can be classified at the item 2 of the above
list.

Unfortunately I get pretty lost when achieving second and third
items.
The first is achievable by using an overlay class thus overwiting the
component NVDA uses to handle a given component. About The second and
third would involve navigating through the window structure and
searching components by some kind of id and then querying information
or
sending actions too that component.

In the old times when NVDA didn't exist other screen readers offered
higher level API's to do just this kind of thing. Speaking of JAWS,
the
home row utility helped a lot when exploring an application internal
structure so that we could try to find an id, a class, a Window name
and
such and then search for that identifier to get access to the wanted
component state. How can I do something similar via NVDA addon? Is
there
any addon, any piece of the source code I should look into to start
figuring it out?


Thanks,

Marlon












Luke Davis
 

Is there anything that even slightly works on Postman?

It seems on the cusp of being accessible, but doesn't pass the threshold.

On Fri, 12 Jun 2020, Marlon Brandão de Sousa wrote:

I use and appreciate your addon. This question was exactly because I wanted to send a pull request to it. Glad your are here.


I know VS Code is an electron application. What I do not know is how to, via NVDA, navigate through the accessibility tree to find and read what I need. I also do not know how to access the accessibility tree so that I know beforehand what I am looking for. This is what I am trying to figure out so that I can collaborate with addons that will work on vscode, discord and postman and other apps.


Do you have any insight?


On 12/06/2020 08:28, Pawel Urbanski wrote:
I've worked on an VS Code add-on. VSC is an Electron application. I
guess you could read though the code of Development Kit by Andy who
recently used Beautiful Soup library to parse HTML. The line number is
in teh status bar, which itself is hosted in some DIV container.
I will try to figure out if I could add line announcement to my add-on
by using textInfos offsets.
On 12/06/2020, Marlon Brandão de Sousa <splyt.lists@...> wrote:
Hello,
I have a need for an addom. Basically, in VSCode, I many times need to
know the line / column I am at the moment.
I know that performing now a read status line will give me this
information, but there are many other things that I don't need on the
status bar when quickly in needs to know the line / collumn positions.
Further more, this is a specific case for a generic question so that
others can also learn the strategy to achieve the goals I am trying to
reach at the moment.
Basically, when scripting an addon, we are trying to do one of three
things:
- Modify the way a given component is handled by NVDA so that it is read
correctly.
- Query information from parts of the application and present them in an
easier way for the addon user, so that they can gater needed information
that is visually easy to get but that without sight would require the
user to perform object navigation and to visit several parts of the UI
to get.
- Similarly to the above, perform clicks or other manipulations on the
UI to achieve actions that are common to the work flow of the
application but that are either inaccessible via keyboard or very hard
to perform in a productive way, many times making an efficient use of
that application not viable at all.
My need with VS Code can be classified at the item 2 of the above list.
Unfortunately I get pretty lost when achieving second and third items.
The first is achievable by using an overlay class thus overwiting the
component NVDA uses to handle a given component. About The second and
third would involve navigating through the window structure and
searching components by some kind of id and then querying information or
sending actions too that component.
In the old times when NVDA didn't exist other screen readers offered
higher level API's to do just this kind of thing. Speaking of JAWS, the
home row utility helped a lot when exploring an application internal
structure so that we could try to find an id, a class, a Window name and
such and then search for that identifier to get access to the wanted
component state. How can I do something similar via NVDA addon? Is there
any addon, any piece of the source code I should look into to start
figuring it out?
Thanks,
Marlon