Re: Proposal: enforce minimum and last tested NVDA version flags for add-on reviews and certification upon 2019.3 release?

Noelia Ruiz
 

Hi, really, imo they shouldn't be exempt, since this could break things in the system like crashes, and I think that the current flags are appropriate, though I read the last developer guide and code, and in fact now I think that this value may be documented in a clearer way addressing the revised approach.

- lastTestedNVDAVersion (string): The last version of NVDA this add-on has been tested with.
- e.g "2019.1.0"
- Must be a three part version string I.E. Year.Major.Minor, or a two part version string of Year.Major. In the second case, Minor defaults to 0.
- Defaults to "0.0.0"
Imo, if this value is not always the maximum version supported, this should be clarified and even, the developer guide, since is updated for each NVDA stable release, could indicate the apiCompatto value, which is the minimum value required of last tested version in order than the current release can be compatible.


An option to clarify things about Andy's proposal could be, when apiCompatTo value is not the same a

El 03/10/2019 a las 04:54, derek riemer escribió:
I think alpha, beta, and RC should automatically exempt themselves from the last tested and minimum flags. That way, developers can battletest addons easily with these versions.
On Mon, Sep 30, 2019 at 7:21 AM Andy B. <sonfire11@... <mailto:sonfire11@...>> wrote:
I think maximum NVDA version should be required because we are
talking about a range of NVDA versions. It makes it clear to
developers and users, if included in add-on manager, that this
add-on supports the stated NVDA versions, even if they are
alpha/beta builds. Now, when stating a nonexistent version of NVDA,
such as alpha, beta, or threshold builds, maximum NVDA version
should indicate an api version such as 2019.3, but also include a
prerelease branch such as threshold, alpha, beta, or rc. After this,
we would have versions stated such as minimumNVDAVersion =
2019.3-alpha, or maximumNVDAVersion = 2019.3-beta1.____
__ __
Andy Borka____
Accessibility Engineer____
__ __
*From:* nvda-devel@groups.io <mailto:nvda-devel@groups.io>
<nvda-devel@groups.io <mailto:nvda-devel@groups.io>> *On Behalf Of
*Noelia Ruiz
*Sent:* Sunday, September 29, 2019 10:58 PM
*To:* nvda-devel@groups.io <mailto:nvda-devel@groups.io>
*Subject:* Re: [nvda-devel] Proposal: enforce minimum and last
tested NVDA version flags for add-on reviews and certification upon
2019.3 release?____
__ __
Regarding to the enforzement and the original question, I wass
missing, until I shared the code here, that for NVDA the last tested
version is always present and not an empty value, since it defaults
to the current NVDA version when the manifest doesnt declare it. So
the important think is that this doesnt break NVDA or system
stability. If an author doesnt want to specify last tested version
and the add-on is fully compatible, we cannot enforze this since
user experience pass. But if alpha versions of NVDA or the last
stable version is incompatible, since the test version is too low
and the add-on will be blocked, or even worse, if the author doesnt
specify a tested version and for this reason NVDA considers that the
add-on is good but in fact it break things, review should fail and
the ad-on shouldnt remain in the stable section of the website.____
Other question would be, as suggested by Andy (not directly),
clarify, in addons manager, the maximum version supported according
to the last tested value, for instance: last test version: 2019.-3
(max suported version: 2019.-4 of course when available). Not sure
if it would be needed.____
Cheers____
__ __
Enviado desde mi iPhone____
El 30 sept 2019, a las 2:40, Joseph Lee <@joslee
<mailto:@joslee>> escribió:____
Hi,____
The plan (which is really a separate thread at this point)
involves sending current NVDA version, along with add-on names,
their versions and update channel metadata. In return the client
will get update info for add-ons deemed compatible based on the
NVDA version sent to the server.____
Cheers,____
Joseph____
____
*From:* nvda-devel@groups.io <mailto:nvda-devel@groups.io>
<nvda-devel@groups.io <mailto:nvda-devel@groups.io>> *On Behalf
Of *derek riemer
*Sent:* Sunday, September 29, 2019 5:34 PM
*To:* nvda-devel@groups.io <mailto:nvda-devel@groups.io>
*Subject:* Re: [nvda-devel] Proposal: enforce minimum and last
tested NVDA version flags for add-on reviews and certification
upon 2019.3 release?____
____
That makes sense, I'm primarily concerned that users will have
to manually update all addons every version, even if they
literally didn't change. I think we should do the lastTested
thing serverSide, or at least offer a mechanism to update that
on the server eventually, so users don't need to download the
exactt same payload with just a different tested version if that
version is already tested..____
____
On Sun, Sep 29, 2019 at 6:32 PM Joseph Lee
<@joslee <mailto:@joslee>>
wrote:____
Hi,____
At least Add-on Updater enforces this check for now unless
the strategy changes.____
Cheers,____
Joseph____
____
*From:* nvda-devel@groups.io <mailto:nvda-devel@groups.io>
<nvda-devel@groups.io <mailto:nvda-devel@groups.io>> *On
Behalf Of *derek riemer
*Sent:* Sunday, September 29, 2019 4:26 PM
*To:* nvda-devel@groups.io <mailto:nvda-devel@groups.io>
*Subject:* Re: [nvda-devel] Proposal: enforce minimum and
last tested NVDA version flags for add-on reviews and
certification upon 2019.3 release?____
____
I really don't think the last tested bit should be enforced
until we have an updator.____
____
On Sun, Sep 29, 2019 at 10:28 AM Noelia Ruiz
<nrm1977@... <mailto:nrm1977@...>> wrote:____
El 29/09/2019 a las 18:18, Joseph Lee escribió:
> Hi,
> Actually, you do need to change last tested version
once 2019.4 comes out too, because that's how add-ons
manager will check for compatibility
I think this is not right (if I'm wrong I will apologize :)
The add-ons manager (or NVDA doesn't check for
compatibility directly
looking at last tested version, but to the api
containing the last
tested version. This is a way to let authors precissely
not to change
this flag each time NVDA has a new stable release. So,
when NVDA 2019.4
is out, we don't need to change this flag unless NVDA
has important
changes. NVDA 2019.3 is an exception due to Python 3
version and speech
refactor.
But generally this flag, if used, don't need to be changed.
Cheers
El 29/09/2019 a las 18:18, Joseph Lee escribió:
> Hi,
> Actually, you do need to change last tested version
once 2019.4 comes out too, because that's how add-ons
manager will check for compatibility (some folks found
out the hard way when attempting to install an add-on
that wasn't marked for 2019.3). The purpose of that flag
is for add-on authors to communicate that they did test
their add-ons in a specific NVDA release (or a
development snapshot for it), and ideally, source code
should match what the manifest claims. By doing so, it
reminds authors that they do need to keep up with
changes introduced in NVDA Core - the minimum flag won't
change much, but last tested flag will from time to time.
> As for communicating the purpose of that flag to
users: I can see confusion. Calling it "last tested
version" when in fact it specifies maximum compatible
version isn't a good communication to begin with,
because latest compatible and last tested versions are
likely to be different (for my add-ons at least, they
are really the same, although in public documentation I
always specify latest stable version for maximum
compatibility range because things under development can
change rapidly; but then again, for my add-ons, I
release development snapshots for some add-ons to keep
up with alpha changes, or as Eric Raymond once advised,
"release early, release often"). I think what can help
might be to listening to public definition of this flag,
and if they are not in line with the actual purpose of
it, to educate users as to what it actually means; if
there is indeed difference, perhaps come up with a
compromise or think about this flag again in a few months.
> Cheers,
> Joseph
>
> -----Original Message-----
> From: nvda-devel@groups.io
<mailto:nvda-devel@groups.io> <nvda-devel@groups.io
<mailto:nvda-devel@groups.io>> On Behalf Of Noelia Ruiz
> Sent: Sunday, September 29, 2019 8:54 AM
> To: nvda-devel@groups.io <mailto:nvda-devel@groups.io>
> Subject: Re: [nvda-devel] Proposal: enforce minimum
and last tested NVDA version flags for add-on reviews
and certification upon 2019.3 release?
>
> Hi, if you want to build versions for testers you
have to change the last tested version in the case of
NVDA 2019.3 due to deletions in the API, but probably,
when NVDA 2019.4 pre-releases or stable are published,
you will not need to change the last tested version. So,
the purpose of this field in the manifest maybe
clarified in the add-on template, but for me the name is
right since this doesn't mean the maximum version that
the add-on is intended to support, nor the last version
of NVDA beingbuild, nor the last stable.
> Cheers
>
>
> El 29/09/2019 a las 14:02, Andy B. escribió:
>> Well, not really. If I want to release add-on builds
for testers, I have to set the last tested version flag
to the current prerelease of NVDA. This isn't a problem,
but the flag's name is misleading at best. Minimum
version and maximum version as I mentioned before, are
self-documenting flags. There is no confusion that you
have to run at least NVDA 2019.1.1, and nothing later
than 2019.3 for this add-on to work.
>> Keep in mind that prospective add-on authors will
take a look at some of their favorite add-on's manifest
and source if they are interested in trying add-on
development. Doing this makes sense because the user is
already familiar with how the add-on works, so they can
follow along with their experience while looking at the
source. Unless a user is already somewhat buried in NVDA
development, minimum version makes sense, but last
tested version does not. Therefore, last tested version
should be changed to maximum version with a comment in
the manifest that maximum version is either equal to or
greater than minimum version, and that maximum version
may include prerelease versions of NVDA.
>>
>> # The lowest supported version of NVDA.
>> minimumNVDAVersion = 2019.1.1 # DTK does not support
2019.1 because it was initially created during 2019.1.1
release.
>> maximumNVDAVersion = 2019.3 # The current alpha
prelease cycle.
>>
>> I vote to have the flags changed to represent the
above example.
>>
>> Andy Borka
>> Accessibility Engineer
>>
>> -----Original Message-----
>> From: nvda-devel@groups.io
<mailto:nvda-devel@groups.io> <nvda-devel@groups.io
<mailto:nvda-devel@groups.io>> On Behalf Of Noelia
>> Ruiz
>> Sent: Saturday, September 28, 2019 2:06 PM
>> To: nvda-devel@groups.io <mailto:nvda-devel@groups.io>
>> Subject: Re: [nvda-devel] Proposal: enforce minimum
and last tested NVDA version flags for add-on reviews
and certification upon 2019.3 release?
>>
>> Imo the current name is right, since the latest
tested version can be minor of the latest supported
version. For example, you probably will not to update
the tested version in the manifest for developer toolkit
after when NVDA 2019.-4 is released, unless the api o
NVDA has important deletions and NV Access incrennt the
value for the bac compatibility.
>>
>>
>> Enviado desde mi iPhone
>>
>>> El 28 sept 2019, a las 17:53, Andy B.
<sonfire11@... <mailto:sonfire11@...>> escribió:
>>>
>>> Why don't we change lastTestedNVDAVersion to
maximumSupportedNVDABersion to align with the
minimumSupportedNVDAVersion flag? The purpose is much
more clearly stated and is self-documenting. After all,
the last tested NVDA version is nothing more than the
upper limit on what the add-on author is willing to support.
>>>
>>>
>>> Andy Borka
>>> Accessibility Engineer
>>>
>>> -----Original Message-----
>>> From: nvda-devel@groups.io
<mailto:nvda-devel@groups.io> <nvda-devel@groups.io
<mailto:nvda-devel@groups.io>> On Behalf Of Noelia
>>> Ruiz
>>> Sent: Saturday, September 28, 2019 2:39 AM
>>> To: nvda-devel@groups.io <mailto:nvda-devel@groups.io>
>>> Subject: Re: [nvda-devel] Proposal: enforce minimum
and last tested NVDA version flags for add-on reviews
and certification upon 2019.3 release?
>>>
>>> Hi, imo, the expression used (last NVDA version
tested) is right, since this is not speaking about
specific versions of NVDA (nor last stable, nor beta or
last master or future). Instead, this reflects what
version has been tested. This may be a superold version
of NVDA, and in this case, when you update to a version
of NVDA not tested with an installed add-on, the add-on
can be blocked so that it doesn't cause instability in
the system.
>>>
>>> Also, sometimes, for practical reasons, authors say
that an add-on has been tested with the upcoming version
of NVDA and NVDA allows it just for testing add-ons in
development, so that they can be ready when the next
stable version of NVDA is released.
>>> Cheers
>>>
>>>
>>>> El 27/09/2019 a las 20:02, Travis Siegel escribió:
>>>> I'm still curious how the last tested version of a
plugin can be for
>>>> a version that hasn't even been released yet.
>>>>
>>>> I argued during the whole invention of this scheme
that calling it
>>>> last tested version was a bad idea, but
apparently, nobody payed
>>>> attention. I strongly urge that this parameter get
a new name, since
>>>> it's impossible to test a nonexistent version for
compatibility.
>>>> Call it expected compatible, or next release
version or something,
>>>> but not last tested version, since there hasn't
been any testing for
>>>> a release that doesn't exist.  You're only
confusing people.
>>>>
>>>>
>>>>> On 9/26/2019 6:53 PM, Joseph Lee wrote:
>>>>>
>>>>> Hi all,
>>>>>
>>>>> Yesterday there was a discussion on NVDA add-ons
list about Python
>>>>> 3 compatibility of add-ons and state of their
manifests. While some
>>>>> add-ons do declare 2019.3 as last tested version
and are indeed
>>>>> Python
>>>>> 3 ready, others such as Classic Selection is
Python 3 ready from
>>>>> source code level but not via the manifest. This
is a point of
>>>>> confusion because when installing such add-ons in
NVDA 2019.3, NVDA
>>>>> will say the add-on in question is incompatible.
>>>>>
>>>>> Thus I would like to propose that when reviewing
add-ons for
>>>>> inclusion in community add-ons website, reviewers
should perform
>>>>> manifest checks. If an add-on does use features
from a given
>>>>> release yet the manifest says otherwise, authors
should be notified
>>>>> and corrections must be made before the add-on is
included. In case
>>>>> of an add-on declaring last tested version as
something yet the
>>>>> source code says something else, it still needs
to be tested in the
>>>>> last tested version (and newer) specified so the
source code and the manifest is in sync.
>>>>> This check also helps authors realize that they
do need to update
>>>>> their manifests from time to time to keep up with
NVDA changes,
>>>>> because ideally, reviewers should not be the ones
telling authors
>>>>> to update their manifests unless it is necessary
to do so.
>>>>>
>>>>> As for when to enforce compatibility range check
(minimum version
>>>>> <= current version <= last tested version), I
propose January 1,
>>>>> 2020 as start date. This allows people to prepare
for both NVDA
>>>>> 2019.3 and this check at the same time. As for
what to do with
>>>>> add-ons that does not include minimum and last
tested version flags
>>>>> in their manifests, I think a notice should be
sent to authors so
>>>>> they can take care of it as soon as possible, or
if the author
>>>>> isn’t willing, NVDA community should do it (for
Classic Selection,
>>>>> it is Tyler Spivey who should be contacted).
>>>>>
>>>>> If this proposal is adopted by the community, I
propose sending out
>>>>> a notice about it several times:
>>>>>
>>>>>    * November 1, 2019: giving authors 60 days to
change their
>>>>> manifests
>>>>>      and do something about Python 3.
>>>>>    * One of the 2019.3 betas if NV Access agrees
with this proposal.
>>>>>    * NVDA 2019.3 RC and stable release: to remind
users about this
>>>>>      change if adopted.
>>>>>
>>>>> Thanks.
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Joseph
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>>
>>
>>
>>
>
>
>
>
>
>
____
-- ____
Derek Riemer
Improving the world one byte at a time! ⠠⠊⠍⠏⠗⠕⠧⠬ ⠮ ⠸⠺ ⠐⠕
⠃⠽⠞⠑ ⠁⠞ ⠁ ⠐⠞⠖
•    Accessibility enthusiast.
•    Proud user of the NVDA screen reader.
•    Open source enthusiast.
•    Skier.
•    Personal website: https://derekriemer.com
____
-- ____
Derek Riemer
Improving the world one byte at a time! ⠠⠊⠍⠏⠗⠕⠧⠬ ⠮ ⠸⠺ ⠐⠕ ⠃⠽⠞⠑ ⠁⠞
⠁ ⠐⠞⠖
•    Accessibility enthusiast.
•    Proud user of the NVDA screen reader.
•    Open source enthusiast.
•    Skier.
•    Personal website: https://derekriemer.com
____
__
--
Derek Riemer
Improving the world one byte at a time!        ⠠⠊⠍⠏⠗⠕⠧⠬ ⠮ ⠸⠺ ⠐⠕ ⠃⠽⠞⠑ ⠁⠞ ⠁ ⠐⠞⠖
•    Accessibility enthusiast.
•    Proud user of the NVDA screen reader.
•    Open source enthusiast.
•    Skier.
•    Personal website: https://derekriemer.com

Join nvda-devel@groups.io to automatically receive all group messages.