Re: Proposal: enforce minimum and last tested NVDA version flags for add-on reviews and certification upon 2019.3 release?

Andy B.
 

I think maximum NVDA version should be required because we are talking about a range of NVDA versions. It makes it clear to developers and users, if included in add-on manager, that this add-on supports the stated NVDA versions, even if they are alpha/beta builds. Now, when stating a nonexistent version of NVDA, such as alpha, beta, or threshold builds, maximum NVDA version should indicate an api version such as 2019.3, but also include a prerelease branch such as threshold, alpha, beta, or rc. After this, we would have versions stated such as minimumNVDAVersion = 2019.3-alpha, or maximumNVDAVersion = 2019.3-beta1.

 

Andy Borka

Accessibility Engineer

 

From: nvda-devel@groups.io <nvda-devel@groups.io> On Behalf Of Noelia Ruiz
Sent: Sunday, September 29, 2019 10:58 PM
To: nvda-devel@groups.io
Subject: Re: [nvda-devel] Proposal: enforce minimum and last tested NVDA version flags for add-on reviews and certification upon 2019.3 release?

 

Regarding to the enforzement and the original question, I wass missing, until I shared the code here, that for NVDA the last tested version is always present and not an empty value, since it defaults to the current NVDA version when the manifest doesnt declare it. So the important think is that this doesnt break NVDA or system stability. If an author doesnt want to specify last tested version and the add-on is fully compatible, we cannot enforze this since user experience pass. But if alpha versions of NVDA or the last stable version is incompatible, since the test version is too low and the add-on will be blocked, or even worse, if the author doesnt specify a tested version and for this reason NVDA considers that the add-on is good but in fact it break things, review should fail and the ad-on shouldnt remain in the stable section of the website.

Other question would be, as suggested by Andy (not directly), clarify, in addons manager, the maximum version supported according to the last tested value, for instance: last test version: 2019.-3 (max suported version: 2019.-4 of course when available). Not sure if it would be needed.

Cheers

 

Enviado desde mi iPhone


El 30 sept 2019, a las 2:40, Joseph Lee <joseph.lee22590@...> escribió:

Hi,

The plan (which is really a separate thread at this point) involves sending current NVDA version, along with add-on names, their versions and update channel metadata. In return the client will get update info for add-ons deemed compatible based on the NVDA version sent to the server.

Cheers,

Joseph

 

From: nvda-devel@groups.io <nvda-devel@groups.io> On Behalf Of derek riemer
Sent: Sunday, September 29, 2019 5:34 PM
To: nvda-devel@groups.io
Subject: Re: [nvda-devel] Proposal: enforce minimum and last tested NVDA version flags for add-on reviews and certification upon 2019.3 release?

 

That makes sense, I'm primarily concerned that users will have to manually update all addons every version, even if they literally didn't change. I think we should do the lastTested thing serverSide, or at least offer a mechanism to update that on the server eventually, so users don't need to download the exactt same payload with just a different tested version if that version is already tested..

 

On Sun, Sep 29, 2019 at 6:32 PM Joseph Lee <joseph.lee22590@...> wrote:

Hi,

At least Add-on Updater enforces this check for now unless the strategy changes.

Cheers,

Joseph

 

From: nvda-devel@groups.io <nvda-devel@groups.io> On Behalf Of derek riemer
Sent: Sunday, September 29, 2019 4:26 PM
To: nvda-devel@groups.io
Subject: Re: [nvda-devel] Proposal: enforce minimum and last tested NVDA version flags for add-on reviews and certification upon 2019.3 release?

 

I really don't think the last tested bit should be enforced until we have an updator.

 

On Sun, Sep 29, 2019 at 10:28 AM Noelia Ruiz <nrm1977@...> wrote:

El 29/09/2019 a las 18:18, Joseph Lee escribió:
> Hi,
> Actually, you do need to change last tested version once 2019.4 comes out too, because that's how add-ons manager will check for compatibility


I think this is not right (if I'm wrong I will apologize :)
The add-ons manager (or NVDA doesn't check for compatibility directly
looking at last tested version, but to the api containing the last
tested version. This is a way to let authors precissely not to change
this flag each time NVDA has a new stable release. So, when NVDA 2019.4
is out, we don't need to change this flag unless NVDA has important
changes. NVDA 2019.3 is an exception due to Python 3 version and speech
refactor.
But generally this flag, if used, don't need to be changed.

Cheers



El 29/09/2019 a las 18:18, Joseph Lee escribió:
> Hi,
> Actually, you do need to change last tested version once 2019.4 comes out too, because that's how add-ons manager will check for compatibility (some folks found out the hard way when attempting to install an add-on that wasn't marked for 2019.3). The purpose of that flag is for add-on authors to communicate that they did test their add-ons in a specific NVDA release (or a development snapshot for it), and ideally, source code should match what the manifest claims. By doing so, it reminds authors that they do need to keep up with changes introduced in NVDA Core - the minimum flag won't change much, but last tested flag will from time to time.
> As for communicating the purpose of that flag to users: I can see confusion. Calling it "last tested version" when in fact it specifies maximum compatible version isn't a good communication to begin with, because latest compatible and last tested versions are likely to be different (for my add-ons at least, they are really the same, although in public documentation I always specify latest stable version for maximum compatibility range because things under development can change rapidly; but then again, for my add-ons, I release development snapshots for some add-ons to keep up with alpha changes, or as Eric Raymond once advised, "release early, release often"). I think what can help might be to listening to public definition of this flag, and if they are not in line with the actual purpose of it, to educate users as to what it actually means; if there is indeed difference, perhaps come up with a compromise or think about this flag again in a few months.
> Cheers,
> Joseph
>
> -----Original Message-----
> From: nvda-devel@groups.io <nvda-devel@groups.io> On Behalf Of Noelia Ruiz
> Sent: Sunday, September 29, 2019 8:54 AM
> To: nvda-devel@groups.io
> Subject: Re: [nvda-devel] Proposal: enforce minimum and last tested NVDA version flags for add-on reviews and certification upon 2019.3 release?
>
> Hi, if you want to build versions for testers you have to change the last tested version in the case of NVDA 2019.3 due to deletions in the API, but probably, when NVDA 2019.4 pre-releases or stable are published, you will not need to change the last tested version. So, the purpose of this field in the manifest maybe clarified in the add-on template, but for me the name is right since this doesn't mean the maximum version that the add-on is intended to support, nor the last version of NVDA beingbuild, nor the last stable.
> Cheers
>
>
> El 29/09/2019 a las 14:02, Andy B. escribió:
>> Well, not really. If I want to release add-on builds for testers, I have to set the last tested version flag to the current prerelease of NVDA. This isn't a problem, but the flag's name is misleading at best. Minimum version and maximum version as I mentioned before, are self-documenting flags. There is no confusion that you have to run at least NVDA 2019.1.1, and nothing later than 2019.3 for this add-on to work.
>> Keep in mind that prospective add-on authors will take a look at some of their favorite add-on's manifest and source if they are interested in trying add-on development. Doing this makes sense because the user is already familiar with how the add-on works, so they can follow along with their experience while looking at the source. Unless a user is already somewhat buried in NVDA development, minimum version makes sense, but last tested version does not. Therefore, last tested version should be changed to maximum version with a comment in the manifest that maximum version is either equal to or greater than minimum version, and that maximum version may include prerelease versions of NVDA.
>>
>> # The lowest supported version of NVDA.
>> minimumNVDAVersion = 2019.1.1 # DTK does not support 2019.1 because it was initially created during 2019.1.1 release.
>> maximumNVDAVersion = 2019.3 # The current alpha prelease cycle.
>>
>> I vote to have the flags changed to represent the above example.
>>
>> Andy Borka
>> Accessibility Engineer
>>
>> -----Original Message-----
>> From: nvda-devel@groups.io <nvda-devel@groups.io> On Behalf Of Noelia
>> Ruiz
>> Sent: Saturday, September 28, 2019 2:06 PM
>> To: nvda-devel@groups.io
>> Subject: Re: [nvda-devel] Proposal: enforce minimum and last tested NVDA version flags for add-on reviews and certification upon 2019.3 release?
>>
>> Imo the current name is right, since the latest tested version can be minor of the latest supported version. For example, you probably will not to update the tested version in the manifest for developer toolkit after when NVDA 2019.-4 is released, unless the api o NVDA has important deletions and NV Access incrennt the value for the bac compatibility.
>>
>>
>> Enviado desde mi iPhone
>>
>>> El 28 sept 2019, a las 17:53, Andy B. <sonfire11@...> escribió:
>>>
>>> Why don't we change lastTestedNVDAVersion to maximumSupportedNVDABersion to align with the minimumSupportedNVDAVersion flag? The purpose is much more clearly stated and is self-documenting. After all, the last tested NVDA version is nothing more than the upper limit on what the add-on author is willing to support.
>>>
>>>
>>> Andy Borka
>>> Accessibility Engineer
>>>
>>> -----Original Message-----
>>> From: nvda-devel@groups.io <nvda-devel@groups.io> On Behalf Of Noelia
>>> Ruiz
>>> Sent: Saturday, September 28, 2019 2:39 AM
>>> To: nvda-devel@groups.io
>>> Subject: Re: [nvda-devel] Proposal: enforce minimum and last tested NVDA version flags for add-on reviews and certification upon 2019.3 release?
>>>
>>> Hi, imo, the expression used (last NVDA version tested) is right, since this is not speaking about specific versions of NVDA (nor last stable, nor beta or last master or future). Instead, this reflects what version has been tested. This may be a superold version of NVDA, and in this case, when you update to a version of NVDA not tested with an installed add-on, the add-on can be blocked so that it doesn't cause instability in the system.
>>>
>>> Also, sometimes, for practical reasons, authors say that an add-on has been tested with the upcoming version of NVDA and NVDA allows it just for testing add-ons in development, so that they can be ready when the next stable version of NVDA is released.
>>> Cheers
>>>
>>>
>>>> El 27/09/2019 a las 20:02, Travis Siegel escribió:
>>>> I'm still curious how the last tested version of a plugin can be for
>>>> a version that hasn't even been released yet.
>>>>
>>>> I argued during the whole invention of this scheme that calling it
>>>> last tested version was a bad idea, but apparently, nobody payed
>>>> attention. I strongly urge that this parameter get a new name, since
>>>> it's impossible to test a nonexistent version for compatibility.
>>>> Call it expected compatible, or next release version or something,
>>>> but not last tested version, since there hasn't been any testing for
>>>> a release that doesn't exist.  You're only confusing people.
>>>>
>>>>
>>>>> On 9/26/2019 6:53 PM, Joseph Lee wrote:
>>>>>
>>>>> Hi all,
>>>>>
>>>>> Yesterday there was a discussion on NVDA add-ons list about Python
>>>>> 3 compatibility of add-ons and state of their manifests. While some
>>>>> add-ons do declare 2019.3 as last tested version and are indeed
>>>>> Python
>>>>> 3 ready, others such as Classic Selection is Python 3 ready from
>>>>> source code level but not via the manifest. This is a point of
>>>>> confusion because when installing such add-ons in NVDA 2019.3, NVDA
>>>>> will say the add-on in question is incompatible.
>>>>>
>>>>> Thus I would like to propose that when reviewing add-ons for
>>>>> inclusion in community add-ons website, reviewers should perform
>>>>> manifest checks. If an add-on does use features from a given
>>>>> release yet the manifest says otherwise, authors should be notified
>>>>> and corrections must be made before the add-on is included. In case
>>>>> of an add-on declaring last tested version as something yet the
>>>>> source code says something else, it still needs to be tested in the
>>>>> last tested version (and newer) specified so the source code and the manifest is in sync.
>>>>> This check also helps authors realize that they do need to update
>>>>> their manifests from time to time to keep up with NVDA changes,
>>>>> because ideally, reviewers should not be the ones telling authors
>>>>> to update their manifests unless it is necessary to do so.
>>>>>
>>>>> As for when to enforce compatibility range check (minimum version
>>>>> <= current version <= last tested version), I propose January 1,
>>>>> 2020 as start date. This allows people to prepare for both NVDA
>>>>> 2019.3 and this check at the same time. As for what to do with
>>>>> add-ons that does not include minimum and last tested version flags
>>>>> in their manifests, I think a notice should be sent to authors so
>>>>> they can take care of it as soon as possible, or if the author
>>>>> isn’t willing, NVDA community should do it (for Classic Selection,
>>>>> it is Tyler Spivey who should be contacted).
>>>>>
>>>>> If this proposal is adopted by the community, I propose sending out
>>>>> a notice about it several times:
>>>>>
>>>>>    * November 1, 2019: giving authors 60 days to change their
>>>>> manifests
>>>>>      and do something about Python 3.
>>>>>    * One of the 2019.3 betas if NV Access agrees with this proposal.
>>>>>    * NVDA 2019.3 RC and stable release: to remind users about this
>>>>>      change if adopted.
>>>>>
>>>>> Thanks.
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Joseph
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>>
>>
>>
>>
>
>
>
>
>
>




--

Derek Riemer
Improving the world one byte at a time!        ⠠⠊⠍⠏⠗⠕⠧⠬ ⠸⠺ ⠐⠕ ⠃⠽⠞⠑ ⠁⠞ ⠐⠞⠖
•    Accessibility enthusiast.
•    Proud user of the NVDA screen reader.
•    Open source enthusiast.
•    Skier.

•    Personal website: https://derekriemer.com





--

Derek Riemer
Improving the world one byte at a time!        ⠠⠊⠍⠏⠗⠕⠧⠬ ⠸⠺ ⠐⠕ ⠃⠽⠞⠑ ⠁⠞ ⠐⠞⠖
•    Accessibility enthusiast.
•    Proud user of the NVDA screen reader.
•    Open source enthusiast.
•    Skier.

•    Personal website: https://derekriemer.com




Join nvda-devel@groups.io to automatically receive all group messages.