-----Original Message-----
From: gate-users@groups.io <gate-users@groups.io> On Behalf Of Genevieve M Gorrell
Sent: 13 August 2018 09:13
To: gate-users@groups.io
Subject: Re: [gate-users] Leniant / Average mode in corpus benchmark tool?

Hi.
Do you need to use Corpus Benchmark, or can you get what you need from Corpus QA? Corpus Benchmark is pretty old technology and we're supporting/teaching Corpus QA preferentially.
Genevieve


On 9 August 2018 at 16:29, <a.s.lacey@...> wrote:

Hi - AnnotatioDiff tool has 3 measures of accuracy - strict, lenient
and average, but only the strict measure appears to be shown in the
corpus benchmark tool.

I get a line like: <TD>0</TD> corresponding to <TD><B>Partially
Correct</B></TD>, even though there are many partial matches.

I note there is another line in the output: <BR>No Token annotations
to count words in the document.

Do partial matches in the corpus benchmark tool rely on the presence
of Token annotations in both annotation sets to be compared as
specified in the corpus_tool.properties?

Thanks

--
Genevieve Gorrell
Research Associate, Department of Computer Science University of Sheffield, UK http://www.dcs.shef.ac.uk/~genevieve/

On 9 August 2018 at 16:29, <a.s.lacey@...> wrote:

Hi - AnnotatioDiff tool has 3 measures of accuracy - strict, lenient and
average, but only the strict measure appears to be shown in the corpus
benchmark tool.

I get a line like: <TD>0</TD> corresponding to <TD><B>Partially
Correct</B></TD>, even though there are many partial matches.

I note there is another line in the output: <BR>No Token annotations to
count words in the document.

Do partial matches in the corpus benchmark tool rely on the presence of
Token annotations in both annotation sets to be compared as specified in the
corpus_tool.properties?

Thanks