#729 formatted -X output in an easier way to parse by phonemizer

So the whole idea here is that we need to be able to map phonemized words back to their original text. When running -X with espeak, the original output looked like this:

tedi@Tedis-MacBook-Pro bin % ./espeak-ng  "this is a to be test" -X
Found: 'this' [DIs]  $u+ $strend $verbsf $nounf
Found: 'is' [Iz]   $pastf $only
Flags:  a   $nounf
Translate 'a'
  1	a        [a]
 26	_) a (_  [a#]

Found: 'to be
' [t@bi]   $pastf
Translate 'test'
  1	t        [t]

  1	e        [E]

  1	s        [s]

  1	t        [t]

 DIs Iz a# t@bi t'Est
tedi@Tedis-MacBook-Pro bin %

The new output will look like this:

tedi@Tedis-MacBook-Pro bin % ./espeak-ng "this is a to be test" -X
this~|||~DIs
is~|||~Iz
to be~|||~t@bi

The modified output is much easier to parse by the python wrapper contained inside Phonemizer because Phonemizer can now read in each line from the espeak-ng standard output, then split each line using the ~|||~ delimiter (e.g. string.split("~|||~") ). The text before the the delimiter is input text, while the text after the delimiter is ASCII IPA phonemized text, therefore the goal of being able to map phonemized text back to its original text has been achieved.

This is required for SSML processing inside Resembletron. The reason being that users apply tags to whole text words, the text words are then phonemized - if there is a 1 to 1 relationship between # of text words to # of phonemized words, then the tags can be translated easily. (E.g. hello, after phonemization becomes həlˈəʊ). Now assume the case where there is not a 1 to 1 relationship between # of text words to # of phonemised words, then a mapping of phonemized words to text words is required (which is now possible with this modification) For example, the words "to be" become one phonemized word "təbi" in certain sentences - with this word to phonemized word mapping, we can identify the original phonemized text.

[espeak-ng:master] New Comment on Pull Request #729 formatted -X output in an easier way to parse by phonemizer
By TediPapajorgji:

Sorry.... wrong repository...

[espeak-ng/espeak-ng] Pull request closed by TediPapajorgji:

#729 formatted -X output in an easier way to parse by phonemizer

So the whole idea here is that we need to be able to map phonemized words back to their original text. When running -X with espeak, the original output looked like this:

tedi@Tedis-MacBook-Pro bin % ./espeak-ng  "this is a to be test" -X
Found: 'this' [DIs]  $u+ $strend $verbsf $nounf
Found: 'is' [Iz]   $pastf $only
Flags:  a   $nounf
Translate 'a'
  1	a        [a]
 26	_) a (_  [a#]

Found: 'to be
' [t@bi]   $pastf
Translate 'test'
  1	t        [t]

  1	e        [E]

  1	s        [s]

  1	t        [t]

 DIs Iz a# t@bi t'Est
tedi@Tedis-MacBook-Pro bin %

The new output will look like this:

tedi@Tedis-MacBook-Pro bin % ./espeak-ng "this is a to be test" -X
this~|||~DIs
is~|||~Iz
to be~|||~t@bi

The modified output is much easier to parse by the python wrapper contained inside Phonemizer because Phonemizer can now read in each line from the espeak-ng standard output, then split each line using the ~|||~ delimiter (e.g. string.split("~|||~") ). The text before the the delimiter is input text, while the text after the delimiter is ASCII IPA phonemized text, therefore the goal of being able to map phonemized text back to its original text has been achieved.

This is required for SSML processing inside Resembletron. The reason being that users apply tags to whole text words, the text words are then phonemized - if there is a 1 to 1 relationship between # of text words to # of phonemized words, then the tags can be translated easily. (E.g. hello, after phonemization becomes həlˈəʊ). Now assume the case where there is not a 1 to 1 relationship between # of text words to # of phonemised words, then a mapping of phonemized words to text words is required (which is now possible with this modification) For example, the words "to be" become one phonemized word "təbi" in certain sentences - with this word to phonemized word mapping, we can identify the original phonemized text.

[espeak-ng/espeak-ng] Pull request updated by TediPapajorgji:

#729 formatted -X output in an easier way to parse by phonemizer

WRONG REPOSITORY SORRY!

So the whole idea here is that we need to be able to map phonemized words back to their original text. When running -X with espeak, the original output looked like this:

tedi@Tedis-MacBook-Pro bin % ./espeak-ng  "this is a to be test" -X
Found: 'this' [DIs]  $u+ $strend $verbsf $nounf
Found: 'is' [Iz]   $pastf $only
Flags:  a   $nounf
Translate 'a'
  1	a        [a]
 26	_) a (_  [a#]

Found: 'to be
' [t@bi]   $pastf
Translate 'test'
  1	t        [t]

  1	e        [E]

  1	s        [s]

  1	t        [t]

 DIs Iz a# t@bi t'Est
tedi@Tedis-MacBook-Pro bin %

The new output will look like this:

tedi@Tedis-MacBook-Pro bin % ./espeak-ng "this is a to be test" -X
this~|||~DIs
is~|||~Iz
to be~|||~t@bi

The modified output is much easier to parse by the python wrapper contained inside Phonemizer because Phonemizer can now read in each line from the espeak-ng standard output, then split each line using the ~|||~ delimiter (e.g. string.split("~|||~") ). The text before the the delimiter is input text, while the text after the delimiter is ASCII IPA phonemized text, therefore the goal of being able to map phonemized text back to its original text has been achieved.

This is required for SSML processing inside Resembletron. The reason being that users apply tags to whole text words, the text words are then phonemized - if there is a 1 to 1 relationship between # of text words to # of phonemized words, then the tags can be translated easily. (E.g. hello, after phonemization becomes həlˈəʊ). Now assume the case where there is not a 1 to 1 relationship between # of text words to # of phonemised words, then a mapping of phonemized words to text words is required (which is now possible with this modification) For example, the words "to be" become one phonemized word "təbi" in certain sentences - with this word to phonemized word mapping, we can identify the original phonemized text.

[espeak-ng:master] By spresse1 <steve@...>:
1a5df9ce04aa: Change the name of the ' character to "tick", rather than "quote", as this was difficult to differentiate from """, which reads as "quotes"

Modified: dictsource/en_list

[espeak-ng:master] By Valdis Vitolins <valdis.vitolins@...>:
f38ec9d1d718: Update changelog

Modified: CHANGELOG.md

[espeak-ng/espeak-ng] Pull request closed by valdisvi:

#726 #725: ' (quote) versus " (quotes) is almost impossible to audibly differentiate

Users of screen readers (eg: Orca, NVDA, JAWS) and other TTS assistive technologies rely on accurate and easy-to-understand reading of the text. One context in which this becomes especially critical is systems administration and software development, where TTS users need to be able to clearly and easily differentiate types of punctuation with minimal effort in the midst of reading a larger portion of text. For example, the two characters have significantly different meanings in shells and in some programming languages.

This PR changes the name of the quote (') character to "tick", as this was difficult to differentiate from """, which reads as "quotes"

[espeak-ng:master] New Comment on Pull Request #726 #725: ' (quote) versus " (quotes) is almost impossible to audibly differentiate
By valdisvi:

It seems ok to me and hopefully @rhdunn, who is maintaining English will not object. Close issue if that is it.

[espeak-ng:master] By Valdis Vitolins <valdis.vitolins@...>:
3f3f17f84355: Fix #715: Need better error messages if MBROLA engine or voice files are missing

Modified: src/libespeak-ng/mbrowrap.c
Modified: src/libespeak-ng/synth_mbrola.c

[espeak-ng:master] By Valdis Vitolins <valdis.vitolins@...>:
c13b3e534b01: Add parameter `all` for --voices to show all voices

Update documentation to show already existing features and for
newly introduced `all` parameter.

Modified: src/espeak-ng.1.ronn
Modified: src/libespeak-ng/voices.c

[espeak-ng:master] By Valdis Vitolins <valdis.vitolins@...>:
6fe3aba5a2be: Merge pull request #717

Modified: src/espeak-ng.1.ronn
Modified: src/libespeak-ng/mbrowrap.c
Modified: src/libespeak-ng/synth_mbrola.c
Modified: src/libespeak-ng/voices.c

[espeak-ng:master] Issue #715 Need better error messages if MBROLA engine or voice files are missing closed by valdisvi.

[espeak-ng/espeak-ng] Pull request closed by valdisvi:

#717 Changes in espeak-ng core — improved MBROLA error handling, `all` parameter to list all voices

Reece, please review these changes and merge them if implementation is acceptable. Thanks!

[espeak-ng:master] By Valdis Vitolins <valdis.vitolins@...>:
875ef11d7756: Fix bug #703: update version number for shared library

When update next version number, look at differences for e.g. b47a75f4 commit
and update following files:
- CHANGELOG.md
- android/jni/include/config.h
- configure.ac
- src/windows/config.h
- Makefile.am

Modified: Makefile.am

[espeak-ng:master] Issue #703 Probably misconfigured Makefile.am for release closed by valdisvi.

Actually it was told already, but less explicitly. I hope fix in 541cd90 make it more clear.

[espeak-ng:master] Issue #686 inform that --stdin flag reads multi-line text and then speaks it after closed by sthibaul.

1 New Commit:

[espeak-ng:master] By Valdis Vitolins <valdis.vitolins@...>:
541cd906f762: Issue #686: make description of --stdin more clear

Modified: src/espeak-ng.1.ronn
Modified: src/espeak-ng.c

PR#726 was merged, this issue is resolved. Thank you!

[espeak-ng:master] Issue #725 ' (quote) versus " (quotes) is almost impossible to audibly differentiate closed by spresse1.

[espeak-ng:master] By Valdis Vitolins <valdis.vitolins@...>:
cf4809a1e573: Fix bug #706: pronunciation error with abbreviations followed by 's

Modified: src/libespeak-ng/translate.c
Modified: tests/translate.test

[espeak-ng/espeak-ng] Pull request closed by rhdunn:

#730 Fix bug #706: pronunciation error with abbreviations followed by 's

@rhdunn, please review and merge fix for #706 I also made it work properly for other languages where, plural/genitive with ...s or ...'s doesn't have meaning (i.e. ..s is considered part of the root word) and updated tests.

[espeak-ng:master] New Comment on Pull Request #730 Fix bug #706: pronunciation error with abbreviations followed by 's
By rhdunn:

Reviewed and merged. Thanks for the fix.

[espeak-ng:master] By Valdis Vitolins <valdis.vitolins@...>:
d17a5ecc534b: Fix issue #546: add initial support for Ukrainian language

Added: dictsource/uk_list
Added: dictsource/uk_rules
Added: espeak-ng-data/lang/zle/uk
Modified: CHANGELOG.md
Modified: Makefile.am
Modified: phsource/phonemes
Modified: src/libespeak-ng/tr_languages.c

[espeak-ng:master] New Comment on Issue #546 Ukrainian Support
By valdisvi:

@MissValeska, I have added initial support for Ukrainian language. You can test it online and improve it yourself by following contributor guide.

@salami5 , I have added initial support for Quechua language. You can test it online and improve it yourself by following contributor guide.

1 New Commit:

[espeak-ng:master] By Valdis Vitolins <valdis.vitolins@...>:
da359384face: qu: fix issue #561: add initial support for Quechua language

Added: dictsource/qu_list
Added: dictsource/qu_rules
Added: espeak-ng-data/lang/qu
Added: phsource/ph_quechua
Modified: CHANGELOG.md
Modified: Makefile.am
Modified: phsource/phonemes

@ian4uh, I have added initial support for Hawaiian language. You can test it online and improve it yourself by following contributor guide.

[espeak-ng:master] Issue #578 Hawaiian Language Support closed by ian4uh.

4 New Commits:

[espeak-ng:master] By Valdis Vitolins <valdis.vitolins@...>:
a10a6b71cd84: docs: add links to action steps

Modified: docs/add_language.md

[espeak-ng:master] By Valdis Vitolins <valdis.vitolins@...>:
54723e8492d9: docs: move creation of language definition up

Modified: docs/add_language.md

[espeak-ng:master] By Valdis Vitolins <valdis.vitolins@...>:
7e69e3358871: docs: allow to call any instance of espeak-ng

Modified: docs/add_language.md

[espeak-ng:master] By Valdis Vitolins <valdis.vitolins@...>:
0ee91aaa064b: haw: fix issue #578: add initial support for Hawaiian language

Added: dictsource/haw_list
Added: dictsource/haw_rules
Added: espeak-ng-data/lang/map/haw
Added: phsource/ph_hawaiian
Modified: CHANGELOG.md
Modified: Makefile.am
Modified: docs/languages.md
Modified: phsource/phonemes

[espeak-ng:master] By Valdis Vitolins <valdis.vitolins@...>:
fc886521943f: docs: update total number

Modified: docs/languages.md

[espeak-ng:master] By Valdis Vitolins <valdis.vitolins@...>:
af7f4e3bcf78: docs: update links to supported lanugages

Modified: README.md

[espeak-ng:master] By Valdis Vitolins <valdis.vitolins@...>:
851c3f2cf5ed: docs: remove link from introductory part

It is provided in linked docs/mbrola.md page.

Modified: README.md

[espeak-ng:master] By Valdis Vitolins <valdis.vitolins@...>:
f5b3734a4652: docs: update size of package

Modified: README.md

[espeak-ng:master] Issue #691 Differences between bashkir and tatar? closed by hozosch.

[espeak-ng:master] Issue #705 --voices option doesn't show mbrola voices closed by hadware.

[espeak-ng:master] New Comment on Issue #566 mbrola issues
By valdisvi:

With 1.50 this is fixed.

[espeak-ng:master] Issue #566 mbrola issues closed by sthibaul.

[espeak-ng:master] New Comment on Issue #566 mbrola issues
By valdisvi:

With 1.50 release this is fixed.

[espeak-ng:master] New Comment on Issue #448 Make a release to fix #338
By valdisvi:

With 1.50 this should be fixed.

[espeak-ng:master] Issue #448 Make a release to fix #338 closed by jiangtiandao.

[espeak-ng:master] New Comment on Issue #144 Creating an Ukrainian language for Espeak NG
By valdisvi:

@Giovani93, Ukrainian language is now supported in development version of eSpeak NG. You can test it online and improve it yourself by following contributor guide.

[espeak-ng:master] Issue #144 Creating an Ukrainian language for Espeak NG closed by Giovani93.