Grammar Representation Requirements for Voice Markup Languages

Abstract

The W3C Voice Browser working group aims to develop specifications to enable access to the Web using spoken interaction. This document is part of a set of requirements studies for voice browsers, and provides details of the requirements for grammars for speech recognition.

Status of this document

This document describes the requirements for grammars used for speech recognition, as a precursor to starting work on specifications. Related requirement drafts are linked from the introduction. The requirements are being released as working drafts but are not intended to become proposed recommendations.

This specification is a Working Draft of the Voice Browser working group for review by W3C members and other interested parties. This is the first public version of this document. It is a draft document and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use W3C Working Drafts as reference material or to cite them as other than "work in progress".

Publication as a Working Draft does not imply endorsement by the W3C membership, nor of members of the Voice Browser working groups. This is still a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite W3C Working Drafts as other than "work in progress."

This document has been produced as part of the W3C Voice Browser Activity, following the procedures set out for the W3C Process. The authors of this document are members of the Voice Browser Working Group. This document is for public review. Comments should be sent to the public mailing list <www-voice@w3.org> (archive) by 14th January 2000.

A list of current W3C Recommendations and other technical documents can be found at http://www.w3.org/TR.

0. Introduction

The main goal of this subgroup is to define a speech recognition grammar specification language that will be generally useful across a variety of speech platforms used in the context of a dialog and synthesis markup environment. The process will consist of four main steps:

1. establish an appropriate set of requirements for grammar specifications
2. evaluate existing grammar languages for satisfaction of requirements
3. settle upon a language specification or modify as necessary
4. deliver a specific language proposal to the full W3C working group.

The scope of issues discussed includes semantics and contexts as well as natural language syntax. Therefore the activities of the Grammar Representation Subgroup are to be coordinated with the activities of both the Natural Language Subgroup and the Dialog Subgroup.

The following eight main topic areas have been identified as important:

1. Natural Language Syntax
2. Large Vocabulary/Dictation
3. Grammar Contexts
4. Semantics
5. Post-Processing Issues
6. Efficiency Issues
7. XML Compatibility
8. Grammar Specification Language Syntax

Each topic area consists of several issues that will be discussed in detail in the following sections. Example specifications presented in this document are for illustration purposes only and do not necessarily represent recommended formats.

0.1 Terminology

0.2 Symbology

Regular Expressions:

Proposed Markup Tags (so far):

0.3 Priorities

The following priorities are used to indicate the level of importance of each requirement in this document.

0.4 Subgroup Coordination

The requirements and specification of the Grammar Representation Subgroup will be coordinated with overlapping requirements and specification of the Natural Language, Dialogue, and Universal Access subgroups.

1. Natural Language Syntax

Specification of the natural language syntax in the grammar representation shall conform to the following requirements:

1.1 Context-Free Grammar Support (must specify)

The grammar representation must support the definition of a Context-Free Grammar (CFG) and, by subsumption, a Finite-State Grammar (FSG). Some platforms will not support recursive rules so content developers will need to be aware of specific product limitations.

1.2 CFG Specification (must specify)

CFG's must be represented by specification in a well known format. Each CFG rule will be a regular expression.

An example well-known CFG representation is Backus-Naur Format (BNF) and a regular expression syntax is the well known {expr}?, {expr}*, and {expr}+ for optional, zero or more, and one or more expressions, respectively, and the use of { expr | expr } for exclusive OR.

1.3 N-Gram Grammar Enabled (should specify)

The grammar representation should enable the definition of an N-Gram Grammar (NGG).

An example format is described in the Appendix.

1.4 Out-of-Vocabulary Words (must specify)

The grammar representation must support the processing of out-of-vocabulary (OOV) words and define a method for representing such words. The content developer can specify the action to be taken upon encountering OOV words.

1.5 Disfluency and Noise Management (should specify)

The grammar representation should support the handling of disfluency and noise.

An example token syntax for a speech noise absorbing model is <FILLER>.

1.6 Notational Convenience (nice to specify, future study)

The grammar representation might allow for certain syntactic conveniences. An example is the permutation of items in a list of N items taken M=N at a time.

An example format for permutation is:

Symbols: A, B, C
'A || B' means 'AB | BA'
'A || B || C' means 'ABC | ACB | BAC | BCA | CAB | CBA'

2. Large Vocabulary and Dictation

Special consideration for large vocabularies shall include the following:

2.1 Large Vocabulary Definition (must specify; cf. section 1.3)

The grammar representation must support the definition large vocabularies suitable for dictation applications.  Associated attributes of such grammars shall be made available to the speech recognizer for improving interpretation characteristics. Specifications for large vocabulary will not preclude the definition of small grammars.

2.2 Efficiency (must specify; cf. section 6)

The grammar representation must not preclude efficient processing of speech grammars with large vocabularies.

3. Grammar Contexts

Multiple grammar contexts shall be supported with the following requirements:

3.1 External Grammars (must specify)

The grammar representation must support the inclusion of grammars defined outside of the current context. Access to grammar contexts shall be provided by a suitable reference mechanism.

3.2 Run-Time Definition (must specify)

Grammar constraint rules must be re-definable in part or entirety while the system is operating. Several mechanisms are possible including, but not limited to, unconstrained redefinition of inferior grammar rules, prior declaration of volatile rules, partitioning of the rule space into static and dynamic arenas, etc.  

4. Semantics

Semantic specifications are to be coordinated with the Natural Language, Dialog and Universal Access subgroups. In many cases semantic definitions required by these other groups will be implemented as part of the specification language of associated grammars.

4.1 Semantics Support (must specify)

The grammar representation must support the specification of semantics in association with the grammar syntax.

4.2 Semantic Tagging (must specify)

The grammar representation must support the tagging of syntax for semantic interpretation. Semantic values shall be returned as attribute-values pairs.

4.3 Attributes (must specify)

The grammar representation must include attributes that can be attached to data returned from the speech recognizer. Such attributes can have multiple values, be used to indicate the context for interpretation of recognizer output, and generally pass semantic information to later processing stages The specification must be consistent with the natural language interpreter input format.

4.4 Attribute Processing (must specify)

A grammar referencing another grammar having attributes must be capable of performing a [currently undefined] set of operations upon the referenced attributes.

Examples of such processing include boolean operations, string manipulation and attribute renaming.

5. Post-Processing Issues

5.1 Confidence Scoring (must specify)

The grammar representation must provide information for the post-processing of recognition confidence scores with regard to error rejection processing. Such information can include the language model perplexity (high perplexity would typically reduce confidence, and hence rejection threshold) or direct cues to tighten or relax the normal rejection constraints to provide content based control of performance.

5.2 N-Best Hypotheses (must specify)

The grammar representation must support the post-processing of N-best output of recognition hypotheses. This requirement will be coordinated with the Dialog subgroup.

6. Efficiency Issues (cf. section 2.2)

6.1 Native Grammar Formats (must specify)

The grammar representation must support the downloading of native grammar formats for efficiency purposes. A binary reference format can be defined for this purpose. Native formats will be useful when content is specifically written for particular platforms.

6.2 Grammar Libraries (should specify)

The grammar representation should support the use of grammar libraries, alternatively called grammar objects, that can contain prepackaged collections of sub-grammars to be included in higher level grammar constructs. Such libraries will be accessible via the naming conventions, may contain symbol tables for efficient reference resolution, may be imported and be designated to remain resident in a platform.

7. XML Compatibility

7.1 XML Embedding (must specify)

The grammar representation must support easy embedding of grammars into XML.

7.2 Pure XML Format (must specify)

A pure XML format for specification of grammars, including CFG's, must be supported. XML grammar specifications must be capable of expressive power equivalent to BNF specifications.

8. Grammar Specification Language Syntax

This is a general requirements section into which all other requirements will eventually migrate as the representation syntax is defined to satisfy those requirements.

8.1 Understandability (must specify)

The grammar representation must be easy to understand, using well known methods for specifying the various elements.

Backus-Naur Format is an example of finite-state and context-free grammar representations.

A modified form of the well known MIT format is an example format for representation of N-gram grammars (cf. Appendix).

8.2 Mixed-Mode Grammars (should specify)

The grammar representation should support the simultaneous mixing of finite-state, context-free, and N-gram grammars.

8.3 Language Extensions (should specify)

The grammar representation should support the extension of the grammar representation in an obvious manner.

8.4 Grammar Naming (must specify)

The grammar representation must support the naming of grammars. Reference to full grammars and rules within grammars shall be supported by a suitable multi-part naming mechanism. Easy name resolution and overloading shall be supported. A namespace mechanism to avoid naming conflicts shall be supported. Such reference shall include reference by Universal Resource Identifier (URI). Attribute fields shall be included in the naming format.

8.5 Native Natural Language (must specify)

The grammar representation must support the specification of a native language or locale. This specification can be embedded within a grammar rule to change the native language in mid-sentence.

An example syntax for the specification of English: xml:lang="en".

8.6 Rule Weighting (must specify)

The grammar representation must support the weighting of grammar rules in the CFG format. Weighting is implicit in the N-Gram format.

8.7 Phonetic Pronunciation (must specify)

The grammar representation must support the inclusion of phonetic pronunciation rules. This information may override default rules defined by the speech processing platform. The thorough definition of this subject will be the charter of another subgroup.

An example format is:

a tag identifying the phonetic alphabet in use e.g.
<alphabet=[arpabet|sampa|vendorspecifi]>.

8.8 Grammar File Inclusion (must specify)

The grammar representation must support the inclusion of other grammar files referenced by name via a Universal Resource Identifier (URI). This inclusion method is distinguished from grammar reference by symbol.

8.9 Comments (must specify)

The grammar representation must include a commenting mechanism. This mechanism can be provided by HTML or XML commenting formats.

8.10 Character Encodings (must specify)

The grammar representation must support the use of character encoding for foreign language support.

Example formats can include Unicode and JIS. XML character encoding can be used for XML grammar specifications.

8.11 Recognizer Timeout Periods (should specify)

The grammar representation should support the specification of time limits inherently related to grammar characteristics.  Such inherent characteristics can include the expected (typically maximum) times required to normally speak a sentence from the grammar. Such time limits can directly indicate the maximum sentence length in the grammar and may include, but not be limited to: maximum initial silence waiting time, minimum spoken utterance time, maximum spoken utterance time, and maximum intra-sentence silence time (for ASR endpointing).

Appendix - Additional Examples

N-Gram Grammar Format

An example format is derived from the MIT N-gram format as follows:

Here is a brief description of the MIT bigram file format:

...comments...
ngram 1=A
ngram 2=B
...comments...
P(w₁) w₁ P_bo(w₁)
    ...
...comments...
P(w₂|w₁) w₁ w₂
    ...

where A is the number of unigrams, B is the number of bigrams, P(w₁) is the unigram probability of word w₁ (symbol), P_bo(w₁) is the corresponding back-off probability, P(w₂|w₁) is the probability of w₂ conditioned on prior w₁. Start and end of sentence are indicated by the '#' symbol, such as P(#|w_i) to indicate a sentence begins with w_i and P(w_j|#) to indicate the sentence ends with w_j.

To adapt this to arbitrary N-grams we need to either indicate N or define a section end marker. We can also provide data type markers such as <GRAMMAR> and </GRAMMAR>. How about a start marker <GRAMMAR TYPE="N-GRAM" ARY=[number]> to indicate the n-ary of the grams. Comments can follow HTML style. An alternative to specifying 1=A, 2=B, etc. is to use markers to identify data.  Examples:

(Feedback: N-gram span is implicit within the format.)

    <N-GRAM ARY="1" P="0.01" PBO="0.001">
           "word"
    </N-GRAM>
    <N-GRAM ARY="2" P="0.01">
           "word list"
    </N-GRAM>

(Feedback: This may be too verbose.)

in which case we could intermix the types, but this would make it more difficult for systems designers to automatically allocate resources. Another issue is whether we want to allow the symbols to be word phrases, so I suppose we could require quote marks around symbol strings for this (as shown).

Acknowledgments

Subgroup Members

Michael Brown (Bell Labs)
Deborah Dahl (Unisys)
Charles Hemphill (Conversa)
Andrew Hunt (Sun Labs)
Robert Keiller (Canon)
Tetsuo Kosaka (Canon)
James Larson (Intel)
William Ledingham (SpeechWorks)
Bruce Lucas (IBM)
Jens Marschner (Philips)
Scott McGlashen (PipeBeach)
Michael Phillips (SpeechWorks)
Stephen Potter (Entropic)
David Raggett (W3C/HP)
Ramesh Sarukkai (L&H)
Frank Scahill (BT Labs)
Volker Steinbiss (Philips)
George White (General Magic)