llllll|||l||||IllllllllllllllllllllllIlllllllllllllllllllllllllllllllllllll US005577165A United States Patent 1191
[11]
Patent Number:
Takebayashi et al.
[45]
Date of Patent:
*Nov. 19, 1996
OTHER PUBLICATIONS
[54] SPEECH DIALOGUE SYSTEM FOR FACILITATING I1VIPROVED HUMAN-COMPUTER INTERACTION
IEEE, Sep. 1992, pp. 197-200, Hiroyuki Tsuboi, et al}, “A Real-Time Task-Oriented Speech Understanding System Using Keyword-Spotting”. IEEE, 1111. 1991, pp. 905-908, Yoichi Takebayashi, et al., “A
[75] Inventors: Yoichi Takebayashi, Kanagawa-ken; ‘933$; Ymch‘
_
5,577,165
Yamashit; Hyogo_ke’n_ Yoshifumi
Robust Speech Recognition System Using Work—Spotting
Nagata, Kanagawa-ken; Shigenobu Seto, Kanagawa-ken; Hideaki Shinchi, Kanagawa-ken; Hideki Hashimoto, Kanagawa-ken, all of Japan
wlth Noise Immumty Leammg ' ICASSP-88,April 1988, pp. 183-186, Hiroshi Matsu’ura, et al, “A Large Vocabulary Word Recognition System Based on Syllable Recognition and non Linear Word Matching”.
_
_ _
_
_
,_
ICASSP-92, Mar. 1992, pp. 85—88, Yoichi Takebayashi, et
[731 Assignees‘ Kablfshlkl Kalsha TosplbagKawasakl’
al, “Key word Spotting in Noisy Continuous Speech Using
Toshlba Software Engmeenng Corp"
Word Pattern Vector Subabstraction and Noise Immunity
Tokyo, both of Japan
Leaning”
[*]
Notice:
The term of this patent shall not extend
beyond the expiration date of Pat. No. 5,357,596.
_
_
Prlmary ExammerwAnen R- MacDonald Assistant Examiner—Indranil Chowdhury
[21] Appl. No.: 312,541
Attorney, Agent, or Firm—Ob1on, Spivak, McClelland, ' Mam & N eustadt , P. C .
[22] Filed:
[57]
Sep. 26, 1994
Related US Application Data [63]
A speechdialogue system capable of realizing natural and
Continuation of Ser. No. 978,521, Nov. 18, 1992, Pat. No. 5,357,596. .
[30]
.
.
.
.
[JP]
smooth dialogue between the system and a human user, and easy maneuverability 9f the system‘ In the system, a smart tic content of input speech from a user is understood and a semantic content determination of a response output is made
Forelgn Application Pnonty Data
Nov. 18, 1991
ABSTRACT
according to the understood semantic content of the input
Japan .................................. .. 3-329475
speech. Then, a speech response and a visual response
[51]
Int. Cl. ...................................................... .. GOlL 9/00
[52]
U_-S- CL
according to the determined response output are generated and Outputted to the usen The dialogue between the system and the user is managed by controlling transitions between
6
-- 395/234; 395/279; 395/2-6
Field Of Search ................................ ..
, 42, 43,
331/44, 45; 395/279, 2-84, 2-4, 2-6 [56]
References Cited U's' PATENT DOCUMENTS 4,677,569
8/ 1989
5,068,645
11/1991
5,219,291
Lemelson
.. .. . .. .
. . . . ..
Drumm ..... ..
the input speech is to be entered and
system states during which the system response is to be outputted. The understanding of a semantic content of input speech from a user is made by detecting keywords in the input speech, with the keywords to be detected in the input speech limited in advance, according to a state of a dialogue
6/1987 Nakano et a1. ......................... .. 381/41
4,856,066
user states during
between the user and the System
381/36
340/710
6/1993 Fong et a1. ............ ..
434/323
5,357,596 10/1994 Takebayashi et a1. ..
.. 395/284
42 Claims, 42 Drawing Sheets
2 3
r 3
S 234
USER STATE
DIALOGUE
DETECTION UNIT
S235 M
236]
RESPONSE
MANAGEMENT 1.. GENERATION q 232
UNIT
C)
UNIT
lsflggc? INPUT __ SPEECH UNDERSTANDING 4| UNIT
4231
l_________
237
U.S. Patent
Nov. 19, 1996
Sheet 1 of 42
5,577,165
f, 11 INPUT SPEECH
SPEECH
UNDERSTANDING UNIT
‘
r, 12
DIALOGUE MANAGEMENT ' UNIT
q
14
G (
f 15
, right r?" ACT = ADDITION CONFIRMATION
“Le‘t me con?rm. You Want to add
<s1ze>, right ?"
US. Patent
FIG.22A
5,577,165
Sheet 16 0f 42
Nov. 19, 1996
@ n=0
L S141
‘ SET NUMBER OF ITEMS IN SEMANTIC RESPONSE REPRESENTATION T0 M
w, 5142
Y
FILL ITEM, SIZE & QUANTITY FOR SENTENCE EACH ITEM PATTERN INTO RESPONSE AL/
I1=I1+l NO
L
ACT : PARTIAL CONFIRMATION
FIG.22B
ITEM
SIZE
COLA
LARGE
1
SMALL
3
POTATOES
QUANTITY
ACT = PARTIAL CONFIRMATION
FIG.22C
“Let me confirm. You want
FIG.22D
“Let me confirm. You want
< quantity > < size > < item >, right ‘.7”
one large cola and
three small potatoes, right ?"
US. Patent
Nov. 19, 1996
mu1imTSe>.Ez