WEBVTT

1
00:00:08.660 --> 00:00:14.480
<v Yu Liang>Let's just, you know, just enjoy his presentation. Thank you. Go ahead, Daniel.

2
00:00:14.580 --> 00:00:23.080
<v Dan Mailman>Okay, all right. Okay, so thank you to the members of the committee, Dr. Wu, Dr. Liang, Dr. Gao, Dr. Wang, and Dr. Huang.

3
00:00:26.460 --> 00:00:29.220
<v Dan Mailman>Let's get to the right screen here.

4
00:00:30.280 --> 00:00:45.780
<v Dan Mailman>Okay, so I'm going to talk about my dissertation, which is on gesture-efficient global language input methods, and first I'm going to motivate why this project started in the first place.

5
00:00:45.960 --> 00:00:57.060
<v Dan Mailman>Then I'm going to define exactly the problem that we are trying to solve, briefly touch on the literature review that all of the articles I had to read in order to do this darn thing.

6
00:00:57.980 --> 00:00:58.540
<v Dan Mailman>Then.

7
00:00:59.480 --> 00:01:13.280
<v Dan Mailman>Describe how the program works, describe the results of the study, and then indicate what still needs to be done and the conclusions based on the study itself.

8
00:01:13.780 --> 00:01:28.760
<v Dan Mailman>All right, so the idea is that there are other languages which are hard to type on a QWERTY keyboard, okay, and those are, let's set a timer so I don't go over time here.

9
00:01:32.140 --> 00:01:42.700
<v Dan Mailman>And those, so, and I've used those things and I'm really not very happy with the ones that exist and being a computer software engineer and writer of software I thought I could do better so basically that's it.

10
00:01:42.860 --> 00:01:51.880
<v Dan Mailman>So Spanish was a language I tried to type before, Japanese was a language I had tried to type before, and Chinese was also a language I had tried to type before.

11
00:01:52.960 --> 00:01:58.800
<v Dan Mailman>The softwares for doing those work, and so that's good, but I think that they could be better.

12
00:01:59.220 --> 00:02:08.540
<v Dan Mailman>So being a scientist as well, turn the idea that you can make better software into a hypothesis that can be tested.

13
00:02:08.680 --> 00:02:23.980
<v Dan Mailman>Okay, so the hypothesis that I came up with for this engineering project is that you can do fewer gestures for any language when you're typing individual characters or when you're typing common words and phrases.

14
00:02:24.200 --> 00:02:26.980
<v Dan Mailman>Okay, and rather than try to prove it.

15
00:02:29.820 --> 00:02:39.700
<v Dan Mailman>Logically, I just build something that is, that uses fewer gestures if I can, and then that proves the hypothesis, and if I can't, that disproves the hypothesis.

16
00:02:42.220 --> 00:02:43.840
<v Dan Mailman>And then, so how do I do that?

17
00:02:43.900 --> 00:02:49.840
<v Dan Mailman>You build the thing, IME input method, and you compare the gesture counts with existing IMEs.

18
00:02:50.280 --> 00:02:55.160
<v Dan Mailman>And if you have fewer gestures, your hypothesis test is proven.

19
00:02:55.360 --> 00:02:58.720
<v Dan Mailman>If you can't have fewer gestures, then it is not.

20
00:02:59.220 --> 00:03:01.080
<v Dan Mailman>Not disproven, but not proven either.

21
00:03:01.420 --> 00:03:05.360
<v Dan Mailman>Okay, scope, you can't test every language.

22
00:03:05.880 --> 00:03:08.760
<v Dan Mailman>There are tens of thousands of them.

23
00:03:09.280 --> 00:03:11.040
<v Dan Mailman>So we just pick a few.

24
00:03:11.360 --> 00:03:12.920
<v Dan Mailman>In this case, I picked three.

25
00:03:14.780 --> 00:03:18.280
<v Dan Mailman>English, Spanish, Vietnamese, and Chinese.

26
00:03:19.380 --> 00:03:22.300
<v Dan Mailman>And then test the hypothesis on those.

27
00:03:24.340 --> 00:03:27.840
<v Dan Mailman>So the first one, the first one was Spanish.

28
00:03:27.980 --> 00:03:29.200
<v Dan Mailman>That's the easiest one.

29
00:03:32.400 --> 00:03:39.940
<v Dan Mailman>And currently, or with, if you're trying to type, I am sharing my screen.

30
00:03:40.060 --> 00:03:41.120
<v Dan Mailman>Can everybody see my screen?

31
00:03:42.500 --> 00:03:42.980
<v Dalei Wu>Yes.

32
00:03:43.380 --> 00:03:44.160
<v Dan Mailman>Okay, great.

33
00:03:47.320 --> 00:03:47.800
<v Dan Mailman>Okay.

34
00:03:49.380 --> 00:03:54.200
<v Dan Mailman>Dr. Gao, I don't know how to have you see the screen.

35
00:03:57.200 --> 00:04:00.620
<v Dan Mailman>Anyway, hopefully you can pick up then, just from what I'm saying.

36
00:04:01.160 --> 00:04:01.940
<v Dan Mailman>Maybe, David.

37
00:04:02.260 --> 00:04:06.180
<v Dalei Wu>Dr. Gao can try and exit the room and join again.

38
00:04:06.460 --> 00:04:08.140
<v Yingfeng Wang>Dr. Gao can see it right now.

39
00:04:08.200 --> 00:04:09.520
<v Yingfeng Wang>You'll see the chatter.

40
00:04:10.000 --> 00:04:12.120
<v Yu Liang>Maybe the internet is unstable.

41
00:04:12.620 --> 00:04:14.640
<v Yingfeng Wang>No, no, Dr. Gao can see it right now.

42
00:04:14.740 --> 00:04:15.520
<v Yingfeng Wang>Okay, great.

43
00:04:15.780 --> 00:04:16.960
<v Yingfeng Wang>You'll just see the chatter.

44
00:04:17.720 --> 00:04:18.280
<v Dan Mailman>Good.

45
00:04:18.840 --> 00:04:26.320
<v Dan Mailman>Okay, so for Spanish, the normal way to do those characters is to type alt codes.

46
00:04:26.540 --> 00:04:31.540
<v Dan Mailman>Okay, so for instance, if I'm in Windows and I'm typing Word and I need a Spanish character,

47
00:04:34.380 --> 00:04:43.420
<v Dan Mailman>then you have to hold down the alt key and type 0-2-2-5 if you want an A with an acute accent over it.

48
00:04:43.520 --> 00:04:48.240
<v Dan Mailman>If you want the capitalized version of that, you hold down the alt key, type 0-1-9-3,

49
00:04:48.380 --> 00:04:53.840
<v Dan Mailman>and magically your character will appear in Microsoft Word or Microsoft Excel or whatever.

50
00:04:54.420 --> 00:05:02.040
<v Dan Mailman>And if you're typing Spanish, that becomes kind of a pain in the tuchus because you've got to either look them up or memorize.

51
00:05:02.040 --> 00:05:10.640
<v Dan Mailman>First, okay, and then you've got to type five characters, five keys, in order to get a single character.

52
00:05:10.780 --> 00:05:12.220
<v Dan Mailman>So I think we can do better.

53
00:05:12.400 --> 00:05:12.740
<v Dan Mailman>Okay.

54
00:05:14.880 --> 00:05:20.680
<v Dan Mailman>So it takes, for instance, five keystrokes to type the N with the tilde over it.

55
00:05:20.740 --> 00:05:26.680
<v Dan Mailman>It takes four keystrokes to type the peseta indicator.

56
00:05:26.800 --> 00:05:29.260
<v Dan Mailman>That's the Spanish unit of currency.

57
00:05:29.960 --> 00:05:32.020
<v Dan Mailman>And if you happen to be in Word, it takes six keys.

58
00:05:34.660 --> 00:05:35.760
<v Dan Mailman>So that's bad.

59
00:05:35.960 --> 00:05:38.860
<v Dan Mailman>Five keystrokes for one character is not desirable.

60
00:05:39.140 --> 00:05:39.420
<v Dan Mailman>Okay.

61
00:05:39.480 --> 00:05:43.780
<v Dan Mailman>So the question is, can an IME, an input method, reduce it to one keyboard gesture?

62
00:05:44.660 --> 00:05:47.840
<v Dan Mailman>And if so, can we make it easily learned?

63
00:05:47.960 --> 00:05:56.040
<v Dan Mailman>And the theory or the design for doing that for μLex, or for μTeclas,

64
00:05:56.800 --> 00:06:00.320
<v Dan Mailman>the case study for Spanish on the keyboard only,

65
00:06:00.860 --> 00:06:01.820
<v Dan Mailman>is that we...

66
00:06:02.040 --> 00:06:06.980
<v Dan Mailman>That we can notice that the N with the tilde is the same thing as an N.

67
00:06:07.120 --> 00:06:08.900
<v Dan Mailman>So there's a one-to-one mapping there.

68
00:06:09.140 --> 00:06:15.040
<v Dan Mailman>We notice that the underline or that the upside-down question mark is similar to the right-side-up question mark.

69
00:06:15.140 --> 00:06:16.960
<v Dan Mailman>So there's a graphical similarity.

70
00:06:17.820 --> 00:06:23.080
<v Dan Mailman>There's also a graphical similarity between the Spanish quote marks and the greater than and less than signs.

71
00:06:24.500 --> 00:06:29.920
<v Dan Mailman>And a graphical similarity between peseta and the capital P.

72
00:06:30.380 --> 00:06:31.900
<v Dan Mailman>But in order to get the...

73
00:06:32.040 --> 00:06:40.020
<v Dan Mailman>In order to get the euro mark, we have to rely on a semantic and not a visual association.

74
00:06:40.800 --> 00:06:50.060
<v Dan Mailman>So using each of those keys, we can do a one-to-one interface for both.

75
00:06:50.160 --> 00:06:56.480
<v Dan Mailman>And basically how μLex works for that is if you hold the key down for just an extra beat,

76
00:06:56.660 --> 00:06:58.140
<v Dan Mailman>you'll get the target.

77
00:06:58.260 --> 00:07:00.180
<v Dan Mailman>You'll get the N with a tilde or the...

78
00:07:00.180 --> 00:07:01.920
<v Dan Mailman>If you hold down the question mark...

79
00:07:01.920 --> 00:07:04.760
<v Dan Mailman>For an extra beat, you'll get the upside-down question mark.

80
00:07:05.320 --> 00:07:05.840
<v Dan Mailman>Okay.

81
00:07:05.880 --> 00:07:08.900
<v Dan Mailman>Now, we're not only motivated by the Spanish language,

82
00:07:09.700 --> 00:07:14.840
<v Dan Mailman>but it turns out that there are other languages for which there's only a one character,

83
00:07:14.980 --> 00:07:18.260
<v Dan Mailman>one character that looks like one of the QWERTY characters.

84
00:07:18.560 --> 00:07:23.360
<v Dan Mailman>There are 51 other languages that we can use the same strategy for.

85
00:07:23.460 --> 00:07:28.300
<v Dan Mailman>So once we write a software for, say, Basque or Albanian, okay,

86
00:07:28.480 --> 00:07:29.300
<v Dan Mailman>we can...

87
00:07:29.300 --> 00:07:31.660
<v Dan Mailman>The user will be able to download that software.

88
00:07:31.920 --> 00:07:35.780
<v Dan Mailman>Be able to hold down the key for an extra beat for whatever character they're looking for

89
00:07:35.780 --> 00:07:38.400
<v Dan Mailman>and be able to just go on about their business.

90
00:07:38.660 --> 00:07:48.560
<v Dan Mailman>So here we have taken six keystrokes and reduced them to one on programs that run with just the keyboard.

91
00:07:48.780 --> 00:07:49.160
<v Dan Mailman>Okay.

92
00:07:49.920 --> 00:07:52.600
<v Dan Mailman>But not everybody wants to just use a keyboard.

93
00:07:52.680 --> 00:07:57.160
<v Dan Mailman>So for all of those languages, we can also provide a mouse-operated display

94
00:07:57.160 --> 00:08:00.800
<v Dan Mailman>where you click on what you want and get that as well.

95
00:08:01.920 --> 00:08:02.120
<v Dan Mailman>Okay.

96
00:08:02.220 --> 00:08:02.980
<v Dan Mailman>All right.

97
00:08:03.260 --> 00:08:05.240
<v Dan Mailman>Now, there's a second...

98
00:08:05.240 --> 00:08:08.660
<v Dan Mailman>The second class of language uses letters as well.

99
00:08:08.900 --> 00:08:09.460
<v Dan Mailman>Okay.

100
00:08:09.500 --> 00:08:10.580
<v Dan Mailman>And the second class of...

101
00:08:10.580 --> 00:08:12.780
<v Dan Mailman>Or the third class of input methods.

102
00:08:13.760 --> 00:08:23.580
<v Dan Mailman>You can't do that for Vietnamese, which has two times three times six different versions of the letter A.

103
00:08:23.780 --> 00:08:24.240
<v Dan Mailman>Okay.

104
00:08:24.320 --> 00:08:29.920
<v Dan Mailman>So we need to come up with a system which presents those in a logical manner

105
00:08:29.920 --> 00:08:31.600
<v Dan Mailman>so that they can be...

106
00:08:31.920 --> 00:08:32.260
<v Dan Mailman>Walked through.

107
00:08:32.440 --> 00:08:36.080
<v Dan Mailman>So you can't put up 200 and 200 and some odd characters

108
00:08:36.080 --> 00:08:39.280
<v Dan Mailman>and expect somebody to be able to find it on the display easily.

109
00:08:39.500 --> 00:08:39.840
<v Dan Mailman>Okay.

110
00:08:39.960 --> 00:08:41.960
<v Dan Mailman>So that system...

111
00:08:41.960 --> 00:08:42.440
<v Dan Mailman>Okay.

112
00:08:42.980 --> 00:08:45.160
<v Dan Mailman>There are systems already in place.

113
00:08:45.240 --> 00:08:47.440
<v Dan Mailman>The Windows Vietnamese Telex system,

114
00:08:47.540 --> 00:08:54.200
<v Dan Mailman>which reduces finding those characters to a memorized set of keys.

115
00:08:54.640 --> 00:08:55.160
<v Dan Mailman>Okay.

116
00:08:55.400 --> 00:09:01.780
<v Dan Mailman>And reduces the number of gestures to get those to basically the number of characters

117
00:09:01.920 --> 00:09:03.240
<v Dan Mailman>plus however many marks.

118
00:09:03.440 --> 00:09:07.940
<v Dan Mailman>So if you wanted a simple A or a simple N in this system,

119
00:09:08.080 --> 00:09:09.320
<v Dan Mailman>you would use one key.

120
00:09:09.480 --> 00:09:09.840
<v Dan Mailman>Okay.

121
00:09:09.960 --> 00:09:16.040
<v Dan Mailman>If you wanted to do a vowel with just the form and not a tone mark,

122
00:09:16.260 --> 00:09:17.660
<v Dan Mailman>you could use two keys.

123
00:09:17.700 --> 00:09:22.920
<v Dan Mailman>So for instance, A with an upside down smile over it,

124
00:09:22.980 --> 00:09:24.180
<v Dan Mailman>you type A-W.

125
00:09:24.380 --> 00:09:26.600
<v Dan Mailman>If you wanted the hat, you would type A-A.

126
00:09:26.800 --> 00:09:30.040
<v Dan Mailman>And if you wanted the O with the horn, you would type O-W.

127
00:09:30.180 --> 00:09:30.540
<v Dan Mailman>Okay.

128
00:09:30.760 --> 00:09:31.780
<v Dan Mailman>So that...

129
00:09:31.920 --> 00:09:36.840
<v Dan Mailman>Reduces what would normally be six keystrokes with the alt codes

130
00:09:36.840 --> 00:09:43.080
<v Dan Mailman>to two or three keystrokes with a set of memorized extra keys

131
00:09:43.080 --> 00:09:44.400
<v Dan Mailman>for the marks on the vowels.

132
00:09:47.340 --> 00:09:50.920
<v Dan Mailman>So in trying to do better, we really couldn't.

133
00:09:50.920 --> 00:09:54.760
<v Dan Mailman>We do exactly the same thing, but we use a visual interface instead.

134
00:09:55.160 --> 00:09:58.420
<v Dan Mailman>So here is an example where for Vietnamese characters,

135
00:09:58.880 --> 00:10:01.200
<v Dan Mailman>the hypothesis is in fact disproven,

136
00:10:01.200 --> 00:10:01.900
<v Dan Mailman>at least so far.

137
00:10:01.920 --> 00:10:05.700
<v Dan Mailman>Because we do exactly the same gestures.

138
00:10:05.780 --> 00:10:09.180
<v Dan Mailman>I think the interface is better, but that's for somebody else to decide.

139
00:10:09.500 --> 00:10:14.480
<v Dan Mailman>This is exactly the same gestures as the Microsoft program,

140
00:10:14.900 --> 00:10:16.320
<v Dan Mailman>but we present it visually.

141
00:10:16.620 --> 00:10:20.860
<v Dan Mailman>So here we actually have disproved our hypothesis and say,

142
00:10:20.920 --> 00:10:25.520
<v Dan Mailman>look, we can't do better for Vietnamese characters.

143
00:10:25.940 --> 00:10:26.320
<v Dan Mailman>Okay.

144
00:10:26.740 --> 00:10:27.500
<v Dan Mailman>All right.

145
00:10:27.940 --> 00:10:30.080
<v Dan Mailman>Other languages in the same class.

146
00:10:30.340 --> 00:10:31.900
<v Dan Mailman>There are 67 of them.

147
00:10:31.920 --> 00:10:36.720
<v Dan Mailman>So we could write a similar piece of software for each of these 67 languages

148
00:10:36.720 --> 00:10:44.960
<v Dan Mailman>to make it easier for users of those languages to type their language more fluidly.

149
00:10:45.920 --> 00:10:46.680
<v Dan Mailman>All right.

150
00:10:46.720 --> 00:10:49.940
<v Dan Mailman>So now it turns out the most interesting, and I didn't think about this,

151
00:10:50.020 --> 00:10:52.620
<v Dan Mailman>I didn't discover this until about midway through the study,

152
00:10:52.660 --> 00:10:55.600
<v Dan Mailman>but it turns out that if we concentrate on words and phrases,

153
00:10:55.760 --> 00:10:58.820
<v Dan Mailman>we can actually make an improvement for English as well.

154
00:10:59.020 --> 00:10:59.520
<v Dan Mailman>Okay.

155
00:10:59.620 --> 00:11:01.240
<v Dan Mailman>So this is an example.

156
00:11:01.240 --> 00:11:01.900
<v Dan Mailman>And I'll demonstrate.

157
00:11:01.920 --> 00:11:02.480
<v Dan Mailman>I'll demonstrate it later.

158
00:11:02.700 --> 00:11:09.340
<v Dan Mailman>But if we write our software in such a way that you're inputting words and phrases

159
00:11:09.340 --> 00:11:11.840
<v Dan Mailman>instead of letter by letter,

160
00:11:12.060 --> 00:11:16.760
<v Dan Mailman>then you can indeed reduce the number of gestures to get those words and phrases.

161
00:11:16.960 --> 00:11:20.740
<v Dan Mailman>So this is an example where if you use the μWords,

162
00:11:20.940 --> 00:11:22.380
<v Dan Mailman>that's one of our case studies,

163
00:11:22.480 --> 00:11:24.260
<v Dan Mailman>to type the word understand,

164
00:11:24.400 --> 00:11:29.940
<v Dan Mailman>you can do it in two gestures rather than the 10 gestures it would take

165
00:11:29.940 --> 00:11:31.580
<v Dan Mailman>using a normal QWERTY keyboard.

166
00:11:35.580 --> 00:11:37.320
<v Dan Mailman>These are the statistics for that.

167
00:11:37.360 --> 00:11:38.180
<v Dan Mailman>I'll go over those later.

168
00:11:38.980 --> 00:11:43.840
<v Dan Mailman>Other languages in the same class, there are 50 some odd languages

169
00:11:44.380 --> 00:11:48.400
<v Dan Mailman>that are basically QWERTY and no marks.

170
00:11:48.600 --> 00:11:48.980
<v Dan Mailman>Okay.

171
00:11:49.000 --> 00:11:54.880
<v Dan Mailman>So those can be done exactly the same way once we have a corpus that we can analyze in order

172
00:11:54.880 --> 00:12:00.040
<v Dan Mailman>to determine the most frequent words and phrases, we can present exactly the same kind

173
00:12:00.040 --> 00:12:01.560
<v Dan Mailman>of interface for these languages.

174
00:12:01.920 --> 00:12:06.880
<v Dan Mailman>That we do for English and indeed for, well, we'll see later,

175
00:12:06.960 --> 00:12:08.540
<v Dan Mailman>we can do the same thing for Chinese as well.

176
00:12:09.180 --> 00:12:14.500
<v Dan Mailman>So not only English, but 56 other languages, if we just take QWERTY into account.

177
00:12:14.680 --> 00:12:23.540
<v Dan Mailman>But it also turns out that there are about 150 other languages that put marks on their letters,

178
00:12:23.780 --> 00:12:25.780
<v Dan Mailman>but multiple marks on their letters.

179
00:12:25.840 --> 00:12:31.340
<v Dan Mailman>But if we take, if we compile corpus corpora for those languages,

180
00:12:31.340 --> 00:12:37.780
<v Dan Mailman>and then do a frequency analysis, we can do a word by word interface for those as well.

181
00:12:38.000 --> 00:12:45.020
<v Dan Mailman>And so presumably we can reduce the number of gestures from somewhere around 10 to somewhere

182
00:12:45.020 --> 00:12:52.560
<v Dan Mailman>around four for any of these languages once we've done corpus analysis on them.

183
00:12:53.080 --> 00:12:53.720
<v Dan Mailman>Okay.

184
00:12:53.760 --> 00:12:58.560
<v Dan Mailman>So for Chinese, the software that I got frustrated with is illustrated here.

185
00:12:59.300 --> 00:13:01.120
<v Dan Mailman>And this is the Microsoft.

186
00:13:01.140 --> 00:13:01.320
<v Dan Mailman>Okay.

187
00:13:01.320 --> 00:13:10.380
<v Dan Mailman>Microsoft input method in order to, for instance, if I wanted to input the Chinese word

188
00:13:10.380 --> 00:13:17.120
<v Dan Mailman>for father-in-law, I would have to type Y-U-E, and then the software would graciously give me

189
00:13:17.120 --> 00:13:22.460
<v Dan Mailman>that mark after it to indicate I've finished a symbol, and then I would have to type F-U.

190
00:13:22.640 --> 00:13:29.120
<v Dan Mailman>So five keystrokes to get to there, and then one more keystroke, the number one,

191
00:13:29.120 --> 00:13:31.000
<v Dan Mailman>or clicking on this area.

192
00:13:31.320 --> 00:13:36.800
<v Dan Mailman>In blue would get me that sent to my Word, or my Notepad, or my spreadsheet.

193
00:13:37.040 --> 00:13:37.320
<v Dan Mailman>Okay.

194
00:13:37.460 --> 00:13:43.880
<v Dan Mailman>So can we reduce the number of gestures to get words and phrases in Chinese?

195
00:13:44.280 --> 00:13:44.800
<v Dan Mailman>Okay.

196
00:13:44.880 --> 00:13:45.800
<v Dan Mailman>So that's the question.

197
00:13:45.940 --> 00:13:46.400
<v Dan Mailman>All right.

198
00:13:47.140 --> 00:13:47.660
<v Dan Mailman>Okay.

199
00:13:47.720 --> 00:13:56.720
<v Dan Mailman>Other languages in this class, there are about a dozen of them, and those are listed here.

200
00:13:56.960 --> 00:14:00.340
<v Dan Mailman>I've actually already done one for Cherokee a while back.

201
00:14:01.320 --> 00:14:03.020
<v Dan Mailman>I could probably do a better job.

202
00:14:03.120 --> 00:14:05.040
<v Dan Mailman>That was years ago.

203
00:14:06.640 --> 00:14:09.980
<v Dan Mailman>And I'm sort of halfway doing one for Japanese here.

204
00:14:10.300 --> 00:14:10.980
<v Dan Mailman>All right.

205
00:14:11.140 --> 00:14:13.060
<v Dan Mailman>But those aren't really part of this study.

206
00:14:15.080 --> 00:14:15.560
<v Dan Mailman>Okay.

207
00:14:15.760 --> 00:14:19.140
<v Dan Mailman>So those are the problems.

208
00:14:19.280 --> 00:14:19.520
<v Dan Mailman>Okay.

209
00:14:19.560 --> 00:14:21.840
<v Dan Mailman>Well, you know, we have a bunch of IMEs out there.

210
00:14:21.900 --> 00:14:27.320
<v Dan Mailman>I think there's actually an organization in the central Chinese government that keeps

211
00:14:27.320 --> 00:14:30.760
<v Dan Mailman>track of all of the IMEs that have been proposed for.

212
00:14:34.980 --> 00:14:37.600
<v Dan Mailman>So I'm getting ready to add to that.

213
00:14:39.220 --> 00:14:45.860
<v Dan Mailman>So the goals of this project are to make IMEs, input methods that are immediately usable

214
00:14:45.860 --> 00:14:51.640
<v Dan Mailman>and easily learned, are compatible with ordinary applications, operating systems, and input

215
00:14:51.640 --> 00:14:52.100
<v Dan Mailman>devices.

216
00:14:52.680 --> 00:14:55.480
<v Dan Mailman>That's actually the hardest part of writing the software.

217
00:14:55.700 --> 00:14:58.720
<v Dan Mailman>And increase user productivity over time.

218
00:14:58.960 --> 00:14:59.460
<v Dan Mailman>Okay.

219
00:14:59.460 --> 00:15:01.300
<v Dan Mailman>So now what do we mean by that?

220
00:15:01.300 --> 00:15:01.440
<v Dan Mailman>We mean by those.

221
00:15:01.720 --> 00:15:01.980
<v Dan Mailman>Okay.

222
00:15:02.020 --> 00:15:02.840
<v Dan Mailman>Immediately usable.

223
00:15:03.080 --> 00:15:06.940
<v Dan Mailman>And you can see that basically these are all what you see is what you get.

224
00:15:07.060 --> 00:15:08.680
<v Dan Mailman>If you click something, you'll get something.

225
00:15:09.160 --> 00:15:12.980
<v Dan Mailman>It's for native speakers of those particular languages.

226
00:15:13.100 --> 00:15:16.500
<v Dan Mailman>These are designed to be immediately accessible.

227
00:15:17.620 --> 00:15:18.920
<v Dan Mailman>So easily learned.

228
00:15:19.520 --> 00:15:20.860
<v Dan Mailman>And immediately usable.

229
00:15:21.140 --> 00:15:21.800
<v Dan Mailman>All right.

230
00:15:22.100 --> 00:15:28.120
<v Dan Mailman>To make compatible with ordinary applications, so on Windows, Microsoft Word, Microsoft Excel,

231
00:15:28.320 --> 00:15:31.060
<v Dan Mailman>Notepad, et cetera, et cetera, what we call productivity applications.

232
00:15:31.840 --> 00:15:33.460
<v Dan Mailman>Windows is an operating system.

233
00:15:33.600 --> 00:15:35.000
<v Dan Mailman>Linux is an operating system.

234
00:15:35.360 --> 00:15:39.920
<v Dan Mailman>OS X on Apple computers is one.

235
00:15:40.380 --> 00:15:41.500
<v Dan Mailman>Android is one.

236
00:15:41.700 --> 00:15:50.980
<v Dan Mailman>Every time we add a particular language, we also would have to write a different version

237
00:15:50.980 --> 00:15:53.860
<v Dan Mailman>of the software for that language for each OS.

238
00:15:54.140 --> 00:15:59.000
<v Dan Mailman>So that's really technically the hardest part of getting these into users' hands.

239
00:15:59.280 --> 00:15:59.500
<v Dan Mailman>Okay.

240
00:15:59.500 --> 00:16:05.980
<v Dan Mailman>For keyboards, the idea of the longpress, the extra beat when you hold down the key,

241
00:16:06.600 --> 00:16:11.960
<v Dan Mailman>guarantees that we're not interfering with the alt keys and the shift keys and the control

242
00:16:11.960 --> 00:16:18.520
<v Dan Mailman>keys and the function keys that operating systems and applications sometimes give special

243
00:16:18.520 --> 00:16:19.140
<v Dan Mailman>meaning to.

244
00:16:19.960 --> 00:16:20.560
<v Dan Mailman>Okay.

245
00:16:20.620 --> 00:16:25.680
<v Dan Mailman>So compatibility with applications, operating systems, and input devices.

246
00:16:26.080 --> 00:16:26.860
<v Dan Mailman>All right.

247
00:16:26.960 --> 00:16:27.660
<v Dan Mailman>And then.

248
00:16:27.660 --> 00:16:34.360
<v Dan Mailman>Kind of the most fun part of this was trying to figure out a way to make it so that when

249
00:16:34.360 --> 00:16:38.040
<v Dan Mailman>people are using the software, they're not only just using the software, but they're

250
00:16:38.040 --> 00:16:40.040
<v Dan Mailman>getting better at it at the same time.

251
00:16:40.300 --> 00:16:40.740
<v Dan Mailman>Okay.

252
00:16:40.980 --> 00:16:46.420
<v Dan Mailman>And we do that through arranging the grids in a particular way.

253
00:16:46.640 --> 00:16:49.620
<v Dan Mailman>And here are three aspects that do that.

254
00:16:49.660 --> 00:16:54.140
<v Dan Mailman>First, the fact that we have grids at all—the last item, display guidance—makes it

255
00:16:54.140 --> 00:16:57.220
<v Dan Mailman>very easy for them to just click on what they see and know that that.

256
00:16:57.220 --> 00:16:59.320
<v Dan Mailman>Thing gets put to their application.

257
00:17:01.280 --> 00:17:04.780
<v Dan Mailman>We also do not change the grids around.

258
00:17:05.020 --> 00:17:10.800
<v Dan Mailman>So the same set of cues, whether it's the mouse cues or the keyboard cues, always result

259
00:17:10.800 --> 00:17:12.400
<v Dan Mailman>in the same target.

260
00:17:12.520 --> 00:17:18.660
<v Dan Mailman>So if you always type W-A to get the word, I don't know, understand, right?

261
00:17:19.120 --> 00:17:23.740
<v Dan Mailman>Every time you type W-A when the software is active, you will get “understand” sent to

262
00:17:23.740 --> 00:17:24.120
<v Dan Mailman>your productivity application.

263
00:17:24.480 --> 00:17:25.960
<v Dan Mailman>So that's cue target consistency.

264
00:17:26.360 --> 00:17:26.700
<v Dan Mailman>Okay.

265
00:17:26.720 --> 00:17:27.080
<v Dan Mailman>And then.

266
00:17:27.220 --> 00:17:32.140
<v Dan Mailman>The other aspect that makes these things easier to use and hopefully faster to use over time

267
00:17:32.140 --> 00:17:39.020
<v Dan Mailman>is to arrange the targets on the grids in such a way that the most common are easily

268
00:17:39.020 --> 00:17:41.860
<v Dan Mailman>found in the left to right, top to bottom eye scan.

269
00:17:42.620 --> 00:17:46.160
<v Dan Mailman>And then the less frequent ones are over sort toward the right left.

270
00:17:46.320 --> 00:17:48.720
<v Dan Mailman>So most common is upper left.

271
00:17:48.960 --> 00:17:50.860
<v Dan Mailman>Least common is lower right.

272
00:17:51.000 --> 00:17:56.000
<v Dan Mailman>And you should be able to find what you're looking for in the top few rows of the grids

273
00:17:56.000 --> 00:17:56.980
<v Dan Mailman>that are our.

274
00:17:57.220 --> 00:17:57.740
<v Dan Mailman>Displays.

275
00:17:58.900 --> 00:17:59.500
<v Dan Mailman>Okay.

276
00:17:59.560 --> 00:18:03.660
<v Dan Mailman>So now all of that babbling, let's do walkthroughs of the software.

277
00:18:03.740 --> 00:18:09.480
<v Dan Mailman>So you actually do know what I am talking about. To introduce them, μTeclas—teclas

278
00:18:09.480 --> 00:18:14.080
<v Dan Mailman>is Spanish for keys, uh, is the keyboard only display for letter by letter.

279
00:18:14.180 --> 00:18:19.460
<v Dan Mailman>When there is a one-to-one mapping of the targets to keys on the keyboard.

280
00:18:19.980 --> 00:18:26.860
<v Dan Mailman>μLetras is adding a display to that. μVietChu is Vietnamese, which introduces

281
00:18:26.860 --> 00:18:47.760
<v Dan Mailman>the branching for future choices. Then μWords and μCiYu are there for English words and phrases and Chinese words and phrases. The prototype software that I am going to demonstrate for English uses only 3,000 of 333,000 available words and phrases.

282
00:18:47.780 --> 00:18:56.660
<v Dan Mailman>And for, uh, The Chinese only 5,000 phrases are in the demo and of, of an available 430 some odd thousand.

283
00:18:56.720 --> 00:18:56.840
<v Dan Mailman>Okay.

284
00:18:56.860 --> 00:18:57.400
<v Dan Mailman>Okay.

285
00:18:57.680 --> 00:19:04.720
<v Dan Mailman>These are available on my website if you want to play with me.

286
00:19:04.800 --> 00:19:05.200
<v Dan Mailman>Okay.

287
00:19:05.240 --> 00:19:09.060
<v Dan Mailman>So μTeclas, the keyboard only for Spanish.

288
00:19:11.580 --> 00:19:11.980
<v Dan Mailman>Okay.

289
00:19:12.340 --> 00:19:15.280
<v Dan Mailman>There's a display and the display there is optional.

290
00:19:15.340 --> 00:19:16.300
<v Dan Mailman>You don't really need it.

291
00:19:16.340 --> 00:19:22.440
<v Dan Mailman>It's only there to tell you what key you've hit and when you've, uh, when the extra beat has been crossed.

292
00:19:22.580 --> 00:19:28.810
<v Dan Mailman>So if you want to type in, well, let me make this bigger, but that's, so I think it's bigger.

293
00:19:30.830 --> 00:19:31.230
<v Dan Mailman>Okay.

294
00:19:31.370 --> 00:19:42.250
<v Dan Mailman>So, uh, if I hold, if I just type a, I'll get an a, but if I hold down the a key for an extra beat, then I'll get a, with the acute, uh, accent over it.

295
00:19:42.430 --> 00:19:47.910
<v Dan Mailman>Um, and similarly, if I hold the shift key and an a, the same thing.

296
00:19:47.990 --> 00:19:48.370
<v Dan Mailman>Okay.

297
00:19:48.490 --> 00:19:54.630
<v Dan Mailman>So this way you're using the keyboard with a visual cues for what the, for the special characters.

298
00:19:55.490 --> 00:20:01.110
<v Dan Mailman>Um, and this one, this particular piece of software is only for the keyboard and it's only useful for languages.

299
00:20:01.110 --> 00:20:08.830
<v Dan Mailman>Which is in which there's a one-to-one mapping, uh, of the target characters and the cue characters, which are on the keyboard.

300
00:20:09.250 --> 00:20:13.830
<v Dan Mailman>Um, but we have several, um, dozen of those.

301
00:20:14.090 --> 00:20:14.790
<v Dan Mailman>All right.

302
00:20:14.930 --> 00:20:17.750
<v Dan Mailman>Um, now not everybody always wants to use the keyboard.

303
00:20:17.850 --> 00:20:25.090
<v Dan Mailman>So the next class of, of, um, IME of input method is μLetras.

304
00:20:25.250 --> 00:20:26.310
<v Dan Mailman>This is Spanish.

305
00:20:26.410 --> 00:20:29.710
<v Dan Mailman>Now we're doing letter by letter and the keyboard still works.

306
00:20:29.750 --> 00:20:31.030
<v Dan Mailman>So I'm here holding down the key.

307
00:20:31.110 --> 00:20:31.330
<v Dan Mailman>It's here.

308
00:20:32.690 --> 00:20:36.950
<v Dan Mailman>Um, no normal G just repeats because there's no variation of it.

309
00:20:36.990 --> 00:20:41.270
<v Dan Mailman>But if I wanted to type the capital N with the tilde.

310
00:20:41.350 --> 00:20:51.190
<v Dan Mailman>So those are all shortpress and longpress on the keys, but we can also do shortpress and longpress on the letters on the, on the display here and get the thing.

311
00:20:51.370 --> 00:20:51.570
<v Dan Mailman>Okay.

312
00:20:51.650 --> 00:21:00.270
<v Dan Mailman>So here I do a shortpress and I get an a, I do a longpress and I get an a with the acute, I use the other button and I get a capital a.

313
00:21:00.430 --> 00:21:01.090
<v Dan Mailman>If I hold.

314
00:21:01.110 --> 00:21:06.890
<v Dan Mailman>It down and, uh, and a normal a, if I do it for a short period of time.

315
00:21:07.030 --> 00:21:07.490
<v Dan Mailman>Okay.

316
00:21:07.570 --> 00:21:12.950
<v Dan Mailman>And the display gives you the feedback that tells you, yes, you've crossed the longpress threshold.

317
00:21:13.090 --> 00:21:13.510
<v Dan Mailman>Okay.

318
00:21:14.130 --> 00:21:14.770
<v Dan Mailman>All right.

319
00:21:14.810 --> 00:21:15.130
<v Dan Mailman>Okay.

320
00:21:15.190 --> 00:21:23.970
<v Dan Mailman>So that's the second class of IME, which is one-to-one mapping of cues and targets, uh, with and without the display.

321
00:21:24.170 --> 00:21:24.610
<v Dan Mailman>Okay.

322
00:21:26.990 --> 00:21:27.590
<v Dan Mailman>All right.

323
00:21:27.610 --> 00:21:31.850
<v Dan Mailman>The third one, the third case study, μVietChu,

324
00:21:33.990 --> 00:21:39.910
<v Dan Mailman>Um, it should demonstrates how we deal with multiple targets and one cue.

325
00:21:40.110 --> 00:21:40.430
<v Dan Mailman>Okay.

326
00:21:40.470 --> 00:21:48.950
<v Dan Mailman>So as I said, there were, uh, 36 variations, not well, 36 times, 72 variations of the letter a possible.

327
00:21:49.090 --> 00:21:49.610
<v Dan Mailman>Okay.

328
00:21:49.770 --> 00:21:52.290
<v Dan Mailman>Uh, if you take into account shifting, okay.

329
00:21:52.350 --> 00:21:57.610
<v Dan Mailman>So for this, you can either the same idea here, the black indicates what you can get immediately.

330
00:21:58.250 --> 00:21:58.770
<v Dan Mailman>Okay.

331
00:21:58.850 --> 00:22:02.950
<v Dan Mailman>Into the typing area, the red, which is accessible by longpress.

332
00:22:03.790 --> 00:22:12.710
<v Dan Mailman>Brings up a further grid and now each of the variations of the letter a, with a single mark is available in one further gesture.

333
00:22:12.910 --> 00:22:23.140
<v Dan Mailman>And each of the variations with two marks are available with a second gesture on a second display.

334
00:22:23.420 --> 00:22:23.960
<v Dan Mailman>Okay.

335
00:22:24.060 --> 00:22:29.560
<v Dan Mailman>So in this way, uh, it's essentially one gesture per mark.

336
00:22:29.760 --> 00:22:30.240
<v Dan Mailman>Okay.

337
00:22:30.320 --> 00:22:34.860
<v Dan Mailman>Which is the same as, so not better than too bad, uh, than the existing one.

338
00:22:34.880 --> 00:22:35.140
<v Dan Mailman>Okay.

339
00:22:35.140 --> 00:22:53.360
<v Dan Mailman>So it's the same as the existing telex windows, uh, IME, okay, but the important thing to know about this, to, to, to sort of carry forward about this is that it does lead to branching and branching leads to new displays and new displays give new possibilities of what you want to choose.

340
00:22:53.500 --> 00:22:53.780
<v Dan Mailman>Okay.

341
00:22:53.920 --> 00:23:04.240
<v Dan Mailman>And that is exactly the strategy that we take into the English-language words-and-phrases interface, which we are calling μWords.

342
00:23:06.880 --> 00:23:15.530
<v Dan Mailman>So, um, uh, I'm going to have to shrink this one a little bit so that you can see the typing area.

343
00:23:16.030 --> 00:23:16.610
<v Dan Mailman>All right.

344
00:23:17.870 --> 00:23:18.270
<v Dan Mailman>Okay.

345
00:23:18.490 --> 00:23:19.250
<v Dan Mailman>Here we go.

346
00:23:19.410 --> 00:23:19.810
<v Dan Mailman>Okay.

347
00:23:19.870 --> 00:23:20.450
<v Dan Mailman>Same idea.

348
00:23:20.510 --> 00:23:26.410
<v Dan Mailman>What you see in black is immediately, um, accessible just by clicking.

349
00:23:27.210 --> 00:23:27.610
<v Dan Mailman>Okay.

350
00:23:29.290 --> 00:23:36.670
<v Dan Mailman>Um, and then what you see in the red with the dots says, I'm going to give you a more choices on a different grid.

351
00:23:38.530 --> 00:23:39.990
<v Dan Mailman>And you can choose one of those.

352
00:23:40.070 --> 00:23:47.010
<v Dan Mailman>So in this way, “quarter” is accessible with two gestures.

353
00:23:47.130 --> 00:23:52.730
<v Dan Mailman>And then for user-interface convenience, we hide what is not available.

354
00:23:52.830 --> 00:23:58.250
<v Dan Mailman>So when you are hovering over this area, you get the grid of Q—of G choices.

355
00:23:58.550 --> 00:24:07.190
<v Dan Mailman>And when you are hovering over this area, you get the input, the lexeme, and that gets sent to the productivity app.

356
00:24:07.710 --> 00:24:08.190
<v Dan Mailman>Okay.

357
00:24:08.530 --> 00:24:19.330
<v Dan Mailman>Uh, so that's where English language now, the same strategy is actually, uh, doable with Mandarin Chinese.

358
00:24:19.590 --> 00:24:28.730
<v Dan Mailman>Um, and I'll demonstrate that with, um, words and phrases.

359
00:24:28.890 --> 00:24:29.450
<v Dan Mailman>Here we go.

360
00:24:29.570 --> 00:24:30.430
<v Dan Mailman>All right.

361
00:24:31.130 --> 00:24:40.130
<v Dan Mailman>So the idea here is that words and phrases are accessible immediately when, oops, let's get the typing area visible here.

362
00:24:40.210 --> 00:24:40.450
<v Dan Mailman>Sorry.

363
00:24:40.450 --> 00:24:41.690
<v Dan Mailman>I have to shrink this again.

364
00:24:42.050 --> 00:24:44.810
<v Dan Mailman>Uh, I wish I could do this bigger, but I can't.

365
00:24:44.890 --> 00:24:48.830
<v Dan Mailman>Um, so words and phrases immediately accessible.

366
00:24:49.250 --> 00:24:49.710
<v Dan Mailman>Okay.

367
00:24:50.030 --> 00:24:56.390
<v Dan Mailman>If you don't see what you want, but you do know the first sound of it, let's go, um, uh, let's see.

368
00:24:57.390 --> 00:24:57.850
<v Dan Mailman>D.

369
00:24:58.470 --> 00:24:58.930
<v Dan Mailman>Okay.

370
00:24:59.010 --> 00:25:07.910
<v Dan Mailman>Then now we have a screen of the most common, or I should say grid of the most common words and phrases, which start with that sound.

371
00:25:08.090 --> 00:25:08.550
<v Dan Mailman>Okay.

372
00:25:08.790 --> 00:25:10.230
<v Dan Mailman>So dōu, meaning

373
00:25:10.450 --> 00:25:11.410
<v Dan Mailman>All or both.

374
00:25:11.630 --> 00:25:12.150
<v Dan Mailman>Okay.

375
00:25:12.190 --> 00:25:17.350
<v Dan Mailman>Is immediately accessible, but there are some that are also accessible using DB.

376
00:25:17.590 --> 00:25:17.970
<v Dan Mailman>Okay.

377
00:25:18.030 --> 00:25:24.650
<v Dan Mailman>Now this is DB is actually going to be the first syllable of a multi character word.

378
00:25:24.890 --> 00:25:29.130
<v Dan Mailman>So all of these, this one starts with D, and this one starts with B.

379
00:25:29.370 --> 00:25:29.890
<v Dan Mailman>Okay.

380
00:25:29.970 --> 00:25:31.670
<v Dan Mailman>So these are all accessible that way.

381
00:25:31.770 --> 00:25:40.150
<v Dan Mailman>So here we have—let us see, if I want this—

382
00:25:40.150 --> 00:25:41.830
<v Dan Mailman>débùdào, “unable to get.”

383
00:25:41.890 --> 00:25:44.790
<v Dan Mailman>So in two gestures, maybe it was three gestures.

384
00:25:44.910 --> 00:25:56.610
<v Dan Mailman>We are doing three characters, which are normally accessible—one, two, four, maybe seven gestures—using the normal pinyin-based interface.

385
00:25:57.170 --> 00:25:57.770
<v Dan Mailman>Okay.

386
00:25:57.810 --> 00:26:02.310
<v Dan Mailman>So the, the take home is that the same simple strategy.

387
00:26:02.510 --> 00:26:03.110
<v Dan Mailman>Okay.

388
00:26:03.510 --> 00:26:09.810
<v Dan Mailman>Uh, either input something directly or find a grid that gives you further.

389
00:26:09.810 --> 00:26:12.250
<v Dan Mailman>Inputs is used all the way down.

390
00:26:12.470 --> 00:26:12.990
<v Dan Mailman>Okay.

391
00:26:13.690 --> 00:26:24.190
<v Dan Mailman>It is kind of like a card trick that looks really, really hard, but the mechanics of it are fairly easy, as we will see.

392
00:26:24.570 --> 00:26:25.290
<v Dan Mailman>All right.

393
00:26:25.370 --> 00:26:25.790
<v Dan Mailman>Okay.

394
00:26:25.870 --> 00:26:27.570
<v Dan Mailman>So back to the slides.

395
00:26:29.450 --> 00:26:30.270
<v Dan Mailman>All right.

396
00:26:30.370 --> 00:26:30.630
<v Dan Mailman>Okay.

397
00:26:30.690 --> 00:26:39.750
<v Dan Mailman>So again, uh, the, the prototypes that are on there use 3000 of 330,000 available, uh, words and phrases.

398
00:26:39.810 --> 00:26:46.930
<v Dan Mailman>And for Chinese, it's 5,000 of the 430 available words and phrases, the final versions of these will cover those.

399
00:26:47.330 --> 00:26:47.810
<v Dan Mailman>Okay.

400
00:26:48.230 --> 00:26:48.950
<v Dan Mailman>All right.

401
00:26:51.760 --> 00:26:52.240
<v Dan Mailman>Okay.

402
00:26:52.640 --> 00:26:53.320
<v Dan Mailman>Literature—oops.

403
00:26:54.400 --> 00:27:01.680
<v Dan Mailman>Uh, I had to read a lot of articles in order to get this thing supported and written, uh, and I don't want to go through them individually.

404
00:27:02.000 --> 00:27:11.600
<v Dan Mailman>So I just want to say the classes of them, uh, several, maybe a dozen articles were on languages and Unicode period, somewhere on usability and performance.

405
00:27:11.600 --> 00:27:39.740
<v Dan Mailman>Some were on other strange devices, which I'm not going to talk about because it's a, it's a sort of, it's not as important to the dissertation, um, but things like one handed input devices and voice input, uh, in certain cases, uh, in order to produce the word and phrase, um, uh, grids, I had to do a lot of corporal corp, corpus studies and frequency lists on the corpus studies.

406
00:27:40.280 --> 00:27:41.400
<v Dan Mailman>Uh, and then.

407
00:27:41.600 --> 00:27:50.620
<v Dan Mailman>Several articles were on Mandarin input and Mandarin language, and then, uh, other articles were on input software and platforms.

408
00:27:51.100 --> 00:28:06.360
<v Dan Mailman>So you can see the tawdry list of articles I had to go through, some of which were actually mine. I had a patent on an early version of this, early in this century.

409
00:28:08.960 --> 00:28:11.140
<v Dan Mailman>Um, so too many.

410
00:28:11.780 --> 00:28:12.220
<v Dan Mailman>Articles.

411
00:28:14.340 --> 00:28:14.780
<v Dan Mailman>Okay.

412
00:28:15.020 --> 00:28:15.620
<v Dan Mailman>All right.

413
00:28:15.660 --> 00:28:19.500
<v Dan Mailman>So, so now the guts of the thing, how does the software actually work?

414
00:28:19.660 --> 00:28:19.960
<v Dan Mailman>Okay.

415
00:28:20.120 --> 00:28:24.640
<v Dan Mailman>And this is where, this is where it's a lot simpler than it, than it might seem.

416
00:28:24.820 --> 00:28:25.180
<v Dan Mailman>Okay.

417
00:28:25.360 --> 00:28:40.800
<v Dan Mailman>So the, the core of the, of the software for those, for those of us who are computer scientists is that it runs a finite state automaton or a finite state machine, uh, using, uh, tables to say how we get from one state to another.

418
00:28:40.920 --> 00:28:41.320
<v Dan Mailman>Okay.

419
00:28:41.520 --> 00:28:42.700
<v Dan Mailman>So, and I'll talk about.

420
00:28:42.940 --> 00:28:49.920
<v Dan Mailman>A finite state machines after I show how simple the software really is at its core, using the tables that drive it.

421
00:28:50.100 --> 00:28:50.340
<v Dan Mailman>Okay.

422
00:28:50.720 --> 00:29:04.760
<v Dan Mailman>So for μTeclas, for the keyboard-only interfaces, all we really need to know is: what is the cue, what do we do when we do a shortpress, and what do we do when we do a longpress?

423
00:29:04.940 --> 00:29:05.260
<v Dan Mailman>Okay.

424
00:29:05.340 --> 00:29:11.020
<v Dan Mailman>So the table that drives μTeclas has rows in the table that look like this.

425
00:29:11.080 --> 00:29:12.600
<v Dan Mailman>They tell you the cue is an n.

426
00:29:14.280 --> 00:29:17.500
<v Dan Mailman>If you shortpress it, you are going to get an n.

427
00:29:17.680 --> 00:29:21.100
<v Dan Mailman>If you longpress it, you are going to get an n with a tilde.

428
00:29:21.320 --> 00:29:27.040
<v Dan Mailman>If you hold the shift key down and shortpress it, you'll get a cap, a capital N.

429
00:29:27.260 --> 00:29:37.520
<v Dan Mailman>And if you hit any, if you, uh, uh, longpress it with the shift key down, you'll get a capital N and one shared loop, one event loop.

430
00:29:38.040 --> 00:29:41.280
<v Dan Mailman>That's the finite state machine handles every single record.

431
00:29:41.480 --> 00:29:41.920
<v Dan Mailman>Okay.

432
00:29:41.980 --> 00:29:42.740
<v Dan Mailman>So you can see the.

433
00:29:42.740 --> 00:29:46.900
<v Dan Mailman>Very simple table, uh, that drives the entire software.

434
00:29:47.820 --> 00:29:48.260
<v Dan Mailman>Okay.

435
00:29:48.540 --> 00:29:52.220
<v Dan Mailman>And this is when we add the display, you can actually see that more clearly.

436
00:29:52.300 --> 00:29:54.740
<v Dan Mailman>The cue is what's in the upper left-hand corner.

437
00:29:55.040 --> 00:29:59.660
<v Dan Mailman>The secondary target, the primary target is essentially the cue.

438
00:29:59.820 --> 00:30:04.200
<v Dan Mailman>So you don't need to show that the secondary target is what gets displayed in the middle.

439
00:30:04.600 --> 00:30:05.040
<v Dan Mailman>Okay.

440
00:30:06.000 --> 00:30:07.960
<v Dan Mailman>Because that's the character that's special.

441
00:30:08.080 --> 00:30:11.980
<v Dan Mailman>And that is the one that is hard to get to unless you have the software.

442
00:30:12.040 --> 00:30:12.320
<v Dan Mailman>Whoops.

443
00:30:12.740 --> 00:30:12.980
<v Dan Mailman>Okay.

444
00:30:13.040 --> 00:30:20.000
<v Dan Mailman>So shortpress selects n; longpress selects the n with the tilde. For the Viet application,

445
00:30:20.380 --> 00:30:27.940
<v Dan Mailman>This shows that, uh, we have two, um, two separate targets.

446
00:30:28.140 --> 00:30:32.100
<v Dan Mailman>One is the primary target and that is the letter a okay.

447
00:30:32.180 --> 00:30:42.560
<v Dan Mailman>Then we have a label indicating the secondary target, which opens a further grid or leads to a child grid,

448
00:30:42.560 --> 00:30:49.500
<v Dan Mailman>we can say. And so the indicator gets put here in the cell, to the left center.

449
00:30:49.720 --> 00:30:55.820
<v Dan Mailman>And when you select it, when you hold down something and release it, uh, you'll get an updated grid.

450
00:30:56.160 --> 00:30:56.640
<v Dan Mailman>Okay.

451
00:30:56.740 --> 00:31:02.660
<v Dan Mailman>So, uh, and then once you actually do a selection, you'll go back to the, to the home grid.

452
00:31:02.940 --> 00:31:03.420
<v Dan Mailman>Okay.

453
00:31:03.800 --> 00:31:08.280
<v Dan Mailman>So, uh, um, similarly, okay.

454
00:31:08.740 --> 00:31:12.520
<v Dan Mailman>Uh, when we, uh, when we're.

455
00:31:12.560 --> 00:31:17.520
<v Dan Mailman>Talking about Chinese now, and we're talking about words and phrases, the actual actions are the same actions.

456
00:31:17.660 --> 00:31:17.980
<v Dan Mailman>Okay.

457
00:31:18.120 --> 00:31:23.220
<v Dan Mailman>They're either send something to your productivity application and return to the home grid.

458
00:31:23.340 --> 00:31:24.260
<v Dan Mailman>That's one action.

459
00:31:24.600 --> 00:31:25.080
<v Dan Mailman>Okay.

460
00:31:25.440 --> 00:31:28.180
<v Dan Mailman>The other action is open a secondary grid.

461
00:31:28.340 --> 00:31:28.640
<v Dan Mailman>Okay.

462
00:31:28.700 --> 00:31:37.740
<v Dan Mailman>So in the case of this cell here, which shows yào and the letter S, if you shortpress, you will get the character

463
00:31:37.820 --> 00:31:38.080
<v Dan Mailman>yào.

464
00:31:38.460 --> 00:31:39.160
<v Dan Mailman>It is yào,

465
00:31:39.240 --> 00:31:39.980
<v Dan Mailman>isn’t it? My eyes are bad.

466
00:31:40.780 --> 00:31:40.940
<v Dan Mailman>Yeah.

467
00:31:41.360 --> 00:31:41.840
<v Dan Mailman>Um,

468
00:31:42.560 --> 00:31:59.320
<v Dan Mailman>And if you shortpress, uh, the second that, or longpress that you'll get a further grid, that is the common words and phrases that start with that sound or that letter in pinyin S okay.

469
00:32:00.560 --> 00:32:04.080
<v Dan Mailman>Um, let's see here.

470
00:32:04.380 --> 00:32:05.140
<v Dan Mailman>All right.

471
00:32:05.180 --> 00:32:12.020
<v Dan Mailman>So table on the left, what it looks like when implemented by the software on the right.

472
00:32:12.140 --> 00:32:12.460
<v Dan Mailman>Okay.

473
00:32:12.680 --> 00:32:14.260
<v Dan Mailman>And then, oops.

474
00:32:15.040 --> 00:32:15.640
<v Dan Mailman>Oops.

475
00:32:15.640 --> 00:32:15.980
<v Dan Mailman>Skipped one.

476
00:32:16.880 --> 00:32:17.360
<v Dan Mailman>Okay.

477
00:32:17.420 --> 00:32:18.820
<v Dan Mailman>And similarly for μWords.

478
00:32:18.940 --> 00:32:21.900
<v Dan Mailman>So this table drives that cell.

479
00:32:22.220 --> 00:32:27.140
<v Dan Mailman>And if you, um, shortpress on that cell, okay.

480
00:32:27.200 --> 00:32:30.280
<v Dan Mailman>You will get “have,” H-A-V-E, sent to your productivity app.

481
00:32:30.460 --> 00:32:39.040
<v Dan Mailman>And if you longpress on that cell, it'll open a grid that enables you to, uh, input the most common words and phrases starting with the letter U.

482
00:32:39.160 --> 00:32:39.560
<v Dan Mailman>Okay.

483
00:32:39.680 --> 00:32:45.600
<v Dan Mailman>So in this way, we use the letters similarly, but the letters in English are the letters of the spelling.

484
00:32:45.960 --> 00:32:53.980
<v Dan Mailman>And for Chinese, the letters that you are using on the keyboard are the letters of the pinyin pronunciation system.

485
00:32:54.760 --> 00:32:55.060
<v Dan Mailman>Okay.

486
00:32:55.580 --> 00:32:56.100
<v Dan Mailman>All right.

487
00:32:56.240 --> 00:33:00.940
<v Dan Mailman>So that was, so that's how that's all about tables that run the software.

488
00:33:01.080 --> 00:33:01.280
<v Dan Mailman>Okay.

489
00:33:01.340 --> 00:33:03.160
<v Dan Mailman>But how does the software actually run?

490
00:33:03.340 --> 00:33:03.640
<v Dan Mailman>Okay.

491
00:33:03.700 --> 00:33:11.900
<v Dan Mailman>So there is a very basic, very simple construct in computer science called a finite state machine, or FSM for short.

492
00:33:11.940 --> 00:33:15.220
<v Dan Mailman>And it basically says, uh, your, your.

493
00:33:15.220 --> 00:33:18.720
<v Dan Mailman>Your software maintains an internal state that means something.

494
00:33:18.900 --> 00:33:19.340
<v Dan Mailman>Okay.

495
00:33:19.640 --> 00:33:26.400
<v Dan Mailman>And it reacts to events and every time it reacts to an event, it may or may not do something.

496
00:33:26.500 --> 00:33:26.860
<v Dan Mailman>Okay.

497
00:33:27.120 --> 00:33:42.320
<v Dan Mailman>So I am going to start at the bottom here. There are only three things it can possibly do: change the way the cell looks, send input and return to the home grid, or display a child grid.

498
00:33:42.380 --> 00:33:42.740
<v Dan Mailman>Okay.

499
00:33:42.840 --> 00:33:44.280
<v Dan Mailman>So those are the only events.

500
00:33:44.280 --> 00:33:49.460
<v Dan Mailman>Now, I'm sorry, the only actions that, that this particular program implements, okay.

501
00:33:49.660 --> 00:34:00.820
<v Dan Mailman>And it implements them in, um, Response to only eight things that you can do when you're manipulating the software.

502
00:34:01.040 --> 00:34:04.180
<v Dan Mailman>If you are on a mouse, you can either hover over a particular cell.

503
00:34:04.360 --> 00:34:05.700
<v Dan Mailman>You can leave the cell.

504
00:34:05.900 --> 00:34:14.020
<v Dan Mailman>If you're pressing on the cell, the timer can go off to say when shortpress becomes longpress, and you can release the mouse button or the keyboard.

505
00:34:14.020 --> 00:34:14.660
<v Dan Mailman>Okay.

506
00:34:14.760 --> 00:34:18.900
<v Dan Mailman>On the keyboard, there are only, um, five things you can do.

507
00:34:18.980 --> 00:34:25.440
<v Dan Mailman>You can press down a key, release a key, shift, press down and shift release.

508
00:34:25.680 --> 00:34:26.020
<v Dan Mailman>Okay.

509
00:34:26.100 --> 00:34:31.340
<v Dan Mailman>So those are the possible events that the software pays attention to.

510
00:34:31.420 --> 00:34:38.760
<v Dan Mailman>And given those actions and those events, we only need to remember, and this is what the states are for, what the user has done.

511
00:34:38.960 --> 00:34:42.620
<v Dan Mailman>So the state said, the state says the program knows.

512
00:34:42.700 --> 00:34:43.000
<v Dan Mailman>Okay.

513
00:34:43.040 --> 00:34:44.000
<v Dan Mailman>So if I'm the program.

514
00:34:44.020 --> 00:34:46.020
<v Dan Mailman>I say, okay, I know that I'm ready.

515
00:34:46.160 --> 00:34:46.520
<v Dan Mailman>Okay.

516
00:34:46.560 --> 00:34:47.320
<v Dan Mailman>Nothing is happening.

517
00:34:47.480 --> 00:34:47.700
<v Dan Mailman>Okay.

518
00:34:47.760 --> 00:34:51.220
<v Dan Mailman>Or I know that the user’s mouse is hovering over my cell.

519
00:34:51.760 --> 00:34:55.560
<v Dan Mailman>Or I know that he's pressed something without a shift key.

520
00:34:55.700 --> 00:34:58.420
<v Dan Mailman>So he's getting ready to do the primary key.

521
00:34:58.500 --> 00:34:58.700
<v Dan Mailman>Oops.

522
00:34:58.780 --> 00:35:00.800
<v Dan Mailman>Now I know he's pressed the shift key.

523
00:35:00.980 --> 00:35:03.340
<v Dan Mailman>So now he's getting ready to do a capital N.

524
00:35:03.680 --> 00:35:04.160
<v Dan Mailman>Okay.

525
00:35:04.440 --> 00:35:14.000
<v Dan Mailman>So when the primary gesture is given, when he holds down the key, we move from Ready to Primary.

526
00:35:14.000 --> 00:35:16.800
<v Dan Mailman>When he holds down the Shift key, we move to Shifted.

527
00:35:17.180 --> 00:35:21.480
<v Dan Mailman>When he long presses in either one of those, we move to either of those two states.

528
00:35:21.640 --> 00:35:26.540
<v Dan Mailman>And then when he releases, we move back to the ready state, ready for the next one.

529
00:35:26.680 --> 00:35:26.920
<v Dan Mailman>Okay.

530
00:35:26.960 --> 00:35:28.560
<v Dan Mailman>So all of that's very, very complicated.

531
00:35:29.220 --> 00:35:37.360
<v Dan Mailman>And the entire, all that babbling I just did is accounted for in this finite state diagram.

532
00:35:37.600 --> 00:35:43.980
<v Dan Mailman>But really all this whole thing means is: what do you do in response to whatever the user is doing?

533
00:35:44.900 --> 00:35:45.320
<v Dan Mailman>Okay.

534
00:35:45.480 --> 00:35:48.760
<v Dan Mailman>So and this is why I said this is the hard part.

535
00:35:48.880 --> 00:36:04.820
<v Dan Mailman>So you do something different if it's a keyboard, if it's a two-button mouse, if it's a one-button mouse, and even if it's a mouse and we've done the sort of split cell thing that we do in the word and phrase interfaces.

536
00:36:05.260 --> 00:36:06.300
<v Dan Mailman>IMEs, input methods.

537
00:36:06.800 --> 00:36:07.320
<v Dan Mailman>Okay.

538
00:36:07.760 --> 00:36:08.280
<v Dan Mailman>So.

539
00:36:09.080 --> 00:36:12.020
<v Dan Mailman>Now the core question is.

540
00:36:12.920 --> 00:36:13.920
<v Dan Mailman>How much.

541
00:36:13.920 --> 00:36:16.180
<v Dan Mailman>How much of the language, whatever that means.

542
00:36:16.480 --> 00:36:16.900
<v Dan Mailman>Okay.

543
00:36:16.940 --> 00:36:19.900
<v Dan Mailman>I'm switching from the implementation now to the coverage statistics.

544
00:36:20.240 --> 00:36:25.480
<v Dan Mailman>How much of the language, whatever that means, can be covered and how few gestures.

545
00:36:25.600 --> 00:36:27.780
<v Dan Mailman>And this is where the statistical analysis comes in.

546
00:36:27.880 --> 00:36:33.360
<v Dan Mailman>So not surprisingly, it's one gesture per letter, whether you're talking about English.

547
00:36:34.700 --> 00:36:39.300
<v Dan Mailman>Now, when you're talking about Spanish, it could be as many as two gestures per letter.

548
00:36:39.500 --> 00:36:41.380
<v Dan Mailman>When you're talking about Vietnamese.

549
00:36:41.380 --> 00:36:45.520
<v Dan Mailman>It could be as many as three gestures per letter.

550
00:36:45.700 --> 00:36:46.060
<v Dan Mailman>Okay.

551
00:36:46.240 --> 00:36:59.240
<v Dan Mailman>So it turns out that in the corpus that I used, the maximum word or phrase was 12 letters.

552
00:36:59.340 --> 00:37:02.500
<v Dan Mailman>So 12 gestures out of 460—

553
00:37:02.760 --> 00:37:03.860
<v Dan Mailman>This was all modeled.

554
00:37:03.960 --> 00:37:05.300
<v Dan Mailman>There isn't actually an IME for this.

555
00:37:05.320 --> 00:37:07.700
<v Dan Mailman>I just did the mathematical analysis for the entire set.

556
00:37:09.340 --> 00:37:10.600
<v Dan Mailman>Maximum 12 characters.

557
00:37:10.600 --> 00:37:14.580
<v Dan Mailman>Covers 460,000 some odd words and phrases.

558
00:37:14.720 --> 00:37:15.880
<v Dan Mailman>For English.

559
00:37:16.400 --> 00:37:17.740
<v Dan Mailman>Maximum 12 gestures.

560
00:37:18.160 --> 00:37:19.580
<v Dan Mailman>For Spanish.

561
00:37:20.020 --> 00:37:20.740
<v Dan Mailman>600.

562
00:37:20.900 --> 00:37:23.800
<v Dan Mailman>Covers 600 some odd thousand words and phrases.

563
00:37:24.120 --> 00:37:25.000
<v Dan Mailman>For Vietnamese.

564
00:37:25.580 --> 00:37:26.680
<v Dan Mailman>13 gestures.

565
00:37:26.980 --> 00:37:31.400
<v Dan Mailman>Not surprisingly, because some of those have the extra two marks.

566
00:37:31.900 --> 00:37:38.240
<v Dan Mailman>But my corpus was only 23,000 words and phrases.

567
00:37:38.640 --> 00:37:39.060
<v Dan Mailman>Okay.

568
00:37:39.060 --> 00:37:42.960
<v Dan Mailman>But when we do word by word, okay, it changes.

569
00:37:43.100 --> 00:37:43.300
<v Dan Mailman>Okay.

570
00:37:43.360 --> 00:37:45.080
<v Dan Mailman>We actually can cut it in half.

571
00:37:45.220 --> 00:37:46.880
<v Dan Mailman>And probably even better than that.

572
00:37:48.280 --> 00:37:54.640
<v Dan Mailman>When we do when we limit the number of words and phrases as we did for the prototypes here.

573
00:37:54.880 --> 00:38:00.120
<v Dan Mailman>Four gestures covers 3,000 words and phrases in English.

574
00:38:00.280 --> 00:38:05.760
<v Dan Mailman>Four gestures also covers 5,000 words and phrases in Chinese.

575
00:38:06.720 --> 00:38:07.800
<v Dan Mailman>In Vietnamese.

576
00:38:07.800 --> 00:38:08.740
<v Dan Mailman>I modeled it.

577
00:38:08.860 --> 00:38:15.440
<v Dan Mailman>Five gestures would cover 23,000 and probably more words and phrases.

578
00:38:15.700 --> 00:38:24.580
<v Dan Mailman>And if we take the entire model into account, six gestures will cover 460 some odd thousand words and phrases in English.

579
00:38:24.620 --> 00:38:31.800
<v Dan Mailman>And six gestures will cover 430 some odd thousand different words and phrases in Mandarin.

580
00:38:31.960 --> 00:38:36.880
<v Dan Mailman>So the real savings in terms of gestures comes when we treat words and phrases.

581
00:38:36.880 --> 00:38:37.860
<v Dan Mailman>Okay.

582
00:38:38.860 --> 00:38:39.620
<v Dan Mailman>All right.

583
00:38:39.620 --> 00:38:54.960
<v Dan Mailman>So for the individual case studies, when we take all of the English language words and phrases into account, we can cover all of them in 99%.

584
00:38:54.960 --> 00:39:03.000
<v Dan Mailman>But as you can see from this graph, we cover something like, I don't know, 70% in four gestures.

585
00:39:04.200 --> 00:39:04.800
<v Dan Mailman>Okay.

586
00:39:05.920 --> 00:39:09.980
<v Dan Mailman>Similarly for Spanish, we cover all of them in 12 gestures.

587
00:39:10.060 --> 00:39:11.140
<v Dan Mailman>This is letter by letter.

588
00:39:11.280 --> 00:39:11.620
<v Dan Mailman>Right?

589
00:39:11.820 --> 00:39:13.180
<v Dan Mailman>Not word and phrase.

590
00:39:13.420 --> 00:39:17.620
<v Dan Mailman>But we still cover most of them, let's say, by six or seven gestures.

591
00:39:17.760 --> 00:39:24.360
<v Dan Mailman>Which basically says that most of the words and phrases that we cover in our corpus are seven and eight characters.

592
00:39:24.520 --> 00:39:28.060
<v Dan Mailman>So basically this is an indicator of the number of characters in the word and phrase.

593
00:39:28.920 --> 00:39:29.920
<v Dan Mailman>Same for Vietnamese.

594
00:39:30.180 --> 00:39:32.260
<v Dan Mailman>Not surprisingly, it's more.

595
00:39:32.800 --> 00:39:34.780
<v Dan Mailman>Because of the extra two marks.

596
00:39:34.780 --> 00:39:35.500
<v Dan Mailman>Okay.

597
00:39:35.700 --> 00:39:38.500
<v Dan Mailman>On the words and phrases.

598
00:39:39.780 --> 00:39:40.820
<v Dan Mailman>And let's see.

599
00:39:40.880 --> 00:39:45.000
<v Dan Mailman>So now we get to the word by word gestures.

600
00:39:45.380 --> 00:39:53.000
<v Dan Mailman>And it turns out that even for, oh, these are the case, these are the limited case studies.

601
00:39:53.160 --> 00:39:55.980
<v Dan Mailman>For 3,000 lexemes, it's four gestures.

602
00:39:58.260 --> 00:40:00.200
<v Dan Mailman>And 75% in two.

603
00:40:00.380 --> 00:40:01.180
<v Dan Mailman>Which is pretty good.

604
00:40:01.320 --> 00:40:04.760
<v Dan Mailman>I mean, 75% of your language is covered in

605
00:40:04.780 --> 00:40:06.060
<v Dan Mailman>two gestures; you are doing pretty well.

606
00:40:07.980 --> 00:40:10.100
<v Dan Mailman>Similarly for the Chinese words and phrases.

607
00:40:10.280 --> 00:40:11.720
<v Dan Mailman>It's steeper.

608
00:40:11.740 --> 00:40:15.600
<v Dan Mailman>So 80 some odd percent in three gestures.

609
00:40:16.680 --> 00:40:20.800
<v Dan Mailman>But quickly we get to 99% in four gestures.

610
00:40:21.840 --> 00:40:29.260
<v Dan Mailman>When we model all of the Vietnamese, similar sorts of statistics.

611
00:40:29.540 --> 00:40:31.360
<v Dan Mailman>Except you'll see that it's one extra.

612
00:40:31.360 --> 00:40:37.300
<v Dan Mailman>For that one extra on average, let's say, one to two.

613
00:40:37.640 --> 00:40:40.660
<v Dan Mailman>Average mark on the vowel for each lexemes.

614
00:40:40.740 --> 00:40:41.740
<v Dan Mailman>For each word or phrase.

615
00:40:41.960 --> 00:40:46.760
<v Dan Mailman>And then now if we're talking about words and phrases now.

616
00:40:46.920 --> 00:40:47.720
<v Dan Mailman>Not characters.

617
00:40:48.300 --> 00:40:48.900
<v Dan Mailman>Okay.

618
00:40:49.320 --> 00:40:50.860
<v Dan Mailman>Words and phrases for English.

619
00:40:50.900 --> 00:40:52.700
<v Dan Mailman>Six gestures for 99%.

620
00:40:52.700 --> 00:40:58.480
<v Dan Mailman>Even when you're talking about 460,000 some odd words and phrases.

621
00:40:59.280 --> 00:41:00.840
<v Dan Mailman>And the number of occurrences.

622
00:41:01.360 --> 00:41:02.780
<v Dan Mailman>Actually nearly 100 million.

623
00:41:04.380 --> 00:41:05.060
<v Dan Mailman>Let's see.

624
00:41:06.340 --> 00:41:08.100
<v Dan Mailman>Fairly similar actually for Mandarin.

625
00:41:08.220 --> 00:41:10.080
<v Dan Mailman>Six gestures covers 99%.

626
00:41:11.760 --> 00:41:14.520
<v Dan Mailman>Assuming you've done your cue to target.

627
00:41:17.000 --> 00:41:17.560
<v Dan Mailman>Strategy.

628
00:41:19.080 --> 00:41:20.980
<v Dan Mailman>In such a way that it minimizes.

629
00:41:21.100 --> 00:41:21.260
<v Dan Mailman>Okay.

630
00:41:21.300 --> 00:41:25.060
<v Dan Mailman>So you could come up with a selection strategy that makes this much worse.

631
00:41:25.180 --> 00:41:29.680
<v Dan Mailman>But the initials of the pinyin work pretty well.

632
00:41:29.680 --> 00:41:36.600
<v Dan Mailman>So six gestures covers 430 some odd thousand words and phrases.

633
00:41:37.740 --> 00:41:38.260
<v Dan Mailman>Okay.

634
00:41:39.320 --> 00:41:40.820
<v Dan Mailman>So those are the statistics.

635
00:41:41.020 --> 00:41:42.000
<v Dan Mailman>They basically.

636
00:41:44.640 --> 00:41:49.460
<v Dan Mailman>We did pretty well in terms of reducing the number of gestures for languages.

637
00:41:49.620 --> 00:41:51.420
<v Dan Mailman>Especially when we go to words and phrases.

638
00:41:52.580 --> 00:41:55.100
<v Dan Mailman>We didn't do so well when it's letter by letter.

639
00:41:56.540 --> 00:41:59.540
<v Dan Mailman>What I'd like to do for each of these pieces of software.

640
00:41:59.540 --> 00:42:01.360
<v Dan Mailman>Is, is I would like to.

641
00:42:01.920 --> 00:42:03.820
<v Dan Mailman>For the μLetras

642
00:42:03.940 --> 00:42:04.420
<v Dan Mailman>One.

643
00:42:04.520 --> 00:42:07.120
<v Dan Mailman>I would like to split the cells the same way we did in μWords and μCiYu.

644
00:42:08.400 --> 00:42:09.400
<v Dan Mailman>For μWords,

645
00:42:09.500 --> 00:42:12.200
<v Dan Mailman>I'd like to do capitalization with secondary gestures.

646
00:42:12.700 --> 00:42:15.040
<v Dan Mailman>I'd like to add an interface for.

647
00:42:15.540 --> 00:42:17.820
<v Dan Mailman>Spaces included with the word or phrase.

648
00:42:18.780 --> 00:42:20.140
<v Dan Mailman>And or punctuation.

649
00:42:20.380 --> 00:42:21.720
<v Dan Mailman>Following the word or phrase.

650
00:42:22.000 --> 00:42:23.180
<v Dan Mailman>I would like to.

651
00:42:24.600 --> 00:42:27.420
<v Dan Mailman>Consider alphabetizing the grids for easier lookup.

652
00:42:27.460 --> 00:42:28.780
<v Dan Mailman>For the Chinese one.

653
00:42:28.780 --> 00:42:31.220
<v Dan Mailman>I'd like to alternate between.

654
00:42:31.420 --> 00:42:33.360
<v Dan Mailman>I'd like to be able to hit the.

655
00:42:33.700 --> 00:42:35.480
<v Dan Mailman>Use the secondary gestures.

656
00:42:35.680 --> 00:42:37.120
<v Dan Mailman>For traditional characters.

657
00:42:37.460 --> 00:42:39.420
<v Dan Mailman>And the primary gestures for.

658
00:42:40.600 --> 00:42:43.520
<v Dan Mailman>For simplified gesture for simplified characters.

659
00:42:43.760 --> 00:42:45.140
<v Dan Mailman>And then I'd also be able to.

660
00:42:45.240 --> 00:42:48.180
<v Dan Mailman>I would like to add an interface so that you get the entire pinyin

661
00:42:48.540 --> 00:42:48.860
<v Dan Mailman>Syllable.

662
00:42:49.060 --> 00:42:50.820
<v Dan Mailman>And or the individual characters.

663
00:42:53.020 --> 00:42:53.660
<v Dan Mailman>Conclusions.

664
00:42:54.420 --> 00:42:57.140
<v Dan Mailman>Human language diversity makes it impossible.

665
00:42:57.140 --> 00:42:58.360
<v Dan Mailman>To do.

666
00:42:59.120 --> 00:43:01.300
<v Dan Mailman>All of your hypothesis testing.

667
00:43:02.180 --> 00:43:06.000
<v Dan Mailman>And then basically these cases suggest that the cue systems can.

668
00:43:06.100 --> 00:43:07.100
<v Dan Mailman>Can be engineered.

669
00:43:07.400 --> 00:43:09.840
<v Dan Mailman>So that word and phrase IMEs use.

670
00:43:10.520 --> 00:43:12.660
<v Dan Mailman>Substantially fewer gestures than letter by letter.

671
00:43:12.840 --> 00:43:13.400
<v Dan Mailman>IMEs.

672
00:43:13.800 --> 00:43:14.980
<v Dan Mailman>Thank you very much.

673
00:43:16.640 --> 00:43:17.360
<v Dan Mailman>That's it.

674
00:43:17.440 --> 00:43:18.740
<v Dan Mailman>That's all I got.

675
00:43:25.200 --> 00:43:25.760
<v Yu Liang>Okay.

676
00:43:26.360 --> 00:43:28.640
<v Yu Liang>Now is the question to the, I think.

677
00:43:29.560 --> 00:43:30.120
<v Yu Liang>Audience.

678
00:43:30.380 --> 00:43:30.620
<v Yu Liang>Yeah.

679
00:43:30.720 --> 00:43:31.260
<v Yu Liang>From the audience.

680
00:43:36.850 --> 00:43:37.610
<v Yu Liang>Any questions.

681
00:43:39.750 --> 00:43:40.190
<v Dan Mailman>Any questions.

682
00:43:40.230 --> 00:43:40.850
<v Dan Mailman>Everybody's asleep.

683
00:43:42.270 --> 00:43:43.070
<v Yu Liang>No, no, no.

684
00:43:43.090 --> 00:43:43.690
<v Yu Liang>It's exciting.

685
00:43:43.850 --> 00:43:44.170
<v Yu Liang>Interesting.

686
00:43:44.490 --> 00:43:45.210
<v Yu Liang>I learned a lot.

687
00:43:45.530 --> 00:43:45.970
<v Yu Liang>Okay.

688
00:43:47.170 --> 00:43:48.190
<v Audience member (Zoom label: Schwab iPad)>I've got a question.

689
00:43:48.810 --> 00:43:50.810
<v Audience member (Zoom label: Schwab iPad)>And this may be completely out of scope.

690
00:43:52.410 --> 00:43:53.990
<v Audience member (Zoom label: Schwab iPad)>And I may be uninformed too.

691
00:43:54.130 --> 00:43:55.650
<v Audience member (Zoom label: Schwab iPad)>So just bear with me, but.

692
00:43:55.710 --> 00:43:58.790
<v Audience member (Zoom label: Schwab iPad)>And I've, I've heard that the court.

693
00:43:59.910 --> 00:44:00.350
<v Audience member (Zoom label: Schwab iPad)>Recorders.

694
00:44:01.290 --> 00:44:04.050
<v Audience member (Zoom label: Schwab iPad)>Who are typing out the transcripts of the courts that they have.

695
00:44:04.250 --> 00:44:05.530
<v Audience member (Zoom label: Schwab iPad)>Types of input devices.

696
00:44:05.770 --> 00:44:06.170
<v Audience member (Zoom label: Schwab iPad)>And.

697
00:44:06.170 --> 00:44:09.690
<v Audience member (Zoom label: Schwab iPad)>That are not strictly keyboards and maybe they have phrases on them.

698
00:44:09.730 --> 00:44:10.270
<v Audience member (Zoom label: Schwab iPad)>I'm not sure.

699
00:44:10.530 --> 00:44:11.410
<v Dan Mailman>I haven't looked.

700
00:44:11.450 --> 00:44:12.730
<v Dan Mailman>To the best of my knowledge.

701
00:44:12.830 --> 00:44:16.430
<v Dan Mailman>Those are syllable based and they, I think that's a good avenue for future.

702
00:44:16.570 --> 00:44:16.950
<v Dan Mailman>Study.

703
00:44:17.810 --> 00:44:18.290
<v Chu-Ren Huang>Yeah.

704
00:44:18.810 --> 00:44:20.790
<v Dan Mailman>I think it would be for Mandarin as well.

705
00:44:21.190 --> 00:44:25.430
<v Dan Mailman>That if you, instead of using strict pinyin, you could, you would.

706
00:44:25.510 --> 00:44:26.910
<v Dan Mailman>And people have done this divide.

707
00:44:27.050 --> 00:44:28.570
<v Dan Mailman>The, the syllable up into.

708
00:44:29.430 --> 00:44:30.270
<v Dan Mailman>Initial and final.

709
00:44:30.650 --> 00:44:33.490
<v Dan Mailman>We might be able to even squeeze one more.

710
00:44:33.790 --> 00:44:35.390
<v Dan Mailman>Save gesture into that.

711
00:44:35.390 --> 00:44:38.830
<v Dan Mailman>but I went with these as sort of for prototypes.

712
00:44:39.550 --> 00:44:40.070
<v Audience member (Zoom label: Schwab iPad)>Right.

713
00:44:40.310 --> 00:44:43.550
<v Audience member (Zoom label: Schwab iPad)>Yeah, I understand you have to kind of hone it in

714
00:44:43.550 --> 00:44:45.030
<v Audience member (Zoom label: Schwab iPad)>to exactly what you can work on.

715
00:44:45.170 --> 00:44:46.730
<v Audience member (Zoom label: Schwab iPad)>But yeah, thank you for that.

716
00:44:46.950 --> 00:44:47.950
<v Dan Mailman>Sure, thank you.

717
00:44:52.770 --> 00:44:53.730
<v Yu Liang>Some other questions?

718
00:44:54.510 --> 00:44:55.910
<v Dan Mailman>Looks like Sarah has one.

719
00:44:57.210 --> 00:44:58.390
<v Sarah Tsiang>Thank you, Dan.

720
00:44:58.490 --> 00:44:59.090
<v Sarah Tsiang>That was great.

721
00:44:59.750 --> 00:45:00.190
<v Dan Mailman>Wow.

722
00:45:00.330 --> 00:45:02.210
<v Sarah Tsiang>You kept mentioning Mandarin Chinese.

723
00:45:02.490 --> 00:45:04.270
<v Sarah Tsiang>How much would it take to adjust

724
00:45:04.270 --> 00:45:07.950
<v Sarah Tsiang>what you've already done to other varieties of Chinese?

725
00:45:09.770 --> 00:45:12.830
<v Dan Mailman>I think, well, okay, so I have this theory

726
00:45:13.350 --> 00:45:14.510
<v Dan Mailman>that I really can't prove

727
00:45:14.510 --> 00:45:17.510
<v Dan Mailman>that if you do a really good phonemic analysis

728
00:45:17.510 --> 00:45:20.890
<v Dan Mailman>of a language, you can get it down to six gestures

729
00:45:20.890 --> 00:45:22.490
<v Dan Mailman>no matter what the language is.

730
00:45:23.450 --> 00:45:26.090
<v Dan Mailman>But so I'm going to go out on the limb and say,

731
00:45:26.230 --> 00:45:27.910
<v Dan Mailman>I can get it down to six,

732
00:45:27.970 --> 00:45:29.330
<v Dan Mailman>but it would take me a month or two

733
00:45:29.330 --> 00:45:31.430
<v Dan Mailman>to come up with a system that does it.

734
00:45:32.590 --> 00:45:33.690
<v Dan Mailman>Have I answered your question?

735
00:45:34.170 --> 00:45:34.570
<v Sarah Tsiang>Yes.

736
00:45:34.990 --> 00:45:36.470
<v Dan Mailman>Okay, and I can't prove that.

737
00:45:36.510 --> 00:45:38.210
<v Dan Mailman>It's just this intuition from the work

738
00:45:38.210 --> 00:45:39.750
<v Dan Mailman>that I've already done.

739
00:45:42.630 --> 00:45:45.990
<v Chu-Ren Huang>Dan, I think Sarah's question is basically,

740
00:45:46.110 --> 00:45:48.330
<v Chu-Ren Huang>can you apply it to Cantonese, Taiwanese,

741
00:45:48.790 --> 00:45:50.050
<v Chu-Ren Huang>Hakka and stuff, right?

742
00:45:50.850 --> 00:45:52.550
<v Chu-Ren Huang>Yes, I think so.

743
00:45:53.810 --> 00:45:59.470
<v Dan Mailman>The normal systems that scholars and missionaries

744
00:45:59.470 --> 00:46:01.250
<v Dan Mailman>have developed for Romanizing those

745
00:46:01.250 --> 00:46:03.970
<v Dan Mailman>tend to give more letters

746
00:46:03.970 --> 00:46:05.810
<v Dan Mailman>than I think are actually necessary,

747
00:46:07.010 --> 00:46:09.270
<v Dan Mailman>even given the actual pronunciation.

748
00:46:09.770 --> 00:46:11.410
<v Dan Mailman>Of those languages, okay?

749
00:46:11.470 --> 00:46:15.190
<v Dan Mailman>And I'm only passingly familiar with Shanghainese

750
00:46:15.190 --> 00:46:16.950
<v Dan Mailman>or Cantonese.

751
00:46:18.390 --> 00:46:21.370
<v Dan Mailman>So I can't say that's true for all of the dialects,

752
00:46:21.370 --> 00:46:23.890
<v Dan Mailman>but it seems to me from the very little I've read

753
00:46:23.890 --> 00:46:25.930
<v Dan Mailman>of the other dialects, the same thing would be true,

754
00:46:26.050 --> 00:46:27.750
<v Dan Mailman>that you would come up with a system

755
00:46:27.750 --> 00:46:33.790
<v Dan Mailman>that gives recognizable cues leading up to the characters.

756
00:46:34.070 --> 00:46:35.970
<v Dan Mailman>And those cues would be recognizable

757
00:46:35.970 --> 00:46:39.310
<v Dan Mailman>by a speaker of Cantonese or a speaker of Hakka,

758
00:46:39.830 --> 00:46:48.320
<v Dan Mailman>or a speaker of, let's say, Sichuan dialect, I think.

759
00:46:48.620 --> 00:46:51.600
<v Dan Mailman>Again, no proof, but I have this intuition we can do it.

760
00:46:51.640 --> 00:46:53.720
<v Dan Mailman>We can engineer such a system

761
00:46:55.980 --> 00:46:58.800
<v Dan Mailman>that will result in six cues maximum

762
00:46:58.800 --> 00:47:00.660
<v Dan Mailman>for 99% of the characters.

763
00:47:06.060 --> 00:47:07.020
<v Yu Liang>Any other questions?

764
00:47:08.200 --> 00:47:09.320
<v Audience member (Zoom label: Schwab iPad)>Yeah, I was going to,

765
00:47:09.460 --> 00:47:12.160
<v Audience member (Zoom label: Schwab iPad)>I'm not sure if I can even quite put this into a question,

766
00:47:12.240 --> 00:47:13.600
<v Audience member (Zoom label: Schwab iPad)>but just something that kind of came to mind.

767
00:47:13.680 --> 00:47:15.980
<v Audience member (Zoom label: Schwab iPad)>And, you know, some of the limit

768
00:47:15.980 --> 00:47:18.640
<v Audience member (Zoom label: Schwab iPad)>to typing speed these days

769
00:47:18.640 --> 00:47:20.660
<v Audience member (Zoom label: Schwab iPad)>is also the layout of the QWERTY keyboard.

770
00:47:21.240 --> 00:47:23.220
<v Audience member (Zoom label: Schwab iPad)>And I'm sure you're aware there are other layouts

771
00:47:23.220 --> 00:47:25.620
<v Audience member (Zoom label: Schwab iPad)>that have been optimized for common keys

772
00:47:25.620 --> 00:47:28.140
<v Audience member (Zoom label: Schwab iPad)>to be in a different position that's easier to type to.

773
00:47:29.840 --> 00:47:32.520
<v Audience member (Zoom label: Schwab iPad)>So again, I'm not sure I actually have a question on that.

774
00:47:32.700 --> 00:47:34.980
<v Audience member (Zoom label: Schwab iPad)>It's just, you know, the ultimate,

775
00:47:35.060 --> 00:47:36.540
<v Audience member (Zoom label: Schwab iPad)>it seems to me the ultimate goal here

776
00:47:36.540 --> 00:47:39.080
<v Audience member (Zoom label: Schwab iPad)>is to make people input to the computer quicker.

777
00:47:39.700 --> 00:47:41.260
<v Audience member (Zoom label: Schwab iPad)>So maybe a question would be,

778
00:47:41.280 --> 00:47:42.740
<v Audience member (Zoom label: Schwab iPad)>how do your methods compare to like

779
00:47:42.740 --> 00:47:44.200
<v Audience member (Zoom label: Schwab iPad)>just changing the layout of the keyboard?

780
00:47:47.560 --> 00:47:51.640
<v Dan Mailman>Well, in fact, that's the reason I chose the letters as cues.

781
00:47:52.680 --> 00:47:54.600
<v Dan Mailman>So that's to separate the problem.

782
00:47:54.880 --> 00:47:58.540
<v Dan Mailman>So people can get faster or slower

783
00:47:58.540 --> 00:48:01.800
<v Dan Mailman>depending on the keyboard layout, perhaps, right?

784
00:48:02.040 --> 00:48:03.760
<v Dan Mailman>But an A is still an A

785
00:48:03.760 --> 00:48:07.120
<v Dan Mailman>and you know how to get to the A on the keyboard, right?

786
00:48:07.240 --> 00:48:10.340
<v Dan Mailman>So whether or not it's, you know, one layout or another,

787
00:48:10.880 --> 00:48:14.540
<v Dan Mailman>you can use the same cues for the same,

788
00:48:16.300 --> 00:48:17.900
<v Dan Mailman>targets for the same outputs.

789
00:48:18.380 --> 00:48:18.820
<v Dan Mailman>Right.

790
00:48:18.940 --> 00:48:19.460
<v Dan Mailman>Does that make sense?

791
00:48:19.640 --> 00:48:21.440
<v Dan Mailman>So separate, you know, it's at the engine,

792
00:48:21.560 --> 00:48:22.680
<v Dan Mailman>you know, divide and conquer.

793
00:48:22.860 --> 00:48:24.740
<v Dan Mailman>If you want to speed up your typing

794
00:48:24.740 --> 00:48:25.840
<v Dan Mailman>using a different layout,

795
00:48:26.080 --> 00:48:28.540
<v Dan Mailman>you can still use this software, right?

796
00:48:28.600 --> 00:48:30.080
<v Dan Mailman>Because an A is an A,

797
00:48:30.160 --> 00:48:32.140
<v Dan Mailman>but it doesn't matter where it is on the keyboard.

798
00:48:32.580 --> 00:48:33.980
<v Audience member (Zoom label: Schwab iPad)>Right. Yeah, that makes perfect sense.

799
00:48:34.140 --> 00:48:34.780
<v Audience member (Zoom label: Schwab iPad)>Yeah. Thank you.

800
00:48:35.140 --> 00:48:35.580
<v Audience member (Zoom label: Schwab iPad)>Surely.

801
00:48:41.930 --> 00:48:42.790
<v Yu Liang>More questions?

802
00:48:46.700 --> 00:48:48.860
<v Yingfeng Wang>So can committee members ask questions?

803
00:48:50.200 --> 00:48:50.640
<v Yu Liang>Later.

804
00:48:51.020 --> 00:48:52.660
<v Yu Liang>It is the closed-door discussion.

805
00:48:52.960 --> 00:48:53.180
<v Yu Liang>Yeah.

806
00:48:53.620 --> 00:48:56.500
<v Yu Liang>You can keep your question for the closed door.

807
00:48:56.620 --> 00:48:56.840
<v Yu Liang>Yeah.

808
00:48:56.840 --> 00:48:57.260
<v Yu Liang>Discussion.

809
00:48:58.480 --> 00:48:58.960
<v Yu Liang>Okay.

810
00:48:59.320 --> 00:48:59.800
<v Yu Liang>Yeah.

811
00:48:59.900 --> 00:49:01.700
<v Yu Liang>If no more question from audience.

812
00:49:01.980 --> 00:49:02.160
<v Yu Liang>Yeah.

813
00:49:02.280 --> 00:49:03.100
<v Yu Liang>Please just, yeah.

814
00:49:03.360 --> 00:49:06.860
<v Yu Liang>Let's give a big hand to Daniel's great presentation.

815
00:49:06.980 --> 00:49:07.680
<v Yu Liang>Thank you.

816
00:49:08.000 --> 00:49:08.740
<v Dan Mailman>Thank you.

817
00:49:08.860 --> 00:49:09.340
<v Dan Mailman>Yeah.

818
00:49:09.380 --> 00:49:11.900
<v Yu Liang>And committee member, please stay.

819
00:49:14.150 --> 00:49:16.210
<v Yingfeng Wang>Daniel, should you stop recording?

820
00:49:17.470 --> 00:49:17.950
<v Dan Mailman>Yes.

821
00:49:18.050 --> 00:49:18.490
<v Dan Mailman>Good for you.

822
00:49:18.550 --> 00:49:18.890
<v Dan Mailman>Thank you.

823
00:49:19.110 --> 00:49:20.690
<v Dan Mailman>I'm going to do that right now.

824
00:49:21.090 --> 00:49:21.570
<v Yu Liang>Okay.

825
00:49:21.650 --> 00:49:22.370
<v Yu Liang>Thanks, everyone.

826
00:49:22.490 --> 00:49:23.230
<v Yu Liang>Thank you.

827
00:49:23.290 --> 00:49:23.490
<v Yu Liang>Yeah.

828
00:49:23.530 --> 00:49:24.490
<v Yu Liang>Let's just wait a minute.

