Drawings for Alternative Communications Patent

1 / 43

Inventor(s): TEDESCO et al.Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL,

BEHAVIORAL, AND BIOLOGICAL CONTEXTSDocket No.: 2001-011.0001

Filing Date: FILED HEREWITH

1 / 4

3

22

FIG. 1

AT&T 3G 8:22 AM

Text Calendar Photos Camera

Stocks Maps WeatherRecorder

Calculator Notes SettingsClock

AAC+

Mail Safari iPodPhone

3

SMS 11

Thursday

73*

+ -

x /

1014

16

20

18

12

AAC+

2 / 43

Inventor(s): TE

DE

SC

O et al.

Title: M

ET

HO

DS

FO

R A

UG

ME

NT

AT

IVE

AN

D A

LTE

RN

AT

IVE

CO

MM

UN

ICA

TIO

NS

INF

OR

ME

D B

Y E

NV

IRO

NM

EN

TA

L, B

EH

AV

IOR

AL, A

ND

BIO

LOG

ICA

L CO

NT

EX

TS

Docket N

o.: 2001-011.0001F

iling Date: F

ILED

HE

RE

WIT

H

2 / 4

3

FIG. 2PRIOR ART

MICROPHONE SPEAKER KEYBOARDDISPLAYCAMERA

USER INTERFACE

10

24 16 26 14 18

12

CPU

28

ACCELEROMETER

34

MEMORY

30

SOFTWARETRANSCEIVER

CIRCUITRY

36

32

WIRED

PORT

40

BATTERY

42

MISCELLANEOUS

PERIPHERALS

44

38

3 / 43




3 / 4

3

FIG. 3PRIOR ART

PSTN

INTERNET

PLMN

46

50

52

10D

56

10C

10B 10A

48

54

4 / 43




4 / 4

3

SMS11

73*

RH

10

58e

58d

58b

FIG. 4

58c

58a

62

6058f

64

5 / 43




5 / 4

3

FIG. 5

CAREGIVER ACQUIRES MOBILE TERMINAL

SOFTWARE LOADED ONTO MOBILE TERMINAL

ESTABLISH A USER PROFILE

STORE A CONTEXTUAL CRITERION FOR

OUTPUTTING A SPEECH ICON

OUTPUT THE SPEECH ICON

100

102

104

106

110

108

IS CRITERION SATISFIED?

YES

NO

6 / 43




6 / 4

3

AT&T 3G 8:22 AM

Profiles

AAC+BACK

AAC+

10

16

18

General Settings

Profiles and Settings

FIG. 6

AAC+

TO FIG. 8

“Caregiver Mode” Enabled

(tap to change to “End User Mode”)

7 / 43

Inventor(s): TE

DE

SC

O et al.

Title: M

ET

HO

DS

FO

R A

UG

ME

NT

AT

IVE

AN

D A

LTE

RN

AT

IVE

CO

MM

UN

ICA

TIO

NS

INF

OR

ME

D B

Y E

NV

IRO

NM

EN

TA

L, B

EH

AV

IOR

AL, A

ND

BIO

LOG

ICA

L CO

NT

EX

TS

Docket N

o.: 2001-011.0001F

iling Date: F

ILED

HE

RE

WIT

H

7 / 4

3

FIG. 7

USER SELECTS PROFILES COMMAND

USER SELECTS PRE-EXISTING PROFILE OR TEMPLATE

150

152USER CREATES NEW PROFILE

158

LOAD154

GO TO BLOCK 108 OF FIGURE 5

EDIT156

GIVE NAME TO PROFILE160

GO TO BLOCK 104 OF FIGURE 5

8 / 43




8 / 4

3

FIG. 8

AT&T 3G 8:22 AM

AAC+

Profiles

BACK

Custom profiles

Templates

Nonverbal adolescent with autism

Help with expressing feelings

Adult user with semantic-pragmatic...

Jimmy Smith

Laura’s normal settings

Settings for Laura’s vacation

TO FIG. 10

TO FIG. 9200

202

204

9 / 43




9 / 4

3

FIG. 9

AT&T 3G 8:22 AM

AAC+

Profile: Jimmy Smith

BACK

208

220

218

EDIT

Icon size: Large

Default voice: Young Male

Assoc. #1: “McD Menu” at McDonalds

LOAD

210

212

206

Assoc. #2: “Help group” after “Help!”

214

Assoc. #3: “Basic conversation” near Mrs.

Miller

216

10 / 43




10

/ 43

FIG. 10

AT&T 3G 8:22 AM

AAC+

Create / Edit a Profile

BACK

Profile Name:

Enter name here

Speech/Language Impairment Info

General Settings for This Profile

Icons

SAVE PROFILE

To Fig. 11

To Fig. 15

To Fig. 30

User’s Schedule/Calendar

Icon-Context Associations To Fig. 18

Contexts To Fig. 21

11 / 43




11

/ 43

FIG. 11

=

AT&T 3G 8:22 AM

AAC+BACK

How would you like to provide speech/

language impairment information?

Take a survey

Enter specific impairment info

Perform a user evaluation

TO FIG. 12

TO FIG. 13

TO FIG. 14

Import data from another source

12 / 43




12

/ 43

FIG. 12

=

AT&T 3G 8:22 AM

Language Development SurveyBACK

In response to the following questions, please

mark “Yes” or “No”.

Does the user know the correct names of

some colors?

YES NO

Does the user say about 50 different words?

YES NO

Does the user say about 100 different words?

YES NO

Does the user say words that tell something

about other words, such as “loud” in the

utterance a “loud noise”?

13 / 43




13

/ 43

FIG. 13

=

AT&T 3G 8:22 AM

Enter Impairment InformationBACK

Dyslexia

Does user have dyslexia?

Vocabulary

General vocabulary level

Known vocabulary words

Stuttering and Annunciation

Below Avg Avg Above Avg

NO

Does user stutter? YES

Known annunciatory problem words:

Type here (e.g., “bathroom”)

CATEGORIES Colors

Aqua

Blue

Brown

Burgundy

Crimson

Forest Green

Gold

Green

Lavender

14 / 43




14

/ 43

FIG. 14

=

3G 8:22 AM

Say: “bathroom”BACK

Say the word “bathroom”:

PLAYSTOP

SaveUndo

RECREW 66

66a

14

Say, “bath-room”

please.

15 / 43




15

/ 43

FIG. 15

=

AT&T 3G 8:22 AM

AAC+BACK

Which general settings would you like to

program for this profile?

Caption font

Caption size

Icon style

Icon size

Language

Voices

Repetitions

Participatory voice output

Icons per screen

TO FIG. 16

TO FIG. 17

Select a default voice:

Young male (synthetic)

Adult female (synthetic)

Rosa

Voice 124153

Jimmy Smith’s voice

Save

68

Volume

16 / 43




16

/ 43

FIG. 16

=

AT&T 3G 8:22 AM

AAC+BACK

Configure default “Repetitions” settings that

should apply to this profile:

Number of repetitions

Accuracy threshold

Penalties

Rewards

Prompt users to repeat

voice outputs

Number of required repetitions:

Save

ON

0

1

2

3

4

How accurate must the repetition be?

Save

30%

40%

50%

60%

70%

Configure “Repetitions” for a particular speech

icon or group of icons:

Browse categories

Search

Penalties

Save

Enforce a penalty if

user fails to repeat

Disable all voice output

OFF

Disable a speech icon

Restrict game/Web access

Rewards

Save

Give a reward if user

completes repetition

Tokens and points

ON

Messages

Enable game/Web access

TO

FIG. 30

17 / 43




17

/ 43

FIG. 17

=

AT&T 3G 8:22 AM

AAC+BACK

Configure default “Participatory voice output”

settings that should apply to this profile:

Frequency of participation

Accuracy threshold

Penalties

Rewards

Prompt users to participate

in voice outputs

Frequency of participation:

SaveON

How accurate must the participation be?

Save

30%

40%

50%

60%

70%

TO FIG. 30

Sometimes

Word or phrase length

Word or phrase type

Always

Browse categories

Search

Specific words, phrases, etc:

Type of participation

Save

User finishes

Step by step

Increase MLU over time

User begins

Type of participation:

18 / 43




18

/ 43

FIG. 18

AT&T 3G 8:22 AM

Icon-Context AssociationsBACK +

“BK Menu” at 32 Main Street Burger King

“Help group” after “Help!”

Rule 1 (11/15/10 4:57 pm)

“Basic conversation” near Mrs. Miller

Rule 2 (11/23/10 6:18 am)

“Recess icons” on Mon-Fri from 12:15 to...

“Nervous emotions” based on heart rate and…

TO FIG. 19

“Swing set” at McNamee Park

19 / 43




19

/ 43

FIG. 19

=

AT&T 3G 8:22 AM

BACK Create TO FIG. 20

WQ E R T Y U I O P

SA D F G H J K L

XZ C V B N M

space return.?123

Name your Icon-Context Association:

Icon-Context Assoc.

20 / 43




20

/ 43

FIG. 20

=

AT&T 3G 8:22 AM

Icon-Context AssociationsBACK

How would you like to create an

association between a speech icon and a

particular context?

Start with a context, then select

icon(s) that should be presented

in that context

Start with an icon, then select

context(s) in which that icon should

be presented

TO FIG. 21

TO FIG. 30

21 / 43




21

/ 43

FIG. 21

=

AT&T 3G 8:22 AM

ContextsBACK

Sounds, noises or speech

Physical movement

Objects or people near this device

TO FIG. 22

Temperature

Lighting

The location of this device

The user’s body

Times, dates and scheduled events

+ TO FIG. 29

Search

22 / 43




22

/ 43

FIG. 22

=

AT&T 3G 8:22 AM

Icon-Context AssociationsBACK

Specify a location that should trigger

output of a speech icon:

Search/browse by location name

Find a location on a map

Draw a “zone” on a map

How would you like to identify a

location?

70

TO FIG. 23

Browse categories

Search

23 / 43




23

/ 43

FIG. 23

AT&T 3G 8:22 AM

Results for “McDonalds”BACK

1. 5065 Main Street | Anycity, USA | 2.84 mi

2. 390 Route 105 | Anycity, USA | 4.31 mi

3. 12 Drawley Lane | Nextown, USA | 5.39 mi

4. 101 Elm Street | Nextown, USA | 7.01 mi

5. 65 Far Drive | Farville, USA | 12.4 mi

Show More Results

TO FIG. 24View Selected

24 / 43




24

/ 43

FIG. 24

AT&T 3G 8:22 AM

View/edit context (McDonald…)BACK

McDonald’s

5065 Main Street | Anycity, USA

General Information

TO FIG. 25

ATM WIFI

View/Edit Icons and Settings for “McDonald…”

Add New Speech Icon(s)

Browse categories

Search

More

info

TO FIG. 27No icons have been saved in

association with this location.

25 / 43




25

/ 43

FIG. 25

AT&T 3G 8:22 AM

Icon results for “fries”BACK

FRENCH

FRIES (1)

Saved to 13 contexts

Frequency of use: High

Add Selected Icons to Context

FRENCH

FRIES (2)

Saved to 4 contexts

Frequency of use: Medium

FRIED

DOUGH

Saved to 0 contexts

Frequency of use: Low

More

info

More

info

More

info

26 / 43




26

/ 43

FIG. 26

AT&T 3G 8:22 AM

Add “FRENCH FRIES (1) to…”BACK

FRENCH

FRIES (1)

Add speech icon:

FRENCH FRIES (1)

to “McDonald’s

5065 Main St.”?

TO FIG. 27

McDonald’s

5065 Main

Street

Save Icon to This Location

27 / 43




27

/ 43

FIG. 27

AT&T 3G 8:22 AM

Icons and settings for “McD…”BACK

BURGER FISH

SANDWICH

FRENCH

FRIES

Caption font

Caption size

Icon style

Icon size

Language

Voices

Repetitions

Participatory voice output

Icons per screen

Icons (6):

Settings for this context:

Icon ranking

Screen

Editor

TO FIG. 28

Volume

28 / 43

Inventor(s): TE

DE

SC

O et al.

Title: M

ET

HO

DS

FO

R A

UG

ME

NT

AT

IVE

AN

D A

LTE

RN

AT

IVE

CO

MM

UN

ICA

TIO

NS

INF

OR

ME

D B

Y E

NV

IRO

NM

EN

TA

L, B

EH

AV

IOR

AL, A

ND

BIO

LOG

ICA

L CO

NT

EX

TS

Docket N

o.: 2001-011.0001F

iling Date: F

ILED

HE

RE

WIT

H

28 / 4

3Screen Editor for “McDonald’s 5056 Main Street”BACK

FIG. 28

Screen Editor for “McDonald’s 5056 Main Street”BACK

BURGER

FISH SANDWICH

FRENCH FRIES

ICE CREAM

PLEASECOLA

72

74

76

78

80

82

SAVE

SAVE

29 / 43




29

/ 43

FIG. 29

AT&T 3G 8:22 AM

Create a new locationBACK

Name this location:

Drag and drop pins to define boundaries:

SaveUndo

Enter name here

84

30 / 43




30

/ 43

FIG. 30

AT&T 3G 8:22 AM

IconsBACK +

actions

animals

around the house

art

body parts

clothing

colors

Search TO FIG. 31

31 / 43




31

/ 43

FIG. 31

AT&T 3G 8:22 AM

Icon results for “burger”BACK

BURGER

Saved to 9 contexts

Frequency of use: HighTO FIG. 32

BURGER

WITH BACON

Saved to 2 contexts


HAM

BURGER

Saved to 0 contexts


32 / 43




32

/ 43

FIG. 32

AT&T 3G 8:22 AM

Icons: BURGERBACK

BURGER

Saved to 9 contexts

Frequency of use: High

Related icons

Caption font (Arial)

Caption size (Normal)

Icon style (Photo)

Icon size (Medium)

Language (English)

Voices (Young male)

Repetitions (OFF)

Participatory voice output (OFF)

Settings for this icon:

Volume (Default)

TO FIG. 21Add “BURGER” to a new context

33 / 43




33

/ 43

FIG. 33

AT&T 3G 8:22 AM

View/edit context (Low blood…)BACK

Low blood sugar

(beneath 70 mg/dL for more than

15 minutes)

General Information

View/Edit Icons and Settings for “Low blood…”

Add New Speech Icon(s)

Browse categories

Search

More

infoBURGER

70

34 / 43

Inventor(s): TE

DE

SC

O et al.

Title: M

ET

HO

DS

FO

R A

UG

ME

NT

AT

IVE

AN

D A

LTE

RN

AT

IVE

CO

MM

UN

ICA

TIO

NS

INF

OR

ME

D B

Y E

NV

IRO

NM

EN

TA

L, B

EH

AV

IOR

AL, A

ND

BIO

LOG

ICA

L CO

NT

EX

TS

Docket N

o.: 2001-011.0001F

iling Date: F

ILED

HE

RE

WIT

H

34 / 4

3

FIG. 34A

=

8:22 AM

Create a new location

Name this location:

SaveUndo

FIG. 34B FIG. 34C

10

Drag and drop pins to define boundaries:

“Maiden Ridge” Neighborhood

STEPHEN

“Maiden Ridge” Speech Icons (Screen 1)BACK

SALLY

GO

JOE PETER

PASS TURN



ME

WANT

I MRS. SMITH

HOME PLEASE

SKATEBOARD

RAMP

BASKETBALL

FOOTBALL

COOL

FUN

COOL

FUN

86a

86b

86c

88a

88b

90

88c

92

16

35 / 43

Inventor(s): TE

DE

SC

O et al.

Title: M

ET

HO

DS

FO

R A

UG

ME

NT

AT

IVE

AN

D A

LTE

RN

AT

IVE

CO

MM

UN

ICA

TIO

NS

INF

OR

ME

D B

Y E

NV

IRO

NM

EN

TA

L, B

EH

AV

IOR

AL, A

ND

BIO

LOG

ICA

L CO

NT

EX

TS

Docket N

o.: 2001-011.0001F

iling Date: F

ILED

HE

RE

WIT

H

35 / 4

3

FIG. 35A

Set maximum levels for:

FIG. 35B FIG. 35C

130 BPM

Heart Rate

Med-High

Electrodermal Activity

Heart R

ate (BP

M):

Levels Exceeded!

Transmitting message...

GS

R:

8:22 AM

AAC+

10

58b

58d

-5000

0

5000

10000

15000

=

8:22 AM

11

7

3

*

MedLow Med-Low

54

AAC+ -- Caregiver Workstation

Profile: Karen Settings History Progress

Save

Changes

Undo

Now editing Icon-

Context Association:

“Nervous emotions

based on heart rate

and electrodermal...”

AAC+

Are you feeling

NERVOUS or

SCARED?

“Nervous Emotions...” Speech Icons (Screen 1)BACK

NOW

94b

94a

I FEEL

NERVOUS EXCITED SCARED FINE

Active Voice OutputBACK

NOW”“I FEEL NERVOUS

14

94c

SPEAK

96

10

56

36 / 43

Inventor(s): TE

DE

SC

O et al.

Title: M

ET

HO

DS

FO

R A

UG

ME

NT

AT

IVE

AN

D A

LTE

RN

AT

IVE

CO

MM

UN

ICA

TIO

NS

INF

OR

ME

D B

Y E

NV

IRO

NM

EN

TA

L, B

EH

AV

IOR

AL, A

ND

BIO

LOG

ICA

L CO

NT

EX

TS

Docket N

o.: 2001-011.0001F

iling Date: F

ILED

HE

RE

WIT

H

36 / 4

3

58e

FIG. 36A

=

8:22 AM

“MILLER!” (voiceprint)

Record audio for “MILLER!”:

PLAYSTOP

SaveUndo

RECREW

62

Cur

rent

aud

io:

MATCH!

Opening “Basic Conv…”

“MIL

LER

!”:

8:22 AM

AAC+

FIG. 36B FIG. 36C

“Basic Conversation near Mrs. Miller” Speech IconsBACK

MILLERHI MRS.

14

Repetition Exercise!BACK

MILLERHI MRS.

JIMMY - NOW YOU SAY:


MILLERHI MRS.

Listening...

98a

98b 98c

37 / 43

Inventor(s): TE

DE

SC

O et al.

Title: M

ET

HO

DS

FO

R A

UG

ME

NT

AT

IVE

AN

D A

LTE

RN

AT

IVE

CO

MM

UN

ICA

TIO

NS

INF

OR

ME

D B

Y E

NV

IRO

NM

EN

TA

L, B

EH

AV

IOR

AL, A

ND

BIO

LOG

ICA

L CO

NT

EX

TS

Docket N

o.: 2001-011.0001F

iling Date: F

ILED

HE

RE

WIT

H

37 / 4

3

58e

FIG. 36D FIG. 36E FIG. 36F


MILLERHELLO MRS.

14


JIMMY - NOW YOU SAY:


98i

98j 98k


MILLERHI MRS.

Listening...

98d


MILLERHI MRS.

Listening...

98e


MILLERHI MRS.

Repetition Complete!

98f


Congratulations!

98g

14

MILLERHI MRS.

You earned 10 points

toward your goal.

Goal: Trip to moviesBACK

98h

90%

Your goal:

Progress:TRIP TO MOVIES

THERE

MILLERHELLO MRS.THERE

Congratulations!

You earned:

MILLERHELLO MRS.THERE

100%

TRIP TO MOVIES

38 / 43

Inventor(s): TE

DE

SC

O et al.

Title: M

ET

HO

DS

FO

R A

UG

ME

NT

AT

IVE

AN

D A

LTE

RN

AT

IVE

CO

MM

UN

ICA

TIO

NS

INF

OR

ME

D B

Y E

NV

IRO

NM

EN

TA

L, B

EH

AV

IOR

AL, A

ND

BIO

LOG

ICA

L CO

NT

EX

TS

Docket N

o.: 2001-011.0001F

iling Date: F

ILED

HE

RE

WIT

H

38 / 4

3

LOADED AS CURRENT PROFILE?: YES

FIG. 37

I-001

PROFILE ID: P-0001

PROFILE NAME: JIMMY SMITH

HOT HOT.TXT NONE DEFAULT 0 OFF HOT.WAV

I-002 FOOD FOOD.GIF NONE MEDIUM 0 OFF (TTS)

I-003 LOST LOST.MOV C-001 LARGE 0 OFF LOST.MOV

NONE

NONE

“EMERGENCY”

I-004 REFRIGERATOR FRIDGE.JPG C-003 DEFAULT 1 OFF FRIDGE.MP3

I-00N ETC. ETC. ETC. ETC. ETC. ETC. ETC.

“IN THE

KITCHEN”

ETC.

ICON

IDICON LABEL ICON FILE

ASSOCIATED

CONTEXT IDICON SIZE REPETITIONS PARTICPATION

ASSOCIATED

VOICE

ASSOCIATED

CONTEXT

LABEL

SPEECH ICON DATABASE

39 / 43

Inventor(s): TE

DE

SC

O et al.

Title: M

ET

HO

DS

FO

R A

UG

ME

NT

AT

IVE

AN

D A

LTE

RN

AT

IVE

CO

MM

UN

ICA

TIO

NS

INF

OR

ME

D B

Y E

NV

IRO

NM

EN

TA

L, B

EH

AV

IOR

AL, A

ND

BIO

LOG

ICA

L CO

NT

EX

TS

Docket N

o.: 2001-011.0001F

iling Date: F

ILED

HE

RE

WIT

H

39 / 4

3


FIG. 38

C-001

PROFILE ID: P-0001


EMERGENCYI-003, I-138,

I-284, I-019...LARGE

DEFAULT TO

PROFILE

BODY TEMP >

100* F

PRESSURE

SENSOR

DETECTS...

BRIGHTNESS

“VERY HIGH”

OR “VERY L…”

C-002BULLY

NEARBYI-675, I-029,

I-444, I-572LARGE

DEFAULT TO

PROFILE

MT-ID-001

WITHIN 10

METERS

LOCATION IS

“ELM SCHOOL”ETC.

C-003IN THE

KITCHEN

I-002, I-004,

I-887,I-373...

DEFAULT TO

ICON

DEFAULT TO

PROFILE

WORD

“KITCHEN” IS

UTTERED

CAMERA

DETECTS... ETC.

0

DEFAULT TO

ICON

DEFAULT TO

ICON

C-004NERVOUS

EMOTIONS BA...

I-432, I-071,

I-273, I884...

DEFAULT TO

PROFILEON

HEART RATE >

130 BPM

GSR DATA

INDICATES...ETC.

I-00N ETC. ETC. ETC. ETC. ETC. ETC. ETC.

DEFAULT TO

ICON

ETC.

CONTEXT

ID

CONTEXT

LABEL

ASSOCIATED

ICON ID(S)ICON SIZE PARTICPATION TRIGGER 1 TRIGGER 2 TRIGGER NREPETITIONS

SAVED CONTEXT DATABASE

40 / 43

Inventor(s): TE

DE

SC

O et al.

Title: M

ET

HO

DS

FO

R A

UG

ME

NT

AT

IVE

AN

D A

LTE

RN

AT

IVE

CO

MM

UN

ICA

TIO

NS

INF

OR

ME

D B

Y E

NV

IRO

NM

EN

TA

L, B

EH

AV

IOR

AL, A

ND

BIO

LOG

ICA

L CO

NT

EX

TS

Docket N

o.: 2001-011.0001F

iling Date: F

ILED

HE

RE

WIT

H

40 / 4

3

FIG. 39

WEDNESDAY

FEBRUARY 13:13 PM

PLAYDATE

WITH SALLY30- 40 DB

“HELP,” CAR

HORN, ...JIMMY, SALLY

MCNAMEE

PARK

DATE TIMEEVENT/

SCHEDULEAMBIENT

VOLUME

LAST 5

SOUNDS

PEOPLE

PRESENTLOCATION

SLIDE MEDIUM-HIGH 77* F 99* F 82 MG/DL NORMALNORMAL

DAYLIGHT

OBJECTS

PRESENT

MOTION

ACTIVITY

OUTSIDE

TEMPERATUREBODY

TEMPERATURE

BLOOD

GLUCOSE

LEVEL

BLOOD

PRESSUREBRIGHTNESS


PROFILE ID: P-0001


CURRENT CONTEXT DATABASE

41 / 43

Inventor(s): TE

DE

SC

O et al.

Title: M

ET

HO

DS

FO

R A

UG

ME

NT

AT

IVE

AN

D A

LTE

RN

AT

IVE

CO

MM

UN

ICA

TIO

NS

INF

OR

ME

D B

Y E

NV

IRO

NM

EN

TA

L, B

EH

AV

IOR

AL, A

ND

BIO

LOG

ICA

L CO

NT

EX

TS

Docket N

o.: 2001-011.0001F

iling Date: F

ILED

HE

RE

WIT

H

41 / 4

3


I-039

PROFILE ID: P-0001


SHAREACTIONS,

SOCIAL

WEDNESDAY,

2/1, 1:13 PM

VOICEPRINT

OF SALLY

SLIDE, BALL,

SALLY, ...NONE 97*

I-447 BULLYPEOPLE,

SCHOOL, ...

THURSDAY,

2/9, 12:17 PM

60 DB+

(UNINTEL...)UNKNOWN MT-ID-001 97*

I-602 NONE DRINKSSUNDAY,

2/12, 11:15 AM(NONE) SUN NONE 100*

PLAYGROUND

SCHOOL

BACKYARD

ICON

IDICON LABEL

ICON

CATEGORY/

CATEGORIES

DATE / TIMEDETECTED BY

MICROPHONE

DETECTED BY

CAMERA

OTHER

MOBILE

TERMINALS

NEARBY

BODY

TEMPERATURELOCATION

USAGE HISTORY DATABASE

FIG. 40

I-00N ETC. ETC. ETC. ETC. ETC. ETC. ETC.ETC.

42 / 43

Inventor(s): TE

DE

SC

O et al.

Title: M

ET

HO

DS

FO

R A

UG

ME

NT

AT

IVE

AN

D A

LTE

RN

AT

IVE

CO

MM

UN

ICA

TIO

NS

INF

OR

ME

D B

Y E

NV

IRO

NM

EN

TA

L, B

EH

AV

IOR

AL, A

ND

BIO

LOG

ICA

L CO

NT

EX

TS

Docket N

o.: 2001-011.0001F

iling Date: F

ILED

HE

RE

WIT

H

42 / 4

3


I-039

PROFILE ID: P-0001


SHARE (21)ACTIONS (61),

SOCIAL (17)

WEDNESDAY

(11)

PLAYGROUND

(13)

SALLY

(10)

BALL

(10)

MOM’S PHONE

(4)

TUESDAY

(5)

HOME

(6)

“BALL”

(5)

OTHER

(11)

NONE

(17)

SATURDAY

(5)

OTHER

(2)

OTHER

(6)

12:00 – 1:00

PM (8)

3:00 – 4:00 PM

(7)

2:00 – 3:00 PM

(6)

ICON

IDICON LABEL

ICON

CATEGORY/

CATEGORIES

DATE LOCATIONDETECTED BY

MICROPHONE

DETECTED BY

CAMERA

OTHER

MOBILE

TERMINALS

NEARBY

TIME

USAGE TRENDS DATABASE

(INCIDENCE IN PARENTHESIS)

FIG. 41

43 / 43

Inventor(s): TE

DE

SC

O et al.

Title: M

ET

HO

DS

FO

R A

UG

ME

NT

AT

IVE

AN

D A

LTE

RN

AT

IVE

CO

MM

UN

ICA

TIO

NS

INF

OR

ME

D B

Y E

NV

IRO

NM

EN

TA

L, B

EH

AV

IOR

AL, A

ND

BIO

LOG

ICA

L CO

NT

EX

TS

Docket N

o.: 2001-011.0001F

iling Date: F

ILED

HE

RE

WIT

H

43 / 4

3


FIG. 42

PROFILE ID: P-0001


LANGUAGE

DEVELOPMENT

SURVEY

QUESTION 1

YES

LANGUAGE

DEVELOPMENT

SURVEY

QUESTION 2

YES

GENERAL

VOCABULARY

LEVEL

40TH

PERCENTILE

KNOWN

VOCABULARY:

COLORS

BLUE,

YELLOW

KNOWN

VOCABULARY:

PEOPLE

ME, DAD,

MOM, SARAH...

KNOWN

VOCABULARY:

TRANSPORT

BIKE, CAR,

TRAIN...

DYSLEXIA

NO

LANGUAGE

DEVELOPMENT

SURVEY

QUESTION 3

NO

LARYNX

TRAUMA

NO

LANGUAGE

REGRESSION

TRIGGER 1

LOUD AREAS

LANGUAGE

REGRESSION

TRIGGER 2

NEW PEOPLE

STUTTERING

YES

(MODERATE)

STUTTERING

PROBLEM

WORD 1

BATHROOM

STUTTERING

PROBLEM

WORD 2

BARKING

STUTTERING

PROBLEM

WORD 3

CHICKEN

STUTTERING

PROBLEM

WORD 4

CHURCH

USER IMPAIRMENT DATABASE

Documents

Drawings for Alternative Communications Patent