Kaytetye coronal contrasts withoutcontours
Susan Lin, Benjamin Davies, and Katherine [email protected]
ARC Centre for Excellence in Cognition and its DisordersMacquarie University
October 6, 2013Ultrafest VIEdinburgh
Kaytetye coronal contrasts withoutedge-detection
Susan Lin, Benjamin Davies, and Katherine [email protected]
ARC Centre for Excellence in Cognition and its DisordersMacquarie University
October 6, 2013Ultrafest VIEdinburgh
IntroductionProcedures
Results
Ultrasound in practiceA place for qualitative observations?Kaytetye coronal contrast: a test case
Why use ultrasound?
Benefits
I Good spacial resolution
I Reasonable temporal resolution (and improving!)
I Portability
I Immediacy of visual information
Consequence
Portable ultrasound machines are being employed in fieldwork anddocumentation, in increasing numbers.
2
IntroductionProcedures
Results
Ultrasound in practiceA place for qualitative observations?Kaytetye coronal contrast: a test case
Why use ultrasound?
Benefits
I Good spacial resolution
I Reasonable temporal resolution (and improving!)
I Portability
I Immediacy of visual information
Consequence
Portable ultrasound machines are being employed in fieldwork anddocumentation, in increasing numbers.
2
IntroductionProcedures
Results
Ultrasound in practiceA place for qualitative observations?Kaytetye coronal contrast: a test case
Typical procedures
1. Collect
2. Post-processI Extract relevant framesI Edge-detection of contours
3. Analyze
4. Interpret
Post-processing may be time-consuming
I Infrastructure cost (in time) high for small projects and/orprojects with simple questions
I Processing time may be too high for performing while on site
3
IntroductionProcedures
Results
Ultrasound in practiceA place for qualitative observations?Kaytetye coronal contrast: a test case
Typical procedures
1. Collect
2. Post-process
I Extract relevant framesI Edge-detection of contours
3. Analyze
4. Interpret
Post-processing may be time-consuming
I Infrastructure cost (in time) high for small projects and/orprojects with simple questions
I Processing time may be too high for performing while on site
3
IntroductionProcedures
Results
Ultrasound in practiceA place for qualitative observations?Kaytetye coronal contrast: a test case
Typical procedures
1. Collect
2. Post-processI Extract relevant framesI Edge-detection of contours
3. Analyze
4. Interpret
Post-processing may be time-consuming
I Infrastructure cost (in time) high for small projects and/orprojects with simple questions
I Processing time may be too high for performing while on site
3
IntroductionProcedures
Results
Ultrasound in practiceA place for qualitative observations?Kaytetye coronal contrast: a test case
Typical procedures
1. Collect
2. Post-processI Extract relevant framesI Edge-detection of contours
3. Analyze
4. Interpret
Post-processing may be time-consuming
I Infrastructure cost (in time) high for small projects and/orprojects with simple questions
I Processing time may be too high for performing while on site
3
IntroductionProcedures
Results
Ultrasound in practiceA place for qualitative observations?Kaytetye coronal contrast: a test case
Typical procedures
1. Collect
2. Post-processI Extract relevant framesI Edge-detection of contours
3. Analyze
4. Interpret
Post-processing may be time-consuming
I Infrastructure cost (in time) high for small projects and/orprojects with simple questions
I Processing time may be too high for performing while on site
3
IntroductionProcedures
Results
Ultrasound in practiceA place for qualitative observations?Kaytetye coronal contrast: a test case
Typical procedures
1. Collect
2. Post-processI Extract relevant framesI Edge-detection of contours
3. Analyze
4. Interpret
Post-processing may be time-consuming
I Infrastructure cost (in time) high for small projects and/orprojects with simple questions
I Processing time may be too high for performing while on site
3
IntroductionProcedures
Results
Ultrasound in practiceA place for qualitative observations?Kaytetye coronal contrast: a test case
Typical procedures
1. Collect
2. Post-processI Extract relevant framesI Edge-detection of contours
3. Analyze
4. Interpret
Post-processing may be time-consuming
I Infrastructure cost (in time) high for small projects and/orprojects with simple questions
I Processing time may be too high for performing while on site
3
IntroductionProcedures
Results
Ultrasound in practiceA place for qualitative observations?Kaytetye coronal contrast: a test case
Reducing the barrier to entry
Question
How can we reduce the barrier to entry (whether real orperceived)?
At least two ways:
I Speed up or automate edge-detection
I Develop alternatives to edge-detection
4
IntroductionProcedures
Results
Ultrasound in practiceA place for qualitative observations?Kaytetye coronal contrast: a test case
Reducing the barrier to entry
Question
How can we reduce the barrier to entry (whether real orperceived)?
At least two ways:
I Speed up or automate edge-detection
I Develop alternatives to edge-detection
4
IntroductionProcedures
Results
Ultrasound in practiceA place for qualitative observations?Kaytetye coronal contrast: a test case
Reducing the barrier to entry
Question
How can we reduce the barrier to entry (whether real orperceived)?
At least two ways:
I Speed up or automate edge-detection
I Develop alternatives to edge-detection
4
IntroductionProcedures
Results
Ultrasound in practiceA place for qualitative observations?Kaytetye coronal contrast: a test case
Reducing the barrier to entry
Question
How can we reduce the barrier to entry (whether real orperceived)?
At least two ways:
I Speed up or automate edge-detection
I Develop alternatives to edge-detection
4
IntroductionProcedures
Results
Ultrasound in practiceA place for qualitative observations?Kaytetye coronal contrast: a test case
An observation (from a colleague)
./images/poster.jpgaNg@n@Na at”@N@
5
IntroductionProcedures
Results
Ultrasound in practiceA place for qualitative observations?Kaytetye coronal contrast: a test case
Testing ground: Kaytetye
I Arandic language
I Spoken 300km north ofAlice Springs, NT
I Dozens of speakersremaining
6
IntroductionProcedures
Results
Ultrasound in practiceA place for qualitative observations?Kaytetye coronal contrast: a test case
Kaytetye coronal inventory (phonemic)
dental alveolar retroflex palatal pre-palatal
oral stop t” t ú c jcnasal n” n ï ñ jñ
pre-stopped nasal t”n” tn úï cñjcñ
lateral l” l í L jL
Test question
Does production of alveolar compared to dental stops in Kaytetyeinvolve different motion of the tongue?
7
IntroductionProcedures
Results
Ultrasound in practiceA place for qualitative observations?Kaytetye coronal contrast: a test case
Kaytetye coronal inventory (phonemic)
dental alveolar retroflex palatal pre-palatal
oral stop t” t ú c jcnasal n” n ï ñ jñ
pre-stopped nasal t”n” tn úï cñjcñ
lateral l” l í L jL
Test question
Does production of alveolar compared to dental stops in Kaytetyeinvolve different motion of the tongue?
7
IntroductionProcedures
Results
Ultrasound in practiceA place for qualitative observations?Kaytetye coronal contrast: a test case
Kaytetye coronal inventory (phonemic)
dental alveolar retroflex palatal pre-palatal
oral stop t” t ú c jcnasal n” n ï ñ jñ
pre-stopped nasal t”n” tn úï cñjcñ
lateral l” l í L jL
Test question
Does production of alveolar compared to dental stops in Kaytetyeinvolve different motion of the tongue?
7
IntroductionProcedures
Results
DataSightedMeasured
Stimuli
I Target stops elicited in/#a a/ or /#a @/ contexts
I 8-12 repetitions of each item
I 7 female native speakers ofKaytetye (all multi-lingual tosome extent)
I Probe held by researcher
8
IntroductionProcedures
Results
DataSightedMeasured
Frames extracted
I 1 vocalic frame, from preceding vowel (a)
I 1 consonantal frame, at maximal constriction of targetconsonant
at”@N@
9
IntroductionProcedures
Results
DataSightedMeasured
Frames extracted
I 1 vocalic frame, from preceding vowel (a)
I 1 consonantal frame, at maximal constriction of targetconsonant
at”@N@ at”@N@
9
IntroductionProcedures
Results
DataSightedMeasured
Post-processing: sighted
at”@N@
1. Color consonantal frame (red)
2. Turn blacks in consonantal frame transparent
3. Combine frames
10
IntroductionProcedures
Results
DataSightedMeasured
Post-processing: sighted
at”@N@
1. Color consonantal frame (red)
2. Turn blacks in consonantal frame transparent
3. Combine frames
10
IntroductionProcedures
Results
DataSightedMeasured
Post-processing: sighted
at”@N@
1. Color consonantal frame (red)
2. Turn blacks in consonantal frame transparent
3. Combine frames
10
IntroductionProcedures
Results
DataSightedMeasured
Heights sighted
Naive coder instructed to “mentally trisect the white line in thecenter of the image” and rate
I +1 if red line is further from center than white
I 0 if red line is the same distance as the white from center
I -1 if red line is closer to center than white
11
IntroductionProcedures
Results
DataSightedMeasured
Heights sighted
+1 FRONT-1 MID0 BACK
Naive coder instructed to “mentally trisect the white line in thecenter of the image” and rate
I +1 if red line is further from center than white
I 0 if red line is the same distance as the white from center
I -1 if red line is closer to center than white
11
IntroductionProcedures
Results
DataSightedMeasured
Post-processing: measured
at”@N@
I Edge-detect both vocalic and consnoantal contours viaEdgeTrak (Li et al., 2005)
I Define three radial lines roughly approximating tonguetip/blade (FRONT), body (MID), and back (BACK)
12
IntroductionProcedures
Results
DataSightedMeasured
Post-processing: measured
at”@N@
I Edge-detect both vocalic and consnoantal contours viaEdgeTrak (Li et al., 2005)
I Define three radial lines roughly approximating tonguetip/blade (FRONT), body (MID), and back (BACK)
12
IntroductionProcedures
Results
DataSightedMeasured
Distances measured
-4 FRONT-21 MID21 BACK
I Distance along radial lines from vocalic (black) to consonantal(red) contours measured (pixels)
I Positive polarity if consonantal contour further from centerthan vocalic
13
IntroductionProcedures
Results
Measured vs. sighted resultsMeasured vs. sighted values
Results: measured
Test question
Does production of alveolar compared to dental stops in Kaytetyeinvolve different motion of the tongue?
Yes.
●
●
●
●
●
●
●
alveolar dental
−40
−20
020
40
Back
Place
Dis
tanc
e be
twee
n V
and
C (
px)
●●
alveolar dental
−40
−20
020
40
Middle
Place
Dis
tanc
e be
twee
n V
and
C (
px)
●
alveolar dental
−40
−20
020
40
Front
Place
Dis
tanc
e be
twee
n V
and
C (
px)
n.s. * p < 0.05 * p < 0.05
14
IntroductionProcedures
Results
Measured vs. sighted resultsMeasured vs. sighted values
Results: measured
Test question
Does production of alveolar compared to dental stops in Kaytetyeinvolve different motion of the tongue? Yes.
●
●
●
●
●
●
●
alveolar dental
−40
−20
020
40
Back
Place
Dis
tanc
e be
twee
n V
and
C (
px)
●●
alveolar dental
−40
−20
020
40
Middle
Place
Dis
tanc
e be
twee
n V
and
C (
px)
●
alveolar dental
−40
−20
020
40
Front
Place
Dis
tanc
e be
twee
n V
and
C (
px)
n.s. * p < 0.05 * p < 0.05
14
IntroductionProcedures
Results
Measured vs. sighted resultsMeasured vs. sighted values
Results: sighted
Test question
Does production of alveolar compared to dental stops in Kaytetyeinvolve different motion of the tongue?
Yes, but only in MIDregion.
Back
Place
Hei
ght R
espo
nses
alveolar dental
−1
01
0.0
0.2
0.4
0.6
0.8
1.0
Middle
Place
Hei
ght R
espo
nses
alveolar dental
−1
01
0.0
0.2
0.4
0.6
0.8
1.0
Front
PlaceH
eigh
t Res
pons
es
alveolar dental
01
0.0
0.2
0.4
0.6
0.8
1.0
n.s. * p < 0.05 n.s.
15
IntroductionProcedures
Results
Measured vs. sighted resultsMeasured vs. sighted values
Results: sighted
Test question
Does production of alveolar compared to dental stops in Kaytetyeinvolve different motion of the tongue? Yes, but only in MIDregion.
Back
Place
Hei
ght R
espo
nses
alveolar dental
−1
01
0.0
0.2
0.4
0.6
0.8
1.0
Middle
Place
Hei
ght R
espo
nses
alveolar dental
−1
01
0.0
0.2
0.4
0.6
0.8
1.0
Front
PlaceH
eigh
t Res
pons
es
alveolar dental
01
0.0
0.2
0.4
0.6
0.8
1.0
n.s. * p < 0.05 n.s.
15
IntroductionProcedures
Results
Measured vs. sighted resultsMeasured vs. sighted values
Results vs. results
Measured Means Sighted Means
BA
CK
MID
*
FRONT*
BA
CK
MID
*
FR
ON
T
Alveolar -5.02 -0.25 11.04 -0.11 -0.12 0.99Dental -5.63 -12.79 8.35 0.13 -0.88 0.98
Results comparison
Sighted ratings capable of distinguishing between polarity, but arenot great at expressing variation in magnitude.
16
IntroductionProcedures
Results
Measured vs. sighted resultsMeasured vs. sighted values
Results vs. results
Measured Means Sighted Means
BA
CK
MID
*
FRONT*
BA
CK
MID
*
FR
ON
T
Alveolar -5.02 -0.25 11.04 -0.11 -0.12 0.99Dental -5.63 -12.79 8.35 0.13 -0.88 0.98
Results comparison
Sighted ratings capable of distinguishing between polarity
, but arenot great at expressing variation in magnitude.
16
IntroductionProcedures
Results
Measured vs. sighted resultsMeasured vs. sighted values
Results vs. results
Measured Means Sighted Means
BA
CK
MID
*
FRONT*
BA
CK
MID
*
FR
ON
T
Alveolar -5.02 -0.25 11.04 -0.11 -0.12 0.99Dental -5.63 -12.79 8.35 0.13 -0.88 0.98
Results comparison
Sighted ratings capable of distinguishing between polarity, but arenot great at expressing variation in magnitude.
16
IntroductionProcedures
Results
Measured vs. sighted resultsMeasured vs. sighted values
Values vs. values
●
●
●
●●
●
●
●
●
−1 0 1
−40
−20
020
40
Back
Height Responses
Dis
tanc
e (p
x)
●●
●
−1 0 1−
40−
200
2040
Mid
Height Responses
Dis
tanc
e (p
x)
●
0 1
−40
−20
020
40
Front
Height Responses
Dis
tanc
e (p
x)
−1 < 0 < +1 −1 < 0 < +1 n.s.
Values direct comparison
Sighted rating is a reasonably good predictor for measureddistances.
17
IntroductionProcedures
Results
Measured vs. sighted resultsMeasured vs. sighted values
Values vs. values
●
●
●
●●
●
●
●
●
−1 0 1
−40
−20
020
40
Back
Height Responses
Dis
tanc
e (p
x)
●●
●
−1 0 1−
40−
200
2040
Mid
Height Responses
Dis
tanc
e (p
x)
●
0 1
−40
−20
020
40
Front
Height Responses
Dis
tanc
e (p
x)
−1 < 0 < +1 −1 < 0 < +1 n.s.
Values direct comparison
Sighted rating is a reasonably good predictor for measureddistances.
17
IntroductionProcedures
Results
UsabilityViabilityAutomation
Summary
Suitable for
I First glances
I Small projects
I Visualization
I Teaching / demonstration?
Not suitable for
I Measuring distances automatically
I Comparisons involving small effects
I Projects with large amounts of data
18
IntroductionProcedures
Results
UsabilityViabilityAutomation
Summary
Suitable for
I First glances
I Small projects
I Visualization
I Teaching / demonstration?
Not suitable for
I Measuring distances automatically
I Comparisons involving small effects
I Projects with large amounts of data
18
IntroductionProcedures
Results
UsabilityViabilityAutomation
Typical procedures (revisited)
Time investment
Where can we save time?
1. Collect
2. Post-processI Extract relevant framesI Edge-detection
3. Analyze
4. Interpret
19
IntroductionProcedures
Results
UsabilityViabilityAutomation
Typical procedures (revisited)
Time investment
Where can we save time?
1. Collect
2. Post-processI Extract relevant framesI Edge-detection
3. Analyze
4. Interpret
19
IntroductionProcedures
Results
UsabilityViabilityAutomation
Qualitative vs. quantitative methods
Edge-detection or alternative: ?
I Measured (quantitative) –time-consuming
I Sighted (qualitative) – canbe automated?
Analysis: quantitative
I Sighted (qualitative) –human component
I Measured (quantitative) –can be largely automated
Automation
If qualitative methods are to be competitive, automation mustbe made possible.
20
IntroductionProcedures
Results
UsabilityViabilityAutomation
Qualitative vs. quantitative methods
Edge-detection or alternative: ?
I Measured (quantitative) –time-consuming
I Sighted (qualitative) – canbe automated?
Analysis: quantitative
I Sighted (qualitative) –human component
I Measured (quantitative) –can be largely automated
Automation
If qualitative methods are to be competitive, automation mustbe made possible.
20
IntroductionProcedures
Results
UsabilityViabilityAutomation
Qualitative vs. quantitative methods
Edge-detection or alternative: ?
I Measured (quantitative) –time-consuming
I Sighted (qualitative) – canbe automated?
Analysis: quantitative
I Sighted (qualitative) –human component
I Measured (quantitative) –can be largely automated
Automation
If qualitative methods are to be competitive, automation mustbe made possible.
20
IntroductionProcedures
Results
UsabilityViabilityAutomation
Qualitative vs. quantitative methods
Edge-detection or alternative: ?
I Measured (quantitative) –time-consuming
I Sighted (qualitative) – canbe automated?
Analysis: quantitative
I Sighted (qualitative) –human component
I Measured (quantitative) –can be largely automated
Automation
If qualitative methods are to be competitive, automation mustbe made possible.
20
IntroductionProcedures
Results
UsabilityViabilityAutomation
Automation: first (naive) attempt
1. Increase contrast of both frames
2. Color all pixels above a white threshold (in one frame)
3. Make transparent all pixels below threshold (in same frame)
4. Overlay colored frame over uncolored frame
21
IntroductionProcedures
Results
UsabilityViabilityAutomation
Automation: first (naive) attempt
1. Increase contrast of both frames
2. Color all pixels above a white threshold (in one frame)
3. Make transparent all pixels below threshold (in same frame)
4. Overlay colored frame over uncolored frame
21
IntroductionProcedures
Results
UsabilityViabilityAutomation
Automation: first (naive) attempt
1. Increase contrast of both frames
2. Color all pixels above a white threshold (in one frame)
3. Make transparent all pixels below threshold (in same frame)
4. Overlay colored frame over uncolored frame
21
IntroductionProcedures
Results
UsabilityViabilityAutomation
Automation: first (naive) attempt
Some reasonably successful...
at”@p@
22
IntroductionProcedures
Results
UsabilityViabilityAutomation
Automation: first (naive) attempt
Some not so much
aú@p@: too much colored
23
IntroductionProcedures
Results
UsabilityViabilityAutomation
Automation: first (naive) attempt
Some not so much
at@k@: too little colored
23
IntroductionProcedures
Results
UsabilityViabilityAutomation
Automation: second attempt
1. Increase contrast of both frames
2. Split frame to be colored into blocks
3. For each block, if block contains “interesting information”
3.1 Color all pixels above a white threshold3.2 Make transparent all pixels below threshold
4. Overlay colored frame over uncolored frame
24
IntroductionProcedures
Results
UsabilityViabilityAutomation
Automation: second attempt
1. Increase contrast of both frames
2. Split frame to be colored into blocks
3. For each block, if block contains “interesting information”
3.1 Color all pixels above a white threshold3.2 Make transparent all pixels below threshold
4. Overlay colored frame over uncolored frame
24
IntroductionProcedures
Results
UsabilityViabilityAutomation
Automation: second attempt
1. Increase contrast of both frames
2. Split frame to be colored into blocks
3. For each block, if block contains “interesting information”
3.1 Color all pixels above a white threshold3.2 Make transparent all pixels below threshold
4. Overlay colored frame over uncolored frame
24
IntroductionProcedures
Results
UsabilityViabilityAutomation
Automation: second attempt
1. Increase contrast of both frames
2. Split frame to be colored into blocks
3. For each block, if block contains “interesting information”
3.1 Color all pixels above a white threshold3.2 Make transparent all pixels below threshold
4. Overlay colored frame over uncolored frame
24
IntroductionProcedures
Results
UsabilityViabilityAutomation
Automation: second attempt
Hmmm...
at”@p@
25
References
Li, M., Akgul, Y., and Kambhamettu, C. (2005). Edge Trak [Compuer Program]. Version 1.0.0.4. RetrievedOctober 3, 2008, from vims.cis.udel.edu/EdgeTrak/.