(12) United States Patent (io) Patent No.: US 10,028,697 B2 · 111111111111111111111111111111111111111111111111111111111111111111111111 (12) United States Patent (io) Patent No.:

111111111111111111111111111111111111111111111111111111111111111111111111

(12) United States Patent (io) Patent No.: US 10,028,697 B2Newman et al. (45) Date of Patent: *Jul. 249 2018

(54) SYSTEM AND METHOD FOR MEASURINGSKIN MOVEMENT AND STRAIN ANDRELATED TECHNIQUES

(71) Applicant: Massachusetts Institute ofTechnology, Cambridge, MA (US)

(72) Inventors: Dava J. Newman, Cambridge, MA(US); Ashley M. Mateus, Cambridge,MA (US)

(73) Assignee: Massachusetts Institute ofTechnology, Cambridge, MA (US)

(*) Notice: Subject to any disclaimer, the term of thispatent is extended or adjusted under 35U.S.C. 154(b) by 371 days.

This patent is subject to a terminal dis-claimer.

(21) Appl. No.: 14/837,455

(22) Filed: Aug. 27, 2015

(65) Prior Publication Data

US 2016/0317079 Al Nov. 3, 2016

Related U.S. Application Data

(63) Continuation of application No. 13/274,992, filed onOct. 17, 2011.

(51) Int. Cl.A61B 5/00 (2006.01)A61B 5/11 (2006.01)

(52) U.S. Cl.CPC ............ A61B 5/442 (2013.01); A61B 5/1121

(2013.01); A61B 5/1127 (2013.01); A61B5/7225 (2013.01); A61B 5/7278 (2013.01);

A61B 510064 (2013.01); A61B 510077(2013.01); A61B 516828 (2013.01); A61B

256210261 (2013.01)

00p

7$a- 0Line Of 0

Non-Extension #1

isLine ofmwimurn

Contraction

(58) Field of Classification Search

CPC ...... A61B 5/442; A61B 5/0064; A61B 5/0077

See application file for complete search history.

(56) References Cited

U.S. PATENT DOCUMENTS

4,000,524 A 1/1977 Rinehart4,654,896 A 4/1987 Rinehart4,969,106 A 11/1990 Vogel et al.5,056,530 A 10/1991 Butler et al.

(Continued)

FOREIGN PATENT DOCUMENTS

WO WO 93/22624 11/1993WO WO 2009/100020 A4 8/2009

(Continued)

OTHER PUBLICATIONS

Ross et al., Recovery Force_l: U.S. Appl. No. 61/701,329, filed

Sep. 14, 2012; 18 pages.

(Continued)

Primary Examiner Sean Dougherty

(74) Attorney, Agent, or Firm Daly, Crowley, Mofford& Durkee, LLP

(57) ABSTRACT

Described herein are systems and techniques for a motioncapture system and a three-dimensional (3D) tracking sys-tem used to record body position and/or movements/motionsand using the data to measure skin strain (a strain field) allalong the body while a joint is in motion (dynamic) as wellas in a fixed position (static). The data and technique can beused to quantify strains, calculate 3D contours, and derivepatterns believed to reveal skin's properties during naturalmotions.

20 Claims, 9 Drawing Sheets

14 tine of

maxim-n'Extension

a ~

lei 0R-1&6

9 line & Non-Eztensinn

he'ang t ers that ji f II 12

boiano txafh S 1 I! Reformation Cirdathe circle and elipse L

l

https://ntrs.nasa.gov/search.jsp?R=20180004379 2019-08-13T11:59:55+00:00Z

US 10,028,697 B2Page 2

(56) References Cited

U.S.PATENT DOCUMENTS

5,167,038 A 12/1992 Rinehart5,641,955 A 6/1997 Bonmau et al.5,757,473 A 5/1998 Kanduth et al.6,345,191 B1 2/2002 Hartmann et al.6,389,200 B1 5/2002 Foltzer6,809,462 B2 10/2004 Pelrine et al.7,281,275 B2 10/2007 Bitzer

2003/0013994 Al 1/2003 Rubinstenn et al.2003/0065278 Al 4/2003 Rubinstenn et al.2005/0264561 Al 12/2005 Angst et al.2006/0056661 Al 3/2006 Einighammer et al.2007/0167879 Al 7/2007 Cochran2007/0186642 Al 8/2007 Sano et al.2008/0234607 Al 9/2008 Hunter-Jones et al.2009/0255531 Al 10/2009 Johnson et al.2009/0315989 Al 12/2009 Adelson2010/0000547 Al 1/2010 Johnson et al.2011/0288447 Al 11/2011 Cochran2011/0319791 Al 12/2011 Harry et al.2012/0238914 Al 9/2012 Goldfield et al.2013/0116601 Al 5/2013 Tomazic et al.2014/0081187 Al 3/2014 Wyatt et al.2014/0277739 Al 9/2014 Kornbluh et al.2014/0311187 Al 10/2014 Amarasiriwardena et al.2015/0073318 Al 3/2015 Holschuh et al.2015/0073319 Al 3/2015 Holschuh et al.

FOREIGN PATENT DOCUMENTS

WO WO 2011/109029 Al 9/2011WO WO 2016/077150 Al 5/2016

OTHER PUBLICATIONS

Bethke, Kristen "The second skin approach: skin strain field analy-

sis and mechanical counter pressure prototyping for advanced

spacesuit design"; with Abstract and Chapters 3, 4 and 5, pp. 41-114

(78 pages), 2005; Retrieved from the Internet. <URL:http://dspace.

mit.edu/bitstream/handle/1721.1/32443/61719483-MIT.

pdf?sequence=2>.PCT Search Report and Written Opinion of the ISA, PCT/US2015/053978 dated Jan. 6, 2016; 8 pages.Bethke, et al.; "Bio-Suit Development: Viable Options for Mechani-cal Counter Pressure;" SAE Technical Paper Series; 34 h Interna-tional Conference on Environmental Systems (ICES); Jul. 19-22,2004; 14 pages.Newman, et al.; "Astronaut Bio-Suit System to Enable PlanetaryExploration;" 55'th International Astronautical Congress; 2004; pp.1-22.

Newman, et at; "Revolutionary Design for Astronaut ExplorationBeyond the Bio-Suit System;" vol. 880, Jan. 30, 2007; 12 pages.Wolfram, et al.; "An Automatic Procedure to Map the Skin StrainField with Application to Advanced Locomotion Space SuitDesign;" 5' World Congress of Biomechanics, Munich, Jul.29-Aug. 4, 2006; 2 pages.Iberall; "The Use of Lines of Nonextension to Improve Mobility inFull-Pressure Suits;" Technical Report AMRL-TR-64-118;Wright-Patterson Air Force Base; Nov. 1964; 44 pages.Iberall; "Use of Lines of Nonextension to Provide Mobility inFull-Pressure Suits;" an ASME Publication; American Society ofMechanical Engineers; Nov. 16-20, 1969; 17 pages.Domingues, et al; "Skin Strain Field Analysis of the Human AnkleJoint;" 4rh Congress Nacional De Biomecanica; Feb. 4-5, 2010; 6pages.Marreiros; "Skin Strain Field Analysis of the Human Ankle Joint;"Dissertation; Instituto Superior Tecninco; Faculdade De Medicina;Nov. 2010; 73 pages.Non-Final Office Action dated Aug. 1, 2014 for U.S. Appl. No.13/274,992; 20 pages.Response filed on Dec. 1, 2014 to on Final Office Action dated Aug.1, 2014; for U.S. Appl. No. 13/274,992; 21 pages.Final Office Action dated Jan. 28, 2015, for U.S. Appl. No.13/274,992; 11 pages.Response filed on May 28, 2015 to Final Office Action dated Jan.28, 2015; for U.S. Appl. No. 13/274,992; 7 pages.Supplemental Response filed on Jun. 1, 2015 to Final Office Actiondated Jan. 28, 2015; for U.S. Appl. No. 13/274,992; 12 pages.Notice of Allowance dated Jun. 11, 2015, for U.S. Appl. No.13/274,992; 9 pages.Ambrosino, et al.; "Novel Magnetic Sensor Based on Fiber BraggGrating and Magnetic Shape Memory Alloys;" I" InternationalConference on Sensing Technology; Nov. 21-23, 2005; 6 pages.Holschuh, et al.; "Morphing Compression Garments for SpaceMedicine and Extravehicular Activity Using Active Materials;"Aerospace Medicine and Human Performance; vol. 87; No. 2; Feb.2016; 9 pages.Holschuh, et al.; "Two-Spring Model for Active CompressionTextiles with Integrated NiTi Coil Actuators;" IOP Publishing;Smart Materials and Structures; Feb. 6, 2015; 14 pages.Park, et al.; "Exoskeletal Force-Sensing End-Effectors With Embed-ded Optical Fiber-Bragg-Grating Sensors;" IEEE Transactions onRobotics; vol. 25; No. 6; Dec. 2009; 13 pages.Stirling, et al.; Applicability of Shape Memory Alloy Wire for anActive, Soft Orthotic; Journal of Materials Engineering and Perfor-mance; ASM International; Feb. 8, 2011; 5 pages.Witt, et al.; "Medical Textiles with Embedded Fiber Optic Sensorsfor Monitoring of Respiratory Movement;" IEEE Sensors Journal;vol. 12; No. 1; Jan. 2012; 9 pages.PCT International Preliminary Report on Patentability dated May26, 2017 from International Application No. PCT/US2015/053978;7 Pages.

U.R. Patent Jul. 249 2018 She!! of US 10,028,697 G2

sk

~0

x0

0\

~: \

) ^ 0 0

~ p/~

0 000

0 00 0

/

/ } %0 0

------------------ CC,00

\ 0 0

\: / 0 0 /gmO g .. m ~ l

!. , .

m e e",;~~; ,

enemas» w t / ! ~12_. gm t

z vh Cis @

the cm \ -amdDdwr.a,

\ ) .

U.R. Patent Jul. 249 2018

y / ,

~% ±

:

~+N`~ w's

. /~ ,~

<Z

6 !

\ ~\

Sheet 2of9

~~~ a

IN\ -

\~

; ~ ~

,\~

.

® a

~! z. ~ Z_!

.y~ r ! ~

~». ^~~~

> ,\®a ©~~

US 10,028,697 G2

S

d3

Q

~C

/2C

U.S. Patent Jul. 249 2018 Sheet 3 of 9 US 10,028,697 B2

E~ EG

place rke.,.' is Bxxly .. nd; `r e,,Z

30

2

34.

\~:a`-p' `<1 (; D) `^\.,:\:

BodySkdm, an Each.3

10

\\ .s..\ S! ♦ inns ::: w, .\\ \ ; \`\\ v Axisti\

28)

U.S. Patent Jul. 249 2018

o D, c,.'i

y

nn re, ~ an

Sheet 4 of 9 US 10,028,697 B2

No

44

; Lmes,None

ti.,

>o; ... .•V.:U ,...\ o„m E.. \.•d y &.:,r 3:G cps>_

.........

....

S0

54

..~ \. .a.


U.R. Patent Jul. 249 2018 Sheet of US 10,028,697 G2

N

k v G

~

~~~


« 01 N ,

? \ § p0.e o %

/ \ ;

©

° G ~ 0 -S y v. ©©a;° ƒ \ » 10

( \,© ©00 !4w0CD » °e ®W, v;'>


514

} \ }! .a !

)Test)} y /

! ( A \

81

~


US 10,028,697 B2

SYSTEM AND METHOD FOR MEASURINGSKIN MOVEMENT AND STRAIN AND

RELATED TECHNIQUES

CROSS REFERENCE TO RELATEDAPPLICATIONS

This application is a continuation of, and claims thebenefit under 35 U.S.C. § 120 of the filing date of, co-pending patent application Ser. No. 13/274,992, filed Oct.17, 2011, all of which is incorporated by reference herein inits entireties.

STATEMENT REGARDING FEDERALLYSPONSORED RESEARCH

This invention was made with government support underGrant No. CNS0932015 awarded by the National ScienceFoundation and under Grant Nos. NAS5-03110 and NAS5-98051 awarded by NASA. The government has certainrights in this invention.

FIELD OF THE INVENTION

The concepts described herein relate to a system andtechnique for measuring and modeling skin movement andmore particularly to a system and technique for quantifyingskin movement and deriving strain fields, contours andthree-dimensional patterns.

BACKGROUND OF THE INVENTION

As is known in the art, while there is extensive under-standing of human skin properties based on active tensiletesting, both in vitro and in vivo, there is little currentknowledge of the strains experienced by skin during naturalmovements.

Understanding the skin's material properties and naturalmotion is important to help provide better understanding ina number of areas including: creating tissue engineeringscaffolds that integrate smoothly with little scarring; under-standing skin growth; aging; and changes due to diseases.Past work has been completed to ascertain material propertydata of isolated skin samples with in vitro tensile testing andby applying external loads (pulling, twisting, indenting, andsuctioning) and measuring the resulting deformations invivo. These experimental methods are limited because theskin behaves differently when removed from surroundingtissues in vitro and in vivo techniques do not take intoaccount the skin's biaxial prestress. Because of the aniso-tropic behavior, it has been theorized that the skin containslines of non-extension (LONEs), or contours of the skin thatstay a constant length with minimal stretching capacity thatonly rotate during joint motion. As the body moves, par-ticularly close to the joint, these LONEs do not always exist;in those cases the contours that are most important are thosewith the minimum extension or compression. Previouswork, however, have only been able to qualitatively findnon-extending lines.

SUMMARY OF THE INVENTION

The concepts, systems and techniques described hereinresult in a new understanding of the relationship between thestructure of skin and the strains it experiences during naturaljoint movement. In particular, concepts, systems and tech-niques allow one to quantify strains, calculate three-dimen-

2sional (3D) contours, identify lines of non-extension(LONEs) and derive patterns that are believed to revealskin's properties during natural motions. The system andtechniques described herein may thus find application in a

5 wide variety of areas including, but not limited to, design oftissue-engineering scaffolds, medical diagnosis for skin sur-gery, the design and development of soft exoskeletons, aswell as commercial spacesuits and athletic garments.In accordance with the concepts, systems and techniques

l0 described herein, a motion capture system (including acamera, video acquisition, processing and tracking markers)and a three-dimensional (3D) tracking system (e.g. a laserscanner) are used to record body movements/motions. By

15 recording body movements/motions a new method is pro-vided for measuring the skin's strain all along the body (astrain field) while a joint is in motion (dynamic) as well asin a fixed position (static). Thus, while prior art techniqueswere able to qualitatively find non-extending lines, having

20 dynamic information during the entire motion, allows one toquantify strains, calculate 3D contours, and derive patternsthat are believed to reveal skin's properties during naturalmotions.From motion capture or laser scan position data, a lines of

25 non-extension (LONEs) processor 522 calculates lines ofnon-extension by analyzing the changes in distance betweeneach marker point and its closest neighbors from initialposition to deformation, and calculating strain values. Whena rectangular grid (such as that shown in FIGS. 3A, 3B is

30 used), closest neighbors are considered to be the two mark-ers adjacent in the same row and also the three markers inthe rows above and below in the corresponding columns. Inone embodiment, second order Lagrangian strains s are

35 calculated using the following equation (it should be notedthat the second order equation is used because the strains areso large):

40 /2 - 102 Al (AI)2

s= 212

_ ~ + ,200 0

In which:1 is the new distance between two points

45 1, is the original length between two points, andAl is the difference between the two.The 3D strains surrounding each data point are projected

onto a two-dimensional (2D) plane tangent to the body skin

50 at each marker position. In one exemplary embodiment, this2D plane is created by first averaging the normal vectors tothe planes between each neighboring pair of strain vectors.This new "average" normal of (eight) 8 different planesdefines the normal vector to the tangential plane created at

55 the marker position. It should be appreciated that othertechniques to compute the plane, may also be used. Afterbeing projected onto the tangential plane, the strains are thenrotated in pairs onto the axes defined by the location of themarker directly above and then averaged together to give the

60 longitudinal strain (sy), the circumferential strain (sx), andthe shear strain (s y).

This is followed by eigenvector analysis to determine theprinciple strains (sI and c,,). If the principle strains are ofopposite signs, meaning there is both extension and com-

65 pression, they are used to mathematically determine theangle of the lines of non-extension (~) with the knownequations:

US 10,028,697 B2tj

tan2o= -1(2+-1)

-Sp(2+su)

t.(1- _1)2(_, (2+-,))

~~ (1 +-11)2(1- (1 +-11)2)

where the first equation is the angle between the primary

eigenvector and the line of non-extension projected onto the

initial position and the second equation is the same angle

projected onto the deformed position. If the principle strains

are of the same sign, meaning there is only local extension

or local compression, then the minimum extension or mini-

mum compression is recorded, respectively. These angles

are projected onto the body surface (e.g., leg) and, during

primarily stationary analysis only, they are connected to

continuous lines (e.g. using basic spline functions). This

method of finding tangent planes and creating strain tensors

requires all eight (8) surrounding points. Other techniques

may, of course, also be used.

With this particular methodology, body motion and skin

movement are provided and skin-strain field analysis and

patterning/contours are provided. In one embodiment, the

motion capture system measures skin movement and strain

around the knee during a squatting motion. It should, of

course, be appreciated that the system and techniques

described herein above are not limited to use with anyparticular body part (e.g. knee regions) or any particularmotion (e.g. squatting), rather the system and techniquesdescribed herein may be used in any body region and for anybody movement including body joint movements fordynamic motions (e.g., squatting, bending, walking, etc.).

In accordance with a further aspect of the conceptsdescribed herein, a new technique to analyze skin movementand strain for a static body as well as a body in motion (e.g.around the knee during a squatting motion) includes calcu-lations of lines of non-extension, minimum and maximumskin movement, compression and tension.

In accordance with a further aspect of the concepts,systems and techniques described herein, a method formanufacturing a garment (e.g. an athletic garment, a space-suit, etc.... ) comprises using a material formed into a shapeby at least considering lines of non-extension of the bodypart over which the garment will be disposed wherein thelines of non-extension of the body part are determined, atleast in part, from position data of body skin of the body partwherein the position data is generated from a plurality ofposition data points obtained at one or more locationsdefined by one or more corresponding markers disposed onthe body skin wherein the position data is computed using aninitial position and a deformed position and computing linesof non-extension from the position data measure at least atthe initial position and a deformed position.

Thus, in accordance with a further aspect of the concepts,systems and techniques described herein, a method fordesign of tissue-engineering scaffolds comprises formingtissue-engineered scaffolds into a shape by at least consid-ering lines of non-extension of the body part over which thetissue-engineered scaffolds will be disposed wherein thelines of non-extension of the body part are determined, atleast in part, from position data of body skin of the body partwherein the position data is generated from a plurality ofposition data points obtained at one or more locationsdefined by one or more corresponding markers disposed onthe body skin wherein the position data is computed using aninitial position and a deformed position and computing lines

4of non-extension from the position data measure at least at

the initial position and a deformed position.

Thus, in accordance with a further aspect of the concepts,

systems and techniques described herein, a method for soft

5 exoskeletons comprises forming a soft exoskeleton into ashape by at least considering lines of non-extension of thebody part over which the tissue-engineered scaffolds will bedisposed wherein the lines of non-extension of the body partare determined, at least in part, from position data of body

10 skin of the body part wherein the position data is generatedfrom a plurality of position data points obtained at one ormore locations defined by one or more corresponding mark-ers disposed on the body skin wherein the position data iscomputed using an initial position and a deformed position

15 and computing lines of non-extension from the position datameasure at least at the initial position and a deformedposition.

BRIEF DESCRIPTION OF THE DRAWINGS20

The foregoing features of the concepts, systems, circuitsand techniques described herein may be more fully under-stood from the following description of the drawings inwhich:

25 FIG. 1A is a graphical representation of lines of non-extension;FIGS. 113-11) illustrate three (3) cases of a deformation

ellipse. The circle outlined in completed (i.e. solid) linesrepresents to initial position and the circle with dotted lines

30 represents the deformed position. Case 1 (FIG. 113): LONESexist. Case 2 (FIG. 1C) omplete extension: the importantdirection is the line of minimum extension. Case 3 (FIG.1D) complete compression: the important direction is theline of minimum compression;

35 FIGS. 2 and 2B are a series of flow diagrams that illustratea process for recording body positions and movements;motions for measuring skin strain along a body (a strainfield) while a join is in motion (dynamic) as well as in a fixedposition (static);

40 FIGS. 3A and 3B illustrate exemplary placement of onehundred forty four (144) four 4 millimeter (mm) and 6 mminfrared reflecting, spherical markers on a right leg. FIG. 3Ais the anterior of the leg and FIG. 3B is the posterior of theright leg;

45 FIGS. 4A and 4B are a graphical explanation of howstrain pairs are rotated onto the new axes defined by theposition of the marker directly above;FIG. 5A illustrates a motion capture reconstruction of

reflective markers on a bent leg;50 FIG. 5B is a map of longitudinal strains that occur as the

leg moves from extended to flexed;FIG. 5C shows a pattern of lines of non-extension calcu-

lated from the strain field seen in FIG. 413;FIG. 6 is a block diagram of a system for measuring and

55 modeling skin movement and for quantifying skin move-ment and deriving strain fields, contours and three-dimen-sional patterns; andFIG. 7 is a block diagram of a processor of the type which

may be used in the system of FIG. 6.60

DETAILED DESCRIPTION OF THEPREFERRED EMBODIMENTS

Referring now to FIG. 1A, a graphical representation of65 lines of non-extension includes a reference circle 10, a

deformation circle 12, a line of maximum extension 14, aline of maximum contraction 16, and two lines of non-

US 10,028,697 B25

extension portions 18a, 18b of which illustrate diametersthat belong to both the circle and the ellipse.

Referring now to FIGS. 113-11), three (3) cases of adeformation ellipse are shown. The circle outlined in com-pleted (i.e. solid) lines 20 represents an initial position andthe circle with dashed lines 22 represents a deformedposition. As visible in FIG. 113, (i.e. Case 1) lines ofnon-extension (ZONES) exist.

Referring now to FIG. 1C, as with FIG. 113, the circleoutlined in completed (i.e. solid) lines 24 represents aninitial position and the circle with dashed lines 26 representsa deformed position. As visible in FIG. 1C (i.e. Case2—complete extension) the important direction is the line ofminimum extension.

Referring now to FIG. 1D, as with FIGS. 1B and 1C, thecircle outlined in completed (i.e. sold) lines 28 represents aninitial position and the circle with dashed lines 29 representsa deformed position. As visible in FIG. 1D (Case 3—com-plete compression) the important direction is the line ofminimum compression.

Referring now to FIGS. 2A and 213, a process for mea-suring the skin's strain all along the body (a strain field)while a joint is in motion (dynamic) as well as in a fixedposition (static) begins by placing markers on a body (or aportion of a body) under test to mark the points at which datawill be obtained as shown in block 30. One particularexample of marker placement on a knee is described in detailbelow in conjunction with FIGS. 3A, 3B. It should, ofcourse, be appreciated that the particular placement ofmarkers depends upon a variety of factors including, but notlimited to, the portion of the body being measured (e.g. knee,elbow, etc.... ), and the limitations of the tracking system.As shown in block 32, once the markers are placed such

that acceptable data can be obtained, a measurement systemobtains data at each marker position. In one exemplaryembodiment, the measurement system includes a motioncapture system (including, for example, a camera, videoacquisition, processing and tracking markers) and a three-dimensional (31)) tracking system (e.g. a laser scanner)which record body positions and movements/motions (i.e.measurement are made while the body (or body portione.g. joint) is in motion (dynamic) as well as in a fixedposition (static). One exemplary system is described belowin conjunction with FIG. 6.Once the data is collected, as shown in block 34, 3D strain

values are computed for each point. In one embodiment,second order Lagrangian strains s are calculated using thefollowing equation:

[1 — 101 0/ (AW£ =

2~ _ [o + 2~

;

In which:1 is the new distance between two pointsto is the original length between two points, andAl is the difference between the two.Processing then proceeds to processing block 36 in which

a two-dimensional (21)) plane tangent to the body skin ateach marker position is computed. In one embodiment, thetwo-dimensional (21)) plane is created by first averaging thenormal vectors to the planes between each neighboring pairof strain vectors. This new "average" normal of eight (8)different planes defines the normal vector to the tangentialplane created at the marker position. It should be appreciatedthat other techniques may also be used to compute this

6plane. However, it is believed that this technique yields themost accurate results if a measurement system of the typedescribed below in conjunction with FIG. 6 is used.Once the two-dimensional (21)) plane tangent to the body

s skin at each marker position is computed, the 3D strains areprojected onto the 2D tangent planes as shown in block 38.

Next, as shown in blocks 40 and 42, longitudinal strainfry), the circumferential strain (sx), and the shear strain (s y)are computed (see FIGS. 4A, 413). In one embodiment, this

to is accomplished in the following manner. After the 3Dstrains are projected onto the tangential plane (processingblock 38), the strains are then rotated in pairs onto the axesdefined by the location of the marker directly above (asillustrated in FIGS. 4A and 4B below) and then averaged

15 together to give the longitudinal strain fry), the circumfer-ential strain (sx), and the shear strain (s y).

Also, as shown in processing block 44, the principlestrains (sr and srr) are determined. In one embodiment, thisis accomplished via an eigenvector analysis to determine the

20 principle strains (sr and srr).As shown in decision block 46, if the principle strains are

of opposite signs, meaning there is both extension andcompression, they are used to mathematically determine theangle of the lines of non-extension (~) as shown in block 48.

25 In one embodiment this can be accomplished with theequations:

s~(2+s1)

30t.

~~ -

20 (1 +-11)2(1 - 0 +ell )

35 where the first equation is the angle between the primaryeigenvector and the line of non-extension projected onto theinitial position and the second equation is the same angleprojected onto the deformed position.

Processing then flows to block 50, in which these angles4o are projected onto the body surface (e.g., leg).

Processing then flows to decision block 52. If decisionblock 52 indicates that stationary analysis should be per-formed (i.e. only a single frame from the imaging system isbeing processed) then processing proceeds to block 54 and,

45 during stationary analysis, the angles are connected tocontinuous lines (i.e. contours) using basic spline functions.It should be noted that techniques other than spline functionscan also be used to connect to the continuous lines (includ-ing but not limited to two straight lines, etc). To date,

5o however, it has been found that a spline is the smoothest andsimplest technique. It should also be noted that the above-described method of finding tangent planes and creatingstrain tensors requires all eight (8) surrounding points.Processing then ends.

55 If, on the other hand, a decision is made in decision block46 that the principle strains are of the same sign, meaningthere is only local extension or local compression, then theminimum extension or minimum compression is recorded,respectively, as shown in block 49. Processing then flows to

6o decision block 52 and to appropriate ones of blocks 54, 56as described above. If decision block 52 indicates thatmotion analysis should be performed (i.e. the imagingsystem produces multiple frames to capture motion of abody part), then processing flows to decision block 56 which

65 implements a loop in which the process from blocks 34 to 56is repeated until all frames are processed. Once all framesare processed, then processing ends.

US 10,028,697 B27

Referring now to FIGS. 3A and 313, a body part has aplurality of exemplary reflecting, spherical markers, gener-ally denoted 60, disposed thereon. In this exemplaryembodiment, the body part is a right leg having one hundredand forty-four (144) markers 50 disposed thereon. It shouldbe noted that FIG. 3A shows the anterior of the leg and FIG.3B shows the posterior of the right leg.

Also, in this exemplary embodiment, the markers areprovided as four millimeter (4 mm) and six (6) mm infraredreflecting, spherical markers. It should, of course, be appre-ciated that other types, sizes and shapes of markers may beused, In short, any marker which allows the collection ofdata suitable for use in the processing described herein maybe used.

Referring now to FIGS. 4A and 413, a graphical explana-tion is shown of how strain pairs are rotated onto the newaxes defined by the position of the marker directly above.Eight points 68a-68h are used. It should be noted that thepoint corresponding to 6270 is not used because its similarityto 6y causes large error. The strain values 6 y and 6x areaveraged for the final results.

FIG. 5A illustrates a motion capture reconstruction ofreflective markers on a bent leg. The left-most column 401indicates the anterior of the leg.

FIG. 5B is a map of longitudinal strains that occur as theleg moves from an extended position to a flexed position.

FIG. 5C shows a pattern of lines of non-extension calcu-lated from the strain field seen in FIG. 5B. Two lines 410,412 consistent with the pattern on the posterior of the kneeare highlighted.

Referring now to FIG. 6 a system 500 for measuring andmodeling skin movement and for quantifying skin move-ment and deriving strain fields, contours and three-dimen-sional patterns includes a data collection system 510 whichcollects data from an appropriately marked body under test512. System 500 is appropriate, for example, to perform themeasurement and data processing techniques describedherein.

Significantly, data collection system collects from thebody under test while the body is at rest (i.e. static datacollection) and while the body is undergoing a motion (i.e.dynamic data collection). It should be appreciated that bodyunder test 512 is here shown in phantom since it is notproperly a part of the system 500 for measuring and mod-eling skin movement and for quantifying skin movementand deriving strain fields, contours and three-dimensionalpatterns.

Data collection system 510 provides the data to a pro-cessing system 514 which processes the data to model andquantify skin movement and to derive strain fields, contoursand three-dimensional patterns in accordance with the tech-niques described hereinabove.

In the exemplary embodiment of FIG. 5, data collectionsystem 510 includes a motion capture system 516 and atracking system 518 which may be provided for example asa three-dimensional laser tracker system. In one embodi-ment, motion capture system 516 is provided as an eightcamera Vicon (Centennial, Colo.) motion capture system.

Motion capture system 510 collects or otherwise obtainsposition data at the locations of the marker points as dis-cussed above in conjunction with FIGS. 3A, 3B. Datacollection system 510 provides the data to processing system514 and in particular to a data store 520 of processing system514. Processing system 514 further includes a lines ofnon-extension (LONEs) processor 522, a principle strainprocessor 524 and a three-dimensional pattern processor 526

8all of which are coupled to receive (either directly orindirectly) data from the data store.LONEs processor 522 analyzes changes in distance

between position data at each marker point and its closest5 neighbors from initial position to deformation. When a

rectangular grid (such as that shown in FIGS. 3A, 3B isused), closest neighbors are considered to be the two mark-ers adjacent in the same row and also the three markers inthe rows above and below in the corresponding columns. In

10 one embodiment, because the strains are so large, LONEsprocessor 522 calculates second order Lagrangian strainsusing the following equation:

15 h — to Al (AI)2

s= 20 _ + 2i;0 0 0

In which:20 1 is the new distance between two points;

to is the original length between two points; andAl is the difference between the two;LONEs processor 522 then projects the 3D strains sur-

rounding each data point onto a two-dimensional (2D) plane25 tangent to the body skin at each marker position. The 2D

plane is created by first averaging the normal vectors to theplanes between each neighboring pair of strain vectorswherein the new average normal of eight (8) different planesdefines the normal vector to the tangential plane created at

30 the marker position. After being projected onto the tangen-tial plane the strains are then rotated in pairs onto the axesdefined by the location of the marker directly above (asdescribed above in conjunction with FIGS. 4A and 413) andthen averaged together to give the longitudinal strain (6y),

35 the circumferential strain (6x), and the shear strain (6,Y).Principle strain processor 524 performs an eigenvector

analysis to determine the principle strains (6r and 6rr). Inresponse to the principle strains being of opposite signs,meaning there is both extension and compression, they are

40 used to mathematically determine the angle of the lines ofnon-extension ~ with the known equations:

"(2+--,)

45

t.

~~

20 (1 +s„)~(1 - (1 +s„)~)

50 where the first equation is the angle between the primaryeigenvector and the line of non-extension projected onto theinitial position and the second equation is the same angleprojected onto the deformed position.In response to the principle strains having the same sign,

55 meaning there is only local extension or local compression,then processor 524 records the minimum extension or mini-mum compression and projects these angles onto the bodysurface and, during primarily stationary analysis only, con-nects them to continuous lines. This may be accomplished,

60 for example, using basic spline functions. This exemplarymethod of finding tangent planes and creating strain tensorsrequires all (eight) 8 surrounding points. It should of coursebe appreciated that in some embodiments less than eight (8)points can be used. In the exemplary embodiment described

65 herein, the eight (8) points (e.g. as illustrated in FIGS. 4A,413) are in close proximity and increasing the number of(close) points increases the accuracy of the measurement.

US 10,028,697 B2I

In one embodiment, with respect to stationary data col-lection, the motion capture system was able to accuratelycapture a grid of markers spaced approximately 3 cm apart(e.g. as shown in FIG. 5A). Preliminary analysis identifiedmaximum longitudinal strains less than 0.40, within thephysical limitations of skin, in an appropriate distribution(FIG. 513). The process for calculating lines of non-exten-sion from the strain field data (e.g. as described in conjunc-tion with FIGS. 2A, 2B above) found a pattern of lines ofnon-extension as shown in FIG. 5C.The entire range of strain (maximum and minimum of the

total strain), longitudinal strain, circumferential strain, andsheer strain for each data point are found.A three-dimensional pattern processor 526 processes the

data provided thereto to connect the calculated angles fromLONES processor 522 from each data point into anatomi-cally feasible contours. In this way, the system is capable ofgenerating designs for flexible apparel. Such apparelincludes but is not limited to athletic or other clothing,portions of space suits and in particular. The system is alsouseful for generating designs of tissue-engineering scaffolds,medical diagnosis for skin surgery and the design anddevelopment of soft exoskeletons.

In one exemplary embodiment the above described tech-nique was utilized on a knee. In this exemplary embodiment,an eight camera Vicon (Centennial, Colo.) motion capturesystem was used to track one hundred forty four (144) 4-mmand 6-mm spherical reflective markers positioned approxi-mately 3 cm apart around a healthy adult female knee jointwith care taken to make a grid with the markers aligned inrows and columns (see FIGS. 3A, 313). In the exemplaryembodiment shown in FIGS. 3A, 3B the markers are dis-posed in a substantially rectangular grid pattern. Otherpatterns (both grid and non-grid patterns e.g. lattice pat-terns) may, of course, also be used. In preferred embodi-ments, the markers are disposed in a uniform grid pattern.Such a grid pattern facilitates use of nearest neighborcomputations. It should also be appreciated that the distancebetween markers impacts the accuracy of anatomical repre-sentations and the ability to make physiological implica-tions. Thus, the closer the spacing of the markers, the moreaccurate the anatomical representation and the greater theability to make physiological implications based upon theanatomical representation. Of course, by spacing markersmore closely in a given area (e.g. 1 cm spacing or less), thelarger the number of data points which will be generated andconsequently the more time it takes to perform computa-tions. Thus, a trade-off may be made between the amount ofdata needed and the time required to measure and processthe data. In one embodiment, the resolution of the motioncapture system is the limiting factor in selecting spacingbetween markers.

With the markers are properly positioned, stationary (i.e.static) data collection is performed. Position data is capturedby the motion capture system with the body joint (e.g., knee)extended (initial position) and then again with the body jointflexed (final position).

The acquired position data is filtered and averaged sothere is one position for each marker in each position, whichthen serves as input to the analysis/calculations methodol-ogy described above.

Also, with the markers properly positioned, motion (i.e.dynamic) data collection can be performed. It should benoted that the same body marker grid should be used in boththe stationary (i.e. static) data collection and the motion(dynamic) data collection. Motion capture position data arecollected for an entire flexion and extension cycle (-6

10seconds) for the entire movement for every/any body joint.In the dynamic case (e.g. 2-6 seconds of movement capturedat a rate of about 15-30 frames per second), an analysisprogram compares each frame to the initial position (first

5 frame). The resultant output is strain data for each point thatcan be combined and analyzed over time or for the durationof the movement. Significantly, the system also can be usedto produce video files to display the strain and directions ofthe non-extending lines graphically on a 3D reconstruction

io of the body (e.g., leg).The technique described herein allows new data to be

generated and with this new data, one can calculate the linesof non-extension, or contours of the skin that remain aconstant length during motion as described herein above.

15 The system and techniques described herein result in anew understanding of the relationship between the structureof skin and the strains it experiences during natural jointmovement. The system and techniques described herein mayfind application in a wide variety of areas including, but not

20 limited to, design of tissue-engineering scaffolds, medicaldiagnosis for skin surgery, the design and development ofsoft exoskeletons, commercial spacesuits and athletic gar-ments.

It should also be appreciated that the above-described25 techniques can be used to calculate the angle of the line of

non-extension, line of minimum extension, or line of mini-mum compression for each frame.

Referring now to FIG. 7, a computer 752 suitable forsupporting the operation of an embodiment of the inventive

30 systems, concepts, and techniques described herein includesa processor 754. Processor 754 may, for example, be pro-vided as a dual-core processor, such as one of the typesavailable from the Advanced Micro Devices Corporation offrom Intel Corporation. However, it should be understood

35 that computer 752 may use other microprocessors. Com-puter 752 can represent any server, personal computer,laptop, or even a battery-powered mobile device such as ahand-held personal computer, personal digital assistant, orsmart phone.

40 Computer 752 includes a system memory 756 which isconnected to the processor 754 by a system data/address bus762. System memory 756 includes a read-only memory(ROM) 758 and random access memory (RAM) 760. TheROM 758 represents any device that is primarily read-only

45 including electrically erasable programmable read-onlymemory (EEPROM), flash memory, etc. RAM 760 repre-sents any random access memory such as SynchronousDynamic Random Access Memory (SDRAM). The BasicInput/Output System (BIOS) 196 for the computer 752 is

50 stored in ROM 758 and loaded into RAM 760 upon booting.Within the computer 752, input/output (I/O) bus 764 is

connected to the data/address bus 762 via a bus controller766. In one embodiment, the I/O bus 764 is implemented asa Peripheral Component Interconnect (PCI) bus. The bus

55 controller 766 examines all signals from the processor 754to route signals to the appropriate bus. Signals betweenprocessor 754 and the system memory 756 are passedthrough the bus controller 766. However, signals from theprocessor 754 intended for devices other than system

60 memory 756 are routed to the I/O bus 764.Various devices are connected to the I/O bus 764 includ-

ing internal hard drive 768 and removable storage drive 770such as a CD-ROM drive used to read a compact disk 771or a floppy drive used to read a floppy disk. The internal hard

65 drive 768 is used to store data, such as in files 774 anddatabase 776. Database 776 may include, for example, astructured collection of data, such as a relational database. A

US 10,028,697 B211

display 772, such as a cathode ray tube (CRT), liquid-crystaldisplay (LCD), etc. is connected to the I/O bus 764 via avideo adapter 778.A user enters commands and information into the com-

puter 752 by using input devices 780, such as a keyboard and 5

a mouse, which are connected to I/O bus 764 via I/O ports781. Other types of pointing devices that may be usedinclude track balls, joy sticks, and tracking devices suitablefor positioning a cursor on a display screen of the display772. 10

Computer 752 may include a network interface 786 toconnect to a remote computer 782, an intranet, or theInternet via network 784. The network 784 may be a localarea network or any other suitable communications network.Data from external systems (e.g. a data collection system 15such as that described above in conjunction with FIG. 6)may be provided tom computer 752 through various com-puter I/O ports and/or network connections.

Computer-readable modules and applications 788 andother data are typically stored on memory storage devices, 20which may include the internal hard drive 768 or thecompact disk 771, and are copied to the RAM 760 from thememory storage devices. In one embodiment, computer-readable modules and applications 788 are stored in ROM758 and copied to RAM 760 for execution, or are directly 25executed from ROM 758. In still another embodiment, thecomputer-readable modules and applications 788 are storedon external storage devices, for example, a hard drive of anexternal server computer, and delivered electronically fromthe external storage devices via network 784. 30

The computer-readable modules 788 may include com-piled (or compilable) instructions for implementing embodi-ments of one or more of. a LONEs processor, a principlestrains processor or a skin orientation processor and methodsdescribed herein. Skin movement and/or strain data may be 35rendered and outputted to display 772 to enable users tographically view the data (e.g. as shown in FIGS. 3A-5C).

In a further embodiment, the computer 752 may executevarious processes on separate processors, such as a firstprocessor and a second processor of a dual core processor. 40As by way of a non-limiting example, control of datacollections operations (e.g. to receive and respond to datafrom a data collection system) may be executed by the firstprocessor and skin and strain operations (e.g., to computeLONEs and strain values) may be executed by the second 45processor. Alternatively, the first and second processors maybe respective first and second computing devices.The computer 752 may execute a database application

790, such as OracleTM database from Oracle Corporation, tomodel, organize, and query data stored in database 776. The 50data may be used by the computer-readable modules andapplications 788 and/or passed over the network 784 to theremote computer 782 and other systems.

In general, the operating system 792 executes computer-readable modules and applications 788 and carries out 55instructions issued by the user. For example, when the userwants to execute a computer-readable module 788, theoperating system 792 interprets the instruction and causesthe processor 754 to load the computer-readable module 788into RAM 760 from memory storage devices. Once the 60computer-readable module 788 is loaded into RAM 760, theprocessor 754 can use the computer-readable module 788 tocarry out various instructions. The processor 754 may alsoload portions of computer-readable modules and applica-tions 788 into RAM 760 as needed. The operating system 65792 uses device drivers 794 to interface with variousdevices, including memory storage devices, such as hard

12drive 768 and removable storage drive 770, network inter-face 786, I/O ports 781, video adapter 778, and printers.

It should be appreciated that the processes describedherein (e.g. in conjunction with FIGS. 2A and 213, forexample) are not limited to use with the hardware andsoftware of FIG. 7. Rather, they may find applicability in anycomputing or processing environment and with any type ofmachine or set of machines that is capable of running acomputer program.

It should also be appreciated that the processes describedherein (e.g. in conjunction with FIGS. 2A and 213, forexample) may be implemented in hardware, software, or acombination of the two. The processes described herein maybe implemented in computer programs executed on pro-grammable computers/machines that each includes a pro-cessor, a storage medium or other article of manufacture thatis readable by the processor (including volatile and non-volatile memory and/or storage elements), at least one inputdevice, and one or more output devices. Program code maybe applied to data entered using an input device to performone or more of the processes and/or to generate outputinformation.The system may be implemented, at least in part, via a

computer program product, (e.g., in a machine-readablestorage device), for execution by, or to control the operationof, data processing apparatus (e.g., a programmable proces-sor, a computer, or multiple computers)). Each such programmay be implemented in a high level procedural or object-oriented programming language to communicate with acomputer system. However, the programs may be imple-mented in assembly or machine language. The language maybe a compiled or an interpreted language and it may bedeployed in any form, including as a stand-alone program oras a module, component, subroutine, or other unit suitablefor use in a computing environment. A computer programmay be deployed to be executed on one computer or onmultiple computers at one site or distributed across multiplesites and interconnected by a communication network. Acomputer program may be stored on a storage medium ordevice (e.g., CD-ROM, hard disk, or magnetic diskette) thatis readable by a general or special purpose programmablecomputer for configuring and operating the computer whenthe storage medium or device is read by the computer toperform processes. The processes described herein may alsobe implemented as a machine-readable storage medium,configured with a computer program, where upon execution,instructions in the computer program cause the computer tooperate in accordance with processes.The processes described herein are not limited to the

specific embodiments described. For example, the processdescribed in FIGS. 2A and 2B are not limited to the specificprocessing order shown in FIGS. 2A and 213, respectively.Rather, unless otherwise precluded, any of the processingblocks of FIGS. 2A and 2B may be re-ordered, combined orremoved, performed in parallel or in serial, as necessary, toachieve the results set forth above.The processing described herein (e.g. in conjunction with

FIGS. 2A, 213) associated with implementing the systemand/or the techniques described herein may be performed byone or more programmable processors executing one ormore computer programs to perform the functions of thesystem. All or part of the system may be implemented as,special purpose logic circuitry (e.g., an FPGA (field pro-grammable gate array) and/or an ASIC (application-specificintegrated circuit)) and/or neural networks.

Also, it should be appreciated that elements of differentembodiments described herein may be combined to form

US 10,028,697 B213

other embodiments not specifically set forth above. Otherembodiments not specifically described herein are alsowithin the scope of the following claims.

Having described preferred embodiments of the concepts,systems, circuits and techniques described herein, it willnow become apparent to those of ordinary skill in the art thatother embodiments incorporating these concepts may beused. For example, it should now be appreciated that one canapply the topologies described herein to rectifier systems(e.g. for grid-connected power supplies) as well and forbidirectional power flow converter systems. Accordingly, itis submitted that that the concepts, systems, circuits andtechniques described herein, should not be limited to thedescribed embodiments but rather should be limited only bythe spirit and scope of the appended claims.The invention claimed is:1. A method for computing lines of non-extension

(LONEs) in a skin movement and strain measurementsystem, the method comprising:(a) obtaining, by a motion capture system of the skin

movement and strain measurement system, positiondata by:obtaining three-dimensional (3D) positions of a plural-

ity of marker points disposed on body skin surround-ing a joint of a test subject with the joint of the testsubject in an initial position;

moving the joint of the test subject to a deformedposition; and

obtaining 3D positions of the marker points with thejoint of the test subject in the deformed position,wherein the plurality of marker points form a grid onthe body skin of the test subject;

by a LONEs processor of the skin movement and strainmeasurement system:

(b) analyzing changes in distance between each markerpoint and one or more adjacent marker points from theinitial position to the deformed position;

(c) determining one or more 3D strains surrounding eachmarker point;

(d) projecting 3D strains surrounding each marker pointonto a two-dimensional (2D) plane tangent to the bodyskin at the corresponding marker point;

(e) after projecting 3D strains onto the 2D tangential planefor each marker point, rotating the projected strains inpairs onto axes defined with respect to the location ofthe corresponding marker point;

(f) averaging the rotated pairs for each axis associatedwith each marker point to generate a longitudinal strain(sy), a circumferential strain (s,), and a shear strain

(r');(g) determining principle strains (aI and c,,) for each

marker point;(h) if the principle strains associated with a first marker

point have opposite signs, determining an angle of aline of non-extension for the first marker point usingthe principle strains associated with the first markerpoint; and

(i) if the principle strains associated with the first markerpoint have the same sign, recording an angle of mini-mum extension or minimum compression for the firstmarker point and projecting the angle of minimumextension or minimum compression onto the bodysurface.

2. The method of claim 1 further comprising:repeating (h) and (i) for other marker points in the

plurality of marker points; andconnecting angles of lines of non-extension into contours.

143. The method of claim 1 wherein:obtaining 3D positions of the marker points comprises

obtaining 3D positions of the marker points with thejoint of the test subject in positions other than the initial

5 position and the deformed position as part of a motionanalysis; and

repeating (b) through (i) for successive frames of themotion analysis.

4. The method of claim 1 wherein determining one orl0 more 3D strains surrounding each marker point comprises

calculating second order Lagrangian strains using the fol-lowing equation:

15 h —101 Ol (A/)zs= 212

_ T + ;0 0 0

in which:20 s is the second order Lagrangian strain for two points;

1 is a new distance between the two points;to is an original distance between the two points; andAl is a difference between the new distance and the

original distance.25 5. The method of claim 1 wherein determining an angle of

a line of non-extension for the first marker point using theprinciple strains associated with the first marker pointincludes using the equations:

30

s,(2+s,)

t.20 (1 +s„ s)2(1 - 0 +„ )T)

35

wherein the equations correspond to the angle between aprimary eigenvector and the line of non-extension projectedonto the initial position and the deformed position, respec-

40 tively, wherein sI and £zz are the principle strains.6. The method of claim 1 further comprising:generating the 2D tangential plane for each marker point,

before projecting the 3D strains, by averaging normalvectors to the planes between each neighboring pair of

45 strain vectors associated with the corresponding markerpoint, wherein a neighboring pair of strain vectorscomprises strain vectors associated with two closestneighbor marker points that are adjacent to one another.

7. The method of claim 6 wherein averaging normal50 vectors includes averaging normal vectors associated with

eight (8) different planes.8. The method of claim 1 wherein determining the prin-

ciple strains (sI and c,,) comprises performing an eigenvec-tor analysis to determine the principle strains (sI and £zz)•

55 9. A method to measure skin movement and strain in askin movement and strain measurement system, the methodcomprising:(a) obtaining position data associated with body skin of a

test subject using a motion capture system of the skin60 movement and strain measurement system, the position

data corresponding to a plurality of marker pointsdisposed on the body skin around a joint of the testsubject, wherein obtaining position data includesobtaining position data for multiple different orienta-

65 tions of the joint of the test subject;(b) computing, by a processor of the skin movement and

strain measurement system, angles of lines of non-

US 10,028,697 B215

extension (LONEs) for various marker points on thebody skin of the test subject using the collected positiondata by:determining one or more three-dimensional (3D)

strains surrounding each marker point;computing a two-dimensional (2D) plane tangent to thebody skin at each marker point at which position dataof body skin was taken in (a);

projecting the 3D strains surrounding each markerpoint onto the corresponding 2D plane tangent to thebody skin;

computing values for longitudinal strain fry), circum-ferential strain (sx), and shear strain (s y) for eachmarker point;

determining principle strains (sr and srr) for eachmarker point;

if the principle strains associated with a first markerpoint have opposite signs, using the principle strainsto mathematically determine an angle of a line ofnon-extension for the first marker point; and

(c) connecting the angles of lines of non-extension intocontours for use in performing at least one of: flexibleapparel design, tissue-engineering scaffold design, softexoskeleton design, and skin surgery.

10. The method of claim 9 wherein computing angles oflines of non-extension further comprises:

repeating (b) for other marker points within the pluralityof marker points.

11. The method of claim 10 further comprising if theprinciple strains associated with the first marker point havea common sign, recording an angle of minimum compres-sion or minimum extension for the first marker point.

12. The method of claim 10 further comprising projectingstrains onto the surface for analysis.

13. The method of claim 9, wherein determining one ormore 3D strains surrounding each marker point comprisescalculating second order Lagrangian strains using the fol-lowing equation:

12-101 Ol (OW£ = 212 = [o + 2~ ;

in which:s is the second order Lagrangian strain for two points;1 is a new distance between the two points;to is an original distance between the two points; andAl is a difference between the new distance and the

original distance.14. A method to measure skin movement and strain in a

skin movement and strain measurement system, the methodcomprising:(a) obtaining position data of body skin using a motion

capture system of the skin movement and strain mea-surement system, wherein obtaining position data com-prises obtaining three dimensional (3D) positions of aplurality of marker points disposed on the body skinsurrounding a joint of a test subject with the joint of thetest subject in an initial position, moving the joint of thetest subject to a deformed position, and obtaining 3Dpositions of the marker points with the joint of the testsubject in the deformed position, wherein the pluralityof marker points form a grid on the body skin of the testsubject;

(b) analyzing, by a processor of the skin movement andstrain measurement system, changes in distance

16between each marker point and one or more adjacentmarker points from the initial position to the deformedposition;

(c) determining one or more 3D strains surrounding each5 marker point;

(d) generating a two-dimensional (2D) plane tangent tothe body skin at each marker point by averaging normalvectors to the planes between each neighboring pair ofstrain vectors associated with the corresponding marker

l0 point, wherein a neighboring pair of strain vectorscomprises strain vectors associated with two closestneighbor marker points that are adjacent to one another;

(e) projecting the 3D strains surrounding each marker15 point onto the corresponding 2D plane;

(f) after projecting the 3D strains onto the 2D tangentialplane for each marker point, rotating the projectedstrains in pairs onto axes defined with respect to thecorresponding marker point;

20 (g) averaging the rotated pairs for each axis associatedwith each marker point to generate a longitudinal strainvalue (sy), a circumferential strain value (sx), and ashear strain value (s y) for the marker point;

(h) determining principle strain values (sr and srr) for each25 marker point using eigenvector analysis;

(i) if the principle strain values associated with a markerpoint have opposite signs, using the principle strainvalues to determine an angle of a line of non-extensionfor the marker point; and

30 0) if the principle strain values associated with a markerpoint have a common sign, recording an angle ofminimum extension or an angle of minimum compres-sion for the marker point.

35 15. The method of claim 14 wherein obtaining positiondata comprises obtaining 3D positions of the marker pointswith the joint of the test subject in positions other than theinitial position and the deformed position.

16. The method of claim 14 wherein determining one or40 more 3D strains surrounding each marker point comprises

computing one or more second order Lagrangian strains.17. The method of claim 14 wherein generating a 2D

plane tangent to the body skin at each marker point includesaveraging eight normal vectors for each marker point.

45 18. The method of claim 14 wherein using the principlestrains associated with a marker point to determine the angleof a line of non-extension for the marker point comprisesusing the principle strains to mathematically determine theangle of the line of non-extension using the equations:

50

-,(2+-,)~~ _ --„(2+-11)

2 (1—£1)2(£1(2+-1))55 ran

— (1 +-11)2(1 — (1 +-11)2)

wherein the equations correspond to the angle between aprimary eigenvector and the line of non-extension projected

60 onto the initial position and the deformed position, respec-tively, wherein sI and £zz are the principle strains.

19. The method of claim 14 further comprising projectingthe angle of minimum extension or the angle of minimumcompression for the marker point onto the body skin.

65 20. The method of claim 16, wherein calculating secondorder Lagrangian strains comprises using the followingequation:

17

[1- go Ol (AW2120 Io 2120

US 10,028,697 B2

in which:s is the second order Lagrangian strain for two points;1 is a new distance between the two points;to is an original distance between the two points; andAl is a difference between the new distance and the 10

original distance.

18

Documents

(12) United States Patent (io) Patent No.: US 10,028,697 B2 · 111111111111111111111111111111111111111111111111111111111111111111111111 (12) United States Patent (io) Patent No.: