FRUSTRATION REDUCTION - JMP · SAS INSTITUTE INC. SPRING 1997 ISSUE 4 FRUSTRATION REDUCTION John Sall Senior Vice President SAS Institute Inc. As work continues on Version 4 of JMP,

SAS INSTITUTE INC.SPRING 1997

ISSUE 4

FRUSTRATION REDUCTIONJohn Sall

Senior Vice PresidentSAS Institute Inc.

As work continues on Version 4 ofJMP, we find that some of the greatesteffort goes not into new features, butinto reducing whatever frustrationusers have with the current features.We have shifted gears a little in howwe see our obligation as softwaredevelopers. This is especially the caseas JMP matures and has fulfilled itsfirst charter. JMP will grow more, butwe see our effort best spent to makeour product do its current job better,not take on too many new jobs.

IÕm an afficionado (gourmet reader) ofgood junk mail. I think that copywriting is a difficult and high art.Three years ago, I received one of thegreatest pieces of junk mail ever, asample newsletter called the SamaraiSword from Corporate Visions¨. Eventhough I didnÕt buy their product, thenewsletter did change my thinkingfrom that point on. The newsletter wasabout sales techniques, and the bigmessage was ÒFind the pain.Ó Thereare people out there that donÕt haveyour product and are suffering becauseof it. You think about a situation in

which having your product makes allthe difference, and then all you haveto do is tell that story.

For JMP, one story is discoveryÑthatwithout the graphics and interactivity,you are painfully missing thosediscoveries that would make all thedifference.

JMPer CableSAS INSTITUTE INC.

SPRING 1997, ISSUE 4

-2-

Recently we started reviewing otherstatistical packages. We needed to findout what kinds of pain the users ofthese other packages are suffering.This will help us identify ways topresent our product. For example,most products donÕt integrate statisticsand graphics into the same window,and we need to show that itÕs reallyfrustrating to keep track of whatgraph goes with what text reportwhen they appear in different places.

But of course, in looking at otherproducts we also find out more aboutour own product, by contrast. So weask ourselves, what frustrations areour users feeling, or are susceptible tofeeling? We want to find problemsbefore our users do, and before ourcompetitors do. If we find them, wecan fix things for a future release.

I wonÕt tell you about all the things weare finding because we all have somehypochondriac tendencies, and wouldsuffer more if we were made moreaware of our problems. Or, you canconsider it your job to tell us what

is bothering you. We welcome yourinput.

But I might mention a few items, justto prove that we are feeling the painsignals. One problem concerns theMacintosh, one problem concernsWindows, and one problem isportable. All three are good examplesof frustrations that we see every dayand have learned to cope with, but areimproving for Version 4.

On the Macintosh, if you paste apicture from JMP into your wordprocessor, then shrink the picture andprint it on a laser printer, some pointsin plots are distorted and show asrectangles instead of squares; smoothcurves donÕt always have smoothjoints. This is a problem that JMP hashad from Day 1. Why didnÕt we fix it?Well the problem is that the MacOSªQuickdraw does its shrinking withinteger coordinates that are 72 dots-per-inch. Whether the square stays asquare, or turns into a tall or widerectangle depends on if a pointÕscoordinates have similar remainderswhen scaled.

We could have cured that by going toPostScript¨-customized graphics butthere were big drawbacks. If you hada plot with thousands of points thepicture would be huge. We do havesome space-efficient custom PostScriptfor a number of things on theMacintosh including dashed lines,



-3-

rotated text, and in certain casessmoothing joined segments.

We also experimented with forms ofcopying in which we collect thedrawing into a bitmap that is four oreight times the original size, and thenshrink it into the picture. When itdraws on the laser printer it comes outgreat. The problem is that each picturemight use several megabytes. We alsoconsidered using the new graphicsinterface, called Quickdraw GX, but itis something that not all users install.

On Windows, there are analogousproblems. For example, when we dorotated text on the side of a plot, itcomes out looking funny when pastedinto a document. That is because it isdone as a rotated bitmap at 300 dotsper inch. It looks fine on a laserprinter, but on a 72 dot per inchscreen, too many pixels get fuzzed inwhen it displays the high resolutionbitmap at the lower resolution.

We tried it both ways underWindows: you have to suffer in oneplace or the other and we made thecall that the printed outputwas what really counted. In JMPversion 4 we will use a differentimaging model that will resolve bothof these issues more directly. We willmake the effort, and most of the effortwe make might not show up inanything you notice immediatelyexcept in the absence of frustration.

A general frustration concernsscrolling. JMP, unlike most statisticalproducts, does a lot of interactionthough hypertext-like popups in thereport itself. We have two placeswhere we put popupsÑon the lowerleft beside the horizontal scroll bar(stationary icons), and within thereport. Popups within a report movewith scrolling but stationary iconsdonÕt. Each one has a big problem.

¥ The problem with scrollingcontrols is that they go off screenand out of sight. To click on themyou have to scroll back to findthem. Sometimes when the graph isin a small window and popup iconsare below the graph you donÕteven notice them unless youhappen to scroll down.A few releases ago, we noticedthat if you asked for some optionsfrom a popup, for example to testnormality in Distribution, youdidnÕt see any change to thescreen because the new report wasbelow the bottom of the window.Now when you request additionalreports the window automaticallyscrolls down to where the reportwas added. But if you wantanother report you have to scrollback up to where you were inorder to ask for it.

¥ The stationary icons have a differ-ent problem. They work fine when



-4-

the report is all about one analysis,but they donÕt work well whenthere are several kinds of reportsin the same window. For example,Fit Y by X produces four differentkinds of output, so the stationarypopup menu canÕt have anything init because the commands wouldnÕtbe right for all four types ofanalysis.

We are committed to solving allthese problems in Version 4.

I believe that the software industryhas much work to do in fixing featuresthat it already has. As users, featuresthat it already has. As users, we getused to things that are badlydesigned. So it is a challenge to bothidentify the problems, as well as findnew designs to solve them.

In April over 40 million householdssaw JMP Statistical Discovery Soft-ware featured on the nationallysyndicated television program,ÒTechnology Today.Ó JMP wasselected for the program because ofits wide acceptance by engineers as atool to gain competitive advantage.The program was developed andcreated by Global Solutions Net-work. If you missed the broadcastyou can visit the Technology Todayweb site <http://www.gsnetwork.com/>

where the feature has been adaptedfor the internet.

JMP SERVICES UPDATE

Professional Services DivisionSAS Institute Inc.

SAS InstituteÕs Professional ServicesDivision is pleased to offer thefollowing JMP Training courses,which will be held publicly in SASTraining centers across the US during1997.

¥ Interactive Data Analysis Using JMPSoftware

¥ Categorical Data Analysis Using JMPSoftware

¥ Design and Analysis of ExperimentsUsing JMP Software

¥ Statistical Data Exploration Using JMPSoftware

¥ Advanced Design and Analysis ofExperiments Using JMP Software

¥ Multivariate Statistical Methods UsingJMP Software

¥ ANOVA and Regression MethodsUsing JMP Software

¥ Statistical Quality Control Using JMPSoftware

For information about specificlocations and dates, call

919-677-8000 ext 7205.

If you need personalized instruction,Institute staff can come to yourlocation to provide onsite training.You can customize training bychoosing segments to meet yourspecific needs. All instructor-basedcourses include computer workshops.To schedule JMP onsite training,technical service, or consulting, callJMP Training at

919-677-8000 ext 7312.



-5-

THERE AND BACK AGAIN:

A Data TableÕs Tale (or, Moving YourData From One Computer To Another)

Part I: Floppy Disks

by Michael HechtSAS Institute Inc.

I like my Mac. I prefer to use it for allmy work, including the work I do onJMP. However, sometimes I have touse a data table that was created withthe Microsoft Windows version ofJMP or send a data table from myMac to a coworker who prefersWindows.

JMP Data TablesWe designed JMP to use the same fileformat for data tables on both theMacintosh and Windows, so transferbetween them is quitestraightforward. However, there are afew ÔgotchasÕ involved and they allconcern how you convince the othersystem that the file you're moving isreally a JMP data table.

The mechanism for this is differenton each system. Windows uses themethod of the file extensionÑathree-letter code joined to the endof the file name with a period. Thefile extension defines the contentsof the file and, in most cases, whatapplication should be used to openit. For JMP data tables we chose thethree-letter code ÒJMPÓ because itseemed appropriate. So a JMP datatable might be namedÒBIGCLASS.JMPÓ on Windows.

With Windows 95, the file name is notrestricted to a maximum of uppercasecharacters, so you are free to namethe file ÒBig Class.jmpÓ. Note that theletter case of the file extension is notimportant; but, the file extension muststill be there. Windows 95 hasadditional options that cause it toshield the file extension from youonce itÕs been registered with thesystem.

Instead of file extensions the Mac usestwo attributes called file type and filecreator.¥ The file type is a special four-letter

code that is not a part of the filename. In fact, the file type is afeature of the file that you cannotchange. The file type onlyidentifies the contents of a file; notwhat

¥ This second attribute is defined bythe file creatorÑ another four-lettercode, which is also inaccessible toyou. For JMP, the file creator isÒSGP Ó. Note that I said S-G-P-space. The trailing space issignificant.

The history of the JMP data table filetype code is long and colored, andfilled with many wild and conflictinglegends. Some say that the developerresponsible decided that these filesShould Give Predictable Data, andthus chose the fabled letters. Otherssay that the developer was told bySAS Security to ÒStop Getting PizzaDeliveredÓ while working late into



-6-

the night, and chose this code as areminder. But I personally believe thatthe code is an acronym for thematerials used to construct the veryfirst data table, Stellar Globules ofPinkish Dust. Whatever the origin, ona Mac a file with a type code of SGPDrepresents a JMP data table.

Moving Tables Via SneakerNet:PC to MacThe easiest way to get the data fromhere to there is to put it on a floppy.How-ever, the thing to rememberhere is that Macs are more accommo-dating than PCs, so you should usethe PC floppy disk format. All PCsuse this format, so if youÕre startingon the PC you just copy the file on thedisk and walk it over to the Mac. Theonly problem is that the file type willnot be set properly. There are twoways around this problem. If youÕreonly confronted with the problemonce a year or less, you may opt forthe first solution. If you do this on adaily basis, IÕm sure youÕll prefer thesecond solution.

1) Solution number one is to launchJMP, then choose the Opencommand, navigate to the floppyor the folder containing the file,and turn on the Show all filescheck box in the Open dialog.Presto! Your data table appears inthe list of files ready to be opened.

2) Solution number two is to open thePC Exchange Macintosh controlpanel (see Figure A next page)

which lets you map PC fileextensions to Macintosh file types.To do this:

¥ Click Add and in the resultingdialog type Ò.JMPÓ into the DOSSuffix field.

¥ Navigate to your JMP applicationand select it (donÕt double-click it).The JMP application icon appearsunder Application Program and theDocument Type popup becomesenabled.

¥ Click on it and choose ÒSGPDÓ,which should have the icon of aJMP data table. Click OK andyouÕre done! From now on, any fileon a PC floppy that ends withÒ.JMPÓ automatically gets the JMPdata table icon. You can double-click it to launch JMP, just like youcan with a data table you createdwith JMP on your Mac.

Mac to PCIf youÕre going the other way (fromMac to PC), youÕll have to format thedisk so that the PC thinks itÕssomething other than pinkish spacedust. To do this, use a high densityfloppy, which has the stylized ÒHDÓlogo and the extra hole. Unlock it,insert it into the Mac, select it, andchoose the FinderÕs Erase Diskcommand. On the dialog that appearschoose DOS 1.4 MB from the Formatpopup. You might need to give thedisk a new name to suit the rigidityof the PC file system. Click Erase andgo get a tall glass of fruit juice. When



-7-

embossed on its icon. Just drag your data table onto it and

you back, the disk should have a cute little ÒPCÓ

youÕre set. Sneaker the diskover to your PC and insertit. From JMP, choose Open(or click that snazzy toolbarbutton) and navigate to thefloppy. ItÕs likely that youÕllneed to choose All files (*.*)from the Files of typepopup. Even then, youmight not recognize our fileby name. If your Mac filename wonÕt fit neatly intothe DOS 8.3 convention, theMac runs it through a meatgrinder to make it fit. Forexample, after copying thedata table BIRTH-DEATHSUBSET to a PC floppy theMac helpfully renamed it to!BIRTH-D.EAT. Lovely.Why it canÕt use Ò.JMPÓ foran extension, like it knowsit should from the PCExchange control panel, Iknow not. Oh well, itseems even the Mac hasroom for improvement.You can see this namemangling on the Mac, too.

Figure A PC Exchange Control Panel

Each assignmnet below determines whichMacintosh application program is used whenyou open DOS documents with a particular suffix.

PC Exchange

DOS Suffix Application Program Document type

.TXT Simple Text TEXT

.JMP JMP SGPD

Add...

OptionsOn Off

.

Assign an application to a DOS suffix:DOS Suffix Application Program Document type

.JMP .JMP

AppleScript folder

.JMP

JMP

Sample Data Desktop

OK

Relaxed

Just select a file on a PCfloppy and Get Info for it.Then OPTION-click on the

file name in the Get Info window to see what thefile name will look like when you plug the diskinto a PC.



-8-

The example in Figure B shows whatthe Mac will do to modify the datatable name Birth-Death Subset Égofigure.

Figure B Click in Table Info

The bottom line here is that if youÕremoving files back and forth, youshould put Ò.JMPÓ on the end of thefile name.

THERE AND BACK AGAINPart II: Sharing Data Tables on aReal Network


When your data sharing needsoutgrow SneakerNet, you will wantto use something a bit moresubstantial for transferring files, like afile server. Here at SAS Institute weuse a Novell NetWare server, whichruns on a PC. This server, bless itssoul, comes with an extension calledNetWare for Macintosh that makes itspeak the AppleShare protocol. Thatmeans our Macs can connect to it

through the Chooser as though itwere a real AppleShare File Server.You may have a similar setup at yoursite. Unfortunately, the DOS fileextension to Macintosh file typemappings you set up in PC Exchangefor floppy disks (see There and Back:Part I in this issue) have no bearing onfiles placed on servers. Therefore, youare forced to follow these two rules:1) Always put the Ò.JMPÓ file

extension on JMP data tables thatreside on a server.

2) On the Mac, you must use theShow all files check box to accessJMP data tables on the server.

Tables Across The InternetIf youÕre connected to the Internetyou might want to use it to send yourJMP data out into the world. In therealm of the Internet, files are notidentified by file extensions or evenby file type codes. Instead, yetanother standard called MIME isused. MIME stands for theÔMultipurpose Internet MailExtensionsÕÑReally, IÕm not kidding!ItÕs a standard for tagging data whichoriginated as a way to send e-mailthat contained styled text, graphics,sound, movies, or other enclosures.The MIME standard has since beenconscripted for use with FTP and theWorld Wide Web.

MIME can define many data types,each of which is assigned a descriptorof the form type/subtype. For example,plain old text uses the descriptor



-9-

text/plain. Text with embedded markupcodes for styles and formatting usesthe descriptor text/enriched. In general,the main type text means this data issomething a human can read. Someother main types are:¥ image for graphics¥ audio for noises¥ video for moving pictures¥ multipart for multiple

representations of the same data ordata that consists of aconglomeration of different types

¥ application, for everything else.The most generic MIME type isapplication/octet-stream, which means thedata is a bunch of bytes for which nointerpretation is known or imagined.When received, data of this type istypically just saved to disk. Also, ifthe MIME type is not recognized, it isusually treated as

application/octet-stream.

If you donÕt tell your computer whatMIME type to use when transmittinga JMP data table across the Internet, itis forced to assume thatapplication/octet-stream must be used.The other end then gets untyped datacontaining the bytes you sent. If theother end is a PC, thatÕs coolÑso longas the file itÕs dumped into has aÒ.JMPÓ extension. If the other end is aMac, thatÕs cool tooÑbut be preparedto use the Show all files check boxwhen opening the data table.

A more elegant method is to tag thedata as a JMP data table. To do this,

you simply invent a mutually agreedupon MIME type specifically for thatpurpose. In keeping with the spiritand the specifications of MIME, youshould use

application/x-jmp-data

You use the main type applicationbecause a JMP data table is not text,image, audio, video, or multipartdata. If the receiving applicationprogram doesnÕt recognize thesubtype, the main type application tellsit to treat the data as application/octet-stream, which is still reasonable. Thesub-type for JMP starts with an x- ,which means an unregistered, private,or experimental data type. We putjmp- in there because the MIMEspecification recommends that yourdata tag include the name of theintended application program.

We could leave it at that, but just incase we ever want to send some otherJMP file type (should one ever exist)we put data on the end to identify thisas a data table.

Now that weÕve agreed on a MIMEtype, how do we tell our computerabout it? On the Mac, you use thishandy system extension namedInternet Config. Internet Configmanages the preference settings thatare commonly used by your variousInternet tools, such as your name ande-mail address. In particular, it keepsup with the mappings of MIME typesto Macintosh file type/creator codes,described in Part I.



-10-

HereÕs how to define a mapping fromapplication/x-jmp-data to actual JMP datatables. Launch Internet Config andproceed as shown in Figure A.

On Windows, thereÕs no such beast asInternet Config. Each Internet tool youuse must be separately configured.HereÕs how to configure Eudora, apopular eÐmail client. Other toolswork similarly. EudoraÕs MIMEmappings are found in its dot-inny file,which has nothing whatsoever to dowith belly buttonsÑit is a text filenamed EUDORA.INI.

If youÕre running Eudora, quit it first;then open EUDORA.INI in a texteditor. This file has a section thatbegins with the line

[Mappings]

and is followed by a countably infinitenumber of one-line entries that lookvaguely like this:

both=rtf,MSWD,TEXT,application,rtf

Save the .INI file and launch Eudora.When you send someone a JMP datatable Eudora consults its MIMEmappings and sees that files with theÒ.JMPÓ extension are to be tagged asÒapplication/x-jmp-dataÓ. If youÕresending it to my Mac, my e-mail client(which happens to be AppleÕsCyberdog) recognizes this MIME tagas belonging to files with thetype/creator pair of ÒSGPD/SGP Ó.Almost like magic, the enclosureappears to me as a JMP data table.This tells Eudora that both incomingand outgoing data, say an enclosure

in an e-mail message, with a MIMEtype of application/rtf should be giventhe Ò.RTFÓ extension. The ÒMSWDÓand ÒTEXTÓ items happen to beMacintosh file creator and type codes!Why? Because Eudora was portedfrom the Macintosh to Windows andit retains its Mac interoperability,even on a PC. So, the thing to do isadd a line to this file that looks likethis:both=jmp,SGP ,SGPD,application, x-jmp-data

ConclusionIn a perfect world, the issuesdiscussed here would not even exist.It shouldnÕt matter what I name myfile, and I should never need to knowabout such things as type/creatorcodes. Unfortunately, life is filledwith niggling details such as these.However, with a bit of configurationwork we can come close to perfection.Apple could help by picking a cross-platform file type to file extensionconvention and sticking with it in allsituationsÑremovable media, fileservers, and networked applications.The Internet is showing us the waywith its MIME standard, and InternetConfig is a reasonable means ofintegrating MIME support into thesystem. Now if only PC Exchange andfile servers would make use of it!

Until we achieve cybernirvana, let meleave you with the following rules:

1) JMP data tables with legs shouldbe named using the Ò.JMPÓ extension.



-11-

2) Networking tools should use theMIME type Òapplication/x-jmp-dataÓ for JMP data tables.

For more information see the UserÕsreference material for:

Cyberdog <http://cyberdog.apple.com/>

Eudora <http://www.eudora.com/>

Internet Config <http://www. quin.echidna.id.au/Quinn/Config/>

MIME < http://sunsite.auc.dk. RFC/rfc/rfc2045.html>

PC Exchange <http://www.apple.com. au/Pub/Datasheets/PCEx2.html>

Share and enjoy!

Figure A Use the Internet Config Extension to Define a MIME Type

JMP Data Table

.jmp

application/x-jmp-data

Name:

Extension:

MIME Type:

Mapping Entry

Extension Application

AddSimpleText - Text Readme.1st

Sort:

Plain TextBinary DataMacintosh

Not for Incoming

Not for Outgoing

SGPD

SGP

Choose Example...

JMP

<none>Post Process:

Defaults

File Type:

File Creator:

Cancel OK

• Launch the Internet Config application and click the File Mappings icon.

• Click Add on the File Mapping dialog to see the Mapping Entry dialog.

• Fill it in as shown here.

Don't forget the space.



-12-

CALCULATING FITTED VALUESFROM A Y BY X SPLINE FIT

By Annette SandersSAS Institute Inc.

To fit a spline relationship to twonumeric variables you first use the FitY by X command from the Analyzemenu. This example uses the BIGCLASS data table from the SAMPLEdata, with height as Y and weight as X.The Fit Y by X platform begins byshowing a scatterplot of the X, Y datapoints. The Fitting popup menubeneath the plot accesses the fittingoptions shown in Figure A.

The Fit Spline option fits a smoothingspline using a smoothing parameteryou specify. The spline is displayedon the X, Y scatterplot (weight andheight in this example) and a tableappears showing the R-Square andSum of Squares Error.

The Save Predicteds popup menucommand for the spline fit creates anew data table column and savespredicted values for each row.However, the spline does not have aprediction equation so you cannotfind fitted values (predicted values) fordata points that are not in the datatable.

In order to compute fitted values youneed more information about thespline fit; specifically, the coefficientsof the splineÕs prediction formula. Toget these you first use the Output CoefTable popup menu command on the

Spline Fit table, which creates a newJMP table and saves a set of splinecoefficients for each unique value ofthe X variable. Figure B shows thecoefficients table for the height byweight example.

Note that although the BIGCLASS datatable has 40 observations the weight (X)variable has only 29 unique values (29rows) listed in the O column. Thesevalues are called knot points. The knotpoints are points at which third degreepolynomials are spliced together. Thepolynomial values and their firstderivatives agree at these points,which results in a continuous andsmooth curve.

You can use this new table and theJMP calculator to build a formula thatwill compute fitted values for any Xvalue. The way you do this is togenerate the X values you want, andcreate an index that associates each Xwith the appropriate row of splinecoefficients for that X value.

The fitted values are for any numberyou choose of equally spaced X valuesthat fall within the range for whichthere are spline coefficients (the minand max of the weight variable in thisexample).

Follow these steps to generate thefitted values.

1) In the spline coefficients table, addthe number of rows so the totalrows is the number of fitted



-13-

Figure A Fitting a Spline Curve

The Fit Spline option fits a smoothing spline using a smoothing parameter you specify. The spline is displayed on the X, Y scatterplot .

The Fitting popup menu beneath the plot accesses the fitting options.

Nonpar Density

Grouping Variable…

Show PointsFit MeanFit LineFit PolynomialFit SplineDensity EllipsesPaired t-test

1000000-stiff100000100001000100101.1-flexible.01Other…

Line of FitConfid Curves: FitConfid Curves: IndivColor

Save PredictedsSave ResidualsPlot ResidualsRemove Fit

heig

ht

50

55

60

65

70

60 80 100 120 140 160 180weight

Fitting Smoothing Spline Fit, lambda=10

R-SquareSum of Squares Error

0.737448184.2851

Smoothing Spline Fit, lambda=10Output Coef Table

height By weight

Figure B Table of Spline Coefficients

O A B C D The O column contains uniquevalues of the weight variable. Thecolumns A , B , C, and D are thespline coefficients associated witheach unique weight value. Theyare the constant, linear, quadratic,and cubic coefficients respectively.



-14-

points you want to obtain. In thisexample we added 71 rows (to the29) in order to obtain 100 fittedpoints. You can create as many ofthese new X values as you want atany time by increasing the numberof rows in the data table.

2) Use Tables®New Column to create anew column of X values (call itnewX). In the New Column dialogselect Formula as the Data Source.

3) Use the calculator Count functionwith Quantile functions as argumentsto generate equally spaced valuesfrom the minimum to maximum ofthe knot points:

count (from quantile0Oto quantile1Oin n steps, 1 time)

Recall that Quantile0 of O is its mini-mum, Quantile1 is its maximum, andn is the total number of table rows.Note that the newX values are inthe coefficients data table but theirordering and physical relation tothe coefficients are not relevant;you are storing unrelated pieces ofinformation within the same rowof the data table. It is likely thatthe coefficients needed to computethe fitted value for a specific newXvalue will not come from its ownrow. The row of coefficientsneeded is identified by an indexvariable you need to create, asdescribed next.

4) Create an index column (call itindex) with calculated values

index) with calculated valuesbetween 1 and 29 that point to eachunique weight value. This indexassociates each newX value with theappropriate row of coefficients:

¥ Use the Sum function toincrement the value of index by 1each time the value of newX isgreater than or equal to the nextconsecutive knot point:

n

(Oj £ newXi and Oj ¹ •)j = 1å

¥ When the value of newX is lessthan the next consecutive knotpoint nothing more is addedÑthat index value identifies therow of coefficients needed tocalculate the fitted value.

5) Now you can use the JMPcalculator to compute fitted heightvalues from newX using the appro-priate coefficients. The formula tocompute the fitted value from newXwith the index value i is:

fitted valuei = A i +d•Bi ++d2•C i + d3•Di

where d is the difference betweennewX and the observed weightassociated with it by its index value.An efficient way to construct thisformula is with a temporaryvariable and an Assignment function.You construct the formula like this:¥ Create a new column called

fittedHt.• Use the calculator and select New

Variable from the Variables



-15-

functions list. Create a tempo-rary numeric variable (call it diff).

• Select Assignments from theConditions functions. Use the firstclause in the assignment tosubtract the knot point (O)subscripted by the computedindex (index) from the newX valueand assign this difference to diff:

diff Ü –newX Oindexresults

This is done because thecoefficients are defined for theincrement from the lower end of

the known point rangecontaining the newX value.

¥ The last step is to enter thecomputation for the fitted valueas the clause for results givingthe formula and results shownin Figure C.

As a visual verification that the fittedvalues follow the Fit Y By X spline fityou can use Graph®Overlay Plots withfittedHT as Y and newX as X. Figure Dshows that the computed curve isalmost the same as the fitted spline.

Figure C Computations to Produce Fitted Spline Values

diff Ü newX – Oindex

results +Aindex •diff Bindex ++ •diff 2 Cindex •3diff Dindex

Compute the fitted value for height

using the coefficients in the row identified by the index variable.

O A B C D newX index fittedHT

Figure D Comparison of Spline Fit (left) and Fitted Values (right)

50

55

60

65

70

60 80 100 120 140 160 180

weight .

heig

ht

fitte

dHT

50

55

60

65

70

60 80 100 120 140 160 180newX



-16-

THE RUNS TEST: NonparametricTesting for Randomness in a Seriesof Runs

by Annie DudleySAS Institute Inc.

When monitoring process control on amanufacturing line, one problem thatoccurs is a cyclic run of defectiveunits. By testing for randomness youcan identify periodic runs that mightnot be visible in control charts.

Nonrandomness can occur either withtoo many or too few runs, where arun is a sequence of like events. Thetotal number of runs in a sample givesan indication of whether the sample israndom. If there are few runs, a timetrend or grouping of like events dueto lack of independence could beoccurring. Many runs might indicatesome systematic short-period cyclicalfluctuation.

As an example, suppose an engineer ismonitoring the unit failures off aprocess line and found the sequenceof successes (s) and failures (f) shownin Figure A.

To set up a runs test, you firstidentify the runs. The success andfailure events are recorded in a JMPtable (event). You identify the numberof runs by tagging the firstoccurrence of each run in the series;that is, tag the row whenever thetype of event changes. To do this:

¥ Create a new column called r andchange the data source to formula.

¥ Enter the formula as shown inFigure A to assign a 1 to thebeginning of each new run.

The total number of runs in thesample is simply the sum of the rcolumn.

Figure A Event Sequence

In a data table, record sequence of events in the order they occurred.

s f s f s s s f f s f s f s f s s s s f s f s f s s f f f s f s f s f s s f s s f s s s s f s f s s

Tag the beginning of each run.

{ 1 ,otherwise,if (eventi ¹ event1–1)



-17-

As a rule of thumb, when either the number of successes or the number offailures is greater than 20, an approximation of the z test can be constructed totest whether there are too many (or too few) runs in a series. This example has20 failures and 30 successes.

Let n1 be the number of successes, n2 be the number of failures, n be totalsample size, and r be the number of runs (the sum of the r column). The largesample approximation treats the distribution of r as normal with

(2•n1•n2)(n1+n2)

mr = and sr =(2•n1•n2•(2•n1•n2–n1–n2))

(n1+n2)2•(n1+n2–1)

These parameters give the z statistic:

z = r – mr sr

This z statistic formula is bulky but not complicated and can beconstructed with the JMP calculator as shown in Figure B.

First, create a new column (call it z test) and use the calculator to construct theformula as follows:1) Use Variables in the function browser to create three new temporary

variables, called n1, n2, and r.2) Use Assignments from the Conditions functions to assign values to n1, n2, and r.3) Enter the equation for the z-approximation as the results clause.4) The last step is to find the 2-tailed probability associated with the z test

value. Create another column (call it p-value) and use the normDist functionfound in the probabilities functions.

Figure B Construction of the z Statistic Formula

results

r

n1 Ü “s”(eventj = )nåj=1

n2 Ü n–n1

Ü rjnåj=1

ÜÜÜ

results

assign values to n1, n2, and r

create assignment clauses

(n1+n2)

(n1+n2)2•(n1+n2–1)

r –(2•n1•n2)( )+ 1

(2•n1•n2•(2•n1•n2–n1–n2))results

r

n1Ünåj=1

n2Ü n–n1

Ü rjnåj=1

compute z statistic as result

“s”(eventj = )



-18-

Note that the normDist function returns theprobability that a value is less than or equalto its argument, so you use 1 Ð normDist,and multiply by 2 to find the desired 2-tailed probability 2•(1–normDist(|z test|)).The z test and probability values show asconstant columns in the data table as inFigure C).

You can delete all the rows andsave this kind of table as atemplate. Then whenever youwant a large-sample runs test,paste the sequence of events inthe event column and thecalculations will proceedautomatically.

Figure C Runs Test and Probability This example is from Seigel(1956), with the sequence beingthe order of gender in a queueat a theater.

References:Siegal (1956), Nonparametric Statistics,McGraw-Hill Book Company, NY

Mendenhall, Scheaffer, and Wackerly (1986),Mathematical Statistics with Applications, 3rdEd., Duxbury Press, CA

Calculator Corner


Cumulative SumHere is an example that uses a lagvariable. Suppose for each row youwant to compute the cumulativetotal of a column called expenses. To

LAGGING BEHINDThe SAS DATA step has a nicefeature called LAG, which gives youthe value a variable had on aprevious observation. To do this inJMP you use the Subscript operatoron a column. If the subscriptÕs valuecomes before the current row, youÕveachieved a lag. JMPÕs Subscriptoperator can also subscript rows thatcome after the current row. This iscommonly called a LEAD.

do this you create a new column(call it cumulative exp) and give itvalues with the formula:

expenses + cumulative expi–1

where i is the special Terms operatorthat is the current row number and1 £ i £ n.

WhatÕs happening here? Well, foreach row, you want the value ofcumulative expenses to be thecurrent rowÕs expenses plus the



-19-

previous rowÕs cumulative expenses.However, thereÕs a problemconditionÑfor the first row i = 1 andthe subscript for cumulative expbecomes 0. The solution is to guardagainst the border case by using an ifclause, as shown in Figure A.

Cumulative Sum by GroupsA more challenging problem isfinding a cumulative sum withingroups, as well as guarding againstmissing values anywhere in theexpenses column. For example,suppose you have a groupingvariable, group, and you want thecumulative values as shown in thedata table in Figure B.

Figure A Accumulate a Totalcumulative exp i–1, if i> 1 { 0 , otherwise

expenses+

expenses cumulative expenses

An efficient formula to do this uses atemporary variable with anassignment condition as follows:

1) From the function browser selectVariables®New Variable. Name thevariable e.

2) From the function browser selectConditions®Assignment to set up

an assignment, which looks likethis:

Üresults

3) Use Conditions®If to give thetemporary variable e the value ofzero whenever expenses ismissing, and the value of expensesitself when it is not missing:

results

e { otherwise0, if expenses = •expenses ,

Ü

4) For the assignmentÕs result clause,use another If clause that assigns efor the 1st row, or when thegroup changes. Otherwise assignthe cumulative,e + the lagged grouped cum,as shown in Figure B.

results e, if i=1 or group ¹ groupi–1

e+grouped cumi-1, otherwise{e { otherwise

0, if expenses = •expenses ,

Ü

Figure B Grouped Totals

expenses group grouped cum



-20-

SAVING DATA TABLE DISKSPACE

ItÕs easy to calculate how much diskspace (bytes) a data table will require.

1) First, add together¥ 8 bytes for each numeric variable¥ field width + 1 for each character

variable¥ 2 for each row state variable.

2) Multiply the total by the number ofobservations in the table.

3) Add 102 bytes overhead for eachcolumn.

4) To be exact, add in 1 byte for eachcharacter used in the Column Infonotes and in the Table Info notes.

Under Windows, the number of bytesa table uses shows next to the tablename when you select Details from theView menu. To see the number ofbytes used by a data table on theMacintosh, click the table icon in theFinder to select it, and use the Get Infocommand in the File menu.

As an example take the familiar BIGCLASS data table in the sample datafolder. It has 40 observations and 5variables:

¥ name - width 12+1 = 13

¥ age - 8¥ sex - width 1+1 = 2¥ height - 8¥ weight - 8

for a total of 39 bytes, giving39*40 = 1560 bytes used by the data.Add in overhead of 102 bytes*5variables = 510, giving a total of 1560+ 510 = 2070 bytes used for the table.The Get Info dialog for the BIG CLASStable at the top in Figure A, shows2,208 bytes, which includes thecharacters used in column and tablenotes. Windows rounds up andshows 3,000 bytes.

Using List CheckWhen you have a variable that has256 or fewer unique values (eithernumeric or character) you can use theList Check option in the New Column(or Column Info) dialog. When youclick the List Check radio button in theColumn Info dialog for an existingcolumn, such as age in this example,the List Check dialog appears andlists the values of the variable asshown in Figure A. List Checkperforms these two functions:¥ Validation of data entry is in ef-

fect; you can only enter the valueslisted in the List Check Dialog.

¥ Data is stored more efficiently;List Check codes each value into asingle byte and maps it to theactual value for spreadsheetdisplay and analysis.



-21-

Figure A Effect of Using List Check Using List Check on thevariable age reduces thenumber of bytes neededon disk from 2208 to1956. This is a spacereduction of about 11%(see the Get Info dialogsat the top and bottom inFigure A). Windowsshows a reduction from3k to 2k.

This may not seem likemuch of a savings, butconsider the effect whenthe table is very large(say 10,000 observationsor more) and there aremany codable variables,especially long charactervariables.

The magnitude ofsavings became clear tous when we beganprocessing a table with32,138 rows and 19columns, which used4.404 meg of space.Turning List Check on forthree character variablesof length 2 with 10values, length 24 with 8values, and length 35with 30 values reducedthe disk space used to2.446 megÑ a savings ofover 44%!



-22-

Enabling Short NumericsBy default, each numeric variable uses8 bytes; changing the field width onlychanges the width used forformatting the values in thespreadsheet.

However, a new Preference calledEnable Short Numerics adds 3 newdata types to the Data Type popupmenu in the Column Info dialog:Integer 1, Integer 2, and Integer 4. Theyreduce the number of bytes used tostore a numeric variable and can beused when you have these numericinteger values:

¥ Integer 1 for Ð126 to +127¥ Integer 2 for Ð 32,766 to +32,767¥ Integer 4 for Ð 2,147,483,646 to

+ 2,147,483,646In the BIG CLASS example, using ListCheck as before, changing height fromNumeric to Integer 1, and weight toInteger 2, the file size reduces to1,440k from the original 2,208. Manysurvey items have ordinal integerresponses that can be storedefficiently with these new Integerdata types.

You are invited to visit and talkwith us at these conventionsand trade shows

Apr 06-09 Experimental BiologyNew Orleans, LA

Aug 6-8 MacWorldBoston, MA

Apr 22-24 Quality ExpoChicago, IL

Aug 9-15 ASAAnaheim, CA

May 5-7 ASQCOrlando, FL

Sep 07-11 ACS FallLas Vegas, NV

May 5-7 SematechSan Antonio, TX

Sep 21-23 SESUGJacksonville, FL

May 14-17 Interface '97Houston, TX

Sep 28-30 MWSUGChicago, IL

Jun 12-15 ICE '97Minneapolis, MN

Oct. 5-7 NESUGBaltimore, MD

Jun 17-19 PC ExpoNew York City, NY

Oct. 22-24 WUSSUniversal City, CA

Jun 23-25 Drug Information Assoc.Annual conferenceMontreal, Canada

Nov 9-11 SCSUGHouston, TX

Nov 30-Dec 5 RSNAChicago, IL



-23-

ÒThe Key isnÕtwhat you know.The key is whatyou can teachothers and havethem apply.Ó

by Colleen JenkinsSAS Institute Inc.

In November JMP users from acrossthe country attended the first JMPData Discovery Conference held atSAS Institute Inc.Õs corporateheadquarters in Cary, NC. This week-long training conference was designedto increase the return on investmentthat each attendee has made whenapplying the statistical methodsavailable in JMP to collecting andmaintaining their data.Attendees included industry-leadingcompanies like Procter & Gamble,Motorola, Immunex Corporation,Glaxo-Wellcome, Intel Corporation,Duke Comprehensive Cancer Center,Blue Cross Blue Shield of Florida, TheGoodyear Tire and Rubber Company,Dow Chemical Company, EastmanKodak Company, Hughes Aircraft,and Eli Lilly Company.Some of the conference objectiveswere:

¥ to provide training on keystatistical methods and analysesusing JMP

¥ to help client attendees identifymethods and strategies to improvetheir competitive position

¥ to provide a forum for attendees tointeract and share ideas

¥ to meet with JMP softwaredevelopers to ask questions andgive input about future softwaredevelopment.

The conference kicked off with akeynote speaker, Dr. Tom Little,Director of Engineering Support atRead-Rite Corporation. Hispresentation, ÒTen Keys to AchievingRobust Product and Process Designs,Ófocused on the competitiveadvantages gained when analyticaltechniques are used to do more,better and faster! At Read-RiteCorporation Dr. Little is responsiblefor all SPC applications, use andtraining of DOE methods, character-ization of new products andprocesses, measurementcharacterization and control, andwafer fab product engineering.



-24-

The objectives of the conference weremet through lecture and workshopsessions. Five one-day courses wereconducted addressing:

¥ categorical data analysis:investigation of Mosaic plots,frequency tables, odds ratio, theCochran-Mantel-Haenszel test,logistic regression, andcorrespondence analysis.

¥ ANOVA and regression methods:evaluation of models with a singlecontinuous response and simple,crossed, or nested categorical andcontinuous predictors.

¥ multivariate statistical methods:introduction to multivariatemodeling, principal componentsanalysis, canonical correlation, anddiscriminant analysis.

¥ advanced design of experiments:design and analysis of experimentswith single or multiple responsesincluding repeated measures,optimization designs, andnonstandard designs, with bothfixed and random effects.

¥ reliability and survival analysis:analysis of reliability and survivaldata, using the Kaplan-Meiermethod, parametric models, andproportional hazards models.

Other activities included an eveningdinner event that gave attendees theopportunity to interact with eachother and learn how JMP is used indifferent corporate environments.

JMP developers and staffparticipated in breaks and lunches toencourage feedback from customerson needs and wants in futureversions of JMP software. Also,round-table lunch discussions hostedby JMP staff focused on specificaspects of the JMP product and itsfuture development.During 1997 two more JMP DataDiscovery Conferences are scheduledto be held at SAS Institute in Caryduring the weeks of

July 15 to July 18October 28 to October 31

Mark these dates on your calendar asconference enrollment is limited. Formore information about the upcomingconferences, or to register, call

919-677-8000 x5005

or send FAX to 919-677-8225

Attendees commented,ÒI loved getting to sit around and talkstatistics with people who really know

what they are talking about. There are afew of us at work that bumble around

blindly together, but IÕve been enlightenedthis week.Ó

ÒItÕs obvious that a lot of time andthought went into the courses. It was greatfun interacting with such knowledgeable

folks!Ó



-25-

EDITORAnn Lehman

CONTRIBUTORSAnnie Dudley Michael Hecht Colleen JenkinsAnn Lehman John SallAnnette Sanders

If you have questions or commentsabout JMPer Cable write to

JMPer CableSAS Institute Inc.SAS Campus Drive

Cary, NC 27513

© Copyright 1997 SAS Institute Inc.All rights reserved.

JMPer Cable is sent only to JMPusers who are registered with SASInstitute.

For more information on JMP, or toorder a copy, contactSAS Institute, JMP Salesphone: 919-677-8000 x 5071FAX: 919-677-8224

You can also browse our web site at

<http://www.sas.com/jmp>

SAS, JMPer Cable, and JMP areregistered trademarks of SASInstitute Inc. Other brand andproduct names are registeredtrademarks or trademarks of theirrespective companies.

Documents

FRUSTRATION REDUCTION - JMP · SAS INSTITUTE INC. SPRING 1997 ISSUE 4 FRUSTRATION REDUCTION John Sall Senior Vice President SAS Institute Inc. As work continues on Version 4 of JMP,