135© The Author 2014. Published by Oxford University Press on behalf of the European Orthodontic Society. All rights reserved. For permissions, please email: journals.permissions@oup.com

Original article

Limitations of a method used for adolescent assessment of smile aestheticsSophy K. Barber*, Nadine Houghton** and R. James Spencer*

*Department of Orthodontics, Leeds Dental Institute and Pinderfields Hospital, Wakefield and **Department of Orthodontics, Leeds Dental Institute and St. Lukes’ Hospital, Bradford, UK

Correspondence to: Sophy K.  Barber, Department of Orthodontics, Leeds Dental Institute and Pinderfields Hospital, Leeds LS2 9LU, UK. E-mail: sophy.barber@googlemail.com

Summary

Aims: To establish whether adolescent orthodontic patients with hypodontia have a preference between the aesthetic outcomes of two treatment strategies for lateral incisor agenesis.Materials/Methods: Standardized photographs of pre-orthodontic patients with missing lateral incisors were manipulated to produce images that represented space opening and tooth replacement in the lateral incisor space and space closure with canine substitution into the lateral incisor space. Adolescent orthodontic patients with hypodontia were recruited to assess the aesthetics of the images. A control group of subjects without tooth agenesis was recruited. Each examiner undertook two tests to assess the smile aesthetics of the images: (1) rating attractiveness using visual analogue scale (VAS) and (2) choice of preference between pairs of images.Results: Difficulties experienced with image manipulation and poor intra-examiner reliability of the VAS make interpretation of the results challenging. Care should be taken if findings are used to aid clinical decisions, as the validity of the main findings is questionable. Results suggest that although adolescents perceive a difference in the aesthetic result of space opening and space closure for missing lateral incisors, the impact on the smile attractiveness is not clinically significant. When forced to choose between the aesthetics of space opening or space closure, the majority of examiners chose space opening with tooth replacement. No difference was found in smile ratings or preferences between adolescents with hypodontia and those with no missing teeth.Clinical implications: The methods used in this study may not be reliable for adolescent assessment of aesthetics.

Introduction

Hypodontia is the developmental absence of one or more primary or secondary teeth excluding the third molars (1). Reported preva-lence of hypodontia in the permanent dentition varies greatly but is accepted as being approximately 4–6 per cent in Caucasians, with a 3:2 female predilection (1, 2). The maxillary lateral incisor is the second most commonly missing tooth after mandibular second pre-molar, with a prevalence of 1.55–1.78 per cent. In contrast to all other teeth, bilateral agenesis occurred more commonly than unilat-eral agenesis in maxillary lateral incisors (3).

Three main treatment strategies exist for the management of missing lateral incisors:

1. No orthodontic treatment with no/minimal restorative modifica-tion of adjacent teeth.

2. Open and restore lateral incisor space.3. Orthodontic space closure with canine substitution.

Historically, opening and restoring the lateral space was con-sidered to be the optimum treatment as it restores the dentition to fulfill Andrews’ Six Keys of Occlusion (1972). It has been suggested that space opening may produce a superior result in patients with a Class III skeletal pattern with a retrusive, concave mid-face; with no crowding and stable Class I buccal segments; or with unfavourable canine characteristics, such as large, dark-coloured teeth with high gingival margins. Additionally, if there are poor prognosis teeth or

Page 135 of 141 | European Journal of Orthodontics, 2013, Vol. 00, No. 00

D. Karoline et al. | Page 135 of 141

European Journal of Orthodontics, 2015, 135–141doi:10.1093/ejo/cju021

Advance Access publication July 12, 2014

other missing teeth in the quadrant space opening may be preferable (4–7). The main advantages are restoration of canine guidance and optimal posterior occlusion, no destruction to tooth tissue through re-shaping and potentially superior aesthetics. These must be bal-anced against the cost, reliability and long-term maintenance of the restoration, and the risk of long-term loss of alveolar bone width and height (8).

More recently, significant support has been expressed for ortho-dontic space closure, with substitution of the canines into the lateral incisor position and canine modification where required (7, 9, 10). The substantial advantage of maintaining a natural dentition with-out reliance on prosthetic work maximizes biological compatibil-ity and stability and minimizes long-term maintenance. However, a number of common problems are associated with space closure and canine substitution (11, 12). These include colour and shape of the canine, relative size discrepancy of the premolar, differing gingival heights, crown torque differences, and an unstable final dentition prone to relapse with space reopening.

This study used digital images and computer-aided manipula-tion to assess the aesthetic outcome of the two methods for treating developmentally absent lateral incisors. Schabel et al. (13) demon-strated that standard digital photographs are valid tools for the analysis of the post-treatment smile compared to digital video clips. Image manipulation allows virtual treatment to assess the outcome, without any detrimental effects of interventive treatment. Small facial and dental changes within a single subject can be assessed while controlling all other variables (14).

The aim of this study was to establish whether adolescent ortho-dontic patients with hypodontia have a preference between the aesthetic outcomes of two treatment strategies for lateral incisor agenesis. The null hypotheses were as follows:

1. Adolescent orthodontic patients will not show a difference in rat-ings or preference between the aesthetic outcome of space open-ing with prosthetic replacement and space closure with canine substitution.

2. There is no difference in smile ratings or preference between examiners with hypodontia and those with no missing teeth.

Materials and methods

The study was undertaken in two hospitals in Yorkshire, UK. Seventeen pre-treatment orthodontic patients aged 11–16 years with unilateral or bilateral missing lateral incisors were recruited to act as photographic subjects. Any subjects with other absent anterior teeth, poor restorations, or severe aspects of malocclusion in the anterior region were excluded. Following enrolment and consent, standard-ized digital colour photographs were taken of their dentitions. The photographs were then manipulated using Adobe® Photoshop® CS2 software to produce two images to represent the two treatment strategies:

• Space opening and tooth replacement in the lateral incisor

space (T1).

• Space closure with canine substitution into the lateral

incisor space (T2).

A total of 34 images were produced. For T1, the prosthetic teeth were generated by copying the central incisors to mimic the tooth shape but with reduced dimensions. Minimal alterations were made to the existing gingival contour. The prosthetic tooth was given a slight grey hue to mimic the common appearance of resin-bonded

bridges. The aesthetics of the pontic was created to be satisfactory, rather than excellent, to reflect current standards of tooth replace-ment. For T2, minimal canine modification was undertaken, again to reflect current practice in the UK where tooth bleaching, gingi-val procedures, and veneer restorations are not widely offered. The only modifications that were made were those that would mimic the results of orthodontic means of camouflage (e.g. tooth positioning) and some enameloplasty.

Facial photographs were taken of five Caucasian female subjects aged 13–14 years to produce a composite face, into which the altered dentitions were inserted. It was decided to use a female image as non-syndromic lateral incisor agenesis has been reported as more prevalent in females than in males. The final image was considered to be of average attractiveness, symmetrical, and with frontal facial proportions closest to clinically accepted norms. These values were based on findings from an extensive literature review of components of facial attractiveness.

In the second part of the study, orthodontic patients aged 11–16  years were recruited as examiners to complete the tests. The study group consisted of 51 orthodontic patients who had developmentally missing anterior teeth (maxillary or mandibular, canines or incisors), to compare to a control group of 51 patients without missing teeth. Parental consent and examiner assent was obtained.

Two PowerPoint presentations were made to allow two tests to be undertaken to assess the aesthetic outcome of the images. The presentations were both timed and ran through on a pre-determined timing sequence from the appearance of the first image and examin-ers were not able to look back through the images. Standardized verbal and written instructions were given to all subjects, but partici-pants were not given any further background information regarding the study and the images. The following tests were undertaken:

1. Rating attractiveness of each smile using a visual analogue scale (VAS): 34 test images plus 4 repeat images to test reliability.

2. Forced choice between pairs of images: 17 pairs plus 3 repeat pairs to test reliability.

When rating with the VAS, the subjects were asked to rate how attractive each image was from highly unattractive to highly attrac-tive by placing a vertical mark on an unmarked 100 mm scale. For the ipsative pairs, the examiner was asked to choose between the pair of images created from each subject (T1 and T2).

Ethical approval was obtained prior to starting the study and a small pilot study involving four subjects who were not included in the study was conducted to determine whether the experimental procedure detailed above was satisfactory.

The primary method of assessing examiner preference was the VAS. A 15% VAS difference, 15 mm on a 100 mm scale, was taken as a clinically significant difference in this study, as previously used by Parekh et al. (15) and Ioi et al. (16). For a power of 90% and a significance level of 0.05, a sample size of 17 was calculated as neces-sary. Therefore, 17 original images were required for manipulation. For the ipsative pairs, it was agreed that, for this study, a preference of more than 20% towards either treatment would be clinically sig-nificant. For a power of 80% and a significance level of 0.05, a sam-ple size of 51 was calculated as necessary. Therefore, 51 examiners were needed in both the study group and the control group, giving a total of 102 examiners.

Appropriate statistical analysis was undertaken using IBM SPSS Statistics version 20.0 software. The following tests were undertaken:

European Journal of Orthodontics, 2015, Vol. 37, No. 2136

Visual analogue scale

• Descriptive statistics for each image—mean, standard

deviation (SD), and minimum/maximum score.

• Shapiro–Wilk test to test for normality.

• Bland–Altman plot for intra-examiner agreement and for

examiner agreement in the VAS measurement.

• Paired t-test to compare T1 and T2 for all examiners com-

bined.

• Independent t-test to compare scores for T1 and T2 for

those with hypodontia (study group) against those with-

out hypodontia (control group).

Ipsative pairs

• Cronbach’s alpha to test for intra-examiner agreement.

• Z-test to establish if there is a difference in the proportion

of examiners who prefer T1 or T2.

• Chi-square test to test if the proportion of examiners who

prefer T1 or T2 is the same for the hypodontia and con-

trol groups.

Results

Data collection was undertaken from April to May 2013, provid-ing a total of 102 participants to act as examiners. Examiner demo-graphics are shown in Table 1.

Examiners were recruited consecutively, with all suitable patients being invited to participate. Interestingly, the number of male and female examiners with hypodontia was equal despite the condition having a female predilection. In the control group, the majority of participants were females (3:2), which may reflect the increased demand and provision of orthodontic treatment for females. Examination of the data revealed no differences between genders in their scoring. The majority of participants were 14–16 years old, with a slightly older demographic in the hypodontia group.

Visual analogue scaleTable  2 shows the mean, SD, and minimum/maximum aesthetic scores assigned to T1 and T2 by all examiners using the VAS. The image rated as most attractive was image 33 for T1 (score 70.5, SD 20.3) and image 17 for T2 (score 53.4, SD 23.9) (Figure 1). The least attractive images were 12 (score 28.2, SD 18.0) and 29 (score 19.8, SD 15.7), respectively (Figure 2).

Normality testing of the mean scores for each image confirmed a normal distribution enabling parametric testing (Shapiro–Wilk, P = 0.470). Paired t-test of the combined aesthetic scores of all exam-iners found a statistically significant difference (P = 0.007) in rat-ings between the aesthetic outcome of space opening with prosthetic replacement and space closure with canine substitution. However,

Table 1. Demographics of the examiners.

Hypodontia group Control group Total

Males 25 13 38Females 26 38 64Age (years) 11 2 2 4

12 3 7 1013 9 6 1514 15 12 2715 10 13 2316 12 11 23

Table 2. Aesthetic scores from the visual analogue scale (VAS) for the outcome of space opening (T1) and space closure (T2) for all examin-ers (combined hypodontia and control group).

Image number

Space opening (T1)Image number

Space closure (T2)

Mean score SD Minimum Maximum Mean score SD Minimum Maximum

1 46.0 19.8 4.0 98.0 3 41.4 19.2 0.0 96.52 30.2 16.0 2.5 82.5 4 23.3 16.6 0.0 83.55 38.7 21.6 1.0 93.0 7 41.2 19.5 4.0 92.06 58.2 22.5 7.0 97.0 9 28.2 17.4 0.0 72.58 44.1 20.2 3.5 100.0 10 24.5 15.8 0.0 70.511 42.2 21.8 5.0 97.0 13 20.4 17.3 0.0 79.512 28.2 18.0 0.0 85.5 14 50.5 22.3 3.0 100.015 54.7 22.3 7.0 99.0 16 28.2 17.9 0.0 85.518 46.5 25.2 0.0 96.0 17 53.4 23.9 4.0 100.021 58.5 24.0 7.0 100.0 19 29.3 17.4 0.0 70.024 37.8 20.9 0.0 91.5 20 31.4 18.6 0.0 95.525 38.0 19.3 0.0 87.0 22 51.1 22.4 4.5 98.526 43.7 19.8 1.0 86.5 23 33.4 17.8 0.0 87.027 48.1 21.0 3.0 92.0 28 26.0 19.4 0.0 97.531 55.0 22.8 3.5 100.0 29 19.8 15.7 0.0 82.532 43.6 18.4 3.0 89.5 30 30.2 17.5 0.0 90.033 70.5 20.3 13.0 100.0 34 44.5 18.9 3.5 82.5Total 46.1 10.7 0 100 33.9 11.0 0 100

SD, standard deviation.Space opening (T1) versus space closure (T2), P = 0.007.Hypodontia versus control: T1, P = 0.559.Hypodontia versus control: T2, P = 0.919.Bold italic values indicate highest rating and bold values indicate lowest rating.

S. K. Barber et al. 137

the difference in mean scores between T1 (mean 46.1, SD 10.7) and T2 (mean 33.9, SD 11) was 12.1 mm, and this was not deemed to be clinically significant.

The scores for the hypodontia group and control group for T1 (space opening) and T2 (space closure) were analysed separately. Independent t-tests found no statistically significant difference in smile ratings between examiners with hypodontia and the control group for either T1 (P = 0.559) or T2 (P = 0.919).

Ipsative pairsPreferences for space opening and prosthetic replacement (T1) and space closure with canine substitution (T2) images for the hypodontia and control groups are shown in Table 3. The major-ity preferred T1 (78.5%) compared to T2 (21.5%) with pref-erence for T2 expressed in only two pairs. The proportion of examiners in each group who preferred each option was remark-ably similar; 39.5 per cent in the hypodontia group and 39 per cent in the control group for T1 and 10.5 and 11 per cent, respec-tively, for T2.

Z-test found a significant difference between examiner preferences towards T1 compared to T2 (P < 0.001), and this preference (78.5 per cent) was deemed to be clinically significant. This indicates that

adolescent orthodontic patients preferred the aesthetic result of space opening with prosthetic tooth replacement for managing missing lat-eral incisors, compared to space closure with canine substitution.

Chi-square test for association failed to find a statistically sig-nificant difference in the preferences expressed by the control and study groups (P = 0.157), indicating that there was no difference in smile preference between examiners with hypodontia and those with no missing teeth.

ReliabilityReliability of the VAS measurements was tested by re-measuring five examiners’ answers, providing a total of 190 measurements. The Bland–Altman plot shows that the agreement of the measure-ments was excellent (Figure 3). The mean error in measurement was 0.04 mm (SD 0.488).

The intra-examiner reliability of the VAS and ipsative pairs was obtained using repeat images within the PowerPoint. The Bland–Altman plot shows poor intra-observer reliability for the repeat images, with a high variability between the scores assigned to the original and repeat images (Figure 4).

Cronbach’s alpha indicated that the reliability of the ipsative pair preferences was acceptable. Although a low value was found (0.5), this may be attributed to the fact that correlation was only being tested between two variables. In this circumstance, it has been rec-ommended to report the inter-item correlation, which was found to be within the optimal range of 0.2–0.4 (0.334) (17).

Discussion

The findings of both the aesthetic scores and the preferences indi-cate that examiners perceived a difference between the aesthetic result of space opening with tooth replacement and space closure with canine substitution. A statistically significant preference was expressed towards space opening although this was only deemed to be clinically significant in the ipsative pairs. Contrary to our findings, Armbruster et al. (18) found that orthodontists, dentists, and lay people all rated post-treatment photographs of people with missing lateral incisors who had undergone space closure and canine substitution higher than those who had prosthetic tooth replacement. Robertsson and Mohlin (19) assessed patients’ perception of their own dental appearance following either space opening or space closure for missing lateral incisors. The results indicated that generally patients who underwent space closure were more satisfied with the aesthetic result than those who had space opening and prosthetic replacement. However, the study subjects were older and aesthetics of the prosthetic replacements were not objectively judged.

The aesthetic scores and preferences awarded by the two groups of examiners were assessed separately to establish if adolescents with missing teeth have a different perception of smile aesthetics to those without missing teeth. The VAS scores and pair preferences were not found to be statistically or clinically different between the hypodontia and control group. This suggests that those with miss-ing teeth are no more aware or critical of missing teeth than other orthodontic patients. A study assessing the impact of hypodontia on the OHRQoL in adults found that patients became more aware of their condition with age (20). It is possible that some of the younger examiners in our study were not yet concerned by their agenesis and as a result were no more critical of others.

These results must be interpreted with caution as two key areas of concern were identified with the methodology used. First, significant

a b

Figure 1. The images with the highest aesthetic scores from all examiners combined: (a) space opening (T1) and prosthetic replacement and (b) space closure (T2) with canine substitution.

a b

Figure  2. The images with the lowest aesthetic scores from all examiners combined: (a) space opening (T1) and prosthetic replacement and (b) space (T2) closure with canine substitution.

European Journal of Orthodontics, 2015, Vol. 37, No. 2138

difficulty was experienced with the image creation and as a result the images used were of questionable validity. Recruitment of suitable photographic subjects was difficult, not due to unwillingness to par-ticipate, but due to a lack of suitable cases being seen in the hospital departments in the study period. Some subjects had to be excluded as they were in the mixed dentition with unerupted permanent teeth, ectopic canines, or other aspects of malocclusion that precluded image manipulation. One subject had to be excluded as racial pigmentation in the gingiva made compatibility with the composite face difficult. The recruitment period was limited, and this difficulty in recruiting pho-tographic subjects meant some inferior initial images had to be used.

Further complications were experienced during the manipulation of the original images. Rotations, tip errors, crowding, unerupted

teeth or the presence of primary teeth and discrepancy in the inter-arch relationship and occlusion made image manipulation difficult. For less severe discrepancies, the teeth were moved to an accept-able position, but this was difficult for certain malocclusions and the resulting image was less aesthetic than would have actually been produced by treatment. In the cases, where the canines were moved mesially to replace the lateral incisors the resulting maxilla was often left narrow relative to the mandible. Attempts were made to manage this by magnifying the image using a vector that recreated the correct perception of the buccal corridor, but this was limited as excessive widening of the maxillary arch to correct inter-arch relationship led to enlargement of the teeth. In some cases, it was not possible to correct the dental shade to match the face due to slight differences

Table 3. Results from the ipsative pairs, showing the preference of the hypodontia and control groups for space opening (T1) and space closure (T2).

Pair Image number

Space opening (T1)

Image number

Space closure (T2)

Total, n Hypodontia, n Control, n Total, n Hypodontia, n Control, n

1 25 87 42 45 20 15 9 62 1 75 40 35 3 27 11 163 33 93 48 45 17 9 3 64 24 81 42 39 23 21 9 125 27 92 46 46 16 10 5 56 15 68 33 35 14 34 18 167 21 81 42 39 22 21 9 128 6 85 38 47 34 17 13 49 11 89 41 48 10 13 10 310 12 4 3 1 7 98 48 5011 8 83 44 39 30 19 7 1212 32 96 49 47 13 6 2 413 26 97 49 48 28 5 2 314 31 99 50 49 9 3 1 215 18 95 46 49 29 7 5 216 2 47 24 23 19 55 27 2817 5 90 48 42 4 12 3 9Total 1362 (78.5%) 685 (39.5%) 677 (39%) 372 (21.5%) 182 (10.5%) 190 (11%)

Preference between T1 and T2: P < 0.001.Preference between those with hypodontia and controls: P = 0.157.Values in bold italic indicates preferred image.

Figure 3. Bland–Altman plot for agreement of visual analogue scale (VAS) measurements.

Figure 4. Bland–Altman plot showing poor intra-examiner agreement in the visual analogue scale (VAS) scores.

S. K. Barber et al. 139

in exposure, which restricted the amount of post-exposure modifica-tion that could be undertaken. The complications with image manip-ulation can be seen in the low scoring images (Figure 2), where there are problems with colour, tooth position, and inter-arch relationship.

The second concern was with the reliability and validity of the VAS. The VAS is widely reported in dental research, and it has been shown to be simple, easy to use in a variety of settings, and reproduc-ible (15, 16, 21–30). However, the Bland–Altman plots that were used to assess intra-examiner agreement indicated poor reliability between the test–retest images. The repeat images were at the end of the tests and possible explanations for the variation in scores on the repeat images could be examiner fatigue or a change in the examiner scor-ing after seeing a number of images and dentitions. This should have been avoided by altering the order of the images, including the repeat images, during testing. The VAS has been used extensively for assess-ing perception of aesthetics (15, 16, 21–30), but in these studies the examiners were adults. Only one study that assessed aesthetics using a VAS involving adolescents aged 13–17 years could be found (31). The study reported the validity and reliability of results to be acceptable, although the exact reliability score was not reported.

The high frequency of maximum and minimum scores from the extreme ends of the scale suggests that the examiners may not have fully grasped the concept of the scale. Examiners were told that 0 represented ‘highly unattractive’ and should be only used for images that they felt were considerably worse than average, whereas 100 was ‘highly attractive’ and should be used for images that were felt were much nicer than an average smile. While the most and least attractive images may have fulfilled this description, it is possible that the examiners did not actually feel this strongly about all images that were given an extreme score. The wide range in scores for each image may have occurred because no calibration image was used prior to the testing, so each examiner had to set their own baseline score against which to measure images. A means for reducing this ‘self-calibration’ would have been to use a control image against which to the examiners could score, as used in other studies (32, 33). This method was not used as the aim was to obtain an absolute, rather than relative judgement of attractiveness, but in hindsight the use of a reference image may have reduced the wide SD and improved reliability.

The second means of assessing aesthetic preference was through ipsative pairs, a type of forced-choice testing requiring respondents to make a choice between two or more options from the same scale with no neutral response allowed. Research has demonstrated increased validity in forced-choice testing due to a reduction in central ten-dency bias and social desirability responding (34, 35). Examiners reported that the ipsative pairs testing was easy to understand and acceptable, and the comparison of from repeat images this study show intra-examiner reliability to be acceptable. Unfortunately, as it is unclear if the VAS was a valid tool against which to compare, it is difficult to draw any conclusions or make recommendations about ipsative pairs as a method of testing.

Conclusions

1. Adolescent orthodontic patients expressed a preference in the aesthetic result of space opening with prosthetic tooth replace-ment for managing missing lateral incisors, compared to space closure with canine substitution. This was both statistically and clinically significant.

2. There was no difference in smile ratings or preferences between adolescents with hypodontia and those with no missing teeth.

3. Other aspects of malocclusion may limit image manipulation of pre-orthodontic dentitions. Difficulties in image manipulation may have compromised the validity of the findings of the study.

4. The VAS may not be valid or reliable for measuring perception of aesthetics in adolescents.

AcknowledgementsMany thanks to Colin O’Sullivan for his help with image creation. The research was undertaken as part of an MSc with sponsorship from the University of Leeds.

References 1. Larmour, C.J., Mossey, P.A., Thind, B.S., Forgie, A.H. and Stirrups, D.R.

(2005) Hypodontia—a retrospective review of prevalence and etiology. Part I. Quintessence International, 36, 263–270.

2. Dhanrajani, P.J. (2002) Hypodontia: etiology, clinical features, and man-agement. Quintessence International, 33, 294–302.

3. Polder, B.J., Van’t Hof, M.A., Van der Linden, F.P. and Kuijpers-Jagtman, A.M. (2004) A meta-analysis of the prevalence of dental agenesis of per-manent teeth. Community Dentistry and Oral Epidemiology, 32, 217–226.

4. Lewis, B.R., Gahan, M.J., Hodge, T.M. and Moore, D. (2010) The ortho-dontic-restorative interface: 2. Compensating for variations in tooth num-ber and shape. Dental Update, 37, 138–140, 142–4, 146–8 passim.

5. McNeill, R.W. and Joondeph, D.R. (1973) Congenitally absent maxillary lateral incisors: treatment planning considerations. Angle Orthodontist, 43, 24–29.

6. Savarrio, L. and McIntyre, G.T. (2005) To open or to close space—that is the missing lateral incisor question. Dental Update, 32, 16–18, 20–2, 24–5.

7. Rosa, M. and Zachrisson, B.U. (2010) The space-closure alternative for missing maxillary lateral incisors: an update. Journal of Clinical Ortho-dontics, 44, 540–549; quiz 561.

8. Sabri, R. (1999) Management of missing maxillary lateral incisors. Journal of the American Dental Association, 130, 80–84.

9. Zachrisson, B.U., Rosa, M. and Toreskog, S. (2011) Congenitally missing maxillary lateral incisors: canine substitution Point. American Journal of Orthodontics and Dentofacial Orthopedics, 139, 434, 436, 438 passim.

10. Tuverson, D.L. (2004) Close space to treat missing lateral incisors. Ameri-can Journal of Orthodontics and Dentofacial Orthopedics, 125, 17A.

11. Rosa, M. and Zachrisson, B.U. (2007) Integrating space closure and esthetic dentistry in patients with missing maxillary lateral incisors. Jour-nal of Clinical Orthodontics, 41, 563–573; quiz 424.

12. Rosa, M. and Zachrisson, B.U. (2001) Integrating esthetic dentistry and space closure in patients with missing maxillary lateral incisors. Journal of Clinical Orthodontics, 35, 221–234.

13. Schabel, B.J., Baccetti, T., Franchi, L. and McNamara, J.A. (2010) Clinical photography vs digital video clips for the assessment of smile esthetics. Angle Orthodontist, 80, 490–496.

14. Carlsson, G.E., et  al. (1998) An international comparative multicenter study of assessment of dental appearance using computer-aided image manipulation. International Journal of Prosthodontics, 11, 246–254.

15. Parekh, S.M., Fields, H.W., Beck, M. and Rosenstiel, S. (2006) Attractive-ness of variations in the smile arc and buccal corridor space as judged by orthodontists and laymen. Angle Orthodontist, 76, 557–563.

16. Ioi, H., Nakata, S. and Counts, A.L. (2009) Effects of buccal corridors on smile esthetics in Japanese. Angle Orthodontist, 79, 628–633.

17. Briggs, S.R. and Cheek, J.M. (1986) The role of factor analysis in the development and evaluation of personality scales. Journal of Personality, 54, 106–148.

18. Armbruster, P.C., Gardiner, D.M., Whitley, J.B. and Flerra, J. (2005) The congenitally missing maxillary lateral incisor. Part 1: esthetic judgment of treatment options. World Journal of Orthodontics, 6, 369–375.

European Journal of Orthodontics, 2015, Vol. 37, No. 2140

19. Robertsson, S. and Mohlin, B. (2000) The congenitally missing upper lateral incisor. A retrospective study of orthodontic space closure versus restorative treatment. European Journal of Orthodontics, 22, 697–710.

20. Meaney, S., Anweigi, L., Ziada, H. and Allen, F. (2012) The impact of hypodontia: a qualitative study on the experiences of patients. European Journal of Orthodontics, 34, 547–552.

21. Roden-Johnson, D., Gallerano, R. and English, J. (2005) The effects of buccal corridor spaces and arch form on smile esthetics. American Journal of Orthodontics and Dentofacial Orthopedics, 127, 343–350.

22. Anderson, K.M., Behrents, R.G., McKinney, T. and Buschang, P.H. (2005) Tooth shape preferences in an esthetic smile. American Journal of Ortho-dontics and Dentofacial Orthopedics, 128, 458–465.

23. Pinho, S., Ciriaco, C., Faber, J. and Lenza, M.A. (2007) Impact of den-tal asymmetries on the perception of smile esthetics. American Journal of Orthodontics and Dentofacial Orthopedics, 132, 748–753.

24. McNamara, L., McNamara, J.A., Ackerman, M.B. and Baccetti, T. (2008) Hard- and soft-tissue contributions to the esthetics of the posed smile in growing patients seeking orthodontic treatment. American Journal of Orthodontics and Dentofacial Orthopedics, 133, 491–499.

25. Kokich, V.O., Kokich, V.G. and Kiyak, H.A. (2006) Perceptions of dental professionals and laypersons to altered dental esthetics: asymmetric and symmetric situations. American Journal of Orthodontics and Dentofacial Orthopedics, 130, 141–151.

26. Kokich, V.O., Jr., Kiyak, H.A. and Shapiro, P.A. (1999) Comparing the perception of dentists and lay people to altered dental esthetics. Journal of Esthetic Dentistry, 11, 311–324.

27. Martin, A.J., Buschang, P.H., Boley, J.C., Taylor, R.W. and McKinney, T.W. (2007) The impact of buccal corridors on smile attractiveness. European Journal of Orthodontics, 29, 530–537.

28. Flores-Mir, C., Silva, E., Barriga, M.I., Lagravere, M.O. and Major, P.W. (2004) Lay person’s perception of smile aesthetics in dental and facial views. Journal of Orthodontics, 31, 204–209; discussion 201.

29. Jornung, J. and Fardal, O. (2007) Perceptions of patients’ smiles: a com-parison of patients’ and dentists’ opinions. Journal of the American Dental Association, 138, 1544–1553; quiz 1613–4.

30. Shaw, W.C. (1981) The influence of children’s dentofacial appearance on their social attractiveness as judged by peers and lay adults. American Journal of Orthodontics, 79, 399–415.

31. de Oliveira, S.C., Furquim, R. and Ramos, A.L. (2012) Impact of brackets on smile esthetics: laypersons and orthodontists perception. Dental Press Journal of Orthodontics, 17, 64–70.

32. Faure, J.C., Rieffe, C. and Maltha, J.C. (2002) The influence of different facial components on facial aesthetics. European Journal of Orthodontics, 24, 1–7.

33. Honn, M., Dietz, K., Eiselt, M.L. and Goz, G. (2008) Attractiveness of facial profiles as rated by individuals with different levels of education. Journal of Orofacial Orthopedics, 69, 20–30.

34. Baron, H. (1996) Strengths and limitations of ipsative measurement. Jour-nal of Occupational and Organizational Psychology, 69, 49–56.

35. Christiansen, N.D., Burns, G.N. and Montgomery, G.E. (2005) Recon-sidering forced-choice item formats for applicant personality assessment. Human Performance, 18, 267–307.

S. K. Barber et al. 141