Thoughts on Procedural...

Tom Kelly (tkelly@dcs.gla.ac.uk)

first year PhD report, Jan 2010

Contents

1 What is Procedural Geometric Modelling? 3

2 Biologically Inspired Morphology 32.1 Emergent behaviours . . . . . . . . . . . . . . . . . . . . . . . . . 42.2 Growth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.3 Regulative Development . . . . . . . . . . . . . . . . . . . . . . . 9

3 Commercial uses 123.1 Procedural Incs CityEngine . . . . . . . . . . . . . . . . . . . . . 133.2 Gamr7s Urban PAD . . . . . . . . . . . . . . . . . . . . . . . . . 143.3 Maxiss Spore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143.4 IDVs SpeedTree . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.5 Bentleys GenerativeComponents . . . . . . . . . . . . . . . . . . 16

4 Some Research Directions 164.1 Flora . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4.1.1 The Algorithmic Beauty of Plants . . . . . . . . . . . . . 174.1.2 Other Reading in Flora . . . . . . . . . . . . . . . . . . . 22

4.2 Urban . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294.2.1 Shape Grammars . . . . . . . . . . . . . . . . . . . . . . . 294.2.2 Architectural Theory . . . . . . . . . . . . . . . . . . . . . 294.2.3 City Modelling . . . . . . . . . . . . . . . . . . . . . . . . 334.2.4 Broader Topics . . . . . . . . . . . . . . . . . . . . . . . . 374.2.5 Reverse Procedural Techniques . . . . . . . . . . . . . . . 40

5 Straight Skeleton 425.1 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

5.1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . 435.1.2 Data Structure . . . . . . . . . . . . . . . . . . . . . . . . 435.1.3 Collisions Between Three Edges . . . . . . . . . . . . . . . 445.1.4 Ensuring Collisions Only Occur On Faces . . . . . . . . . 455.1.5 Many Edges Colliding . . . . . . . . . . . . . . . . . . . . 495.1.6 Horizontal Output Edges . . . . . . . . . . . . . . . . . . 505.1.7 Negative edge gradients . . . . . . . . . . . . . . . . . . . 535.1.8 Ambiguous Weighted Skeleton . . . . . . . . . . . . . . . 535.1.9 Reconstructing Faces . . . . . . . . . . . . . . . . . . . . . 55

5.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

6 Conclusions 56

7 Future Directions 577.1 Spectrum of Proceduralizations . . . . . . . . . . . . . . . . . . . 577.2 Evaluation of Procedural Systems . . . . . . . . . . . . . . . . . . 587.3 Thesis Subject . . . . . . . . . . . . . . . . . . . . . . . . . . . . 597.4 The Future . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

2

1 What is Procedural Geometric Modelling?

Procedural geometric modelling is the engineering of algorithms that create ge-ometry.

Whilst a standard 3D work flow allows a user to create a single chair of aspecific height, a procedural work flow might let a user create an algorithm thatproduce a chair of any height.

The benefits of procedural content include:

Removes the restriction that the size of a virtual environment is propor-tional to the time spent creating it.

The quality of the environment is consistent at no additional cost. In the long term efficient procedural geometry tools could lead to runtime

environment generation. A virtual world would be generated as the userexplores, giving an experience with more variety and less repetition to theuser.

Procedural methods offers the potential to generate content that reactsto various stimuli. For example it could respond to current hardwareavailability, to users level of expertise or to the length of their attentionspan.

It is easier to for users to create custom content given a procedural model,rather than starting from scratch.

Before we begin, a few words and phrases that should be defined:

Model An algorithm that produces a (normally geometric)model

Instance The output of a single invocation of a model.Morphogen A substance which causes a change in form of an

object.Morphospace of a model The conceptual space created by all the different pos-

sible instances that a model could output

2 Biologically Inspired Morphology

Several ancient examples of design systems can be found in nature. In generalthese techniques are inherently parallel and robust, but have properties thatmake it hard for humans to engineer.

Evolution, the modification of organisms to survive in their environment,gives one view of a procedural system. The controlling algorithm is an evolu-tionary system that adapts the design of a creature so that it may survive andreproduce.

Another classical example is organism growth - complex structures are grownfrom simple beginnings, often a simple cell.

3

It is instructive to examine a few of the phenomena that give natural sys-tems their intricate and compelling morphospace. Biologists are not yet able topresent a complete system for many of these complex systems, however whatthey have discovered shows many insights into the properties of complex pro-cedural systems. While they have been divided into different groups discussingdifferent properties of natural morphology, many of the examples belong to morethan one group.

2.1 Emergent behaviours

An emergent system shows behaviours that were never directly stated or pro-grammed. In our case we are interested in behaviours that create novel andinteresting forms.

Perhaps the classic example are cellular automata. These systems manipu-late a symbolic environment over time. The classic example is Conways Gameof Life[22], in which a grid of black and white squares, and some simple rules,give rise to some quite unexpected behaviours.

The rules allow for cells, represented by black grid squares, to be created ordestroyed, based on their neighbours after the previous iteration. Cells are bornin cells adjacent to three neighbours on the previous iteration. Cells with onlyone, or four or more neighbours die, while cells with two or three neighbours onthe previous iteration remain alive.

Figure 1: a) The cycle of a glider pattern in the Game of Life. b) The Gosperglider gun, a formation that creates a stream of gliders every 30 iterations.

Given just these simple rules and a simple starting arrangement, large andcomplex patterns can be produced. There are several patterns that, once theyhappen to occur, are self perpetuation, such as the glider in figure 1,a, whichmoves through space. Initial conditions have even been discovered that createa continuous stream of these gliders, such as figure 1,b.

Indeed it was cellular automata like these that caused Doyne Farmer andChristopher Langton to discuss life at the edge of Chaos[32], while investigat-

4

ing the properties of cellular automata. This is generally taken to mean thatcomputation, and with it interesting form, mostly occurs in situations undergo-ing phase-transition. That is there are not so many events to make informationstorage impossible, but enough interactions to perform computations at all.

My feelings about cellular automata are that, while they produced intricatepatterns, because they were abstract and not founded on physical rules theywerent true to biological systems. I also wished to see if it was possible toengineer successful outcomes. I performed a short experiment using a computerrunning a 2D physics simulator[58].

A number of cells were modelled as circles. They are assigned types, repre-sented as colours, and each diffuses a particular morphogen. These morphogensdisperse over time. Each type of cell changes its acceleration beased on the theother morphogen gradients. For example the green cells may attempt to movein the direction of the blue morphogen, while avoiding the morphogen that theythemselves emit, as in figure 2.

By changing the response to the morphgen gradient, it was indeed possibleto create quite complex behaviour. For example the cells quickly created thethree-layered objects in figure 2, however the results were dramatically differentevery time the simulation was run. In comparision with a pure geometric design,(where we might have simply stated a center and three radii) it is a difficultsystem to work with. However if we didnt know the volume that the layeredobject would be bounded by, the morphogen-gradient method may be preferable.

Indeed the process of evolution itself can be regarded as an emergent be-haviour of simple proteins. Like the other emergent behaviours, their is nopredefined goal or destination to the morphological forms.

Generally emergent behaviours to have the following properties:

Robust. Generally the shape and forms that are generated can be brokenup, but will often reform with similar characteristics.

Variety of output. The unpredictable nature of emergent behaviours tendsto lead to a large variety of results.

Difficult to Control. Almost by definition, emergent behaviours do some-thing other than that they were designed for. This makes them difficultfor humans to engineer in a standard graphical workflow.

Expensive. The phenomena examined here are parallel mechanisms. Thatis a set events happen in parallel inside a simulation. Because of theserial von Neumann bottleneck these scenarios are expensive to compute.

2.2 Growth

When two growing bodies compete for space, they subdivide that space. Forexample cells within a membrane, such as occurs in an embryonic mammal,grow and divide, whilst carving the space within the membrane into discreteregions. Partitioning space in this way turns out to be a useful procedural tool,figure 3.

In two dimensions, with bodies growing at a constant speed, the resultis very similar to a computational geometry algorithm known as the Voronoitessellation[60], figure 4. The tessellation has some useful properties - it will

5

Figure 2: Simulated cells (red, green & blue) following morphogen gradients(green type morphogen shown in white). Panels 1-4 shown as the simulationcontinues.

colonise all the space, that is growth continues until every region belongs to onecell or another, and it will terminate with as many partitions as specified whenstarting.

The Voronoi algorithm classifies all space as being nearest one or another ofthe cell centers. Viewed from the perspective of growth, each area is colonisedfirst by the nearest cell.

There are several algorithms to calculate the Voronoi diagram[60], the edge-cutting technique was used to create the city street layouts for one of the au-thors earlier projects (figure 5), while Fortunes algorithm is faster in lowerdimensions[21].

Additionally we may wish to have our objects in 2D grow at different rates,or to partition space in three or more dimensions. This leads to the weightedVoronoi diagram[10], as in figure6, and higher dimensional Voronoi diagrams[61]as in figure 7, respectively.

Using this class of space-subdivision algorithms provides some of the geo-metric robustness that biological systems have, but remain easily controlled. Itis also requires less computing power, and is entirely deterministic. The straightskeleton[3] can also be used as a space subdivision technique, I work on thealgorithm in great detail in later chapers.

6

Figure 3: Belousov-Zhabotinsky reaction forming a Voronoi-like pattern becausethe boundaries occur where the chemicals phases collide [2].

Figure 4: A Voronoi tessellation with one million points, from [56]

7

Figure 5: An earlier project that takes sets of points, and uses the Voronoitesselation to create city-street-like maps, such as those on the right. To addrealism, randomly selected plots are merged with their neighbours.

Figure 6: An additively weighted Voronoi diagram. The points are the cellcenters [28].

8

Figure 7: Three dimension Voronoi tessellation. Cell centers not show [52].

2.3 Regulative Development

Regulative development is the study of the interactions that cause an egg todivide and grow. There are several documented[4] techniques for mechanismsto change the developmental path of one particular cell. While many cellsbegin identical, regulative development may cause it to become different typesof tissue, or behave in some specific way.

When certain types of cells are adjacent, new cell-division behaviours canbe turned on. This induction can happen several times to build up layers ofdifferent cells as the adjacency changes, as in figure 8. Because the techniquedepends on contact between cells, induction only works locally.

Chemicals which affect cell development paths are known as morphogens.Based on the concentrations of morphogens, cells can develop in down differentpathways to their eventual form. Additionally, cell age also has an effect, sothat older cells can behave in different ways to new cells.

in this way the final specification of how a limb cell should behave is built upcombinatorially; first it is supplied with information as whether it is to be a leg

or wing then signals within the growing limb bud specify more fine-grainedcomponents of positional value, reflecting the precise position within the

limb.[4]

So a programmer might say that values have been curried into the codefor each part of the limb. You can see this in an experiment - if the tip of aembryonic chickens foot is transplanted onto the end of the wing, the result inan adult is a wing with a toe on the end; that is the tissue has already beencurried (fated) as part of a leg.

As this chicken example shows these fields are robust when dealing withgeometrically changing features (as disrupted the system can re-organise itself

9

Figure 8: Cell Induction. i) two adjacent regions of cells, ii) new cell typeinduced by adjacency, iii) additional strata of cells via induction

ignoring inconsistencies) and may well prove to be well suited to proceduralgeometry that adapts to new configurations.

Further experiments have shown these principles in practice by taking a flyegg (Drosphila) and removing some of the fluid containing the morphogens fromthe head end leading to a fly without a head. Alternately if some of the criticalmorphogenic chemical is removed from the rear of the fly and injected into thehead, a fly with two sets of abdominal segments is produced. Morphogenic fieldscertainly solve the problems of symmetry in simple grammars (since diffusionis equal in all directions). There are several techniques that seem to be used -differentiating features by field strength or using a combination of fields.

Figure 9 shows how a single cell may have several different morphogenicfields present. Development in different parts of the cell depends on the relativeconcentrations of these chemicals. By detecting the gradient of the morphogen,some processes can even be orientated. Complex harmonic patterns can even bebuilt up by releasing a morphogen in response to two other morphogens havingequal concentrations, figure 10.

Another technique is shown by the evaluation of pair-rule genes. For exampleDrosphila has a number of genes (hairy, even-skipped, paired, fushi-tarazu) ofdifferent lengths which are activated at intervals of two body segments. Booleancombinations of these genes allow more complex patterns to be expressed, with-out any additional input data, as shown in figure 11. This is very similar toMullers 01 paper [47], that uses the union of 1D square wave functions topositions elements on a buildings facade.

Finally, much later in the embryos development cycle, there is a techniquethat creates occasional features called lateral inhibition, figure 12. All the cells

10

Figure 9: A cell with several different morphogenic fields. [4], plate 21-57.

Figure 10: Top:Cell differentiation may depend on the concentration of a mor-phogen. Bottom: By releasing a third morphogen (3) when two others are equal,harmonic sequences of morphogens can be built up.

start off emitting the same concentration of signal chemical. Cells near thosewith a larger signal stop signalling, the result is an interspersed pattern. In thisexample each of the selected cells then becomes a mother cell to a clutch of

11

Figure 11: The different repeating genes (white rectangles) can be combinedusing different boolean operators to create a wide range of boolean patterns(bottom) [4].

hair-cells, while the remaining cells carry on as normal to become skin.

Figure 12: Lateral inhibition is used in the locating of hair follicles. i) The signalstrength starts out the same, ii) random noise ensures that some cells emit astronger signal that others. iii) Lateral inhibition causes cells near a large signalto stop signalling, leaving an interspersed pattern that designates the growthlocations for the follicles. [4]

All these regulative techniques rely on physical phenomena to create mor-phologies. Emulating these phenomena is computationally expensive, becausethere are many events occurring in parallel. These techniques are however veryrobust to shape changes, and disruption during growth. I believe that they arealso fairly understandable from a human engineers point of view, and can beengineered to create interesting forms.

3 Commercial uses

Much of the leading work on procedural modelling happens in the commercialworld, and many of the results are never published. I am most of aware of workin the video gaming communities.

12

Over the last two decades video games have adapted with Moores law tobecome increasingly complex. One of the consequences of this is that our gamesare more detailed. For example the complexity of each frame has risen fromthe tens of polygons in the 1980s to the tens of millions today [1]. Additionaltechniques such as displacement and bump mapping are adding additional detailbeyond polygon meshes.

As figure 13 illustrates, the increasing requirement for detail is visible in thestructure of video game development companies. The majority of the man-yearsrequired to make a modern video game are spent by artists creating models ofthe world.

Unfortunately this bottleneck of creating geometry means that it costs morethan ever to create video games. Development costs for the like of Ubisofts FarCry 2 (2008) are expected in be in the tens of millions of pounds.

Discussions with staff from Frontier Software describe some of the tools usedto accelerate the generation of large urban environments. Because much of thiswork remains unpublished, it is worth detailing a few companies that developprocedural technologies.

Figure 13: Typical staff composition in a Japanese game developer[45].

3.1 Procedural Incs CityEngine

This is the accumulation of Pascal Muellers research in the form a company.Their major selling points are:

Rapid Digital Prototyping Shorter Time-to-Market Cost Reduction High-Quality Content Redefine Design Flawless Integration

13

The market cost is $4000, and they seem to be pitching to a wide range ofcompanies, everything from video games, architecture, archaeology, simulation,film & TV.

A typical work flow involves selecting a terrain map, and a population map.From these the application can create a road network from one of several designs.After this subdivision into lots and buildings occurs. To construct buildings,shape grammars are used and driven by a domain specific Python-like scriptinglanguage. This final step means that using the package takes considerable train-ing, but allows complex features such as recursion and responding to changes interrain. A complex example is shown in figure 14.

Figure 14: A promotional image from CityEngine.

3.2 Gamr7s Urban PAD

Like Procedural Inc. Gamr7 have academic roots, but they are concentrating onthe gaming market. The software isnt able to procedurally generate geometry,rather instance buildings at multiple location (figure 15). However complexroad networks may be generated by small teams in short periods of time. Theirselling points:

Automatic Geometric adaptation Automatic Height Field adaptation Automatic LOD removal of objects Exports to Collada for easy mock-ups and visualisation Exports to XML for easy integration into pipeline Height field Import Configurable Forbidden Zone : Water, game play constraint Configurable Road Network : Spline, Bridges, Etc.

3.3 Maxiss Spore

Spore is a video game that allows users to control a creature as it evolves froma single cell life form, through a land-based stage to an advanced civilisationwith buildings and vehicles. Maxis use procedural technologies to great effectto allow the user to design a creature, whilst maintaining a worst-case quality

14

Figure 15: A promotional image Gamr7

threshold. That is, it is difficult to build a creature that doesnt look like acreature.

A small set of operations allows the effective creation of the torso and limbsof a creature. The skeleton of the creature, and the width of the muscles aroundthe skeleton can be specified by the user. An unknown system then creates afitting skin around the skeleton and calculates where the skin should crease asthe creature moves. Finally the user can position pre-made meshes such as eyesand horns, each of which has a parametrised range of variations.

This very usable domain specific geometry tool borders on a proceduralsystem. Considering that the game has been designed for young children, it isquite believable that a simple artificial intellegence could create a large varietyof creatures of good quality.

A procedural animation system is then able to take the data about whichlimbs are legs and which are arms to animate the creature. This is impressiveas there may be 0 or more limbs (figure 16).

Spore costs $40.

3.4 IDVs SpeedTree

SpeedTree is a commercial product that allows artists to design trees and incor-porate the results into 3D environments, usually video games as in figure 17. Theoffering is a suite of software, the most impressive part of which is SpeedTreeRT,which calculates physical effects and level-of-details in real time for video games.It is very widely used in the industry as it provides integration with existingvideo game engines.

A CAD system provides the artists with an inteface to specify what theirtrees will look like. The output from this can be used in real time or, of course,taken as a mesh and edited further in a 3D editor.

The cost is $8,500 per title released.

15

Figure 16: A user-generated character from EAs Spore. (An AVATAR Dire-horse by CopperLou, from http://bit.ly/7AYJkZ)

Figure 17: An screen capture of SpeedTrees trees in the video game Fallout 3,by Bethesda

3.5 Bentleys GenerativeComponents

Bentley has produced a CAD application, GenerativeComponents, in whichelements can be procedural. They can be defined by parametric or associativemodelling. An example of the data flow that creates a staircase is shown infigure 18. This software is important as it is used by architects to acceleratethe design of modern geodesic structures such as the Setun Hills Business Park(Moscow), and Lansdowne Road Stadium (Dublin).

4 Some Research Directions

Procedural technology has been a long time in maturing. In the geometricdomain we have seen early successes in terrain generation, then flora. Todaywere starting to see moderate success in urban environments.

16

Figure 18: An example of a staircase from GenerativeComponents. Source:http://www.aecbytes.com/review/2009/BentleyArchV8i.html

4.1 Flora

The field of procedural botany has a larger, more structured corpus than urbanprocedural modelling. The prevalent technology has been the L-system for thecreation of structured branches and leaves. This has been used to great effectin commercial projects. For example IDVs Speedtree allows for the proceduralcreation of a variety of foliage very quickly (figure 17), and provides an extensivelibrary of plant models to be customised.

In this chapter I will first review Lindermayers work with L-Systems, fol-lowed by a broader overview of the state of procedural botany.

4.1.1 The Algorithmic Beauty of Plants

Lindenmayer described his (L) systems in 1968 [34] as an attempt to describethe structure of plants using a formal grammar. The book The AlgorithmicBeauty of Plants[49] (TABOP) gives an introduction to his work, this sectionoutlines some of its main themes.

To model plant growth convincingly the traditional, sequential replacementof a Chomsky grammar was discarded in favour of a parallel, generational growthmodel. For example, given the non-terminals X & Y, and the terminals + & -,we might start with the initial string

initial:X

and apply our production rules. For example every instance of an X may be

17

replaced by the string (X XY), and similarly (Y -X+Y). This happens toevery symbol, once per iteration.

iteration 1: XYiteration 2: XY-X+Yiteration 3: XY-X+Y-XY+-X+Y

We can perform as many iterations as we wish, and observe the length ofthe string grow or shrink as the pattern progresses.

Finally the results are fed into a turtle that evaluates the string as a seriesof instructions (for example, X might be move forward, and - might be turnleft). This gives a graphical result.

The result from these systems is generally pleasing, and intented to show thegrowth and development of a plant as it grows. Each iteration is taken to bea time step. However the system is limited to repetitive structures, as TABOPobserves:

However, production application may also depend on the predecessors context.This effect is useful in simulating interactions between plant parts, due for

example to the flow of nutrients or hormones.[49]

This context sensitivity is very similar to the biological principle of induction(figure 8), and echos a move from Chomskys type 2 grammar to a type 1.

A notation is introduced to describe context sensitive production rules. Forexample to state that a b between an a and a c should be replaced with a d wewould use the notation:

a b c d

Together with the conventions that the number of iterations is n, the anglethat the turtle turns on a + or - symbol is , and Fd moves the turtle forwardd units, allows us to evaluate the grammar given in figure 19 to figure 20. TheL-system interpreter was written in Java, and the image rendered to a ScalableVector Graphics image.

These basic L-Systems are given various extensions to overcome certain lim-itations. TABOP gives the following modifications:

Context sensitive. As described above, symbol replacement can occurbased on the surrounding elements.

Stochastic. Each production rule is given a corresponding probability withwhich it occurs on each iteration.

3D. By using a three dimensional turtle, instead of the two dimensionalstandard, three dimensional structures can be created.

Stack operations. Push and pop stack operations, normally representedby the characters [ and ], save and restore the turtles position during

18

n=40, =22.5

#ignore:+-Fstarting string: F1F1F1

0 0 0 10 0 1 1[-F1F1]0 1 0 10 1 1 11 0 0 01 0 1 1F11 1 0 1? + ? -? - ? +

Figure 19: example 1.31,b from TABOP, with some small modifications.

Figure 20: A simple L-System I encoded form TABOP[49], example 1.31,b(figure 19) over 40 iterations.

evaluation. This gives better representation of the hierarchical nature oftree - when the turtle finishes drawing a flower, it can return to createmore branches.

19

Parametric L-systems. Here each symbol is given a set of parameters, anda replacement can only take place if a logical statement associated withthe parameters is true.

Table L-systems. These use a different table of production rules to simu-late changes in the environment, for example one for generating the plantin a winter (non flowering) state and summer (flowering) state.

Map L-systems. These are remarkably similar to Stinys Shape Gram-mars [55]. As shown in figure 21 a Map L-System operates on geometricinput and searches for sets of lines that match the production rule, andthen applies one of them. Because this system works directly on geometryit is unnecessary to use a turtle to evaluate the output of the system.

Figure 21: A Map-L-system from [49]. Using the two production rules (bottom,yellow), the initial rectangle (top left) can be subdivided into a tiling.

There are many additional extensions to L-systems in the literature [50][9].The problem with L-systems is that each extension attempts to make up forsome short coming of the previous models. Perhaps this shows L-systems pro-vide a valid framework, because it is so easily extended. However, from ansoftware engineering standpoint, when you see a model overloaded continuallyit points towards the original system being not sufficiently detailed.

It seems it may be preferable to model the two systems individually; L-systems seem to work well for the hierarchical structure common in plants, butare clumsy when simulating growth or environmental interaction. This is a viewshared by Hart in the introduction to [25]:

The most prominent of these models has been the L-system. In an effort tomake these models produce more realistic results, some have encoded the

environmental influences, such as geotropism (sagging branches), directly intothe L-system. While this allows the genetic model to include more detailedinformation about its reaction to external forces, it makes the model much

more complex and difficult to decypher.[25]

More recent work on L-system in environments have come to similar conclu-sions, that when representing a buildings facade as an L-System, a lot of workmust be done to change one particular window[35]. L-systems also represent

20

only one hierarchy of components. For example a grid of windows on a facadeis either defined in terms row-major or column-major. In this situation there isno requirement to specify a hierarchy, but the L-system enforces one.

As Chomsky points out[16], a grammar should reveal some insight into thestructure being examined. By the time several of the above extensions have beeninstigated, the language becomes a mess of concepts and corrections (figure 22),showing little more insight than if if the same structures where specified in anyother programming language.

Figure 22: Five rules from a parametrised, programmable stochastic L-System[9]

TABOP leaves the subject of L-systems, and continues to discuss widertopics, such as the phenomena of Phyllotaxis in plants. This is an interestingstudy in the limits of procedural modelling from form alone. When modellingnatural phenomena better (procedural) models can be built when we betterunderstand the causes of a given pattern or form.

Phyllotaxis is the name given to the spiral patterns of seeds that grow in asunflowers head, or the needles on a pine cone. Similar patterns occur in dif-ferent phases. There are several commonly observed modes, alternate, oppositeor spiral, shown in figure 23.

TABOP describes a model for the spiral phenomena, we simply measure thedesired angle of the mode of Phyllotaxis were interested in, and use a simpleformula:

21

Figure 23: a) opposite, b) alternate and c) Fibonacci spiral patterns ofPhyllotaxis[9]

for incrementing n for each point{ = n * angle //NB (correct angle is 137.5)radius = c * sqrt(n)}

This model only manages to represent one mode of Phyllotaxis, and althoughthere are similar models for the other modes, we cannot gradually change theparameters to switch the mode of the model, figure 24, without creating anunobserved pattern.

However, there are biological models [53], that when run as a simulationcorrectly produce all known modes of Phyllotaxis by varying one parameter,figure 25. However these are more expensive to compute than the simple for-mula.

4.1.2 Other Reading in Flora

There is a wide range research into algorithmic botany. Here I review a sampleof papers from the field.

The seminal paper that brought the realisation that mathematical modellingof biological phenomena was Turings A Chemical Basis for Morphogenesis[57].

By a careful treatment of diffusion formula, this paper shows how a feedbacksystem can create patterns from an originally uniform system, figure 26. Theseare chaotic & emergent, examples of form that occurs in systems posed forphase-change.

However, the impact of this paper on the field of graphics has been slight,with the emphasis on recreating, rather than understanding geometry.

Much of the work in procedural flora is a re-invention or reformulation ofLindenmayer Systems [34]. For example Kawaguchi [29] produced vibrant re-sults in 1982, figure 27, by combining tubes, tendrils, and spheres knots.Each chamber in the spiral is constructed as a skew cylindrical prism. Theseprovide growth in the forward direction and are repeated and scaled by a set

22

Figure 24: A range of different spiral generations with different values for alpha.Only one of these uses the correct angle found in nature - it is the spiral withthe best coverage (second from top, right). Looking at the diagram from adistance helps this pattern to become obvious. This image was generated fromjava code, via an SVG renderer.

Figure 25: The suspected patterns of auxin transport that lead to the phenom-ena of Phyllotaxis [53].

23

Figure 26: An example of a dappled pattern as resulting from a morphogensystem.[57]

ratio, a geometric progression. Knots are added to allow the spiral form to ro-tate and as branches to add additional tendrils. There is an adjustment factorto move child branches away from the parents. There isnt a formal grammarbehind this system, rather a successive application of rules. It would be easyto create these designs using a stochastic, 3D L-System, but this option is notevaluated.

Figure 27: Tendril plants[29].

The ever prolific Prusinkiewicz provides the backbone to the field of algo-rithmic research. In [50], a modification to a context sensitive, probabilisticL-system is presented that creates vegetation of a required shape, as for top-iary. By backtracking whenever a boundary is reached, figure 28, a realisticbranching structure is created in the interior, and sufficient foliage is shown onthe exterior.

The original premise behind L-systems was that they simulated the biologicalprocess of induction, and growth. However, the backtracking algorithm heregrows plants in a artificial sequence, very different to natural growth.

24

Figure 28: Top: An implicit surface in the shape of a dinosaur, and the corre-sponding topiary plant. Bottom Left: 2D results pruning to an oval. Bottomright: The process of exploring space, and returning the active-pointer to theprevious branch[50].

Prusinkiewicz and Mech develop the idea of environment interaction in L-systems in[43]. Two concurrent systems run in parallel, one for the plant andanother for the environment. The plant receives information from the envi-ronment, transports the information to the needed location and responds tothis information by changing the form. The environment perceives the plantsactions and simulates the environment (diffusion of chemicals or flow of light)and gives feedback to the plant. The development of this paper was this twoway communication between model and environment. The results from the theenvironmental feedback are returned as parameters to the L-system.

Recently Palubicki et al.[46] presents a accumulation of much of this workwith L-Systems, figure 29. By exploiting the increased computing power avail-able in modern computers, very complex models can be evaluated.

There is a substantial body of literature describing botany generation with-out L-systems. These systems are often more elegant and engineered towards thespecific problem being approached. While they could be recast as L-Systems,they give more insight as domain specific languages.

Greene[24] introduced the quad-tree as an acceleration mechanism to al-low fast lookups to allow the grown geometry to search for light, figure 30.Heliotropism (plants tracking the sun) is simulated by finding the brightest

25

Figure 29: L-systems can create complex environments when used on a largescale[46].

direction, and biasing growth towards that direction.

Figure 30: A model [24] of a house was imported into a voxel grid. The plantmodel was then biased to grow with respect to this model so that the branchesfollowed the roof shape and stayed clear of the doorway.

Once the structure of a plant has been determined by a grammar, it isnecessary to fill in the remaining detail in a consistent and connected way.Bloomenthal[13] gives an early solution to convert a line representation of a treeinto rendered geometry, figure 31. By fitting spline-controlled surfaces to thetree a well-fitting trunk mesh can be created. By x-raying tree bark a realisticbump map was generated, and outlines of leaves are converted to geometricalobjects and attached.

Reffye et al.[17] describe a time-based simulation of tree growth. The al-gorithm steps through time, adding new objects at the buds of branches, andsimulating growth for existing objects by rescaling them. The discrete simula-tion method allows for the simulation of disruptive events, such as trauma orpruning, and forces, such as gravity to be modelled without deviation from themain algorithm.

26

Figure 31: A model of a maple tree[13], note the close-fitting mesh and tex-tures, not normally obtainable from L-Systems turtle-based discrete geometricobjects.

Figure 32: A fir tree with trauma evident on the lower branches[17].

Hart et al.[25] continue the theme of physically based simulations for themodelling of trees. Different trees respond differently to gravity, there are twomain types -

27

Gymnosperms (such as Pine or Spruce trees). These trees support theirweight of their branches by adding extra material below the branches, thatis compression wood.

Angiosperms (such as Apple trees). These trees support their weight byadding additional material above the branches, causing it to be in tension,hence tension wood

These two types of growth produce different shaped branches and this modelsimulates that by widening the area of the trunk above/below the branch forGymnosperms/Angiosperms respectively.

The algorithm used by Hart consists of

1. Compute mass

2. Compute center of mass

3. Compute photosynthesis

4. Compute growth rates

5. Grow branches

Because there is a conflict between the strength of a tree (proportional toarea of trunk) and its weight (proportional to volume) this provides a naturalmechanism for the tree to stop growing.

Turning away from physical simulations, Runions et al.[51] use a sparse setof attractors to direct a growth system. The attractors are scattered, markinga preferred density map of the final tree. As the trees skeleton gets within aset proximity for each attractor, the attractor is removed. Once all attractorshave been reached, growth stops.

By positioning the attractors in different ways (uniformly through a volume,on the surface of a volume) using different volumes (cones, cylinders) a varietyof different plants can be created by an artist.

Figure 33: Space colonisation technique for tree generation. Left: final tree.Right: A sapling and a set of attractors.

28

4.2 Urban

The body of work on urban procedural modelling is expansive and disjoint.Unlike the modelling of flora, where the L-System is the key concept, no suchtechnique has appeared for Urban Modelling. Because urban modelling requiresapproximating the designers thought processes, it is generally a much morechallenging field, and most of the corpus resorts to what looks about right:

Voronoi points Pv are generated along an axis that approximately splits theblock into two halves. We have observed that for most queries vertices blocksthis axis corresponds closely to the central segments of the medial axis of the

block.[8]

Parts of this chapter are from my MEng Research.

4.2.1 Shape Grammars

Stiny and Gips created the concept of shape grammars in 1975, during his PhDat UCLA[55]. These grammars have been used in design systems for urban [54],and other domains (such as furniture [30]).

A shape grammar consists of a starting state, a set of transformation rulesand some termination rules. A transformation rule specifies a geometrical con-figuration that is matched against the current state, and a replacement for thatconfiguration, figure 35. Unlike an L-System, the rules are not applied in paral-lel, rather in a serial manner reminiscent of Chomsky grammars[16]. Termina-tion rules ensure that the evaluation terminates when a particular configurationis achieved.

Another difference between a shape grammar and an L-System is that ashape grammar is not deterministic, that is it doesnt always give the sameresult. The result depends on the choices of transformation rule made and thematching sections of geometry to be replaced. For example only one of rules 20and 21 from Stinys shape grammar analysis (figure 35) of Palladian villas[54]may be applied in one situation. Furthermore a transformation rule may matchan infinite number of situations, such as locating a pillar anywhere along a wall.This means that shape grammars must be guided through evaluation. In anideal grammar, every possible choice leads to a valid outcome.

A large variety of interesting forms can be created using shape grammars.Generally they are more powerful than L-systems, being able to create compli-cated forms, figure 34, but require guidance as to which rules should be applied.

4.2.2 Architectural Theory

There is a large body of work in architectural thoery. In general these are notvery helpful because the are not generative, that it is more suited to analysing,rather than generating geometry. A Pattern Language[5](APL) attempts to givethe rules for building a city, but because of lack of algorithmic detail, is onlyable to say whether a particular design complies with its theories or not. APLdescribes its language as:

The elements of this language are entities called patterns. Each patterndescribes a problem which occurs over and over again in our environment, andthen describes the core of the solution to that problem, in such a way that you

29

Figure 34: The complete Palladian villa floor plan [54].

Figure 35: Rules 20 and 21 from Stinys analysis of Palladian villas[54]. Thematching case on the left of each example is replaced with the geometry on theright hand side. In this example the central block can either be lengthened orshortened. The letter c is used as a marker to match later features.

can use this solution a million times over without ever doing it the same waytwice.[5]

And procedes to list rules for the construction of a city from the large-scaleto the small:

1. independent regions

2. major city structures

3. self governing community

4. networks between communities

5. neighborhood environment

6. local centres

7. streets or clusters of houses

8. work communities

9. local road and path network

10. public land

30

11. common land (smaller than public land)

12. social institutions, family houses

13. groups in society

14. local shops and gathering places

15. layout of a group of buildings

16. relation of each building to the environment

17. indoor/outdoor divide for each building

18. paths and spaces between buildings

19. interior floor plan of the house

20. use of home rooms

21. use of office/work rooms

22. rooms for different social groups

23. relationship of the room to the outdoor

24. garden plan

25. features of the room

26. features of walls and stairs

27. curiosities in the rooms

28. materials and building methodology

29. engineering/internal structure of building

30. structure of the the floor and walls

31. building doors and windows

32. other building details

33. materials for indoor finishing

34. outdoor finishing

35. furnishing

It is worth noting that this list does not represent a hierarchy, but rathergroupings of patterns important to architects. Some of the patterns of tangi-ble/implementable:

Pattern 159: Light on two sides of every Room...Locate each room so that ithas outdoor space outside it on at least two sides, and then place windows inthese outdoor walls so that natural light falls into every room from more than

one direction.[5]

31

While others are vague:

Pattern 10. Magic of the City...Put the magic of the city within reach ofeveryone in a metropolitan area. Do this by means of collective regional

policies which restrict the growth of downtown areas so strongly that no onedowntown can grow to serve more than 300,000 people. With this population

base, the downtowns will be between two and nine miles apart.[5]

However these patterns do not form a generative system, there is no algorithm,just a hierarchy of goals. Texts like APL would be most useful when evaluatinga generative procedural system. We can ask an artist - can you reliably createPattern 10 using our procedural system?.

As Hiller[27] notes, architectural theory books tend towards representing anidealised view of architecture.

[With reference to Architectural Theory books]...first, most have been stronglynormative, and weakly analytic, in that they have been too concerned to telldesigners how buildings and environments should be and too little concerned

with how they are.[27]

Because of this bias, and the lack of generative detail, much of the corpusof architectural theory books is hard to use for procedural modelling. Howeversome broad overviews can be useful.

One exception is the book How Buildings Learn[14], in this Bond describesthe way houses age, and how their appearance changes over time; Buildings thathave been used for a while end up looking very different to how they were built.

Figure 36: How Buildings Learn [14], front cover, left: houses New Orleans1857, right same houses 1993.

In old cities each house in an initially identical row will have had manydifferent modifications made to it by different owners through the years. Somehouses will have had similar alterations to do with current trends (such as PVCwindows), others will have unique changes by generations of rich or poor owners(such as telescopes and flag poles). Bond describes numerous examples of howthe floor plan of a house can over time grow.

Bond continues to give a description of the effect of wealth on house devel-opment. Rich home owners are more likely to make changes to their properties.Poorer home owners will not alter the appearance of their houses, and this maypreserve many historic features, such as has happened in the older parts of New

32

Orleans, figure 36. The end result is that the old buildings seen in Europeancities are a complex mess of features with underlying patterns. There are morelikely to be correlations between buildings close to each other. Such structuresare built to similar plans and similar materials and modifications that are madeto one house are likely to be applied to other local houses. For examples whenconservatories became popular they were added to old and new houses alike.Because of historic building protection laws uniform appearance is not corre-lated with the age of the property. This huge variety provides a challenge forcity generation. Again, this book is not generative, and only gives an idea ofwhat the outcome should be, rather than saying how to achieve it.

Mitchell [39] starts to describe this problem of computability and makes oneof the first attempts at describing a machine learning centric approach to pro-cedural modelling. He describes the huge potential search space for designingbuildings in real world architecture setting. Many parameters - from local build-ing codes, to the quantity of light a window would produce must be encodedand solved to produce a technically correct building. Trickier is the problem ofdesigning an algorithm with a sense of aesthetic beauty, which complicates thissearch space again. For example we may desire that windows across a buildingare different sizes in odd and even columns. Formulating a problem that is suf-ficiently constrained, without over constraining the problem is a tough problemitself, before solving this massive logic problem is considered. Mitchell suggestsa range of possibilities, from using a computer to design the buildings and hu-mans to verify them to the opposite as possibilities for making use of computersin architecture.

In [40] Mitchell describes the kind of first order logic predicates that may beused to describe architecture. Throughout this lengthy volume many conflictingapproaches for describing architecture in logical form are given. None of theseare concise enough to implement and the NP-complete nature of inferring factsfrom logic problems make this approach a little unrealistic. Mitchell does notcombine these different ways of viewing architecture into a single model.

4.2.3 City Modelling

Parish and Mullers 2001 paper [47] sparked a recent renewed interest in pro-cedural urban geometry in graphics research, well detailed by [59]. Parish usesa L-system to generate roads and basic buildings. This work formed the basisfor the foundation of the company Procedural Inc, and one of the few commer-cial procedural modelling tools, CityEngine. As figure 37 outlines, the systemuses number of processing steps to create a city. Population density and terrainmaps are given as input. Main roads are laid between the population cen-ters, and a selfsensitive constrained Lsystem builds a tighter network of smallerroads around these. Between the roads another Lsystem constructed simpleskyscrapers. This paper sees the introduction of mass-modelling, in which basicprimitive shapes are combined to give the outline of a building. Generally theresults, figure 38 were impressive.

By 2006 Muller et al. had advanced their work, replacing these undecoratedmodels with a sophisticated shape grammar [42], figure 39. The grammar uses L-system-like production rules to specify the situations in which a transformation

33

Figure 37: Overview by the author of Procedural modelling of cities[47].

Figure 38: Results of Procedural modelling of cities[47].

rule is applied. However the authors do not describe their algorithim as anL-system as it contains many other geometric constructs:

...and it is important to have the possibility to distinguish between absolutevalues (values that do not scale) and relative values (values that do

scale). [42]

By defining methods to subdivide surfaces, and to snap geometry to repeatinglocations, a wide range or features can be portrayed. For example a wall can bedivided to accommodate an integer number of windows, and the windows can bepositioned to avoid arbitrary roof lines. A large set of production rules are stillnecessary, but a wide range of intricate and realistic geometry can be created.Examples given involve skyscrapers and a reconstruction of Roman Pompeii.However the path through the morphospace (design space) is predetermined bythe grammar, only taking into account stochastic choices, or inputs from thedata-maps.

In [63], Wonka et al. introduce an attribute system that allows the design

34

Figure 39: Left: A mass-model of a complex building, Note the way the win-dows avoid the roof-lines. Right: the corresponding snap-lines used to positioncomponents.

space to be purposefully evaluated. A large library of components and a split(continuous subdivision) grammar a large design space is possible. A controlgrammar runs in parallel and distributes attributes, ready for the next iteration.For example every element may have a simplicity value, and if the user asksfor a simplistic building, this can be used to guide the evaluation of the splitgrammar. Other elements have attributes that ensure they are only used ifthere is sufficient geometric space. When all attributes are taken into account,there may be a tie-break between several rules. When such an arbitrary choicehas to be made, the choice is baked in until the end of the evaluation of thatparticular building. So if a window type is chosen in a particular building, it willbe used in all identical situations in the same building, providing consistancy ofdecisions.

A different pair of hierarchies are used by Finkenzeller [20] to describe thecourse and fine details of a building, figure 40. The course system starts witha floor plan - this comprises a concave set of abutting polygons that representsthe main building, a wing or a balcony. The second hierarchy of geometry isfor the style, the finer components, such as wall subdivisions or windows/doors.Cornices (decorative moulding strips) are extruded from a sequence of pathsand curves. The different components of the windows and door are created bya multitude of cornice and frame elements.

Another model for the generation of new urban environments is to extrap-olate existing data. Aliaga et al.[8] generate new areas of aerial photographsby extrapolating from existing road networks and photographs. To create newareas of the city (fragment synthesis), the road network is filled in in an hier-archical order of road size, as in [47]. The direction is derived from a statisticalwalk between the specified points, based on information from the example data(average area of blocks in the vicinity, road type/hierarchy or number of roadsat a junction). Styles are attributed to the new points/intersections from aclassification system based on the example model. Groups of similar (types ofroad hierarchy in an area, spacial frequency) points are grouped together usinga quadtree to maintain adjacency. The blocks (parcels) are then split intoplots using the Voronoi tessellation of points scattered along the longest axis ofthe block.

Each block is then filled in by morphing the best-matching block in the

35

Figure 40: Two hierarchies give impressive detail, without exploding the com-plexity of the grammars [20].

example model to fit the space in the synthesised road network. This is imple-mented by subdividing the quad and using an affine texture transform. Becauseof the large number of blocks in a typical input image, there is likely to be agood match (so not so much stretch or distortion) in the synthesised image.However it is a weakness of the system, that it is less suitable for use with smallquantities of input data. The results are convincing for large sets of input data,figure 41, but largely statistically derived from certain properties of the existinggeometry.

Figure 41: The fragment on the left, and GIS data is applied and extrapolatedto produce the new geometry on the right.[8]

Once a model has produced a building, it is useful if certain elements canbe manually edited, giving artists direct control of the computational output.As Lipp et al. demonstrate[36] it is possible to add meta-data to an L-system,to specify exceptions, figure 42. Furthermore, by representing these features as

36

semantic patches that are laid on top of an L-system, the system can be re-evaluated, and the artist-defined modifications are still present. Two conceptsare defined to help a user edit find the feature desired -

exact instance locator - the decision path through the underlying gram-mar is stored. By remembering the element number at each level in thehierarchy (e.g. the 34th house, 3rd floor, 2nd window) it is fairly robustto changes.

semantic instance locator - this is the technique for selecting more abstractconstructions, e.g. columns of windows. Additional meta-data is added tothe grammar, so that split or repeat operations tag each production withthe given semantic property. Between this system, and an intuitive userinterface, editing existing grammars is a faster process, that involves lessprogramming.

Figure 42: Left: The red controls allow interactive window resizing[36]. Right:The bricked-up window can be identified in an L-system generated facade.

4.2.4 Broader Topics

There are a large number of scattered topics that are related to urban proce-dural modelling. Some subjects include castle construction, entire generativemodelling languages, real time rendering of cityscapes and growing cities overtime. Here I will outline a few of these directions.

The subject of stitching different procedural modelling primitives together,and transferring data between geometric routines is studied by Havemann in [26].A stack-based Generative Modelling Language (GML) is introduced in whichthe output of different geometric, or otherwise, routines can be used as inputto another routine in much the same manner as Postscript. Each transfor-mation can access the output of any previous stage removing a bottleneck ofstrict-procedural systems. The primitives themselves are defined using bound-ary representation and Euler macros. Sharp faces are just triangulated butsmooth surfaces act as CatmullClark subdivision surfaces.

Geographic Information System (GIS) databases present a standard inputfile for working with cartographic data. Rendering large landscapes from these

37

files is approached by [15]. This paper details the task of rendering 3D scenesin real time from a vector (rather than the normal raster) source. Because theinput data is in vector form, it is easy to define some objects (rivers, roads) assmooth splines.

An octree stores each patch of land, with larger octs being used for furtheraway areas. The maps (texture and displacement) for the leaves of the octreecan be stored onto the GPU to increase speed. Its also possible to edit vectorson the fly - by only re-processing the elements that change with each edit itspossible to edit spline features in real time. Standard techniques like frustumculling and LOD allow real time rendering of these massive terrains.

Figure 43: Screen captures from the real time display of GIS data [15].

The subject of Weber et al.s paper [62] is the growth of cities over time. It isbased on simulation for the numerical aspects, rather than the pure estimation& guesswork for the geometrical aspects. Similar to Mullers earlier work [47]the input to this system is a range of maps (height, water, forest...) and aninitial starting state (e.g. a single street). Subdivision occurs in the followingmanner:

Major Streets Quarters Minor Streets Block Lots Building EnvelopesIn contrast to the earlier work, the streets in the simulation are grown for

each time step, and must be generated in chronological order, thus L-systemscannot be used. This allows the user to influence the street pattern interactively.

Standard traffic and land-use simulating techniques from urban planners areused to predict the growth of the city. The output of the model is comparedwith the real world growth of Las-Vegas. Because the model is interactive,subtle changes can be made at different points in time to keep the growth on

38

Figure 44: The growth of a city over time [62].

track. The final result is quite similar to current-day Vegas, with few changes,given that so much theory is involved, figure 44

Ali et al. address the problem of rendering massive cities, once weve pro-cedurally created them in [6]. Their approach is to use ray casting techniquesagainst a displacement mapped surface for each wall of each house. To cre-ate a fast displacement-map intersection mechanism, the orthogonality of thegenerated geometry is exploited on the GPU. Cubes can be matched to emptyvolumes of space to allow fewer samples when casting each ray. By exploitingthe expected orthogonality, a lossless matrix-based algorithm is presented thatis carefully tailored for facades. The results of these techniques are shown to beimplementable on a GPU.

Gerth et al. present an example [23] of a domain specific tool for non-expertusers to create castles. GML is again used to describe the interactions of theprimitives. Users are able to draw the outlines of buildings and select the pointswhich walls should be built though. This presents a simple interface that cancreate quite complicated castles. The domain specific information about castlescan be embedded in the system, gives good results, figure 45.

Figure 45: GML modelled castle [23].

39

4.2.5 Reverse Procedural Techniques

Reverse engineering a model, or inverse proceduralization is the method of build-ing a procedural model from a single example. This process gives many insightsas to how a general procedural modelling package could work.

For example, given the polygon mesh of a house, we may want to infer thatthe windows are a repeated elements on a certain grid. This lets us compressthe geometry, correct for errors, and build procedural models.

To extract a building grammar from a set of photographs, [7] takes a setof several photographs and a simple 3D model. The input mesh is manuallyconstructed, and aligned to the photos using a semi-automatic process. Afterorthogonal subdivision the layout of each floor is represented by strings repre-senting blocks. Analysis on theses strings with a few simple rules can producea useful grammar. Then several assumptions are made about the grammar ofa house. For example that there is a single ground floor, a single top floor, andany floors between the two are repeated when the facade is re-sized. For sim-ple examples, figure 46, reasonable results are achieved, however the grammaranalysis technique cannot work on more complicated designs.

Figure 46: A photo (left) is converted to a photograph, from which a gram-mar is extracted, modifications to the floor plan are made, and grammar thenregenerates the building (middle bottom), before being stitched back into theoriginal photograph (right).

Creating geometry from existing geometry is a hard problem, because youneed to understand the structure of the geometry on many different scales.Merrell [37] [38] makes a very successful attempt at only copying local features,using a very similar technique to 2D texture synthesis, figure 47. The inputexample geometry is subdivided to blocks. These are analysed to find allowableneighbours in each axis direction. To evaluate the model a backtracking searchis performed to find configurations of allowable (previously observed) blocks.As with texture synthesis the output can be constrained to volumes force thestructure to grow in a specific location. There is no gross control of the systemand the stochastic element cannot be controlled, but could be guided by thelikes of a mass model. It can construct unexpected and new features, such asthe arches within arches in figure 47.

Another direction borrowed from research in textures was studied by Cabralet al. in extending the concept of content aware image resizing [11] to polygongeometry. A geometric structure is pre-processed to create handles that can berepositioned in real time to deform the building, figure 48.

An input model is pre-processed into a set of constraints and a solution.Then the model can be reshaped using a set of point handles, while the con-

40

Figure 47: Top Left: The input models. Right: The result of model synthe-sis [38]

straints are satisfied to position each vertex in real time. The geometry isanalysed and a set of hard and soft constraints are entered into a matrix andsolved using least-squares. At interactive speeds, the users inputs are projectedusing the found solution and applied - if the result is degenerate, the users editis rejected.

While this method cannot create new geometry, it provides a fast an in-teractive solution for resizing rectilinear meshes, and infers a model using thematrix-inverse technique.

Figure 48:

Li et al. examine reverse engineering feature trees from a given model [33].This paper takes the approach that you have a 3D mesh and want to recreatethe constructive-solid-geometry primitives and operations used to create thatmesh. These operations are presented in a tree (similar to a parse tree) tospecify their inputs, outputs and ordering.

The algorithm identifies recoverable vertices, edges and faces that the origi-nal mesh hints at. These might be external to the volume defined by the mesh,or internal and are identified by intersecting existing features, such as edges orfaces.

One of the hardest parts of working with raw-geometry is determining wheresymmetries lie. Because there is no construction intent, all symmetry informa-tion is lost. Mitra et al. [41] recover this data by identifying many pairs ofsymmetrical points. The algorithm identifies the prevalent symmetry trans-forms by sampling many symmetric pairs of points, and clustering the lines ofsymmetry to to reveal the strongest patterns. By quickly rejecting pairs biasedon the local features, the number of points to compare is reduced by an orderof magnitude.

Mean shift-clustering is used to merge nearby points. The Euclidean sym-metry space used is 7 dimensional. Clustering is needed because noisy input

41

mesh is rarely perfectly symmetrical. Each clustered set of points indicates onepossible symmetry in the model.

Convincing results, such as the decimation of the castle mesh, figure 49 to14% of its original size (transparent boxes are removed data) make this aninteresting technique.

Figure 49: The found symmetries in a model of a castle [41]

Mitra later extends his work to find patterns expressed by two combinedtransforms [48] by searching for lattices in a transform space. Some of the typesof pattern detected are shown in figure 50.

Figure 50: Different two-transform patterns [48].

5 Straight Skeleton

The straight skeleton is a geometrical construction that I have become interestedin because of its applications to urban modelling, but also its ability to emulatethe expansion of an arbitrary polygon, in ways very similar to biological growth.

While exploring the possibilities for roof generation I came across a paper [3]from Franz Aurenhammer explaining the issues generating a typical hatchedroof from an arbitrary floor plan. This problem, finding the straight skeleton,is more complex than it initially appears. The straight skeleton is a mathemat-ical structure that is defined by tracing the corners shrinking of a 2D polygon(figure 51). The best solution found runs sub-quadratic time O(n17/11+e [18],given n input vertices and an arbitrarily small constant e, which is traded offfor space. While Eppstein and Erickson describe this fast approach, the detailsof the algorithm are best described by Felkel and Stepan [19], who provide asensible route for implementation.

Laycock and Days paper [44] investigates how to use the straight skeletonto produce different types of roof edges, from Mansard to Gable. By adjustingthe result of the straight skeleton after it has run, they show that it is possi-ble to produce a wide range of roof styles. My initial implementation of theskeleton revealed several modifications that might make the skeleton a powerfulprocedural primitive:

42

A more interesting skeleton may be constructed if each edge is assigneddifferent slope. This could be used to create gabled ends of roofs withoutLaycock and Days messy alteration after the skeleton algorithm has run.For example the slope of a roof may be changed at any time to create say avertical wall, followed by an overhang, followed by a roof. This constructis known as the weighted straight skeleton (WSS). [18]

The skeleton can be truncated at any height during its construction, leav-ing a bevelled object, this may be used as a wall if the edges slope isnear-vertical or for constructing a window or mitered door frame.

It should be possible to construct skeletons from non-planar polygons. Forexample the walls and roof of a house on a hillside.

5.1 Algorithm

In my Masters Thesis I was only able to create a crude straight skeleton imple-mentation. What follows now is an explanation of a more careful algorithm Ihave developed from Felkels description to include weights per edge.

It contributes a method for dealing with multiple co-sited split events in afloating point environment, as well as a resolution strategy for horizontal roofedges, this leads to a new degenerate case in which the weighted straight skeletonis ambiguous.

The straight skeleton takes a simple polygon as input and outputs a graphconnecting the input vertices, figure 51. Strictly the skeleton is a two dimen-sional construction, however here I regard it as a three dimensional roof -given the floorplan of a house, the skeleton is the gutters and crest lines of aroof when viewed from above.

5.1.1 Overview

The algorithm uses a sweep plane (figure 52) - this is always parallel to theplane in which in the input polygon is embedded. It starts at height zerofrom the input polygon, and ascends towards the top of the roof. Intersectionsbetween edge-planes, or sides of the roof, are considered in height order as thesweep plane passes them. The edge-planes (sides of the roof) are defined bythe gradient associated with each edge of the input polygon. In the unweightedcase these are all assumed to be the same positive gradient, whose value has noeffect on the output.

There are intersections that occur between faces that arent valid, thesemust be ignored as in figure 53. After all collisions have been accounted for, thealgorithm is complete and will have output the edges of the WSS.

5.1.2 Data Structure

The primary structure used is a set of corners, figure 54. Each corner haspointers to:

the adjacent (next and previous) input edges. I assume a counter clockwiseordering of points.

the adjacent corners.

43

Figure 51: An example of the 2D unweighted skeleton (thin black lines) of aninput polygon (thick black lines), and the calculated faces (shades of grey).

Via the normal computer graphics convention, holes are represented by back-wards (clockwise) loops of corners. There is a separate list of all input edgesEachedge has:

a list of corners that reference it. a list of the inactive corners that form its boundary.Additionally there are a set of current edges at the height of the sweep plane.

Each edge may be referenced by more than one pair of corners. This happensin the case that two parallel roof edges exist.

As the algorithm executes there is one loop of corner pointers for every areain which the roof volume that intersects the current sweep plane. For exampleif a skeleton produces a roof with two peaks, there would be two loops of cornerpoints at some point during execution.

5.1.3 Collisions Between Three Edges

As the sweep plane ascends the edge-planes may collide to a point, and we haveto modify the data structures accordingly. Often we will remove old corners,insert new corners or add output edges to the polygon. The ways in which edgescan collide are shown in figure 55.

There are also two other important cases to understand. Firstly, where thereis only one plane at a point, it is just a plane(!). Secondly, when there are two,it forms only a line in space. These arent registered as events and no action istaken. Figure 55 1,2 & 3 covered all possible topologies.

44

Figure 52: An overview of the straight skeleton algorithm processing an inputpolygon.

Each edge-plane cannot collide with just any old other edge-plane. Epp-stein [18], gives more ideas on a fast approach, but concentrating on correctnesswe can start by noting that when two planes collide it forms an edge. We needthree planes to collide to form a point; Before three planes can collide, two ofthese planes must already be adjacent. Using the above list of possible inter-sections, we can see we should only be testing for collisions between pairs ofadjacent edges and one other edge.

Figure 56 gives an overview of how the data structure is updated for eachthree plane case. These three types of collisions are sufficient to calculate thestraight skeleton of almost all random input polygons. The other cases aredegenerate and are discussed in the next section. It is unfortunate that mostcommon roof shapes are classified as degenerate as many input edges are paralleland include symmetrical shapes that cause more than three planes to meet ata point.

5.1.4 Ensuring Collisions Only Occur On Faces

One way to view the algorithm, is that after the first event, we calculate a newinput polygon at the given height and start again - so we might see the algorithmas executing on a pile of contours, one for every event, figure 57.

There are two obvious places to perform this check:

45

Figure 53: The intersections between edge-planes bcd and abd should be ignored.

Figure 54: The pointer structure between live corners and edges.

Store a list of events between polygons - After an event, find new eventsand remove old events involving each edge or bisector thats changed.

Store a list of events between planes (planes dont have bounds) and beforeevaluating an event, ensure it occurs within all three faces. This is thesimpler method I implemented.

One trick here (as Felkel[19] describes) is to realize that if two adjacent

46

Figure 55: In 1)ab two adjacent edges meet another edge in its centre, 2) showsthree consecutive edges colliding in 3)ab we see a variation where three consec-utive edges that form a loop collide.

Figure 56: An overview of the data structure manipulation that occurs at eachtype of event. See 60 for detailed explanation of the general case.

edges are involved, these two edges overlap and no collision test is needed. Theseadjacent edges form Felkels[19] bisectors. As Eppstein[18] explains, reduces this

47

Figure 57: So far the collisions have been presented as infinite planes, howeverwe are really colliding a set of three sided polygons or faces. For example theback edge (bold, 2) in the above figure changes its topology (red, 2) after someof the events. After event b the face belongs to two volumes, and hence gives twopolygons to collide against. This means that for all of the three faces involvedin a collision we must ensure that a collision occurs within its bounds.

to a ray-casting problem against triangles that has very fast known solutions.Each of the three types of collisions involves one or more pairs of adjacentedge-planes colliding. The faces that are intersected are sometimes triangles,but sometimes they are unbounded triangles, akin to quads with infinite areas.This occurs when the bisectors do not collide.

Floating point calculations sometimes miss collisions that we expect, topo-logically, to happen. However we can increase the chances of getting a topologi-cally sound result if we increase the margin (epsilon-error range) until we catchall collisions. There are two places where we need to be careful - in checkingthat a collision really happens on an edges face and in finding events that hap-pen at the same height or in the same place. I use a cylinder shaped volumeto accumulate events that occur at a given height range and within a certaindistance to a centre point58, similar to work in [31].

Figure 58: Given a set of detected collision points (circles), they should becombined using an robust-epsilon technique for height, 2 and radius, 3

48

5.1.5 Many Edges Colliding

In a degenerate case, more than three planes can meet at one point. Thishappens in the center of regular polygons, as well as examples such as figure 59,where five or eight faces meet single points. Here I present a resolution algorithmthat processes pairs of adjacent corners, for any number of edges meeting at asingle point.

Figure 59: Many buildings footprints lead to more than three edge-planes col-liding at one point. In this example there are two points where 5 and 8 edgescollide at once.

It is important that these occasions are detected and treated correctly, as weassume that the linked-list of corners and their locations form a simple polygon(when projected down onto the input plane) at all times. One of my contribu-tions here is generalising some rules for handling point collisions, whether threeedge-planes, or more, meet at a point.

The collision resolution technique takes a set of planes that meet at a pointas input. It processes these into lists of chains of adjacent corners that referencethese edges. While constructing these chains we perform the face collisions todetermine which (if any) corner references the correct portion of the edge.

Chains consist of a counter-clockwise ordered lists of edges meeting at apoint. If the edges are adjacent during the collision, they are in the same chain.

We then perform the pointer manipulations intra-chain (figure 60) and inter-cha(figure 61).

49

In figure 60, a we see the input structure - a chain of edges abc and theirassociated corners (dark blue circles). We now process each pair of edges thatare adjacent (orange lines) as in b and c:

We remove each of these corners from the master-list. (note that inter-corner pointers, green arrows, dont need to be adjusted).

We create output edges to the skeleton.

Then we process the non-adjacent edge pairs as in figure 60, d, adding in a newinput corner and finally adjusting the inter-corner pointers, (cutting out theinternal corners). However non-adjacent edge pairs take a little more thoughtas multiple chains can collide at one point.

We then procede to process between the chains, figure 61, that is inter-chainpointer updates. We still process corners by pairs, but this time we process thethree pairs that are non-adjacent. We order these around the collision point(in the unweighted case this can be done by the angle any corner on the chainmakes with the collision point, but see later comments on the weighted case),and process the pair made by first corner in the first chain, with the last cornerof the next.

Then we process the non-adjacent edge pairs as in figure 61 d, adding in anew input corner and finally adjusting the inter-corner pointers, (cutting out theinternal corners). However non-adjacent edge pairs take a little more thoughtas multiple chains can collide at one point.

For each of these pairs we add one additional corner that is adjacent to eachedge. This corner is at the location of the collision, and cuts out the inter-chaincorners from any edge loops.

5.1.6 Horizontal Output Edges

When evaluating rectilinear shapes, such as the cross-shape in figure 59, we oftensee horizontal skeleton lines. This is another degenerate case that we often seeat the same time as the > 3 planes at a point case.

We can note that before processing events at one height, there should be noadjacent edges that form a horizontal edge in the skeleton. We can see that thisis true because there must also be a collision at the other end of the horizontaledge that should have already been processed and removed the edge. Also thatafter processing there should be no such edges.

As all events co-heighted occur simultaneously there is no interaction be-tween events and the algorithm is deterministic. That is it produces the sameresult, whatever order the collisions are in, so we let them occur in an arbitraryorder.

After one event occurs we may be in the situation where two consecutive &adjacent edges form a horizontal edge. Because this edge has no height, thereis no way to determine which events it causes can occur first, figure 62.

For this reason all face-collision checks for events at one height are performedbefore any are evaluated. This is done by caching a list of chains. This bringswith it the problem that the corner used in a chain may have been superseded,or cut off from its intended collision (see figure below) by an earlier event atthe same height.

50

Figure 60: Intra chain pointer manipulations.

Figure 61: Inter chain pointer manipulations.

The resolution strategy here depends on the chain type. If the chain hasmore than one corner in it, it represents the intersection of two faces, so therewill always be a corner lying on this intersection whose adjacencies will lead tothe correct corner. If the chain is of length 1, we must resort to more geometriccalculations to project all referenced corners onto the input edge, and choosethe preceding corner.

As figure 63 shows, this is quite and involved procedure. Faces a,b,c,d ande collide at point x. We find their positions in the given volumes by locating

51

Figure 62: In the above figure, there is a collision at x, then at the same height,faces a,b,c &d collide at y. Face a should be rejected because were outside thebisector. However as the edge forms a horizontal skeleton line, the bisector ispoorly defined (green boundaries), so we would have to accept that all four facescollide, resulting in deformed output.

the corresponding corners. As co-heighted events occur in an arbitrary order,by the time figure 2 occurs the corners (u & v) are no longer valid.

In the case of u (or any chain with 2+ edges), we can can use the follow-ing corners previous corner to find u. This is because the corner referencestwo edges used in a collision, so must take part it (and horizontal events areindependent - they cant effect each others edges or validity).

This trick wont work to find v as it is part of a chain of length 1. Weproject all corners that reference the edge onto the collision line. We then usethe point (v) that is just before the projected collision as the previous corner.We ignore any points that reference the edge before the found corner. This isvalid as all these points lie on the edges plane, and all the points occur at thesame height, so their order is defined by their projection.

52

Figure 63:

5.1.7 Negative edge gradients

By changing the weight or gradient of an edge, we can drastically change theoutput shape. There is even the possibility of giving the edges a negative (out-wards) gradient, to create overhangs.

No modifications are required to the given algorithm to allow these gradients.It is interesting to note that negating the gradient, figure 64 produces the

same shape as reversing the polarity (loop direction) of the input, but with adifferent final meaning - no longer is the result the top of the roof-shape (asurface whose normals point upwards), it becomes the ceiling of the buildingfrom the inside (a surface whose normals point downwards).

When edges have negative gradients, it becomes much harder to guaranteetermination of the skeleton; That is it may define an infinite volume. Sometimesit will terminate, others not. Determining which polygons will terminate andwhich wont would be a good future research direction. Of course it is alwayspossible to specify an arbitrarily high horizontal plane, which terminates thealgorithm.

5.1.8 Ambiguous Weighted Skeleton

If two skeleton edge-planes are parallel and have the same direction, the inter-section of their planes with a third is no longer a point (it is either a plane, aline or not an intersection at all). This leaves the WSS ambiguously defined.This is one of my contributions. This condition can occur in two places:

the input polygon

53

Figure 64: The effect of reversing the polarity on a collision.

when intermediate edges are removed to being two parallel edges together

Figure 65: 1) an event in the construction of a WSS that is ambiguous. 2,3)alternative resolution strategies that minimise or maximise the enclosed volume.

When two roof-planes of parallel edges collide, they form a horizontal bi-sector. That is, the line of collision is parallel to the input polygon. If thetwo edges have different directions then this is a roof ridge and the algorithmproceeds as normal, assured that at the opposite end of the ridge the two roofridges will collide again. However if the directions are the same then we areonly able to obtain a degenerate bisector, figure 65. The ambiguity arises in notknowing which surface gives way to which, because we cannot determine usingthe horizontal bisector.

This is a mathematical ambiguity arising from a singularity in the weightedcase, and unrelated to issues of robustness.

54

Figure 66: Work done outside my PhD. Two identical bay windows that leadto the same ambiguous case. The one on the left is resolved using the volumeminimising technique, the one on the right is volume maximising.

Depending on the configuration as the edges approach parallel, the resultis quite different, and so the limiting case is of little help when resolving thesesituations. Because of the utility of a robust straight skeleton, this is an impor-tant situation to define. We resolve the situation by selecting one roof-plane toreplace the others in the corner data structure, as in figure 66. This is the pri-mary reason for the vertical clustering outlined earlier - we need to know whichambiguous events must be treated at the same time, to preserve a meaningfultopology.

Aichholzer et al.[3] do not describe the weighted skeleton, and so do notcome across this situation in their text. Im not aware of a description of theweighted skeleton that avoids this ambiguity. However Eppstein [12] describesthe equivalent case in three dimensions.

5.1.9 Reconstructing Faces

Once the edges of the skeleton are known, it is desirable to be able to definefaces (we might want to tile our roof). There is one edge for each face. Bykeeping a list of which corners are used by which edges, we are able to traversereconstruct the face, figure 67. We traverse the edge map once for each inputedge, following the complete graph created by the subset of the output edgesthat are associated with an input edge.

5.2 Results

The results here are presented for completeness. They were generated fromwork that did not take place under my PhD time quota.

This section has been censored by the Author. In lieu he gives these prettypictures (figure 68, 69).

55

Figure 67: Two cases of face reconstruction by following the map of edges, tocorners (orange dots) referenced by the input edge (orange line). The outputpolygon (yellow) is another simple, monotonic, polygon.

6 Conclusions

In general Procedural Modelling is still a loosely defined field, it shows muchpromise, but no single system of operation has emerged.

Procedural botany is a strong subject with solid commercial uses, meanwhileprocedural urban studies are just starting to make progress, and very little workhas been done towards procedural fauna.

Biologically inspired techniques are robust, but hard to control. They arealso expensive to evaluate. Theyve shown us that attempting to engineeringalgorithms that imitate phenomena, such as Phyllotaxis, isnt as good as un-derstanding the phenomena directly. However when dealing with man-madegeometry, it is too complicated to model the thought processes and decisionsthat are made, so sometimes there are no options, except to pick an algorithmthat gives a close result.

There are some discrete geometric algorithms, such as the straight skele-ton and Voronoi tessellation that may make it affordable to create biologicallyinspired models.

Architectural theory is at best analytical, rather than generative. Littleuseful work in generative architectural studies has been performed. The bestgenerative work is still coming out of the computer science community. Howeverarchitectural theory may be useful for evaluating any results. It is easy to checkthat a solution