Delft University of Technology ColorTracker

Delft University of Technology

ColorTracker

Holzheu, Stefanie; Lee, Sang

Publication date2016Document VersionFinal published versionPublished inProceedings of the 34th eCAADe Conference

Citation (APA)Holzheu, S., & Lee, S. (2016). ColorTracker. In A. Herneoja, T. Österlund, & P. Markkanen (Eds.),Proceedings of the 34th eCAADe Conference: Complexity & Simplicity (Vol. 2, pp. 209-216). eCAADe.

Important noteTo cite this publication, please use the final published version (if applicable).Please check the document version above.

CopyrightOther than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consentof the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Takedown policyPlease contact us and provide details if you believe this document breaches copyrights.We will remove access to the work immediately and investigate your claim.

This work is downloaded from Delft University of Technology.For technical reasons the number of authors shown on this cover page is limited to a maximum of 10.

ColorTrackerStefanie Holzheu1, Sang Lee21Aleatorix 2Delft University of Technology & [email protected] [email protected]

With the work-in-progress research project ColorTracker we explore color as aformal design tool. This project-based paper describes a novel softwareapplication that processes color composition of a place and transcribes the datainto three-dimensional geometries for architectural design. The researchcomprises two parallel trajectories: a theoretical survey and the softwareapplication design. The theoretical survey presents the historical background ofcolor. The project-based research seeks to develop digital methods andtechniques that analyze the color compositions of the environment. Subsequentlythe objective is a novel application software for smart mobile devices in order todemonstrate the potentials of examining the color composition and chromaticparameters of a given environment and how it can contribute to the design.

Keywords: color, architecture, urbanism, tracking, form-generation

As the cities become denser, competition for visibil-ity between buildings and businesses increase. Withdensification,wealsofindmoreandmore spaces thatfall out of the desirable visual perception. Anothercompelling problem stems from the leftover or in-between urban places where no one cares to go. Onpaper, these places may not be seen problematic.But from the pedestrians' point of view, they maystand out as dark spots that appear undesirable andthus avoided. This research is motivated by the de-sire to use new encoding technology to translate thecontextual color combinations for architectural andurban design. ColorTracker (hereafter CT) envisionsa unique tool to augment the simulation and devel-opment of chromatic formal composition for archi-tectural design.

Digital technologies have greatly augmented ar-chitectural work. They help increase quality and pre-cision, as well as efficiency and performance. With

the computer technologies in architecture and de-sign becoming increasingly sophisticated, the disci-pline has transitioned from simply emphasizing pro-ductivity toward generative design experimentationand simulation. Now we use digital tools and tech-niques to simulate, explore, organize, and solveprob-lems with enhanced visual or structural complexity.With the ongoing parametric design research projectColorTracker we aim to explore one specific elementof environmental variables in architectural and urbandesign: Color.

With the preceding points in mind, this on-going research project consists of two continuousphases: the theory-based research and the project-based one. While the theory-based research aimsto establish the conceptual framework, the project-based research, based on the scientific yet theoreti-cal premises of the first phase, directly engages theadvanced inquiries in software development and de-

DESIGN TOOLS | Applications - Volume 2 - eCAADe 34 | 209

sign application. In the following section, thoughnota comprehensive summary, we will start with illus-trating the complexity of the historical precedents oncolor research.

TheoryTwo people might agree about the color of the ob-jects surrounding them but the perception of coloris actually a highly subjective experience. Physio-logical as well as cultural factors play an importantrole. The potential disjunction between the humanperception of color and the physical composition ofan object that exhibits color has been argued sincethe ancient Greek philosophy. In "Color for the Sci-ences," Jan J. Koenderink explains the dispute as fol-lows: "The pessimistic notion that colors are 'meremental paint' andhaveno relation to thephysical andchemical constitution of things at all is popular in sci-ence and (especially) in philosophy, but it has no ba-sis in fact" (Quoted inKuehni 2013:3). Thus, color con-cerns a large body of research ranging from naturalsciences, to engineering and to metaphysics.

Definition. Color: Attribute of visual perception con-sisting of any combination of chromatic and achro-matic content. This attribute can be described by chro-matic color names such as yellow or brown, red, pink,green, blue purple, etc. or by achromatic color namessuch as white, gray, black, etc., and qualified by bright,dim, light, dark, etc., or by combinations of such names(Kuehni 2013:3).

Color Values. The modern understanding of colorbegan with Isaac Newton. His observation of lightpassing through a prism led to the discovery thatwhite light is a mixture of distinct color rays. New-ton also states, "For the rays, to speak properly, are notcolored. In them there is nothing else than a certainpower and disposition to stir up a sensation of this orthat color" (Quoted in Kuehni 2013:3).

Thomas Young (1773-1829) most notably con-tributed the wave theory of light and brought in-sight into the nature of light. He related color towavelength and calculated the approximate wave-lengths of the seven colors Newton recognized. Fur-

thermore, YoungandHermannvonHelmholtz (1821-1894) postulated that the human eye has three dif-ferent types of color receptors with different spec-tral sensitivities, specifying that those photorecep-tors (cones) roughly correspond to red, green andblue (RGB). Based on this trichromatic vision theorythe RGB color spacewas developed in the 1920. Sub-sequent experiments proved that the RGB primariescould indeed match all visual colors within a certainrange of color spectrum. To this day we use the RGBcolor-space model and represent colors in terms ofthe tri-stimulus values.

Kurt Nassau (1927-2010) contributed to howmatter alters the composition of light and causes di-verse effects of color. He describes fourteen cate-gory of causes, "[...] four dealing with geometry andphysical objects, and those remaining dealingwith var-ious effects involving electrons in atoms ormolecules ofmaterials and causing absorption or emission of lightat selected wavebands" (Kuehni 2013:4). Nassau alsostates that " [...] perceived color is merely the eyes'measure and the brain's interpretation of the domi-nantwavelengthor frequencyor energyof a lightwave"(Byrne & Hilbert 1997:3).

Cultural Preferences and Color Connotations.Color conveys information andaffects our perceptionand response to our environment and what we see.Depending on the environmental circumstances onecolor may convey different meaning from one placeto the other. A color seen alone might hold a differ-ent meaning than when it is seen alongwith anothercolor. When working with color we also have to rec-ognize the cultural, personal and emotional disposi-tion colors carry. Colors have different meaning indifferent cultures for example: in Russia the color redis associated with communism while in South Africait is the color of mourning. Likewise, the color blue inIndia is associated with Krishna while in the Westernculture in general it is seen as a cool, masculine color(Livingston 2014:133).

Goethe conducted one of the earliest formal ex-plorations of color theory. Derived by the poet's in-tuition (later corroborated in part by science) he pro-

210 | eCAADe 34 - DESIGN TOOLS | Applications - Volume 2

vided the first catalogueof howcolor is perceivedun-der different circumstances. Goethe observes:

"Green: If yellow and blue, which we consider asthe most fundamental and simple colors, are united asthey first appear, in the first state of their action, thecolor which we call green is the result. The eye experi-ences a distinctly grateful impression from this color. Ifthe two elementary colors are mixed in perfect equal-ity so that neither predominates, the eye and the mindrepose on the result of this junction as upon a simplecolor. The beholder has neither the wish nor the powerto imagine a state beyond it. Hence for rooms to live inconstantly, the green color is most generally selected"(Goethe 1840:316).

Following Goethe's lead, Johannes Itten furtherexplored the psychological aspects of color. Itten'sThe Art of Color is a synopsis of his studies and teach-ings where he describes among other aspects hissubjective association of particular colors with spe-cific emotions. Itten "represented colors as expressivemoods and descriptions over the importance of differ-ent types of color contrast: hue, light-dark, cold-warm,complementary, simultaneously, extension and satura-tion" (Opara & Cantwell 2014:172). In his writings It-tendescribes the impact of colors on theviewer: "Col-ors are forces, radiant energies that affect us positivelyor negatively, whether we are aware of it or not" (Itten1970:12). Most of all, "Color is life; for a world withoutcolor appears to us as dead... Colors are the children oflight, and light of their mother" (Itten 1997:8).

ProjectHow do we embody the cultural significance of colorconnotations and include the scientific parameters ofvisible light in architectural design? CT tackles thisquestion through geometry. CT grows out of the re-search work on apparatus-centricity (Lee 2016). Onefacet of the theory posits that the cultural constructis largely driven by implementation and use of per-ceptual and cognitive apparatuses that are designedto augment and sometimes to replacehumancapaci-ties. The CT research aims to develop an architecturaland environmental instrument to analyze, map and

transcribe the color palette of a given urban placeat the eye level for architectural analysis and form-giving. CT approaches the topic of colorscape froma numerical point of view. (Here the term colorscapeindicates a combination of colors that characterize aspecified environment. It will also be referred to invideo recording as creating the sensation of experi-encing the colors of a particular environment or ascompositions created using the found color ensem-ble of a particular environment, either exclusively orin combination with visual performances.) It keepstrack of chromatic parameters such as brightness,contrast, hue and saturation that can be organizedas data, not the perceptual and emotional interpre-tation.

Within this framework, CT aims to develop a wayto track color compositions of cities and landscapes,and utilize the information as a form-making tool. Inaddition, the extended capabilities and applicationmay verywell include thepotential as an analysis toolto diagnose and evaluate the color palette, satura-tion and balance of a given architectural and/or ur-ban environment. Therefore the application may beused inmapping and studying the color compositionof cities and its in-between spaces. CT makes a toolfor both data visualization and form-making.

Iteration 1. In the first series of exercises, CT wasused to analyze colorscapes and map them in 2D inthe vocabulary of lines, shapes and materials. Thisprocess defines spatial relations and expressions thatcorrespond to different arrangements of color in theformal vocabulary.

Iteration2.During the second series of exercises, thecolorscapes are treated as a design tool where the 3Dshapes generated represent the colorscapes. Whilearchitects usually describe spatial relations in termsof form, CT allows todescribe the relations in termsofthe color qualities, which can also inform about lightor material. The CT app outputs a series of 3-d data-forms in the .obj file format that can be easily used bycommonmodeling and design applications. The .objfile from theCT application can thenbemodified andelaborated on using various additional parameters.


CT.1.4. In the current iteration CT.1.4 consists ofa custom application that can read from real-timevideo camera feed or a prerecorded video file froma video camera ormobile device. Here we apply digi-tal programmingmethods to create a newdesign ap-plication. The resulting custom CT application trans-lates the color composition and chromatic parame-ters of a given video input to inform and manipulatea three-dimensional geometry and surface.

As mentioned before algorithmic processes areused in the actual color analysis and form makingprocess. The program seen in Figure 1 was writtenin Max/Msp/Jitter. Max/Msp/Jitter has been used byperformers, artists, and composers extensively in or-der to customize applications for use in computermusic, interactivemedia and sound design. It is com-parable to Grasshopper for Rhino in architecture asit also follows a graphic approach rather than themore traditional coding. It offers a unique and di-verse collection of tools so called objects that arevisual boxes containing singular instructions to per-form specific functions. Some create noises, someother make video effects or perform simple calcula-tions. These objects are added to a visual canvas andconnected with patch cords (virtual wires). By com-bining objects, one can design an interactive, uniquesoftware application.

Figure 1CustomMax/Msp/Jittercolor trackingsoftware; Aleatorix.

The patch consists of three segments. The first seg-ments contains the main components for the color

tracking and the shape generator. The second sec-tion are the sub-components that are necessary toexecute each function, such as the timer and the fourindividual shape components for each color range.The third section finally consists of the componentsnecessary to generate, save and export a shape as an.obj file.

Analyzing the contents of the video input. Eventhough the basic algorithms for tracking motion andcolor have been around for a while (such as those inanimated movies), the dynamic nature of the tech-nology has been hardly put to use in architecture.Therefore CT takes advantage of such dynamic, real-time algorithms in the analysis and signification ofcolorscapes. This part of the application is designedto define and track a particular color range in a videoas its position changes from frame to frame. Thistechnique helps locate a particular color within agiven scene. First the video is captured anddisplayedin a separate window. It is possible to adjust upfrontthe image quality, saturation, hue, contrast, and soforth within the image control function of the pro-gram. TheMaxMSP Jitter object suckah is positionedover the window of the video and will output, aftersimply clicking on the display window, the color ofchoice as RGB color value. This value is used to al-ter the video to a preferred color range. In the sec-ond step, the altered video is channeled through ajit.chromakey object. This object takes two inputs:the video on the left and a reference color, in thiscase, black, on the right input. This object measuresthe chromatic distance between each of the left in-put's cells (pixels) and the reference black. This stepwas performed to achieve a "green screening" effect.A second suckah object over the new picture is usedto define the final color value for the tracking func-tion. The output value is then routed through a smallsub-patch that defines a specific color range. Us-ing only one definite value would not allow any lightergo color changes in the input live video and there-fore not deliver accurate data.

The object used to locate a particular color in animage is jit.findbounds. It tracks the previously de-


fined color range in the video using themin andmaxattributes. When jit.findbounds receives a video in-put, it looks through the entire input for values thatfall within the specified range. It thenoutputs the cellindices (coordinates) that describe the region whereit located the designated values. In effect, it outputsthe indices of the bounding region within which thevalues fall. The bounding region is a rectangle, sojit.findbounds will output the indices for the left-topand bottom-right cells of the region inwhich it foundthe specified values.

The process described here is executed for eachcolor range individually and the coordinates are sendto the corresponding part of the form generator aswell as the sound output component for further use.

Figure 2NURBS displaywindow; Aleatorix.

Figure 3NURBS with fourcontrol points asflat plan; Aleatorix.

SoundOutput (alternativeway to express the col-lected data). This section transforms the receivedvalues into audible sound. A makenote object out-puts MIDI sound information paired with a velocity

value which can be adjusted. With the pgmaout ob-ject a specific color is assigned to a specific soundtype. This component executes four times in total,one for each color range. The result is a unique soundmontage based on the urban colorscape samples.

Form Generator. Starting point of the actual formgeneration process is the jit.gl.nurbs object, whichrenders a NURBS surface. Displaying the NURBS re-quires a jit.gl.render object that renders the shape.In total there are four NURBS displayed as there arefour different color ranges in the current iteration ofthe application and each color range generates itsown shape as part of the final composition (Figure 2).Through various control points defined in a jit.matrix,the shape of the NURBS surfaces can be altered. Inthe specific case here, the jit.matrix object specifiesthe amount of control points to 40 x 40 points, whichdescribes the maximum amount of points in X andY direction. In the dimension component, the or-der and initial position of these control points is pre-defined to a regular grid. Several attributes can beadded to the jit.gl.nurbs object. On one hand thereare attributes equal for all four NURBS surfaces suchas position, rotation, predefined viewports and scale.On the other hand there are the attributes specificto each individual NURBS surface such as appearanceand color. At the start of the run-cycle the grid is setto display four control points as a flat plane (Figure 3).

Figure 4coordinate system;Aleatorix.


Figure 5NURBS point andX-Y-Z coordinates;Aleatorix

Figure 6NURBS with fouraltered controlpoints; Aleatorix.

Figure 7shape example;Aleatorix

The values obtained from the color tracking processare then used to alter the location of these controlpoints and therefore the geometry of the NURBS sur-

face (Figure 4).To manipulate the location of a single control

point, 3 values X,Y and Z are needed. Figure 5 showswhich points are altered andhow the values for thesepoints are registered to X, Y and Z coordinates. In thiscase the point in column 0 (first column) and row 0(first row) is altered. The initial location was X= 1.0,Y= 0.0, Z= 0.0 which was altered to X= 16.0, Y= 19.75,Z= 14.48 (Figure 6). Y and Z coordinates are definedthrough the color tracking process. To the X coordi-nate equals duration, a time length set by the user.The X coordinate therefore changes as the user chan-ges the time settings. Every time a sequence is com-pleted the column value will increase as there will beonemore control point added to each row, while therow value itself stays constant, set to 2 rows. Thistechnique can be describes as amono-directional (orlinear) growth process. At a given time, 2 controlpoints will be manipulated simultaneously.

The values of a given set of control points arestored in the sub-matrix. In the next sequence theconsequent set of control points will be transformed.This entire process can be repeated at least 4 timesbut beyond that as many times as needed (Figure 7).

Once the video input has ended it is possible tosave the shape in the .obj file format that for exam-ple can be read in AutoCAD or Rhino (Figure 8). Theshape will be exported in a rectangular grid mesh.Therefore the amount of control points must be amultiple of 4. A sub-patch monitors the amount ofcontrol points. If the user chooses to save the shapeit will check if the value is in multiples of 4. If it mis-matches, it will set the amount of control points tothe next smaller multiple of four.

Calibration. The visual performance of the camerasensor presents a series of complexity in calibration.The settings to produce equal images throughoutdifferent cameras vary from one to the next and donot remain consistent throughout. For the sake ofclarity, we will adopt the zone system and color tem-perature based on the existing conventions of pho-tography. The zones indicate the brightness range oflight and dark, and the color temperatures, the light


Figure 8input/ outputexample; Aleatorix.

source that directly affects the color rendition. Thezone 0 indicates pure white and the zone 10 pureblack when printed on white paper. The zone gra-dients are measured using the neutral photo grey(80% grey). The color temperatures refer to threemain reference points: 2700K (incandescent light),3200K (tungsten photo light) and 5500-6500K (sun-light). The white balance on the video camera in-put is calibrated according to the three color tem-perature references. The exception to the color tem-perature references include fluorescent lights (green-ish tint) and sodium vapor street lights (orange). Forthese two kinds of lights, we will rely on the auto-matic white balance function of the camera.

CTMobile App (Work in Progress)The CTMobile Appwill take advantage of the photo-and video-capturing capabilities of today's mobile

devices. The app is aimed at an architectural and ur-ban use for color-to-geometry conversionwith amo-bile device. The CT Mobile App utilize hand-held de-vices such as smartphones or tablets equipped witha camera for site analysis and design. Here the appis intended to work as location-based mobile appli-cation that translates color disposition of architec-tural and urban places immediately at the scene intothree-dimensional data-geometry. Such hand-heldmobiledevicesmayalsobenetworked; exchange thevisualized color data; and superimpose the informa-tion on different locations.

Application &UseCT can be used to assess and evaluate colorscapesand identify the perceptual black holes as well as toevaluate how they can be improved in order to re-store the chromatic balance and composition of the


place. Furthermore CT enables its users to comparethe color palettes of different architectural and urbanplaces.

Complexitybehind theMakingof SimplicityThe application that drives CT has a simple interfacedesign to deliver pure forms that are free of the cul-tural and enculturating baggage human constructsalways carry. But those forms are derived from thecomplex combination of hardware and software. Akit of hardware (e.g. smartphone, webcam, com-puter, etc.) requires a complex string of existing,mass-producedparts and assemblies. A piece of soft-ware is often a behemoth of conflicting algorithmsthat have accumulated and become too complex tofix but too expensive to be abandoned. In essence,CT embodies such contradictions of science versusculture and a never-ending pursuit of creating some-thing simple from a warehouse-full of complex bitsand pieces of technology.

In ClosingThe significance and contribution of the overall re-search can be summarized as the development of aneasy-to-use, portable application. It enables archi-tectural and landscape professionals to analyze thecolor disposition of a place and experiment with theformal expression by means of color composition.The value of the research output can be summarizedas a tool for documenting and visualizing the com-plex urban conditions in terms of color and provid-ing the effective means to express the combinationof urban colors in 3D-shapes.

REFERENCESByrne, A and Hilbert, DR (eds) 1997, Readings on Color:

The science of color, Band 2, MIT Pressvon Goethe, JW 1840, Theory of Colors, MIT PressItten, J 1970, The Elements of Color, John Wiley & SonsItten, J 1997, The Art of Color, John Wiley & SonsKoenderink, JJ 2010, Color for the Sciences, MIT PressKuehni, RG 2013, Color: An Introduction to Practice and

Principles, WileyLee, S 2016, Architecture in the Age of Apparatus-Centric

Culture, Unpublished Manuscript

Livingston, J 2014, Designing With Light: The Art, ScienceandPracticeofArchitectural LightingDesign, JohnWi-ley & Sons

Opara, E and Cantwell, J 2014, Best Practices for GraphicDesigners, Color Works, Rockport


Documents

Delft University of Technology ColorTracker