69
Sketching in Code / Processing Marcin Ignac @ JUG, May 2010, Poznań How I use Processing to do prototyping in almost every project. (screenshot from Wiild Life interactive animation http://marcinignac.com/projects/ 2008/10/12/wiild-life )

Sketching in Code

Embed Size (px)

DESCRIPTION

My presentation about Processing and the role it has in the prototyping (and very often development) phase of almost every project I take part in.

Citation preview

Page 1: Sketching in Code

Sketching in Code / Processing

Marcin Ignac @ JUG, May 2010, Poznań

How I use Processing to do prototyping in almost every project.

(screenshot from Wiild Life interactive animation http://marcinignac.com/projects/2008/10/12/wiild-life)

Page 2: Sketching in Code

Me, 27, programmer, designer, artist

I live and work in Copenhagen, DK

Page 3: Sketching in Code

PROCESSINGopen source, cross-platform, multimedia programming language

What is it?

Processing was founded by Ben Fry and Casey Reas in 2001 while both were John Maeda's students at the MIT Media Lab. http://processing.org

Page 4: Sketching in Code

Environment

Code editor. Set of basic libraries. Script compiler. Application and applet exporter.

Page 5: Sketching in Code

Programming Language

Beginner

Basic scripting. Static images.

Page 6: Sketching in Code

Programming Language

Intermediate

Functions. Draw loop is executed every frame. Easy acces to keyboard and mouse for interaction.

Page 7: Sketching in Code

Programming Language

Advanced

Modularity. Classes. External libraries.

Page 8: Sketching in Code

Programming Language

Expert

Eclipse. Beside better code editor it’s easier to manage your code and writing your own libraries.

Page 9: Sketching in Code

Open Source

Community

http://processing.org - forum, exampleshttp://openProcessing.org - gallery of projects with source code

Page 10: Sketching in Code

Open Source

Libraries

3rd party libraries can provide us with better video handling, more advanced 3d, procedural geometry or physics.

Page 11: Sketching in Code

Cross-platform

Java: Linux, Mac, Mobile, PC

Page 12: Sketching in Code

Cross-platform

Android

(Photo by creativeapplications.net)

More: http://android.processing.org/

Page 13: Sketching in Code

JavaScript

Cross-platform

All basic Processing functionality was recently ported to Javascript. Audio, video and 3d soon will be also possible with technologies like HTML5 and WebGL.

More: http://processingjs.org/

Page 14: Sketching in Code

Example Projects

Why I called it multimedia or interactive media programming language.

Page 15: Sketching in Code

‘Art’

After I finished Computer Science at Poznan University of Technology I studied at the Academy of Fine Arts in Poznan where I was exposed to both traditional art...

Page 16: Sketching in Code

Media

... and some more contemporary stuff like video and interactive installations.

Page 17: Sketching in Code

“That what is the first”

One of my favourite art pieces I made. There is an image of a woman projected on the wall. When visitor comes closer to the picture, the woman becomes younger and younger.

It was was the first time I used Processing (for motion tracking).

Page 18: Sketching in Code

Motion Tracking

Recorded image is processed and moving objects are detected to termine their distance to the target.

Page 19: Sketching in Code

Context

The art work was exhibited in a abandoned building in the center of Poznan.

Page 20: Sketching in Code

My Mother

Video: http://vimeo.com/666152

Page 21: Sketching in Code

Copenhagen Institute of Interaction Design

CIID

In 2008 I moved to Denmark to take part in the pilot year of new interaction design course.

Page 22: Sketching in Code

Greetings gestures

On of our first projects at CIID completed during Computational Design course. The idea was to record hand movements of people greeting each other. They were comming from different cultures so the gestures varied a lot.

(Photo by just.Luc / Flickr)

Page 23: Sketching in Code

Wii Remote

Controlling device for Nintendo Wii was an obvious choice. It has bluetooth connection and motion sensors.

Page 24: Sketching in Code

Recording

Page 25: Sketching in Code

Darwiin Remote

We used DarwiinRemote to connect to the Wii Remote via Bluetooth and record the data.

Page 26: Sketching in Code

Korea

USA

Denmark #1

Denmark #2

Italy

IndiaData

We have collected data for both hands and heads. I decided to use hand data only.Colors: xAcc (red), yAcc (green), zAcc (blue), pitch (yellow), roll (light blue)

Page 27: Sketching in Code

Experiments

I started buy just displaying the raw data (previous slide). Next step was to use the time and acceleration values to alternate shape of set of cubes in 3d space.

Page 28: Sketching in Code

Demo

I made an interactive application to explore different possibilited and parameters.

Video: http://vimeo.com/1905763

Page 29: Sketching in Code

Final posters

Three final designs were made.

Page 30: Sketching in Code

Exhibition

All the posters were exhibited at Danish Design School at the end of 2 weeks course.

(Photo by toujjval / Flickr)

Page 31: Sketching in Code

VJ-ing

I recorded some of the experiments and used them later at one of my vj shows.

Video: http://vimeo.com/2630365More: http://marcinignac.com/blog/2008/11/03/vj-vorg/

Page 32: Sketching in Code

Have Fun

My first Wii Remote test :) “White brick in space”

Page 33: Sketching in Code

3D Drawing

At the exhibition I met Flemming Tvede Hansen (Ph.D-Scholar, DKDS) who asked me if it was possible to use Wii to draw in 3d. I said yes. And few days later I had working prototype. We used Wii Remote’s build in IR-Camera and some IR LEDs.

Video: http://vimeo.com/2238077

Page 34: Sketching in Code

We proceeded by making interactive app that Flemming used to explore various dynamic shapes. This first version was able to do just flat 2D drawings.

Page 35: Sketching in Code

Second version was able to export 3d models. It was still limited to one 2d plane though.

Page 36: Sketching in Code

Final version was fully capable of drawing in 3d. It was controlled by both mouse or Wii Remote.

Page 37: Sketching in Code

Computational Design

Some of the designs were even printed in 3d.

Page 38: Sketching in Code

3D Print

Page 39: Sketching in Code

3D Print

Page 40: Sketching in Code

External Hardware

Serial Port

Processing can be very easily connected to external hardware. For example throught serial port.

Page 41: Sketching in Code

Left Rotate Right Next

Arduino Controller

Arudino board with 3 buttons and light sensor.

Want to know more how to build your own stuff?http://arduino.cc - Arduino home pagehttp://protolab.pbworks.com - Stanford prototyping tutorialshttp://fritzing.org - Circuit design toolhttp://www.partly-cloudy.com/misc/toolkits/listall - list of Arduino alternatives

Page 42: Sketching in Code

Demo

I showed game controller made in 30min. Tree buttons control position and rotation of the brick. When you cover the light sensor the current brick is dropped down.

I used modified version of Tetris code from http://leo.lobato.org/processing/tetrisclone/index.html

More about my Arduino projects: http://marcinignac.com/blog/tag/arduino/

Page 43: Sketching in Code

External Hardware

Wi-Fi

Another possibility is to use Internet / WiFi connection.

Page 44: Sketching in Code

Demo

I used multitouch TouchOSC app to simultaneously control two particle systems. oscP5 library was used to handle communication part in Processing (parsing messages in OSC protocol).

Page 45: Sketching in Code

Realtime, everytime.

In september 2009 I joined shiftcontrol studios (http://shiftcontrol.dk/)

Page 46: Sketching in Code

Live Visuals

DR P3 Guld’09

Our first project together. Live audio reactive visuals for Mike Sheridan during danish radio music awards show.

More & Video: http://marcinignac.com/projects/2009/01/26/DR-P3-Guld'09

Page 47: Sketching in Code

Live Visuals

DR P3 Guld’09

Page 48: Sketching in Code

Control App

For the purpose of the show we developed audio reactive app. Generated visuals were then displayed on LED screen on the stage.

Page 49: Sketching in Code

Projection Mapping

Page 50: Sketching in Code

Projection Mapping

When we project an image on the surface in front of the projector the image looks nice.

More: http://marcinignac.com/blog/2009/12/09/projection-mapping-in-3d/

Page 51: Sketching in Code

Projection Mapping

The bigger the projection angle the more image is distored.

Page 52: Sketching in Code

Projection Mapping

We can fix that by projecting image from projector point of view. Together with some cleaver masking we can achieve very good results.

Page 53: Sketching in Code

Experiments in the studio

Audio reactive bars on the pipes in our studio.

Video: http://vimeo.com/7504037

Page 54: Sketching in Code

Experiments in the studio

Sun ‘behind’ the pipes.

Video: http://vimeo.com/7504182

Page 55: Sketching in Code

Projection Mapping

Sun’s trick explained. In the places where there are pipes we simply display black color.

Page 56: Sketching in Code

Experiments in the studio

Some more advanced masking.

Video: http://vimeo.com/7504276

Page 57: Sketching in Code

Experiments in the studio

I use just one projector here.

Video: http://vimeo.com/7630261

Page 58: Sketching in Code

Eskulap Club

Poznań 2009

I fall 2009 I was invited to do some interactive installation in Eskulap Club, Poznan. I decided to do something with the ceiling.

Page 59: Sketching in Code

Mapping Setup

I designed software that allowed me to map particular sides of the cubes and display animations on them.

Page 60: Sketching in Code

3xI

Vivisesja, Poznań 2009

In the final setup I was using one laptop and two projectors. There was also in InfraRed sensor on the floor so whenever somebody was passing by the animation was changing.

More: http://marcinignac.com/projects/2009/11/20/3xI

Page 61: Sketching in Code

ProjectedQuads

More & Source Code: http://marcinignac.com/blog/2010/01/31/projectedquads-source-code/

Page 62: Sketching in Code

a.k.a. prototyping

Sketching in Code

Sketching on paper, Prototyping interactions, Interactive SketchesReasons to sketch: testing ideas, exploring posibilities, when i start i kind of know what i want to achieve but actual results my vary ;)

Page 63: Sketching in Code

Keep everything

Save all ideas / code. You never know when you will need to go back to an older version.

Page 64: Sketching in Code

Dynamic Identity

In our latest project we are looking at properties of paper and how this style can be replicated in code.

Page 65: Sketching in Code

Paper Folding

Processing

My first experiment was to simulate origami-like paper folding. You can select two triangles and bend virtual paper along the edge between them.

Video: http://vimeo.com/11810679

Page 66: Sketching in Code

Mesh Deformation

C++ / Lib Cinder

Second sketch was build for iPad to test multitouch interactions. Using fingers you can squeeze the mesh and deform it.

Video: http://vimeo.com/11899479

Page 67: Sketching in Code

Cloth Simulation

Processing / Traer Physics

Latest sketch involved some physics. The two highest points is where the fingers will be. I has to be yet tested on multitouch device.

Video: http://vimeo.com/11911657

Page 68: Sketching in Code

Exploring by prototyping.

Understanding by making.

Learning while playing.

Simply start making stuff. Right now! Even if you don’t fully understand what you want to do. Just try it, whatever is in your mind. You can’t fully understand something until it’s visible, tangible so don’t wait. Play with it, learn from it observe, fix it or build it again.

Page 69: Sketching in Code

Thank you!

marcinignac.com