Dress.code #3 - Interactive Technologies

presented by

What We Do & Who We Are With Roots In Design, Development, And Film, We Meandered The Crowded Landscape Of Web And App Creation, Eventually

Casting A Wider Net With More Innovative And Imaginative Solutions For Our Diverse Clientele.

3 1 50+ ∞Offices Promise Awards Bad Jokes

July 2015

A Digital Production CompanyEqual Parts Craft & Curiosity

Development Content Creation Experiential

July 2015

THIS IS DRESS.CODEA Community Where Like-Minded Individuals Can Come Together To Learn To Subvert Traditional Approaches To

Technology In Media. 

CREATIVE TECHNOLOGY IS OUR FORTE

July 2015

Jean-Michel KristofferJustin

July 2015

THE MAKING OF THE MOLSON’S GLOBAL BEER FRIDGE (ARDUINO, ANDROID, GOOGLE APIS)

HUNGER GAMES MOCKINGJAY FACE RECOGNITION ACTIVATION (KINECT, FACE TRACKING)

July 2015

INTERACTIVE TECHNOLOGIES

July 2015

Molson’s Global Beer FridgeCASE STUDY

Objective

Locally translate multiple languages from the original spoken word to English.

Determine if the phrase “I am Canadian” was spoken.

Arduino Android

July 2015

Google APIs

July 2015

Problems

- Language / speech is complex

- Very large amount of processing power is required

- Series of very broad corpora (bodies of preprocessed text and speech patterns)

- The availability of such corpora was limited at best (ex: restricted to Universities / selection of languages)

July 2015

✓ Most advanced

✓ Largest number of languages

✓ Best API

๏ Not a commercially available product

๏ Development purpose only

PROS CONS

Google Speech Recognition API

July 2015

Process

1. Record phrase and convert to sound file (.flac)

2. Transmit to Google Speech Recognition API with selected languages to compare against with

3. Receive transcribed speech or a failure message

4. Compare transcribed speech to known phrase in that language

5. Confirm that the required phrase was spoken

Google Speech Recognition API

July 2015

Limitation

Google Speech Recognition API

๏ 50 requests per day

๏ Limits access to predefined IP addresses

๏ New configuration would be necessary on every location or IP address change

July 2015

✓ Chromium and Chromium OS

✓ An open source web browser for Linux that emulates Chrome

✓ Includes a Speech API key which can be used by developers within reason

✓ Daily requests and the IP limitation was overcome

Chromium Project

July 2015

Web Speech API

(Demo)

July 2015

Android app

✓ Main controller

✓ Voice recognition

✓ Display

✓ Audio visualizer

July 2015

Hardware

July 2015

Arduino

✓ Open-source computer hardware and software

July 2015

Arduino IDE

✓ Open-source software

✓ Windows, Mac, Linux

✓ Written in Java

✓ Code in C/C++

July 2015

HARDWARE / APIs

Arduino

Android app

Microphone

Bluetooth

Serial Data

Google Speech Recognition API6 RGB Addressable LEDs

Magnetic Lock

Relay

Button

July 2015

Arduino Box

✓ Arduino Uno

✓ Relay Shield

✓ Bluetooth

March 2015

Hunger Games MockingjayCASE STUDY

Hunger Games MockingjayCASE STUDY

Objective

Kinect Camera Face Tracking Website (share / gallery)iPad (share)

July 2015

Take a selfie and a branded face-paint graphic will be applied on their face

Share their poster

July 2015

✓ Points (1347), Triangles (2340)

✓ Hair Color, Skin Color

✓ Face properties (glasses, mouth, eyes)

๏ Output quality (1080p resolution video camera)

PROS CONS

FACETRACKING - R&D

HD Face Tracking SDK (Kinect)

(Example)

July 2015

✓ Performance

✓ Open source๏ Tracking

PROS CONS

FACETRACKING - R&D

ofxFaceTracker (openFrameworks)

(Example)

July 2015

✓ Performance

✓ Tracking

✓Cross-platform [ C++ (Android / iOS) - AS3 - JS ]

๏ $$$ (around $2500 / project)

PROS CONS

FACETRACKING - R&D

Beyond Reality Face Nxt

(Example)

July 2015

✓ SDK

๏ Liveview (via Wi-FI)

DSLR camera(Canon)

Mirrorless Camera (Sony)

CAMERA - R&D

Camera

๏ Shut down after 2h (with liveview) ๏ API (via Wi-FI)

✓ Liveview

July 2015

Kinect

(Example)

✓ Skeletal tracking

✓ Face tracking

✓ 1080p camera

✓ Speech Recognition.

July 2015

HARDWARE

TV

Kinect

Camera

Laptop

iPad

July 2015

TECHNOLOGIES

Create a poster

Kinect

Camera Video stream (as Webcam)

Kinect API (via ANE)

Custom filters (liveview)

Generate picture Upload to Server(http://localhost:4000/)

Face Tracking

July 2015

TECHNOLOGIES

Share a posterServer iPad app (Web App)

fs.watch

Send email (via SendGrid)

Upload pictures

www

REST API

Refresh gallery

Form completed

Show pictures

Show gallery

April 2015

Challenges

✓ UI / Kinect Gesture

✓ Lights setup

✓ Focus

THANK YOU

Jean-Michel KristofferJustin

July 2015