BeyondLab Tokyo #2 : Roboticsin Softbank HQ

“Coming Future of Robotics”Student Innovation through VR+Robotics

Akihiko Shirai, Ph.D

Associate professorKanagawa Institute of Technology, JAPAN

http://www.shirai.la/project/expixel

Biography1973 Born in Yokohama, Japan1983 (10) 1st programming1988 (15) Play Nintendo too much.1990 (17) News paper boy and arcade game1992-1998 Tokyo Polytechnic Institute

Photo Engineering, Virtual Reality in Lab1998 Hired, Canon Inc. Printing company.

(Got a chance of planner in Namco)2000 Moved to Video Game industry

Criterion Inc. Guildford U.K.Real-time Rendering Engine for PS2Meet up with Ph.D in Engineering

2001 Back to AcademiaTokyo Institute of TechnologyVirtual Reality Entertainment Systemsfor infancy. (Ph.D in Engineering)

2003-2004 Post-Doc in NHK ES.2005-2007 Move to France

ENSAM Presence & Innovation Lab.Prototype of VR Theme park project

2008-2009 Science Communicatorin National Science Museum

2010 Kanagawa Institute of Technology

[definition] Entertainment system is System which affects to human amusement.Video game, museum exhibition, entertainment VR, media arts, e-sports, broadcasting, social network…

1. Device and/or System

2. Service and/or Contents

3. Evaluation methods

Research in Entertainment system

Tangible Playroom: Penguin Hockey (2001)

AceSpeeder2 + GPU Vision (2006)

https://www.youtube.com/watch?v=N4WtupBoRMI

https://www.youtube.com/watch?time_continue=2&v=75jA7F8AXC4

“The song of Anagura”, Miraikan常設展示『アナグラのうた 〜消えた博士と残された装置〜』

Manga Generator Project

Robo Remote, Yamazaki Lab in KAIT

Concept: Solving common issue in living room

Pixel-Shader Technology that can show multiple images on one screen at same time.

ExPixel

Extended Pixel

X-Pol (FPR for 3D)

Extra Image…

What is ExPixel

Scritter to ExPixel: Scritter

NAGANO, K., UTSUGI, T., HIRANO, M., HAMADA, T., SHIRAI,A., AND NAKAJIMA, M. 2010.A new ”multiplex content” displaying system compatible with current 3d projection technology.

In ACM SIGGRAPH 2010 Posters, ACM, New York,NY, USA, SIGGRAPH ’10, 79:1–79:1.

Multiplex bytwo projectors.

Scritter to ExPixel: ScritterH

Wataru FUJIMURA, Yukua KOIDE, Robert SONGER, Takahiro HAYAKAWA, Akihiko SHIRAI, Kazuhisa YANAKA,"2x3D: Real-Time Shader for Simultaneous 2D/3D Hybrid Theater", ACM SIGGRAPH ASIA 2012 Emerging Technologies, Singapore, 2012

Hidden imagery with two projectors.

ScritterH, Projector based hidden imagery

Merge an arbitrary image A fornaked eyes and an image B thatyou would like to hide to generatea different image C and multiple-project B and C.

Image A visible to naked eyes whileimage B with glasses.

* Figure from ScritterH project

“2x3D”, hybrid 2D+3D theater

2-3 ハイブリッドシアター[2X3D]

Palarax!

Naked eye: Left + Filter: Right.

It is Implemented as a GPU based shader.

Good for theater but not suitable for consumer usage.

2x3D: Hybrid 2D + 3D display

Wataru FUJIMURA, Yukua KOIDE, Robert SONGER, Takahiro HAYAKAWA, Akihiko SHIRAI, Kazuhisa YANAKA,"2x3D: Real-Time Shader for Simultaneous 2D/3D Hybrid Theater", ACM SIGGRAPH ASIA 2012 Emerging Technologies, Singapore, 2012

Line-by-line: Even odd selection

Only even lines Only odd lines

Everybody can make multiplex hidden content with Unity 3D

Content Creation tool

1) Import Package 2) Set Images 3) Run!!

Multi-language presentation

Naked Channel (Janpanese)

Hidden Channel (English)

UbiCode

Naked Channel (Advertisement)

Hidden Channel (QR code)

POLAR: Physical Optical Layered AR

AR marker on the main content

POLAR: Physical Optical Layered AR

AR marker on the hidden channel

Road Mapping of Multiplex-Hidden technologies

1st gen “Scritter”Multiplex

2nd gen“ScritterH Multiplex Hidden

3rd gen on LCD Projector

4th gen on Flat Panel

5th gen Public Media

2010 2012 2014 2016 2018 2020

A. Shirai 2015.8.10

Multilanguage Presentation, education application

▲Medial appli.

▶ “PARAOKE”(2012)

◀ 4K/240 inch large screen(2010)

◀ “Scritter” by Koki Nagano (2010)

▲ “2x3D”, Hybrid theater （2012）▲ “UbiCode”Interactivity for digital signage （2012）

▶ private on public signage

◀「ニコニコメガネ」(2013)

Website: “blog.shirai.la” and @o_ob

WiiRemote Programming(2009)

Books

The Future of Game Design– Science in Entertainment systems (2013)

Come to see & join to Laval Virtual and IVRC(1) What is IVRC?

(2) What is Laval Virtual?

(3)What is ReVolution?

Today’s conclusion

Sponsored by

Symposium On Education

Education Workshop: Innovation in the Age of Virtual

Reality through Organizing International Student Competition

国際学生コンテストを通したVR世代のイノベーション

Akihiko SHIRAI, Ph.D

IVRC Executive Committee

Kanagawa Institute of Technology, Japan白井暁彦神奈川工科大学／IVRC実行委員会

Sponsored by

Agenda of this workshop

Innovation in the Age of Virtual Reality through Organizing International Student Competition

11:00-11:30 [Part1]Free Play of past IVRC projects

11:30-11:50 [Part2] Opening Address by Prof. Susumu TACHI

11:50-12:00 coffee break

12:00-12:30 [Part3] Panel Talk

12:30-13:30 [Part4] Panel Discussion (Japanese/English trans)

13:30 Photo session

-14:00 [Part5]Demo and Discussion

Innovation in the Age of Virtual Reality through Organizing International Student Competition

Education Workshop, Symposium on Education, SIGGRAPH ASIA 2015

Exhibition Hall No.2, Meeting Room 2A

11:00 – 14:00, Tuesday, November 3, 2015

http://ivrc.net

What is IVRC? International collegiate Virtual Reality Contest

establish in 1993 by Prof. Susumu Tachi

Contest of interaction

and virtual reality, which

fuses art and technology

Honors the greatest Virtual Reality and Interactive works of students

New education system through contest

What is IVRC? Organized by Virtual Reality Society of Japan

Supported by Ministry of Economy, Trade and Industry and Ministry of Education, Culture, Science and Technology

Ambassade de France au japon, Service pour la Science et la Technologie

Held at Miraikan, National science museum of Emerging Science and Innovation

Virtual Reality Society of Japanhttp://www.vrsj.org/

IVRC: contest of the students, by the students, for the students

IVRC is the acronym of “International collegiate Virtual Reality Contest”, which has been held since 1993 and is the contest of Interactive products using advanced technologies such as virtual reality and robots. Our English name changed for internationalization in 2004, because we began to invite awarded teams from the French VR contest “Laval Virtual”.

When you hear the words such as “Virtual Reality” and “robot”, you should feel that special expensive machines are necessary to realize them. However, there are many ideas which can be actualized only with handmade machines. The main purpose of this contest is for participants to gain deeper understanding of and familiarity with interactive technologies through activities such as planning interactive systems, making devices themselves, and improving quality of their contents.

IVRC has not only the function of contest but also the function of education. It aims to construct the new education system of the 21st century to develop excellent human resources, who can think, learn, and work by themselves.

Call for Submission

Deadline of submission

2nd stage, Demonstration

Final stage, Demonstration

SIGGRAPH/ Laval Virtual

Online submission

Improvement phase

1st stage, Paper review

Development phase

IVRC Flow Chart

4 +1 projects

Annual Conference of VRSJ

DCEXPO: Digital Content Expo

April

Late June or Early July

September

October

Semi final at the annual conference of the Virtual Reality Society of Japan (9-11th September, Tokyo)Demonstration in academic conference

Grand final in Digital Contents EXPO 201522-25th October, Miraikan, Tokyo.Demonstration in Public

U-18: Youth section

International Video Selection for the final stage

The First Presence at ACM SIGGRAPH 2002

IVRC Committee helps the champion team submit to SIGGRAPH as a reward to the winner.

“Virtual Chambara”, SIGGRAPH2002

“Virtual Chambara”, SIGGRAPH 2002

History of IVRC Presence at SIGGRAPH

2003 THE DIMENSION BOOK

2004 Fragra / Dis-Tansu

2005 Straw-like user interface / Kobito: Virtual Brownies / Ton2

2006 bubble cosmos / INVISIBLE ~The Shadow Chaser~ / Powder Screen: A Virtual Materializer …..

History of IVRC Presence at SIGGRAPH 2007 BYU-BYU-View / CoGAME

2008 Ants in the Pants / Landscape Bartender

2009 Baby type robot “YOTARO” / Back to the mouth /Funbrella: making rain fun

2010 Haptic Canvas

2011 Mommy Tummy

2015 Childhood/ vibroSkate

International Cooperation with Laval Virtual

“The winner of the IVRC Award in France will be invited to theTokyo stage of the IVRC in Japan as a challenger. Similarly, thewinner of the Laval Virtual Award in Japan will be invited to thefinal stage of Virtual Fantasy, category “Demos”, studentcompetition of Laval Virtual, in France as a challenger. Conditions ofinvitation will be defined every year by each part of this agreement.”

Agreement on International Cooperationat Laval Virtual 2004

IVRC and Laval Virtual will maintain an award partnership for three years (2004-2007): the Laval Virtual Award at the IVRC in Japan, the IVRC Award at “Laval Virtual’s Le Village de la Creations” in France.

J.F. Fontaine(Laval Virtual)

Prof. S. Tachi(IVRC)

International Cooperation with ETC, CMU

“The best work certified by CMU among other works created as student project will be invited for the final contest of IVRC. The invited work will be assessed in the final contest in the same way as other selected works in the preliminary contest of IVRC and will be entitled to receive each award of IVRC.”

Partnershipin SIGGRAPH 2010

Don MarinelliExecutive producer of ETC

http://ivrc.net/

Carnegie Mellon University (CMU) Entertainment Technology Center (ETC) and International collegiate Virtual Reality Contest (IVRC), in recognition of the growing importance of the roles Virtual Reality and Entertainment Technology play in the 21st century, have come to the agreement:

Agreement on International Cooperation in Virtual Reality and Entertainment Technology

Michelle Macau(ETC) Ambassador

Agent for Donald MarinelliExecutive Producer of ETC

Prof. S. Tachi(IVRC)

1st 3D Craze（the1950s)

Dawn ofVR

(the1960s)

1940 1960 1980 2000 20201920

Dawn of 3D

（the1920s)

2nd 3D Craze（the1980s)

3rd 3D Craze（the2010s)

2ndVR Wave(the2020s)

?1st

VR Wave(the1990s)

movie movie

Theme-park

computer

game

TV

movie movie

Virtual Reality in the 1990s 1989 VPL Research Data Glove and Eyephone. “Virtual reality" was first used.

1990 Santa Barbara Conference (The Engineering Foundation)

Virtually all the researchers were summoned.

1991 ICAT (International Conference on Artificial Reality and Telexistence) was

held in Japan.

1992 Committee on Virtual Reality was established in National Research Council.

Its report was published in 1995.

1993 IVR（Industrial Virtual Realty) was held in Japan.

1993 VRAIS (Virtual Reality Annual International Symposium) and Research

Frontiers in Virtual Reality were held. They gathered together to form IEEE

Virtual Reality Conference.

1993 IVRC（International Virtual Realty Contest) was held.

1996 VRSJ (Virtual Reality Society of Japan) was established.

2001 IEEE Virtual Reality Conference was held in Japan.

2008 National Academy of Engineering (NSF) announced 14 grand engineering challenges for the 21st century on February 15, 2008. Enhance Virtual Reality is chosen as one of the 14 challenges.

23 years… More than 200 projects have been realized through 23

IVRCs including 21 projects selected as SIGGRAPH E-Tech.

Average 5 persons for each project mean over 1,000 persons have deeply experienced and understood VR!

IVRC students become staffs, and staffs now become world famous professors, designers, creators, and engineers.

IVRC Alumni Members

21 CENTURY'S GRAND ENGINEERING CHALLENGES

National Academy of Engineering announced 14 grand engineering challenges for the 21st century on February 15, 2008. http://www.engineeringchallenges.org/

The final choices fall into four themes that are essential for humanity to flourish -- sustainability, health, reducing vulnerability, and joy of living.

Tremendous advances in quality of life have come from improved technology in areas such as farming and manufacturing. If we focus our effort on the important grand challenges of our age, we can hugely improve the future.

Enhance Virtual Reality is chosen one of the 14 challenges.

Second VR Wave（the 2020s ？)

VR Challenges

Autostereoscopic Display

Haptic VR

Augmented Reality

Telexistence

Human Augmentation

1st 3D Craze（the1950s)

Dawn ofVR

(the1960s)

1940 1960 1980 2000 20201920

Dawn of 3D

（the1920s)

2nd 3D Craze（the1980s)

3rd 3D Craze（the2010s)

2ndVR Wave(the2020s)

?1st

VR Wave(the1990s)

movie movie

Theme-park

computer

game

TV

movie movie

IVRC Wishes to thank…

New comer via “International Video Session”

The Maze Game(Puzzle, SAS VR, Oman)

Laval Virtual 2014 to IVRC 2014

Stimulus(ESIEA Ouest, Laval, France)In Digital Contents Expo 2015

Collaboration with KAIT students

Sponsored by

IVRC2015 to Laval Virtual 2016

Laval Virtual 201623rd-27th March, 2016 in Laval France

[Invited Project]Winner: “Nyokinyoki beans of the tree”(Jack and bean stalk) Univ. Keio

<ReVolution>Open competition in VR

11th Edition Theme “Real-Virtuality”(submission deadline: End of Dec.)

http://www.laval-virtual.org/

Come to see & join to Laval Virtual and IVRC(1) What is IVRC?

(2) What is Laval Virtual?

(3)What is ReVolution?

Today’s conclusion

Laval Virtual

Laval Virtual

Academic Conference “ACM VRIC”Award, Competition, Public Day

Keynotes

Workshop + Aural Presentation

ACM VRIC: Virtual Reality International Conference

Dr. Pattie Maes, MIT Media LaboratoryPr Alain Berthoz, College de FranceVirtual avatars and real brains: how do they interact? AR and VR for learning and collaboration

Dr. Oliver Bimber, Johannes Kepler Univ. Linz

Light-Field Technology: A Revolution to Imaging and Display

Dr. Carolina Cruz-Neira, Emerging Analytics Center, University of Arkansas at Little Rock

Beyond Fun and Games: VR as a Tool of the Trade

Christine Perey, AR for Enterprise AllianceThe Future of Work with Augmented Reality

Dr. Skip RizzoDirector for Medical Virtual Reality Institute for Creative

Technologies - Research Professor USC Davis School of

Gerontology and USC Keck School of Medicine Department of Psychiatry & Behavioral Sciences

Beyond Video Games:A Virtual Reality Revolution in Behavioral Health

Workshop + Aural Presentation

ACM VRIC: Virtual Reality International Conference

Special Session on Friday14:00-14:20 Opening Address “Real-Virtuality”, Prof. Akihiko SHIRAI (KAIT)

14:20-14:30 “IVRC to Laval Virtual / Jack and the Beanstalk”, Mr. Shota SUGIMOTO, (Univ Keio)

14:30-14:50 “Embodied Media”, Prof. Kouta MINAMIZAWA (Univ keio)

14:50-15:00 “Metamorphosis Hand: Interactive Experience of Embodying Virtually Transformed Hands”, NamiOgawa (Univ of Tokyo)

15:00-15:30 “VR to Superhuman Sports” Prof. Masahiko INAMI (Univ of Tokyo)

15:30-16:00 Round table and debate, exchanges with the audience

ACM VRIC: Virtual Reality International Conference

http://superhuman-sports.org/

AR/VR Contents, and for the next year…

Thierry Frey, chair of AR/VR contents and VR Startup Contest. At Final day of Laval Virtual 2016

URANIOM in Laval

Student Competition

Student Competition

RIKKYO IKEBUKURURO HIGH SCHOOL

MATHEMATICAL SCIENCE CLUB

SOARING

BIKE

LAVAL VIRTUAL 2016

Virtual Fantasy DEMO Category

“The flying bike that everyone can enjoy.”

The concept of SOARING BIKE is...

The fascinating parts

・Doesn’t have a target of age group

All Hand made

How to experience SOARING BIKE

・You could fly anywhere inside the stage.

Enjoy your flight.

Come to see & join to Laval Virtual and IVRC(1) What is IVRC?

(2) What is Laval Virtual?

(3)What is ReVolution?

Today’s conclusion

Open Competition of VR demo

Technical innovation / Interaction

Academic + Public demo testing

Theme “Real-Virtuality”

…Realizing Virtual, Bring real to virtual, Fantasy to be Real…

ReVolution

Glassless Augmented Display, KAIT

Gifts of Blizzard, Tama Univ.

"Real Virtuality"

We Wave II, Paris8

Submission deadline: end of year (31/Dec/2016)

Theme: Beyond visuals, Bring your body (temp.)

See you next year in Laval