Adaptive portable sensors such as beacons offer new ways to interact with content-driven apps. Museums are already using these to create rich immersive, interactive displays that could scale to be available in our homes.

Sensors

Sensors in the future home will let us design highly personalised and contextual experiences based on who is present, their activities and their environment.

What do sensors ‘sense’ that content makers can use?

z Who’s in the room? How many bodies, their age, size, ability level, interests, and more.

z Are people sitting, standing, moving? Are they going from room to room?

z What are they doing? Is there multitasking happening that shifts their attention?

z What’s in the space and how much room is there to move?

z Other data about the immediate environment such as temperature, humidity or light levels.

RESEARCH + DEVELOPMENT

The ABC will discover how to use sensory information for storytelling. We’ll have a broader range of data inputs to work with - there’ll be potential to add a new dimension to the experiences we design for audiences.

Is this for real?

You can buy this stuff now. Sensors already in the market give us a taste of what could become a standard feature in future homes:

Commercially available sensors are already used to adapt the physical environment (ie. lighting and temperature) through a control centre.

Google Nest thermostat

Withings Home camera

Estimote Beacon

People imagined

Audio that follows me around from room to room.

Visual displays that I can see no matter where I’m looking.

A space that adapts to my personal needs—my sound, temperature and light levels—even if I’m sharing the room with someone else.

Food in my fridge is analysed and monitored for nutrient levels and use-by-dates.

People were bothered by All of these sensors make me feel as if I’ve invited Big Brother into my home. What will they see? Will my privacy be compromised? Who controls the captured data?

The costs of acquiring and installing the technology.

They can currently track brain basic function, heartrates or even fertility. Sony Xperia Ear wireless smart earbuds allow for more advanced media players that react to the body’s response in order to deliver content.

Oculus Touch Controllers Wireless hand controllers for manipulating objects in virtual reality.

Is this for real?

Smart watches, fitness bands and sleep trackers have already hit the marketplace in a high profile way but wearable technology available now includes things like:

Nike Run and Google Project Jacquard are examples of smart fabrics and shoes that track performance and health data.

Melon EEG and Bellabeat Leaf are part of a growing range of health products that can track the body.

Wearables & BiofeedbackWearables will soon be smarter, more discreet and diverse, and the data they collect from our audience will open up new content opportunities and user interactions.

What data will wearables collect that we might use?

z Data from inside the body; heart rates, blood sugar, vitamin levels and brainwaves.

z Physical data related to moods and emotional responses.

z Proximity, positioning and movement; wearables can have sensors that understand where our bodies are in relation to other objects.

People are willing to share their personal data to get content that benefits their health and wellbeing. As a trusted provider, the ABC might also use this information to augment other services - mood altering programming perhaps?

FitbitBeacon

EmotivEPOC neuroheadset

BedditSleep Tracker

People imagined

A perfect night’s sleep as a result of biofeedback interventions. For instance, sounds that encourage REM sleep.

People were bothered by

Interpreting all of the data. Now I have the information, so what? What next?

The fear of someone or something reading my mind.

That I’ll lose my ability to listen to my own body.

I am worried about electromagnetic radiation.

RESEARCH + DEVELOPMENT

Teddy The Guardian, Ubooly, Disney Infinity are smart toys designed with inbuilt connectivity, companion apps and games.

Lights like the Philips Hue, which allow us to control and adjust lighting settings remotely or responsively.

Connected, multi-purpose speakers like the Triby speaker for smart kitchens - this lets you listen to music, make hands-free calls, leave digital notes for other households members.

Smart Objects (IoT)Smart objects in the Internet of Things give us a whole new platform for the distribution of content in the home. They will also allow us to create media experiences that meaningfully augment the home environment and people’s activities in this space.

What makes smart objects useful for content delivery?

z They collect and store data from cameras and sensors used to make content more contextual, more personal.

z They’re web-enabled and can be networked to ‘talk’ to each other.

z They can be contextually placed, fitting naturally into rooms they are designed for.

z Surfaces can be smart too (e.g. smart walls, glass, tables), and offer interesting new possibilities for display.

RESEARCH + DEVELOPMENT

The ABC could use smart objects as physical props and as triggers for multisensory elements (e.g. lights, fans, aromas) in the experiences we create for people in their homes. Smart surfaces will also help us to supply people with right place, right time information in a convenient way.

Is this for real?

Right now when we say ‘future home’, people’s first thoughts are around smart fridges. Networked appliances have captured our attention when it comes to smart objects, but other “Internet of Things” examples for the home include:

Smart ToysUbooly

PhilipsHue

Invoxia Triby

People imaginedA smart mirror that knows who I am and gives me relevant information and entetainment at the right moments.

Kitchen appliances that decipher the contents of my fridge or pantry and help me to eat and cook better. Lighting and speaker systems that know when I enter room and adjust accordingly.

People were bothered byWill my home be hacked?

It’s expensive! The tech’s not good enough yet. I want my fridge to last at least 10 years! I don’t want to be spied upon, or for companies to start selling advertising on my toast.

Is this for real?

While some people already use projectors for home cinemas and gaming today, projections in future homes will be much more sophisticated. Here are some recent examples that help us understand both projections and holograms in the future home space:

The Holus is an interactive, holographic, tabletop display with 3D games for the whole family.

High lumens, short throw portable projectors like these from Sony Life Space are starting to become more affordable.

Holograms and in-home projections for play are starting to be used for kids content - e.g. Star Wars Force Trainer and Lumo Play.

Projections & HologramsAlthough future homes will still have plenty of screens and surfaces for viewing content, we will also be able to use holographic, light field technologies and projections to create more immersive and portable displays.

What are some of the benefits of using holograms and projections?

z They are flexible. Their size and location can be adjusted to suit specific situations and available space.

z They are immersive. They can be displayed all around a viewer to create atmosphere, or transform their setting by bringing content to life in the space around them.

z Holograms can also be used to create very life-like scenes in 3D.

FitbitHolus

SonyLife Space Projector

Start WarsHologram Toy

People imaginedCreating a relaxing ambience in my home with big and beautiful arty visuals.

I want to see things in a larger scale, so we don’t all have to crowd around a small screen.

I want to collaborate with others on a project in a 3D visual way.

People were bothered byI don’t want my personal stuff like emails projected on the wall for everyone to see! Just things that are good for everyone.

Digital hygiene. I’m not sure I want more digital stuff my bedroom.

Will the projectors be noisy? Will images be bright enough for day time, or will I have to close my blinds?

RESEARCH + DEVELOPMENT

RESEARCH + DEVELOPMENT

Smart assistants like Apple’s Siri, Google Now and Microsoft Cortana use basic artificial intelligence and voice recognition to respond to requests for information.

Gesture &Voice (NUIs)Natural User Interfaces (NUIs) add a new dimension to how people can interact with media. Voice and gestures can be used to either bring richer engagement, or to leave the rest of the body free to do other things.

What will voice recognition and gestural interaction let us do that’s different?

z Effortless interaction. Combining voice with artificial intelligence means information is there for the ‘asking’.

z Design experiences to make multitasking easier. No more messy hands on the kitchen iPad!

z Use the entire body as a conduit to engage with media. Right now we are limited to click, touch, tap.

z Create more accessible experiences by allowing people to interact in the way that suits them.

Is this for real?

Voice and gestural interactions are already widely available but still rudimentary. Voice recognition technology has many commercial applications - most notably smartphone assistants. Gestural interfaces are also very accessible but haven’t yet crossed successfully into mainstream consumer applications.

Microsoft’s motion sensor device, Kinect, is one of the most well known gestural tools of today.

More recently, interaction sensors that use radar technology have been developed to track gestural interactions very precisely, e.g. Google Project Soli.

Google Soli

MicrosoftKinnect

Apple Siri

People imagined

Finally, I can look up! I can control my media without having to look at a display.

I’ll be free to multitask and roam around the house without being tethered to my phone.

The body movements that I make every day - such as when I’m exercising or chopping vegetables could be detected to trigger certain actions.

People were bothered by Will the system understand my accent?

I don’t like speaking in the morning, so I don’t want to use my voice.

For certain tasks it might take more effort for me to use my body. I may as well just tap a button.

Will my dog think that I’m talking to it?

I might feel like a bit of an idiot waving my arms around.

Google’s Deepmind researches a subfield of artificial intelligence called Deep Learning that aims to create an artificial ‘brain’.

AI and Virtual AssistantsThe future home will be (artificially) intelligent, using data from sensors, smart objects and devices, and personal wearables to learn about it’s occupants and then meaningfully augment their lives.

What will A.I. mean for content in the home?

z Machines will ‘decide’ what content to suggest based on what they know about their household.

z Content will be designed for machine identification as well as human consumption - even more than now.

z Virtual assistants will come to feel like a household member with a human voice and name.

Is this for real?

A.I. is the top research priority for the world’s largest technology companies (Google, Facebook etc) who use the information they acquire from their existing search engines.

Currently virtual assistants utilise a basic machine intelligence that is embedded in, or at least catered for, in a growing number of technology products:

Amazon Echo is a ‘home hub’ with a virtual assistant called Alexa that uses simple A.I. to answer questions and make recommendations.

Google Now/Ok Google already displays a sophisticated understanding of individual patterns of behaviour and delivers responsive content suggestions.

Google Now

AmazonEcho Alexa

Xibot TechXibot

People imagined

At last, my life will be so organised. I will spend less time on household planning, and spend more time with my family.

I won’t ever need to actively seek out weather or traffic updates because my assistant will anticipate my information needs.

People were bothered by I don’t want a machine telling me what I like. I’m a little worried about my assistant giving me biased information, or trying to sell me stuff. The costs of acquiring and installing the technology. I’m am doubtful that the foundational technology supporting Siri or Cortana will ever going to be good enough. Will my virtual assistant always be listening to me? That’s a bit creepy. Can I turn it off?

RESEARCH + DEVELOPMENT

Magic Leap, the Google funded, super secret MR startup periodically releases astounding videos but have stated ‘it’s not coming until it’s ready’.

Artificial RealityIn many ways, the home space offers the most exciting possibilities for ‘artificial realities’. It’s the most practical context, and this is where people will feel the most comfortable using immersive and physically exploratory experiences.

What are some exciting ways we can use VR, AR and MR in the home?

z Virtual reality can literally place our audience inside a story or event, or transport them to a location they might otherwise not be able to experience - all from the comfort of their own home.

z Augmented and mixed reality will allow us to create story experiences that make use of objects, furnishings and the physical space in homes. Find out more in the “Future of Family Entertainment” section of this expo!

Is this for real?

Virtual Reality is here and making waves in 2016, and there is already talk that AR/MR technologies are said to be the next big disruptors of mobile. Supporting this are significant investments in VR and AR/MR content by companies like 20th Century Fox, Disney, Industrial Light and Magic, HBO, Facebook and Amazon.

Oculus Rift, the first consumer ready VR headset from the Facebook owned company is on the market now. The HTC Vive is a close competitor with Sony’s Playstation VR headset due later this year.

Microsoft have done a number of high profile demonstrations of Hololens, their AR/MR headset - but the beta version still has a way to go to reach it’s full potential.

OculusRift

Mini Augmented Vision

MicrosoftHololens

People imagined

Practical hands-on activities will be so much easier! Great for following a recipe, cutting a pattern, or stepping through instructions. I’d like to experience places that I will probably never get to visit in my lifetime. I like the idea of information that follows me around.

People were bothered by

Do I have to wear a headset? It looks and feels awkward.

Make sure I can easily turn AR on and off.

RESEARCH + DEVELOPMENT

Sound

How people will consume and listen to audio in future homes is a great story all of its own. There will be a range of new ways for media makers to immerse audiences, and smarter forms of seamless, personal listening experiences.

What are the possibilities for future audio technologies in the home?

z Binaural audio (also known as 3D sound) replicates how you would actually hear sound in a room, space or environment. It’s used to add a greater sense of presence and deeper immersion in virtual and augmented reality experiences.

z Augmented reality earbuds will listen to the environment around you and then remix the soundscape in real time.

z Smart wireless speakers will no longer be static objects - they’ll be multi-functional, and more contextually aware (knowing who’s in the room and what they’re listening to).

Is this for real?

Various production companies and broadcasters are already experimenting with binaural audio production. Many manufacturers of speakers and home audio systems offer a range of future-focused products targeted specifically for a smart home context:

Sonos are designing smart speakers with software and sensors that will make them responsive to listener activities.

Hearables like Sony Xperia Ear allow you to send and receive messages, check your schedule, search and navigate, all without a screen.

XperiaEar

SonosPlay: 5

Neumann heads DPA 5100

People imagined

Sound that follows me as I move from room to room without distortion or volume differentiation. I want to listen to whatever I like, and wherever I am in my home without disturbing other people. I want high quality, 3D sound experiences so I can feel like I’m really there!

People were bothered by

I don’t own this house, so I don’t want to have to pay to install a wireless speaker system! I don’t want to be completely isolated from those around me, as tends to happen when I wear earbuds or headphones.

RESEARCH + DEVELOPMENT

Sony’s Life Space UX (above) Glass Sound Speaker is a bluetooth speaker with LED lighting to create a welcoming, ambient glow.