2011-2012 Report for AOARD Grant 11-4039

“Natural Models for Autonomous Control

of Spatial Navigation, Sensing, and Guidance, Part 1”

Date 13-2-12

Name of Principal Investigators: Justin Marshall - e-mail address : Justin.marshall@uq.edu.au - Institution : The University of Queensland - Mailing Address : Queensland Brain Institute, The University of Queensland,

St Lucia, Queensland 4072, Australia. - Phone : +61 7 33651397 - Fax : +61 7 33654522

Thomas Cronin - e-mail address : Cronin@umbc.edu.au - Institution : The University of Maryland Baltimore County - Mailing Address : Biological Sciences, UMBC, 1000 Hilltop Circle, Baltimore

MD 21250, USA - Phone : +1 4104553449 - Fax : +1 4104553875

Period of Performance: 1/1/11 – 1/2/12 (UK format dates) The personnel involved in this project have worked well together and those specifically associated with the output below are summarized below: Justin Marshall PI Tom Cronin PI Nick Roberts (UK) Supporting collaborator Nader Engheta Supporting Collaborator (Viktor Gruev) Shout Chiou PostDoc, salary supported, Moved December 2011 Martin How PostDoc, salary supported Shelby Temple PostDoc, research supported Vincenzo Pignatelli PostDoc, research supported Hanne Thoen PhD, salary supported (started April 2011) Chris Talbot PhD, research supported 50% Wen-Sung Chung PhD, research supported 30% Yi-Hsin Li PhD, research supported 30% Mathilda Bue Undergraduate Honours Awarded 2011 Rachel Templin Undergraduate Honours Awarded 2011 Mickey Pascoe Undergraduate Honours

Report Documentation Page Form ApprovedOMB No. 0704-0188

Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering andmaintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information,including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, ArlingtonVA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if itdoes not display a currently valid OMB control number.

1. REPORT DATE 21 MAR 2012

2. REPORT TYPE Annual

3. DATES COVERED 05-04-2011 to 05-04-2012

4. TITLE AND SUBTITLE Natural Models for Autonomous Control of Spatial Navigation, Sensing,and Guidance

5a. CONTRACT NUMBER FA23861114039

5b. GRANT NUMBER

5c. PROGRAM ELEMENT NUMBER

6. AUTHOR(S) Justin Marshall

5d. PROJECT NUMBER

5e. TASK NUMBER

5f. WORK UNIT NUMBER

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) The University of Queensland,St. Lucia,Brisbane, Queensland 4072,Australia,NA,NA

8. PERFORMING ORGANIZATIONREPORT NUMBER N/A

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) AOARD, UNIT 45002, APO, AP, 96338-5002

10. SPONSOR/MONITOR’S ACRONYM(S) AOARD

11. SPONSOR/MONITOR’S REPORT NUMBER(S) AOARD-114039

12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release; distribution unlimited

13. SUPPLEMENTARY NOTES

14. ABSTRACT The development of a behavioral screening technique for polarization sensitivity in any animal (trialed oncrustacean, cephalopods, fish and next turtles).

15. SUBJECT TERMS

16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF ABSTRACT Same as

Report (SAR)

18. NUMBEROF PAGES

19

19a. NAME OFRESPONSIBLE PERSON

a. REPORT unclassified

b. ABSTRACT unclassified

c. THIS PAGE unclassified

Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18

On 16-1-2011 a final report was submitted for the first half of this work (AOARD

09-4073) with the following summary of achievements:

a) The development of a behavioural screening technique for polarization sensitivity in any animal (trialed on crustacean, cephalopods, fish and next turtles).

b) Demonstration of polarization sensitivity ten times more acute than

previously documented in cephalopods (cuttlefish and octopus) and

crustaceans (mantis shrimps and crabs). This opens a potentially new

chapter in polarization communication and camouflage.

c) Electrophysiological characterization of linear polarization, circular polarization and colour photoreceptors in 3 species of mantis shrimp (stomatopod). This includes confirmation of elliptical encoding in 2 species, again suggesting a covert communication language?

d) Behavioural acuity threshold of linear polarization in a stomatopod

partially achieved.

e) Anatomical basis of circular polarization characterized in 3 stomatopod species.

f) Modeling of ideal ¼ wave retardation advanced using biological parameters as input – it is this area that has the highest chance of practical application through data-storage and nanotechnological biomimetics.

g) Demonstration of new forms of linear and circular polarization reflection

in several stomatopod species.

h) Characterisation of the polarization reflection properties of marine

creatures other than cephalopods and stomatopods has begun with over

20 species of fish so far.

The current grant, AOARD 11-4039 which is part of this overall project, grows from these achievements and four of these, b,d,g and h, are expanded on in this initial report as we have exciting new findings there. Findings b,d and come under original Objective 4: (Objective 4) To characterize and explain polarization optics and visual sampling

of natural stimulus fields in model organisms. A general principle of neural

sampling is that both pre-receptor and post-receptor mechanisms operate in series to

enhance signal detection. Our objective here is to examine some of the pre-filtering

mechanisms used, both optically and behaviorally. The work is expected to involve

a variety of animals to explore diverse systems of managing difficult environmental

challenges. These systems include object detection, covert signalling, and

navigation.

Finding h comes under original objective 1:

(Objective 1) To measure and explain structural properties and their diversity in

natural materials that preferentially reflect or absorb linearly and circularly

polarized light. Our research has revealed that certain animals use very unusual

structures to control the reflection, transmission, or absorption of linearly or

circularly polarized light from their surfaces or in their photoreceptors. These

structures often have no man-made counterparts, and the optics underlying

polarization absorption or reflection are poorly understood. In this project we will

examine new types of natural polarizers and polarization-sensitive photoreceptors,

characterizing their spectral, ultrastructural, and theoretical optical properties.

At the end of the report I also project towards new projects for the next 4 years for which we are currently seeking funding and which now have collaborative elements in two laboratories in the USA, one in Australia and one in the UK.

Introduction:

The ultimate goal of this research is to understand polarization vision, polarization communication and polarization camouflage (information transfer) in marine animals, with a view to using what we learn in technological applications. Spatial navigation, sensing and guidance are tasks animals behaving in the real world accomplish every day and some of this is achieved using polarized light, a form of electromagnetic radiation that humans are not capable of accessing without resorting to technology such as filters, cameras and specialized sensors. Our aim is to use the power of biological design in a realm of vision to which we are only now becoming dimly aware. To carry out this research, two laboratory groups, both of which have worked for several years in collaboration with the Air Force, have joined forces. These laboratories, one in the United States (Tom Cronin) and one in Australia (Justin Marshall), have served as research centers in the areas of visual physiology and ecology of marine and terrestrial animals, with special expertise in visual aspects of ultraviolet and polarized light.

Results 1 - Behavioural results: experiments with LCD polarisation monitors (b

and d)

The news concept behind this behavioural screening technique is as follows:

Conventional liquid crystal display (LCD) computer monitors are based on the manipulation of linearly polarised light by the electrical activation of liquid crystal filters. Light passes, first through a vertically oriented linear polaroid filter, then through an LCD array, and finally through a second horizontally oriented polaroid filter. Without electrical activation the LCD layer causes the e-vector of the incoming vertically polarised light to be rotated by 90 degrees, allowing most of it to be transmitted through the second horizontally oriented polaroid filter. However, when electrically activated, the liquid crystals reorient themselves so that the plane of polarised light is rotated to lesser amounts depending on the level of electrical stimulation. This results in different amounts of light being transmitted through the second polaroid filter.

It follows that the removal of the second polaroid filter will produce a polarisation-only monitor, in which, what was previously brightness contrast, now corresponds to polarisation e-vector contrast. In collaboration with Drs. Nick Roberts and Shelby Temple (University of Bristol) we adapted several standard computer monitors in this way to test the response of various species of marine organism to polarised light stimuli (Fig. 1).

Polarisation vision in the cuttlefish Sepia plangon

Cuttlefish from the waters near the UQ fieldwork station on Stradbroke Island were housed in glass seawater aquaria. A polarisation monitor was then placed against one side of the aquarium and fast looming circular stimuli (equivalent to an approaching predator) were presented at 3 minute intervals (Fig. 1). The angle of polarisation contrast was altered for different presentations, so that polarisation contrast varied from 90° to 0°. Cuttlefish responded to perceived looming stimuli in a number of ways, including swimming movements, skin colour and texture changes, which were recorded using digital video cameras. We developed an automated system for measuring cuttlefish responses from the digital video sequences, based on changes in the distribution of image pixel intensity values within the body region of the cuttlefish.

The remarkable result from this work is that these animals display a very sensitive, around 1o sensitivity to the direction of polarised light. This would enable them to use and analyse signals from their world in the same way as we utilise colour vision.

Fig. 1. Cuttlefish (Sepia plangon) body patterns as a behavioural assay for testing polarisation vision. (A) Schematic of testing tank and relative position and size of stimulus. (B) Body colour pattern change showing deimatic (brightening) response to potential predator, top is pre-stimulus body pattern bottom is post stimulus body pattern. (C) Time course of change in body colour pattern, stimulus onset marked by double headed arrow. Line follows the mean change in intensity value of pixels on the mantle.

This work is about to be published in the leading biological journal Current Biology and a copy of this manuscript (embargoed until 21st of this month, is attached at the end of this report. The press release that will accompany this work is copied below

and we already have interest from several mainstream science communication groups such as New Scientist and Discovery channel.

Embargoed until 21st February 12.00am AEST / 5pm GMT 20th February Australian and UK researchers have made new findings about a form of secret language in the animal kingdom using polarization, a type of light that humans cannot see. In a new paper published in Current Biology, researchers at UQ’s Queensland Brain Institute and at the University of Bristol in the UK have examined polarization vision and its significance in biological signalling. They focused on a type of cuttlefish (close relative to octopus and squid) to demonstrate how polarization could be used as an important kind of communication in the animal kingdom. The new paper shows cephalopods such as cuttlefish have the ability to see in many more directions of polarized light than previously thought. Co-author Professor Justin Marshall, who has published more 20 scientific papers on polarization, said humans had not yet developed the language to describe all the roles of polarization in nature. Professor Marshall said most people would be familiar with the concept of polarization through the use of polarized sunglasses, but polarization also had an important application in the detection of skin cancer in humans, as a viewing scope containing polarized light was a technology currently used to detect melanomas. “Our work is borrowing from millions of years of evolution, so perhaps we can learn a lot from nature in terms of better solutions to the problems mankind faces,” he said. It has been known for years that many animals have better colour vision than humans and also many have polarization vision (P-vision). They literally see things that we can’t. The polarization of light is a dimension of reality invisible to most people without specialized instruments. “Mammals and some other groups, don’t appear to have P-vision, although many parts of the animal kingdom do,” he said. “For example, other studies have found that animals such as ants and bees and even fish may used polarization to navigate. “Polarization in animals has previously just been categorized as just an unusual and interesting phenomenon but the work we’ve done in the past few years shows that animals use P-vision in the same way we use colour, to communicate with each other.” Professor Marshall said ironically animals such as cephalopods (cuttlefish, squid and octopi) and many crustaceans were colour blind. Instead they have concentrated on polarisation vision.

“They have evolved perfectly to see light we cannot see and also use polarized skin patterns to camouflage into their backgrounds, giving them an advantage over some predators who did not have P-vision. “While colour is very useful in terrestrial or shallow-water environments, it is an unreliable cue deeper in water due to the spectral modification of light as it travels through water of various depths or of varying optical quality,” he said.

“Here, polarization vision and communication has special utility and consequently has evolved in numerous marine species, as well as at least one terrestrial animal.”

Professor Marshall, of UQ's Queensland Brain Institute, last year received $962,000, including a Discovery Outstanding Researcher Award (DORA), for a three-year study of colour and polarization vision on the Great Barrier Reef.

As the tightest-packed ecosystem on the planet, coral reefs make for a competitive environment, from which has evolved unique sensory adaptations that Professor Marshall's research will investigate. His laboratory has developed a range of techniques and methods, such as underwater spectrophotometry, photography, unique behavioural tests and mathematical modeling of animal vision to understand the design of visual signals and systems in the light of ecology, behaviour and evolution.”

Polarisation vision in intertidal crustaceans – fiddler crabs

Using a very similar experimental protocol, we investigated the polarisation sense of an intertidal crustacean, the fiddler crab Uca vomeris. Because this species normally inhabits intertidal mudflats and is active during the low-tide period, we could not use a standard aquarium system. Instead, we suspended the crab on top of a polystyrene ball supported over a steady flow of compressed air, so that the crab was able to ‘walk’ freely causing the ball to rotate underneath it. We were then able to present polarisation stimuli (again the predator-like looming stimulus) of varying angular contrast to the tethered animal. The crabs responded to perceived looming stimuli, either by running away from the monitor, or by ‘freezing’ (Fig. 2).

Using this system, we found that, like the cephalopods, fiddler crabs are extremely sensitive (1-3o) to the angular contrast of polarised light (Fig. 3 and see attached Manuscript by How et al for full details).

a) b)

Fig. 2. Fiddler crab (Uca vomeris) running escape response as a behavioural assay for testing polarisation vision. (a) Schematic of testing running ball and relative position and size of stimulus. (b) Direction of runs away from polarization looming stimulus.

Fig. 3. Response of the fiddler crab Uca vomeris to varying polarisation angular contrasts.

This behavioural assay has also been used to test stomatopod crustaceans and new results confirm a very different result to that in cuttlefish and fiddler crabs in that stomatopods show very coarse sensitivity to polarisation (Fig. 4)

Fig. 4. Response of the stomatopod (mantis shrimp) Haptosquilla trispinosa to varying polarisation angular contrasts. Sensitivity threshold is around 15o.

This observation is in some ways surprising as stomatopods possess what appears to be the most complex polarisation system of animal, sampling linear polarisation with four channels and circular polarisation with two, but this is consistent with one current hypothesis. That is – both the stomatopod colour and polarisation system are not interested in fine detail but in fact seek rapid information through a scanning visual system about features in their environment. This rapid information transfer through multiple serial channels is the subject of a new proposal to AOARD and AFOSR.

Results 2 - Anatomy: a new form of polariser (g)

One objective of this project (Objective 1) has been to examine not just the visual systems of polarisation sense but also the ways in which animals manipulate polarised light. A recent publication in The Journal of Experimental Biology (Fig. 5 front cover) details this work. This Chiou et al paper is also attached with this report.

Fig. 5. A new type of polariser to nature was described from the antennal scales in stomatopod Odontodactylus scyllarus.

Biological signals based on color patterns are well known, but some animals communicate by producing patterns of polarized light. Known biological polarizers are all based on physical interactions with light such as birefringence, differential reflection, or scattering. We describe a novel biological polarizer in a marine crustacean based on linear dichroism of a carotenoid molecule. The red-colored, dichroic ketocarotenoid pigment astaxanthin is deposited in the antennal scale of a stomatopod crustacean, Odontodactylus scyllarus. Positive correlation between partial polarization and the presence of astaxanthin indicates that the antennal scale polarizes light with astaxanthin. Both the optical properties and the fine structure of the polarizationally-active cuticle suggest that the dipole axes of the astaxanthin molecules are oriented nearly normal to the antennal scale’s surface. While dichroic

retinoids are used as visual pigment chromophores to absorb and detect polarized light, this is the first demonstration of the use of a carotenoid to produce a polarizing signal. By using the intrinsic dichroism of the carotenoid molecule and orienting the molecule in tissue, nature has engineered a previously undescribed form of biological polarizer.

Results 3 - Polarising reflections from marine creatures other than cephalopods

and stomatopods (h)

While there is good evidence for specific polarisation reflections from both stomatopods and cephalopods. It is worth asking what these polarisation vision capable animals would see if they viewed other species. Is there a predatory advantage? Are there animals that ‘know through evolution’ that they must respond to this threat and camouflage in polarisation against their backgrounds?

Research developing from previous AOARD contracts (Contract – 064040) in which a polarising camera system was developed to characterise polarisation from animals in situ, has revealed both camouflage and also potential vulnerability in marine animals (Fig. 6).

Fig. 6 Polarizing cameras on the reef, current technology. Photographs of a silvered fish (Herring) returned to underwater light environment after immobilization. Photographs of the fish (left) and a polarizing and a white standard are taken through polarizing filters at 0, 45 and 90 degrees. The resulting images on the right are false colour images calculated in a pixel-by-pixel basis and showing polarization degree (or %), polarization angle (A) and polarization intensity (I). From the key-inserts in % and A, the polarizing filter can be seen to be around 60% polarizing and has its e-vector aligned along its long axis (bottom right to top left). Silvery fish are clearly capable of a very close match to all three parameters and are therefore likely to be well camouflaged in polarization as well as colour or intensity. Other fish lack this ability in one or other aspect (see inserts where mullet are able to match % but not A).

Most remarkable in these observations are firstly the close match to the background (in polarization percent and angle) that silvery fish manage and the lack of match that white fish such as mullet have in angle. While this observation is currently being confirmed, the hypothesis is that this gives us a glimpse into the polarization vision / polarization arms race that is underway in the ocean. We are currently engaged in characterizing the polarizing signatures of a number of marine species as Fig. 7 demonstrates.

Fig. 7. Imaging polarimetry of potential predators and prey of mourning cuttlefish (Sepia plangon). (A) Half beak (Hemiramphus spp.) in colour and (B) polarization contrast with false colours indicating e-vector angle. E-vector angle legend is provided in the corner of each image. (C) Black bream (Acanthopagrus spp.) in colour and (D) polarization contrast, as well as (E) high-resolution polarization contrast (155-175 degrees). (F) Shrimp (Macrobrachium) in colour and (G) polarization contrast, as well as (H) high resolution polarization contrast (135-155 degrees). (I) Tuskfish in colour and (J) in high resolution polarization contrast (140-180 degrees).

List of Publications: SE Temple, V Pignatelli, T Cook, MJ How, T-H Chiou, NW Roberts and J

Marshall (in Press February 2012) High Resolution polarisation vision in a cuttlefish. Curr. Biol.

T-H Chiou, AR Place, RL Caldwell, NJ Marshall, and TW Cronin (2012). A novel function for a carotenoid: astaxanthin used as a polarizer for visual signalling in a mantis shrimp. J. Exp. Biol. 215:584-589.

M.J. How, V.Pignatelli S.E. Temple, N.J. Marshall and J.M. Hemmi (in press 2012) High e-vector acuity in the polarisation vision system of the fiddler crab Uca

vomeris J. Exp. Biol.

Future directions

As this project is due to finish at the end of February 2013, we are already considering the next stage and building upon the results from the last three years. Below is a summary of a proposal currently underway, an attempt at re-building the stomatopod eye (Fig. 8) and understanding some of the information transfer principles behind this remarkable system.

Re-engineering the stomatopod eye, nature's most comprehensive visual sensor

ABSTRACT

Stomatopod (mantis shrimp) vision is both unique among animals and extraordinarily complex at the receptor level, comprising a total of 20 different photoreceptor types or functional input channels. 12 channels for color (including several in the UV, the whole system sampling from 300-720nm), 6 for linear polarization (4 sampling at 0°, 90°,45° and 135°) with peak spectral sensitivity close to 500nm and 2 in the UV sampling at 0° and 90°, with peak spectral sensitivity close to 350nm) and (in some species), 2 for circular polarization (Left and Right-handed), also with peak spectral sensitivity close to 500nm. Why do stomatopods sample the light that is available to them in such great detail? This research project builds on our previous work in this system and with a combination of our current state of knowledge and fresh intellectual and methodological input to the project, aims to explain the complexity of the system for the first time. Stomatopods brains are small and are unlikely to deal in multi-dimensional data sets but, in common with most invertebrates, more likely send a set of simple ‘command messages’ to the brain from the eye and outer visual system neuropils. The mechanisms that function to reduce and analyse the complexity of 20 data streams make up an important component of the research we propose here. Added leverage to approach both receptor function and data streaming / filtering comes from new collaborations that can mimic or re-engineer the stomatopod eye opto-electronically. By sharing coding principles between disciplines (engineering and biology), and through continued behavioural, anatomical, molecular and physiological investigations, we aim to decode the inner principles of stomatopod vision as well as provide spin-offs to inform more efficient and smart sensor design.

STATEMENT OF OBJECTIVES

The proposed research represents an unusually integrative effort, bringing together four world-class laboratories – two headed by senior scientists with years of collaborative experience and two by up-and-coming junior scientists with whom we have worked in recent years. The team provides strength in diverse areas of research, from molecular genetics and cell biology to advanced imaging and spectral/polarization analysis of scenes and living organisms, to optics and

optoelectronics, to fabrication of nanodevices, photosensors, and real-time analytical systems. The research will use basic biological investigation of the most complex visual sensory analyser yet found, that of stomatopod crustaceans, to learn fundamental processes of sensory analysis and neural integration and to use these findings to inspire the design of new types of imaging devices and analysers. Specific objectives are: (1) To learn the design foundations stomatopods use for optimized, rapid

processing of multi-channel information. We will examine the limits of resolution polarization vision in stomatopods, using new methods of presenting stimuli that provide precise tuning of stimulus polarization angle, degree and ellipticity. Similar approaches permit us to explore spectral resolution and see further through scattering media. Central to this objective is discovering how stomatopods scanning eye behaviour organizes and inputs multi-channel spatiotemporal information. (2) To discover how information is processed in stomatopod visual systems. At the most fundamental level we will undertake studies of the molecular and cellular basis of rapid photoreception and phototransduction (by examining the genes and cytoskeletal elements of mantis shrimp receptors). At higher levels, we will examine neural interconnectivity and processing principles at various stages of analysis in the visual system both anatomically and with electrophysiological approaches.

(3) To discover the natural complexity of visual scenes the stomatopod imaging

system has evolved to see. It is fundamental that we understand the visual information stomatopods see. This crucial component of the project takes findings generated by the above-stated objectives, with inspiration from our earlier work as well, to inform the design of artificial stomatopod inspired imaging devices. Our findings related to camouflage patterns or biological communication systems will inspire future image analysis of natural and machine vision environments.

(4) To investigate the co-evolution of visual systems, visual signals and

camouflage in the natural environment. We will proceed from the above discoveries to relate how stomatopods see and what they see to our observations that species vary in the types of signals they produce (including spectral, polarization, circular polarization, and temporal motion features). We will examine whether polarization signals are matched to properties of the polarization receptors among species, and detail the evolutionary paths of the optical devices in stomatopod eyes, polarization sensitivity and polarization signals.

0.0

0.2

0.4

0.6

0.8

1.0

300 400 500 600 700

a b c d Fig. 8 The stomatopod eye, biological and engineered.

a-c) Views at increasing magnifications of the stomatopod eye, showing the mid-band and surrounding peripheral retina. In (c), the overlays show the task of each part of the eye: Color processing occurs in 12 channels by mid-band rows 1-4, where each row has a 3-channel, stacked spectral analysis set. The dorsal peripheral region examines two different e-vectors at ±45o, while the ventral periphery, analyzes e-vectors that are horizontal and vertical. Rows 5&6 in the mid-band examine circularly polarized light (in some species). These polarization components are copied from Fig. 4.1 in “Fundamentals of Polarimetric Remote Sensing” by John R Schott, and represent the filters used to characterize the Stokes vector. This illustration in Schott’s book was produced with no knowledge of or connection with stomatopod vision. Here precisely the same diagrams can be used to represent the underlying anatomy and polarization function of the stomatopod eye-region. d) The proposed design of the re-engineered stomatopod eye at the level of the chip. Each symbol represents a nano-engineered pixel containing polarization and color information (in some simultaneously). This technology already exists within the Gruev lab, but has yet to be applied in an actual imaging system.