1 | P a g e

Optimal Design Report

Eye Robot

by Andrew Bligh

Tyler Harrington Matthew Morra

Ian Rogers Team 9

Client Contact: Dr. John D. Enderle

Biomedical Engineering Book Series Editor for Morgan & Claypool Publishers Professor of Biomedical Engineering

University of Connecticut Bronwell Building, Room 209

260 Glenbrook Road Storrs, Connecticut 06269-2247

Phone: (860) 486 5521 Fax: (860) 486-2500

Email: jenderle@bme.uconn.edu Web: http://engr.uconn.edu/~jenderle

2 | P a g e

1. Optimal Design Project 1.1 Introduction

The purpose of this project is to create an accurate and precise robotic model of the human “Horizontal Fast Eye Movement System” as described in detail by Dr. Enderle’s extensive eye movement research. The object recognition system will be the first to act when any one of the multiple LEDs of the display are lit. The image will be transferred from a camera to a computer via Bluetooth and an Arduino, where it will be analyzed to determine the horizontal position in space. After this is complete the movement parameters of the linear actuators will be computed and manipulated depending on the current location of the eyes. Once the calculation is done, the appropriate voltage will be sent to the corresponding Arduino and applied to the respective actuator, so that the object is centered in the field of vision. This process will repeat again and again until the random LED lights are turned off.

To fully mimic the horizontal fast eye movement system and Ocularmotor model; linear active stage tension generators, viscous and elastic elements will need to be implemented into the design. The active state tension generators will be modeled using Nickel Titanium memory shape alloys, which produce quick linear actuation based on input voltage. These will provide us with the correct movements at high enough speeds to appropriately model the eye movement system. The viscous element will be implemented in the design a cylindrical "snubber like" unit that will enclose a spring . The viscous portion will use kinetic friction to generate a force which will oppose the movement velocity. Likewise the elastic element, the spring will generate an oppositional force but only with respect to position. These components in conjunction with the previously mentioned electronics will make up our robotic eye system.

3 | P a g e

1.2 Subunits - Arduino Uno The Arduino is an open-source physical computing platform based on a simple i/o board and development environment that implements the Processing/Wiring language. These microcontrollers will be implemented in a few different areas of this project. One will be used to receive the Bluetooth image signal and import it into the attached computer. This microcontroller may or may not be necessary depending on the computer’s built in Bluetooth capability. Another set of Arduino’s will collect voltage values and durations from the computer and relay them to the linear actuators to initiate movement. The last Arduino will be used to control the random lighting of the LED strips implemented into a display board. - Light Emitting Diode (LED) Display The Arduino as previously mentioned will be used to randomize the order in which the LED’s will be lit. The display will be 32 inches long and placed every 5 degrees, or in 3.5 inch increments along the display length. Using the Ardunio’s IDE, a simple script to control voltage to the LED’s can accomplish this. 10 bar-segment LED’s will be used and mounted in a wood board. A pane of plexiglass will be painted with the exception of where the LED’s are located and used to cover the front of the display board. This will be both ascetically pleasing and also help to facilitate the image processing feature.

Figure 3: LED Display for testing.

Figure 2: Arduino Uno Microcontroller

4 | P a g e

- Camera The cameras that are to be incorporated into our design are the JPEG Color Cameras by LinkSprite. These cameras are advantageous specifically because of their size, with the dimensions being 32 x 32 mm. The LinkSprite Serial Color Camera (TTL) has the capability to take images up to 30 frames a second. Image resolution for the JPEGS produced is 320x240. - Bluetooth Communication

Our client Dr. Enderle has requested that the eye movement system incorporate Bluetooth technology for data transfer. By using Bluetooth we can effectively eliminate any tension forces that may have otherwise been exerted on the system by running a serial cable out of the back of the globe enclosure. The data transfer system will require a partnership between a transmitter and receiver. A Bluetooth transmitter will be mounted in each of the globes and used to transfer the JPEG images taken by the camera to a Bluetooth receiver which will be mounted on an Arduino. The Arduino will then be fed the JPEGs into a computer for processing. It should be noted that the JPEG camera serial output is not compatible with the Bluetooth transmitter directly. A RS232 to TTL converter circuit will be required to connect the camera directly to the Bluetooth transmitter. In order to accommodate these components within the globe, a custom PCB will be designed using Multisim and uploaded to Ultiboard for ordering.

Figure 4: LinkSprite JPEG Camera

Figure 5: Signal Configuration

5 | P a g e

- OpenCV OpenCV provides a wide range of programmable functions for doing image transformation. OpenCV is available in C, C++, and has a Python binding support capability. OpenCV will be utilized to acquire targets, and from the data, determine positioning information. To accomplish this, a Houge Transformation will be computed. A Houge Transformation will approximate parameters of a shape based on its boundary conditions. Because the LED’s are rectangular this transform will be very easy to apply. - Actuators

In order to model the contraction of the linear and medial rectus muscle’s, a fast and

efficient linear actuator must be employed. The MigaOne fits the project well, and despite a few drawbacks, has many unique properties that can be taken advantage of and utilized.

The MigaOne linear actuator works on the properties of the shape memory alloy NiTi. Shape memory alloys change shape drastically based on temperature. If the temperature of the alloy passes a certain lower threshold, the wire then changes crystalline structure from an Austenite to Martensite. This shape memory alloy is used as a piece of resistive wire in a closed circuit. This resistor will naturally dissipate heat based on input voltage and current based on the following equation known as Ohm’s Law.

Ohm’s Law

Voltage = Current * Resistance

With this relationship in mind and given that in increase of resistance will result in an increase in heat dissipation, it becomes apparent that a driver input can be used to directly tune the actuation speed, distance, and force by means of heating the system.

The horizontal saccade completes a 10 degree rotation in a mere 50ms, measured from the beginning of muscle contraction to the end of eye rotation and in an astounding 30ms

Figure 1: Houge Transformation of a Human hand.

6 | P a g e

below 5 degrees of rotation. Use of the MigaOne motor provides an ideal way to model this, since actuation of the MigaOne motors covers a .325” stroke in as little as 38ms. If it is found that our system needs a larger stroke, then a lever system can be implemented to sacrifice some motor accuracy for greater actuation length. The motor is capable of outputting 20 N, which is beyond sufficient for our needs. Not only do these motors comply with the technical specifications of the project, but they are also very small and lightweight (weigh of .7 ounces and dimensions comparable to a credit card). -Muscle Model

Fast actuation alone is not enough to correctly and accurately model the movement of the human eye. To simply connect the linear actuators tangentially to the globes would completely ignore the unique properties of eye musculature. In order to ensure that our model exactly follows Dr. Enderle’s horizontal fast eye movement system, we based our design of the diagram above see in figure 6. As one can observe, the active state tension generator (Fag) lies in parallel with a dash-pot and a spring (B1 and Klt, respectively). These act as modifiers to the input, but in a more realistic sense, model the elasticity and viscosity of the eye musculature. A spring acts as a force on the system that resists compression/extension (the force is proportional to the position). A dash-pot acts as a force on the system that resists compression and extension based on velocity (thus it is proportional to the system velocity). Knowing that Force = mass * acceleration and that the actuation with the input of these parameters represent the force, we can draw the following equation, using x for position and dx/dt for velocity.

Figure 6: Dr. Enderle’s Linear Homoemorphic Saccadic Eye Movement Model

7 | P a g e

When it comes to modeling these coefficients (k’s for spring constants and c’s for damping constants), springs can be easily implemented, since they are both widely available for custom order and come in an array of sizes and varying elasticity. Modeling the dash-pot, on the other hand, posed a bit of a problem that requires a unique design solution. Modeling the dash-pot using actual fluid viscosity may introduce difficulties because of the scale of our design (the size would have to be small), the accuracy required (finding the right fluid in the correct dimensional container would require a lot of testing), and the specifics of our arrangement (having the spring and dash-pot in parallel poses it’s own implementation problems). As an alternative, we plan on using another velocity dependent property, kinetic friction. Sliding friction would offer a good solution because friction could be applied to any actuating element. Negatively, this force of friction may in fact be hard to optimize. Also, since we are only interested in modeling kinetic friction, this system may add a latency to the system, as actuation will have to overcome static frictional forces before movement is achieved. This should not pose too much of a problem, since acceleration and force output of the actuator are of high magnitude in our implementation of the MigaOne linear actuators.

Equations for static and kinetic friction In the actual implementation of friction as a dash-pot in parallel with a spring, we are presented with two separate cases.

The first case is in consideration of the spring and dashpot in parallel with each other

and the linear actuator. A unique feature of the MigaOne motor system is that it already requires a spring to re-conform to it’s previous shape (relaxed state, no actuation). If we simply use a custom spring attached directly to the stroke slider as well as apply friction to the moving slider of the actuator, this should account for voigt element. The second case, is for the three instances of a spring and dashpot in parallel without any actuation. Two of these units are attached from each actuating component to a side of the eye, and an additional of voigt element is attached to the back of the globe which models the superior and inferior rectus’ influence on the horizontal system (as represented by variables Bp

Figure 7: Individual Model Elements

8 | P a g e

and Kp). This instance calls for a modular design that can be implemented for any of the dashpots.

The design for this module calls for a spring of the required constant fixed to the insides of two overlapping cups. The spring provides the necessary k constant, and depending on the materials and snug fit of the cups, which are to slide in and out to various lengths, the articulating surfaces of the cups should provide the appropriate friction. Globes The manufactured globes will need to be roughly twice the size of the human eyeball. This will allow enough space for the camera to fit inside as well as a Bluetooth chip which will send the collected image to the computer to be processed. Since the globes will be required to house these components, they will need to be made from plastic. This will allow the Bluetooth signal be picked up by the receiver, whereas a metal globe would encase and attenuate the signal. The plastic will also give the eyes a very light mass which will allow for easy actuation. The globes will also have two sleeve bearings, one on the top and bottom, through which the aluminum rods will pass through the centroid. The bearings will minimize the friction during the rotation of the eyes. Mount/Bearing/Stage The base of the stage will be made out of aluminum which will provide a very strong and light weight foundation. There will be two aluminum rods extending vertically outward from the base which will pass through the bearings of the globes. The bearings will provide frictionless rotation horizontally. 2. Realistic Constraints

Figure 8: Optimum Viscous Element Model

9 | P a g e

The human eye and its processing mechanisms have proven to be an extraordinarily fast system. Modeling such a system with many devices in constant communication with each other will certainly accumulate some latency -- likely more than desired. The Saccade takes 50ms to move from start to finish which is a very small window of operation. There are several bottle-necks that can be introduced to the system if the design is executed poorly. The largest constraint to the design would be the rate of transfer. The most time consuming aspect of our design will come from the system’s reliance on Bluetooth to send and receive messages between camera and computer. Understanding the image size is important for this reason. An image has a resolution size and is denoted by Length x Width. Multiplying these values together will give you the area of the image in pixels. Depending on the type of image you are using, each pixel contains a number of bits. RGB contains 8 bits per pixel, yielding 256 colors. Monochromatic represents 2 colors, black and white, giving you 2 bits per pixel. By multiplying the area of the image by the bits per pixel, the true size of the image can be found. At best case, the Bluetooth transmitter can transfer data at 2Mb/s. With this information we can find how much data can be transferred in 50ms. The last column in the chart below represents the percentage of an image that can be received during the Saccade. Though a 160x120 monochrome image can be sent in about a third of the Saccade time, further testing would need to be done to see if OpenCV could detect the LED in such a small, black and white image.

Figure 9: Rates of Data Transfer

Budget restraints are also a practical hindrance, as this severely narrows the number of linear actuators on the market that would fit our specifications. Also, companies are far more likely to market their motors to other Corporations who look to buy in bulk quantity. The MigaOne motor encompasses all the qualities that were required to make the actuation successful. It provides an adequate velocity for the movement, and the stroke lengths of the motors are large enough to move the eyes from 20 degrees to negative 20 degrees. Most importantly the MigaOne is cost effective, unlike most motors that are available. Though, the MigaOne does have some drawbacks. Because it is a memory shape alloy, when voltage is not being supplied to it relaxes back to its original shape. These relaxation times of these alloys are 3 to 5s. This is going to increase the waiting period between movements in order to allow for the alloy to return to steady state. 3. Safety Issues

10 | P a g e

Based on the optimal design previously described there are both electrical and mechanical dangers involved although they are very limited. The MigaOne linear actuators pose some minor issues. Firstly, these actuators and their associated drivers pose a shock hazard, with input voltage nearing 30V and current may reach high levels as well. Besides a shock hazard, the MigaOne linear actuator poses a burn/fire hazard. These motors operate off of the re-shaping of a NiTi memory shape alloy. A coiled piece of NiTi wire provides the actuation when a battery source applies voltage to the resistive wire, which dissipates heat. The wire reaches above 70 degrees Celsius in order for the NiTi to change conformation and cause actuation. By using the custom MADv5 drivers, overheating is prevented by inclusion of a heat sensitive limiter that kills power to the motor momentarily when a set temperature is reached. Proper protection and fan cooling of the motors must be used to make sure that no possibility exists for burn injury as well as component damage and fire hazard. 4. Impact of Engineering Solutions

The device detailed above provides a major step forward in physiological modeling. The human eye movement system is an evolutionarily optimized system and our understanding and replication of this system only helps to forward progress in both robotics as well as biological systems modeling. An anatomically correct and accurate model is of much use medically, as it provides a good benchmark for how biological systems should work in normal healthy individuals.

These types of eye movement models have much room for future growth and improvement/optimization. The next logical step in modeling this system is to add the functionality of the superior inferior rectus, that is, to model the eye in it’s movement on more than just the horizontal plane. Functionality could even farther increase by modeling eye movements other than the saccade such as tracking eye movements and viewing objects on the eye periphery. Perhaps the future will even yield a model that includes the neck and it’s role in the tracking of objects through eye’s field of vision. A good starting place for all f these future technologies though is rooted in our understanding and accurate modeling of the horizontal saccade. Using the same positional logic and the musculature modeling of our design could provide a jumping off point for these future technologies. Camera technology could also be greatly improved by utilizing some functionalities of the human eye. Any type of security or data collecting camera might be stand to be optimized with the precise, regular, and concise movements of the saccade. At least on a base level our model’s technology could be implemented quite easily. With the growing ease of implementation and shrinking cost of camera technologies, it is likely that our model represents at least a piece of the logical conclusion of motion camera capture technologies. The key is that our eye movement system, already optimized by biological evolution is able to make smart decisions and reactions based on the dynamic properties of a stimulus. It is apparent that the biological eye movement system is the real dominant technology in existence, and our mimicry could help make our technical world catch up in efficiency.

11 | P a g e

The medical world could also benefit from progress in this field. If, in the future, doctors were able to engineer a synthetic, implantable eye, a device like ours may not only just help in discovering relevant biological data for consideration during fabrication of the device, but also could provide insight on implementing am accurate synthetic musculature to suit that eye. 5. Life-Long Learning The development of a prototype takes a great deal of careful planning, time and effort. Before any part of the project can be started, the team must first define the parameters that the product will need to satisfy. Also a budget and timeline must be established to avoid any complications later on. From these guidelines and with the budget in mind, a set of realistic constraints can be formed based on the essential prototype capabilities. Once this is completed research on previously developed products can begin to determine the existing technology available. If the no similar projects have been attempted then extensive research will need to be done to find materials and components that will comply with the specification requirements.

After alternative technologies are found, each should be compared with one another to determine the optimal components. Once these are selected they can start being pieced together into a working prototype. During the development phase of the project lifespan, the product will need to be built and tested thoroughly to find any bugs before being approved for mass production.

The concept of teamwork is vital to the success of a project. The members of the team must always be in communication with one another and kept up to date on the overall status of development. Budget

Component Multiplicity Price

TiNi Linear Actuators 4 $158.80

TiNi Linear Actuator Drivers 4 $55.50

Arduino Uno Microcontrollers 4 $119.80

JPEG Cameras 2 $99.90

JPEG Camera Trigger 2 $59.90

Bluetooth Transmitter 2 $29.90

RS 232 to TTL Converter 1 $14.95

Bluetooth Receiver 1 $64.95

Fabricated Eye Globes 2 $100.00

12 | P a g e

Horizontal LED Strip

-10 Segment LED Bars 9 $17.55

-Framing - $50.00

Support Chassis - $200.00

Shipping Fees - $18.24

Total $971.25

Optimal Design

Figure 10: Optimum Model: Front View

13 | P a g e

Figure 11: Optimum Model: Motor Mounts