Cutting Edge Vision: Metal Embedded Optics for Smart Knives

Abstract We present a novel technique for embedding optic fibers into a metal blade to sense objects that the knife is cutting. In particular, we present a design for a kitchen knife with fiber optics between the edge of the blade and the handle, with a skin-color sensor that overcomes the complex conditions in the kitchen. Hoping this design will lead to future work on minimizing cooking injuries, our handheld device also includes a simple finger-protection mechanism in the form of a retracting blade. We present our novel hardware design, an initial study of imaging capabilities, and a discussion of future directions.

Author Keywords Smart handheld tool, kitchen, cooking, fiber optics, protection, safety, injuries, skin color detection.

ACM Classification Keywords H.5.2. Information Interfaces and Presentation: User Interfaces.

Introduction The knife is an essential kitchen tool, but it is also responsible for many injuries. We explore a new concept for a smart knife, with embedded optics that add a finger-cutting risk-detection device to the blade. We propose to augment the knife’s with an integrated optical skin sensor that overcomes the challenges of

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author. Copyright is held by the owner/author(s). CHI'15 Extended Abstracts, Apr 18-23, 2015, Seoul, Republic of Korea ACM 978-1-4503-3146-3/15/04. http://dx.doi.org/10.1145/2702613.2732700

Amit Zoran MIT Media Lab Cambridge, MA, USA The Hebrew University of Jerusalem (HUJI), Israel zoran@cs.huji.ac.il Nan-Wei Gong MIT Media Lab Cambridge, MA, USA nanwei@mit.edu Roy Shilkrot MIT Media Lab Cambridge, MA, USA roys@mit.edu

Shuo Yan MIT Media Lab Cambridge, MA, USA shuoyan@MIT.EDU Pattie Maes MIT Media Lab Cambridge, MA, USA pattie@media.mit.edu

the cooking environment while preserving the look and interaction of a traditional knife. To demonstrate the system’s potential, we implemented a protection mechanism that acts only when it detects that the cook’s fingers are at high risk for injury. In such cases, the blade will retract. Future solutions might guide the user to change the knife’s cutting plane, or alert the user with an audible alarm or visible signal.

This paper is structured around three topics. After discussing related work, we present Sharp Sight: Optic Blade for Risk Detection - our original design, fabrication methodology, and process for integrating imaging technology inside the metal blade. Blade Retraction for Finger Protection introduces protection mechanisms, including our retracting blade technology. Lastly, we discuss the limitations of the work and suggest directions for future research.

Kitchen Safety and Related Work Smart hand-held tools for fabrication present a new field of research in the human-computer interaction (HCI) community [10], and promise to transform the development of hazardous manual tools. The ultimate goal is for smart tools to autonomously detect their own usage patterns and provide guidance that protects their operators, the materials being processed, or the environment. Our smart knife is augmented with sensing technology that protects the user in a kitchen environment.

Cooking knives are unusually dangerous domestic hand tools, associated with more than 8 million reported injuries in the United States between 1990 and 2008 [9]. Previous attempts to mitigate this risk have used mobile addenda to the blades that prevent the edge of the knife from reaching the fingers [4]. Finger protectors also come in the form of plastic wearables [2] or protective gloves [8]. Outside of the kitchen, conductance sensors have allowed for the development of table saws whose blades can stop within milliseconds of approaching the user’s hand [1], and smart utility knives with retracting blades [5]. [6] has proposed an optical sensor connected to a haptic device, and demonstrated how it protects the yolk of a

Figure 1. (a) Common cutting scenario of our smart knife design, and renderings of a smart knife and its parts, (b) in a retracted blade mode, (c) or a mechanical exploded drawing.

boiled egg while a user cuts into the white. While these projects are highly relevant to our work, our proposal implements an array of such sensors within a traditional knife design.

Sharp Sight: Optic Blade for Risk Detection While capacitive or conductivity sensing are often used to detect proximity or touch, these methods are not practical in the kitchen, where the presence of foods and liquids on cutting surfaces complicate the grounding conditions. Thermal detection is also unreliable, as the temperature of human fingers and processed food may not differ significantly. In light of these constraints, we implemented a human skin-color imager, looking to the visible-light range of the electromagnetic spectrum. Because attaching cameras to the knife while maintaining a desirable field of view may significantly impact its form factor, our imaging sensor is embedded inside the metal of the blade itself. This preserves the usage and inherent operation of the knife and adds sensing exactly where it is needed - in the sharp edge.

Augmenting the Blade The design of our metal blade considers three parameters: its cutting performance, derived from the traditional design; the optical requirements for finger detection in the edge; and the restrictions of the fabrication technologies.

Our optic blade includes trenches dug into the metal, which guide the fiber optics inside the surface and contain a retracting mechanism. Plastruct FOP-30 Fibre Optics (PMMA acrylic fibers with 0.76mm diameter) were chosen over glass-based fibers for their flexibility, durability, multi-modal light transference, thickness and

considerably lower price. Pairs of fibers are threaded through dual trenches 2mm deep and lead to the sharp edge of the blade (see Fig 2(c)). One fiber carries illumination, while the other collects light reflected from objects near the blade. The trenches help prevent the light from the illumination fibers reflecting directly to their sensing pairs. The trenches are sealed with epoxy adhesive (a simple epoxy glue for the early prototypes, or AA-BOND FDA22 FDA Medical Grade Epoxy Adhesive for a food-safe blade). The fibers are collected into two coated and evenly cut bundles, which lead to a powerful light source and an imaging sensor.

Our maximum blade thickness is 3.175 mm, which is the upper thickness of chefs’ knives. Several blades were made from either aluminum or stainless steel, using a variety of fabrication technology: milling aluminum alloy 6061 by ShopBot Desktop CNC; Direct Metal Laser Sintering (DMLS) 3D printing of Stainless Steel 17-4PH (unhardened); and Ultra-Hard Wear-Resistant 440C Stainless Steel (unhardened) using wire electric discharge machining (Wire-EDM, a digital subtractive process where material is removed by discharges of current). After the blade is fabricated and the fibers are inserted into their trenches and covered with epoxy, the blade is manually sanded to remove leftover epoxy and sharpen the edge, creating an angle that properly diffuses the refractive light.

Electronics The embedded platform is based on ATmega 328P, which communicates through a FTDI230XQ-r to the computer over UART. Capacitive sensing detects the user’s grasp on the handle and activates the embedded system. A PWM output is connected to the servomotor to control the movement of the mechanical retracting

Figure 2. End handling of fiber optics (renderings): (a) A heat-shrink tube filled with epoxy holds the fiber together, trimmed with a chisel to create a flat end. (b) The tube’s end is held over the LED/CCD with a custom designed 3D printed enclosure. (c) Cross-section in blade’s trench.

release. A 3D printed enclosure, designed to separate the two bundles of fiber optics,attaches the end of one fiber optics bundle to the surface of the CCD, and the other to the surface of the LED (Figure 2(a-b)).

For illumination, we incorporated a high-power light source (CREE® Xlamp® MX-6 LEDs). Most of the examples here were collected with a cold light LED (CCT 5000K, at 97 mA). However, we tried both the cold light LED and a warm light one (CCT 2200 K, at 97 mA), and recommend incorporating varying LEDs to span a wider detection spectrum. The optical detection is performed with a CCD image sensor module (C329-SPI), connected to the microcontroller via SPI. The fiber bundle edge is attached directly to the CCD surface.

Imaging Data In order to illustrate our goal of detecting human fingers near the blade, we present raw data collected from our sensor in Figure 3. A thorough analysis of the data will be conducted in a future study; it is shown here to demonstrate its potential. Our initial experiment included ten measurements each from the fingers of eleven people with varying complexions (Asian, Caucasian, and African) and numerous food items (banana, butter, white bread, green leaf, kiwi, orange, and pear) in a fixed setup. The background image eliminates noise from ambient light sources and brings out the working fibers, and the rgb color measurements are converted to normalized-RGB (r + g + b = 1)[3] (see Figure 4). At the moment, the data is analyzed offline on a PC rather than on the embedded system.

Figure 3. Image of the fibers when approached by different objects: fingers and a number of food types. Every row represents the response of one or more fibers to the object in the column.

In addition, we experimented with measuring the distance of objects from the blade, with the intent of creating a faux depth sensor or a line scanner. We collected samples of a finger moving away from, up and down the blade to derive the correlation between the light intensity and the position of the user’s finger. Our data suggest that the current setup allows detection within the range of ~5cm.

Blade Retraction for Finger Protection One of the major challenges of our work was to design a finger-protection mechanism with a fast response time, high reliability, and repeatability. As the main contribution of our work is in the design of the optic blade, we present a proof of concept for a protection mechanism, as well as reviewing alternative directions for future research. We present a simple solution for blade retraction before discussing other possible protection technologies.

Magnetic or Mechanical Release A simple protection mechanism might disconnect the blade from the handle (using a magnetic or mechanical release), releasing the blade from the force and inertia of the cook’s hand motion. While the blade might still hit the finger, it would the momentum to penetrate, making any injury less severe. However, this solution introduces new risks, as the sharp, free metal blade might fall on the legs or hit the cook’s body. Moreover, for the reusability of the knife, we need to protect the fiber optics between the blade and the handle.

To control the motion of the released blade, we designed a simple, reusable retracting mechanism (see Figure 5) with latches to hold the blade in use. Two metal ribs clamp the blade; together with a ball-

bearings rail, they restrict the blade’s motion so it can only move backwards. Plastic ribs (Nylon12 or Delrin) between the metal ribs smooth the blade’s motion, and a small steel pin keeps it from moving until a servomotor that pulls it with nylon wires when risk is detected releases it. Two springs then push the blade away from the finger as fast as possible. This allows the blade to retract 13mm in 100ms. An additional spring helps lock the release pin back inside the blade when user slides the blade back into place.

While the current design may not be sufficient for reliable injury prevention, due to its limited retracting distance, it demonstrates our proposed direction. We believe that a mechanical or magnetic release of the blade, allowing a fast and controlled retraction, is the easiest solution. More research will be needed to determine the direction, range, and speed of retraction.

Alternative Future Directions for Finger Protection An alternative direction is to guide the user’s hand so that the motion plane of the knife will not endanger the cook’s finger. One option is to embed a programmable, gyroscopic mechanism for cutting-plane adjustment. The Traxion by Rekimoto demonstrates a future technology using a virtual force sensation to push the hand to a safe plane [7].

Technical Limitations and Future Work The design presented in this paper is constrained by our DIY fabrication abilities. With more accurate machining, the integration of the fibers into the metal can be more precise and dense, providing higher resolution for the imaging sensors. Moreover, in order to deploy the optic blade in real-world kitchens, a dedicated mass production process will be required,

Figure 4. (a)-(e) 1D normalized intensity-ratio colorspace (R/G + R/B + G/B) histograms of various food items (black) compared with a human finger (gray). (f) 2D histograms in the normalized R/G domain for better separation when 1D histogram is insufficient (such as in (e)).

such as casting steel or ceramic in a pre-made mold, or using a diamond-coated cutting disc for trenches.

Our current optics design is suboptimal: the image of the fibers is not precisely focused on the imaging sensor and the sensor itself performs numerous auto-calibration steps to even out the image. Additionally, in our current visible-light sensing, we suspect that uncooked pork or chicken meat will prove hard to distinguish from skin color without a higher-order imaging modality that includes a broader spectrum or improved sensitivity. Our system would benefit from an embedded microcontroller with stronger computational power to perform some of the calculations in real time.

Conclusion This paper presents a novel contribution to the growing pool of smart tools, integrating optic fibers inside a metal blade. More design work and user interaction research is required to implement a sufficiently safe knife, yet we believe that our solution showcases the possibilities of embedding optical sensors inside metal tips and bits. Prior to introducing the study of interaction with this new line of smart utensils, we present a way to overcome the design and fabrication challenges, and demonstrate a new sensorial blade with imaging capabilities as a first step toward that goal. Our knife shows promising capabilities in the collection of visual data, and presents an integrated finger-protection mechanism. With this work, we envision Cutting Edge Vision facilitating the design of new smart tools for activities such as cooking, which involve delicate manual skills and high risks.

Acknowledgements We would like to thank Will Langford, Ami Yaacobi, Adam Simha, Scott Greenwald, Gershon Dublon, Brian Mayton, Barmak Heshmat Dehkordi, Jennifer Jacobs, David Mellis and all the participants in the skin color detection study.

References [1] Gass, S., Chamberlain, R., Fulmer, J., Jensen, J., and Schramm, B. Motion detecting system for use in a safety system for power equipment, Feb. 12 2013. US Patent 8,371,196. [2] Jones, S. Knife guide and finger protection device, Jan. 2013. US Patent App. 13/182,699. [3] Kakumanu, P., Makrogiannis, S., and Bourbakis, N. A survey of skin-color modeling and detection methods. Pattern recognition 40, 3 (2007), 1106–1122. [4] Kim, B. Kitchen knife with finger protection Ł , June 2013. WO Patent App. PCT/KR2011/009,763. [5] MacAloney, N. Recoil - the knife that knows. https://www.quirky.com/products/603-recoil-the-knife- that-knows. [6] Nojima, T., Sekiguchi, D., Inami, M., and Tachi, S. The smarttool: A system for augmented reality of haptics. VR ’02, IEEE Computer Society (2002). [7] Rekimoto, J. Traxion: A tactile interaction device with virtual force sensation. UIST ’13, ACM (2013). [8] Salcedo, A. Finger protector kitchen gloves, Oct. 2012. US Patent App. 13/433,710. [9] Smith, G. A. Knife-related injuries treated in United States emergency departments, 1990–2008. The Journal of Emergency Medicine 45, 3 (Sep. 2013). [10] Zoran, A., Shilkrot, R., Goyal, P., Maes, P., and Paradiso, J. The wise chisel: The rise of the smart handheld tool. Pervasive Computing, IEEE (2014).

Figure 5. The retracting mechanism. (a) Normal state, where the pin holds the blade in place; (b) the pin retracts and the spring pushes the blade up; (c) the user pushes the blade back to its place, and a spring pushes the locking pin into its place.