NHK August 2017 Science & Technology Research Laboratories ... · 1.7 Media transport technologies 11 ... and social media analysis, and built a Smart Production Laboratory in our

August 2017 NHKScience & TechnologyResearch Laboratories

2016

NHK Science & Technology Research Laboratories

20

16NHK

Science & TechnologyResearch Laboratories

2016

Annual Report

Nippon Hoso Kyokai[Japan Broadcasting Corporation]

Greetings ············································· 1 Accomplishments in FY 2016 ··············· 2

Table of Contents

1 8K Super Hi-Vision ································· 4

1.1 8K Super Hi-Vision format 5

1.2 Cameras 6

1.3 Displays 7

1.4 Recording systems 7

1.5 Sound systems providing a strong sense of presence 8

1.6 Video coding 9

1.7 Media transport technologies 11

1.8 Advanced conditional access system technology 12

1.9 Satellite broadcasting technology 12

1.10 Terrestrial broadcasting transmission technology 13

1.11 Wireless transmission technology for program contributions (FPU) 15

1.12 Wired transmission technology 16

1.13 Domestic standardization 17

2 Three-dimensional imaging technology ··········································· 18

2.1 Integral 3D imaging technology 19

2.2 Three-dimensional imaging devices 21

2.3 Multidimensional image representation technology using real-space sensing 22

3 Internet technology for future broadcast services ······························· 24

3.1 Cloud-based media-unifying platform 24

3.2 Device linkage services 26

3.3 Program information utilization technology 28

3.4 Internet delivery technology 29

3.5 Security technologies 30

4 Technologies for advanced content production ··········································· 31

4.1 Content elements extraction technology 31

4.2 Content production support technology 32

4.3 “Smart Production Lab” 33

4.4 Bidirectional field pick-up unit (FPU) transmission technology 33

4.5 Wireless cameras 34

5 User-friendly broadcasting technologies ········································ 35

5.1 Information presentation technology 35

5.2 Speech recognition technology 37

5.3 Automatic audio description 38

5.4 Language processing technology 38

5.5 Image cognition analysis 39

6 Devices and materials for next-generation broadcasting ·············· 41

6.1 Advanced image sensors 41

6.2 Advanced storage technologies 43

6.3 Next-generation display technologies 44

7 Research-related work ························· 46

7.1 Joint activities with other organizations 46

7.1.1 Participation in standardization organizations 46

7.1.2 Collaboration with overseas research facilities 48

7.1.3 Collaborative research and cooperating institutes 48

7.1.4 Visiting researchers and trainees and dispatch of STRL staff overseas 48

7.1.5 Commissioned research 49

7.1.6 Committee members, research advisers, guest researchers 49

7.2 Publication of research results 50

7.2.1 STRL Open House 50

7.2.2 Overseas exhibitions 51

7.2.3 Exhibitions in Japan 52

7.2.4 Academic conferences, etc. 52

7.2.5 Press releases 52

7.2.6 Visits, tours, and event news coverage 52

7.2.7 Bulletins 53

7.2.8 Website 53

7.3 Applications of research results 53

7.3.1 Cooperation with program producers 53

7.3.2 Patents 54

7.3.3 Prizes and degrees 55

NHK Science & Technology Research Laboratories Outline ··················· 57

NHK STRL ANNUAL REPORT 2016 | 1

NHK Science & Technology Research Laboratories (STRL), the sole research facility in Japan specializing in broadcasting technology and part of the public broadcaster

NHK, is working to create a rich broadcasting culture through its world-leading R&D on broadcasting technologies.

Fiscal year 2016 saw the fruits of our persistent R&D efforts over many years. They include IEEE Milestones awarded to the High Definition Television and Emergency Warning Code Signal Broadcasting Systems that we developed and the Philo T. Farnsworth Corporate Achievement Award at the Engineering Emmy Awards, presented for the first time ever in Asia, for our decades of dedication to technological development. We also started test satellite broadcasting of Super Hi-Vision, which we have researched over the past 20 years.

At our Open House in May, we exhibited a 130-inch sheet-type 8K display to give shape to a future home display. During the Rio Olympics in August, we conducted live 8K terrestrial transmission experiments in Rio de Janeiro and Tokyo. We will continue to accelerate our R&D with the aims of realizing terrestrial broadcasting of Super Hi-Vision, a thin and lightweight large-screen display, and “full-featured 8K” to maximize 8K capability as well as enhancing our 8K program production equipment.

ICT infrastructure will be more advanced and society will be more diversified in 2020. To meet the needs resulting from these changes, NHK is implementing the NHK Corporate Plan for FY 2015-2017 to transform itself from a public broadcaster to “public media.” NHK STRL will put further effort into research on utilizing the Internet to enrich broadcasting and on “user-friendly broadcasting” technologies to achieve barrier-free broadcasting. We will also work actively on a three-dimensional television that provides natural 3D images without requiring special glasses and other technologies with a view to future broadcasting services 20 years from now.

This annual report summarizes our research results in FY 2016. It is my hope that this report will help you better understand NHK STRL’s R&D activities and enable us to build collaborative relationships that will promote R&D. I also hope it will help you utilize the results of our efforts.

Finally, I would like to express my sincere gratitude for your support and look forward to your continued cooperation in the future.

Greetings

Toru KurodaDirector of NHK Science & Technology Research Laboratories

2 | NHK STRL ANNUAL REPORT 2016

8K Super Hi-Vision NHK STRL is researching a wide range of technologies for Super Hi-Vision (SHV) in

preparation for the launch of full-scale broadcasting and the widespread use of SHV and future broadcasting services beyond SHV. We built a 130-inch large sheet-type display by combining four 4K organic light-emitting thin panels and exhibited it at the NHK STRL Open House 2016 to give a concrete shape to a future SHV display for home use. We also developed a system for the simultaneous production of high dynamic range (HDR) and standard dynamic range (SDR) video, a full-featured SHV camera that supports 120-Hz frame frequency, 8K full resolution, a bit depth of 12 bits, and HDR in compliance with ITU-R Recommendation BT.2100, and an SHV full-resolution single-chip color camera using a 133-megapixel image sensor. For future SHV terrestrial broadcasting, we demonstrated the world’s first real-time 8K terrestrial transmission using a 60-Hz HEVC real-time codec in Rio de Janeiro and Tokyo simultaneously.

→See p. 4 for details.

Three-dimensional imaging technologyWith the goal of developing a new form of broadcasting delivering a strong sense of

presence, we aim to develop a more natural and viewable three-dimensional television that does not require the viewer to wear special glasses. To this end, we made progress in our research on integral 3D imaging technologies and display devices for 3D images. We also conducted R&D on multi-dimensional image presentation technology using real-space sensing that is applicable to broadcasting live sports for utilization at the 2020 Tokyo Olympic and Paralympic Games. In our research on integral 3D display technologies, we worked to improve image quality by increasing the number of pixels and expanding the viewing zone and prototyped direct-view display equipment using a high-density 13.3-inch 8K OLED display (664 ppi).


Internet technology for future broadcast servicesWe continued researching technologies for utilizing the Internet, targeting new

broadcasting services for an era in which broadcasting and telecommunications will be tightly integrated. In our research on broadcast-linked cloud services, we began a study on a media-unifying platform that automatically selects an appropriate medium (e.g., broadcasting, the Internet) and viewing terminal (e.g., TV, smartphones) for displaying programs in accordance with the viewer’s situation and fabricated a prototype system. In our research on Internet delivery technology, we developed an MPEG-DASH player that can be used on diverse viewing terminals and made it available to commercial broadcasters and content providers through the IPTV Forum. We also began researching a TV-watching robot that provides company to people watching TV alone. Moreover, we began a survey of research on the cyber security of broadcasters.


Integral 3D direct-view display equipment using 8K display

Media-unifying platform prototype terminals

Accomplishments in FY 2016

130-inch sheet-type 8K display


Technologies for advanced content production We progressed with our R&D on advanced program production technologies, which

include production technologies for new content services and wireless transmission technologies used for program contributions such as emergency reporting and live sports coverage. In our research on technologies for transmitting SHV program contributions, we worked to increase the capacity of current field pick-up units (FPUs) and investigated a millimeter-wave-band wireless camera for SHV. We are also researching a technology for assisting program production using artificial intelligence (AI), called “smart production,” to change the working style of program production and expand the capability of producers. In FY 2016, we consolidated our research data of elemental technologies for smart production, including image analysis, speech recognition, and social media analysis, and built a Smart Production Laboratory in our laboratory.


User-friendly broadcasting technologies We are conducting research on technologies for “user-friendly” broadcasting that is

easy to listen to, view, and understand, so that all people, including those with vision or hearing impairments and non-native Japanese speakers, can enjoy broadcast content and services. We progressed with our research on sign language CG translation with facial expressions for weather information. We developed a practical system for automatically generating sign language CGs from weather forecast data distributed by Japan Meteorological Agency and launched an evaluation website for the system on NHK Online. In our research on automatic audio description and speech synthesis, we developed a system for automatically generating audio descriptions of athlete names, scores, and progress in games based on the analysis of competition data provided by Olympic Broadcasting Services, which we used during the Rio 2016 Olympic and Paralympic Games.


Devices and materials for next-generation broadcastingWe are researching the next generation of imaging, recording, and display devices and

materials for new broadcast services such as 8K Super Hi-Vision (SHV) and three-dimensional television. In our work on 3D integrated imaging devices capable of pixel-parallel signal processing, we halved the diameter of connection electrodes to 5 μm and reduced the pixel size to 50 μm square. In our research on holographic recording technology for achieving a very large capacity and high transfer rate for SHV video recording, we developed a prototype drive with a recording density of 2.4 Tbit/inch2 and a transfer rate of 520 Mbps and verified its operation with compressed SHV video. In our work on an OLED with a longer lifetime that could be used for a large SHV display for home use, we researched a device structure and materials that achieve both high efficiency and long lifetime and developed a red OLED device with an internal quantum efficiency of 100% and a lifetime of over 10,000 hours when it is continuously on.


Research-related workWe promoted our research on Super Hi-Vision and other technologies in various ways,

including through the NHK STRL Open House, various exhibitions, and reports. We also actively collaborated with other organizations and program producers. We contributed to domestic and international standardization activities at the International Telecommunication Union (ITU), Asia-Pacific Broadcasting Union (ABU), Information and Communications Council of the Ministry of Internal Affairs and Communications, Association of Radio Industries and Businesses (ARIB), and various organizations around the world. We exhibited our latest research results such as Super Hi-Vision, shortly before the start of its test broadcasting, broadcast technologies utilizing the Internet, 3D television, and “Smart Production” at the NHK STRL Open House. The event was attended by 20,371 visitors. We also held exhibitions in Japan and overseas to increase awareness of our research results.


Smart Production Laboratory

Evaluation website for weather report sign language CGs (NHK Online)

Holographic memory prototype drive for recording SHV video

STRL Open House 2016


1 8K Super Hi-Vision

NHK STRL is researching a wide range of technologies for 8K Super Hi-Vision (SHV), including video formats and imaging, display, recording, audio, coding, media transport, content protection and transmission systems. We are looking ahead to the start of the regular broadcasting and widespread use of SHV and future broadcasting services beyond SHV.

In our research on video formats, we developed a system for the simultaneous production of high dynamic range (HDR) and standard dynamic range (SDR) video and conducted demonstration experiments on HDR live program production.

In our work on imaging, we developed a full-featured SHV camera that is compliant with ITU-R Recommendation BT.2100 and supports a 120-Hz frame frequency, 8K full resolution, a bit depth of 12 bits and HDR. We also developed a full-resolution single-chip color camera using a 133-megapixel image sensor that can operate at a 120-Hz frame frequency in interline transfer. We designed a 1.25-inch, full-featured 8K image sensor with 33 megapixels that supports a 240-Hz frame frequency and a bit depth of 14 bits.

In our work on displays, we reduced the size of our 9.6-inch monitor by separating into display unit and control unit. We also improved the image quality of our projector by using high-power laser light sources that are twice as bright as conventional equipment, while halving the speckle noise that occurs due to interference of the laser light. For the future 8K display in home use, we developed a 130-inch sheet-type display by combining four thin 4K organic light emitting diode (OLED) panels and demonstrated a future living space at the NHK STRL Open House 2016.

In our work on recording, we developed a compact memory package and extended the functionality of our compression recorder. We also succeeded in real-time compression of 8K video in 4:2:0 at a 240-Hz frame frequency for the high-speed capture of SHV video.

In our work on audio, we developed an adaptive downmixing technique to generate high-quality stereo or 5.1 ch sound signals through the signal processing of 22.2 ch audio signals for the simultaneous production of program audio. We also developed a software-based codec for MPEG-H 3D audio with an eye toward 22.2 ch sound in next-generation terrestrial broadcasting. For the reproduction of 22.2 ch sound, we researched binaural reproduction using line array loudspeakers integrated with a display.

Regarding video coding, we investigated the required bit rates for 8K/120-Hz video using High Efficiency Video Coding (HEVC) and started codec development. We also developed elemental technologies for an advanced coding format for next-generation terrestrial broadcasting and proposed a way of improving intra-prediction at an international standardization meeting.

Regarding media transport technologies, we investigated the application of MMT for the IP delivery of 8K content and the synchronized presentation of multiple pieces of content. We also researched MMT technologies for next-generation terrestrial broadcasting, which include IP packet multiplexing and an IP transmission scheme for STL/TTL to enable a SFN.

In our work on content rights protection and conditional access, we contributed to the standardization of the second-generation conditional access system. Our effort led to a revision of the ARIB Technical Report (ARIB TR-B39) and the addition of specifications for combined receivers.

Regarding satellite broadcasting transmission, we worked on the establishment of a new ITU-R Recommendation for ISDB-S3(BO.2098), a transmission system for advanced wide-band satellite broadcasting. We investigated multilevel coded modulation and a way of compensating for nonlinear distortion to increase the capacity and transmission performance of a 12-GHz-band broadcasting satellite. We also researched an array-fed shaped-reflector antenna for a 21-GHz-band satellite broadcasting system as a new satellite channel and a dual-band antenna to receive both 12-GHz and 21-GHz satellite broadcasting.

Regarding terrestrial broadcasting transmission, we prototyped a modulator and demodulator that supports hierarchical transmission in which services for fixed reception and those for mobile reception are multiplexed into a single channel. During the Rio Olympic Games, we demonstrated the world’s first real-time 8K terrestrial transmission using a 60-Hz HEVC real-time codec in Rio de Janeiro and Tokyo simultaneously.

In our work on wireless transmission technologies for program contributions, we researched field pick-up units (FPUs) that use the 6/6.4/7/10/10.5/13-GHz band (microwave band) and 42/55-GHz band (millimeter-wave band) with the aim of enabling SHV live broadcasting of emergency reports and sports coverage. We also worked on the standardization of these FPUs. For the purpose of SHV mobile relay broadcasting, such as road race coverage, in the 1.2-GHz/2.3-GHz band we investigated bidirectional adaptive control and a rate-matching technique that improves reliability by adaptively controlling the coding rate of error correction codes according to the varying channel quality.

In our work on wired transmission technologies, we researched video synchronization and equipment control technologies to develop IP-based program production and program contribution systems. Regarding our channel bonding technology for cable TV transmissions of SHV, we conducted demonstration experiments using commercial CATV channels, developed a compact receiver. We also investigated baseband transmission aimed at the large-capacity transmissions that can be expected in the future.



1.1 8K Super Hi-Vision format

We made progress with our R&D and standardization activities related to the 8K Super Hi-Vision video system.

■ High-dynamic-range television

We worked on the standardization of the high-dynamic-range television (HDR-TV) video format. Our effort led to Recommendation ITU-R BT.2100 issued by the International Telecommunication Union Radiocommunication Sector (ITU-R) in July 2016(1). This Recommendation specifies that the system gamma value in the electro-optical transfer functions (EOTFs) for the display side of the Hybrid Log Gamma (HLG) system should be set according to the peak luminance of the display. To verify the effect of this setting, we conducted video production experiments at different peak luminance with 20 video engineers to compare a case where the system gamma value was varied depending on the peak luminance and a case where it was not. The results showed that changing the system gamma value according to the display peak luminance tends to reduce the dependence of the average picture level (APL) and histogram of video on peak luminance, demonstrating that it is effective for consistent video production independent of viewing conditions(2).

4K/8K test broadcasting started on August 1, 2016 and some of the programs are being offered in the HLG HDR. We contributed to the Next Generation Television & Broadcasting Promotion Forum (NexTV-F) and the Association for Promotion of Advanced Broadcasting Services (A-PAB) for their discussion on HDR operational guidelines for the video level of HDR programs.

With the aim of enabling the simultaneous production of standard dynamic range (SDR) and HDR content, we prototyped a camera capable of capturing SDR and HDR video simultaneously(3). This camera is equipped with a common optical iris used for SDR and HDR and an electronic iris for the gain adjustment of SDR. We demonstrated that controlling the iris for SDR while producing HDR video enables the simultaneous shooting of HDR and SDR video.

We built an 8K HDR live production system that covers the whole process from capture to display by connecting multiple 8K HDR (HLG) cameras, an existing video switcher, a compression recorder, a text superimposer and an HDR liquid crystal display. Using this system, we conducted experiments to capture persons and other objects with large amounts of contrast and demonstrated live video production in the HLG system (Figure 1)(4).

■ Full-featured 8K program production system

We are conducting R&D on program production equipment and systems that support a 120-Hz frame frequency with the goal of achieving full-featured 8K video production. We built a test production system for 8K/120-Hz(5) consisting of a compression recorder, a signal router(6), a waveform monitor(7), a 17-inch compact display, color grading equipment with a

down-conversion function(8) and 120-Hz time code equipment(9). Using this system, we confirmed the normal operation of the switching of 8K/120-Hz video signals, transmissions between devices, signal monitoring and the compatibility of the 120-Hz time code with conventional 60-Hz equipment. We also developed a production switcher by increasing the number of inputs and outputs of the signal router and adding image processing and transition effect capabilities. In addition, we built an 8K/120-Hz off-line video editing system. We also verified the feasibility of a program production system that uses the Precision Time Protocol (PTP), which can coordinate clocks precisely within microseconds, as the synchronizing signal(10).

■ Spatial resolution measurement methods for TV cameras

We standardized two types of measurement methods for accurately assessing the spatial resolution characteristics of TV cameras. One of them is to capture an “HDTV In Mega Cycle Chart,” a resolution chart in which bar patterns having multiple spatial frequencies are arranged spatially, and read the amplitude response of the patterns on a waveform monitor. The other method is to capture a slanted-edge chart and calculate the modulation transfer function (MTF) from the edge. We compiled guidelines for these two measurement methods of resolution characteristics in ARIB Technical Report TR-B41(11).

■ Color rendering properties of LED lighting

We conducted experiments to evaluate the color rendering properties of white LED lighting for wide-color-gamut 4K/8K production. On the basis of the results, we proposed to ARIB that the average color rendering index (Ra) of 90 or higher and the special color rendering index (R9) for red of 80 or higher be adopted for recommended values, leading to ARIB Technical Report TR-B40(12).

■ Standard test sequences

We contributed to the production of Series B of ultra-high-definition/wide-color-gamut standard test sequences provided by the Institute of Image Information and Television Engineers (ITE). We captured sport scenes and other materials that are not included in Series A with our 8K full-resolution camera and edited the images by adding scrolling subtitles.

[References](1) Rec. ITU-R BT.2100: “Image parameter values for high dynamic

range television for use in production and international programme exchange” (2016)

(2) Y. Ikeda, Y. Kusakabe, K. Masaoka and Y. Nishida: “Effect of Variable System Gamma for Hybrid Log-Gamma HDR Video Production,” Proc. IDW/AD’16, pp.1001-1002 (2016)

(3) Y. Ikeda, K. Masaoka and Y. Nishida: “A television camera to capture HDR and SDR images simultaneously,” ITE Tech. Rep., Vol.40, No.15, IST2016-28, pp.29-32 (2016) (in Japanese)

(4) D. Koide, T. Yamashita, R. Funatsu, N. Shirai, Y. Ikeda, Y. Nishida and T. Ikeda: “8K UHDTV-HDR Live-Program Production -System Construction and Trial Production-,” ITE Tech. Rep., Vol.40, No.23, BCT2016-58 (2016) (in Japanese)

(5) T. Hayashida, T. Soeno, J. Yonai, A. Iwasaki, Y. Ikeda, D. Koide, T. Yamashita, Y. Takiguchi, E. Miyashita and Y. Nishida: “Development of an 8K Production System With 120 Hz Frame Frequency,” SMPTE 2016 Annual Technical Conference and Exhibition (2016)

(6) J. Yonai, T. Yamashita and Y. Nishida: “Development of a seamless switcher for the full-featured 8K Super Hi-Vision,” ITE Annual

Figure 1. Exhibit of HDR live production (at the NHK STRL Open House 2016)



Conference, 22D-2, 2016 (in Japanese)(7) T. Soeno, Y. Ikeda and T. Yamashita: “Development of 8K Waveform

Monitor with U-SDI Signal Analyzer,” ITE Tech. Rep., Vol.40, No.14, BCT2016-34, 2016 (in Japanese)

(8) T. Hayashida and T. Yamashita: “Development of Color-grading equipment for the full-featured 8K Super Hi-Vision,” ITE Annual Conference, 22D-1, 2016 (in Japanese)

(9) T. Soeno, N. Shirai, D. Koide and T. Yamashita: “A Time Code Signal Transmission for 4K/8K Program Production with a 120-Hz Frame Frequency,” ITE Tech. Rep., Vol.40, No.14, BCT2016-46, 2016 (in

Japanese) (10) A. Iwasaki, T. Hayashida and T. Yamashita: “Study of the

Synchronization System for 8K 120fps Production,” ITE Annual Conference, 22D-3, 2016 (in Japanese)

(11) ARIB Technical Report TR-B41 1.0, “Measurement methods for resolution characteristics of television camera systems” (2016)

(12) ARIB Technical Report TR-B40 1.0, “Color rendering indexes and recommended values of white LED illuminant for UHDTV programme production” (2016)

1.2 Cameras

We are researching and developing for practical full-featured Super Hi-Vision (SHV) cameras and for imaging technologies to be employed at the upcoming Tokyo Olympic Games.

■ Full-featured SHV camera

We developed the first full-featured SHV camera conforming to ITU-R Recommendation BT.2100, which was standardized in FY 2016(1). This camera supports a high dynamic range (HDR) in addition to a 120-Hz frame frequency, 8K full resolution and a bit depth of 12 bits, which were achieved with our previous cameras. It achieved a setting of 1200% dynamic range by supporting the Hybrid Log Gamma (HLG) format of HDR. We captured various evaluation images with this camera and screened them at NAB Show 2016 and the NHK STRL Open House.

With the aim of making a full-featured SHV camera more compact and practical, we developed an 8K full-resolution single-chip color camera using a 133-megapixel image sensor (Figure 1)(2). Its camera head weighs only 6.3 kg and the CCU was downsized to the 3U size. Regarding the resolution characteristics, the camera achieved an MTF in excess of 35% with 3,200 TV lines. We also conducted a 120-Hz driving experiment in which 60-Hz image sensors were driven by inter-line scanning in two-line units in the 133-megapixel image sensor.

We previously developed a “Cube Camera,” a compact SHV camera that uses a 120-Hz, 33-megapixel single-chip image sensor. Using this camera, we captured fast-moving sport scenes such as bike races and ice hockey games for content screening at NAB Show 2017.

We also researched elemental technologies for improving camera performance. For noise characteristics, we conducted subjective evaluation experiments to investigate the required SN ratio of 8K cameras for HDR images(3). The results showed that an SN ratio of 48 dB or higher is desirable under the conditions of a viewing distance of 0.75 H, a display luminance of 1000 cd/m2, a system gamma of 1.2 and a frame frequency of 60 Hz. For the color reproduction characteristics, we investigated bit depth degradation, which is caused by the signal value clipping of high-chroma signals during color adjustment with a linear matrix. We devised a method for using signal processing to reduce the degradation and demonstrated its effectiveness through experiments. To suppress flicker in images captured at a 120-Hz frame frequency under an environment in which the brightness of lighting changes with a 50-Hz, we developed a technique by shooting at 240-Hz and reconstruct a flicker-free image. We demonstrated its effectiveness in capture experiments. For autofocus (AF) technology, we proposed a hybrid AF system that combines a sensor-based phase-difference detection AF (PDAF) with focus-aid (FA) signal, and demonstrated its effectiveness through simulations.

We are aiming to develop a general-use 8K camcoder combining a camera and recorder for the wide-spread use of

SHV cameras. In FY 2016, we developed prototype equipment for verifying the basic functions. The prototype can record 8K/60-Hz video for more than one hour into four SD cards using conventional compression recording technology.

■ Full-featured SHV image sensor

We made progress in our development of image sensors for a practical full-featured SHV camera.

In FY 2016, we designed a 1.25-inch full-featured SHV image sensor. It has 33 megapixels, each of which has a size of 2.1 μm (photodetecting area of 16.2 mm × 9.1 mm), and operates at 240 Hz. The quantization bit depth of its A/D converter is 14. We designed a new three-stage pipelined ADC architecture for the A/D converter to enable high-precision and high-speed operation. We also incorporated a noise reduction processing circuit to reduce random noise and fixed-pattern noise.

This research was conducted in cooperation with Shizuoka University.

■ High-speed SHV capturing technology

With the aim of enabling the high-speed capture of SHV video at the Tokyo Olympic Games, we made progress in our development of capture equipment, a recorder and a slow-motion player.

We began the development of experimental capture equipment that enables the high-speed capture of video with 33 megapixels at a 240-Hz frame frequency. We developed a recorder that can perform the real-time compression of 8K video in 4:2:0 at a 240-Hz frame frequency by the parallel operation of two compression signal processing boards for full-featured SHV. We are also developing a slow-motion system that can record and reproduce video simultaneously. In FY 2016, we verified its operation at 60 Hz. We also conducted experiments on capturing sports scenes such as speed skating in cooperation with Nippon Sport Science University.

■ Electronic variable ND filter

We are researching an electronic neutral-density (ND) filter for television cameras that can control incident light continuously and also locally. We previously studied the use of electrochromic (EC) materials with a wide variable range of the transmission rate from 80% to less than 1% for dimming materials and prototyped an EC element consisting of an electrolyte solution of silver nitrate (AgNO3) and copper chloride (CuCl2) inserted between ITO transparent conductive films. In FY 2016, we made the dimming element into a two-layered structure, which doubled the operation speed compared with the conventional one.

This research was conducted in cooperation with Murakami Corporation.



[References](1) K. Kitamura, T. Yasue, T. Soeno, H. Shimamoto: “Full-specification

8K Camera System,” NAB Broadcast Engineering Conference Proceedings, pp.266-271 (2016)

(2) T. Nakamura, T. Yamasaki, R. Funatsu, H. Shimamoto: “Development of a full-resolution 8K single-chip portable camera system,” ITE Annual Report, 22E-1, (2016) (in Japanese)

(3) R. Funatsu, K. Kitamura, T. Yasue, D. Koide, H. Shimamoto: “Development and Image Quality Evaluation of 8K High Dynamic Range Cameras with Hybrid Log-Gamma”, Electronic Imaging, Image Quality and System Performance XIV, pp.152-158 (2017)

1.3 Displays

We have made progress in our development of various displays that can handle 8K Super Hi-Vision (SHV) video and continued with our research on large sheet-type displays.

■ Full-featured 8K program production (display-related)

For the purpose of evaluating full-featured 8K panels, we prototyped interface evaluation equipment for 8K displays that can drive panels in low-voltage differential signaling (LVDS). Evaluations using a 55-inch 8K panel demonstrated that the panel can operate at 120 Hz and that a display gamma with different characteristics set by users can be applied in addition to those specified in BT.709 (SDR) and BT.2100 (HLG). We also reduced the size of our 9.6-inch 8K monitor that we developed in FY 2015 by separating its display unit from the power/control unit.

■ Full-featured SHV projector

We improved the image quality of our full-featured SHV projector by increasing the brightness and reducing the coherency of its laser light sources. More specifically, we increased the number of laser diodes to increase the brightness and also widened the half-power band width of light sources by adjusting the center wavelength of individual laser diodes to reduce the coherency. As a result, the projector doubled the brightness and halved the speckle contrast while covering 95% of colors in the xy chromaticity diagram of the BT.2020 color gamut.

■ SHV sheet-type display technologies

We are researching large, lightweight and sheet-type organic light-emitting diode (OLED) displays that can be rolled up and used in the home for showing SHV. An organic light-emitting element is a self-emissive display device and is suitable for a thin display because it does not require a backlight. In FY 2016, we developed a 130-inch large display by arranging four 4K OLED panels that use thin glass substrates. The display, whose thickness is only 2 mm including the back board for fixing the panel, demonstrated the feasibility of an ultrathin display (Figure 1). This display was fabricated in cooperation with LG Display and ASTRODESIGN, Inc. We plan to improve its performance such as panel luminance and 120-Hz operation and to develop a display that can show 8K images with a single panel.

1.4 Recording systems

We are researching compression recorders with the aim of developing full-featured SHV recording equipment. In FY 2016, we worked to downsize the solid-state memory package, implement support for a high-speed interface and extend the functionality of the compression recorder(1).

To reduce the size of the memory package, we investigated the use of an NVMe interface, which supports a wider band than the conventional interface. The conventional memory package uses a bundle of multiple SSDs having a 6-Gbps SATA interface, which has hampered downsizing efforts. We therefore devised a memory package structure that uses a single SSD instead of a bundle of multiple SSDs by supporting

the wide-band NVMe interface (Figure 1). This structure showed the feasibility of a compact memory package with a size reduced to about 1/7 of that of the conventional package. We also conducted experiments using a memory evaluation board and found that parallel writing into the memory chip and an increased writing block size can achieve a writing/reading speed in excess of 20 Gbps, which is equivalent to that of the conventional memory package. The memory package also has versatility as a recording medium because the NVMe interface is an open standard. On the compression recorder side, we implemented the NVMe interface on the substrate firmware that controls recording, and we verified recording and

Figure 1. 8K full-resolution single-chip camera head

Figure 1. Prototype large sheet-type display



reproduction operations with a commercial NVMe drive.To extend the functionality of the compression recorder, we

developed a decoder IP core supporting 8K ProRes, which is a codec for video editing, and incorporated it into the recorder in order to simplify the input to/output from the editing system. We also implemented external input/output of the audio and time code, which is a necessary function for a program production system.

To sustain the quality of the compressed pictures, we improved the color conversion method to make it effective for compression recording without chroma subsampling. We earlier confirmed that the use of a specific color conversion formula for suppressing noise propagation was effective for curbing image degradation when a certain color signal contains large noise. However, we later found that a conventional color conversion formula was more effective for some images. We therefore investigated a method of switching the formulas adaptively according to the identified visual feature value of the image and confirmed through simulations that the method always reduced image quality degradation after compression(2).

[References](1) K. Ogura, K. Kikuchi, T. Kajiyama, E. Miyashita: “Full specification

8K Super Hi-Vision Compression Recorder and Backup system,” ITE Technical Report Vol.41, No.5, pp.139-142 (2017 in Japanese)

(2) K. Kikuchi, T. Kajiyama, K. Ogura, E. Miyashita: “Adaptive Color Space Transforms for 4:4:4 Video Coding Considering Uncorrelated Noise among Color Components,” MMSP2016 Digest, 39 (2016)

1.5 Sound systems providing a strong sense of presence

We are researching a 22.2 multichannel sound (22.2 ch sound) system for 8K Super Hi-Vision (SHV).

■ SHV sound simultaneous production system

We are studying technologies to produce high-quality 22.2 ch sound efficiently and simultaneously while producing stereo and 5.1 ch sound.

We studied a shotgun microphone, which has sharper directivity than a conventional one, and devised a method for predicting the directivity characteristics of the microphones more accurately. Using this method, we designed a 1-m-long acoustic tube and developed a microphone using the acoustic tube(1). We also developed a microphone system consisting of compact shotgun microphones arranged in a spherical array structure (Figure 1) as a way of picking up 22.2 ch sound at a single point. The microphone achieved favorable separation between channels in a wide frequency range(2).

We developed an adaptive downmixing technique focusing on the coherence between 22.2 ch audio signals. The technique suppressed variations in loudness levels, which occur with downmixing using constant coefficients, and thus achieved downmixing that can maintain the mixing balance in 22.2 ch. We also developed an audio monitor for U-SDI signals for full-featured 8K SHV production systems.

For audio coding, we developed a software-based codec based on MPEG-H 3D Audio(3), which is the latest audio coding scheme, with the aim of supporting 22.2 ch sound in next-generation terrestrial broadcasting. The results of a subjective evaluation on the sound quality of coded 22.2 ch audio signals demonstrated that broadcast quality could be obtained at bit rates higher than 512 kbit/s(4).

■ Reproduction of converted SHV sound

We are researching technologies for the easy reproduction of 22.2 ch sound at home. In FY 2015, we formulated a design method for a reproduction controller for transaural reproduction using a line-array loudspeaker system integrated with a flat panel display (Figure 2). In FY 2016, we extended the design

method to develop separate designs according to the bandwidth and verified its effectiveness through computer simulations. We also developed a signal processor capable of real-time execution of extended signal processing in cooperation with Sharp Corporation.

We also studied ways of improving the performance of binaural reproduction. We confirmed that using loudspeakers located on the side of the listener to reproduce audio signals with a suppressed bandwidth produces a perception of sound coming from behind the listener and proposed this method as a rear enhancement filter(5). We also demonstrated that the sense of spaciousness increased when this method is combined with a simple reproduction technique in which loudspeakers are set only in front.

We investigated a method for converting 22.2 ch sound signals into those of an arbitrary number of channels in order to enable 22.2 ch sound reproduction using a home theater system with fewer channels. The results of a listening experiment showed that our proposed fixed downmix coefficients did not undermine spatial impression as much as downmix coefficients minutely calculated to maintain the direction of sound arrival.

Figure 1. Structure of compact memory package

NVMe interface

Memory controller Memory chips

Figure 1. Appearance of single-unit microphone



We also studied how to make 22.2 ch sound easier to listen to at public viewings and other places. We investigated a way of sequentially correcting the reproduction level based on the level difference between audio signals and ambient noises in each frequency band while considering the inter-channel correlation in each bandwidth of audio signals and demonstrated that this method improved the audibility of 22.2 ch sound(6).

■ Standardization

At the International Telecommunication Union Radiocommunication Sector (ITU-R), we contributed to the establishment of a new Recommendation on the sound systems and the order of channels when programs produced by various sound systems, including those for 22.2 ch sound, are transmitted and recorded(7). We also investigated measurement errors caused by the direction of a measurement microphone which occurs in adjusting the sound characteristics of a sound evaluation room or other places and compiled the results into a draft report. In addition, we contributed to revisions of related Recommendations. This includes the addition of a descriptor for specifying downmix coefficients in the Recommendation on sound metadata used by 22.2 ch and other advanced sound systems, the definition of a constant term and the clarification of a calculation method in the

Recommendation on loudness measurement methods, as well as the addition of test sound sources including 22.2 ch sound to the report on compliance material for verifying that a loudness meter meets the specifications within loudness measurement methods.

At the Association of Radio Industries and Businesses (ARIB), we contributed to a revision of the standards on 22.2 ch sound to comply with the ITU-R Recommendation(8). We also helped to revise the loudness operational guidelines to incorporate 22.2 ch test sound sources. Additionally, we revised a 22.2 ch sound standard with regard to the method of adjusting the room response as well as 5.1 ch sound production guideline. We also produced standard sound sources of 3D multichannel sound for the purpose of promoting 22.2 ch sound.

At MPEG, we worked on the incorporation of ARIB standards into an MPEG standard and thus contributed to the international promotion of the 22.2 ch sound broadcast standard. We also participated in a verification test of MPEG-H 3D audio and helped develop the latest coding scheme.

[References](1) Y. Sasaki, T. Nishiguchi, K. Ono, T. Ishii, Y. Chiba, A. Morita:

“Development of shotgun microphone with extra-long acoustic tube,” Audio Engineering Society Convention Paper 9639 (2016)

(2) Y. Sasaki, T. Nishiguchi, K. Ono: “Development of multichannel single-unit microphone using shotgun microphone array,” Proceedings of the 22nd International Congress on Acoustics (ICA2016)

(3) ISO/IEC 23008-3: 2015 (2015)(4) T. Sugimoto, T. Komori: “Audio coding of 22.2 ch audio signal using

MPEG-H 3D Audio,” Autumn meeting of the Acoustical Society of Japan, 3-7-18 (2016).

(5) T. Hasegawa, S. Oode, K. Ono, K. Iida: “Control of a sound image to the rearward direction using band-stop filters and laterally located loudspeakers,” Autumn meeting of the Acoustical Society of Japan, 3-7-11 (2016).

(6) S. Kitajima, T. Sugimoto, K. Ono: “Reproduction method of 22.2 multichannel sound in noisy enviornment considering inter-channel correlation,” IEICE Technical Report, EA2016-142 (2017)

(7) Rec. ITU-R BS.2102: “Allocation and ordering of audio channels to formats containing 12-, 16- and 32- tracks of audio” (2017)

(8) ARIB Standard STD-B59 2.0, “Three-dimensional Multichannel Stereophonic Sound System for Programme Production” (2016)

1.6 Video coding

We are researching video compression techniques for full-featured 8K Super Hi-Vision (SHV) and SHV terrestrial broadcasting.

■ Full-featured SHV video coding

We investigated the required bit rates for 120-Hz video coding for full-featured SHV broadcasting and verified the image quality. The domestic standard for SHV broadcasting (ARIB STD-B32) employs temporal scalable coding that can partially decode 60-Hz video frames from compressed 120-Hz video streams. We conducted experiments in which we varied the rate of increase in bit amount of an entire 8K/120-Hz video relative to the layer of the 8K/60-Hz video. The results of an evaluation using an objective metric showed that the increasing rate of the bit amount was 10% or less. We verified the subjective quality of 120-Hz and 60-Hz videos and confirmed no significant deterioration from original videos(1).

We researched speeding up methods of 8K/120-Hz video coding. We set appropriate block sizes and applied appropriate coding parameters to prediction methods for the differential temporal layer between 60-Hz and 120-Hz videos. This reduced

the coding computation time by 17% while maintaining coding efficiency(2).

We worked out a basic design of an 8K/120-Hz codec that uses High Efficiency Video Coding (HEVC). The encoder divides an 8K/120-Hz image into eight partitions (four in the spatial direction and two in the temporal direction) and performs encoding by using eight 4K/60-Hz equivalent HEVC encoders simultaneously. To verify the output streams and the effect on image quality caused by the partitioning, we developed simulation software of the encoder. For the decoder, which we decided to implement in software, we developed the core part for decoding processing and confirmed the successful decoding of streams created by the encoder simulation software.

We produced evaluation images for R&D on 120-Hz video coding in cooperation with NTT (Nippon Telegraph and Telephone Corporation). We made images that are helpful for evaluating the quality when they are moving at high speed.

■ Video coding for 8K program contributions

We evaluated the image quality of coded video in HEVC in order to design system parameters for wireless contribution

Figure 2. Line-array loudspeaker system integrated with a display



links, which we call field pick-up units (FPUs). We tested one to three codecs in tandem and identified through subjective quality evaluation the required bit rates in each stage to meet the quality criteria for program contributions. The results showed that the bit rates of 285 Mbps for 8K video and 145 Mbps for 4K video were required to meet the criteria of ITU-R Recommendation BT.1868 in the three-stage cascade structure, which was the toughest condition. This evaluation was conducted as part of activities of the ARIB JTG (Joint Task Group) on HEVC performance evaluation for program contributions.

■ Effort toward 8K terrestrial broadcasting

During the Rio Olympic Games (August 5-21, 2016), we conducted experimental terrestrial transmissions of 8K and held public viewings (PVs) in Rio de Janeiro in cooperation with the Brazilian TV broadcaster TV Globo. In the experiment, 8K programs were compressed to 84 Mbps by using our prototype 8K HEVC encoder (Figure 1) and then transmitted over terrestrial waves. At the reception point, the compressed stream was decoded by our prototype 8K HEVC decoder and then displayed on a 98-inch 8K liquid crystal display (LCD) with 22.2 ch loudspeakers. This experiment presented the immersive sensation of 8K and our compression and transmission technologies to many people both in and outside Brazil. We also conducted similar experimental terrestrial transmissions in Japan and screened the received programs at the Rio Olympic PV venue in STRL.

To enable 8K terrestrial transmission at a low bit rate, we developed a video processing to use before HEVC coding. This processing was partially implemented into a device in consideration of interface with 8K video player and HEVC codec. The processing increases the coding efficiency by taking advantage of the noise reduction and band limitation effects of wavelet shrinkage in the spatial domain. This research was conducted as a government-commissioned project from the Ministry of Internal Affairs and Communications titled “R&D on Advanced Technologies for Terrestrial Television Broadcasting.”

■ Next-generation video coding

In FY 2016, we developed technical elements for next-generation video coding to enable 8K terrestrial broadcasting. To improve intra prediction technology, we devised an algorithm that improves the accuracy of the intra prediction for chroma signals. We observed an improvement of the coding efficiency by referring to the prediction modes of neighboring chroma blocks without increasing computational complexity(3). We also developed a method for reducing prediction residues in motion compensated prediction. In addition, we devised a method for improving the motion vector coding used in motion compensation. An improvement of the coding efficiency was observed when controlling the derivation of predictors of the motion vector according to the shape of the block partitioning(4). We also devised a video-format adaptive technique as a new

element of coding systems. In response to the international standardization of HDR video format, we developed a method for reducing the coding degradation of HDR video that occurs in encoding process, and confirmed its effectiveness through experiments.

We proposed some of these methods to an international standardization conference on next-generation video coding technologies. We also provided HDR test sequences of the Hybrid Log-Gamma (HLG) format to promote standardization activities(5).

■ Video coding using super-resolution techniques

We completed a real-time video coding system that delivers 8K and 4K video signals simultaneously by using four pairs of 2K/4K super-resolution inter-layer prediction units with hybrid optimization criteria that can approximate perceptual image quality metric(6). This system transmits 4K HEVC video stream accompanied by ancillary data for super-resolving decoded 4K video into 8K resolution. We developed lossless coding of ancillary data, inter-unit synchronization and automatic synchronization of video and ancillary-data streams to demonstrate the connectivity of the total transmission system (Figure 2). This research was performed under the auspices of a program titled “Research and Development of Technology Encouraging Effective Utilization of Frequency for Ultra High Definition Satellite and Terrestrial Broadcasting System” of the Ministry of Internal Affairs and Communications, Japan.

We are also researching video format conversion using super-resolution techniques. We improved spatial processing by introducing a method for selecting the frequency band to be used by considering the color sampling pattern of a camera(7). We also improved temporal processing by using linear-filtering interpolation with contrast compensation in consideration of the spatio-temporal contrast sensitivity of human vision system(8). For gradational processing, we developed a method for interpolating intermediate gradation by introducing a point spread function that imitates the degradation process of an optical system.

[References](1) Y. Sugito, K. Kanda, S. Sakaida: “A Study of Required Bit-rate on 8K

120Hz Video Coding,” FIT2016, No.3, RI-004, pp.17-22 (2016) (in Japanese)

(2) Y. Sugito, K. Kanda, S. Sakaida: “A Study of Parameters for 8K 120Hz Video Coding,” ITE Annual Convention 2016, 12B-1 (2016) (in Japanese)

(3) S. Nemoto, Y. Matsuo, A. Ichigaya: “Chroma Intra Mode Predictor Based on Modes of Neighboring Blocks,” International Workshop on Advanced Image Technology 2017 (IWAIT2017) (2017)

(4) S. Iwamura, K. Iguchi, A. Ichigaya: “Partition-adaptive merge candidate derivation,” JVET-D0107 (2016)

(5) S. Iwamura, A. Ichigaya: “New 4K HDR test sequences with Hybrid Log-Gamma transfer characteristics,” JVET-E0086 (2017)

(6) T. Misu, S. Iwamura, Y. Matsuo, K. Kanda, S. Sakaida: “Real-time

Figure 1. Equipment for 8K program compressionFigure 2. Coding system using super-resolution reconstruction technologies



8K/4K Video Coding System with Super-resolution Inter-layer Prediction,” 32nd Picture Coding Symposium (PCS 2016), I-12 (2016)

(7) Y. Matsuo, S. Sakaida: “Super-Resolution Method by Registration of Multi-Scale Components Considering Color Sampling Pattern and Frequency Spectrum Power of UHDTV Camera,” 18th IEEE

International Symposium on Multimedia (ISM 2016), pp.521-524 (2016)

(8) Y. Matsuo, S. Sakaida: “Frame-Rate Conversion by Linear-Filtering Interpolation Using Spatio-Temporal Contrast Compensation,” 35th IEEE International Conference on Consumer Electronics (ICCE 2017), pp.262-263 (2017)

1.7 Media transport technologies

In our research on media transport technologies for transmitting video and audio, we are investigating MPEG Media Transport (MMT), which can be used for both broadcasting and broadband networks. We are also studying the IP delivery of 8K content, synchronized presentation technology and the use of MMT for next-generation terrestrial broadcasting.

■ Application of MMT technologies

We developed a receiver for broadcasting and broadband that can receive both 8K programs from satellite broadcasting and 8K content through IP delivery. We employed MMT as the signal format for IP content delivery as with 8K broadcasting to share the same signal processing circuit in the receiver. We conducted an experiment in which 11 channels of 8K content were delivered over a 10-Gbps-class network and any one of the channels was successfully presented at the receiver. This result demonstrated stable content delivery and reception. We also investigated a transmission protocol for the IP delivery of MMT signals and confirmed that it can handle unicast distribution as well as multicast distribution(1) and that it allows protocol conversion from MMT to HTTP(2). We exhibited these results at the NHK STRL Open House 2016, InterBEE 2016 and other exhibitions.

This research was performed under the auspices of a program titled “Research and Development of Technology Encouraging Effective Utilization of Frequency for Ultra High Definition Satellite and Terrestrial Broadcasting System” of the Ministry of Internal Affairs and Communications, Japan.

As an example of the application of MMT technologies, we developed a technique that allows multiple content such as multi-angle images to be delivered over IP and displayed on multiple devices such as TV and tablets in synchronization. We demonstrated that this technique achieves high-accuracy synchronization(3). We also studied a use case in which multiple pieces of content are retrieved from a network based on a designated absolute time and presented in synchronization with each other, and we demonstrated stored formats that can support this use case(4).

■ Next-generation terrestrial broadcasting

Aiming for next-generation terrestrial broadcasting, we researched a multiplexing scheme for IP packets that conforms to the channel coding system for terrestrial broadcasting and an IP transmission system used over studio to transmitter links (STLs) and transmitter to transmitter links (TTLs) to enable a single-frequency network (SFN) in terrestrial broadcasting. We compiled our findings into specifications and conducted verifications with a prototype device.

Part of this research was conducted as a government-commissioned project from the Ministry of Internal Affairs and Communications titled “R&D on Advanced Technologies for Terrestrial Television Broadcasting.”

■ International standardization related to MMT-based broadcasting

We offered to provide MPEG with MTT packet data and an analysis tool as a sample for implementing ISO/IEC TR 23008-13 “MMT implementation guidelines” and our offer was accepted(5). We also reported on our research on the application of MMT technologies for the IP delivery of 8K content at ITU-R SG6 and ITU-T SG16.

[References](1) S. Aoki et al.: “Delivery of 8K Content over Satellite Broadcasting

and Broadband,” International Workshop on Smart Info-Media Systems in Asia 2016, SS5-2, pp.236-241 (2016)

(2) S. Aoki et al.: “A Study on Conversion from MMT Protocol to HTTP,” Proceedings of IEICE Communications Society Conference, 2, B-6-10, p.10 (2016) (in Japanese)

(3) Y. Kawamura et al.: “Implementation of Inter-Terminal Video Synchronization Using MMT on Android,” Entertainment Computing 2016, pp.85-92 (2016) (in Japanese)

(4) K. Otsuki et al.: “A Study of Stored Formats to enable HTTP Access by designating Absolute Time,” ITE Technical Report, Vol.40, No.45, BCT2016-84, pp.17-20 (2016) (in Japanese)

(5) ISO/IEC TR 23008-13:2016: Information technology — High efficiency coding and media delivery in heterogeneous environments — Part 13: MMT implementation guidelines, 2016Figure 1. Experiment on multichannel IP delivery of 8K content



1.8 Advanced conditional access system technology

We are researching an advanced conditional access system (CAS) that provides rights protection and conditional access to Super Hi-Vision content and working toward its practical application. The advanced CAS technology uses a secure scrambling scheme and supports a CAS module software update function to ensure continuous provision and improved security.

We participated in standardization work at the Association of Radio Industries and Businesses (ARIB), which led to the

revision of Chapter 5, “Operational Guidelines for Advanced Digital Broadcasting Conditional Access System and Receiver Specifications” of Technical Report ARIB TR-B39, “Operational Guidelines for Advanced Digital Satellite Broadcasting,” in December 2016. The revised version includes the specifications of a combined receiver for advanced BS digital broadcasting, BS/wide-band CS digital broadcasting and terrestrial digital television broadcasting.

1.9 Satellite broadcasting technology

For the widespread use of 8K Super Hi-Vision (SHV), we are working on standardization and improving the performance of 12-GHz-band satellite broadcasting and also researching next-generation satellite broadcasting systems such as for 12-GHz-band satellite broadcasting with a larger capacity and 21-GHz-band satellite broadcasting.

■ Transmission system for advanced wide-band digital satellite broadcasting

We worked on the international standardization of a transmission system for satellite broadcasting and researched multilevel coded modulation and cross-polarization interference cancellation to improve transmission performance.

We worked on the international standardization of the transmission system for 8K SHV satellite broadcasting, ISDB-S3 (Integrated Services Digital Broadcasting for Satellite, 3rd generation). Our effort led to the establishment of Recommendation BO.2098 at the International Telecommunication Union Radiocommunication Sector (ITU-R) in December 2016. We also contributed to the publication of ITU-R Report BO.2397, which describes ultra-high-definition television satellite transmission experiments and the SHV test satellite broadcasting, which started in August 2016.

Regarding multilevel coded modulation, we designed 64APSK (amplitude phase shift keying) coded modulation using set partitioning in order to further increase the capacity of 12-GHz satellite broadcasting(1)(2). Computer simulations demonstrated that the required C/N was improved by 0.4 dB and 1.0 dB compared with those of DVB-S2X 64APSK and Gray

coded 64QAM (quadrature amplitude modulation), respectively, when a code rate of 4/5 was used (Figure 1).

In FY 2015, we proposed an algorithm that cancels interference of cross polarization in 12-GHz-band satellite broadcasting by receiving RHCP (right-hand circular polarization) and LHCP (left-hand circular polarization) signals, simultaneously. In FY 2016, we implemented the algorithm into hardware and verified its effectiveness.

■ Advanced satellite broadcasting systems

For the next generation of satellite broadcasting, we designed a 12-GHz-band on-board antenna and a 21-GHz-band array-fed shaped-reflector antenna. We also developed a 12/21-GHz-band dual-polarized receiving antenna and a 12-GHz-band on-board solid-state amplifier.

With the aim of increasing the capacity and service availability rate of 12-GHz satellite broadcasting, we investigated a 12-GHz-band antenna that can be mounted on a broadcasting satellite to reduce interference to other countries and increase the output of a satellite transponder. We designed shaped-reflector antennas, one with a single reflector and the other with two reflectors, and compared their radiation patterns. The results showed that the dual reflector antenna is better suited to the requirements of mountability on a launch vehicle and consistency of each radiation pattern in right- and left-hand circular polarization.

We applied shaped reflector to the 21-GHz-band on-board array-fed imaging reflector antenna. Using the aperture diameter and the number of elements as parameters, we studied ways of reducing side lobes, equalizing the RF power distribution of feed array elements and decreasing the number of elements. We designed an array-fed shaped-reflector antenna with an aperture diameter of 1.8 m and 19 elements and investigated a 21-GHz-band 300-MHz-class wide-band transponder system. The investigation results showed that the system achieves an annual service availability rate of 99.9% in Tokyo when using a satellite transmission output of 2.3 kW, a quadrature phase shift keying (QPSK) modulation scheme and a code rate of 1/2(3).

We designed a 12/21-GHz-band dual-polarized receiving antenna using a four-element microstrip array antenna for a feed antenna so that 12-GHz-band right- and left-hand circular polarization and 21-GHz satellite broadcasting can be received with a single parabolic antenna. We used a stacked substrate because the 12-GHz band and 21-GHz band have different optimum antenna substrate thicknesses. Simulation results showed that the antenna has a voltage standing wave ratio of Figure 1. Transmission performance of 64APSK coded modulation

1.E－01

1.E－02

1.E－03

1.E－04

1.E－05

1.E－06

1.E－07

1.E－08

1.E－09

1.E－10

1.E－1115.2 15.4 15.6 15.8 16.0 16.2 16.4 16.6 16.8 17.0

Bit

erro

r ra

te

64APSK coded modulation (4/5)

DVB-S2X 64APSK (4/5)Gray coded 64QAM (4/5)

C/N[dB]



1.2 or less for the 12-GHz band and 1.4 or less for 21-GHz band and can cover both broadcasting bandwidths. To verify the feasibility of a 12/21-GHz-band dual-polarized receiving antenna, we evaluated radiation patterns of an offset parabolic antenna with 45 cm aperture diameter by using design values of the feed antenna. The antenna achieved gains of 33.5 dBi for 12-GHz-band right-hand circular polarization, 33.4 dBi for 12-GHz-band left-hand circular polarization and 37.7 dBi for the 21-GHz band (right-hand), and a cross-polarization discrimination in excess of 31 dB.

In our work on the development of a 12-GHz-band on-board solid-state amplifier, we researched technologies to reduce nonlinear distortion caused by the satellite transponder with the aim of improving the satellite transmission performance of 16APSK. To improve the nonlinear characteristics of on-board amplifiers, we prototyped a high-power on-board solid-state power amplifier that has higher linearity than a traveling-wave tube amplifier. By using gallium nitride, the amplifier achieved an output power of 120 W in the 12-GHz band. We connected the output of the prototype solid-state power amplifier, which is connected to a linearizer to compensate for the nonlinearity, to a receiver equipped with a reception equalizer, and evaluated the transmission performance of 16APSK. The results demonstrated that the required C/N was 1.1 dB better than that of a traveling-wave tube amplifier. This research was funded

by the Ministry of Internal Affairs and Communications, Japan, through its program titled “Research and Development of Technology Encouraging Effective Utilization of Frequency for Ultra High Definition Satellite and Terrestrial Broadcasting System”.

To expand the transmission capacity by using 32APSK and improve the service availability rate of 16APSK, we investigated a thermal transportation method for a satellite carrying a 300-W-class traveling-wave tube amplifier or a 100-W-class solid-state power amplifier. This research was conducted in cooperation with the Japan Aerospace Exploration Agency (JAXA).

[References](1) Y. Suzuki, Y. Koizumi, M. Kojima, K. Saito, S. Tanaka: “A study on

64APSK Coded Modulation part 2 -Performance enhancement by LDPC coding rate optimization-,” Proceedings of the 2016 IEICE Society Conference, B-5-22 (2016) (in Japanese)

(2) Y. Koizumi, Y. Suzuki, M. Kojima, K. Saito, S. Tanaka: “A study on 64APSK Coded Modulation,” IEICE Tech. Rep., Vol.116, No.243, SAT2016-55, pp.51-56, Oct. 2016

(3) S. Nakazawa, M. Nagasaka, S. Tanaka: “A Design of a Compact Transponder for a 21-GHz Band Broadcasting Satellite using an Array-fed Shaped Reflector Antenna,” Proceedings of the 2017 IEICE General Conference, B-1-1 (2017) (in Japanese)

1.10 Terrestrial broadcasting transmission technology

For the terrestrial broadcasting of Super Hi-Vision (SHV), we are researching a next-generation terrestrial broadcasting system, a channel plan and transmission network design.

■ Next-generation terrestrial broadcasting system

We are in the process of establishing a preliminary standard for the purpose of migrating the current terrestrial broadcasting services to the next-generation system. In FY 2016, we developed detailed specifications for a hierarchical transmission system that multiplexes the services for fixed reception and those for mobile reception into a single OFDM modulation signal and prototyped a modulator and demodulator. We improved the frequency interleave of the mobile reception

layer, added a power boost function and verified the effectiveness. We also optimized the pilot signal configuration for mobile reception and confirmed a better required C/N and speed tolerance than the current ISDB-T (Integrated Services Digital Broadcasting - Terrestrial) (Figure 1) (1).

Future TV receivers are expected to receive broadcasting signals not only from conventional terrestrial and satellite broadcasting but through various channels such as the Internet and Wi-Fi networks. To increase compatibility with the Internet, we developed an MMT (MPEG Media Transport) that supports hierarchical transmission using multiple layers and implemented it into a prototype system. We also investigated a mechanism for retransmitting received broadcasting signals to a Wi-Fi network in a room or vehicle so that broadcast content can be provided to terminals in places where terrestrial broadcasting cannot be received directly.

■ Advanced technologies for terrestrial broadcasting

We are researching terrestrial SHV broadcasting under the auspices of the Ministry of Internal Affairs and Communications, Japan as part of its program titled “Research and Development for Advanced Digital Terrestrial TV Broadcasting System”, which started in FY 2016. Using the constellation, forward error correction and MIMO system compliant with the preliminary standard, we began developing a modulator and demodulator that are capable of a larger transmission capacity and hierarchical transmission.

As part of this research, we plan to set up an experimental transmission station in Tokyo. In FY 2016, we developed transmission and equipment specifications for this large-scale experimental station. We also investigated the permissible value of interference caused by radio waves from the experimental station because the test broadcasting signal must not affect current terrestrial TV broadcasting receivers used in the same frequency band.

This research was conducted in cooperation with Sony Figure 1. Comparison of speed tolerance between the preliminary standard and ISDB-T

30

25

20

15

10

5

0250200150100500

requ

ired

C/N

[dB

]

ISDB-T

Preliminary standard

Maximum Doppler frequency [Hz]

( ): FEC code rate( ): FEC code rateQPSK(1/2)QPSK(1/2)

16QAM(1/2)16QAM(1/2)



Corporation, Panasonic Corporation and Tokyo University of Science.

■ Transmission performance evaluation using terrestrial broadcasting propagation characteristics

We evaluated the system performance in a multipath environment with the signals (256QAM) for the fix reception by using a preliminary-standard modulator and demodulator that we prototyped in FY 2015. We prepared a multipath environment for laboratory experiments by simulating the terrestrial broadcasting propagation channels measured around our laboratory and compared the degradation of the required receiver input power between the preliminary standard and the existing ISDB-T format (2). The results demonstrated the superior performance of the preliminary standard, which exhibited less degradation than ISDB-T (Figure 2).

■ Channel planning

In FY 2016, we began a study in cooperation with the Engineering Administration Department to estimate the required scale of frequency reallocation (repacking) of existing digital terrestrial broadcasting, which will be necessary when

allocating new channels for SHV nationwide.

■ R&D on effective use of frequency

We have been researching single frequency network (SFN) technology to use frequencies more efficiently through a government-commissioned project from the Ministry of Internal Affairs and Communications titled “Research and Development of Technology Encouraging Effective Utilization of Frequency for Ultra High Definition Satellite and Terrestrial Broadcasting System,” which started in FY 2014. In FY 2016, we conducted transmission experiments within the service area of the experimental stations in Hitoyoshi City, Kumamoto Prefecture, to compare the performance of a space-time coding (STC)-SFN, a space-frequency coding (SFC)-SFN and a conventional SFN. The results showed that the STC-SFN and SFC-SFN had about 2 dB better required received power than the conventional SFN. We also confirmed that the SFC-SFN had an equal level of reception characteristics to those of the STC-SFN in the propagation environment of Hitoyoshi City (Figure 3) (3).

We also conducted a comprehensive connection experiment by combining STC-SFN technology with an 8K HEVC decoder. The results demonstrated that 8K video could be received successfully in an SFN environment about 18 km and 20 km away from the Hitoyoshi station and the Mizukami station, respectively.

■ Collaboration with overseas organizations

During the Rio Olympic Games, we demonstrated 8K terrestrial live transmissions using the world’s first HEVC real-time codec in Rio de Janeiro and Tokyo simultaneously. In Brazil, the 8K content was transmitted from an experimental station, where our modulator was connected to the transmission equipment of the Brazilian TV broadcaster TV Globo. The transmitted signals were received at a demonstration venue approximately 8 km away for live public viewing of the Games. We also conducted field experiments in Rio de Janeiro in parallel with the demonstration. The results were reported at SET EXPO and added to ITU-R BT.2343, a report describing the results of terrestrial field experiments on ultra-high-definition television around the world.

At the Digital Broadcasting Experts Group (DiBEG), which promotes ISDB-T internationally, we updated a joint document of Associação Brasileira de Normas Técnicas (ABNT) Brazilian technical standards in cooperation with broadcasters in Brazil and other South American countries. We also investigated and reported trends toward next-generation terrestrial broadcasting of other countries at the Japan-Brazil next-generation broadcast study task force.

We dispatched one of our researchers to the Polytechnic University of Valencia in Spain to research the use of MIMO for broadcasting and investigate the trend toward next-generation terrestrial broadcasting in Europe.

[References](1) H. Miyasaka, A. Sato, S. Asakura, T. Shitomi, S. Saito, Y. Narikiyo, T.

Takeuchi, M. Nakamura, K. Murayama, M. Okano, K. Tsuchida, K. Shibuya: “A Study on the Scattered Pilot Patterns for Mobile Reception in the Next Generation Terrestrial Broadcasting,” ITE Technical Report, Vol.40, No.30, BCT2016-68, pp.5-8 (2016) (in Japanese)

(2) A. Sato, T. Shitomi, T. Takeuchi, M. Okano, K. Tsuchida: “Transmission Performance over Multipath Environment of Proposed Specification for the Next Generation Terrestrial Broadcasting - Comparison with ISDB-T by Indoor Experiment -,” ITE Technical Report, Vol.40, No.45, BCT2016-82, pp.5-9 (2016) (in Japanese)

(3) S. Saito, T. Shitomi, S Asakura, A. Satou, M. Okano, K. Murayama, K. Tsuchida: “4x2 MIMO Field Test of Advanced SFN Using Space Time Coding For 8K Transmission,” IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB 2016), IEEE, 10A-2 (2016)

Figure 2. Comparison of the degradation of transmission characteristics in a multipath environment

5

4

3

2

1

0543210

ISDB-T is betterISDB-T is better

Preliminary standard is betterPreliminary standard is better

Degradation of required receiver input power in ISDB-T [dB](64QAM, convolutional code, 3/4 code rate)

Degradation of required receiver input power in ISDB-T [dB](64QAM, convolutional code, 3/4 code rate)

Deg

rada

tion

of r

equi

red

rece

iver

inpu

t pow

er

in th

e pr

elim

inar

y st

anda

rd [d

B]

(256

QA

M, L

DP

C c

ode,

2/3

cod

e ra

te)

Deg

rada

tion

of r

equi

red

rece

iver

inpu

t pow

er

in th

e pr

elim

inar

y st

anda

rd [d

B]

(256

QA

M, L

DP

C c

ode,

2/3

cod

e ra

te)

Figure 3. Field measurement results of STC-SFN and SFC-SFN

1211109876543210-1

30252015100 5

Req

uire

d re

ceiv

ed p

ower

diff

eren

ce b

etw

een

conv

entio

nal S

FN

and

adv

ance

d S

FN

[dB

](R

equi

red

rece

ived

pow

er o

f con

vent

iona

l SF

N/

Req

uire

d re

ceiv

ed p

ower

of a

dvan

ced

SF

N)

Req

uire

d re

ceiv

ed p

ower

diff

eren

ce b

etw

een

conv

entio

nal S

FN

and

adv

ance

d S

FN

[dB

](R

equi

red

rece

ived

pow

er o

f con

vent

iona

l SF

N/

Req

uire

d re

ceiv

ed p

ower

of a

dvan

ced

SF

N)

Field measurement result (STC-SFN)Field measurement result (STC-SFN)Field measurement result (SFC-SFN)Field measurement result (SFC-SFN)Computer simulationComputer simulation

Received power ratio of main waves and SFN waves in receiver input [dB]Received power ratio of main waves and SFN waves in receiver input [dB]

All 25 dataAll 25 data

1.8 dB (average)1.8 dB (average)

1.8 dB (average)1.8 dB (average)2.0 dB (average)2.0 dB (average)2.0 dB (average)2.0 dB (average)

Received Power ratio of main wave and SFN wave is between 5-12 dBReceived Power ratio of main wave and SFN wave is between 5-12 dB

Improvement made by STC-SFNImprovement made by STC-SFN

Improvement made by SFC-SFNImprovement made by SFC-SFN



1.11 Wireless transmission technology for program contributions (FPU)

With the goal of realizing Super Hi-Vision (SHV) live broadcasting of emergency reports and sports coverage, we are conducting R&D on field pick-up units (FPUs) for transmitting video and sound materials. In FY 2016, we researched a microwave-band FPU, a millimeter-wave-band FPU and a 1.2-GHz/2.3-GHz-band FPU.

■ Microwave-band FPU

We researched a microwave-band FPU that can transmit 8K SHV signals at a 200-Mbps-class transmission rate over a distance of 50 km and worked toward its standardization. This FPU achieves a much larger transmission capacity in the 18-MHz transmission bandwidth, which is also used by current Hi-Vision FPUs, by using dual-polarized multiple-input multiple-output (MIMO) technology and orthogonal frequency division multiplexing (OFDM) with a higher-order modulation scheme (Figure 1).

In FY 2016, we upgraded the low-density parity-check (LDPC) codes and the OFDM frame structure of our prototype modulator and demodulator to develop a practical FPU. For the LDPC codes, we employed the 44,880-bit-length codes used in the transmission system for advanced wide-band digital satellite broadcasting (ARIB STD-B44). The new frame structure allows one OFDM frame to contain an integer number of LDPC code blocks. With an optimized number of data carriers and frame length, the structure enabled efficient transmissions. We also contributed to the establishment of a national standard and the standardization at the Association of Radio Industries and Businesses (ARIB) so that microwave-band FPUs can be put into practical use in 2020.

■ Millimeter-wave-band FPU

We researched a millimeter-wave-band (42-GHz-band) FPU that can transmit 8K SHV signals at a 400-Mbps-class transmission rate and promoted standardization activities. In FY 2016, we fabricated a MIMO-OFDM modulator and

demodulator that support a 125-MHz transmission bandwidth, twice the conventional bandwidth (62.5 MHz)(1). By combining it with an RF front-end unit and dual-polarized antenna that we fabricated in FY 2015, we completed a prototype millimeter-wave-band FPU (Figure 2).

We confirmed through laboratory experiments that the FPU can achieve a maximum transmission rate of 600 Mbps (with a 32QAM subcarrier modulation scheme and 3/4 coding rate). We also contributed to a revision of the ARIB standard on a portable millimeter-wave digital transmission system for television program contribution (ARIB STD-B43) in preparation for its practical use. In addition, we conducted field transmission experiments over a distance of 8 km between the NHK Broadcasting Center in Shibuya and our laboratory by connecting a millimeter-wave-band FPU with an H.265/HEVC codec that the Engineering Administration Department developed in FY 2016 for transmitting 8K program contributions. The results proved the feasibility of 8K SHV signal transmissions.

■ 1.2-GHz/2.3-GHz-band FPU

To enable the mobile relay broadcasting of SHV signals by using the 1.2-GHz/2.3-GHz-band, we are researching a MIMO system with adaptive transmission control using the time division duplex (TDD) scheme.

In FY 2016, we confirmed that it is possible to sufficiently reduce the influence of feedback delays caused by TDD even in a time-varying channel similar to that used to broadcast a relay road race(2). We applied space-time trellis coding to the system we prototyped in FY 2015 (Figure 3) in order to increase the robustness of down links.

We also investigated rate matching, which controls the code rate of error correction coding adaptively according to the varying channel quality. We used concatenated codes of a turbo code and a Reed–Solomon (RS) code for forward error correction. Bit puncturing after turbo coding controlled the coding rate between 0.33 and 0.92 depending on the channel quality. Computer simulations using the received data in a radio wave propagation experiment carried out on the Kyoto Ekiden (a marathon relay race) course showed that adaptively controlling the coding rate can prevent a transmission error even when the reception quality deteriorates(3).

Additionally, we prototyped 2.3-GHz-band power amplifiers for field transmission experiments of this system and obtained an experimental radio station license.

Part of this research was conducted as a government-commissioned project from the Ministry of Internal Affairs and

Figure 1. Microwave-band FPU prototype

Figure 2. Millimeter-wave-band FPU prototype Figure 3. Prototype MIMO system with adaptive transmission control



Communications titled “R&D on Highly Efficient Frequency Usage for the Next-Generation Program Contribution Transmission.”

[References](1) J. Tsumochi, F. Ito, Y. Matsusaki, H. Kamoda, T. Nakagawa, H.

Hamazumi: “Development of Wideband Modulator and Demodulator using a MIMO-OFDM Technique for 42 GHz band FPU (in Japanese),”

ITE Technical Report, Vol.40, No.23, BCT2016-64 (Jul. 2016)(2) F. Ito, F. Uzawa, K. Mitsuyama, T. Kumagai, N. Iai: “A Study on TDD

Feedback Scheme for Adaptive Control SVD-MIMO System,” 2017 IEICE General Conference (2017) (in Japanese)

(3) F. Uzawa, F. Ito, K. Mitsuyama, T. Kumagai, N. Iai: “A Study on Rate-matching Applicable to Field Pick-up Units for Mobile Relay,” 2017 IEICE General Conference (2017) (in Japanese)

1.12 Wired transmission technology

We are researching an Internet Protocol (IP) based program production and program contribution system that can be used for 8K programs. We are also studying a channel bonding technology for transmitting 8K programs over cable TV networks and the FTTH digital baseband transmission system.

■ IP based program production and program contribution system

We are conducting R&D on a system that applies IP technology to program production and program transmission for a low-cost and efficient workflow. The system is expected to have various advantages by using widespread IP technology, such as allowing the use of high-speed inexpensive IP devices(1) and transmitting signals of various formats for video, sound, synchronization and control over a single IP network.

In FY 2016, we evaluated the synchronization performance of the Precision Time Protocol (PTP), a synchronization technique standardized by IEEE1588, as a method for synchronizing video devices connected to an IP network. PTP synchronizes the clock between devices by exchanging time information (PTP packets) between a master device and slave devices and adjusting the time of the slave devices to the high-precision clock of the master device. The clock synchronization performance is affected by the transmission latency of PTP packets, and it varies by the number of Ethernet switches connected between the master and slave devices. We therefore conducted experiments and quantified the relationship between the number of connected switches or the transmission rate of data passing through the switches and the clock synchronization performance. We plan to utilize the findings for future system design. We also implemented the function of PTP clock synchronization into IP transmission equipment for uncompressed 8K signals and confirmed that the equipment is capable of the stable transmission of uncompressed 8K signals.

The use of IP technology for program production enables various capabilities such as automatically detecting a device connecting to the IP network and controlling the connection. We participated in activities at the Advanced Media Workflow Association Networked Media Incubator (AMWA NMI), an international organization that discusses and standardizes connection management methods of IP program production systems, and built a test environment in our laboratory for verifying the connection management methods being

discussed at the AMWA NMI. To allow the use of connected devices from any of the studios connected over the IP network, we investigated a new mechanism for preventing the contention of device connection management. We implemented this mechanism into our test environment and verified its operation by referring to the control API format adopted by AMWA NMI.

■ Cable TV transmission of 8K signals

We continued with our R&D on a channel bonding technology to transmit partitioned 8K signals over multiple channels so that 8K programs can be distributed through existing coaxial cable television networks. In FY 2016, we conducted retransmission experiments on 8K satellite broadcasting by using commercial cable networks and demonstrated the feasibility of 8K program retransmission via existing cable TV networks(2). Viewing 8K satellite broadcasting through cable TV also requires a compact and affordable receiver. To address this need, we developed the world’s first compact receiver (tuner) equipped with a demodulator LSI that supports channel bonding technology (Figure 1). Using the same output signals as the demodulated signals of 8K satellite broadcasting, video and audio decoder LSIs for satellite broadcasting receivers can also be used for cable TV receivers.

■ Digital baseband transmission system for FTTH

As a way of distributing broadcasts to homes using FTTH (Fiber to the home), we are studying a digital baseband transmission system. In this system, multiple streams of 8K and Hi-Vision broadcasting are multiplexed with 10-Gbps-class baseband signals by using time division multiplexing (TDM) and transmitted over optical fibers. It can reduce the cost of large-capacity transmission significantly compared with conventional RF transmission. In FY 2016, we prototyped transmission equipment capable of mitigating the difference in transmission latency, which occurs when many streams are multiplexed. We conducted a performance evaluation and demonstration. We also investigated a way of migrating the existing FTTH equipment for RF signal transmission to a digital baseband transmission system in stages.

[References](1) J. Kawamoto, T. Kurakake: “Uncompressed 8K Ultra-high Definition

Television Transmission over 100G Ethernet in Broadcasting Station,” OFC2017, M2I. 5 (2017)

(2) K. Uezono, K. Nakajima, H. Matsumoto, Y. Hakamada, T. Kusunoki, T. Kurakake: “Retransmission trial of 8K UHDTV satellite broadcasting over an existing commercial CATV line,” Proc. of the IEICE Society Conference, B-8-35 (2016) (in Japanese)

Figure 1. Compact receiver



1.13 Domestic standardization

We are engaged in domestic standardization activities related to 4K and 8K ultra-high-definition television satellite broadcasting systems.

In November 2015, the Broadcasting System Subcommittee of the Information and Communications Council's Information and Communications Technology Sub-Council in the Ministry of Internal Affairs and Communications (MIC) began a study on the technical conditions of high dynamic range (HDR) television with the aim of increasing image quality. Following a proposal made by the Association of Radio Industries and Businesses (ARIB) on the video format, coding scheme and identification method of HDR, the Information and Communications Council published a report titled “Technical conditions for ultra-high-definition television broadcasting systems with higher image quality” in May 2016(1). On the basis of this report, a Ministerial Ordinance of MIC (a national technical standard) was revised in July 2016, which enabled HDR broadcasting in ultra-high-

definition television. ARIB also worked on revisions of its technical standards that specify the television systems in cooperation with the Next Generation Television & Broadcasting Promotion Forum (NexTV-F) and the Association for Promotion of Advanced Broadcasting Services (A-PAB) that develop operational guidelines (Table 1).

Members of NHK STRL contributed to these standardization efforts on ultra-high-definition television broadcasting by participating as members of the Information and Communications Council working group, committee chairs of ARIB development sections, and managers and members of ARIB working groups.

[References](1) FY 2016 Information and Communications Council Report, No. 2023,

May 24, 2016

Table 1. Major revisions of the ARIB standards for ultra-high-definition television satellite broadcasting systems

Domain ARIB Standard Major revisions

Transmission system STD-B44Naming of the transmission system for advanced wide-band digital satellite broad-casting as “ISDB-S3”, Incorporation of the results of performance evaluation using an operational broadcasting satellite

Multiplexing (MMT/TLV)

STD-B60Addition and modification of descriptors related to application control and multime-dia services

Conditional access STD-B61 Clarification of the unique information of respective receivers in conditional access

Video coding STD-B32 Part 1Incorporation of the Ministerial Ordinance for HDR, Clarification of operational guidelines for HEVC

Multimedia coding STD-B62Upgrade of the multimedia coding reference model to support HDR, Addition of the symbols for the electronic program guide (EPG), Addition of an information acquisi-tion capability for conditional access

Receiver STD-B63Revision of the recommended performance of optical transmitters and receivers for distributing intermediate-frequency signals for satellite broadcasting reception, Addition of items referring to STD-B32 Part 1 regarding HDR


2 Three-dimensional imaging technology

With the goal of developing a new form of broadcasting, we are pursuing the technology of a three-dimensional (3D) television that shows more natural 3D images to viewers without special glasses. We conducted comprehensive research on 3D imaging which include image capturing and displaying technologies, coding methods and required system parameters for the integral 3D, and display devices development for 3D images. In parallel, we made progress in our R&D on a new image representation technology using real-space sensing that is applicable to live sports broadcasting. We aim to utilize the technology during the 2020 Tokyo Olympic and Paralympic Games.

In our research on display technologies based on the integral 3D method, we are developing basic technologies to increase the number of pixels and expand the viewing zone. In FY 2016, we developed a direct-view display system that combines 4 images produced from four HD liquid crystal panels with a new optical setup. The system reduced the noise and increased the resolution of displayed images. We also built prototype system using five high-definition projectors, which increased the resolution and viewing-zone angle (horizontal and vertical) to approximately 114,000 pixels and 40 degrees, respectively. Moreover, we fabricated direct-view display system that uses a high-density 13.3-inch 8K OLED display (664 ppi).

The MPEG Free-Viewpoint Television (MPEG-FTV) ad hoc group have started its activities in 2013 to standardize new coding technologies for 3D images, and we have been participating in this group. In FY 2016, we demonstrated coding experiments applying 3D High Efficiency Video Coding (3D-HEVC), which is a conventional multiview coding method, into the integral 3D images. We have successfully compressed the integral images using the 3D-HEVC by converting elemental images in the integral images into multi-viewpoint images which can be compressed by the conventional multiview coding method (3D-HEVC).

In our research on image capturing technologies of the integral 3D method, we are studying technologies to obtain spatial information by using multiple cameras and lens arrays for creating high-quality 3D images. In FY 2016, we developed an integral 3D model-based capture technology that generates 3D models of an object from multi-viewpoint images captured by multiple robotic cameras and converts them into elemental images. We have demonstrated how this technology generates a 3D point cloud model of a photographed image and converts it into elemental images by using seven robotic cameras positioned in a hexagonal arrangement.

In our research on the system parameters of the integral 3D method that we started in FY 2015, we are investigating the relation between the display parameters (in terms of the focal length of the lens array and the pixel pitch of the display device, and so on) and image quality (in terms of the depth-reconstruction range, resolution and viewing zone) through subjective evaluations of the simulated integral 3D images. In FY 2016, we developed a nonlinear depth-compressed expression technology that compresses the depth of 3D scenes and performs a no-blur and high-quality 3D visualization on an integral 3D display with a limited depth reconstruction capabilities without inducing a sense of unnaturalness in viewers by taking advantage of the characteristics of human depth perception. The results of the subjective evaluations showed that the amount of unnaturalness was acceptable even when a 3D space in excess of 100 m was compressed into a depth range of 1 m by using this technology.

In our research on 3D display devices, we have been studying electronic holography devices and beam-steering devices. For electronic holography, we continued to study spatial light modulators using spin transfer switching (spin-SLM). In FY 2016, we prototyped an active-matrix-driven spin-SLM (100×100 pixels) using a tunnel magnetoresistance element with a narrow 2-μm pixel pitch, and successfully demonstrated displaying 2D images. With the aim of building an integral 3D display that uses beam-steering devices instead of a lens array, we studied an optical phased array that uses an electro-optic polymer. In FY 2016, we designed and prototyped an optical phased array consisting of multiple channels and demonstrated the operation of one-dimensional beam steering.

In our research on multidimensional image representation technology using real-space sensing, we are developing technologies for new image representation techniques using real-space sensing such as a high-speed object tracking system using a near-infrared camera, a method to estimate the head pose of a soccer player and a studio robot for joint performance with CGs; four-dimensional spatial analysis and image representation of sport scenes that combines multi-viewpoint robotic cameras, object tracking technology and 3D information analysis; a 2.5D multi-motion representation technique that enables time-series image presentation; and a naturally enhanced image representation method for a flying object to generate the trajectory of a golf ball in real time. In FY 2016, we verified the effectiveness of each basic system through prototyping and field experiments and identified issues to be resolved toward practical use.



2.1 Integral 3D imaging technology

■ Improvement of integral 3D image quality

We are researching spatial 3D imaging technology as a way of creating a 3D television that reproduces more natural 3D images and does not require special glasses. The integral 3D method, which is a spatial imaging technology, can reproduce natural 3D images in accordance with the viewer position by faithfully reproducing light rays from objects. In FY 2016, we worked to improve the quality and resolution of integral 3D images.

We have been researching a 3D imaging technology that synthesizes the screens of multiple direct-view display panels in order to increase the number of pixels of integral 3D images. The conventional screen synthesis method, however, had an issue of quality degradation of the produced 3D images because it required an optical system for combining multiple images with a complex lens structure and a diffuser plate (Figure 1 (a)). In FY 2016, we devised a new optical system for combining multiple images with a simple lens structure and eliminated the need for a diffuser plate. Using a prototype of the new optical system, we developed display equipment with four HD-resolution liquid crystal display (LCD) devices. The equipment displayed 3D images with less noise and higher resolution than the previous equipment (Figure 1 (b)). The depth of the display equipment using the new optical system was also reduced to less than 1/5 that of the conventional prototype(1).

In FY 2016, we also developed a 3D display method that can increase the resolution of 3D images and expand the viewing angle by using multiple high-definition projectors. In this method, each projector placed in its optimum position superimposes projected images onto a lens array at a certain angle by using elemental images as parallel rays. This improves the 3D image resolution characteristics and viewing-zone angle characteristics(2). We also developed a technology to enable automatic and high-precision adjustment of the

projection position of elemental images and the lens array position(3). Our prototype equipment using five high-definition projectors achieved a resolution of approximately 114,000 pixels for a front view and a viewing-zone angle of 40 degrees in the horizontal and vertical directions, which is about 1.5 times as wide as that of the conventional equipment. We exhibited the equipment at the NHK STRL Open House 2016.

With the goal of developing a direct-view 3D display, we applied a high-density 8K direct-view panel for 3D imaging. We displayed elemental images on a 13.3-inch OLED display (664 ppi) produced by Semiconductor Energy Laboratory Co., Ltd., and designed the optimum lens array for the pixel structure to build a prototype of 3D display equipment. The prototype successfully displayed 3D images having a resolution of 293 (H)x190 (V) and a viewing-zone angle of 32 degrees. These images had a smaller lens structure and a higher-density pixel structure than those displayed by the conventional device with a larger pixel structure (Figure 2). We exhibited this display equipment at the NHK STRL Open House 2016 and IBC2016, which was held in the Netherlands in September.

■ Coding technologies for integral 3D images

We are researching coding technologies for elemental images used in the integral 3D method. In FY 2014, we began a study on compression efficiency for the case of applying existing video coding techniques to the integral 3D method. In FY 2016, we applied existing video coding schemes to the integral 3D video and conducted evaluation experiments to determine the relationship between compression efficiency and image quality. The results showed that higher compression

(a) (b)

Figure 1. 3D imaging using screen synthesis - images displayed by the conventional method (a) and the method using the new optical system combining multiple images (b)

(a) (b)

Figure 2. Integral 3D image using high-density 8K display (a) Image viewed from the front (b) Image viewed from above/below/left/right (partially enlarged)

From above

From below

From left From right

Figure 4. Capturing of multi-viewpoint images

Area of integral 3D display reproductionMulti-viewpoint robotic cameras

Figure 3. Control of multi-viewpoint robotic cameras

Target area of 3D reconstruction

Robotic cameras



efficiency was obtained by converting elemental images into multi-viewpoint images and then applying the 3D High Efficiency Video Coding (3D-HEVC) method to the multi-viewpoint images than by applying HEVC to elemental images(4). We also contributed to standardization activities at the MPEG-FTV ad hoc group by submitting a document related to the provision of test images.

■ Technologies for capturing spatial information

In integral 3D imaging, it is necessary to capture information on the directions and colors of light rays propagating through the air; such information is called spatial information. In FY 2016, we examined a technology for capturing spatial information without using a lens array by using sparsely arranged multi-viewpoint cameras. This technology generates 3D models of an object from multi-viewpoint images and converts them into elemental images. To generate high-quality 3D images, we developed a multi-viewpoint robotic camera system and an integral 3D model-based capture technology(5)(6). The multi-viewpoint robotic camera system controls multiple robotic cameras cooperatively to follow and capture a specific area of 3D image reconstruction in the air (Figure 3). This makes it possible to capture high-resolution multi-viewpoint images of the area to generate high-quality 3D models. The integral 3D model-based capture technology uses seven multi-viewpoint robotic cameras arranged in a hexagon. The cameras capture multi-viewpoint images of an area of integral 3D display reproduction projected in the air (Figure 4). Then, each camera estimates the depth by stereo matching. On the basis of the estimated depth information, the pixels of each camera are projected into a virtual space to generate a 3D point cloud model (Figure 5). Lastly, the position relationship among the cameras, the object and the display during capture is reproduced in the virtual space and the 3D point cloud model is converted into elemental images.

As a way of integral 3D imaging without using a lens array, we developed a technique to capture spatial information with capture equipment consisting of a camera and a motorized stage. In this technique, the camera, while being moved up and down and from side to side, captures light rays propagating in the air from various angles. On the basis of the information of

light rays passing through a virtual lens array, elemental images are generated. We exhibited integral 3D images generated by this technology at the Open House 2016 (3D image content in Figure 2).

■ System parameters of the integral method

In FY 2015, we began research aimed at determining system parameters that can serve as guidelines for designing integral 3D imaging systems. In the integral method, the spatial frequency of a displayed 3D image deteriorates sharply when displaying deep 3D scenes that have more depth than the display’s depth-reconstruction range, which is determined by display parameters such as the pixel pitch of display equipment and the focal length of lenses. To display a high-quality 3D image of a 3D scene with substantial depth, it is necessary to compress the scene depth so that it fits into the depth-reconstruction range (Figure 6). In FY 2016, we developed a depth-compressed expression technology that compresses the depth of 3D scenes and performs a no-blur and high-quality 3D visualization without including a sense of unnaturalness in viewers by taking advantages of the characteristics of human depth perception. Using a stereoscopic 3D display with viewpoint tracking for reproducing the binocular parallax and motion parallax, which we developed in FY 2015, we evaluated the feeling of unnaturalness of viewers for the depth-compressed 3D scenes generated by using a nonlinear depth compression method with various sizes of the depth range. Figure 7 shows the evaluation results of unnaturalness caused when 3D scenes with a variety of depths (0 m to 250 m) were compressed into depths of 0 m to 10 m. The results showed that the amount of unnaturalness was acceptable even when a 3D scene in excess of 100 m was compressed into a depth range of 1 m(7). On the basis of these results, we began a study on the image quality of integral images generated from depth-compressed images by using stereoscopic 3D display equipment that reproduces the spatial frequency characteristics of integral 3D images in addition to binocular parallax and motion parallax. We plan to derive system parameters for an integral method that can reproduce a 3D space whose depth has been compressed with the depth compression technology.

[References](1) N. Okaichi, H. Watanabe, H. Sasaki, J. Arai, M. Kawakita, T. Mishina:

Figure 5. Example of 3D models generated

Figure 6. Effect of depth-compression expression

Issue with integral 3D television

Integral 3D television

Integral 3D television

Back Front

Front

Range for which the depth can be reproduced without blurring

Compress the scene within this rangeBack

Depth-compression expression technology

Blurring of the foreground and background

Reproduction of a high-quality and natural image

Figure 7. Evaluation of 3D space compressed by nonlinear depth compression

5

4.5

4

3.5

3

2.5

2

1.5

11098760 54321

Short-distance scene 1 (1 m or less)

Short-distance scene 2 (1 m or less)

Medium-distance scene 1 (14 m or less)

Medium-distance scene 2 (14 m or less)

Long-distance scene 1 (250 m or less)

Long-distance scene 2 (250 m or less)

Range of the displayed depth [m]U

nnat

ural

ness

(S

cale

of d

egra

datio

n)

Tolerance limit



“Integral 3D Display Using Multiple Flat-panel Displays,” ITE Technical Report, Vol.40, No.26, CE2016-42, ME2016-82 (Aug. 2016) (in Japanese)

(2) H. Watanabe, M. Kawakita, N. Okaichi, H. Sasaki, J. Arai, T. Mishina: “Integral 3D Display with Multiple UHD Projectors,” ITE Annual Convention 2016, 34C-5 (in Japanese)

(3) H. Watanabe, M. Kawakita, N. Okaichi, M. Kano, J. Arai, T. Mishina: “Projection-type Integral 3D Display with Highly Accurate Method for Auto-compensating Elemental Image,” IDW'16 3DSAp2/3Dp2-25L, pp.1659-1660 (2016)

(4) K. Hara, M. Kawakita, T. Mishina, H. Kikuchi: “Coding Performance

for Moving Picture of Integral Three-dimensional Image using 3D-HEVC,” IDW'16, 3DSAp2/3Dp2-23L, pp.1655-1656 (2016)

(5) K. Ikeya, T. Mishina: “Development of Multi-Viewpoint Robotic Cameras for 3D Reconstruction,” Proc. of IEICE General Conference, （巻号未定） (in Japanese)

(6) K. Ikeya, Y. Iwadate: “Multi-Viewpoint Robotic Cameras and their Applications,” ITE Trans. on MTA, Vol.4, No.4, pp.349-362 (2016)

(7) Y. Sawahata and T. Morita: “Depth-Compressed Expression for Integral 3D Displays – Naturally Expressing a Variety of Scenes with Deep Depth -,” ITE Technical Report, Vol.40, No.41, 3DIT2016-38, IDY2016-47, IST2016-72, pp.17-20 (Nov. 2016)

2.2 Three-dimensional display devices

■ Ultra-high-density spatial light modulator

We are researching electronic holography with the goal of realizing a spatial imaging three-dimensional (3D) television that shows natural images. Displaying 3D images in a wide viewing zone requires a spatial light modulator (SLM) having a very small pixel pitch and an extremely large number of pixels. We are developing an SLM driven by spin transfer switching (spin-SLM) to realize a display device with a pixel pitch under 1 μm. Pixels of the spin-SLM are composed of magnetic materials. The spin-SLM can modulate light by using the magneto-optical Kerr effect, in which the polarization plane of reflected light rotates according to the magnetization direction of the magnetic materials.

We previously developed a tunnel magnetoresistance (TMR) light modulation device(1) that can switch the magnetization direction at a low electric current. The TMR light modulation device consists of three layers: a pinned layer, an insulating layer and a light modulation layer. A transparent electrode common to all devices is placed on the top of the light modulation device. Applying an electric current to this device in the perpendicular direction to its film surface switches the magnetization direction of the light modulation layer. When a polarized light is incident from the transparent electrode side of the spin-SLM, in which many pixels are arranged in two dimensions, the light diffracted by each pixel causes interference in the air. Since the polarization plane of the diffracted light rotates by the magneto-optical Kerr effect of the light modulation layer, a 3D image is displayed in the air by forming a 2D pattern of the hologram in the magnetization direction of the light modulation layer. Rewriting the hologram patterns at high speed also enables the display of moving images. The spin-SLM with many pixels requires an active-matrix pixel selection technology that drives individual pixels by a transistor arranged in each pixel. We previously developed a spin-SLM with 5×10 pixels and a pixel pitch of 5 μm and demonstrated its operation principle(2).

In FY 2016, we prototyped a 2D spin-SLM by forming a TMR light modulation device on the pixel electrode of an active-matrix silicon backplane (2-μm pixel pitch, 100×100 pixels) that we developed in FY 2015. We optimized the film-deposition conditions and the thickness of the magnesium oxide (MgO) layer used for the insulating layer of the TMR light modulation device. This reduced the resistance deviation of each pixel to 1/3 that of a conventional pixel and improved the variation in the driving current. We transferred image data of a Japanese character to the external drive circuit of the prototyped spin-SLM and succeeded in displaying a 2D image by spin transfer switching(3) (Figure 1).

This research was supported by the National Institute of Information and Communications Technology (NICT) as part of a project titled “R&D on Ultra-Realistic Communication Technology with Innovative 3D Video Technology.”

■ Beam-steering device

For a future integral 3D display with much higher performance than current displays, we are developing a new beam-steering device that can control the shape and deflection direction of optical beams from each pixel at high speed without using a lens array. Such a device would enable the reproduction of 3D images having both a wide viewing zone and high resolution. To realize the new beam-steering device, we focused on an optical phased array (OPA) using optical waveguides. We designed, fabricated and evaluated a multiple optical waveguides using an electro-optic (EO) polymer that can control the refractive index at high speed by applying an external voltage(4).

An OPA consisting of multiple optical waveguides (channels) can flexibly change the shape and deflection direction of output optical beams by precisely controlling the phase of each channel by using the changes in the refractive index that occur when a voltage is applied. In FY 2016, we designed and analyzed the operation of the OPA device consisting of eight channels to fabricate a basic device using an EO polymer. We performed numerical simulation using the beam propagation method to confirm the optimum size, pitch and phase conditions of channels We prototyped the designed OPA using an EO polymer and verified the deflection operation of output optical beams by voltage control.

[References](1) H. Kinjo, K. Aoshima, D. Kato, K. Machida, K. Kuga, H. Kikuchi:

“Spin-Spatial Light Modulator with Tunnel Magnetoresistive Effect,” NHK STRL R&D, No.151, pp.39-46 (2015) (in Japanese)

(2) K. Aoshima, H. Kinjo, K. Machida, D. Kato, K. Kuga, T. Ishibashi, H. Kikuchi: “Active Matrix Magneto-Optical Spatial Light Modulator Driven by Spin-Transfer-Switching,” J. Disp. Technol., Vol. 12, No. 10, pp.1212-1217 (2016)

(3) H. Kinjo, K. Aoshima, N. Funabashi, T. Usui, S. Aso, D. Kato, K. Machida, K. Kuga, H. Kikuchi: “Two Micron Pixel Pitch Active Matrix

Figure 1. 2D spin-SLM prototype with 2-μm pixel pitch and 100×100 pixels (a) and 2D image displayed (b)

(a) Spin-SLM prototype

(b) 2D image displayed



Spatial Light Modulator Driven by Spin Transfer Switching,” Electronics, Vol. 5, No. 55, pp.1-9 (2016)

(4) Y. Hirano, Y. Motoyama, K. Tanaka, A. Otomo, H. Kikuchi: “Optical

Beam Steering Devices Using Electro-Optic Polymer Arrayed Waveguides,” ITE Annual Convention, 12C-4 (2016) (in Japanese)

2.3 Multidimensional image representation technology using real-space sensing

We are researching new image representation technologies that can be used for the 2020 Tokyo Olympic and Paralympic Games and various other live sports programs. We are also developing a studio robot for joint performance between live performers and CGs (computer graphics) that can be applied to studio video production.

■ New image representation technique using real-space sensing

We made progress in researching an image representation technology using real-time sensing so that more user-friendly and interesting TV programs can be produced by utilizing various information obtained from camera images and sensors.

We are investigating a method to visualize in real-time the trajectory of an object moving at high speed and in a complex manner in the footage of live sports coverage. In FY 2016, we prototyped a system that uses images from a near infrared camera for robust tracing of a fast-moving object that is not visible to the naked eye. The results of field experiments demonstrated the effectiveness of the system and issues toward its practical use. We plan to improve algorithms so that the system can be used for large-scale live sports broadcasting.

Similarly, for use in live sports coverage, we researched a method to estimate the head pose of a player in soccer game footage captured with a wide-angle lens(1) for the purpose of commenting on game strategies. In FY 2016, we developed a technique that can estimate the head pose with high accuracy in a short calculation time by effectively combining several feature values requiring a small amount of calculation extracted from a low-resolution image of the face area. We confirmed its effectiveness through experiments using actual images. We plan to optimize the process so that the technique can be used for live programs.

For studio video production, we are developing a studio robot that enables natural joint performance between live performers and CG characters. In FY 2016, we prototyped a robot equipped with a locomotive function to move around in response to the movement of a CG character, a sensing function to sense performer motion and enable lively interaction, a function to send back the CG conditions to live performers and a lighting measurement function to integrate CG imagery and live action more naturally. We exhibited the

robot at the NHK STRL Open House and also identified the requirements for practical use through evaluations(2) (Figure 1). For its use in studio video production with a higher degree of freedom, we plan to reduce the noise of the robot and incorporate new features such as the inpainting of cables and other unnecessary objects captured in images.

■ Four-dimensional spatio-temporal image representation technique for sport scenes

To make live sports broadcasts of ball games more comprehensible to viewers, we researched a new image representation technique that utilizes multiview video. This technique combines multi-viewpoint robotic cameras whose directions are controlled to focus on the object that a cameraperson is shooting with object tracking and 3D information analysis technologies. This makes it possible to calculate the time-series 3D position of an object such as a ball in real time and with high precision from images of the moving robot cameras(3). By using the calculated time-series 3D position information, the technique can overlay CGs such as the ball trajectory accurately on captured multiview video (Figure 2). In addition to presenting such information, the technique can change the viewpoint after operation, which enables easy-to-understand image representation with a higher degree of freedom. We also improved the quality of generated images by increasing the resolution of the multi-viewpoint robotic cameras. Our demo system using this technology gained a favorable reception at exhibitions such as Digital Content EXPO and Inter BEE. We plan to improve the accuracy and robustness of time-series 3D position information and increase the number of sports events that can be supported by this technique.

■ 2.5D multi-motion representation technique

We are researching multi-motion representation that shows a series of motion in the form of a serial photograph. We investigated a method to generate a multi-motion image immediately after motion takes place in live sports broadcasting. This image is expressed by a simulated 3D CG in which an athlete area image cut out from the captured image is placed as a billboard in a CG space against a background 3D CG image of the stadium. In FY 2016, we developed an object extraction

Figure 1. Example of video compositing using studio robot

Studio robot Interactive performance between performer and CG character

PerformerComposite CG

Superimpose CG on the robot position

(a) Captured image (b) Compositing image overlaid with CG ball trajectory

Figure 2. Example of composite image using ball tracing



algorithm method with high operability that uses an object region extraction technique (Active GrabCuts) to simplify the adjustment of a region to be extracted from the captured image. We also developed an automatic processing method for the billboard arrangement using a stereo camera. We verified the operation of these methods and confirmed their effectiveness. We conducted simulations of an image representation technique that employs these methods and investigated ways of creating effective production effects. For practical use, we plan to conduct a demonstration experiment and improve the precision of the athlete contour region.

■ Naturally enhanced image representation of a flying object

For more comprehensible and interesting live golf programs, we are researching the use of image processing technologies and physical sensors for measuring the position of a golf ball, which is almost invisible to TV viewers, and for enhancing the ball naturally by using CGs(4). In FY 2016, we developed a method to estimate the 3D coordinates of a ball with high precision and generate its trajectory in real time by using sensor cameras and laser sensors installed near the teeing ground and around the place where the ball lands. We verified the effectiveness of the method through field experiments using a prototype system. We plan to extend this method so that it can be applied to a wide variety of sports.

[References](1) S. Yokozawa et al.: “Study on Head Pose Estimation for Football

Videos”, Proc. of ITE Annual Convention, 32D-1 (2016) (in Japanese)(2) H. Morioka et al.: “A Studio Robot for Joint Performance with CG

characters”, Proc. of ITE Winter Annual Convention, 23B-5 (2016) (in Japanese)

(3) H. Okubo et al.: “Prototyping of a Real-time 3D Ball Tracking System for Sports Graphics”, Proc. of ITE Annual Convention, 32B-2 (2016) (in Japanese)

(4) D. Kato et al.: “A study of estimating 3D Locus for Flying object”, Proc. of Annual Conference of the Robotics Society, RSJ2016AC2B2-04 (2016) (in Japanese)


3.1 Cloud-based media-unifying platform

We are researching technologies for making use of the Internet to provide more user-friendly and attractive broadcasting services. In FY 2016, we studied a platform that enables the easy viewing of content at any time without considering the differences in delivery paths and viewing terminals and developed technologies for an advanced broadcast broadband service on the basis of Hybridcast Technical Specification version 2.0.

■ Media-unifying platform

As media for delivering TV programs diversify to include broadcasting and the Internet, so do terminals that support the viewing of the programs such as smartphones and tablets. Against the backdrop of this diversification of program-viewing methods, we are researching a “media-unifying platform” that automatically selects appropriate media and distribution sources for displaying programs in accordance with the viewer situation. The media-unifying platform consists of servers that

3 Internet technology for future broadcast services

We continued our research on how to use the internet to provide services adapted to diverse viewer environments.

These days, TV programs are distributed through various media, such as broadcasting and the internet, and viewed on diverse terminals including smartphones. Toward this backdrop, we began a study on a cloud-based media-unifying platform that automatically selects appropriate media and distribution sources for displaying programs in accordance with the viewer situation, and we prototyped a system. To realize the system, we investigated a link description scheme and conducted experiments to measure the reception status of broadcasting and broadband networks in living environments.

We conducted verification experiments on a system that synchronizes real-time data on broadcasting by using Hybridcast's high-accuracy synchronization function between broadcasting and telecommunications networks. For SHV multimedia broadcasting, we prototyped a receiver to verify a general data transport method standardized by ARIB for data multiplexed by MPEG Media Transport (MMT).

In our research on device linkage services, we investigated new service systems that link TV with IT technology. We prototyped a system that presents appropriate information based on viewing logs and a smartphone's positioning information. We also developed “8K Time into Space,” which displays a large number of videos on an 8K display and allows the viewer to select one by touching or gesture, and advanced “Augmented TV” that utilizes an augmented reality (AR) technology for TV. We also began a study on a TV-watching robot that provides company to people watching TV alone at home, for which we developed functions to detect the TV position and to generate utterances related to the program.

In our research on the utilization of program information, we are studying more convenient broadcast broadband services that bridge TV viewing with daily activity. We designed and prototyped a base system for linking TV with diverse smartphone applications and Internet of Things (IoT)-enabled devices, and developed extended functions of common companion application for Hybridcast.

For the effective utilization of broadcast data, we provided program data in Linked Data format, developed new educational and sports services using data structuring, and investigated entity linking that links person information contained in closed captions of programs with Wikipedia article headings.

In our research on technologies for promoting Hybridcast, we conducted a browser performance test at the IPTV Forum to improve receiver performance and investigated HbbTV in Europe to examine mutual operation with HbbTV.

In our research on internet delivery technology, we continued our work from FY 2015 by developing an MPEG-DASH player. Our player technologies were made available to the members of the IPTV Forum and have been widely used by commercial broadcasters and content providers, contributing to the creation of new delivery services. For more stable video distribution, we developed a delivery path control technology that carefully controls the delivery path of each viewer terminal in accordance with the congestion status and a transmission rate control technology that continues stable live video distribution even when the access connection increases.

Regarding security technologies, we made progress in research on attribute-based encryption that can restrict information access by service providers according to their attributes while protecting viewer information stored on cloud servers. We developed an encryption scheme that reduces the processing burden of the viewing terminal and an encryption scheme that can search encrypted data without decryption. We also developed lightweight symmetric encryption to conceal information securely in IoT-enabled devices and studied traitor tracing cryptography to prevent illegal copying of the receiver’s decryption key. Moreover, we began a survey on the cyber security of broadcasters.



control the program distribution status of each medium and media-unifying engines that automatically select appropriate media on the terminal side. We exhibited a prototype terminal equipped with a media-unifying engine(1) at the NHK STRL Open House 2016 and demonstrated how programs are viewed through appropriate means selected according to the viewer environment such as at home or outdoors (Figure 1).

To realize the media-unifying platform, we are investigating a link description scheme that can specify content regardless of the type of distribution media. Using this scheme, the viewer can share his/her friends the link information of a program as a recommendation through SNS or email without paying attention to the media type, such as broadcasting or the Internet, and the friends can view the program via a medium appropriate to their viewing environment. In FY 2016, we prototyped a function to identify the same content distributed through different media based on its metadata such as the title and verified its effectiveness(2). Using this function to derive media that can view the content specified by the link enables the viewing terminal to select an appropriate medium.

To assess the reception quality of broadcasting and the Internet in various viewing environments, we developed a portable device for measuring the reception status of broadcasting and communications and conducted measurements in actual living environments such as on a train, on a bus and on foot (Figure 2). The collected data showed that the service availability rate can be improved and the amount of communication data can be reduced significantly

by using the media-unifying platform to select an appropriate channel (broadcasting or the Internet) according to the environment(3).

We outlined scenarios where the media-unifying platform is utilized for viewing content in people’s daily lives, based on which we produced video content describing a family’s life in the near future and screened it at the NHK STRL Open House (Figure 3).

■ Technologies for advanced Hybridcast

Since its launch in 2013, Hybridcast has steadily expanded its services by welcoming the participation of commercial broadcasters. In FY 2014, the IPTV Forum published Hybridcast Technical Specification version 2.0, which enabled the high-accuracy synchronization of content between broadcasting and telecommunications and video distribution using MPEG-DASH (Dynamic Adaptive Streaming over HTTP) as we had proposed. Using these capabilities, we conducted R&D to realize more attractive Hybridcast services.

To demonstrate the feasibility of services using the high-accuracy synchronization function between broadcasting and telecommunications networks, we conducted verification experiments on a system that distributes data generated in real time at a site of live sports coverage via the Internet and presents it in synchronization with broadcast images. In the experiments, we used the system with J. League soccer matches recorded live on NHK BS1. The system added time information for synchronization with the measurement data (tracking data), such as player positions. The tracking data is generated and distributed in real time by a data production service provider, uploaded to the cloud server and distributed to a

Figure 1. Example of operation of the media-unifying platform prototype terminal

Figure 2. Device for measuring broadcasting and telecommunications reception status

Figure 3. Video content presenting cases of use of media unification

Figure 4. System structure of the data synchronization and distribution experiment

Venue for on-the-spot broadcastData production service provider

Time server

Data distribution equipment

Stadium

Tracking data

Tracking systemUplink 1 Gbps

Data distribution server

(WebSocket)

Downlink 100 Mbps

Commercial cloud service

Reception environment

Time-mapping generator

Time serverBroadcast

wave (NHK BS1)

Prototype Hybridcast receiver supporting the synchronization

Figure 5. Prototype Hybridcast application

Automatically selection of an appropriate viewing method from broadcasting or the Internet

Media-unifying technology

Broadcasting

Internet



prototype Hybridcast receiver that receives BS1 broadcasting (Figure 4). The results showed that the data was successfully received prior to the arrival of broadcast images and that the data and images were synchronized with high accuracy in the receiver by using the added time information(4). Using this technology, we prototyped a Hybridcast application that displays the player position and running speed in real time during the progress of a live soccer program and exhibited it at the NHK STRL Open House 2016 (Figure 5).

We also studied technologies for a multi-view service that distributes images seen from different angles via the Internet simultaneously with broadcasting and allows the viewer to select the images. We proposed a control method for reproducing MPEG-DASH video in synchronization with the broadcast images and confirmed its operation by using a prototype system(5). We also proposed and verified a method for simplifying the time mapping between broadcast content and content delivered on the Internet to achieve high-accuracy synchronization(6).

■ SHV multimedia broadcasting

Regular SHV satellite broadcasting is scheduled to start in 2018. We worked on revisions of the standards related to SHV multimedia broadcasting and conducted their verification.

We developed an experimental receiver to verify the transmission of data content multiplexed by MPEG Media Transport (MMT) and verified the cache storage control of data resources transmitted by broadcast waves and event message operation. Using this receiver, we demonstrated that closed captions can be presented in more enriched ways by using an SHV multimedia broadcasting application to retrieve the ARIB-TTML format closed caption data transmitted by the MMT multiplexing scheme and by adjusting the size, color and display position of characters (Figure 6). We also verified the operation of the general data transport method specified by an ARIB standard(7) and exhibited it at the NHK STRL Open House 2016. Using this method to push the delivery of image data and binary data on broadcast waves will enable the frequent update of data used for applications.

[References](1) H. Endo, K. Matsumura, K. Fujisawa, K. Kai: “Cross-Media Platform

for the Unification of Broadcast and Broadband Program-Viewing Environments”, IEEE BMSB, 7A-3 (2016)

(2) S. Taguchi, H. Endo, K. Matsumura, K. Kai: “Content Identification Method for Media-Unifying Platform,” ITE Winter Annual Convention, 11A-4 (2016) (in Japanese)

(3) H. Endo, S. Taguchi, K. Matsumura, K. Fujisawa, K. Kai: “Measurement of the Reception Status of Broadcasting and Broadband by Field Experiment for a Validation of Media-Unifying Platform,” ITE Tech. Journal, Vol.40, No.45, BCT2016-83, pp.11-15 (2016) (in Japanese)

(4) Y. Hironaka, M. Onishi, K. Matsumura, M. Takechi: “Evaluation Tests of Synchronized Presentation of Live Broadcast Program and Internet Content on Hybridcast,” ITE Annual Convention, 13D-1 (2016) (in Japanese)

(5) M. Onishi, Y. Hironaka, K. Matsumura, M. Takechi: “Prototype of Synchronization Method between Broadcast and Broadband Using MPEG-DASH Player for Hybridcast,” ITE Tech. Journal, Vol.40, No.14, BCT2016-37, pp.13-16 (2016) (in Japanese)

(6) M. Onishi, K. Matsumura, Y. Hironaka, M. Takechi: “A Consideration of Method to Convert Timestamp for Synchronization between Broadcast and Broadband on Pre-recorded Program,” ITE Annual Convention, 13D-2 (2016) (in Japanese)

(7) S. Takeuchi, A. Baba, K. Kai, K. Fujisawa: “A Study on General Data Transport Method over MMT in Multimedia Service of UHDTV Broadcasting,” ITE Winter Annual Convention, 11A-3 (2016) (in Japanese

3.2 Device linkage services

We are researching service systems that will offer new user experiences beyond those of conventional TV broadcasting by taking advantage of IT technologies such as the Internet and augmented reality (AR) and by linking TV with various devices such as smartphones, tablets and sensors.

■ Technology for providing program-related information matching the user’s location

We researched a content-matching service that connects program viewing to new discoveries and values not only in the living room but in various scenes in daily life.

We developed a prototype system that presents the user with topics and information related to the programs by utilizing the user’s program-viewing logs in accordance with their location, identified by the positional information of their smartphone. We also studied cases of use of the content-matching service by considering the business requirements of commercial broadcasters. We demonstrated that these cases of use can be supported on a unified application at the NHK STRL Open

House 2016 (Figure 1)(1). We thus demonstrated the feasibility of a new TV experience in which topics and information provided by TV programs can be utilized in daily activities, both indoors and outdoors, by using smartphones as a bridge

Figure 6. Example of an application using closed caption data

Figure 1. Example of prototype content-matching services



to connect TV viewing and user behaviors on the Internet and in the real world.

■ Interactive technology on 8K display

We are researching user interfaces (UIs) using versatile web technologies for enabling interactive services on 8K displays and the “8K Time into Space” image presentation technology that can reproduce a huge number of videos simultaneously on one display(2).

In our work on UIs, in FY 2016, we incorporated a gesture recognition device into our prototype system in addition to a touch sensor so that users can intuitively operate web content on an 8K display. We achieved an environment that allows not only touch operation but noncontact operation of the 8K display screen from a distance.

We developed a new image presentation technology taking advantage of high-definition 8K displays, called “8K Time into Space.” This technology subdivides a single TV program or movie into hundreds of moving images in chronological order and plays them simultaneously on one screen so that the viewer can obtain a view of the whole sequence of the video content. Our new mechanism of showing continuous still images similarly to a moving image and our new web technology that reduces the processing load of playback made it possible to reproduce a huge amount of video content simultaneously at a rate more than 15 frames per second regardless of the length or number of partitions of the video. In addition, by using the aforementioned UI, we developed a new viewing style in which the viewer can view a program by freely going back and forth between the thumbnails of the entire program and a selected scene displayed with high definition. 8K Time into Space was selected by the Ministry of Economy, Trade and Industry as one of 20 innovative technologies in 2016 to promote innovation in digital content. We demonstrated of 8K Time into Space at Digital Content EXPO 2016 as a selected technology. This research was conducted in cooperation with Massachusetts Institute of Technology(MIT) Media Lab.

■ Device-linked system technology using AR

We are studying AR technology for broadcasting services. We conducted R&D on a device linkage system called “Augmented TV” that will provide a new viewing experience through mobile devices such as smartphones and tablets. This system will enable extra broadcast content to be shown in front of a TV screen by overlaying 3D CGs on TV images shown on a mobile device.

In FY 2016, we developed a method for representing the continuous space between the inside and outside of a TV screen by modifying the arrangement and shape of 3D CG objects(3). The method formulates the 3D space within the TV screen by focusing on the geometric coherency of the “appearance” according to the viewing position. By doing so, the method can create a natural representation of an object

appearing to fly straight out of the TV screen regardless of the position of the mobile device. We also presented at a conference a technology for enabling viewers not having a mobile device to also enjoy the service, which we developed in FY 2015 jointly with the Kanagawa Institute of Technology(4).

Augmented TV received the 2015 Technical Promotion Award (Content Technology Award) from the Institute of Image Information and Television Engineers.

■ TV-watching robot

As the number of single-person households increases due to Japan’s aging society, along with trends toward nuclear families and diverse styles of TV viewing, an increasing number of people are watching TV alone, instead of enjoying it with other people. Against the backdrop of these changes in society, we began developing a TV-watching robot to allow viewers to share the enjoyment of watching TV with others by using a communication robot.

In FY 2016, we developed a function to automatically detect the position of the TV and viewer in a living room and a function to generate utterances related to the program being viewed, and we implemented these functions into an experimental communication robot.

For the automatic detection of the TV position, a camera mounted on the robot set in the living room captures the surrounding environment. The TV position is detected by extracting the feature values of the luminance difference and edges from the captured video and calculating each image moment. Experiments in a laboratory simulating a living room (Figure 3) demonstrated that this detection method achieved an accuracy of about 88%, which is higher than that of calculating each feature value separately(5). We also developed a function to detect the position of a person by applying a face detection technology using the camera built in the robot and a sound localization technology using a microphone array and incorporated it into the prototype robot.

For the generation of utterances related to the program being viewed, we developed a technology to enable a robot to say what it “feels” about the program as people do. This technology generates an utterance text by extracting keywords that are highly related to the genre of the TV program from the program information and subtitles and combining them with template sentences that express emotions (e.g., want to go, want to see, want to eat). We incorporated this utterance generation function into the prototype robot and enabled the robot to turn to the TV or to the viewer and make utterances.

[References](1) C. Yamamura, M. Ikeo, A. Baba, H. Fujisawa, M. Uehara: “Design of

information provision for connecting TV and Life Activities,” Proc. of ITE Annual Convention, 13D-3 (2016) (in Japanese)Figure 2. 8K Time into Space

Figure 3. Experiment on TV detection by TV-watching robot



(2) H. Ohmata, A. Lippman: “8K Time into Space: Web-based Interactive Storyboard with Playback for Hundreds of Videos,” Proceedings of 2016 IEEE International Symposium on Multimedia (ISM 2016), pp.413-418 (2016)

(3) H. Kawakita, M. Uehara: “Visual Coherence Between 2D Moving Pictures and 3D Real Space for Augmented TV,” 33rd Technical Committee Conference of Cyberworlds IEICE, pp.1-4 (2016) (in

Japanese) (4) H. Kawakita, M. Uehara, T. Nakagawa, Y. Suzuki, H. Suzuki, A.

Shirai: “Investigation of an Application for Augmented TV with ExPixel,” Proc. of ITE Annual Convention, 11C-1 (2016) (in Japanese)

(5) Y. Hoshi, Y. Kaneko, Y. Kawai, M. Uehara: “A Study on detection methods of TV position in living room,” Proc. of ITE Winter Annual Convention, 22B-6 (2016) (in Japanese)

3.3 Program information utilization technology

With the aim of making content more accessible and increasing user satisfaction in the era of convergence of broadcasting and internet services, we are studying how broadcasters and service providers can provide viewers with content that matches their lifestyle.

■ System for connect daily activity and TV viewing

In FY 2016, we began research on more convenient broadcast-broadband services that connect TV viewing and daily activities in our life by using internet. Based on the current situation of increasing the amount of time spent for smartphone, we designed a system architecture that enables TV to interact with various applications and IoT-enabled devices via smartphones and an application framework which constitute a core part of this architecture. In addition, we prototyped this framework as extended function of companion application for Hybridcast(1)(2). We verified the system with various use cases and confirmed the versatility of the framework for various services.

■ Data linkage and data-structuring technologies

We are investigating how broadcaster data can be utilized for diverse services offered by non-broadcasters. To make the coverage of the investigation as wide as possible, we progressed the work from both data creation and its utilization aspects. For data creation aspect, we made program guide data and data related to video for the web applications “NHK Video Map MICHISHIRU” and “NHK Chiiki Dukuri (local initiatives) Archives”, available to external parties by providing them in Linked Data format. For data utilization aspect, we participated in following contest events for developers, “LOD Challenge Japan 2016” and “RESAS×JAPAN Hackathon” as a data provision partner, and “Mashup Awards 2016” as an application programming interface (API) partner. We also presented NHK World data in Linked Data format to be used by the smartphone app for participants of the ISWC2016 international conference held in Kobe to verify a method for utilizing data for the international promotion of content.

We began developing new services that utilize NHK content

in new areas such as education and sports by using data-structuring technology. This included content systematization based on the government guidelines for schoolteaching and structuring of information on sports players. This technology was utilized in the NHK web pages for “NHK for School” and the Rio 2016 Olympics.

■ Entity linking of program information

As an elemental technology for linking the information in programs with entities in the real world such as “objects” and “concepts,” we are researching entity linking that identifies what is actually indicated by a keyword mentioned in closed captions of programs. In FY 2016, we focused on person-related keywords that are of great utility value and developed an entity-linking method that links person-related keywords in closed captions with Wikipedia article headings(3). The method, which is based on the analysis of the relationship among various keywords appearing in Wikipedia content, can disambiguate keywords by analyzing the link information among person keywords. The experimental application of this method to closed caption data of programs demonstrated that this method is more effective than conventional ones.

■ Promotion of Hybridcast

In addition to technological development for services and platform of Hybridcast, we are also studying technologies for promoting Hybridcast services. We conducted a performance test event with IPTV Forum Japan. Purpose of this event is to share the knowledge on the performance of Hybridcast browsers that run on TV receivers(4). Through this event, the current situation and issues of receiver performance were successfully shared between broadcasters and receiver manufacturers and recognized as a good opportunity to help improve services and receiver performance.

To promote Hybridcast services globally, we began studying ways of enabling the cross-execution of applications on both Hybridcast and HbbTV that is developed and deployed mainly in Europe. We studied a basic architecture and application creation methods for ensuring the compatibility of applications between Hybridcast and HbbTV, and verified them by proto-type implementation. We also conducted a survey in Europe on the current situation of actual HbbTV services and reported our findings at the IPTV Forum Japan and other parties/organizations.

[References](1) M. Ikeo, H. Ohmata, H. Ogawa, C. Yamamura, M. Miyazaki, H.

Fujisawa: “An application Framework for collaboration with TV, Internet and Real World,” Proc. of 79th National Convention of IPSJ (2017) (in Japanese)

(2) H. Ogawa, M. Ikeo, H. Ohmata, H. Fujisawa: “IoT Device Connection Architecture for Broadcast Content,” Proc. of 79th National Convention of IPSJ (2017) (in Japanese)

(3) M.Urakawa, H.Fujisawa: “Data Interchange between Sports Event and Program Information based on Ontology”, Proc of ITE Winter Annual Convention, 25C-3(2016) (in Japanese)

Figure 1. System model for services to connect TV viewing and daily activities

VoD

Broadcaster

In front of a TV Outside of a home

Service providerBroadcaster’s program-related information

Services and data of various providers

Data linkage

Secure utilization of viewing data and content targeting of broadcaster for each user’s situation

Various daily activitiesTV linkage (Mobile-centric)

IoT linkage IoT linkage

Location linkage

Activity data

Activity dataViewing data

Program-related information

Current Hybridcast linkage (TV-centric)

Healthcare

Shopping Awaking

Broadcasting

Hybridcast-enabled TV

CA＊

Extension

Companion Screen

＊ Companion Application



(4) M. Naemura, M. Miyazaki, M. Urakawa, H. Fujisawa: “Entity Linking of Person Keywords using Group Tree Structured Graph”, IEICE Technical Report, Vol.116, No.213, NLC2016-21, pp.51-56 (2016)

(5) M. Ikeo, K. Matsumura, H. Fujisawa, M. Takechi: “A Study of HTML5 Web Browser Performance on Hybridcast TV Receiver”, Proc. of ITE Winter Annual Convention, 15B-3 (2016) (in Japanesse)

3.4 Internet delivery technology

For large-scale and stable video distribution over the Internet, we are researching video player technologies, delivery path control in accordance with reception status and transmission rate control for video distribution.

■ Video player technologies

We made progress in our development of an MPEG-DASH (Dynamic Adaptive Streaming over HTTP) video player that can playback MPEG-DASH Internet video stably on HTML5 browsers of various viewing terminals such as Hybridcast-enabled TVs, PCs and smartphones (Figure 1).

In FY 2015, we made our MPEG-DASH player available to the MPEG-DASH interoperability study group at the IPTV Forum to help realize a stable video delivery service using MPEG-DASH. Our player, named “dashNX,” is available to members of the study group. In FY 2016, we equipped our player with extended functions for satisfying broadcasters’ service requirements, such as inserting another video into a program seamlessly, and with application programming interfaces (APIs) to customize viewing operation using these functions. Moreover, we provided a sample player under development equipped with basic operations of video playback and API samples as well as a test website for operation verification so that the member companies can immediately embark on service development.

As a result of these efforts, our player has been widely used by commercial broadcasters and distribution service providers, contributing to demonstration experiments on Hybridcast-based 4K video delivery and the creation of new video delivery services such as a 4K video delivery service. In recognition of these achievements, we were awarded the METI Minister’s Award (MPTE Award) by Motion Picture and Television Engineering Society of Japan, Inc.(1)

■ Delivery path control in accordance with reception status

For low-cost and stable video delivery, we made progress in our development of a method for carefully controlling the delivery path to each terminal using diverse delivery paths including multiple content delivery networks (CDNs) and distribution equipment built on cloud servers. In this method, the distribution side decides the best delivery path to each viewing terminal by collecting the reception-status information

such as the network throughput measured by each terminal and determining the congestion status of each delivery path. In response to instructions from the distribution side, each terminal changes the delivery path. We built a prototype system by incorporating a function to measure and notify reception status and a function to change the delivery path into the MPEG-DASH player that we developed in FY 2015. We operated this prototype system on a cloud server and verified the operation of delivery path control using multiple distribution servers. The results showed that the degradation in viewing quality could be suppressed by changing the delivery path to a different distribution server with sufficient capacity in the case of congestion on one distribution server(2). We also exhibited the actual operation at the NHK STRL Open House 2016 (Figure 2).

■ Transmission rate control for video distribution

We developed a method for carefully controlling video transmission rates in accordance with the congestion status of distribution servers as a way of ensuring the stable quality of live video distribution even if the number of accessing terminals increases. The method constantly monitors the number of viewing terminals connecting to distribution servers and controls transmission rates in real time to ensure that the total of the video transmission rates to all the connected terminals is less than the transmission capacity of distribution servers. We built a prototype distribution system equipped with this method in a cloud server environment and verified its operation. The results showed that this method can suppress fluctuation in video quality and provide video with a stable quality compared with conventional methods, even when an increase in the number of viewing terminals causes congestion in distribution servers(3). We also exhibited the actual operation at the NHK STRL Open House 2016.

[References](1) S. Nishimura: “Development of a Hybridcast compatible MPEG-DASH

player,” The Motion Picture and Television Engineering, No.771, pp.46-47 (2016) (in Japanese)

(2) S. Tanaka, S. Nishimura, M. Yamamoto: “Adaptive Delivery Path Figure 1. Playback video on various terminals using MPEG-DASH player

Figure 2. Comparison of viewing quality between systems with/without delivery path control

Without delivery path control (Conventional method)

With delivery path control (New method)

Temporal transition of the quality of received video

Temporal transition of measured network throughput

(Different colors for each distribution server)



Control Method Based on Each Terminal's Reception-Status in Video Distribution,” IEICE Society Conference, B-6-16, 2016, p.16 (2016) (in Japanese)

(3) M. Kurozumi, S. Oda, S. Nishimura, M. Yamamoto: “Video

Distribution Technology by a Transmission Rate Control According to a Congestion Situation of Distribution Servers,” ITE Annual Report, 11D-1 (2016) (in Japanese)

3.5 Security technologies

To ensure the security and reliability of programming in the era of the convergence of broadcasting and telecommunications, we researched cryptography and security technologies that can be used to provide secure services.

■ Cryptography algorithm for cloud services

Service providers need information about viewers in order to personalize services. Also, to allow viewers to receive unknown and unexpected services, many service providers need to have this information. An efficient way to meet this requirement is to store viewer information in the cloud server and allow various providers to access the information, but it is necessary to restrict access to the information to preserve viewers’ privacy. To meet these conflicting conditions, we researched attribute-based encryption scheme that can specify the access rights of providers to information according to their attributes. To reduce the burden of the encryption process on the viewer terminal, we developed an encryption scheme that divides up the process so that part of it can be handled by cloud servers(1)(2)(3). We also developed an encryption scheme that can search encrypted data without decryption in order to reduce the data that providers can access to the minimum necessary(4).

In addition, we developed an access control system that enables viewers to view paid broadcasting services even from outside the home when the attribute-based encryption scheme that we developed in FY 2015 is used for paid broadcasting services(5).

■ Lightweight encryption algorithm

We researched symmetric encryption that would enable safe and secure information transmission in an environment where diverse Internet of Things (IoT)-enabled devices are connected to the Internet. We developed a lightweight symmetric encryption scheme so that even data from devices with a low-performance CPU can be concealed securely.

■ Traitor tracing encryption scheme

We researched a traitor tracing encryption scheme to identify

an illegally copied decryption key as a countermeasure against illegal copying of receiver decryption keys. To identify which decryption key has been used, each receiver needs to have a unique decryption key, and a pirate receiver created using the copied decryption key needs to be tested in order to determine the key. If the pirate receiver detects this test, however, it can interrupt the test by stopping operation. To prevent this from happening, we devised a content delivery method that makes a test indistinguishable from an actual service(6).

■ Survey on cyber security of broadcasting systems

We began a survey on the cyber security of broadcasting systems. We investigated actual cyber-attack technologies and analyzed the risk of cyber-attacks against broadcasting systems. We also surveyed cyber security trends in the domestic and international broadcasters.

[References](1) N. Attrapadung, G. Hanaoka, K. Ogawa, G. Ohtake, H. Watanabe, S.

Yamada: “Attribute-Based Encryption for Range Attributes,” SCN2016, pp.42-61 (2016)

(2) G. Ohtake, R. Safavi-Naini, L. Zhang: “Implementation and Evaluation of Outsourcing Scheme of Attribute-Based Encryption,” IEICE ISEC Technical Report, Vol.116, No.129, ISEC2016-30, pp.129-136 (2016) (in Japanese)

(3) G. Ohtake, R. Safavi-Naini, L. Zhang: “Outsourcing Scheme of ABE Encryption Secure against Malicious Adversary,” ICISSP2017, pp.71-82 (2017)

(4) G. Ohtake, R. Safavi-Naini, L. Zhang: “Efficient Verifiable Attribute-Based Keyword Search,” IEICE ISEC Technical Report, Vol.116, No.505, ISEC 2016-119, pp.195-202 (2017) (in Japanese)

(5) K. Ogawa, S. Tamura, G. Hanaoka: “Functional Encryption to Enrich Broadcasting Services,” Symposium on Cryptography and Information Security (SCIS) 2017, 4F2-3 (2017)

(6) K. Ogawa, G. Hanaoka, H. Imai: “How to Make Traitor Tracing Schemes Secure against a Content Comparison Attack in Actual Services,” IEICE Trans. Fundamentals, Vol.E100-A, No.1, pp.34-49 (2017)


4.1 Content elements extraction technology

■ Video retrieval technology

Raw video footage stored in video archives is a valuable resource for program producers. To make active use of such footage, we are researching video retrieval technology.

We refined the technology for identifying persons appearing in program footage that we investigated in FY 2015. The refined technology uses feature values calculated for each divided region and has a better identification model. Recognition experiments using about 150 episodes of dramas showed that the technology improved the accuracy by 36% compared with conventional techniques. We also developed a face detection technology for specifying face positions in images which is necessary for face recognition. To support TV program footage that has frequent changes in head directions and facial expressions, the technology employed visual features emphasizing the information around feature points in a face and a classifier using cascaded decision trees. After training with two months’ worth of TV footage, the technology achieved favorable detection accuracy with 71.0% recall rate (i.e., percentage of full detection coverage) and 96.7% precision rate (i.e., percentage of correct detection)(1).

We continued to improve our technology for detecting character strings in video frame images (scene text). We increased the accuracy of detecting slanted character strings by using features based on the aspect ratio of the candidate character region rotated by an arbitrary angle(2). Also, with the accelerated feature calculation of a region that combines multiple candidate character regions, the technology can detect Japanese character strings in images at a rate of 10

frames per second. We exhibited this technology at the NHK STRL Open House 2016.

We are also researching visual-based similar-image retrieval that uses visual similarities in images. While our previous method arranged fixed-size blocks by focusing on the object region, our upgraded method can vary block sizes so that the blocks can be arranged closely without unnecessary spaces between them. This improved the retrieval accuracy. Moreover, we began a study on new visual features focusing on the symmetry of object appearances in order to realize “fine” similar-image retrieval that searches for identical objects such as specific buildings.

In our work toward the practical application of video retrieval technology, we prototyped a video material management system equipped with an automatic image-sorting capability using object recognition technology and a visual-based similar-image retrieval capability in response to requests from program producers handling CG synthesis and imaging effects. We began the experimental use of the system at production sites. We also continued our support for the experimental use of the metadata supplementation system for earthquake disaster archives at three NHK stations in the Tohoku region (Sendai, Morioka and Fukushima) for the purpose of efficiently managing the huge number of video reports on the Great East Japan Earthquake. We equipped this system with a person retrieval function using face recognition technology. In addition, we participated in international standardization activities for a mechanism to generate desired metadata by flexibly combining various analysis technologies as a capability of future media program production systems.

4 Technologies for advanced content production

We are progressing with our research on program production technologies for high-quality attractive content services and wireless technologies for program contributions such as live sports coverage and music programs.

In our work on content element extraction technology, we upgraded our previous technologies of face recognition, scene text detection and visual-based similar image retrieval and developed a face detection technology that can handle variations in head directions and facial expressions. We also began research on video summarization technology. We developed a mechanism to automatically generate summarized video according to the significance of each video section based on multilateral factors such as tweets on the program and camera movement.

In our research on content production support technology to help broadcasters in program production, we developed a system that presents information necessary for program production by analyzing text big data including social media and program archives.

In our research on bidirectional field pick-up units (FPUs), we worked on elemental technologies for high-speed bidirectional transmission. We increased the transmission capacity by improving the hybrid automatic repeat request (HARQ) retransmission control scheme, reduced latency by redesigning the transmission frame structure and improved the operability by using an automatic IP-address mapping function.

We are also researching a transmitter and receiver using millimeter-wave-band radio waves for a 4K and 8K Super Hi-Vision wireless camera. In our research on single-carrier transmission technology for a more compact wireless camera, we prototyped an experimental modulator and demodulator with a wider transmission bandwidth and confirmed through laboratory experiments that they are capable of 200-Mbps-class transmission. In our research on a 4×4 MIMO-OFDM (multiple-input multiple-output orthogonal frequency division multiplexing) scheme with four transmitters and four receivers for larger transmission capacity, we demonstrated the real-time transmission of video transport streams using this scheme. We also improved the operability of our Hi-Vision wireless camera and used it for program production of the NHK Kouhaku year-end music show.



■ Video summarization technology

With the aim of supporting the production of Internet content such as program websites and NHK On Demand, we launched a research on automatic video summarization technology.

We developed a method for detecting scene boundaries in program video so that video sections to constitute summarized video can be equally extracted from individual scenes. The method can detect scene boundaries in various genres of program video with an accuracy in excess of 70% by using the change in the color formation based on the image segmentation of each shot image as a determinate criterion(3).

To estimate important video sections that should be included in summarized video, we researched ways of collecting and exploiting program-related tweets on Twitter from the Internet.

We developed a method for calculating the chronological change in the number of tweets that have been broadly categorized into several topics(4) and for calculating the significance score of each shot based on the mapping between the topics and closed caption data. We also defined a method for calculating the significance score of each shot by attaching great importance to shots that include elements such as appearance of opened-captions (telops), close-up of faces, particular camera movement (e.g., zoom-in) and pattern images (e.g., illustrative CG). On the basis of this, we developed a method for automatically generating summarized video based on the “overall significance score” of each shot derived by carrying out a weighted summation of each element score as well as a mechanism for allowing the user to adjust the weight allocation to create various types of summarized video (Figure 1).

In our research on the significance evaluation of video sections, we demonstrated that analyzing face deformation data collected from sensors can extract subtle changes in facial expressions more accurately than by conventional facial expression recognition technologies and that each viewer has a particular tendency. Part of this research was conducted in cooperation with Waseda University.

[References](1) Y. Kawai, T. Mochizuki, M. Sano, H. Sumiyoshi: “Training and

Evaluation of Joint Cascade Face Detection Targeting TV video,” Proc. of ITE Annual Convention, 13B-5 (2016) (in Japanese)

(2) R. Endo, Y. Kawai, H. Sumiyoshi, M. Sano: “A Robust Method of Detecting Multi-oriented Scene Text Candidate,” Proc. of ITE Annual Convention, 22B-4 (2016) (in Japanese)

(3) T. Mochizuki, Y. Kawai, M. Sano, H. Sumiyoshi: “Scene Segmentation of Program Video by Color Formation Based on Superpixel,” Proc. of ITE Annual Convention, 14B-3 (2016) (in Japanese)

(4) A. Matsui, T. Kobayakawa, Y. Yamanouchi: “A Visualization of TV-related Tweets based on the Target Scene Contents,” Proc. of IEICE General Conference, D-9-12 (2016) (in Japanese)

4.2 Content production support technology

With the aim of supporting broadcasters with program production, we are researching a technology to extract and present the information necessary for program production and the structure information of past related programs by analyzing text big data including social media and program archives.

To make use of information from social media, we built a social media analysis system that can analyze the co-occurrence and frequency changes of words from tweets on Twitter. This system shows, for example, that tweets containing a word “ambulance” are often accompanied with particular words such as “hospital” and “accident”. Also for prompt reporting, we developed a method for automatically collecting useful information for program production in real time from Twitter. This method determines whether each tweet is useful for programs by entering the tweet text into a recurrent neural network (RNN) that extracts and learns the text features precisely. We incorporated an attention mechanism for

assigning weights to important parts and a multitask learning capability for processing multiple tasks simultaneously into this process and confirmed their effectiveness.

In our work on production support using TV program archives, we developed a system that presents producers with past program topics in chronological order and proposes candidate topics to be featured next on the basis of analysis of the information of previously broadcast programs such as TV program scenarios, closed captions and program summaries. We equipped this system with the similarity-based program retrieval algorithm using a network of semantic relations between words. We also added a function to search program structure information.

[References](1) T. Miyazaki, I. Yamada, K. Miura, M. Miyazaki, A. Matsui, J. Goto, H.

Sumiyoshi: “TV Program Retrieval using Semantic Relations Dictionary,” IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB 2016) IEEE, 10C-1 (2016)

(2) K. Miura, H. Sumiyoshi, A. Matsui, T. Miyazaki, I. Yamada, J. Goto: “A Proposal of Time Series Analysis Method on TV Program Themes,” ITE Winter Convention, 13C-3 (2016)

(3) T. Miyazaki, S. Toriumi, Y. Takei, I. Yamada, J. Goto: “Automatic Tweet Extraction for News Writers,” NLP2017, D4-1 (2017)

Figure 1. Mechanism of generating summarized video

Tweet content score

Opened-caption score

Face close-up score

Camera movement score

Pattern image score

0.5 0.2 0.3 0.1 0.00.3 0.0 0.5 0.0 0.70.0 0.0 0.2 0.6 0.50.3 0.0 0.3 0.2 0.40.0 0.8 0.0 0.0 0.0

S1S2S3S4S5

w1=w2=1.0, w3=w4=w5=0.0

0.8 0.2 0.8 0.1 0.7

w1=w2=0.0, w3=w4=w5=1.0

0.3 0.8 0.5 0.8 0.9

Program videoShot 1 Shot 2 Shot 3 Shot 4 Shot 5

Shot 1 Shot 2 Shot 3 Shot 4 Shot 5 Shot 1 Shot 2 Shot 3 Shot 4 Shot 5

Shot 1 Shot 3 Shot 5 Shot 2 Shot 4 Shot 5

Summarized video 1 Summarized video 2

Overall significance score S Overall significance score S

Weight allocation adjustable by the user

Overall significance score of each shot S ＝ Σwisi

Figure 1. Information collection from social media

Social media information such as tweets

Production site

Social media analysis

Useful information for news report

[Useful] Truck accident

[Useful] Fire

Unnecessary tweet

Unnecessary tweet



4.3 “Smart Production Lab”

R&D on the practical use of artificial intelligence (AI) and big data are becoming a growing trend around the world. The use of AI technology could also benefit broadcasters, who handle a massive amount of program-related video, audio and text data, by enabling the automatic extraction of useful information for program production and the speedy and effective production of programs and content.

We are researching a program production assistance technology using AI in order to change the working style of program production and expand the capability of producers. We call this technology “smart production”. In FY 2016, we aggregated our research outcomes of elemental technologies for smart production, including image analysis, speech recognition and social media analysis, and built a “Smart Production Lab” in our laboratory (Figure 1). We plan to accelerate our effort toward the practical application of AI in cooperation with program producers.

4.4 Bidirectional field pick-up unit (FPU) transmission technology

We are researching bidirectional field pick-up units (FPUs) for the high-speed wireless transfer of file-based video footage. In FY 2016, we worked to establish elemental technologies for high-speed bidirectional transmission. This included increasing the transmission capacity by improving the hybrid automatic repeat request (HARQ) retransmission control scheme, reducing latency by redesigning the transmission frame structure, and improving the operability by using an automatic IP-address mapping function.

For high-speed file transfer, we improved the HARQ function that controls retransmissions in error correction block units. Computer simulations confirmed that the signal-to-noise ratio (SNR) at a bit error rate of 10-7 and with a coding rate of 8/9 was improved by about 6 dB when a turbo code, instead of a convolutional code, was used as the error correction code. We investigated an adaptive control method of the retransmission block sizes of the HARQ in accordance with the channel quality. The results of evaluation using a prototype device demonstrated that the method can achieve file transfer in excess of 190 Mbps in the 18-MHz bandwidth, which is used by existing FPUs.

A bidirectional FPU must be able to transmit a live video stream without delay. We investigated a transmission frame

structure with low latency in order to send studio video and audio back to the reporting site and operate devices remotely. The new frame structure has its payload subdivided into multiple regions and stores incoming data in each region until just before it is sent. This makes it possible to store the data that was entered during the transmission of a frame into the same frame and send it together. The results of evaluation using a prototype device implemented with this transmission frame structure showed that the round trip transmission latency that occurs with an FPU was halved compared with that of a conventional device. We thus demonstrated the feasibility of low-latency transmission equivalent to 200 ms, which is required for fixed-line phones(1).

When using a bidirectional FPU, it is necessary to send an outside broadcast (OB) van equipped with the FPU to the reporting site and open up a dedicated high-speed IP link to enable bidirectional video transmission, remote control and IP connection with the broadcast station. Previously, communication settings such as the IP codec needed to be set manually. To improve the operability, we developed a function that allows a bidirectional FPU itself to set IP packet channels automatically.

[References](1) T. Koyama, F. Uzawa, T. Kumagai, K. Mitsuyama, K. Aoki, N. Iai: “A

Study on Frame Format of Bidirectional FPU to Decrease Transmission Delay,” 2017 IEICE General Conference, (2017) (in Japanese)

Figure 1. Smart Production Laboratory

Figure 1. Bidirectional FPU experimental equipment



4.5 Wireless cameras

With the goal of realizing a 4K and 8K Super Hi-Vision wireless camera, we researched a transmitter and receiver using millimeter-wave-band radio waves. We prototyped a modulator and demodulator adopting a single-carrier transmission with a frequency domain equalization (SC-FDE) scheme that could enable a more compact transmitter than by the orthogonal frequency division multiplexing (OFDM) scheme (Figure 1). This wide-band SC-FDE modulator and demodulator achieved a transmission bandwidth of 125 MHz, twice that of a conventional modulator and demodulator(1). They use the concatenated coding of Reed-Solomon codes and convolutional codes for forward error correction. Laboratory experiments demonstrated that the modulator and demodulator are capable of 200-Mbps-class transmission with 32APSK (amplitude and phase shift keying) and a 1/2 coding rate. We also studied the application of 2×2 (two transmitters and two receivers) multiple-input multiple-output (MIMO) technology to the SC-FDE scheme in order to achieve 400-Mbps-class transmission capacity, and we prototyped experimental equipment.

To further expand the transmission capacity, we are researching 4×4 MIMO-OFDM transmission technology. We previously proposed a method of using “block QR decomposition” to reduce the number of computations in maximum likelihood detection (MLD) used for signal detection on the receiver side and built a prototype(2). In FY 2016, we added an error correction function to the prototype and demonstrated real-time transmission of video transport streams by using 4x4 MIMO-OFDM.

[References](1) Y. Matsusaki, H. Kamoda, K. Imamura, H. Hamazumi: “Development

of Wideband SC-FDE Modulator and Demodulator for Millimeter-wave Band,” Proc. of 2017 IEICE General Conference, B-5-45, (2017) (in Japanese)

(2) F. Ito, T. Nakagawa, H. Hamazumi, K. Fukawa: “Development of Complexity-reduced 4x4 MIMO-MLD Demodulator with Block QR Decomposition,” IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), 2A-3 (2016)

Figure 1. Wide-band SC-FDE modulator


5 User-friendly broadcasting technologies

5.1 Information presentation technology

NHK STRL is researching technology for translating weather information into sign language CGs as well as kinesthetic display technology for conveying the shape and hardness of a 3D object so that people with hearing and vision impairments can enjoy broadcasts.

■ Sign language CGs with facial expressions for presenting weather information

To enhance broadcasting services for viewers who mainly use sign language, we are researching technology for automatically generating sign language animations using

computer graphics (sign language CGs) for weather reports(1).In sign language, manual gestures, facial expressions and

mouth movements (“mouthing”) play an important role in conveying text information. Our conventional sign language CGs create a continuous movement by connecting motion data that is predefined for each word of sign language. In FY 2016, we developed a function to automatically modify motion junctions between words and an interface for manually changing the order of sign language words in order to achieve more natural and smoother manual movements. We also integrated a function to express facial expressions and mouthing into our previously developed sign language CG

We are conducting R&D on user-friendly broadcasting that conveys information quickly and accurately to all viewers, including people with hearing or visually impaired and non-native speakers, by converting broadcast content data automatically.

In our research on information presentation, we continued our development of sign language CG characters that have facial expressions as presenters of weather information. We developed an automatic modification function to improve the manual gestures of CG characters. We also added a function to express facial expressions and mouthing to our sign language CG translation system and conducted subjective evaluations. We put our system to practical use to automatically generate sign language CGs from weather forecast data distributed by Japan Meteorological Agency and launched an evaluation website in NHK Online.

In our study on technologies for the kinesthetic presentation of the contour and hardness of a 3D object, we developed a device that can instantly convey the size and shape of a virtual CG object by letting the user “hold” it with their thumb, forefinger and middle finger and “active touch” its surface gently.

In our research on speech recognition technology for closed captioning, we developed an end-to-end speech recognition technique that does not use a pronunciation dictionary for information programs containing background noise and inarticulate speech. We also developed a method of estimating the topic of a program and the probabilities of word sequences for a topic for programs in which topics are changed frequently. This method can maintain the accuracy of word sequence estimation even when the accuracy of topic estimation is low, improving the overall recognition accuracy.

We began research on automatic audio descriptions with the aim of providing a new commentary service that can be used for live programs. For use in the Rio 2016 Olympic and Paralympic Games, we developed a system for automatically generating audio descriptions of athlete names, scores and game progress based on the analysis of competition data provided by Olympic Broadcasting Services. The system provided automatic audio descriptions for 1,929 games as of September 2016 and for 2,496 games as of January 2017. We conducted experiments on the subjective evaluation of automatically generated audio descriptions in cooperation with people with visual impairment and sighted people and confirmed their effectiveness for both. In our research on speech synthesis technology for automatic audio descriptions, we built a versatile speech synthesis system using a deep neural network (DNN)-based acoustic model.

In our research on language processing technology, we studied the feasibility of providing news scripts with reading assistance information for non-native speakers in Japan. Reading assistance information, such as explanations in easy Japanese, kana syllables of Chinese characters, dictionary information for difficult words, coloring of proper nouns and translations into foreign languages, can make news content easier to understand. We developed a system that automatically produces reading assistance information by using machine translation technology and allows the operator to modify errors manually.

In our research on image cognition analysis, we researched image features suitable for a wide field-of-view environment such as 8K Super Hi-Vision. We investigated the relationship between the preferred image size, image features and the viewing distance by using 100 different types of images. The results indicated that changing the viewing distance does not significantly affect the relationship between the preferred image size and image content. We also identified the influence of the shaking duration and viewing angle on the degree of unpleasantness caused by shaking images.



translation system and asked hearing impaired people to evaluate the system. The results showed that adding facial expressions and mouthing achieved higher evaluation scores than those of the conventional system.

Our sign language CG generation system using weather data automatically converts the weather forecast data provided in XML format by Japan Meteorological Agency into sign language CGs. In FY 2016, we developed a function to generate sign language for weather warnings and advisories in addition to regular weather forecasts(2). We also improved the processing speed and stability of the system by integrating processes from the reception of weather data and the generation of sign language CG animations to the uploading of data to the website. We released a website for evaluating the weather report sign language CGs automatically generated by this system (Figure 1) in NHK Online in February 2017.

We also prototyped a demonstration system that automatically displays sign language CGs urging to evacuate on tablet devices at the time of receiving an Emergency Earthquake Warning on TV, by using the Hybridcast receiver’s functionality for linkage with other terminals(3).

Part of this study was conducted in cooperation with Kogakuin University.

■ Technologies for kinesthetic display of the shape and hardness of an object

We are researching a technology to convey the information of images and 3D objects to the user’s fingers. We previously developed a device that expresses the shape of a virtual object expressed in a CG when the user holds and touches it with their thumb and forefinger. In FY 2016, we developed a new device. When the user holds a virtual CG object with their thumb, forefinger and middle finger and moves their hand over the surface of the object gently, the device activates its linear actuator, which pushes the user’s fingers outward according to their hand movements to express the size and shape of the virtual object instantly. We also implemented a function to present the hardness or softness of the surface that the user is touching (Figure 2). Moreover, we conducted experiments to study how people recognize the hardness and softness of objects and obtained useful knowledge for system design (4).

We are also researching a technology for presenting 2D

information such as diagrams and graphs in textbooks to people with visual impairment. We continued with our development of a system that expresses such information by using pin arrays that move up and down electronically to form lines and surfaces. We previously developed a device that gives vibrations to pin arrays and mechanically guides the user’s finger. In FY 2016, we confirmed from evaluation experiments by visually impaired people that the mechanical guiding helps them form a cognitive map in their mind and promotes understanding of the information. We also improved the system for the purpose of practical use at schools for visually impaired people, by adding a function that allows the teacher to remotely control a finger-guiding device for multiple students via the Internet. This function, which would enable the teacher to convey the information of a specific part to multiple students simultaneously, demonstrated the applicability of the system for remote teaching in distance learning (5). Part of this research was conducted in cooperation with Tsukuba University of Technology.

Figure 2. Device for presenting the shape of an object to three fingers

[References](1) S. Umeda, T. Uchida, M. Azuma, T. Miyazaki, N. Kato, N. Hiruma:

“Automatic Production System of Sign Language CG Animation for Meteorological Information”, International Broadcasting Convention Conference (2016)

(2) M. Azuma, N. Hiruma, T. Uchida, T. Miyazaki, S. Inoue, S. Umeda, N. Kato: “Development of Automatic Sign Language Animation System to Express Weather Warnings”, IEICE Technical Report, Vol.116, No.248, WIT2016-35, pp.11-15 (2016) (in Japanese)

(3) T. Uchida, S. Umeda, M. Azuma, T. Miyazaki, N. Kato, N. Hiruma, S. Inoue, Y. Nagashima: “Provision of Emergency Information in Sign Language CG Animation over Integrated Broadcast-Broadband System”, IEEE International Symposium on Broadband Multimedia Systems and Broadcasting, 11C-3 (2016)

(4) T. Handa, K. Murase, M. Azuma, T. Shimizu: “A Haptic Three-dimensional Shape Display with Three Fingers Grasping”, IEEE Virtual Reality 2017, p.325-326 (2017)

(5) T. Sakai, T. Handa, M. Sakajiri, T. Shimizu, N. Hiruma, J. Onishi: “Development of Proprioception-Tactile Display and Evaluation of Effect for the Local Vibration Presentation Method,” Journal of Advanced Computational Intelligence and Intelligent Informatics (JACIII) vol.21, no.1, p.87-99 (2017)

Figure 1. Evaluation website for weather report sign language CGs (in NHK Online) http://www.nhk.or.jp/strl/sl-weather/

Figure 2. Device for presenting the shape of an object to three fingers



5.2 Speech recognition technology

We are researching speech recognition for efficiently producing closed captions for live programs so that more people including the elderly and those with hearing difficulties can enjoy TV programs. Live information programs often have conversations during cooking and interviews with people in the street. Background sound such as work noise and inarticulate pronunciation pose a challenge in speech recognition.

■ Conversation recognition

The accuracy of speech recognition in a TV program declines when there is background noise and inarticulate speech during a conversation. When possible, a re-speaker can be employed to increase the recognition accuracy in such situations. A re-speaker works in a quiet room and speaks or rephrases the original noisy or inarticulate speech, which is then automatically recognized for captioning. However, many local broadcasters cannot afford to employ a re-speaker. Thus, for them to produce closed captions for information programs, it is an urgent task to develop a way of automatically recognizing program audio without the need for rephrasing.

For the direct recognition of conversational speech, we previously adopted deep neural networks (DNNs) to develop an acoustic model that estimates vowels and consonants of input speech in order to increase recognition accuracy. We also added the estimated speech variability into our pronunciation dictionary to recognize imprecise words that are spoken differently from the original pronunciation in the dictionary.

In FY 2016, we developed an algorithm to select only possible speech variations from a huge amount of possible variations generated by the variation estimation technique. This algorithm reduced the number of recognition errors caused by false estimation(1). To recognize speech that cannot be handled by the pronunciation dictionary of phoneme sequences that have been accumulated manually, we developed a speech recognition technique that does not require a pronunciation dictionary, called “end-to-end speech recognition.” This technique trains a DNN that maps input speech and a recognition output text directly without using phonemic symbols. Using the technique’s capability to identify the correct reading of characters from the context, we succeeded in developing end-to-end speech recognition for Japanese(2)(3).

In some information programs, topics change one after another at short intervals, covering diverse areas such as cooking, events and travel reports. Speech recognition for such programs requires a language model that gives the probabilities of word sequences corresponding to individual topics. As it is difficult to accurately estimate the topic when it changes

frequently in a program, we developed a DNN that was trained to estimate the probabilities of word sequences and the topic concurrently(4). This method can maintain the accuracy of word estimation even when it is difficult to identify shifts in the topic in a program and thus increases the recognition accuracy.

Out of the elemental technologies for speech recognition that we have developed previously (Figure 1), feature extraction, the acoustic model, the language model and the search technology have become replaceable by a DNN, which improves the accuracy of each technology. With these new technologies, we reduced the recognition error rate of program speech from 9.5% to 6.9% in an experiment using the local information program “Hirumae Hotto.”

■ Speech recognition for program production

The transcription of speech in video footage is indispensable to produce useful programs out of video materials collected for program production. Now that a huge amount of news video materials can be collected easily thanks to the development of network technologies, a system is required to produce speech transcription swiftly and efficiently. Against this backdrop, we began developing a transcription system using speech recognition technologies and a user interface (Figure 2) for correcting recognition errors efficiently by mapping recognition results with video materials.

[References](1) M. Ichiki, A. Hagiwara, H. Ito, K. Onoe, S. Sato, and A. Kobayashi:

“SMT-Based Lexicon Expansion for Broadcast Transcription,” APSIPA ASC, Paper ID 111 (2016)

(2) H. Ito, A. Hagiwara, M. Ichiki, T. Mishima, S. Sato, and A. Kobayashi: “Word Modeling for End-to-end Japanese Speech Recognition,” IEICE technical report, vol. 116, no. 279, SP2016-47, pp. 31-36 (2016)

(3) H. Ito, A. Hagiwara, M. Ichiki, T. Mishima, S. Sato, and A. Kobayashi: “End-to-end Speech Recognition Considering Reading of Kanji,” Spring Meeting of the Acoustical Society of Japan, 1-5-11, pp. 31-32 (2017)

(4) A. Hagiwara, H. Ito, M. Ichiki, T. Mishima, S. Sato, and A. Kobayashi: “Domain Adapted Language Model by Multi Task Learing,” Spring Meeting of the Acoustical Society of Japan, 1-5-7, pp. 19-22 (2017)

Figure 1. Speech recognition elemental technologies that can be replaced by DNN

Training data automatically generated from broadcast programs

Speech data

Text corresponding to the speech data

Topic-trackinglanguage

model

Acoustic model Language model

Feature extraction

Noise reduction Search

Deep learning acoustic modelCooperative

conversation

Noise reduction autoencoder End-to-end speech recognition

Recognition result

Figure 2. User interface for speech transcription



5.3 Automatic audio description

We are researching “automatic audio description” for producing commentaries for live broadcast programs so that people with visual impairment can enjoy live sports coverage.

■ Survey on the necessity of automatic audio description

We investigated past Olympic programs to determine the percentage of programs that had running commentaries on the competition status(1). The results showed that information on the score displayed on TV was rarely spoken in some competitions and that only 35% of athlete lists and standings was spoken in athletics and swimming, indicating that it is difficult for listeners to understand the competition status only from the TV sound of sports programs.

Audio descriptions of TV dramas are produced under the restriction that they must not overlap program speech. For sports and other programs that have frequent speech, however, it is difficult to insert commentaries into silent intervals. We therefore produced an evaluation sports program that has program speech and commentary overlapping with each other, which we asked visually impaired people to evaluate(2). We confirmed from the results that commentaries describing the competition status are necessary for them to enjoy sports programs regardless of the amount of overlap.

■ Experiments in the Rio 2016 Olympics and Paralympics

We conducted an experiment on generating automatic audio descriptions of the competition status such as athlete names and scores by using the Olympic Data Feed (ODF) competition data distributed by Olympic Broadcasting Services. In this experiment, we used video that NHK distributed via the Internet without running commentaries. We created templates for generating speech texts for each sport in advance. When the data on a shot or goal was distributed by the ODF, an automatic audio description text was generated, synthesized and superimposed on the video. We also developed a transliteration method that considers athletes’ nationalities because some names have different readings depending on the language even if they have the same spelling(3). During this experiment, we generated automatic audio descriptions for video of 1,929 competitions in 17 Olympic sports and seven Paralympic sports(4). Figure 1 shows an example of commentary automatically generated form ODF. We asked people with visual impairment and sighted people to subjectively evaluate the video content with automatic audio descriptions and confirmed that the content was useful for both of them. We also created speech templates for other sports after the

experiment to generate automatic audio descriptions. So far, we have produced video content with automatic audio descriptions for a total of 2,498 competitions in 22 Olympic sports and seven Paralympic sports. We also studied a speech synthesis technology for automatic audio descriptions. We changed the acoustic model for the versatile speech synthesis system from the conventional hidden Markov model (HMM) to a deep neural network (DNN) model that can express a more complex structure. A subjective evaluation demonstrated that the DNN method can produce more diverse speech styles, which are required for automatic audio descriptions, with higher sound quality than the conventional HMM method(5).

■ Application for practical use

We developed a reading technology that allows the user to select the reading speed of the synthesized speech of electronic books for visually impaired people from 3x, for which the content is audible, to 8x, for which the outline can be grasped.

[References](1) S. Sato, T. Kumano, N. Seiyama, A. Imai, I. Yamada: “A System

Verification for Automatic Speech Guidance for Sports Broadcasts,” The Special Interest Group Technical Reports of IPSJ, Vol.2016-SLP-113, No.6, p.6 (in Japanese)

(2) A. Imai, R. Tako, K. Onoe, N. Seiyama, S. Sato, T. Miyazaki, T. Kumano, I. Yamada, M. Iwaki: “A Study on New Service for Overlapped Audio Information on TV Programs,” Proc. of IEICE General Conference, H-4-11, 2016, p.322 (in Japanese)

(3) T. Miyazaki, T. Kumano, A. Imai: “Improving Transliteration Quality of People’s Name by Considering Nationality Information,” Proc. of IPSJ FIT2016, No.2, E-018, 2016, pp.145-146 (in Japanese)

(4) A. Imai, N. Seiyama, T. Kumano, T. Miyazaki, S. Sato: “A study of automatic audio description service for TV sports programs,” CSUN Assistive Technology Conference 2017

(5) K. Kurihara, N. Seiyama, A. Imai, T. Takagi: “The DNN-based speech synthesis using speaker codes and emotional codes,” Proc. of IEICE General Conference (in Japanese)

5.4 Language processing technology

As a new way to provide information to non-native speakers in Japan, we began research on news scripts with reading assistance information. We are also studying a system that enables the one-person production of easy Japanese news scripts for NEWSWEB EASY.

■ News scripts with reading assistance information

News texts available on the Internet are not easily understandable for many non-native speakers. We therefore

began research on various kinds of reading assistance information to make news scripts easier to understand for them. We employed the followings for reading assistance: easy Japanese explanations, kana syllables of Chinese characters, dictionary information for difficult words, coloring of proper nouns and translations into foreign languages. We prototyped example content using them (Figure 1). We also surveyed teachers and students at Japanese language schools about the effectiveness of the news content with such reading assistance information and were told that it is effective and should be

Figure 1. Example of a commentary automatically generated from competition data



provided.

■ System for producing reading assistance information

Reading assistance information such as easy Japanese explanations, translations into foreign languages and kana syllables are automatically generated by a machine translation system and a morphological analyzer. We developed a system to correct the errors and insufficiency of reading assistance information that may occur during automatic generation. This system allows the operator to mouse-click a part of the reading assistance information displayed above and below words such as kana syllables for manual correction. The system is also equipped with a function to easily change the difficulty level of words by mouse operation, which decides whether dictionary

information should be displayed or not.

■ Machine translation technology

We studied the use of neural machine translation technology for the automatic generation of English reading assistance information. Neural machine translation can miss out some parts of the input text during translation. To address this problem, we developed a technique to detect such missing parts. We also developed a method for using this technique to select one of several alternative translations that have no missing parts. Experiments demonstrated the improvement in the translation performance(1). We also prototyped a Japanese-Korean translation system and an English-Spanish translation system and confirmed that high translation performance can be achieved for these combinations of languages.

■ One-person production system of NEWSWEB EASY

Today easy Japanese news scripts for NEWSWEB EASY are produced jointly by a reporter and a Japanese teacher familiar with easy Japanese. To produce easy Japanese news scripts in emergencies or at local broadcast stations where such a production structure cannot be always secured, we are conducting R&D on a single-person production system that enables a reporter to produce scripts by himself/herself. In FY 2016, we prototyped a text editor that uses statistical machine translation technology to propose alternative scripts based on the knowledge of Japanese language instructors to a reporter writing an easy Japanese news script.

[References](1) I. Goto and H. Tanaka: “Detecting Missing Translation Part for Neural

Machine Translation,” Proc. of the annual meeting of Natural Language Processing (NLP), P18-3 (2017) (in Japanese)

5.5 Image cognition analysis

We are engaged in researches for identifying image features suitable for wide-angle viewing with the aim of contributing video production taking advantage of the wide field-of-view (FOV) environment of 8K Super Hi-Vision (SHV).

■ Measurement of preferred image size

In FY 2016, we conducted an evaluation experiment using 8K SHV images that contain various types of objects and scenes captured in various ways to find out the features of images that people prefer to view with a wide FOV. We measured the preferred size of the FOV of 100 different kinds of images. The experiment was performed by displaying them to

the participants while varying the displaying-size (image size) and the distance from the screen. We investigated the effects of features of the image content and the viewing distance. The results showed that the preferred size varied with the type and size of the main object in the image. We also found that the change in the viewing distance does not significantly affect the relationship between the preferred image size and the image content(1). We plan to analyze the influence of the real size and screen size of objects and the impressions given by the entire image on the preferred image size. We also plan to evaluate the effect of the screen size in order to extract the effective features of images adaptable to wide-angle viewing environments.

■ Shaking image analysis technology

Viewers may experience unpleasantness similar to motion sickness when viewing video showing a lot of movement over a large viewing angle. We are researching a technology for analyzing such images and estimating the degree of unpleasantness caused by viewing them. In FY 2016, we conducted psychological experiments using 4K images to investigate how the amount of recognizable shaking motion of the screen (cognitive quantity of shakiness) and the degree of unpleasantness vary with the duration of image shaking and the viewing angle. The results showed that the cognitive quantity of shakiness saturates but the degree of unpleasantness continues to increase when the duration of shaking exceeds

Figure 1. Example of easy Japanese news scripts with reading assistance information

Figure 1. Experiments on evaluating the preferred sizes of various types of images

Viewing distance 0.75H / 1.5H (H: Screen height) = Viewing angle of the screen:100˚/61̊

Screen size 85-inch 8K display

Image size Enlargement ratio : 1 - 1/4



about three seconds. We found a relationship between the degree of unpleasantness and the cumulative amount of the cognitive quantity of shakiness. We also confirmed that both the cognitive quantity of shakiness and the degree of unpleasantness increased with the viewing angle of shaking images(2).

On the basis of the results, we refined our algorithm for estimating the cognitive quantity of shakiness and the degree of unpleasantness from the physical characteristics of images such as the size and shape of the moving area and the constituents of the spatio-temporal frequency. We verified the validity of values estimated by this algorithm through experiments using general images.

[References](1) M. Harasawa, Y. Sawahata and K. Komine: “The effect of viewing

distance on preferred physical size of high-resolution moving images,” IEICE Technical Report, Vol.116, No.513, pp.31-36 (2017) (in Japanese)

(2) M. Tadenuma: “Basic Relationship between Physical Cognition Degree of Shakiness and Unpleasantness,” ITE Technical Report, Vol.41, No.5, pp.33-36 (2017) (in Japanese)


6.1 Advanced image sensors

■ Three-dimensional integrated imaging devices

We are researching imaging devices with a 3D structure in our quest to develop an image sensor having a large number of pixels that can be used as part of a future three-dimensional imaging system. These devices have a signal processing circuit for each pixel directly beneath the photodetector. This enables signals from all pixels to be read out simultaneously so that both a large number of pixels and a high frame rate can be

achieved at the same time (Figure 1). We previously prototyped a two-layered device with 128×96 pixels (80 μm square each)

6 Devices and materials for next-generation broadcasting

We are researching the next generation of imaging, recording, and display devices and materials for new broadcast services such as 8K Super Hi-Vision (SHV).

In our research on imaging devices, we made progress in developing 3D integrated imaging devices, low-voltage multiplier films for solid-state image sensors, and organic image sensors. In our work on 3D integrated imaging devices capable of pixel-parallel signal processing, we reduced the pixel size to 50 μm square by halving the diameter of connection electrodes to 5 μm and modifying the circuit layout. We also developed a circuit for eliminating noise and improved the fabrication process. Our work on low-voltage multiplier films for solid-state image sensors with high sensitivity included reducing the dark current by changing the fabrication process and reducing the noise of signal-reading circuits. In our work on single-chip organic image sensors with an image quality comparable to that of a three-chip camera, we improved the efficiency of organic photoconductive films and prototyped a transparent cell for green having a maximum quantum efficiency of 80%.

In our research on recording devices, we continued with our work on holographic memory with a large capacity and high data transfer rate for SHV video signals, and on a high-speed magnetic recording device with no moving parts that utilizes the motion of magnetic domains in magnetic nanowires. In holographic memory, we developed a prototype drive that has a recording density of 2.4 Tbit/inch2 and a data transfer rate of 520 Mbps and verified its operation by recording and reproducing compressed SHV video signal. We also began studying multi-value recording to increase the recording density and data transfer rate. In magnetic nanowires, we investigated for suitable magnetic nanowire materials, conducted simulations of magnetic domain formation and driving domain analysis, and widened the bandwidth of our recording and reproduction evaluation system in order to increase the driving speed of magnetic domains. This led to magnetic domain driving in excess of 1 m/s, more than 10 times that of conventional devices.

In our research on displays, we studied an organic light-emitting diode (OLED) with a longer lifetime and solution-processed devices for large SHV displays for home use. We also developed elemental technologies for a next-generation display with higher image quality and lower power consumption. For an OLED with longer lifetime, we researched a device structure and materials that achieve both high efficiency and long lifetime and developed a red OLED device with an internal quantum efficiency of 100% and a lifetime of beyond 10,000 hours. For solution-processed devices, we developed a technology for increasing the mobility of solution-processed oxide TFTs and a technology for improving the efficiency of quantum-dot light-emitting diodes (QD-LEDs). In our work on displays with higher image quality and lower power consumption, we investigated oxide semiconductor materials suited for high-mobility TFTs. We also prototyped driving equipment that controls the temporal aperture in line units to suppress motion blur on hold-type displays such as OLED displays and demonstrated its effectiveness.

Figure 1. Concept diagram of 3D integrated imaging device

EGB

Pixel-parallel signal processing

Light

PixelPhotodetector

Signal processing circuit

Figure 2. Variance of output relative to variance of standard voltage

-30 -20 -10 0 10 20 30-20

20

10

0

-10Var

ianc

e of

out

put (

%)

With correlated double sampling circuits

Without correlated double sampling circuits

Variance of standard voltage (%)



and confirmed that it can output a signal of 16 bits with a wide dynamic range by utilizing a circuit structure taking advantage of pixel-parallel signal processing.

In FY 2016, we reduced the pixel size, developed a circuit that can eliminate noise, and improved the fabrication process. We successfully miniaturized pixels to 50 μm square each by reducing the diameter of electrodes that connect the upper and lower circuits from 10 μm to 5 μm and modifying the circuit layout. For noise reduction, we devised a correlated double sampling (CDS) circuit that is capable of pixel-parallel operation and stacking into pixels(1). We conducted measurements by simulating changes in the standard voltage of a photodetector due to noise and confirmed that this circuit can reduce the variance of the output (Figure 2). Regarding the improvement of the fabrication process, we found that impurities that adhered to the bonding surface during the wafer bonding process decrease the bond strength, and we developed a pretreatment technique for preventing the adherence of impurities.

This research was conducted in cooperation with the University of Tokyo.

■ Low-voltage multiplier films for solid-state image sensors

The sensitivity of cameras incorporating solid-state image sensors decreases as the number of pixels and the frame rate increase because the amount of light incident on each pixel decreases. To address this problem, we are developing a solid-state image sensor overlaid with a photoconductive film (low-voltage multiplier film) on a CMOS circuit. A photoconductive film is able to multiply the electric charge by only applying a low voltage. In FY 2016, we improved the characteristics of crystalline selenium films and chalcopyrite CIGS films, which are two candidate materials for low-voltage multiplier films. We also studied the reduction of the noise of CMOS imaging devices on which a multiplier film is overlaid.

We considered that the dark current of crystalline selenium films is caused by tellurium, which is added to prevent films from separating from the substrate. We therefore developed a new deposition method for thinning the tellurium layer. The method achieved a tellurium layer with a thickness only one-tenth that of a conventional film while maintaining the original effectiveness that reduced the dark current at room temperatures by almost half (Figure 3)(2). We also improved the crystallinity of chalcopyrite CIGS films by changing the film deposition process.

A CMOS imaging device to be overlaid with a multiplier film has a different pixel circuit structure and signal-reading operation from those of an ordinary CMOS imaging device and thus requires a new process for reducing noise that occurs at the time of the reset operation. We investigated a method of performing digital correlated double sampling over two frames by using a prototype device. Evaluation results showed that

performing this process halved the noise to about 15 electrons (at a 60Hz frame frequency) and clarified the relationship between noise and the dark current of circuits(3).

■ Organic photoconductive film for single-chip cameras with high S/N

We are conducting research on organic image sensors with an image quality comparable to that of three-chip color broadcast cameras. These sensors consist of alternating layers of three different organic photoconductive films (organic films) sensitive to each of the three primary colors of light and transparent thin-film-transistor (TFT) circuits for reading the signals from the photoconductive films (Figure 4). The electrodes of organic image sensors that sandwich each organic film must be transparent in order to transmit light into the lower layers of the stacked organic films. In FY 2016, we developed a technology for improving the quantum efficiency of a transparent cell in which an organic film for green is sandwiched between transparent electrodes.

The quantum efficiency of the organic films of transparent cells is reduced to about 10% because the films are damaged by the high energy of material particles when transparent counter electrodes are formed on the films. To solve this problem, we adopted an electron beam evaporation technique that can theoretically reduce the energy of material particles as a new way of forming transparent electrodes. We also inserted a transparent buffer layer with a robust molecular frame between the organic film and transparent counter electrode in order to suppress the damage on the organic films (Figure 5(a)). Evaluation results of the spectral sensitivity of the prototype transparent cell for green demonstrated that a quantum efficiency of 80% was achieved when green light with 500-nm wavelength was irradiated (Figure 5(b)) and that a transparent counter electrode was successfully formed without damage on the organic films(4).

[References](1) M. Goto, Y. Honda, T. Watabe, K. Hagiwara, M. Nanba, Y. Iguchi, T.

Saraya, M. Kobayashi, E. Higurashi, H. Toshiyoshi, T. Hiramoto: “In-Pixel A/D Converters with 120-dB Dynamic Range Using Event-Driven Correlated Double Sampling for Stacked SOI Image Sensors,”

Figure 3. Dark-current characteristics of crystalline selenium film

0 1 2 3 4 5 6 71

100

10

Dar

k cu

rren

t den

sity

(pA

/cm

2 )

Voltage (V)

Room temperature (25℃)

Tellurium layer thickness : 1.0nm

Tellurium layer thickness : 0.1nm

Figure 4. Structure of organic image sensor

LightGreen

Blue Red

Transparent TFT circuit

Organic film for blueOrganic film for green

Organic film for red

Blue output

Green output

Red output

Figure 5. Cross section (a) and spectral sensitivity (b) of prototype transparent cell for green

400 500 600 700 8000

100

80

60

40

20

Applied voltage 15V

Buffer layer

Glass substrate

Transparent electrode

Organic film for green

(a) (b)

Wavelength (nm)

Qua

ntum

effi

cien

cy (

%)

Transparent counter electrode



Proc. of IEEE SOI-3D-Subthreshold Microelectronics Technology Unified Conference (IEEE S3S), 6b.3 (2016)

(2) S. Imura, K. Mineo, Y. Honda, K. Hagiwara, T. Watabe, K. Miyakawa, M. Namba, H. Ohtake, M. Kubota: “Improvement of Dark Current in c-Se-based Photodiode by Reducing the Thickness of Adhesion Layer,” Ext. Abstr. of 77th JSAP Autumn Meet., 16a-A35-5 (2016) (in Japanese)

(3) T. Watabe, Y. Honda, M. Namba, T. Kosugi, H. Ohtake, M. Kubota:

“Random Noise and Dark Current of Readout Circuit for Stacked CMOS Image Sensor,” Proc. of IEICE Society Conference, C-12-13 (2016) (in Japanese)

(4) T. Takagi, Y. Hori, T. Sakai, T. Shimizu, H. Ohtake, S. Aihara: “Characteristic Improvement in Color-Selective Photodetector with Organic Photoconductive Film Sandwiched between Transparent Electrodes ,” Ext. Abstr. of 64th JSAP Spring Meet., 17a-P4-20 (2017) (in Japanese)

6.2 Advanced storage technologies

■ Multi-level recording holographic memory

An archive system for the long-term storage of 8K Super Hi-Vision (SHV) video will need a very high data transfer rate and large capacity. We have been researching holographic memory to meet these needs. In FY 2016, we developed a practical prototype drive and improved the signal-to-noise ratio (SNR) for multi-level recording.

The prototype drive has a laser light source wavelength of 405 nm, a recording density of 2.4 Tbit/inch2, and a data transfer rate of 520 Mbps. It can record 2 terabytes’ worth of data into a 130-mm disk medium (Figure 1). We confirmed that SHV video signals compressed to 85 Mbps can be recorded in a disk medium and reproduced in real time using the prototype drive. We exhibited the drive and reproduced video at the NHK STRL Open House 2016(1). This drive is equipped with a wavefront compensation technology for suppressing the deterioration of reproduced signals due to hologram distortion. We confirmed that this technology is effective for reducing the error rate by more than 50% when reproduced signals have poor quality.

In our work on elemental technologies for holographic memory, we developed the dual-page reproduction technology and achieved a reproduction transfer rate of 1 Gbps in FY 2015(2). In FY 2016, we began studying multi-level recording to further increase the recording density and data transfer rate. For multi-level recording, it is necessary to develop technologies for improving the SNR. We investigated the use of a roll-off filter to reduce intersymbol interference, and confirmed using numerical simulation that it can improve the SNR by 1.9 dB. We also improved the SNR by 1.5 dB by applying a method that divides reproduced signals by a fixed-pattern noise element. This means that we can improve the SNR by more than 3 dB by combining these methods.

The prototype drive was developed in cooperation with Hitachi, Ltd., and Hitachi-LG Data Storage, Inc.

■ Magnetic high-speed recording devices utilizing magnetic nanodomains

With the goal of realizing a high-speed magnetic recording device with no moving parts, we are developing a recording device that utilizes the motion of nano-sized magnetic domains

in magnetic nanowires. We previously verified the operation principle for this recording device, i.e., the formation (recording), detection (reproduction), and current driving of magnetic nanodomains by adopting a magnetic recording head used in hard disk drives(3). In FY 2016, we developed fundamental technologies for stable recording, reproduction and high-speed driving.

On the basis of our finding that the misalignment of magnetic nanowires and the magnetic recording head deteriorates the recording efficiency and SNR, we added a mechanical system for precisely adjusting the contact angle and position of the head in the experimental setup for evaluating the recording and reproduction. In our quest for magnetic materials with low magnetization that would enable the high-speed driving of magnetic domains, we fabricated an artificial ferrimagnetic material that has an ultrathin (0.3-0.4 nm) ruthenium interlayer between cobalt/palladium multilayered films. The new material reduced the net magnetization by 80%(4). We prototyped a nanowire structure using this material and drove magnetic domains with our experimental setup. The results demonstrated the successful high-speed driving of magnetic domains with 1/12 of the driving current density and more than ten times the driving speed of conventional materials (Figure 2).

As elemental technologies for further increasing the driving speed, we modified the signal preamplifier system of a magnetic recording head that is used to detect magnetic domains in magnetic nanowires. In particular, we increased the bandwidth of the reproduction system from the conventional value of 100 MHz to about 1 GHz. We also conducted simulations using the Landau–Lifshitz–Gilbert (LLG) equation, which describes magnetization dynamics and damping in general magnetic materials, to investigate the directional dependence of a recording magnetic field on the magnetization reversal time. We found that domain nucleation for magnetization reversal starts at 10 picoseconds by applying a magnetic field obliquely to magnetic nanowires and demonstrated that the speed of magnetic domain formation can be increased by changing the direction of a recording magnetic field.

Figure 1. Prototype drive

Figure 2. Cross section of the prototype artificial ferrimagnetic nanowire and results of magnetic domain driving

100

–100

–60

–20

20

60[Co/Pd]

[Co/Pd]

500nm

Multilayered film

Surface oxidized Si wafer substrate

Multilayered film

Ru interlayer

Under layer

Mag

netiz

atio

n (a

rbitr

ary

unit)

Before driving

After driving

(a) (b)



[References](1) Y. Katano, T. Muroi, N. Kinoshita, N. Ishii: “Prototype Holographic

Drive with Wavefront Compensation for Playback of 8K Video Data,” Proc. IEEE ICCE, pp.317-318 (2017)

(2)Y. Katano, T. Muroi, N. Kinoshita and N. Ishii:“ Efficient High-Speed Readout in Holographic Memory by Reusing Transmitted Reference Beam,” MOC, 14B-2 (2016).

(3) M. Okuda, Y. Miyamoto, M. Kawana, E. Miyashita, N. Saito, S.

Nakagawa: “Operation of [Co/Pd] nanowire sequential memory utilizing bit-shift of current-driven magnetic domains recorded and reproduced by magnetic head,” IEEE Trans. Magn. Vol.52, No.7, pp.3401204-1—3401204-4 (2016)

(4) M. Okuda, M. Kawana, Y. Miyamoto: “Current-Driven Magnetic Domains Motion in [Co/Pd] Nanowire with Ru Interlayer,” Proc. ITE Annual Convention, 34D-5 (Aug. 2016) (in Japanese)

6.3 Next-generation display technologies

We are researching a flexible organic light-emitting diode (OLED) display with a longer lifetime, solution-processed devices for large rollable displays, and technologies for a next-generation display with higher image quality and lower power consumption.

■ Flexible OLED displays with longer lifetime

We are researching organic device structures and materials that extend the operating/storage lifetime and reduce the power consumption of flexible OLED displays. OLED devices use active materials such as alkali metals for their electron injection layer. These materials are sensitive to moisture and oxygen and deteriorate on the substrate. This poses the greatest challenge in realizing a flexible OLED display. To address this issue, we are developing an OLED that does not use alkali metals and can better withstand oxygen and moisture, called an “inverted OLED”. In FY 2016, we studied a power-saving inverted OLED with a long lifetime with the aim of realizing a flexible OLED display. We developed a new doping technique for improving the electron injection performance of the inverted OLED. This led to the development of a practical red device that requires a driving voltage of only 3.4 V at a luminance of 200 cd/m2, 2 V less than that for a conventional device, and has an internal quantum efficiency of about 100% and a lifetime in excess of 10,000 hours (1). The device was developed in cooperation with Nippon Shokubai Co., Ltd.

We also worked toward the development of new materials for an OLED with longer lifetime and higher efficiency. Increasing the performance of OLEDs requires the development of a peripheral material that transports positive holes and electrons as well as the development of a luminescent layer material. Until now, there have been few reports on hole-transporting materials designed for high-performance green OLEDs. We therefore analyzed the device characteristics of several hole-transporting layer materials through molecular orbital calculations and identified the molecular structure of a hole-transporting layer material suitable for long-life and highly efficient OLEDs(2). This finding is a major step toward the development of a practical OLED with longer lifetime and higher efficiency.

■ Solution-processed devices for large rollable displays

With the goal of realizing a very flexible, large rollable display, we are conducting R&D on solution-processed oxide TFTs that can be fabricated easily without using vacuum apparatus. Solution-processed oxide materials often contain solvent-related defects and have low mobility. In FY 2016, we developed a technique that can improve mobility by adding fluorine to solution-processed oxide materials(3). We found that solution-processed IGZO (In-Ga-Zn-O) with fluorine added improved the mobility of TFTs from 1.8 cm2/Vs to 4.7 cm2/Vs. By additionally applying a film quality improvement technique using hydrogen injection and oxidation that we are developing,

we achieved a maximum mobility of 7.0 cm2/Vs.We are also researching electroluminescent devices using

quantum dots (QDs), called quantum dot light-emitting diodes (QD-LEDs), as a luminescent material that is solution-processable and capable of light emission with high color purity. A QD, which is a semiconductor nanocrystal with a size of about 10 nm, can control the wavelength and full width at half maximum of the emission spectrum by using the capability of size control. However, most of the high-color-purity quantum dot materials that have been reported use toxic cadmium and there is a need for the development of a cadmium-free material. In FY 2016, we fabricated a QD-LED using specific cadmium-free materials in cooperation with external organizations. What is also important for increasing the efficiency of QD-LEDs is the development of a carrier transport material that can be used with QDs. In contrast to the island growth of a conventional carrier transport material in deposition on a QD film, the material we developed forms a uniform amorphous film, demonstrating that it is useful for increasing the efficiency of QD-LEDs(4).

■ Technologies for increasing image quality and lowering power consumption

We are conducting R&D on high-mobility TFTs to increase the image quality and lower the power consumption of sheet-type displays. In FY 2016, we developed a high-mobility TFT that uses zinc oxynitride (ZnON) as the semiconductor material. Although ZnON exhibits high mobility, it causes significant time degradation of device characteristics. As a way of suppressing the deterioration of device characteristics, we developed a technique for adding a small quantity of silicon to ZnON(5). Our prototype TFT achieved a mobility of 54 cm2/Vs, about five times as high as that of the conventional IGZO (In-Ga-Zn-O) TFTs.

We continued with our research on adaptive temporal aperture control for suppressing motion blur on hold-type displays such as OLED displays and extending the lifetime of OLEDs. In FY 2016, we derived a driving method to control the temporal aperture in line units, insert a transition area between a dynamic area and static area, and change the temporal aperture gradually when shifting between the dynamic and static areas to suppress image quality degradation by blinking artifacts and screen flickering. We prototyped driving equipment that controls the aperture time line by line and demonstrated the effectiveness of this method with actual equipment, which was evaluated through simulations(6).

In our research on the driving technology of multiple-division-scanning-drive displays with a higher frame rate and larger screen, we proposed a driving method of suppressing image distortion, which is an issue with multiple-division-scanning driving, by changing the time from data writing to light emitting for each horizontal line to minimize the difference in the light-emitting timing(7). We verified its effectiveness through simulations.



[References](1) H. Fukagawa, K. Morii, M. Hasegawa, T. Oono, T. Sasaki, T. Shimizu,

T. Yamamoto: “Demonstration of Highly Efficient and Air-Stable OLED Utilizing Novel Heavy-Doping Technique,” SID Digest, pp.790-793 (2016)

(2) H. Fukagawa, T. Shimizu, H. Kawano, S. Yui, T. Shinnai, A. Iwai, K. Tsuchiya, T. Yamamoto: “Novel Hole-Transporting Materials with High Triplet Energy for Highly Efficient and Stable Organic Light-Emitting Diodes,” Journal of Physical Chemistry C, Vol.120, pp.18748-18755 (2016)

(3) M. Miyakawa, M. Nakata, H. Tsuji, Y. Fujisaki, T. Yamamoto: “Improvement of TFT characteristics by fluorine additive on solution-processed oxide semiconductor,” Abstract of 64th Japan Society of Applied Physics Spring Meeting, 16a-502-7 (2017) (in Japanese)

(4) T. Tsuzuki, G. Motomura, T. Yamamoto: “Quantum dot light-emitting diode using 2,2’-bis(N-carbazolyl)-9,9’-spirobifluorene as a morphologically and thermally stable hole-transporting material,” Physica Staus Solidi A,Vol.213, No.12, pp.3194-3198 (2016)

(5) H. Tsuji, T. Takei, M. Nakata, M. Miyakawa, Y. Fujisaki, T. Yamamoto: “Suppression of degradation of electrical characteristics in ZnON-TFTs by silicon doping,” Abstract of 64th Japan Society of Applied Physics Spring Meeting, 16a-502-5 (2017) (in Japanese)

(6) T. Usui, Y. Takano, T. Yamamoto: “Development of OLED Display using Adaptive Temporal Aperture Control Driving Method with Transition Area Insertion,” IDW’16, DES2-2, pp.1289-1292 (2016)

(7) T. Usui, T. Okada, Y. Takano, T. Yamamoto: “A study of a driving method for suppressing image distortion of tiled OLED displays,” ITE Technical Report, Vol.41, No.2, pp.93-96 (2017) (in Japanese)


7.1 Joint activities with other organizations

7.1.1 Participation in standardization organizationsNHK STRL is participating in standardization activities within

and outside Japan, mainly related to broadcasting. In particular, we are contributing to the creation of technical standards that incorporate our research results.

The ITU Radiocommunication Sector (ITU-R) Study Group 4 (SG4) handles international broadcasting standardization for satellite broadcasting. As part of this group, we proposed that the transmission system for UHDTV satellite broadcasting be named as “ISDB-S3” and our proposal was accepted as ITU-R Recommendation BO.2098-0. We contributed to active discussion at Study Group 5 (SG5) for terrestrial broadcasting by submitting input on incorporating the system parameters of a 42-GHz-band field pick-up unit (FPU) for the wireless transmission of program contributions into ITU-R Recommendation F.1777 “System characteristics of television outside broadcast, electronic news gathering and electronic field production in the fixed service for use in sharing studies.” As part of Study Group 6 (SG6) for broadcasting services, we

contributed to the establishment of Recommendations for the standard viewing conditions based on the requirements of high-dynamic-range television (HDR-TV) and for the optical-optical transfer function (OOTF). A new Recommendation that specifies both the Hybrid Log-Gamma (HLG) method based on a joint proposal by the British Broadcasting Corporation (BBC) and NHK and the perceptual quantization (PQ) method based on a proposal by Dolby Laboratories, Inc., was established as ITU-R Recommendation BT.2100. We also submitted supporting documents to ITU Telecommunications Sector (ITU-T) Study Group 9 (SG9) for cable TV for the specification of three Recommendations (J.94, J.183, J.288) for the channel bonding technology used in the cable transmission scheme for 4K and 8K.

At the Moving Picture Experts Group (MPEG), we participated in the next-generation coding system standardization group with the aim of achieving 30% improvement in the coding efficiency of High Efficiency Video Coding (HEVC), which is the

7 Research-related work

NHK STRL promotes the use of its research results on 8K Super Hi-Vision and other technologies in several ways, including through the NHK STRL Open House, various exhibitions and reports. It also works to develop technologies by forging links with other organizations and collaborating in the production of programs.

We contributed to domestic and international standardization activities at the International Telecommunication Union (ITU), Asia-Pacific Broadcasting Union (ABU), Information and Communications Council of the Ministry of Internal Affairs and Communications, Association of Radio Industries and Businesses (ARIB) and various organizations around the world. We also promoted Japan’s terrestrial digital broadcasting standard, ISDB-T (Integrated Services Digital Broadcasting - Terrestrial), by participating in activities at the Digital Broadcasting Experts Group (DiBEG) and the International Technical Assistance Task Force of ARIB.

The aim of the FY 2016 NHK STRL Open House was to present our future vision of broadcasting and service in an understandable way. It featured 27 exhibits on our latest research results such as Super Hi-Vision, just before the start of its test broadcasting, new broadcasting technologies utilizing the Internet, 3D television, “Smart Production” content production technologies that connect people with society and next-generation devices. The event also had 13 poster exhibits and four interactive exhibits and was attended by 20,371 visitors. We also held 44 exhibitions in Japan and overseas.

We conducted 81 tours of our laboratories for 1,649 visitors. Thirty-one of these tours were for visitors from overseas.

We published 592 articles describing NHK STRL research results in conference proceedings and journals within and outside Japan and issued 10 press releases. We continued to consolidate our intellectual property rights by submitting 351 patent applications and obtaining 263 patents. As of the end of FY 2016, NHK held 1,907 patents.

We are also cooperating with outside organizations. Last year, we participated in 21 collaborative research efforts and three commissioned research efforts. We hosted two visiting researchers and 18 trainees. We also dispatched five of our researchers overseas.

The equipment resulting from our research was used in the production of NHK television programs. It included a millimeter-wave-band mobile camera system that transmits Hi-Vision video with high quality and low latency used in sports programs and an ultra-high-speed camera used in natural science programs. With the start of the test broadcasting of Super Hi-Vision, our 3D reverberator and loudness meter were also used for the production of 22.2 ch sound. In FY 2016, NHK STRL collaborated with the parent organization in making 28 programs. Finally, in recognition of our research achievements, NHK STRL received a total of 47 awards in FY 2016, including an IEEE Milestone and a Primetime Emmy Award.



coding scheme of current 4K/8K Super Hi-Vision satellite broadcasting, and set the target of the final standard issuance for October 2020. We also provided a draft with MPEG Media Transport (MMT) packets and analysis tools for the MMT implementation guidelines (ISO/IEC TR 23008-13), which are based on ARIB Standard STD-B60 for media transport schemes. We conducted a final performance verification test of MPEG-H 3D Audio, a coding scheme for three-dimensional sound including 22.2 ch sound, to help with the publication of MPEG-H 3DA ver. 2, which specifies a low-complexity (LC) profile while maintaining a balance between the decoder circuit size and sound quality. Regarding MPEG-4 Advanced Audio Coding (AAC), we standardized supplements that specify a 22.2 ch downmixing method and a new profile level for 22.2 ch for the revision of ARIB Standard STD-B32. We also contributed to the standardization of a new 3D video coding technology at the MPEG-Free-Viewpoint Television (FTV) ad hoc group by providing elemental images of integral 3D images and test sequences of multi-viewpoint images captured with two-dimensional camera arrays.

At the Society of Motion Picture and Television Engineers (SMPTE), we engaged in the standardization of flag signals for detecting high-dynamic-range signals and wide-color-gamut signals in transmission streams of video signals and also in the

revision of the ST2036-4 standard for the ultra-serial digital interface (U-SDI), which is a next-generation digital interface, to incorporate HDR detection signals.

The technical committee and general meetings of the Asia-Pacific Broadcasting Union (ABU) were held in Bali, Indonesia. We presented our latest technologies for 8K Super Hi-Vision and user-friendly broadcasting. We also participated as a project manager in discussions on topics such as metadata technology for program production, next-generation terrestrial broadcasting, hybrid broadcasting and OTT technologies.

In addition to the above activities, we engaged in standardization activities at international and domestic standardization organizations, including the European Broadcasting Union (EBU), the International Organization for Standardization (ISO), the International Electrotechnical Commission (IEC), the Advanced Television Systems Committee (ATSC: an organization standardizing TV broadcasting systems in the U.S.), the Audio Engineering Society (AES), the World Wide Web Consortium (W3C), which oversees the HTML5 standard for describing content delivered through broadcasting and telecommunications, the Japan Electronics and Information Technology Industries Association (JEITA) and the Telecommunication Technology Committee (TTC) of Japan.

■ Leadership activities at major standardization organizations

■ International Telecommunication Union (ITU)

Committee name Leadership role

International Telecommunication Union Radiocommunication Sector (ITU-R)

Study Group 6 (SG 6, Broadcasting services) Chair

■ Asia-Pacific Broadcasting Union (ABU)


Technical committee Chair

■Information and Communications Council of the Ministry of Internal Affairs and Communications


Information and communications technology subcommittee

ITU section

Spectrum management and planning committee Expert member

Radio-wave propagation committee Expert member

Satellite and scientific services committee Expert member

Broadcast services committee Expert member

Terrestrial wireless communications committee Expert member

■ Association of Radio Industries and Businesses (ARIB)


Technical committee

Broadcasting international standardization working group Chair

Quality evaluation methods investigation research section Committee chair

Monitoring evaluation methods working group Manager

Sound quality evaluation methods working group Manager

Digital broadcast systems development section Committee chair

Multiplexing working group Manager

Download methods TG Leader

Video coding methods working group Manager

Data coding methods working group Manager

Advanced data imaging (H.264) TG Leader

Data broadcasting methods working group

Application control ad hoc group Leader

Data content exchange methods JTG Leader

Copyright protection working group Manager

Digital receivers working group

Ultra-high-definition TV broadcast receivers TG Leader

Digital satellite broadcasting working group Manager

Mobile multimedia broadcasting methods working group Manager

Digital terrestrial broadcasting transmission path coding working group

Manager

Studio facilities development section

Studio sound working group Manager

Contribution transmission development section

Terrestrial wireless contribution transmission working group

Manager

Millimeter-wave contribution transmission TG Leader

New frequency FPU study TG Leader

Promotion strategy committee

Digital broadcasting promotion sub-committee

Digital broadcasting experts group (DiBEG)

International technical assistance task force Manager

Standard assembly

Low-power radio station working group

Radio microphone WG

New digital transmission format study TG Leader

■ Telecommunication Technology Committee (TTC)


Multimedia application working group

IPTV-SWG Leader



7.1.2 Collaboration with overseas research facilitiesWe are participating in technical study groups under the

technical committee of the European Broadcasting Union (EBU). In the Beyond HD group, which studies the promotion and implementation of UHDTV, we took part in discussions on operational guidelines for HDR program production, wide-color-gamut displays for UHDTV and camera requirements, and provided updates on 4K and 8K satellite broadcasting in Japan. In the VR group, we provided input for a survey on VR initiatives at the Broadcast Technologies Future (BTF), an alliance of R&D laboratories sponsored by the EBU. On the basis of the survey results, we compiled a report on the VR services that should be offered by public broadcasters and the initiatives currently being worked on.

We dispatched one of our researchers to BBC R&D for research on an IP-based program production system using the prototype system developed by the BBC. We also engaged in activities at the Advanced Media Workflow Association Networked Media Incubator (AMWA NMI), which standardizes equipment control systems under the leadership of BBC R&D. We participated in discussions on control functions required by broadcasters and conducted a demonstration of our transmission format converter.

A joint task force involving the AMWA and EBU is standardizing the framework for interoperable media services (FIMS) as a mechanism for a future, flexible content production infrastructure. As the upcoming version of the FIMS standard includes a metadata production assistance function using media analysis technology, we began participating in the activities in December.

MPEG-H 3D Audio, an audio coding standard for 3D sound, was established in FY 2015. In response to this, we developed a codec for 22.2 ch sound that can be used for next-generation terrestrial broadcasting in cooperation with Fraunhofer Institute for Integrated Circuits in Germany. We derived bit rates that satisfy broadcast quality and reported them at a conference.

Brazil adopted Japan’s ISDB-T standard as a basis for their digital terrestrial broadcasting in June 2006. Since then, the public and private sectors in Japan have worked together to promote ISDB-T worldwide. This effort has so far resulted in 18 countries adopting ISDB-T. This year, we promoted the standard by participating in the Digital Broadcasting Experts Group (DiBEG) and International Technical Assistance Task Force of ARIB.

7.1.3 Collaborative research and cooperating institutesIn FY2016, we conducted a total of 21 collaborative research

projects and 34 cooperative research projects on topics ranging from system development to materials and basic research.

We collaborated with graduate schools in eight universities (Chiba University, the University of Electro-Communications,

Tokyo Institute of Technology, Tokyo Denki University, Tokyo University of Science, Toho University, Tohoku University and Waseda University) on education and research through activities such as sending part-time lecturers and accepting trainees.

7.1.4 Visiting researchers and trainees and dispatch of STRL staff overseasWe hosted one visiting researcher from the BBC (UK) to

honor our commitment to information exchange with other countries and the mutual development of broadcasting technologies. As part of a program for hosting young researchers from ABU (Asia-Pacific Broadcasting Union) member institutes, we hosted one researcher from Vietnam (Table 1).

We provided guidance to a total of 18 trainees from eight

universities (Waseda University, Tokyo University of Science, the University of Electro-Communications, Tokai University, Nagaoka University of Technology, Tokyo City University, Tokyo Denki University and the University of Tokyo) in their work towards their Bachelor’s and Master’s degrees.

Five STRL researchers were dispatched to research institutions in the United States, the United Kingdom and Spain (Table 2).　

Table 2. Dispatch of NHK STRL researchers overseas

Location Term Research topic

Carnegie Mellon University, USA 2016/1/9 to 2017/1/1 Information security

BBC, UK 2016/1/19 to 2016/7/16 Survey and research on program production systems using telecommunications network technology

Polytechnic University of Valencia, Spain

2016/10/1 to 2017/3/31 Next-generation terrestrial broadcasting technology

Pompeu Fabra University, Spain From 2016/11/7 Next-generation video coding technology

University of California, USA From 2017/3/1 New video experience applying AR and other technologies

Table 1. Visiting researchers

Type Term Research topic

Visiting researcher From 2017/2/1 High-efficiency coding technology for 8K video

ABU visiting researcher 2016/1/28 to 2016/5/31 Super Hi-Vision imaging system



7.1.5 Commissioned researchWe are participating in research and development projects

with national and other public facilities in order to make our research on broadcast technology more efficient and effective. In FY 2016, we took on three projects from the Ministry of Internal Affairs and Communications.

• R&D of technology encouraging effective utilization of frequency for ultra-high-definition satellite and terrestrial

broadcasting systems• R&D on highly efficient frequency usage for the next-

generation program contribution transmissionDevelopment of variable transmission capacity

technology• R&D on advanced technologies for terrestrial television

broadcasting

7.1.6 Committee members, research advisers, guest researchersWe held two meetings of the broadcast technology research

committee and received input from academic and professional committee members. We held 15 sessions to obtain input from

research advisers. We also invited researchers from other organizations to work on seven research topics with us.

■ Broadcast Technology Research Committee Members March 2017

** Committee chair, * Committee vice-chair

Name Affiliation

Kiyoharu Aizawa** Professor, University of Tokyo

Tadashi Ito Senior Vice President, NTT Information Network Laboratory Group

Toshihiko Kanayama Vice President, National Institute of Advanced Industrial Science and Technology (AIST)

Toshiaki Kawai Executive Director and Chief Engineer, Tokyo Broadcasting System Television Inc.

Yasuhiro Koike Professor, Keio University

Tetsunori Kobayashi Professor, Waseda University

Yoichi Suzuki Professor, Tohoku University

Junichi Takada Professor, Tokyo Institute of Technology

Atsushi Takahara Director, Institute for Materials Chemistry and Engineering, Kyushu University

Fumihiko Tomita* Vice President, National Institute of Information and Communications Technology (NICT)

Yasuyuki Nakajima President/CEO, KDDI Research, Inc.

Yasumasa Nakata Executive Director, Fuji Television Network, Inc.

Tatsuhiro Hisatsune Section Manager, Broadcast Technology Division, Information and Communications Bureau, Ministry of Internal Affairs and Communications

Ichiro Matsuda Professor, Tokyo University of Science

Masayuki Murata Professor, Osaka University

■ Research Advisers March 2017

Name Affiliation

Makoto Ando Executive Vice President, Tokyo Institute of Technology

Susumu Itoh Professor, Tokyo University of Science

Makoto Itami Professor, Tokyo University of Science

Tohru Ifukube Emeritus Professor, University of Tokyo (Project Researcher, Institute of Gerontology)

Hideki Imai Emeritus Professor, University of Tokyo

Tatsuo Uchida Specially Appointed Professor, Sendai National College of Technology

Juro Ohga Emeritus Professor, Shibaura Institute of Technology

Tomoaki Ohtsuki Professor, Keio University

Jiro Katto Professor, Waseda University

Yoshimasa Kawata Professor, Shizuoka University

Satoshi Shioiri Professor, Tohoku University

Takao Someya Professor, University of Tokyo

Fumio Takahata Professor, Waseda University

Katsumi Tokumaru Emeritus Professor, University of Tsukuba

Mitsutoshi Hatori Emeritus Professor, University of Tokyo

Takayuki Hamamoto Professor, Tokyo University of Science

Hiroshi Harashima Emeritus Professor, University of Tokyo

Takehiko Bando Emeritus Professor, Niigata University

Masato Miyoshi Professor, Kanazawa University

■ Guest Researchers March 2017

Name Affiliation

Kazuhiro Iida Professor, Chiba Institute of Technology

Mamoru Iwabuchi Associate Professor, University of Tokyo

Tokio Nakada Contract Professor, Tokyo University of Science

Takefumi Hiraguri Professor, Nippon Institute of Technology

Kazuhiko Fukawa Professor, Tokyo Institute of Technology

Toshiaki Fujii Professor, Nagoya University

Tetsuya Watanabe Associate Professor, Niigata University



7.2 Publication of research results

7.2.1 STRL Open HouseThe aim of the NHK STRL Open House 2016 was to present

our future vision of broadcasting to visitors in an understandable way. It featured 27 exhibits on our latest research results such as Super Hi-Vision, whose service launch was on the horizon, internet technology for future broadcast services, 3D television, “Smart Production” technologies that connect people with society. The event also had 13 poster exhibits and four interactive exhibits and was attended by 20,371 visitors. At the entrance hall, we exhibited images of future goals that our technology research aims to achieve in four different sections. One of the sections showcased realistic 8K video screened on a 2-mm-thick large sheet-type 8K OLED display with a size of 130 inches. Other sections featured internet utilization technology for providing appropriate content and information according to various scenes of daily life and a 3D television with improved quality. The lecture focused on research areas that follow our 8K technology. A special presentation and research presentations introduced to visitors our R&D on the use of broadcasting and internet video services, next-generation terrestrial broadcasting, internet utilization technology and integral 3D television.

Schedule• May 24 (Tuesday) Opening ceremony• May 25 (Wednesday) Open to invitees• May 26 - May 29 (Thursday to Sunday) Open to the public

Entrance

Sheet-type 8K display

■ Lecture

Title Speaker

Expectations of Future Broadcasting Boosted by Video, Media, and Technology Kiyoharu Aizawa, Professor, Ph.D., University of Tokyo

■ Special presentation

Title Speaker

How People Choose Between Television and Internet VideoThe use of video services ––now and the future

Maki Shigemori, Head of Public Opinion Research Division, NHK Broadcasting Culture Research Institute

■ Research presentations

Title Speaker

R&D on the Transmission System for Next-Generation Terrestrial Broadcasting Madoka Nakamura, Advanced Transmission Systems Research Division

Working Towards a New TV Experience Using the Internet Chigusa Yamamura, Integrated Broadcast-Broadband Systems Research Division

R&D on Integral 3D Television Masato Miura, Three-Dimensional Image Research Division



7.2.2 Overseas exhibitionsThe world’s largest broadcast technology exhibition, the

National Association of Broadcasters (NAB) Show 2016, was held in April. We exhibited the latest 8K technologies, including an 8K theater with a 350-inch screen using an 8K laser projector, a full-featured 8K camera, a compact 8K full-resolution single-chip camera, 8K high dynamic range (HDR) displays and an 8K outside broadcast (OB) van, highlighting our great strides toward 8K test satellite broadcasting and full-featured 8K Super Hi-Vision. The show attracted about 103,000 visitors from around the world.

During the Rio Olympic Games in August, we conducted a demonstration of 8K terrestrial transmission and public

viewings in cooperation with the Brazilian TV broadcaster TV Globo. A total of about 30,000 viewers enjoyed Olympic game events with the excellent sense of presence offered by 8K video and 22.2 multichannel sound.

The International Broadcasting Convention 2016 (IBC 2016), the largest broadcast technology exhibition in Europe, was held in September. We exhibited an 8K sheet-type display aligning four 4K OLED panels and an 8K HDR display and screened the highlight of the Rio Olympic Games. We also exhibited sign language CG, an automatic audio description system and an integral 3D television using an 8K OLED display. The convention drew about 56,000 visitors.

■ Research exhibits

A New Development of Super Hi-Vision B3 Video Distribution Technologies Adapted to Diverse Viewing Styles

B Internet Technology for Future Broadcast Services C1 Scene Text Detection to Automatically Generate Metadata for Videos

C Smart Production C2 Studio Robot for Joint Performance with CG

D 3D Television C3 Automatic Sign Language Animation System to Express Weather Warnings

A1 8K-HDR Live Program Production C4 News Service with Reading Assistance

A2 Developments Toward Full-featured Super Hi-Vision C5 Haptic Presentation Technology for Conveying 3D Shape of Object

A3 Full-resolution Single-chip 8K Camera System C6 Three-dimensional Information Analysis for Live Sports Graphics

A4 Holographic Memory for Archival Use D1 Integral 3D Television

A5 Three-dimensional Sound Production Equipment D2 Device Technologies for Future 3D Display

A6 Retransmission Technology of Super Hi-Vision Satellite Broadcasting for Cable TV Networks E1 Future Image Sensor Technologies

A7 8K/4K Video Coding System with Super-resolution Reconstruction E2 Elemental Technologies for Sheet-type Displays

A8 Next-generation Terrestrial Broadcasting Systems E3 Magnetic Nanowire Memory

A9 Delivery Technology Using MMT for 8K Super Hi-Vision F1 Utilization and Development of NHK’s Technologies

A10 8K Super Hi-Vision Wireless Links for Program Contribution F2 Live Streaming Service for Smartphones

B1 Technologies to Realize a “New Television Experience” Spread by the Internet F3 The Time Has Come to Launch Super Hi-Vision Broadcasting

B2 New User Experience by Hybridcast for a Live Sport Program F4 IEEE Milestones Awarded for NHK’s Technical Achievements

■ Poster exhibits

A-P1 Ultra-reality Meter D-P2 Optical Beam Steering Device

A-P2 Future Video Coding Technologies E-P1 Solid-state Image Sensor Overlaid with Photoelectric Conversion Layer

A-P3 Super-resolution Technique for Full-featured 8K Video E-P2 Organic Image Sensors

B-P1 Encryption Scheme for Privacy Protection E-P3 Solution-processed Oxide TFTs (Thin-film Transistors)

B-P2 Content Search Behaviors on Time Shift Zapping System E-P4 Inverted OLED (Organic Light-emitting Diode)

C-P1 Advanced CG Sign Language Technology E-P5 Driving Technology for Enhanced Video Image Quality and Longer Lifetime

D-P1 Depth-compressed Expression Technology

■ Interactive exhibits

1. Experience Full-featured 8K Super Hi-Vision 3. Augmented TV

2. Immerse Yourself in a 3D Sound System 4. Let’s Move and Watch! Integral 3D Quiz

■ Three overseas exhibitions

Event name Dates Exhibits

NAB Show 2016 (Las Vegas, USA) 4/18 to 4/21 8K laser projector, Full-featured 8K camera, 8K full-resolution compact single-chip camera, 8K HDR LCD, 8K outside broadcast (OB) van

Rio Olympic Games (Rio de Janeiro, Brazil) 8/5 to 8/21 Terrestrial transmission demonstration, 8K public viewing (including live screening)

IBC2016 (Amsterdam, Netherlands) 9/9 to 9/13 8K sheet-type display, 8K HDR LCD, Sign language CG, Automatic audio description, Integral 3D television



7.2.3 Exhibitions in JapanThroughout the year, NHK broadcasting stations all over

Japan hosted events and exhibitions of the latest broadcast technologies resulting from our R&D. We conducted 8K Super Hi-Vision (SHV) public viewings at our auditorium during the

Rio Olympic Games and screened sports scenes with the sense of presence to many viewers. We also demonstrated the reception of SHV at some conferences to publicize the start of SHV test satellite broadcasting.

7.2.4 Academic conferences, etc.We presented our research results at many conferences in

Japan, such as the ITE and IEICE conferences, and had papers published in international publications such as IEEE Transactions, Journal of Physical Chemistry C and Journal of the Society for Information Display.

7.2.5 Press releasesWe issued 10 press releases on our research results and

7.2.6 Visits, tours, and event news coverageTo publicize R&D on 8K Super Hi-Vision and integral 3D

television, we held tours for people working in a variety of fields including broadcasting, university and academic research and the arts. We welcomed visitors from around the world, including officials of standardization and international broadcasting conference organizations, such as the European Broadcasting Union (EBU) and International Broadcasting Convention (IBC), and broadcasters from various countries.

Inspections, tours 81 (31 from overseas)1,649 visitors (214 from overseas)

News media 14 events

Academic journals in Japan 56 papers

Overseas journals 26 papers

Academic and research conferences in Japan 231 papers

Overseas/International conferences, etc. 153 papers

Contributions to general periodicals 54 articles

Lectures at other organizations 72 events

Total 592

■ 24 exhibitions in Japan

Event name (Only major events) Dates Exhibits

Thanks in Shibuya (Shibuya de domo) 5/3 to 5/5 8K Super Hi-Vision (SHV)

Rio 2016 Olympic Games 8K public viewing 8/6 to 8/22 8K SHV

NHK Disaster Prevention Park 2016 8/27 to 8/28 Sign language CG technology, Reading assistance technology

CEATEC JAPAN 2016 10/4 to 10/7 Sheet-type display, SHV cable transmission technology

Digital Content Expo 2016 10/27 to 10/30 8K Time into Space, Real-time object tracing

NHK Osaka Station Open House “BK Wonderland” 10/29 to 10/30 Haptic TV

InterBEE 2016 11/16 to 11/18 Augmented TV, Real-time object tracing, MMT multichannel transmission

NHK Science Stadium 2016 12/3 to 12/4 Reading assistance technology, Integral 3D quiz

IDW/AD'16 12/7 to 12/9 Reception of SHV test satellite broadcasting

NHK Kouhaku year-end music show 8K public viewing 12/31 8K SHV

We issued 10 press releases on our research results and other topics.

Dates Press release content

2016/4/7 Announcement of the STRL Open House 2016

4/7 The pioneering high-definition television system and emergency warning code signal broadcasting system were dedicated IEEE Milestones

5/24 Development of an OLED device for flexible displays with strong resistance to oxygen, long service life and high energy-saving capability

5/24 Development of a holographic memory drive for 8K Super Hi-Vision archive

5/24 Development of an 8K Super Hi-Vision wireless link

5/24 Start of joint evaluation of transmission schemes for cable TV retransmission of 4K/8K Super Hi-Vision

5/24 Successful multichannel transmission of 8K Super Hi-Vision over 10-Gbps-class optical internet lines

9/29 Development of a compact cable TV receiver system for 4K/8K Super Hi-Vision

10/6 NHK STRL wins Engineering Emmy Award

2/20 Evaluation website for weather forecast sign-language CGs opened



7.2.7 BulletinsWe published bulletins describing our research activities and

achievements and special issues on topics such as wired transmission technologies, video evaluation based on human science, and large-capacity and high-speed storage technologies. The Broadcast Technology journal, which is directed at overseas readers and features in-depth articles about our latest research and trends, included articles such as “Standardization Trends in Video Coding Technologies,” “MPEG-DASH and Hybridcast” and “Image Evaluation Using Biological Information.”

■ Domestic Publications

STRL Dayori (Japanese, monthly) No. 133 to No. 144 NHK STRL R&D (Japanese, bimonthly) No. 157 to No. 162 Annual Report (Japanese, annually) FY2015 Edition

■ Publications for overseas readers

Broadcast Technology (English, quarterly) No. 64 to No. 67 Annual Report (English, annually) FY2015 Edition

7.2.8 WebsiteThe NHK STRL website describes our laboratories and their

research and posts reports and announcements on events such as the Open House, and the organization’s journals. We enhanced the capabilities of the website for smartphones so that it can be viewed by more people. To promote people’s understanding of NHK STRL, we made our latest brochures available on the “About NHK STRL” page.

7.3 Applications of research results

7.3.1 Cooperation with program producersEquipment resulting from our R&D has been used in many

programs. Our millimeter-wave mobile camera system, which uses millimeter-wave-band radio waves to transmit HD video with high quality and low latency, performed well in live sports coverage of golf tournaments and triathlon competitions. For natural science programs, our ultra-high-speed camera

captured the mechanism of electric discharge machining and the vibration of the strings of a string instrument, which cannot be observed with human eyes. With the start of test broadcasting of Super Hi-Vision, our 3D reverberator and loudness meter were also used for the production of 22.2 multichannel sound. We collaborated in the production of 28 programs in FY 2016.

STRL Dayori Broadcast TechnologyNHK STRL R&D

Laboratories’ outline page



7.3.2 PatentsNHK participates in a patent pool, which bundles licenses of

patents required by digital broadcasting and high-efficiency video coding standards under reasonable conditions. The pool especially promotes the use of patents held by NHK to help with the promotion of broadcasting. We are protecting the rights to our broadcasting and communications-related R&D as part of our intellectual property management efforts. We are

also actively promoting contracts on transfers of patented NHK technologies by enhancing our Technology Catalogue, which summarizes NHK’s transferrable technologies, and at events such as the STRL Open House 2016, CEATEC JAPAN 2016, Technical Show Yokohama 2017, the 46th NHK Program Technology Exhibition and other events we held in cooperation with local governments and other organizations.

■ Patents■ Patents and utility model applications submitted

Type New Total at end of FY

Domestic Patents 322 1,259

Utility models 0 0

Designs 0 0

Overseas 29 111

Total 351 1,370

■ Patents and utility models granted


Domestic Patents 254 1,757

Utility models 0 0

Designs 0 1

Overseas 9 149

Total 263 1,907

■ Patents and utility models in use (NHK Total)


Contracts 16 287

Licenses 38 486

Patents 24 255

Expertise 14 231

■ Technical cooperation (NHK Total)

Type Total

Technical cooperation projects 20

Commissioned research projects 3



7.3.3 Prizes and degreesIn FY 2016, NHK STRL researchers received 47 prizes, including IEEE Milestones, a Primetime Emmy Award, a Meritorious Award on

Radio and the Takayanagi Memorial Award. Four researchers obtained a doctoral degree in FY 2016, and at the end of FY 2016, 87 STRL members held doctoral degrees.

Award Winner Award Name Awarded by In recognition of Date

NHK IEEE Milestone IEEE High-Definition Television System Many years of R&D, which led to the realization and inauguration of the high-definition broadcasting system and brought viewers high-definition, wide-screen images with a strong sense of reality

2016/4/7

NHK IEEE Milestone IEEE Emergency Signal Code Broadcasting System Many years of R&D, which led to the realization and inauguration of the world’s first emergency code signal broadcasting system in 1985

2016/4/7

Kazuya Kitamura, Toshio Yasue Ichimura Prize, Contribution Prize New Technology Development Foundation

Development of a full-featured 8K Super Hi-Vision image sensor 2016/4/25

WISDOM X and DISAANA development group (Jun Goto)

Award of Excellence National Institute of Information and Communications Technology (NICT)

Development and release of WISDOM X and DISAANA 2016/5/11

Satoshi Oode ITU-AJ Award, Encouragement Award (ICT Field)

ITU Association of Japan Contribution to the establishment and revision of sound-related recommendations at ITU-R SG6

2016/5/17

International Technical Assistance Task Force (Masahiro Okano)

ITU-AJ Award, Encouragement Award (International Cooperation Field)

ITU Association of Japan Contribution to broadcasting technology development in countries adopting ISDB-T through seminars and standardization support

2016/5/17

Keji Ishii Presidential Citation Society for Information Display (SID) Contribution to International Display Workshops ’15 as executive chair

2016/5/24

Hiroyuki Kawakita, Toshio Nakagawa, Makoto Sato (Tokyo Institute of Technology)

Technology Promotion Award, Content Technology Award

Institute of Image Information and Television Engineers (ITE)

Augmented TV content such as Giant Squid AR at NHK Science Stadium event

2016/5/27

Seiji Mitsuhashi (Engineering Dept.), Kazuya Kitamura, Hiroshi Shimamoto, Tsuyoshi Sakiyama, Kazuhiko Nemoto (Broadcast Engineering Dept.), Yuji Oguri (Broadcast Engineering Dept.), Ryoichi Sato (Hitachi Kokusai), Hideaki Ariga (Ikegami Tsushinki)

Image Information Media Future Award, Next-generation TV Technology Award


Development of a practical 8K single-chip camera system 2016/5/27

Genichi Motomura, Yoshiki Nakajima, Tatsuya Takei, Toshimitsu Tsuzuki, Hirohiko Fukagawa, Mitsuru Nakata, Hiroshi Tsuji, Takahisa Shimizu, Yoshihide Fujisaki, Toshihiro Yamamoto, Katsuyuki Morii (Nippon Shokubai), Munehiro Hasegawa (Nippon Shokubai)

Niwa–Takayanagi Award, Research Paper Award


A Flexible Display Driven by Oxide-Thin-Film Transistors and Using Inverted Organic Light-Emitting Diodes

2016/5/27

Kenji Machida, Hiroshi Kikuchi, Kiyoshi Kuga, Ken-ichi Aoshima, Nobuhiko Funabashi, Daisuke Kato, Hidekazu Kinjo, Shintaro Aso

Image Information Media Future Award, Frontier Award


Research on spatial light modulator driven by spin-transfer switching for electronic holography

2016/5/27

Takehiro Sugimoto, Yasushige Nakayama, Tomoyasu Komori, Kaoru Watanabe, Satoshi Oode, Toru Chinen (Sony), Mitsuyuki Hatanaka (Sony), Minoru Tsuji (Sony), Hiroyuki Honma (Sony)

ASJ Technical Development Award Acoustical Society of Japan 22.2ch Audio Encoder/Decoder Using MPEG-4 AAC Establishment of 22.2ch audio coding standards and development of an MPEG-4 AAC encoder

2016/5/28

Hiroki Endo IEEE BMSB 2016 Best Paper Award IEEE Cross-Media Platform for the Unification of Broadcast and Broadband Program-Viewing Environments

2016/6/3

Yoichi Suzuki, Masaaki Kojima, Masashi Kamei, Masafumi Nagasaka, Susumu Nakazawa, Shoji Tanaka

Satellite Communications Research Award

Institute of Electronics, Information, and Communication Engineers (IEICE)

8K Super Hi-Vision satellite broadcasting experiment at the NHK STRL Open House 2015

2016/6/3

Kazuho Ono AES Fellowship Award AES (Audio Engineering Society) Remarkable contribution to the development of a high-quality professional microphone

2016/6/4

Moe Nagahata (Senshu Univ.), Masamitsu Harasawa, Shun Sato (Senshu Univ.), Hitoshi Nakazawa (Senshu Univ.), Hiroshi Ishikane (Senshu Univ.)

2015 Annual Convention Outstanding Presentation Award

Japanese Psychological Association “Eating disorder tendencies affect size perception of human body picture”

2016/6/16

Masamitsu Harasawa, Kazuteru Komine

2015 Annual Convention Outstanding Presentation Award

Japanese Psychological Association “Visual attention oriented toward very large visual field affected cortical activities: an ERP study”

2016/6/16

Masafumi Nagasaka, Shoji Tanaka, Masayuki Takada (Media Planning Office), Kenshin Saeki (Engineering Dept.)

Meritorious Award on Radio, Ministerial Commendation

Association of Radio Industries and Businesses (ARIB)

Practical application of a dual-polarized receiving antenna for 12-GHz-band broadcasting satellite

2016/6/27

Kensuke Ikeya, Masanori Kano, Tomoyuki Mishina

Hoso Bunka Foundation Awards Hoso Bunka Foundation Development of a new image presentation technology using “Gurutto Vision” with multi-viewpoint robotic camera

2016/7/5

Satoshi Nishimura Technology Development Award Motion Picture and Television Engineering Society of Japan, Inc.

Development of a Hybridcast-enabled MPEG-DASH player 2016/8/1

Takuya Shitomi Suzuki Memorial Award Institute of Image Information and Television Engineers (ITE)

Transmission technology for the next-generation terrestrial broadcasting

2016/9/1

Kazuhiro Hara Suzuki Memorial Award Institute of Image Information and Television Engineers (ITE)

A study on coded image quality of integral three-dimensional image using 3D-HEVC

2016/9/1

Mitsunobu Okuda Suzuki Memorial Award Institute of Image Information and Television Engineers (ITE)

Formation, driving and detection of magnetic domains in magnetic nanowire

2016/9/1

Taro Miyazaki, Tadashi Kumano, Atsushi Imai

Forum of Information Technology Encouragement Award


Improving Transliteration Quality of People’s Name by Considering Nationality Information

2016/9/9

Kazuto Ogawa IWSEC2016 Contribution Award Information Processing Society of Japan

Contribution to IWSEC2016 as program chair 2016/9/13

Hideki Tanaka Award for Person of Cultural Merits of Tokyo Citizen

Tokyo Metropolitan Government Award for Person of Cultural Merits of Tokyo Citizen 2016/10/3

Mitsuru Nakata, Hiroshi Tsuji, Yoshihide Fujisaki, Tatsuya Takei, Genichi Motomura, Hirohiko Fukagawa, Toshimitsu Tsuzuki, Takahisa Shimizu, Toshihiro Yamamoto, Yoshiki Nakajima, Naoki Shimizu (NHK Engineering Systems)

Technical Committee Prize Paper Award (Third Prize)

IEEE Industry Applications Society 2015

Oxide Thin-Film Transistor Technology for Flexible Organic Light-Emitting Diode Displays

2016/10/3



Award Winner Award Name Awarded by In recognition of Date

Yukiko Iwasaki 11th ICEL Best Poster Award ICEL Organizing Committee Observation of clear differences in PHOLED performances using similar TADF materials as phosphorescent host

2016/10/4

Satoshi Nishimura METI Ministerial Award (MPTE AWARDS 2016)

Motion Picture and Television Engineering Society of Japan, Inc.

Development of a Hybridcast-enabled MPEG-DASH player 2016/10/6

Yuki Koizumi, Shoji Tanaka, Kyoichi Saito, Masaaki Kojima, Yoichi Suzuki

JC-SAT2016 Best Paper Award Institute of Electronics, Information, and Communication Engineers (IEICE)

A study on 64APSK coded modulation 2016/10/13

Nobuyuki Hiruma (NHK Engineering Systems), Makiko Azuma, Tsubasa Uchida, Shuichi Umeda, Taro Miyazaki, Naoto Kato

ABU 2016 Best Paper Award ABU Research on sign language CGs 2016/10/21


Primetime Emmy Award, Philo T. Farnsworth Corporate Achievement Award

The Academy of Television Arts & Sciences

Development across several decades of pioneering technologies which have expanded the possibilities of broadcasting technology in Japan and all over the world

2016/10/26

Hisayuki Ohmata, MIT Media Lab Innovative Technologies 2016 Ministry of Economy, Trade and Industry

8K Time into Space 2016/10/27

Kazuho Ono, Takehiro Sugimoto, Akio Ando (Univ. of Toyama), Keishi Imanaga (Sanken Microphone), Yutaka Chiba (Sanken Microphone), Takeshi Ishii (Sanken Microphone)

Kanto Region Invention Award, Invention Encouragement Award

Japan Institute of Invention and Innovation

A Narrow−angle Directional Microphone with Suppressed Rear Sensitivity

2016/11/10

Takenobu Usui Best Paper Award International Display Workshop Research presentation on adaptive temporal aperture control of OLED displays

2016/12/9

Daiichiro Kato (NHK Engineering Systems), Yuko Yamanouchi, Hideki Mitsumine, Kazutoshi Muto

SI2016 Best Lecture Award Society of Instrument and Control Engineers

Lecture on “Development of a hybrid sensor that enables virtual presentation using a handy camera”

2016/12/18

Naoto Okaichi Best Research Presentation Award Institute of Image Information and Television Engineers (ITE)

Research presentation on “Integral 3D display using multiple direct-view display panels”

2016/12/21

Yutaro Katano, Tetsuhiko Muroi, Nobuhiro Kinoshita, Norihiko Ishii

IEEE ICCE 2017 Best Paper, Third Place

IEEE ICCE (IEEE International Conference on Consumer Electronics)

Prototype Holographic Drive with Wavefront Compensation for Playback of 8K Video Data

2017/1/9

Hiroshi Shimamoto Kenjiro Takayanagi Memorial Award Kenjiro Takayanagi Foundation Development of 8K Super Hi-Vision image sensors 2017/1/20

Kenichiro Masaoka 2017 Special Recognition Award Society for Information Display (SID) R&D on wide-color-gamut display and color gamut calculation criteria

2017/1/20

Masafumi Nagasaka Research Encouragement Award ITE Technical Group on Broadcasting and Communication Technologies

Three lectures in FY2016 2017/2/24

Junichiro Kawamoto Research Encouragement Award ITE Technical Group on Broadcasting and Communication Technologies

Research on error correction coding in IP transmission 2017/2/24

Kouji Kozaki (Osaka Univ.), Makoto Urakawa, Tomo Onodera (Infinito Lab), Takeshi Masuda (Osaka Univ.), Fumihiro Kato (National Institute of Informatics), Terufumi Kawasaki (A’ Work Souzoukan)

Urban Data Challenge 2016, Activity Division, Silver Prize

Association for Promotion of Infrastructure Geospatial Information Distribution

AfterFive Development Team 2017/2/25

Hisayuki Sasaki, Naoto Okaichi, Hayato Watanabe, Masahiro Kawakita, Tomoyuki Mishina

IEICE Image Engineering Technical Group IE Award

IEICE Image Engineering Technical Group

Color moire reduction technique for direct-view-type integral three-dimensional display

2017/2/27

Isao Goto, Hideki Tanaka ANLP 23rd Annual Meeting Excellent Paper Award

Association for Natural Language Processing

Detection of omissions in neural machine translation 2017/3/16

Reiko Takou, Nobumasa Seiyama, Hiroyuki Segi (Seikei Univ.), Tohru Takagi (NHK Engineering Systems), Yuko Uematsu (Secom), Hideo Saito (Keio Univ.), Shinji Ozawa (Keio Univ.)

32nd Telecom System Technology Award, Encouragement Award

Telecommunications Advancement Foundation

Research on automatic reading of weather report 2017/3/24

Naoto Kato IEICE Distinguished Achievement and Contributions Award


Many years of contribution to the promotion of I-Scover 2017/3/24


■

Deputy Director of STRL Kohji Mitani

Director of STRL Toru Kuroda Executive Research Engineer Makoto YamamotoExecutive Research Engineer Tomohiro Saito

(at end of FY2016)

Head

NHK STRL Organization

Research planning/management, public relations, international/domestic liaison, external collaborations, etc.

Patent applications and administration, technology transfers, etc.

Integrated broadcast-broadband technology, Hybridcast, IT security, broadband video service technology, video content analysis, etc.

Satellite/terrestrial transmission technology, millimeter-wave and optical 8K contribution technology, multiplexing technology, IP transmission technology, etc.

8K program production equipment, video coding for efficient transmission, highly realistic audio systems, etc.

Speech recognition, advanced language processing such as simple Japanese and sign language CG creation, transmission of tactile/haptic information, etc.

Spatial 3D video system technologies (integral 3D, etc.), 3D display device technology, cognitive science and technology, etc.

Ultrahigh-resolution and ultrasensitive imaging devices, high-capacity fast-write technology, sheet-type display technology, etc.

Personnel, labor coordination, accounting, building management, etc.

Toru Imai

Tomoko Okamoto

Toshio Nakagawa

Shunji Nakahara

Tetsuomi Ikeda

Masakazu Iwaki

Hiroshi Kikuchi

Naoto Hayashi

Taisuke Yamakage

Planning & Coordination Division

Patents Division

Internet Service Systems Research Division

Advanced Transmission Systems Research Division

Advanced Television Systems Research Division

Human Interface Research Division

Three-Dimensional Image Research Division

Advanced Functional Devices Research Division

General Affairs Division

U.S.-made television purchased for the home of the first subscriber

■

Established in June 1930June 1930 - January 1965January 1965 - July 1984

July 1984 - Present

Technical Research LaboratoriesTechnical Research Laboratories,Broadcast Science Research LaboratoriesScience & Technology Research Laboratories

Patents held: DomesticInternational

STRL by numbers

Employees 257

Degree-holding personnel

(at end of FY2016)

(including 229 researchers)

87

1,758149

■ STRL Open House

The STRL Open House is held every year in May to introduce our R&D to the public.

■ Current research building

Completed March 2002High-rise building: 14 �oors above ground, two below ground Mid-rise building: 6 �oors above ground, two below ground Total �oor space: Approx. 46,000 m2

Total research area: Approx. 16,000 m2

Total land area: Approx. 33,000 m2

■ History and near future of broadcasting development and STRL

The NHK Science & Technology Research Laboratories (NHK STRL) is the sole research facility in Japan specializing in broadcasting technology, and as part of the public broadcaster, its role is to lead Japan in developing new broadcasting technology and contributing to a rich broadcasting culture.

1925: Radio broadcasting begins

1930: NHK Technical Research Laboratories established

1953 : Television broadcasting begins

1964 : Hi-Vision research begins

1966 : Satellite broadcasting research begins

1982 : Digital broadcasting research begins

1989 : BS Analog broadcasting begins

1991 : Analog Hi-Vision broadcasting begins

1995 : Super Hi-Vision research begins

2000 : BS Digital broadcasting begins

2003 : Digital terrestrial broadcasting begins

2011 : Analog television broadcasting ends

2006 : One-Seg service begins

2018 : Super Hi-Vision broadcasting

2016 : Super Hi-Vision test broadcasting

NHK Science & Technology Research Laboratories Outline

Seijogakuen-mae Soshigaya-Okura Odakyu Line

To ShinjukuNHK STRL

N

Bus stop

Bus stop

Bus stopToho

Natl. Ctr. for Child Health

and Development

TomeiExpressway

Kinuta Koen（Park）

Ring Road No. 8/Kanpachi Dori

Setagaya Dori

Yoga

Tokyu

Den-en-toshi line

To Shibuya

Shuto

Expressway

Yoga I. C.

■Odakyu line, from Seijogakuen-mae station, south exit:［Odakyu Bus/Tokyu Bus］・Shibu 24（渋24） toward Shibuya Station［Tokyu Bus］・To 12（等12） toward Todoroki-soshajo・Yo 06（用06） toward Yoga Station（weekdays only）・Toritsu 01（都立01） toward Toritsu Daigaku Station, north exit

■Tokyu Den-en-toshi line, fromYoga station:［Tokyu Bus］・To 12（等12） toward Seijo-gakuen-mae station・Yo 06（用06） toward Seijo-gakuen-mae station（weekdays only）

In all cases, get off the bus at the “NHK STRL”（NHK技術研究所）bus stop

Nippon Hoso Kyokai（NHK）Science & Technology Research Laboratories（STRL）1-10-11 Kinuta, Setagaya-ku, Tokyo

Edited and Published byb :

Access to NHK STRL

Directions

http://www.nhk.or.jp/strl/index-e.htmlTel:+81-3-3465-1111

August 2017 NHKScience & TechnologyResearch Laboratories

2016


20

16NHK

Science & TechnologyResearch Laboratories

2016

Annual Report

Nippon Hoso Kyokai[Japan Broadcasting Corporation]

Documents

NHK August 2017 Science & Technology Research Laboratories ... · 1.7 Media transport technologies 11 ... and social media analysis, and built a Smart Production Laboratory in our