Application design supporting structured radiology reports · 2007. 2. 13. · Administrators (AHRA) ... The standardized presentation also provides another advantage, as referring

An application design supporting structured

radiology reports

Linn Wikström

UMEÅ UNIVERSITY DEPARTMENT OF COMPUTING SCIENCE

SE – 901 87 UMEÅ SWEDEN

February 12, 2007 Master’s Thesis in Computing Science, 20 credits

Internal advisor at CS-UmU: Helena Lindgren External advisor at Sectra: Henrik Magnusson

Examiner at CS-UmU: Per Lindström

Abstract Radiologists use medical images to diagnose diseases. Today, medical images are commonly stored and displayed electronically. The radiologist describes the image interpretations in a report, and the electronic viewing of images opens a possibility for the radiology report to become computerized. By creating an application that supports the radiologist’s report generation, tools for creation of structured reports can be provided, generating more complete, clear reports, connected to the examination images. The structured report can also be encoded, to provide data for medical research and for statistics. This masters’ thesis establishes the requirements on a structured reporting application, by means of a literature review and by user interviews and meetings. Using the established requirements, a structured reporting application was designed, and partly implemented as a prototype. The results show that an application for structured reporting has to be made adaptive, to support different usage areas, each having their own requirements on navigation, organization and workflow. It also has to support reuse of report components, and automatic generation of sections. The resulting reports should be encoded in DICOM SR (Digital Imaging and Communications in Medicine – Structured Reporting), using the report pattern established by IHE (Integrating the Healthcare Enterprise). To provide users with a complete set of radiological terms for generation of natural language, a medical dictionary should be used. User input can be made through data entry controls, and it should be possible for the user to, in an easy way, link images and add measurement values to the report content.

Sammanfattning Idag lagras medicinska bilder vanligtvis elektroniskt, och röntgenläkare undersöker bilderna för att upptäcka sjukdomar. De fynd som görs och de slutsatser som dras redovisas i ett utlåtande. Den datoriserade miljön för medicinska bilder gör det möjligt att skapa elektroniska rapporter. Genom att skapa en applikation som stöder röntgenläkaren i skapandet av dessa kan applikationer som genererar strukturerade utlåtanden tillhandahållas. Sådana strukturerade utlåtanden är tydligare, mer kompletta och möjliggör relationer mellan rapporterad text och undersökningsbilder. Strukturerade rapporter kan också kodas för att möjliggöra forskning och statistik. Det här examensarbetet fastställer vilka krav som finns på en applikation som ger stöd åt genereringen av strukturerade utlåtanden. Utifrån dessa krav utformades en sådan applikation, och den implementerades delvis i en prototyp. En applikation för strukturerade utlåtanden måste göras adaptiv, för att stödja olika användningsområden med olika krav på navigering, organisation och arbetsflöde. Den måste också stödja återanvänding av olika delar i utlåtanden samt automatisk generering av stycken. Det strukturerade utlåtande ska sparas i DICOM SR-format (Digital Imaging and Communications in Medicine – Structured Reporting), och följa IHEs (Integrating the Healthcare Enterprise) mall för rapporter. För att ge användare tillgång till en komplett terminologi för skapandet av naturligt språk bör ett lexikon användas. Datainmatning kan ske genom att använda formulär, och det ska vara möjligt att på ett enkelt sätt länka bilder och mätvärden från bilder in i utlåtandet.

Table of contents 1 Introduction ................................................................................................................................................ 2

1.1 Radiology reporting ........................................................................................................................... 2 1.2 Motivation ........................................................................................................................................... 2 1.3 Concepts and abbreviations ............................................................................................................. 4 1.4 Sectra PACS ........................................................................................................................................ 5 1.5 Goal ...................................................................................................................................................... 6 1.6 Resources ............................................................................................................................................. 6 1.7 Limitations........................................................................................................................................... 6 1.8 Thesis outline ...................................................................................................................................... 6

2 Methods ....................................................................................................................................................... 8 2.1 Design process .................................................................................................................................... 8 2.2 Identifying requirements ................................................................................................................... 9 2.3 Application development ................................................................................................................10

2.3.1 Design .......................................................................................................................................10 2.3.2 Implementation .......................................................................................................................11

2.4 Evaluation..........................................................................................................................................11 3 SR Design Guidelines ..............................................................................................................................12

3.1 Functionality .....................................................................................................................................12 3.2 Encoding ...........................................................................................................................................13

3.2.1 DICOM SR ..............................................................................................................................13 3.2.2 HL7 ...........................................................................................................................................14 3.2.3 The IHE SINR Profile ...........................................................................................................14

3.3 Language & Terminology ...............................................................................................................16 3.3.1 SNOMED CT .........................................................................................................................16 3.3.2 UMLS ........................................................................................................................................17 3.3.3 RadLex ......................................................................................................................................18

3.4 Interaction .........................................................................................................................................18 3.4.1 Interaction styles .....................................................................................................................18 3.4.2 Control design .........................................................................................................................19

3.5 Usability .............................................................................................................................................20 3.5.1 Usability goals ..........................................................................................................................20 3.5.2 Design principles .....................................................................................................................21 3.5.3 Design principles for data entry............................................................................................22

3.6 Summarized guidelines ....................................................................................................................23 4 Results ........................................................................................................................................................25

4.1 Established requirements ................................................................................................................25 4.1.1 The general interface ..............................................................................................................25 4.1.2 The general reporting application.........................................................................................26 4.1.3 The CMIV application ...........................................................................................................26

4.2 Application design............................................................................................................................27 4.2.1 Conceptual design ...................................................................................................................27 4.2.2 Physical design .........................................................................................................................32

4.3 Implemented prototype ..................................................................................................................33 4.3.1 System description ..................................................................................................................33 4.3.2 The main interface ..................................................................................................................34 4.3.3 Sample application 1: The general radiology report ..........................................................36

4.3.4 Sample application 2: The CMIV annotation report.........................................................37 4.4 Evaluation..........................................................................................................................................38

4.4.1 The general radiology report .................................................................................................39 4.4.2 The CMIV annotation report ...............................................................................................39

5 Discussion and conclusions ....................................................................................................................40 5.1 Methods .............................................................................................................................................40 5.2 Structured Reporting .......................................................................................................................40 5.3 Prototype ...........................................................................................................................................41 5.4 Future work .......................................................................................................................................42

6 Acknowledgements ..................................................................................................................................43 References ............................................................................................................................................................44 7 Appendix...................................................................................................................................................... I

A. Unstructured interview template ............................................................................................................. I B. Interview summaries ................................................................................................................................. II C. CMIV Meetings ......................................................................................................................................... V D. Evaluation interviews ............................................................................................................................VII

List of Figures Figure 1.1 Sectra PACS ...................................................................................................................................... 5 Figure 1.2 Image window ................................................................................................................................... 5 Figure 2.1 Design process .................................................................................................................................. 8 Figure 2.2 Sketch of main application .............................................................................................................. 9 Figure 2.3 Sketch of tool drawer ....................................................................................................................... 9 Figure 3.1 The SINR Profile (adopted from IHE(15)) ...............................................................................15 Figure 3.2 The SINR pattern ...........................................................................................................................16 Figure 4.1 Starting the SR application ............................................................................................................28 Figure 4.2 Application areas ............................................................................................................................28 Figure 4.3 Split image window ........................................................................................................................29 Figure 4.4 Menu options ..................................................................................................................................29 Figure 4.5 Image selection window ................................................................................................................31 Figure 4.6 Measurement insertion ..................................................................................................................31 Figure 4.7 System overview .............................................................................................................................33 Figure 4.8 Report template ..............................................................................................................................34 Figure 4.9 Section template .............................................................................................................................34 Figure 4.10 Sliding drawer ...............................................................................................................................35 Figure 4.11 Print preview .................................................................................................................................36 Figure 4.12 Implementation of "Open" ........................................................................................................36 Figure 4.13 Tool drawer ...................................................................................................................................37 Figure 4.14 Collection of annotated images ..................................................................................................38

1 Introduction

Linn Wikström Application design supporting structured radiology reports

2

1 Introduction

This chapter provides initial information about radiology, and how it is used in medical work today. The purpose is also to explain the use of computerized systems for medical images, and why the computerization should be extended to include radiology reports. Some abbreviations and concepts are introduced, and the thesis goals, resource and limitations are presented.

1.1 Radiology reporting

Radiology is the area of medical work where medical images are used to diagnose, and sometimes treat, diseases. Originally, it dealt with the medical use of X-rays for the purpose of obtaining visual information. Today radiologists also direct other imaging technologies such as ultrasound, computed tomography (CT) and magnetic resonance imaging (MR). The radiologist examines the images, and reports the contents of the images by creating a radiology report. The traditional radiology reporting process consists of five main steps (1): image analysis The examination images are analyzed by the radiologist. dictation The radiologist makes a voice recording of the report transcription A secretary creates a written report out of the voice recording approval The radiologist approves the written report distribution The report is distributed by (internal) mail, sometimes electronically. The radiology report is the main medium through which the results of radiology exams are communicated (2). The report is then used as one of the tools to determine the patient’s health state (3).

1.2 Motivation

The radiology report is essentially the end product on which radiologists are judged by their clinical colleagues(4). Despite the importance of the report, the acquisition of the necessary skills to create it are largely left to medical training. Thus, with conventional dictation, radiology reports are largely non-standardized (2). Because of this, traditional radiology reports suffer from several problems, concerning content, correlation with images, reporting time and availability for research. The report content problems are due to reports being commonly incomplete, vague and hard to interpret. Because the radiologist has to remember what to include in the report, without any memory support, sections are sometimes left out or forgotten, which makes the report incomplete (2). If specific organs are not mentioned in the report, the clinician reading the report is not sure whether the organs were examined or not. Another issue concerning report content is that reports are sometimes vague, because the radiologists are not required to express the certainty of the finding, or the impression (2). Also, because there is no standardized terminology, the reports are often hard to interpret. One large analysis of 8426 reports found up to 14 different terms used to describe a single common abnormality, and 23 synonyms for reporting the presence of the same pathology (4).

Poor correlation between report text and actual image data is another problem for the clinician receiving the report. The referring physician often wants to review the actual images described by a radiology report. Even with a PACS system, this task is often difficult or even impossible, because of

3 Introduction

limited access to image data, or because the task of identifying one image among the thousands of images in one examination is very time consuming (5).

The traditional reporting process is also time consuming. As it contains several steps that take time, it is the cause of untimely reports (2). A report turnaround time on more than 24 hours negatively impacts radiology's quality of service and a patient's quality of care. Many clinicians raise this issue as the greatest shortcoming of radiology (5). A survey by the American Healthcare Radiology Administrators (AHRA) found that report turnaround time is a constant source of referring physician dissatisfaction.

Finally, traditional radiology reports are difficult to use in medical research. Free-form text is difficult to use for quantitative analysis, and though use of natural language offers flexibility of expression, summarizing natural-language data requires a human encoder or automated natural-language processing (6). Some groups have attempted to apply natural language processing to free-form text reports, in order to produce organized output (7), but this is difficult to achieve.

Structured Reporting (SR) is a radiological term and research topic, covering the different aspects of a system that allows information to be standardized and presented in a clear, organized format, tracking the attributes of each finding, and prompting the radiologist for input (8). Structured Reporting is meant to provide an efficient mechanism for the generation, distribution, and management of clinical reports. Compared to transcription, the structured reporting system has the possibility to improve the radiology reporting process in several ways.

The content of reports may be improved, as structured reporting provides a standardized method of presentation (7)(9). This way, the radiologist is prompted, which encourages the production of a complete and comprehensive report (7). The standardized presentation also provides another advantage, as referring clinicians do not have to go through vast amounts of descriptions to get to the results. This improves quality and speed of communication between radiologists and referring physicians(7). Another important feature for the reporting radiologist is the automation of technique sections. A single menu selection (or a small number of selections) may contribute a complex procedure protocol to the report (7). There is evidence that structured data entry provides more complete information than comparable natural-language reporting. With structured reporting, the use of standardized language improves the possibility for the clinician to interpret the reports.

Structured reporting also shifts radiology reporting from an exam-centric to a finding-centric emphasis. The former involves tracking the progress of a finding (such as a lesion) by retrieving multiple historical reports and reading through many paragraphs of text. A finding-centric emphasis, in contrast, involves tracking the attributes of each finding across reports, prompting the radiologist for changes or impressions(9). When comparing different types of reports, a majority of radiologists and referring clinicians preferred itemized reports. Prose reports foster a lack of standardization of content among different radiologists. Itemized reports facilitate complete documentation of information and measurements and are more popular with both radiologists and referring clinicians(10). As mentioned previously, if specific organs are not mentioned in the report, then the clinician is unsure whether the organs were examined. This would explain the preference by clinicians for reports with more detail, regardless of whether there was complex disease (10).

Another advantage of Structured Reporting is the ability to link clinical documents with the referenced images for simultaneous retrieval and display (11). An SR solution can create links between image coordinates of specific diagnostic findings and the SR. A referring physician can easily pinpoint a radiologist's findings in the images when the SR is reviewed(5). When SR is combined with PACS on an integrated platform, greater efficiency and potential time savings are possible(12).


4

These links between text and images are not limited to the patient’s current imaging exam. Historical comparison images or from complementary imaging studies can be embedded in the reporting document (1).

Structured reporting enables faster report turnaround time (5), because the dictation, transcription and approval phases are all replaced with the radiologist’s creation of a structured report, and the distribution can be made electronically. This enables rapid report generation, immediate verification and potential immediate delivery of results (10). The instantaneous delivery of reports can improve patient care and decrease hospital stays(7). Also, the transcription costs are eliminated as the transcription phase is removed(7)(10) .

As for research, the encoding of structured reports makes it possible to use the content for data mining. Not only does Structured Reporting include encoding of terminology in one language, but it offers the possibility to use the same code in different languages, thereby supporting worldwide medical research(8). Structured data entry also can be integrated more directly with decision support tools (6) and for physician profiling. Structured reporting methods may be used to measure how efficiently and accurately radiologists review examinations. During the process of computerized reporting, the total review time, time spent per finding, and number of findings can be recorded for utilization management studies. The specific findings in SR can be correlated (5).

Potential disadvantages of a computer-generated reporting system include the requirement of a rudimentary level of computer literacy. Typing skills, for reports of cases of complex abnormalities, are of benefit to augment the basic report. Interpretation errors and the need for a quiet working environment are drawbacks to the use of voice recognition (10).

A misconception about menu-based reporting is that the number of possible findings is so large that a decision tree must be unwieldy to navigate. Research shows that this is not the case. However, the way in which the options are organized has a huge effect on the usefulness of the system (7).

1.3 Concepts and abbreviations

Some words and concepts will be used frequently throughout this thesis, and therefore an explanatory list of these is provided below. API Application Programming Interface. CMIV Center for Medical Image Science and Visualization. A multidisciplinary research

center in Linköping, with the mission to develop methods and tools, both for image analysis and for visualization within health care and medical research applications.

DICOM Digital Imaging and Communications in Medicine. The de facto standard for

medical image communication, defining data structures and services for exchange of medical images and related information (13).

HL7 Health Level Seven. A non-profit organization that provides standards for both

exchange, management, and integration of clinical data and management, delivery, and evaluation of healthcare services (13).

IHE Integrating the Healthcare Enterprise. An industrial initiative to integrate healthcare

resources (13).

5 Introduction

PACS Picture Archiving and Communication System. Computers or networks dedicated to the storage, retrieval, distribution and presentation of medical images.

RADLEX A Lexicon for indexing and retrieval of radiology information resources (14). RIS Radiology Information System. RSNA Radiology Society of North America.

SINR Simple Image and Numeric Reports (15).

SNOMED CT Systematized Nomenclature of Medicine - Clinical Terms(16). Dictionary tools,

created to reveal patient information.

SR Structured Reporting. A radiological term for a system that allows information to be standardized and presented in a clear, organized format, tracking the attributes of each finding, and prompting the radiologist for input (8).

UMLS Unified Medical Language System. A compendium of vocabularies and

classifications and mapping between these vocabularies.

1.4 Sectra PACS

Sectra Imtec AB provides a complete Picture Archiving and Communication System (PACS) solution to radiology departments all over the world. A PACS system consists of computers or networks dedicated to the storage, retrieval, distribution and presentation of medical images. The system gives the radiologists a reading tool in an environment filled with data. In addition to display features of workstations that can improve productivity, PACS can also contribute to major changes in logistics or workflow (12)(17). Figure 1.1 shows the starting interface of Sectra PACS, where worklists are displayed and patients or examinations can be selected for viewing. Figure 1.2 illustrates the image window, used to examine images.

Figure 1.1 Sectra PACS

Figure 1.2 Image window


6

The most useful advantage that workstations provide over film is the availability of stack viewing, in which images are electronically stacked up and viewed in sequence in the space of a single image, rather than displaying a number of images across the monitor. Stack viewing also permits a radiologist to maintain his or her gaze on a specific spatial location while images change. This is very difficult to accomplish in tile mode as the gaze is shifted from image to image. Many radiologists believe that stack viewing allows for a better interpretation of the three-dimensional relationship of various structures (12).

Two aspects are important in the process of interpreting medical images, where the first is automation. In many high-volume film reading environments, the radiologist’s time is optimized if all she or he is doing is interpreting examinations and generating reports. Through the workstation login, the radiologist is known to the PACS, and the correct images can automatically be displayed the way the user wants. The second aspect is the integration of the images with other medical information and notes. In film-based operation, the preloaded film alternator is usually either accompanied by slips of paper or by cards bearing the patient name and location on the alternator. A PACS station handles digital information such as patient and exam data, and can also provide functionality to insert image notes, and support work lists (12).

1.5 Goal

The goal of this thesis is to design an application supporting the creation of structured radiology reports in the existing PACS software. As part of the design process, a prototype of the application will be implemented as a plug-in to the PACS. This prototype will illustrate the system functionality for two different situations;

1. The general radiology reporting case 2. A specific reporting case at CMIV.

1.6 Resources

Related to the existing Sectra PACS software is an API which can be used to access PACS information. As some data was not available through the API, a Sectra application called wdump was used to retrieve information from the PACS database. To convert XML documents to the DICOM SR standard described in chapter 3, a DICOM Offis application called xml2dsr was used.

1.7 Limitations

The application design should take into consideration several different aspects of the area of Structured Reporting, but the resulting design propositions will not all be implemented in the prototype. First of all, the encoding of reports from natural language to coded entries will not be complete. To fully take advantage of the structured reporting entries, each concept should be encoded to enable data mining. This will not be implemented in the prototype. As for the creation of natural language, the prototype should generate complete sentences, but not with the extended vocabulary and grammatical variation needed in a real application. The fast distribution of reports is a part of the Structured Reporting concept, but this will not be addressed in the application design or prototype implementation. Finally, the prototype will be limited by the API functionality and the environment in which it is implemented.

1.8 Thesis outline

7 Introduction

Chapter 2 describes the methods used to establish requirements for the structured reporting application, as well as the way the application was designed, and how the prototype was implemented and evaluated. The results are then presented in two separate sections, chapter 3 and chapter 4. Chapter 3 presents the literature review made to establish application requirements, resulting in a set of structured reporting design guidelines. All other results, consisting of requirements, the designed application, a description of the implemented prototype, and the prototype evaluation results are described in chapter 4. The final section, chapter 5, presents discussion and conclusions on used methods, the designed application, the implemented prototype and future work.


8

IDENTIFY NEEDS

(Literature, reporting tool,

Interviews, meetings)

(RE)DESIGN

(Conceptual design, physical

design)

BUILD INTERACTIVE VERSION

(Visual Studio .NET)

EVALUATE

(Quick and dirty)

2 Methods

This chapter describes the methods used to achieve the goal of designing a Structured Reporting application, and implementing a prototype adopted for two different usage areas. The interaction design process as described by Preece (18) has been used as a foundation, but has been adapted somewhat to suit the situation.

2.1 Design process

The interaction design process describes the methodology of interaction design. Preece (18) defines the three characteristics of the interaction design process as focus on users; specific usability and user experience goals; and iteration. She also describes the four basic activities of interaction design as: identifying needs and establishing requirements; developing alternative designs; building interactive versions; evaluating the interactive versions. To visualize how the activities are related to each other, a lifecycle model can be used. The design process of this project can be described with the lifecycle illustrated in Figure 2.1.

Figure 2.1 Design process The general requirements were created first, based on literature, report tool presentation, and interviews. After this, the general interface and the general sample could be designed and built. The process of creating a specific application for CMIV did not start until parts of the general interface had been designed. Therefore, the sample application created for CMIV started a new design process, closely related to, but in a somewhat different phase than the design process of the general sample application. Both these processes affected the design of the general interface The life cycle illustrates the design process used, and as they are based on Preece’s model, they include the three characteristics and the four activities of interaction design(18).

Figure 2.1 Design process

9 Methods

Detailed descriptions of the methods used throughout the design process are found in the following sections. The results of the conducted literature review are presented in Chapter 3, while all other results can be found in Chapter 4.

2.2 Identifying requirements

The initial way of establishing application requirements for the general interface and the general sample application was by conducting a literature review, presented in Chapter 3. Also, a presentation of a reporting tool used at Rikshospitalet in Oslo, held by one of the tool developers, was attended. This presentation took about 30 minutes, and notes were taken to support recollection of the presentation.

In addition to this, two unstructured interviews were performed with two radiologists from Swedish hospitals, one large hospital and one small clinic, both located in Stockholm. These two radiologists were found through contacts at Sectra, and familiar with Sectra PACS. Each of the interviews took about an hour in time, and sound recordings were made throughout. The interview protocol used can be found in appendix A. Interface sketches, as well as a report template sample, were used to illustrate concepts and to get feedback. These sketches had been created using the guidelines described in Chapter 3, with the purpose of providing the interviewees with something concrete to have opinions about. A sketch of the main interface can be seen in Figure 2.2 and an example of a sketch of a tool drawer is illustrated in Figure 2.3Error! Reference source not found..

The report template sample was simply a paper with text divided into different sections. The sections were then filled with nonsense text to avoid user focus on language style and content, rather than on report structure. The section headings were (in order): Patient Data, Exam Data, Clinical History, Findings, Impressions, Recommendations and Technical exam description.

Figure 2.2 Sketch of main application Figure 2.3 Sketch of tool drawer


10

To generate requirements for the specialized application for CMIV, a total of four meetings, separated in time by one to two weeks, were held to discuss the functionality of the system. The author of this masters’ thesis attended all meetings, while the other attendants varied. Notes were taken throughout the meetings. Summaries of the questions addressed at each meeting can be found in appendix C. The first meeting was conducted to establish the main functionality of the specialized application requested by a CMIV radiologist. The CMIV radiologist, a CMIV research engineer and a Sectra software engineer attended the meeting. The unfinished prototype of the general sample application was presented at this meeting. The second meeting was attended by the CMIV radiologist, and the Sectra software engineer. The purpose of this meeting was to change some of the system requirements, as the plug-in environment limited the possibilities to meet all the functionality the user wished for. At this meeting, no materials were used. The third meeting was made as an unstructured interview, with the only attendant being the CMIV radiologist. Screen shots of the CMIV sample application, as it had been implemented thus far, were brought to this meeting, to provide some discussion material. The final and fourth meeting was attended by the CMIV radiologist and the CMIV researcher who took part in the first meeting. This meeting was another unstructured interview, during which the user could try the application functionality and comment on it while testing, as the current prototype of the application was presented.

2.3 Application development

The system was developed from two different aspects simultaneously. One aspect was the design, controlling what the system should do, behave and look like, and the other was the prototype implementation, where the system structure was determined.

2.3.1 Design

The interface design can be separated into two parts: conceptual and physical. There is no exact border between the two parts, but simply put, conceptual design describes functionality of the system, while physical design describes the appearance. Typically, the conceptual design answers questions like: What functions will the product perform? How are functions related to each other? What information needs to be available? Conceptual design develops a model of what the product will do and how it will behave. During the conceptual design, established requirements and guidelines were used. Most of them were known early in the process, but some requirements were added later in the process, which forced new functionality of the system, and thus the conceptual design was changed. Physical design specifies the details of the design, such as screen layout and menu structure. The menu design does not involve what functionality should be included, but rather the representation and order of elements. Screen design involves making users focus on the correct thing, as well as it concerns good organisation and grouping of elements. The physical design was mainly determined in the end of the design phase. Design principles established during the literature review were used as a support for creating the physical design.

11 Methods

2.3.2 Implementation

The prototype was created in Visual Studio.NET with the programming language C#. A plug-in API (CAI API) was used to access functionality in the PACS and to create a plug-in to the existing system. A Sectra application called wdump was used to retrieve data from the database, as a supplement to the limited functionality of the CAI API. XML documents were created to provide the prototype with templates for report and interface content. The structured reports were also saved and stored as XML documents. To convert the structured reports into DICOM SR format, a DICOM Offis (19) tool, called xml2dsr was used. To use this tool, reports had to be stored as XML according to the XML scheme related to the xml2dsr application. Another tool, provided by Businessware Architects(20), was used to automatically generate C# classes from the xml scheme provided by DICOM Offis. Instances of the generated classes could easily be serialized into an XML document, which had the correct structure to be converted into DICOM SR by the xml2dsr tool. The purpose of the encoding into DICOM SR was to make sure that the application design would work with the structure needed for encoding. A database was created in Microsoft Access, containing tables with terminology from the RadLex lexicon, and a table listing existing SR application reports. Three terminology tables were created; one for terms for anatomic locations, one for types of findings, and one for possible procedures. There were no complex structures or relationships within the database.

2.4 Evaluation

At the end of the project, the two sample applications were evaluated to get user feedback, using the “Quick and dirty” paradigm. The goal for both evaluations was to find out if the application fulfils the usability goals described in chapter 3.5. The interview template with summarized answers for each evaluation can be found in Appendix D. A shorter presentation of the evaluation results can be found in section 5.4. Each sample was evaluated separately, as if that sample application was the only way to use the prototype. This way, both users only tried one of the sample applications, partly because the CMIV application was too specialized to be interesting to the user evaluating the general sample application, and partly because an evaluation of both sample applications at once would take up too much time. For the evaluation of the general radiology reporting application, the prototype was presented to a radiologist at Linköping University hospital in Linköping, Sweden. The user saw the prototype for the first time during the evaluation session. After a short introduction, explaining the evaluation situation, the user was observed while exploring the application, and was asked to “think out loud”. After this, the user was questioned about the prototype in a semi-structured interview. The application used an English lexicon for language creation, and for consistency, the full interface was presented in English, although this was not the user’s native language. The full evaluation session was sound recorded. For the evaluation at CMIV, a semi-structured interview was made with the same user that had taken part in the design process, a radiologist working at CMIV in Linköping, Sweden. The prototype had been installed on a workstation at CMIV about two weeks prior to the evaluation, and had been used twice before the evaluation. Since the user had been involved in the whole development process, the concept of Structured Reporting, and the purpose of the prototype was already familiar to the user, so no elaborate explanation was needed.


12

3 SR Design Guidelines

This chapter presents a literature study of different aspects of Structured Reporting, concerning the functionality and encoding needed in an SR application. The problem of generating medical language is also addressed, as well as interaction and usability issues. The purpose of each section is to identify critical issues and guidelines for the application design. The findings in each section are summarized as a guideline at the end of the chapter.

3.1 Functionality

The functionality of an SR system is dependent on how well it suits the general workflow and work of the radiologist. Some characteristics related to this process are navigation, sequences and reuse of components. An effective structured reporting system must allow the process of menu navigation to be either unbounded, or a fixed path, or some user-defined process between the two extremes, for various and differing reporting methods and tasks. In other words, the navigation must be different for various types of tasks. An effective structured reporting system will allow interpreting of medical images in any progression, for example right to left or top to bottom, in some situations or in a predetermined sequence in other situations(21). The amount of obtained, distributed and archived information is greater than the ability to organize and interpret the data. This requires innovation to develop new ways to navigate through large datasets (1). One dilemma has been how to index images without standardization protocols. This is one of the reasons which have restricted the content of the radiology report. If information like images were available in an easy-to-use, indexed format, radiologists could refer to the images. The multimedia report (including SR) is a new tool, and radiologist challenge is now to find a way to incorporate the new tools into a new reporting-mechanism, without sacrificing the radiologist time and resources. By embedding the images into the report document, the radiologist can identify the pertinent images and provide the clinicians with necessary quantitative measures to assist in the decision-making process. This has the ability to save time for the referring clinician (1). The success of any SR system depends crucially on the completeness of structured report templates, as well as the ability to produce natural language from structured input (7). An ideal system would create fully structured reports that are consistently organized (21). The primary organization layout of a report, or the categorization of its elements, can be thought of as options ranging between a variety of elements at one end and a numeric coding system at the other end. An effective structured reporting system must be able to accommodate all possible types of categories (21). The process of generating a report is more efficient if a new report can be quickly assembled from components or macros that allow the reuse of previous reports, in part or in whole. How the macros are organized and recalled affects the speed and robustness of the reporting process. There must be an organization, a system, to facilitate the creation and editing of macro-level content. An effective structured reporting system will allow frequently used components of the knowledge base to be reused in different ways, and allow short cuts to all modes of using components repeatedly (21). Other needed functionality is related to the need for creation of complete, informative reports. As Trevino experiences from interviews with heads of radiology departments, merely structured format does not provide the flexibility of matching what the radiologists want to convey (9). Because of this,


not all input can be controlled, but free text input is also needed. One of the interviewees claimed that "Structured reporting helps to improve the organization of medical records and reduces ambiguity. Ambiguity is the enemy. Structured reporting will allow a radiologist to express uncertainty about a diagnosis, but it allows the level of certainty to be quantified and expressed in a fashion that is more universally understood"(9). Therefore the SR system should prompt the radiologist for an explicit expression of certainty.

3.2 Encoding

An ideal system would contain end-user hidden codes and data relationships that can be used for data mining and billing (21). By implementing encoding, the data is immediately available for medical research. When text within the report is created by point-and-click manner, the text is captured as structured information, and an underlying clinical database is created and can be used for billing, coding, quality assurance, clinical follow-up and research (4). There are a variety of health care standards and guidelines available, but those reviewed in this section are either commonly used in the area of radiology (HL7), or have been created for Structured Reporting specifically (IHE’s SINR pattern), or fulfill both of the criteria above (DICOM SR).

3.2.1 DICOM SR

DICOM (Digital Imaging and Communications in Medicine) is known as the de facto standard for medical image communication. The standard defines data structures and services for the vendor-independent exchange of medical images and related information. In its modern form, the standard has been available since 1993 which means that DICOM precedes the development of Web technology based on Web services and XML encoding (13). DICOM is a highly specific and explicit specification. The standard specifies all data interchange parameters from hardware factors (industry standards adopted) up through the bit/byte stream, services, protocols, to the domain knowledge layer. A very practical and useful level of interoperability can be achieved with the standard. The need for local configuration agreements is minimized (22). In the year 2000, an extension to the DICOM standard that covers medical reports and other clinical data was officially released, Structured Reporting (SR). In addition to the header information that is also used for DICOM images, the actual content of a structured report is represented by a document tree. Each content item (node) of the tree contains some piece of information, for example, a text paragraph or a reference to an image. A set of well-defined relationships describes how parent and child content items in the hierarchical document structure are related to each other. For example, the standard defines templates to harmonize the document structure and groups of codes in order to limit the choice for a particular context. In the last couple of years, DICOM SR has been implemented many times but mainly as part of prototypes like the ones that vendors show at public demonstrations. Outside of the imaging world, DICOM is not that common and, therefore, it is rather unlikely that DICOM SR will become accepted as a standard outside the imaging area (13). The standard includes seven different relationship types between parent nodes and child nodes (23): Contains specifies that a child and all its descendants are contained within/ are part

of its parent (which has to be a container) Has properties specifies that the parent has some characteristics, which are conveyed by

the child (and its descendants) Inferred from specifies that the parent is a conclusion, deduction or inference made from

the information described by the child and its descendants. Selected from specifies that the parent is a set of coordinates selected from a child (an

image, waveform, or set of coordinates) Has observation context extends the parent and all its children with additional context.


14

Has acquisition context extends the acquisition context of a parent. Has concept modifier can be used to convey equivalent meanings in the same or different

languages, or to clarify the meaning of a specified concept. The purpose is to avoid highly specific concept encoding.

DICOM SR describes content, and does not depend on how it is presented. Synonyms are unified with a single coded entry for a concept, or by a mapping between different codes for synonyms (23). Advantages of coded entries are: the ability to indirect lists of defined terms through separate tables; the potential for reuse of common lists of terms; the potential to outsource the definition and maintenance of lists or terms; the potential to achieve consistency of terminology between different specialties. The disadvantages are that they are more complex to parse and that they expand data sets. SR does not only handle the encoding, storage and transmission of documents, but also provides mechanisms for referring to any (DICOM) object, such as images and waveforms (11). The main problem in SR implementation is to consider the combination of these customization factors to provide the end users with an extremely user-friendly SR toolkit while handling the complicated DICOM SR objects transparently. The DICOM standard specifies only the SR information model and document management, regardless of how the information is presented to the user(11). PACS-specific implementation problems and challenges due to both the necessity to cover all clinical specialties and to be able to handle the format and content of SR documents generated by external devices or clinics. Private templates are commonly used (11).

3.2.2 HL7

Founded in 1987, HL7 (Health Level Seven) is a non-profit, ANSI accredited Standards Developing Organization that provides standards for the exchange, management, and integration of data that supports clinical patient care and the management, delivery, and evaluation of healthcare services (13). The HL7 standard is developed with the assumption that a trigger event in the healthcare world causes exchange of messages between applications. There are currently two different message protocols (version 2 and version 3), where version 2 messages have no defined underlying information model, while version 3 uses a document markup standard, called Clinical Document Architecture (CDA). CDA defines structure and semantics of medical documents, and messages are encoded in XML (13).

3.2.3 The IHE SINR Profile

IHE (Integrating the Healthcare Enterprise) is an industrial initiative with the goal to stimulate integration of healthcare information resources. While IHE does not develop standards as such, it selects and recommends appropriate standards for specific use cases and also develops restrictions, or application profiles for these standards that allow for a simplified system integration. The resulting set of specifications, is published and revised annually. IHE is strongly supported by the industry: more than 160 companies have developed IHE-compliant systems between 1999 and 2005 and participated in the cross-vendor testing events organized by IHE, including most of the market leaders in the modality, RIS (Radiology Information System), and PACS (Picture Archiving and Communication System) sectors. This means that standards recommended by IHE have a high probability of a quick uptake in the medical market (13). IHE presents an effort to agree on how to use the powerful, but flexible tools, such as DICOM and HL7. One of the resulting profiles is the Simple Image and Numeric Reports (SINR), that separates the reporting process into a number of functions, including report creation, report management, storage and viewing (1).


Reports exchanged with the SINR Profile have a simple structure attractive to many imaging departments, including a title, an observation context, and one or more sections, each with a heading, observation context, text, image references, and optionally coded measurements. Some elements can also be coded to facilitate computer searches. Such reports can be input to the formal diagnostic report, thus avoiding re-entry of information. The SINR Profile uses the DICOM SR standard for encoding of reports (15). The profile is visualized in Figure 3.1.

Figure 3.1 The SINR Profile (adopted from IHE(15))

The SINR Profile does not only include reporting content, but separates functions of reporting into discrete actors for creation, management, storage and viewing, illustrated in Figure 3.2. By separating these functions while defining transactions of reports, vendors are enabled to include one or more of the functions in an actual system (15). Reports are created in the Report Creator, processed and modified by the Report Manager, permanently stored by the Report Repository and queried and retrieved by Report Reader. The External Report Repository Access is a gateway to access other enterprise department reports, such as Laboratory, or Pathology, and the Enterprise Report Repository received reports in HL7 format (15).

Document Title

(CONTAINER)

Section Heading (CONTAINER)

Report Text (TEXT)

Image Reference (IMAGE)

Measurement (NUM)


Measurement (NUM)

Coded Entry (CODE)


Measurement (NUM)

Observation Context

Observation Context

HAS OBS CONTEXT

HAS OBS CONTEXT

CONTAINS

CONTAINS

INFERRED FROM INFERRED FROM


16

Figure 3.2 The SINR pattern

3.3 Language & Terminology

In much of radiology, no standardized language exists. The practice of radiology reporting has not changed since Roentgen discovered x-rays in 1898 (9). As imaging systems, the radiology report, and the medical record all move online, there is an increasing need for a unified language to organize and retrieve this information (14). Correct language creation is crucial, as it is through the reports that radiologists’ knowledge is communicated. A structured report depends on the assembly of report elements into a natural language sentence like composition. An effective structured reporting system allows language composition software to link assembled relationships in a semantic net. The selected elements can then be composed into grammatically and syntactically correct narratives as intended by a report author (21). Historically, the use of various imaging modalities such as ultrasound and magnetic resonance imaging (MRI) has sometimes resulted in differences in vocabularies for findings and anatomy. For instance, echocardiographers might use different terminology for cardiac anatomy than nuclear medicine specialists (16)(22). If Structured Reporting is to be successful, a consistent lexicon that would provide a uniform method and common language is needed (4). As of 2002, there was still a lack of widely accepted standardized report formats and lexicons, which limited the possibilities to create more standardized reports. (2). The success of any SR system depends crucially on the ability to produce natural language from structured input (7). Below is an overview of different language tools, possible to use in an SR application: the SNOMED coding system, the UMLS vocabulary and the RadLex lexicon. .

3.3.1 SNOMED CT

SNOMED CT-(Systematized Nomenclature of Medicine - Clinical Terms) is a coding system, controlled vocabulary, classification system and thesaurus, designed to "capture information about a patient's history, illnesses, treatment and outcomes" introduced 2001 (16)(22). SNOMED CT is a dynamic, scientifically validated clinical health care terminology and infrastructure, which has been created to make health care knowledge more usable and accessible. The SNOMED CT provides a

Report Creator

Report Manager

Report Repository

Report Reader

Enterprise Report Repository

and

External Report Repository Access

Report Manager


common language that enables a consistent way of capturing, sharing and aggregating health data across specialties and sites of care (16). The imaging-related content provides broad representation of the procedures, drugs, anatomical structures, and findings specific to clinical imaging. SNOMED CT is considered to be the first international terminology (16)(22), and is probably the most comprehensive medical terminology developed to date. It can be used to support patient data capture, transfer, querying and storage via an electronic patient record. The vocabulary has over 7 million defined relationships, and includes concepts covering multiple use scenarios, e.g. diagnosis, drug definitions, findings, procedures, anatomy. SNOMED CT has been licensed in at least 30 countries worldwide, and has been translated into Spanish and German.

3.3.2 UMLS

UMLS (Unified Medical Language System) is a controlled vocabulary that consists of a metathesaurus, semantic network and lexical applications. It is a compendium of over 100 national and international vocabularies and classifications and provides a mapping structure between them. UMLS was developed by the National Library of Medicine (NLM), USA and introduced in 1986 (24). The purpose of (UMLS) is to facilitate the development of computer systems that behave as if they understand the meaning of the language of biomedicine and health. To achieve that purpose, NLM produces and distributes databases and programs for use by system developers in building or enhancing electronic information systems (25)(26). The Metathesaurus is a collection of medical concepts and inter-concept relationships (24). It is a very large, multi-purpose, and multi-lingual vocabulary database that contains information about biomedical and health related concepts, their various names, and the relationships among them. The Metathesaurus is organized by concept or meaning. In essence, its purpose is to link alternative names and views of the same concept together and to identify useful relationships between different concepts (25)(26). The Semantic Network specifies categories to which medical concepts can belong and the semantic relationships that can be assigned between these concepts and their attributes (24). The purpose of the Semantic Network is to provide a consistent categorization of all concepts represented in the metathesaurus and to provide a set of useful relationships between these concepts (25)(26). The SPECIALIST Lexicon contains syntactic, morphological, and orthographic information for biomedical and common words in the English language. The Lexicon and its associated lexical resources are used to generate the indexes to the metathesaurus and also have wide applicability in natural language processing applications in the biomedical domain.(24). The Lexicon has been developed to provide the lexical information needed for natural language processing. Coverage includes both commonly occurring English words and biomedical vocabulary (25)(26). UMLS supports conversion of terms from one controlled medical vocabulary to another. "The UMLS approach assumes continuing diversity in the formats and vocabularies of different information sources and in the language employed by different elements of the biomedical community. It is not an attempt to build a single standard biomedical vocabulary."(24). The lexical programs or tools are designed to address the high degree of variability in natural language words and terms. Words often have several inflected forms which would properly be considered instances of the same word. Multi-word terms in the Metathesaurus and other controlled vocabularies may have word order variants in addition to their inflectional and alphabetic case variants. The lexical tools allow the user to abstract away from this sort of variation (25)(26). The UMLS includes the English and Spanish editions of SNOMED CT (25).


18

3.3.3 RadLex

RadLex is called “A Lexicon for Uniform Indexing and Retrieval of Radiology Information Resources”, and is an RSNA project. Although a variety of terminologies and standards are currently in use by radiologists, no single lexicon currently serves all the needs of radiology. The RadLex project is a pilot program for development of a comprehensive radiology lexicon. The first portion of RadLex, a draft lexicon for thoracic radiology, has been developed(14).

The purpose of the RadLex lexicon is to provide a uniform structure for capturing, indexing, and retrieving a variety of radiology information sources, such as teaching files, research data, and radiology reports. RadLex will unify and supplement radiology terms in other lexicons, such as the American college index, SNOMED, UMLS, the Fleischner Society Glossaries, and DICOM. The terms will be made freely available on the Internet with cross references to these other lexicons and standards(14). When complete, RadLex will be used to describe all the salient aspects of an imaging examination (e.g., modality, technique, visual features, anatomy, and pathology). The lexicon will facilitate the organization of multiple diverse information sources including teaching files, clinical trials databases, clinical structured reports, web-based information resources, image archives, and other radiology information systems (14).

In 2006, the RSNA-sponsored RadLex project has made major strides, which will culminate with the release of version 1.0 of the entire lexicon in conjunction with the 2006 RSNA Annual Meeting(27). Currently, nearly 100 volunteer radiologists participate in the RadLex project, which covers all major subspecialties and represents over 26 separate subspecialty organizations, both nationally and internationally. Moreover, collaborative relationships have been forged between the RSNA and other groups conducting related terminology efforts: the College of American Pathologists (SNOMED, UMLS, and the National Cancer Institute. As the successor of the American College of Radiology’s Index for Radiological Diagnoses, RadLex will eventually serve as the single source for radiology-specific concepts and terms and will fill gaps where existing terminologies are lacking (27).

3.4 Interaction

“What users want is convenience and results. But all they see is the interface. As far as the customer is concerned, the interface is the product” (28). One of the technical challenges facing radiologists in creating the multimedia report is the status of the user interface. If alternative workflow-enhancing technologies could complement or replace mouse and keyboard, the adopting of multimedia reporting would possibly improve (1). This will not be the case in an SR application, but user interface such as knowledge about interaction styles and control design may still be used to improve the user interface of any application.

3.4.1 Interaction styles

As described by Schneiderman (29), there are three main interaction styles: direct manipulation, menu selection and form fill-in. Before designing, the choice of interaction styles and the relationship between these styles should be determined. Blending several interaction styles may be appropriate when users and tasks are diverse. For example, commands can lead the users to form fillin, or menus can be used to control a direct manipulation environment when suitable visualization of actions cannot be found. Also, keyboard commands can provide shortcuts for experts who seek more rapid performance than menu selection (29). Direct manipulation has three fundamental properties: objects and actions are continuously represented; actions are rapid, reversible, and provide immediate feedback about the object of interest; and “physical” actions such as button pressing or drag and drop replace command issuing.


Direct manipulation is when objects and actions are represented visually, and the users can rapidly carry out tasks and see the results, as is the case with, for example, dragging and dropping. Keyboard entry of commands or menu choices are replaced by pointing devices which control the visual representations of objects and actions. This interaction style makes interfaces easy to learn and remember, and provide rapid interaction by experts. There is hardly ever any need for error messages, as the feedback is immediate, and users feel in control (18). Disadvantages of direct manipulation are that it might consume screen space, and that users must learn the meanings of visual representations. For experienced users, it might take more time to move the hand to the mouse, rather than typing the command (29). Form fill-in is used when data entry is required. Users see a display of related fields, move a cursor along the fields, and enter data where desired. Since knowledge of the keyboard, labels, and permissible fields is required, some training may be necessary. This type of manipulation is most appropriate for knowledgeable intermittent users or frequent users. Advantages of this interaction style are that it simplifies data entry, requires modest training, gives convenient assistance and permits use of form-management tools. Form fill-in is an attractive way of data entry because the full complement of information is visible, giving users a feeling of being in control of the dialog. The visibility of information reduces the necessity of instructions, but a disadvantage of the interaction is that it consumes screen space (29). Menu selection is when people read a list of items, select the most appropriate for the task, and watch the effect. The benefit is that it has a clear structure to decision making. It is appropriate for novice and intermittent users. Terminology has to be chosen carefully and consistently. Menus are effective because they offer cues to elicit recognition, rather than to recall actions (29). The primary goal for menus and form fill-in is to create a sensible, comprehensible, memorable and convenient organization relevant to the user’s tasks (29).

3.4.2 Control design

Controls are the visual elements that let the user interact with the application. GUI designers are faced with an unending variety of controls to choose from. Each new control brings with it expected behaviors and characteristics. Choosing the appropriate control for each user task will result in higher productivity, lower error rates, and higher overall user satisfaction(30). Cooper and Reimann (31) divides controls into four different kinds depending on user goals (some controls combine more than one). The imperative controls are used to initiate a function, selection controls are used to select options or data, entry controls are used to enter data, and display controls are used to directly manipulate the program visually. The pros and cons of some of the most commonly used controls are described here, to create a base for choosing suitable controls for the SR application interface.

Imperative controls

The most common imperative control is the push button, which also is the most visually compelling control, as it indicates pressability (31). Another imperative control is the butcon, which is a button with an icon, most commonly used on a toolbar. The butcon is easy to use, as it is always visible and fast to access, and easy to memorize. The problem with the butcon is the dependence of quality of the icon that describes the function.

Selection controls

There are a number of different selection controls. One of the most common selection controls are the check boxes. Like buttons, they give the user a strong visual clue for clicking. It is primarily a text based control, and as such the checkbox supports the text rather than the other way around. Therefore it is unambiguous, and it forces the user to slow down to read it. The explanation of the checkbox function might also take up a lot of space (31), and might still leave the user guessing what


20

the alternative is, as happens when the checkbox is marked with, for example, “Save to archive on closing” (28). Another common type of selection controls are radio buttons, which provide the functionality of deselecting all other options than the currently selected. This is quite space consuming, but is useful to show the user the full set of choices at once (31) .

Another category of selection controls are list controls. They can differ somewhat in appearance, but they all allow users to select from a finite set of text strings. This is generally a very dry control, but important text items can be distinguished with graphic icons. Similar to list controls are comboboxes, which provide a precise method of data entry by allowing either entering of a value, or selecting of a value in a drop-down-list. This type of control is superb for a single selection, space efficient, and provides a clear separation between the text-entry part and the selection part. Tree controls are also selection controls, especially suitable for presenting hierarchical data. It can be problematic for users to understand data structures, and therefore tree controls should only be used in the case where what is being represented is naturally thought of as a hierarchy.

Entry controls

Entry controls can be separated into three categories. Bounded controls restrict the available set of values that the user can enter. Examples of these are sliders and scrollbars. As the name indicates, these controls should be used were bounded values are needed. Unbounded controls, on the other hand, are open ended. The best examples of this are text fields. The last category of entry controls is spinners, and they blur the difference between bounded and unbounded controls. These controls permit data entry using either the mouse or keyboard.

Display controls

There are a variety of different display controls, where the most common probably is the text control. This control simply displays a written message. Another display control is the scrollbar, which has the visual clue of proportional rendering of content. The drawback of the scrollbar is that it contains no information about its terminal values (current page number, total number of pages etc). Drawers and levers are panes that can be opened and closed in a single action. Drawers are a great place to put controls and functions that are less frequently used, but when they are used, it is in conjunction with the main work area of the application.

3.5 Usability

Usability is usually defined as ensuring that the interactive products are easy to learn, effective to use, and enjoyable from the user’s perspective. It involves optimizing the interactions people have with interactive products to enable them to carry out activities in everyday life. One way of describing usability is through usability goals. Another, more practically intended conceptualization, are design principles, intended to orient designers towards thinking about different aspects of the design (18).

3.5.1 Usability goals

Preece (18) uses a set of usability goals to describe usability, meaning that the essence of the concept can be broken down to these goals. These are: Effectiveness This is a general goal, referring to how good a system is at doing what it is supposed

to do. Efficiency The concept refers to how the system supports the user in carrying out the task. Are

there many or few steps to carry out the action? Once a user has learned hot to use the system, can they sustain a high level of productivity?


Safety The safety issues involves protecting the user from dangerous or undesirable conditions by reducing the risk of users pushing the wrong button, and by providing help to recognize, diagnose and recover from errors. A safe system allows the users too explore, and feel confident about the interaction.

Utility This term describes to what extent the system provides the needed or wanted functionality? Can the users do what they want, the way they want it?

Learnability The concept refers to how easy the system is to learn. How fast can a user get started with core tasks? A wider set of tasks?

Memorability This is a term for how well a user remembers how to use the system, once it has been learned.

3.5.2 Design principles

There is an almost unlimited amount of design principles available, some more specified than others. Below is an explanation of the design principles that are most applicable to this project. Design principles are more concrete that usability goals, and may be used like a checklist by designers to create more usable designs. Visibility This is a principle for the visibility of functions. The more visible functions

are, the more likely that the user knows what to do next (18). Feedback Feedback means sending back information about what was done and what

has been accomplished (18) (29). The hourglass is an example of poor feedback ,as it gives no information about what is happening or how long it will take to finish. Knowing how much longer a given operation will take is very useful for the user, any time the process is longer than ten minutes (30).

Constraints Different types of constraints are available to limit users’ actions. Physical constraints limit movement, logical constraints use common sense, and cultural constraints are learned conventions, like knowing that the color red means “stop”. Constraints limit the way a user can interact with the system, which reduces mistakes and prevent errors (18)(29).

Mapping This term refers to the relationship between controls and their effects (18). A classical example is how light switches on the wall are related to the lights. Good mapping is then when the left switch controls left light.

Consistency When similar operations and elements are used for similar tasks, there is good consistency. This make interfaces easier to learn and use (18)(29). Good user interfaces apply consistent behavior throughout the application and build upon a user's prior knowledge of other successful applications (30). Basic behavior and placement of controls should be consistent throughout the application. As soon as you change the behavior of these basic controls, your user will feel lost (30). A common complaint among users is that certain terms are not clear or consistent (30).

Affordance This describes an attribute of an object that allows people to know how to use it. One common example is the shape of door knobs, which afford pushing or pulling. Physical objects have real affordances, while user interfaces have perceived affordances, which is the same as learned conventions (18). An interface object with good affordance is the button, because it signals “pressability”.

Universal usability This term refers to how a user interface may be designed for both experts and novices (29). Accelerators for experts are examples of how flexibility and efficiency of use is increased (18). Keyboards shortcuts provide an efficient means to enter text and data, but this is not suitable for novice


22

users, so complete and equal keyboard and mouse support are needed for all operations (30).

Reduce memory load As far as possible, users should not have to memorize things within the interface (29). People learn more easily by recognition rather than recall (18), so it is usually better to provide a list of data values to select from rather than have the users key in values from memory. The average person can recall about 2,000 to 3,000 words, yet can recognize more than 50,000 words (30).

Minimize work There are different kinds of work, and all should be minimized to create a good user interface. Logical work is when the user has too understand things to be able to use the interface, such as comprehending the navigation or structure. Perceptual work involves decoding visual layouts, mnemonic work is a term for recalling of passwords and locations. Finally, there is physical work, which concerns number of keystrokes, movement and gestures(31).

Compatibility This term refers to the match between the designed system and the real world. If there are words, phrases and concepts familiar to the user, it is preferable to use them rather than inventing new (18).

User control User control and freedom is a concept for describing the user experience of the interface. If there are emergency exists, permit easy reversal of actions, there is a high level of user control (18)(29). Users should be able to try out functionality, make mistakes, and control what is happening rather than having their actions controlled by the interface. Controlling the user is completely contradictory to event-driven design in which the user rather than the software dictates what events will occur. User control is also achieved by supporting internal locus of control and making sure there are no surprises, users feel they control the interaction (18)(29).

Aesthetic design The design should be aesthetic and minimalistic (18). The look and feel must be consistent. On the basis of users' experiences with one screen or one dialog box, they should have some sense of how to interact with the next screen or control(30).

3.5.3 Design principles for data entry

Some guidelines have been created for data entry in general and form fill-in in particular. Not all of them are applicable to this project, but the following might be found useful. Only the guidelines that are not mentioned as general design guidelines are described here. Minimal input actions Allow users to do as few actions as possible to achieve a goal. Fewer input

actions means greater productivity and fewer errors. Selecting from a list of choices eliminates need for memorization, structures decision-making tasks and eliminates typographic errors. One thing to note is that if users must move their hands from a keyboard to another device, the advantage is negated. Redundant data entry should also be avoided (29).

Clear terminology Terminology needs to be clear (29), and items should be labeled with adjectives or nouns, not verbs (28).

Cursor movement Provide a simple and visible mechanism for moving the cursor between fields. Enabling tab movement is one way to provide convenient cursor movement, keyboard arrows are another (29).

Error correction Entry forms should allow error correction for individual characters and entire fields. Use of the back space key and overtyping should be allowed to enable users to correct mistakes (29).


Error prevention Prevent users from entering incorrect values. For example, in a field for positive integers, do not allow the user to enter letters, minus, or decimal points (29).

Error messages If users enter unacceptable values, the error message should indicate permissible values of the field (29).

Immediate feedback Instant information about errors is preferable. When feedback can only be provided after the entire form has been submitted, the location of the field needing corrections should be made visually clear (29).

3.6 Summarized guidelines

The preceding sections all set their own specific requirements on the design of an SR application, but to be able to use the information, decisions on what applies on this project are needed. Therefore, each section will be discussed here, and as an outcome of each section, guidelines for SR application design are presented. Some functionality is required for creating a successful SR application. Navigation, organization and workflow all have to be adapted to suit the task. Shortcuts to frequently used components are needed, and it has to be possible to reuse components from previously created reports. It is crucial that the report template is complete, and that the resulting report content can be presented as grammatically correct natural language. As of specific functions, free text input has to be made possible, as well as image linking. There is also a great need for prompting the users to determine the certainty of the exam conclusions. As DICOM SR is the only standard found, that is specifically developed for encoding Structured Reports, it seems wise to make use of it. However, while DICOM SR presents a format for encoding, it is not specific enough, as individual modifications can be made. These individual modifications make it difficult to assure compatibility with DICOM SR encoded documents from other manufacturers. It is precisely for that reason IHE has developed the SINR pattern, which limits the ways in which DICOM SR can be used. Since the SINR pattern is not in conflict with DICOM SR, but rather is a simplification of it, the conclusion is that the reports should be encoded DICOM SR using the SINR pattern. As for the prototype implemented in this project, there are resources available to create DICOM SR encoded reports. However, the tool does not follow the SINR pattern, and neither will the resulting reports. As the prototype is already limited by the prototyping environment, any future, complete system has to be built from scratch. Such a system can therefore use a different tool. Reports consist of language, and any tool for creating Structured Reports will never be better than the lexicon used for generation of natural language. SNOMED is an extensive product, already available in more than one language. RadLex is not yet fully developed, but is created to be more complete and suit the radiology area better than any other lexicon available today. The UMLS Lexicon SPECIALIST is in turn very general, using concepts for the entire biomedical world. Comparing the three, SNOMED seems more suitable for the international market, while RadLex seems to be cover all needs of radiology, but in English only, and SPECIALIST is too general. The most important feature while creating an SR application must be to fulfill as many needs as possible, and to develop a tool that supports radiology in particular. Therefore, RadLex seems more suitable than SNOMED, when RadLex is fully developed. To adapt to the international market, UMLS Metathesaurus could be used. As for this project, RadLex is used because there is no need for completeness in a prototype, and because it is easily available on the internet. Also, RadLex is not as extensive as SNOMED. After reviewing these terminology tools, it seems like Radlex will present the most complete lexicon for terminology.


24

Concerning interaction styles, an SR application motivates a mix of styles. Some functionality needed is very similar to functionality provided by menu selections in other applications, and for the sake of consistency, menu selection should be used as interaction style for this functionality. For functionality for report modification, form fill-in is the most suitable interaction style. Fundamental functionality in report modification could preferably be provided in more than one interaction style, to make it as available as possible for the user. To avoid confusion over what functionality can be reached in what way, there should, as far as possible, be a clear difference between report modification activities (such as text editing) and report administration activities (such as printing or saving). The requirements on selection controls and entry controls are hard to predict, as the application may and should be used in different ways at different sites. There is a strong need to create an adaptable application, where selection and entry controls may be changed depending on the situation it is used in. The overview of these controls might help in knowing some benefits and drawbacks of each control type and choose the correct control type for the situation. Concerning imperative controls, regular buttons may be used, as they are familiar to most users. As for display controls, tabs or drawers can be used for controls to provide a large amount of tools while not displaying any tools that are not important at that moment. A benefit of drawers compared to tabs, is that they can be created vertically, and therefore drawers have a stronger connection to the vertical outline of the report. The usability goals and design principles are useful throughout the application development, as a reminder of how to create user friendly applications, but also as a tool for evaluation. The design principles and usability goals are already formulated as guidelines, and there is no need to repeat them here. One general rule, though, is to make sure that there is a strong motivation whenever a design choice goes against a design principle.

25 Results

4 Results

Below are descriptions of the results for each part of the design process. The requirements not already established in the literature review in Chapter 3 are presented in this chapter. Furthermore, the system design describes how an SR application should be implemented in the PACS, and the implementation section describes the created prototype. Results from the evaluation of the prototype are summarized and presented.

4.1 Established requirements

Requirements were partly based on the review, presented in Chapter 3, partly from studying another application, and partly from user interviews and meetings. Some requirements found in the user meetings and interviews are the same as some functionality requirements that were established in the literature study. These requirements were considered to be especially important during the design. Below is a summary of the results of the two initial interviews, and of the CMIV meetings. The main issues reviewed here are those concerning requirements on the system. More extensive summaries of the interviews and the meetings can be viewed in appendixes B and C.

4.1.1 The general interface

The reporting system used in Oslo, as presented by the developer, is an adaptable tool, created as part of a RIS (Radiology Information System), and connected to Microsoft Office Word templates. The reporting application had an administration tool, where users could connect the reporting tool to the word template. The specific reporting tool, created in the administration tool, could then be constructed to contain a limited number of data entry controls. The reporting tool itself was used to enter data in a fast manner, and as this data is entered, a word document is created. The report templates could be set up with images, to describe or clarify data, but images could only be part of the reports only if they were added to the Word template beforehand. Since the reporting tool was specialized for each usage area, it was made evident that an SR application has to be made adaptable. It was also obvious that some areas need a significant number of data entry controls, which means that a lot of space has to be made free for controls. As described by the developer, users request a possibility to use a variety of different controls in the tool, and they also request the possibility to automatically create graphs in the report. Besides this, it should be possible to add images to the report. The interviewees saw Structured Reporting both as a tool for guiding the user in what to include in a report and what to explain or take a position into. None of them thought that the tool can be made universal. The form has to be adaptable to different types of exams. The largest problem, according to the users, is to force the users to create similar reports at the same time as a allowing for user freedom. Both interviewees believed that it would be necessary to force people into using such a limiting tool. One comment concerning image links was that it has to be made extremely simple if it is to be used. During the interviews, more specific requirements also became clear. Headlines with no content should be removed from the report. The user has to be able to save unfinished reports, and to continue working on them later on. It is not necessary to be able to edit some one else’s report, but it


26

is necessary to be able to make extensions on a finished report. It was also mentioned that national and international health care guidelines could be used to create the correct set of templates for an area. Both interviewees were very optimistic about the SR contribution to medical research, but still believed that it would be hard to motivate the users to use the system. The reluctance to use an SR application may depend a lot on the time this takes to create a report, because just two extra clicks might get people to avoid an action. It was not perceived as good to have to click to get into or through writing a report. One user saw great importance in not displaying the tools on top of the report, because then the full report can not be viewed while working with it. As a graphical user interface, the tabs do work fine, but the colors of the sketches were perceived as confusing.

4.1.2 The general reporting application

The interviews brought some specific requirements for the general sample application. A “clinical history” section should be included in the report, and contain the referring doctor’s text. The same goes for the “intention”, or the question asked by the referring clinician. The referring clinician’s question has to be put in there, and there has to be a separation between the clinician’s text and the radiologist’s report. The radiologists believed that this kind of tool, where a form is used to make selections, can only work if combined with free text. One positive aspect is that the headlines alone will make it easier to remember to include all parts of a report. The ”Certainty” expression, prompting the radiologist for a description of how certain the findings or impressions seemed, was perceived as good, and as something that should be commonly used.

4.1.3 The CMIV application

Some of the functionality the user wished for was not directly made into requirements, but adapted to suit the implementation environment. Not all the wanted functionality could be accessed through the plug-in API, and therefore some changes in the requirements were created together with the user. For example, the linked image can not be displayed with measurements within the plug-in, neither can the plug-in control what images are viewed in the image window. Because of such limitations, the following requirements should not be looked upon as complete, but rather as requirements for this specific prototype. Below is a summary of the requirements gathered at the different meetings. For more extensive information about what requirements were considered at each meeting, see Appendix C. For simplicity, all types of measurement or text annotations are hereby referred to as annotations. The main functionality wanted in the SR application was the ability to get a summary of all annotations made in the examination images. Access to annotations in old examinations should be enabled at the same time as new examinations are viewed, in order to compare the old and new images. A click on a representation of an annotation should display the image in an image window. The annotations should then be gathered in some format, and made available as research data. The annotation data should be presented in a table, with the name, image number, series number, alias(label), type of annotation and value. It is also good if the annotation data could be marked, by the user, with an alias or label, and then easily copied into Excel. Functionality of the table could be made similar to Microsoft Office tables. It is better if all annotations are included by default, and then the user could exclude the not wanted annotations from the report. It should also be made visible what images are marked as key images in the image series, because this indicated that it is possible to rapidly access the images. Besides the annotation data, information displayed in the report should include the author, the name of the examination, request number,

27 Results

exam number, and the time of the examination. When printing the document, the user should be able to select what images are going to be printed, and what size the images should be printed in. All annotated images should be displayed as thumbnails in the tool area, and when one of them is clicked, highlight the corresponding text. Display image number, series number below thumbnail images, and put an icon on thumbnail images to visualize what images are key images. Select a full column when the column header is pressed. More than one author should be able to make changes in the report. When the application is started, it should be immediately visible if there are any previously created reports for this patient or examination. These reports should be sorted by examination, and label the examinations by body part, date and time. It should be possible to save a new version of a previously created report.

4.2 Application design

As described in chapter 4, the design can be separated into two different aspects, conceptual and physical design. Simply put, conceptual design concerns the functionality of the application, while the physical design concerns the look and feel of the system. The SR application described below is presented as it should be designed to be a fully functional part of the Sectra PACS that fulfils all the established requirements. The main focus is on the application interface, where functionality and interactivity is visible to the user. Other functionality, such as encoding and language issues, are also part of the design, but no description is provided as to how this should be implemented. The application design mainly applies to the general interface, since this is the framework that can be further developed into a real application. The sample applications are merely specific illustrations of how the general interface can be adjusted for different usage areas.

4.2.1 Conceptual design

Adaptability

To provide an adaptable interface, the report content and interface functionality is controlled by templates. Each type of report has its own template, and when the application is started, report sections and report creation tools are created according to the current template. This allows for great variations in navigation, organization and workflow, and is also an efficient use of space, as tools that are not needed are never present. The template controls what type of widgets are added to the drawer, what the contents of the control should be, what order the controls should be added in, and how the sentences, created by the controls in the drawer, should be constructed.

Starting the application

The designed SR application is a fully integrated part of the PACS, because it needs to have a close relationship to the PACS interface and functionality. The first benefit of this relationship is when the application is started, as it allows the reports to be associated with patients and exams. To achieve this, and to take advantage of the interface look and feel already known to the user, the application is started from the original PACS startup environment. All old reports are represented in a list the same way patients and exams are represented, with the only difference being the button for creating a new report. This startup environment is illustrated in figure 4.1.


28

Main areas

The application consists of four different areas: a menu section at the top, a report preview part to the right, a tool area to the left, and a feedback bar at the bottom, as illustrated in figure 4.2. There are three main reasons for allowing the application take up a lot of space, up to one full workstation monitor. First of all, the specialized applications may need a lot of space to fit all necessary controls, so a large tool area is needed. Secondly, a report preview area is needed to convey to the user what is being created, while user interviews showed that covering the report presentation area with the tools is not an option. Finally, navigating through the application should require the user to use a minimum of “clicks”, because clicks are perceived as time consuming. By allowing the interface to consume a lot of space, functionality may be made visible and accessible.

Figure 4.1 Starting the SR application

Menu bar

Tool area Report preview

Feedback bar

Figure 4.2 Application areas

29 Results

It is possible to start the application in one monitor or as part of one monitor, together with an image window, as illustrated in figure 4.3. If the application is opened in an area, too small for displaying both the report preview and the tool area, the report representation could fill up the entire area. This is mainly an option for reading previously created reports, but may also be used if the application is combined with voice input tools.

Figure 4.3 Split image window

Menu

Some of the functionality identified as requirements for the system are most suitable provided to the users as menu options, because of the system’s similarity to other applications. These functions are all report administration options, providing functionality to create a new report, save a report, save a report as a new version, print preview, and print. The menu options are illustrated in figure 4.4.

Figure 4.4 Menu options

The “New from template” menu option creates a new report from the chosen template. If there is only one template available, or if there is one template used more frequently than others, this template is set as the default template that will control the look of the report when the application is started. If there are several other templates, one may be selected from a list by the menu option. As for the “Save” and “Save as new version” options, the functionality is somewhat simplified compared


30

to other applications, as document locations and file formats are already determined. This constrains the way the user saves reports, which limits user control, but allows document control on an organizational level. The “Save as a new version” option simply provides the same functionality as the “Save as” option does in other applications, where the current document is saved in a different name, or at a different location. The only difference is that the location and file format are not determined by the user, but the document is saved the same way as when the “Save” option is selected. The retrieval of documents are similarly constrained when the menu option “Open” is selected. Instead of displaying all of the file system, the previously created reports for the current patient are displayed. These are displayed as a hierarchy of old reports, sorted by examinations, where reports are labelled by signature and date.

Report preview

One part of the application is a representation of the report content, where the user at all times can view the result of any actions. This way the user knows, at all times, how the report is affected by any actions, as user feedback is immediate. The important free text functionality is provided by enabling users to edit and add text inside the report representation. At the same time as this functionality is definitely needed, it might also defeat the purpose of any structured reporting application, because the users may then create reports without using the SR application tools, thus not making use of all the SR encoding benefits.

Feedback bar

The bottom status bar is made available for feedback, displaying messages on system actions, and also providing hints and information on available keyboard shortcuts. Other usable information in the feedback bar is provided using icons that for error, progress and information. As part of the bar, below the report preview, is a password box and a “Sign” button. When the user enters the correct password, and presses the button, the current report is signed and saved, and distributed to the next step in the reporting process. By making the report signing constantly visible, the user is constantly reminded of the importance of singing a report, and functionality that has to be used for every report is always provided the same way.

Tools

The tool area consists of a main area, usable for any report section, and drawers, specific for each individual report section. The main area is used for displaying images or viewing previously created reports, but is also used as an information area, providing instructions on how to create reports, how they are distributed, and available laws, guidelines or instructions. This area is suitable for report administration tasks, such as saving, opening or viewing images, because this area is related to the report in general, and not to any section or report part specifically. The creation of report sentences is made in the tool drawers, which are all created according to the template. Each drawer controls the content of a specific report section, and the drawer headline is the same as the section headline. The need for adaptability of the system is huge, and the adaptability lies mainly in these drawers. One drawer is created for each editable section in the report, and the controls in each drawer should be determined by a specific template.

Image links

As SR is meant to include text, images and measurements according to the SINR report pattern, different possibilities for creating image links and inserting measurements are needed. To make image linking extremely easy and accessible, a variety of different ways to link images is provided. One significant advantage of having SR in the PACS is the closeness to the images, which improves

31 Results

radiologist workflow. One way of doing this is by enabling drag-and-drop functionality from an image, into the report. Another way is to provide a button in the tool drawer controlling the report section where image links may be added. This button opens a selection window, similar to the window illustrated in figure 4.5, which provides functionality to select one image (or image frame) among all collections related to the current examination. The interaction within this selection window is very similar to other image selection tasks within the PACS, as this already is familiar to the user.

Another way of providing functionality for insertion of both image links and measurements is to simply list them inside the drawer, and add the value as part of the created text. Thereby it is possible to make the insertion by a single click, although this requires the user to know the measurement value or image number. This kind of insertion is illustrated in figure 4.6.

Speed

One very important function needed to provide a fast interface, are shortcuts and cursor movement by using the Tab button. It is crucial for the acceptance among users that the interface can be tabbed through, and does not require a lot of mouse clicks. Keyboard shortcuts are therefore provided to switch focus from a report section and the corresponding drawer. This way, the user can go to and from the tools without using the mouse, at the same time as it is possible to tab through the tools or the report sections. To induce the presence of this shortcut, it is described in text on the feedback bar. Keyboard shortcuts are also available for menu options.

Figure 4.5 Image selection window

Figure 4.6 Measurement insertion


32

Below the surface

The construction of natural language depends on an established lexicon or dictionary, preferably the RadLex, developed by RSNA (14). The terms available in the dictionary are the main building blocks for sentence generation, but what terms are included into the report is dependent on the template used, and on the description of the examination. If the exam is a CT examination of the head, only anatomical locations and possible head findings will be available among the tool drawer terminology. The importance of grammatically and vocabulary correct reports language cannot be emphasized enough, because the radiology reports are the main outcome of the radiologists’ work. The created report is encoded into DICOM SR using IHE’s SINR pattern, in order to make the reports compatible with reports from other manufacturers. The encoding also enables data mining of reports.

4.2.2 Physical design

The main goal for the physical design was to strive for consistency and compatibility, primarily with the main PACS application, but also with other similar applications, such as word processors and data entry applications. The physical design is created to be as similar to the main interface as possible, to give the user a feeling of unity and an initial knowledge of how to interact with the system. This consistency of look, feel and interaction is important for the user, as it decreases learning time and reduces memory load. Compatibility with the rest of the system means that words, phrases and concepts are familiar to the user. The only unfamiliar objects, not present in any other part of the PACS, are the tool drawers. One benefit of using drawers as containers for data entry controls is that the drawer handles are mapped to the sections of the report, so that they come in the same order. Visibility of functions is created by not making controls or options control different things at different times. The system status is made visible by providing information in the feedback bar, using both text, icons, and a progress bar. Tool tips are also used to provide even more information to the inexperienced user. The concept of SR is meant to reduce the user’s memory load and minimize work, partly by prompting the user for input, so that no sections or conclusions are forgotten, and partly by adding standardized text sections nearly without any input actions from the user. Of course, this somewhat limits the user freedom, but all actions are still driven by user input, and all text may be edited by the user, some control and freedom is still left. As far as possible, errors are prevented by choosing controls that do not allow the wrong type of input. The use of bound and unbound controls in the right applications is very important to limit user actions. Because of the templates controlling the drawer content, the type of control can easily be changed into another to suit the situation and the user, and thereby also creating the best possible environment to prevent the user from making mistakes. Screen design includes making users focus on the correct thing, due to colouring, organisation, grouping etc. There are several reasons for the location of the application areas (menu bar, tool area, report preview area and feedback bar). The main reason for these locations is simple, it is what is commonly used in applications today and therefore it is consistent with what the users have experienced. One reason for locating the tool area to the left and the report presentation area to the right is because people in the western world reads content from left to right. That way, the tools are difficult to ignore, and although this is not always convenient for an (unwilling) user, it should serve as a constant reminder that the tools should be used.

33 Results

4.3 Implemented prototype

The description of the implemented prototype is divided into three parts, starting with a system description of the main application, followed by descriptions of the implemented sample applications. This prototype implements all requirements that were within project limitations, except for the possibilities to include images or graphs in the templates. The plug-in is only meant to illustrate the possibilities with Structured Reporting, the way it could be implemented, and the possible advantages of the chosen structure.

4.3.1 System description

The prototype was developed as a plugin to the Sectra PACS, where the CAI API was used to access information within the PACS. Some data could not be reached through the API, and then the Sectra application called Wdump was used to retrieve information from the PACS database. The system is illustrated in figure 4.7.

Figure 4.7 System overview

The report and the interface are both built up using a report template. This template controls what sections should be included in the report. Each report is based on an XML template, which controls what sections are included in the report. The drawer created for a section is generated by looking up the content of another XML file, the Section template. Some sections, such as patient and examination data, clinical history and request information are non-editable sections that may be included without adding any editing controls in the interface. All other sections that are added also causes a “drawer” to be created in the interface. The report template also contains some language information, so that each generated part of the report can be named differently depending on what language is used. For each of the editable report sections that are created, a tool drawer is generated, as is visualized in figure 4.8.

xml2dsr


34

Figure 4.8 Report template

Each section in the report also has its own section template. This template defines what content is added to the tool drawer, and how the section sentences are generated. The functionality of the section templates is illustrated in figure 4.9. What section template is used for a certain section is defined by the report template. This way, sections may be reused, which means that a variety of report structures can be created from one set of section templates.

Figure 4.9 Section template

The prototype uses the database to store information about existing reports, so that information about reports may be displayed to the user without retrieving the actual report. The database also contains lexicon information, used when creating report sentences. There are three tables containing words and phrases defined by the Radlex Lexicon, one with body part terminology, one with terms for findings, and one for procedures. All three tables have the same structure, where one field is an id number, one field is the term, and the last field is a relationship number, defining a hierarchy of terms. As an example, the term “heart” has id value 14 and the term “heart valve” has relationship number 14, because heart valves are part of the heart. Using this database, dynamic lists of terms are produced. Once the report has been created, it can be saved as an XML document. When an already existing report is retrieved, the XML document is used to recreate the report, which may be changed, resaved, or saved as a new version. The XML reports are also converted into DICOM SR reports using the DICOM Offis tool xml2dsr. The DICOM SR reports are not used further in the prototype.

4.3.2 The main interface

The prototype is started as a plug-in in an image viewer window. The menu bar in the image viewer is visible and available, although it is not connected to any functionality in the prototype. The drawers are “hidden” behind the report preview area, and only a handle to the drawer is displayed in the tool area. Each handle has the name of the section it controls written on it, and the

35 Results

handles are ordered horizontally the same way the sections are ordered in the report, so that there is a spatial relationship between report sections and the controls that edit the section text. Figure 4.10 shows how the drawer slides out when the report handle is pressed, or when the keyboard shortcut is used.

The prototype environment does not allow the top menu to be modified, and therefore the menu was placed at the bottom, in the feedback bar. Some report management tasks, such as opening previously created exams, or choosing image print sizes, were located in the tool area at the left part of the window. This tool area was also used as an image viewer, because the plug-in environment does not provide functionality to control an image window.

One limitation in this prototype is the keyboard movement. The keyboard may be used to access editable text and the toolbar drawer, but the menu options are not always available through shortcuts, as the plug-in environment handles the keyboard input when some controls are in focus.

Figure 4.10 Sliding drawer


36

Figure 4.11 Print preview

Figure 4.10 and 4.11 show how the menu options “Print Preview” and “Open” are implemented. The preview option opens a new window, which displays the report as it will be printed. The open option displays a tree view in the tool area. If a user clicks on an icon in the tree hierarchy, the corresponding report is opened.

4.3.3 Sample application 1: The general radiology report

The general radiology report can be applied to many different situations, although it is too general to be applicable everywhere. It has been developed to illustrate the functionality of the system, and as an example of how the system could be used. Besides non-editable sections such as patient data, exam data, clinical history and the referring clinician’s question, four sections are part of the report. These four sections are “Findings”, “Impressions”, “Recommendations” and “Technique”, and each section has a corresponding drawer that may be used to create text that should be added to it. The “Findings” drawer can be seen in figure 4.13. At the bottom of each drawer, there is a sentence area, where the sentence being created is updated as soon as the user interacts with one of the tools. All sections use words and concepts defined in the RadLex lexicon to create sentences, and some controls are updated dynamically to create a hierarchical structure of words. One example is the anatomic location, where the first word is a body part, such as thorax or abdomen, and the following controls are used to specify what part of the thorax or abdomen the user wants to refer to. The prototype does not include all body parts, all different types of findings, or all possible recommended treatments, but is rather a proof that this type of interaction may be used to describe any part of the body.

Figure 4.12 Implementation of "Open"

37 Results

The general radiology report can not include measurements from the images, as they can not be inserted from the main image window, and they can not be displayed in the plug-in environment. Because of this, only one way of including measurements in report text was implemented. All examination measurements are listed, and can be included in the report the same way other describing words are added, by selecting them when creating a sentence.

4.3.4 Sample application 2: The CMIV annotation report

The main purpose of the CMIV application was to display all annotations and measurements made in exam images, so that a list of these measurements in an old examination could be viewed while looking at images belonging to a new examination. The application creates a table of all measurements and annotations made in the images in the current examination. By default, all annotations and measurements are included, but there is functionality to remove table rows. The user has the option of controlling what images included in the report that should also be printed together with the report, and what size the image should be printed in. The table columns are name, image and series numbers, label, type of annotation, value, image link and key image. The measurement/annotation table contains links to the annotated image, and the image is displayed in the main tool area if a link is clicked. The table, or parts of the table, can be copied to the clipboard using a context menu, or by using a keyboard shortcut. The clipboard content can then be copied into other applications, such as Microsoft Excel. The user requested that the table functionality would be like the Microsoft Office tables, but this was not fully implemented. The key image column displays a flag image if the image containing the measurement/annotation has been marked as a key image. Automatic measurement names are created using series data (id, series number, image/frame number, measurement number). The label column is the only editable column, where the user can make a small label or comment for each annotation/measurement.

Figure 4.13 Tool drawer


38

The section drawer is in this case not used for text creation, as hardly any of the table is editable. Instead, the drawer displays thumbnails of all annotated images, and when one of the thumbnails is clicked, the corresponding rows are highlighted. Below each thumbnail, the image and series numbers of the image are displayed, and if an image is marked as a key image, a small flag icon can be seen in the top right corner. For the CMIV report, all users have access to all other users reports. Below the table is a text field for entering comments.

Figure 4.14 Collection of annotated images

The main limitation of this application is the lack of ability to display the annotation or measurement on an image when the image link is clicked. This functionality can not be accessed through the API. Another problem is that when retrieved images are in inverted grayscale, they can not be displayed in ordinary grayscale. This functionality is also missing in the API.

4.4 Evaluation

No prototype changes were made due to the results of the evaluations. However, a summary of the results will be given in this chapter, while details can be found in Appendix D.

39 Results

4.4.1 The general radiology report

The concept of Structured Reporting is considered as something very positive, and is looked upon as the reporting model of the future. Some of the reasons to use an SR application would be the completeness of reports, the possibilities to create statistic information of examinations, and the ease of communication between hospitals. The prototype is perceived as very detailed and complete and the interface is fairly easy to use. There is no doubt that a user, with some training, can work up some speed in using the application. The main disadvantage of this general sample is that it is too general. The layout and structure works well, but the options need to be adapted to both the examination type and to selected options. It is extremely important that the application is perceived as easy to use, if it is going to be an option to speech recognition and dictation. Another disadvantage is that it is somewhat easier to make mistakes when creating a report by clicking options, than it is when dictating. It is easier to check the wrong box, than it is to say the wrong word. Another important issue is that the images have to be visible at the same time as the reporting tool. One interesting aspect that was addressed, was the language generation. The user did not feel that complete sentences had to be generated, but rather preferred the terms to be presented as separate terms in the report, like: Solid process. Right Lung.

4.4.2 The CMIV annotation report

The listing of annotations and image links works well, and provide workflow support in several different ways. First of all, it makes the user feel more secure when all annotations are included in the final report, and it reduces memory load as the user is not required to remember all annotations. Another aspect is when the radiologist wants to make similar measurements on several exams. With a list of measurements, and the images available, it is easier to repeat measurements. The gathering of annotations also increase report clarity, and the reports do not have to be redone, as sometimes happens when measurements are summarized. In the end, it simplifies the work, not only for the radiologist, but also for those who come after. A lot of the aspects that were considered as missing functionality are due to the limitations in the plug-in environment, mainly because image links can not control the image viewers. This makes the task of finding an image unnecessarily complex. Other ways to easily find the image are also missing, because image series numbers are not displayed in the image viewer, and because there is no possibility to see, in the image viewer, if there are key images in the present image stack. Besides this, the list of annotations is also too separated, because measurements can not be labeled in the image window, and therefore measurement values and their labels are separated in the list. The functionality that was perceived as missing in the SR prototype was the ability to print thumbnail images, and to undo changes made to the table. The naming of annotations could also be improved, so that the name includes information that enables the user to search for an examination using the annotation name. Besides this, functions and menus are not always visible to the new user, and the clarity in the document handling and tables can be somewhat improved by color coding selected items.


40

5 Discussion and conclusions

The outcome of this master thesis is the design and the prototype described in this report. Below follows some discussions and conclusions concerning the methods used, the development of a SR application, the implemented prototype, and future work.

5.1 Methods

The possibility to create the CMIV sample did not occur until the main requirements had been established for the general interface, and parts of the application had already been implemented. This caused the two design processes described and illustrated in chapter 4 to be somewhat out of sync, which was not an ideal way to work. The implemented prototype lacks somewhat in structure, because new requirements were constantly appearing, and the prototype lacks some visibility of functions because there was no plan for such functions when the main interface was created. All in all, the contact with CMIV was an excellent chance to design and implement in close relationship with the user, and though some things could not be planned for, the final results are better because of that input. The method used for establishing requirements at CMIV might not be a method to recommend when implementation possibilities are better. As mentioned, the plug-in environment provided some limitations, and as not all limitations were known from the beginning, requirements had to be redone. As for this case, the continuous meetings served their purpose well, because follow-up meetings were definitely needed. DICOM OFFIS xml2dsr is not suitable for future creation of DICOM SR documents, since it does not enable usage of other variables than the ones that exist in their set format. Other variables, that Sectra customers are used to having access to, such as body part, examination code etc. are not included. Another reason not to use this tool is because it does not fully conform to the IHE SINR recommendations. In both the initial interviews and the evaluation sessions, the participants were too few to allow for any general conclusions to be made. Many of the application needs established through the interviews were also supported by the literature review, while other requirements can be seen as an indication of user opinions. It is important to remember that both interviewees were radiologists working in Stockholm, Sweden, which means that their opinions might differ significantly from opinions of users around the world. The evaluation situation is similar, because each sample application was evaluated by one person only, and both users taking part in the evaluation are working in Linköping, Sweden.

5.2 Structured Reporting

As mentioned in chapter 1, some benefits are true for any SR application. My opinion is that the benefits of implementing an SR application in the PACS are considerable, as it provides the user with a tool for reporting the findings in the same environment that the findings can be viewed. Not only does this provide the user with fast access to the actual data that is being reported on, and saves the user from constantly having to switch to a different application, it also creates a possibility to include the report information in the everyday workflow.

41 Discussion and conclusions

The adaptability is crucial for this concept to work, as there is a large variety of needs. By creating templates, this adaptability can be made totally invisible for the end user, removing all important options that have to be made before starting. The natural language also has to bee seen as very important. Because of the adaptability of the system, a lot of the requirements are fulfilled. Navigation can be made different for different tasks, workflow can be made different for different cases. The layout and content of the controls can be organized in different ways, and previously made report parts can be reused. But, as always, the template approach does not only bring benefits. It is easy to imagine the user confusion and frustration if templates are made very similar to each other, while some options and controls are still different. Therefore, information about the template concept, and knowledgeable creation of templates is extremely important. The templates provide guidance in what should be included in a report. To make templates as complete as possible, they should be developed together with the customer, using national and international guidelines for radiology reporting. Since templates control report structure, this is adaptable to the culture of that specific hospital. It is important to remember that these templates should be created to improve reporting quality, not merely to provide tools for generating standardized reports. With a complete template, the user gets great support in remembering what should be included in the report. The benefits of using drawers as containers for data entry tools are the almost unlimited amount of space. As with tabs, one drawer handle uses much less space than the controls it contains. Compared to tabs, that are usually located horizontally, drawers make it possible to get a correct mapping between report sections and the drawer handles. Another difference between drawers and tabs is that there is still a “main” area available, which can be used together with any part of the report. The tool area is also very close to the sentence it creates, which created a natural relationship between the two. Generally, the possibility to immediately see the results of the actions is very informative to the user, at the same time as the user never has to see tools that are not important. The drawbacks of the designed application are due to the same drawers mentioned as a benefit, as the user is not provided with knowledge of what is behind the drawers that are not open. Another drawback is that the radiologists are not all going to be optimistic about changing their way of working. This does have to be orders from management, because radiologists will get upset about having to do something they are not used to, and having to change their routines. It is possible that they might understand the use from a research point of view. As the radiology reports are made more similar to each other, not depending on the author, the possibility to stand out is reduced. Clinicians may get suspicious, as standardized reports may give the impression that the radiologist has not reviewed the images. It is easy to imagine that the use of SR in radiology reporting potentially will decrease radiologists’ productivity by increasing dictation time. Another and perhaps more worrisome effect of SR use is the possibility of distraction during image viewing, causing decreased interpretation accuracy. Speech accuracy, templates, user customization, and PACS integration minimize these disadvantages.

5.3 Prototype

There are several reasons why it is not appropriate to have the Structured Reporting application implemented as a plug-in to the PACS, even if the relationship to images could be achieved in the plug-in environment. First of all, when an application is started through an image window, there is a natural relationship between the application and the image it has been started from, but no relationship at all to the patient or the exam the report concerns. Secondly, the plug-in replaces an image viewer window without having the possibility to control what images are displayed in other


42

image windows. There are also menu options available in the image windows, that are not applicable to the use of the SR application.

5.4 Future work

Future development of an SR application should rely on more reliable user opinions. The prototype implemented for this masters’ thesis can be used for user evaluation, but opinions have to be gathered from all around the world, and the prototype is suitable for an English or Swedish speaking audience only. To provide crucial functionality, the SR application has to be implemented as an integrated part of the PACS, thus enabling functionality for logical application startup and a close relationship with the images. The SR application also needs an administration interface, for the creation and modification of report and section templates. The adaptability of the system will only come into use when templates can be created, recreated and reused for different exams and languages. To be able to create complex but correct natural language, a close relationship between the SR application and a medical dictionary is needed. A language generator will never be better than the dictionary that controls terminology and grammar. A lot of the words and concepts provided by the dictionary should be DICOM SR encoded, so that the resulting encoded reports may be accessible for research and statistics. As an optional way of specifying an anatomic location of a finding, a 3D image could be used. If the report creator could close in on, and perhaps mark up, a specific body part, the reader could be provided with not only the term for the selected body part, but also with an image of it. Of course, some of the benefits of Structured Reporting will disappear when voice input is used to enter text. To prepare the system for dictation, and to enhance encoding of free text input, all free text should be compared to the encoded words and concepts in the dictionary, so that the encoding does not depend on the input type. Still, with free text input or dictation, the author will not be limited to the dictionary terminology, and a variety of terms not included in the dictionary will be used.

43 Acknowledgements

6 Acknowledgements First of all, I would like to thank all the potential users who have sacrificed their time to help me gather information. By taking part in interviews, meetings and observations, they have all contributed to my masters’ thesis. I believe the results are so much better because of it. I would also like to thank everyone at Sectra Imtec, not only for helping me out with various tasks along the way, but also for making lunch hours and coffee breaks so enjoyable. And thanks to my masters’ thesis companions, Daniel and Kristian, for sharing knowledge, motivation, hopes and frustration with me. A special thanks goes to my advisor at Sectra, Henrik Magnusson. You managed to give me all the freedom I needed, while still being available at any time I was in a tough spot. You never seemed too busy to help out, although I know you must have been. I want to send another special thanks to my advisor at Umeå University, Helena Lindgren. You showed interest from the very beginning, and provided invaluable motivation and input for this report. Finally, great thanks to Erik. Somehow you manage to provide ideas and inspiration, be my peer reviewer, my consultant, and my best friend. My life is so much better because of you.


44

References 1. Reiner, Bruce I, Siegel, Eliot L and Shastri, Kaushal. The future of radiology reporting.

[book auth.] Bruce I Reiner, Eliot L Siegel and Weiss D. Electronic reporting in the digital medical enterprise. Great Falls, VA : SCAR, 2003, 9, pp. 83-104.

2. Johnson, Anette J. Radiology report quality: a cohort study of point-and-click structured reporting versus conventional dictation. 9, September, 2002, Academic Radiology, Vol. 9, pp. 1056-1061.

3. Wikipedia. Radiology. Wikipedia. [Online] November 28, 2006. [Cited: December 1, 2006.] http://en.wikipedia.org/wiki/Radiology.

4. Reiner, Bruce I and Eliot, Siegel. Reinventing the Radiology Report, 2: Time to Adapt. December 4, 2004, Imaging Economics.

5. Vinning, David J. The Next Digital Frontier. June 7, 2003, Desicions in Imaging Economics. 6. Kahn, Charles E Jr, Wang, Kun and Bell, Douglas S. Structured entry of radiology reports using

World Wide Web technology. 1996, RadioGraphics, Vol. 16, pp. 683-691. 7. Berman, Gerald D, et al. Structured Radiology reporting: a 4-year case study of 160,000

reports. [Online] November 20, 2001. [Cited: 10 11, 2006.] www.structuredreporting.com/structurad-rsna-01.pdf.

8. Vivalog LLC. Structured Reporting. [Online] September 8, 2005. [Cited: November 8, 2006.] http://www.structuredreporting.com.

9. Trevino, Merlina. Point-and-click designers take on the radiology report. February, 2003, Diagnostic Imaging.

10. Naik, Sandeep S, Hanbidge, Anthony and Wilson, Stephanie R. Radiology reports: examining radiologist and clinician preferences regarding style and content. 2001, American Journal of Roentgenology, Vol. 176, pp. 591-598.

11. Hussein, Rada, et al. DICOM Structured Reporting Part 2. Problems and Challanges in Implementation for PACS workstations. 2004, RadioGraphics, Vol. 24, pp. 897-909.

12. Reiner, Bruce and Siegel Eliot L, Guest Editors. The cutting edge: strategies to enhance radiologist workflow in a filmless/paperless immaging department. 3, September, 2002, Journal of Digital Imaging, Vol. 15, pp. 178-190.

13. Eichelberg, Marco, et al. A Survey and analysis of Electronic Healthcare Record Standards. 4, December, 2005, ACM Computing Surveys, Vol. 37.

14. RSNA (Radiological Society of North America). RadLex: A Lexicon for Uniform Indexing and Retrieval of Radiology Information Resources. [Online] November 1, 2006. [Cited: December 1, 2006.] http://www.rsna.org/radlex/.

15. IHE (Integrating the Healthcare Enterprise). Technical Framework, vol I: Integration Profiles, Revision 6.0. IHE Technical Frameworks. [Online] May 20, 2005. [Cited: October 11, 2006.] available at. http://www.ihe.net/Technical_Framework/upload/ ihe_tf_rev6.0ft_vol1_2005-05-20.pdf.

16. SNOMED International. SNOMED About/Overview. [Online] May 22, 2006. [Cited: 10 13, 2006.] http://www.snomed.org/about/index.html.

17. Lundberg, Nina. Impacts of PACS on Radiological Work. Gothenburg : Department of Informatics, University of Gothenburg, 2000.

18. Preece, Jennifer, Rogers, Yvonne and Sharp, Helen. Interaction design- beyond human computer interaction. USA : John Wiley & Sons, 2002.

19. OFFIS DICOM Team. DCMTK - DICOM Toolkit. [Online] May 10, 2006. [Cited: September 15, 2006.] http://dicom.offis.de/dcmtk.php.en.

20. Businessware Architects. Code XS. Businessware Architetcts. [Online] August 1, 2006. [Cited: September 15, 2006.] http://www.bware.biz/.

45 References

21. Liu, David, Zucherman, Mark and Tulloss, William B Jr. Six characteristics of effective structured reporting and the inevitable integration with speech recognition. 1, March, 2006, Journal of Digital Imaging, Vol. 19, pp. 98-104.

22. HIPAA (Health Insurance Portability and Accountability Act). Digital Imaging Communication in Medicine (DICOM) Standards Committee. [Online] November 13, 2003. [Cited: October 13, 2006.] http://www.hipaanet.com/hisb_dicom.htm.

23. Clunie, David A. DICOM Structured Reporting. Bangor : PixelMed, 2000. 24. OpenClinical. UMLS (Unified Medical Language System). [Online] May 8, 2006. [Cited:

October 13, 2006.] http://www.openclinical.org/medTermUmls.html. 25. United States National Library of Medicine. About the UMLS Resources. [Online] May

19, 2006. [Cited: October 13, 2006.] http://www.nlm.nih.gov/research/umls/ about_umls.html.

26. United States National Library of Medicine. Unified Medical Language System Fact Sheet. [Online] March 23, 2006. [Cited: October 13, 2006.] http://www.nlm.nih.gov/pubs/factsheets/umls.html.

27. Flanders, Adam E. (Editorial) Informatics at RSNA 2006: Major Informatics Initiatives in a Fresh Learning Environment. 2006, RadioGraphics, Vol. 26, pp. 1259-1261.

28. Raskin, Jef. The Humane Interface - New Directions for Designing Interactive Systems. s.l. : ACM Press, 2000.

29. Schneiderman, Ben and Plaisant, Catherine. Designing the user interface - Strategies for effective human-computer-interaction. s.l. : Pearson Education, 2005.

30. Hobart, James. Principles of good GUI design. [Online] September 20, 1995. [Cited: November 5, 2006.] http://www.iie.org.mx/Monitor/v01n03/ar_ihc2.htm.

31. Cooper, Alan and Reimann, Robert. About Face 2.0 - The essentials of interaction design. Indianapolis : Wiley Publishing, 2003.

I Appendix

7 Appendix

A. Unstructured interview template

(Introduction) How does the reporting work today? What are your opinions about it? Do you see any need for change or improvement? Is there any type of structure to be followed today? Any templates? (Description of Structured reporting) What would you include in a Structured Report? Headlines? What do you think about these report sketches? Are there any crucial aspects to take into considerations when creating SR? What do you think about these application sketches? What functionality would you want to include?


II

B. Interview summaries

Transcription of interview 1. Radiologist working at a large Swedish hospital

Spontaneous comments

Radiologists are special, and everyone has their own specific wordings in their reports. Fast distribution of the reports is something incredibly wanted. It is important to remember the importance of reliability, as emergency patient reports must no be lost on the way.

Iimage links in the report

The web version of Sectra’s IDS5 gives the clinicians the possibility to access reports and image series, but not selected parts of these series. The radiologist may create presentations containing only a few important images, but some technical problem causes problems when clinicians try to access these presentations from their journal management system.

Standardizing reports

Clinicians choose to read different radiologists’ reports with more or less care, and they don’t trust individual radiologists the same. It is impossible to standardize the way the radiologists write their reports, simply because there are too many ways to express something. It might be possible to standardize more simple examinations, such as fractures, and to make people use key expressions.

Report structure

The first part of a report is usually a “talkative” part, concerning the details of the findings. In this part, negations are usually present. After this, there is usually a “-“, and then a summary that will answer the referring clinician’s question. At this hospital, traumas are common. When a trauma patient arrives, a team of doctors from different specialties gathers, and makes a team judgment of the injuries. In these cases, the reports are extensive. These types of reports start with a short summary and clinical history section, and then there are headlines structured by anatomy structures, like “head”.

Suggested report structure

Separating the different parts of a radiology report this way could work, specially with more complex reports. This is kind of like it is already being done. The really short reports can not be separated into headlines. The clinical history part in the report should be the referring doctor’s text. The “intention”, or the question asked by the referring clinician is missing. Headlines with no content should be removed from the report. Some radiologists already make the separation between “Findings” and “Impressions”. “Recommendations” are not always used. About the user interface: From a research point of view, searching for results in a database would be lovely, but it would be hard to motivate the users to use the system. It does depend a lot on the time this takes, and just two extra clicks might get people to avoid an action. The radiologist is not concerned with the response time, as long as the errand is removed from his/her desk. Today there are problems with emergency errands, as the turnaround time gets long when a transcript of the report has to be created and signed after dictation. This kind of system can only work if combined with free text. One positive aspect is that simply the headlines will make it easier to remember to include all parts of a report. As a graphical user interface, the tabs do work fine. The user has to be able to save unfinished reports, and to continue

III Appendix

working on them later on. You don’t have to be able to edit some one else’s report, but it is necessary to be able to make extensions on a finished report. It would be best if you could fill in text in the report, and then be forced to click on the tabs before the report may be signed. This does have to be orders from management, because radiologists will get upset about having to do something they are not used to, and having to change their routines. It is possible that they might understand the use from a research point of view.

Research

If this system could cover 50% of all reports, it has come very far. You can never completely avoid free text areas.

Swedish terminology

The word ”Rekommendationer” is too bold to use as a headline. Perhaps ”Råd om fortsatt behandling” would be better.

Transcription of interview 2: A radiologist at a small private clinic

Spontaneous comments

Structured reporting is partly about what to include in a report and what to explain or take a position into. It can never be made universal. Some important aspects on developing this kind of system is how to make a system that support patient security, as well as identifying what biases in the system that might include risks.

Standardizing

To create general reports is tough, but it is possible to do for specific areas. Lung medicine could have a specific form, as well as oncology. Sometimes the clinicians get suspicious from receiving standardized reports, as it gives the impression that the radiologist has not looked at the images. The problem of structured reporting is then that it contains a lot of standardized text.

Report structure

The first part is a describing part, containing explanatory text about the findings. This part also usually contains some negations. After this, there is sometimes one part discussing the cause of the findings. In a more extensive report, there is a summary, an impression and perhaps recommendations. Some general headlines for these sections could be used. The most simple reports only contain a negation, and with these it should be possible to create standard reports. When there are findings, or when there is a specific question from the referring clinician, some ”obvious” parts are left out.

Suggested report structure

This structure would work, but the terminology might need some changes. The technical exam description is not always in the report. The clinical history is part of the text from the referring clinician, but sometimes radiologists enter a few notes. The referring clinician’s question has to be put in there, and you have to separate what is said by the referring clinician and what is being reported by the radiologist. Perhaps “Recommendations” could be part of “Impressions”.

Swedish terminology

The headline ”Intryck” should be replaced by ”Bedömning”. ”Rekommendation” is ok, but technically this means something that the clinician can not choose not to do, so a different wording


IV

might be preferable. Perhaps ”Åtgärd”, ”Föreslagen åtgärd” or ”Åtgärd att överväga” would be better.

Image links in the report:

It takes a little time to put arrows on images and things like that. Image linking has to be made extremely simple if it is to be used. And how would you refer to one image among three thousand? How would the radiologist know what the clinician is interested in? It is not possible to refer to series of images, and those receiving the reports don’t always have quick access to the images, as the computers might be slow. Some radiologists draw arrows and such, but I find it unnecessary.

GUI

The “Certainty” setting is good, and it would be good to use that kind of expressions at a lot of places. All the checkups being made these days contain comparisons from one time to another. They can have totally different patterns than a first time examination. These have to be separated, so “Checkup” should have its own tab in the user interface. I don’t understand the meaning of using the different colors. It is not good that you have to click your way into and through writing a report. It looks nice, but nobody wants to click themselves further on. In stead of this, the tabs could be displayed in a different window, so that you can see the full report at once. To control this application with speech recognition, perhaps you could tab through the headlines, and Shift-tab to go backwards. That way the headlines would still be visible on screen. Or perhaps the active part could expand, and there could be both links and tabs in the report to give the user some choice.

Content

National and international health care guidelines could be part of the forms. The form has to be adapted for different types of exams. The problem is to force the users to create similar reports as the same time as you want to allow freedom. You have to go through the authorities to force people. You also have to be able to make a discussion in text, as part of the “Impressions”.

General comments

What is in the report is not as important as the clinicians trust or acceptance of the radiologist’s conclusions. In order to develop a system like this, Good Clinical Practice (GCP), should be studied, and NHS (National Health Service) could be useful, as well as the Royal College of Radiologists.

V Appendix

C. CMIV Meetings

Meeting 1

Participants (other than author): CMIV Radiologist, CMIV Research engineer, Sectra software engineer Materials: Interactive prototype of general interface The requirements from CMIV were not so much about the report content, but more about how to introduce the application in the daily workflow. The user needs to access annotations and measurements in old examinations at the same time as new examinations are viewed, in order to compare the old and new images. The initial wish was for the user to be able to click on a measurement representation in an image, and then view the image, with the measurement or annotation, in an image window. Once the measurements have been created, they should be gathered in some format (i.e. Excel), for easily available research data. Automatic measurement names were also requested, so that each measurement would be properly marked and related to in the following examinations. Findings in reports are commonly numbered, and then be referred to by number. It is useful to know that annotations and key images exist, and then be able to access them in a fast manner.

Meeting 2

Participants (other than author): CMIV Radiologist, Sectra software engineer Important content is measurements and annotations. Key images are not as important. A suggested name on the measurement/annotation containing series number and image number could be produced. Pet CT is coming, maybe it could be useful there. Missing: connection between SR and image window

Meeting 3

Participants (other than author): CMIV Radiologist Materials: Screen shots of the existing prototype The annotation data should be presented in a table, with the name, image number, series number, alias(label), type of annotation and value. The table should contain some information showing if the annotation is marked as a key image or not. It should also be possible for the user to add some sort of label to each annotation. Make the annotation table easy to copy into excel. There should also be information in the report about the author, about the name of the examination, request number, exam number, and the time the examination was made. No location details about annotations have to be included. When printing the document, give the user the option of including images, and what size it should be printed in. For an ordinary report, it would probably be better to choose what annotations to include, but for CMIV it is better to get the options to exclude the unwanted measurements/annotations. It is too rigid if only the author can make changes to the report. Use the Swedish (Sectra) concepts for annotations and measurements. Show all annotated images as thumbnails in the beginning, and when one of them is clicked, highlight the corresponding text. Check if there are rules for how you should name images in SR.


VI

Meeting 4

Participants (other than author): CMIV Radiologist, CMIV Research engineer Materials: Interactive prototype The functionality of the table could be like Microsoft Office tables. When a column header is pressed, the full column can be selected, and the entire table can be selected by clicking the upper left corner. Some changes in exam data should be made, so that body part and laterality is presented, if possible. Image numbers and series numbers should be displayed below the thumbnail images. Put a flag icon on thumbnails to visualize what images are key images. Previously created reports have to be visible when the plug-in is started, so that it is obvious if the current examination already has been reported on. The reports could then be presented as a hierarchy, sorted by bodypart, examination date and examination time.. It should also be possible to resave a report as a different version. Sometimes the images are displayed in inverted gray color, and the thumbnail images are sometimes dark. Create a comment text field below the table. It would be great to be possible to paste text into text annotations.

VII Appendix

D. Evaluation interviews

The General application – evaluation session with radiologist at Linköping

University Hospital

“Think out loud-session” Every country probably has its own set of headlines. The reporting is different in different countries. The Americans probably want everything elaborated, while the English are quite the opposite, and want everything to be very concise. Every country has its tradition. I am a little taken aback by the completeness of this, but it is extremely important that it is easy to fill in these options. My first impression is that I need access to the image while this is done. It is a disadvantage when you have to move your focus from the images to the reporting system, as happens when you have to make sure that speech recognition systems write the correct thing. It is very important that the image is easily accessible, and the application can not cover the image, at the same time as it is unwanted to have to switch applications all the time. The provided alternatives must be different for different kinds of examinations. It would be better to see all alternatives, and not have to scroll in a drop-down list. When not all alternatives are visible, you have to know the options to be able to choose. Perhaps it could be like menu options, when the alternatives are made visible as an option is marked. It is hard to understand all these terms. This system probably needs some training to be usable, because it has to be familiar to be usable. This will be an alternative to voice recognition and dictation. It is excellent with digital reports, to be able to search for “How many of our lungs have a 3cm large tumor?”. The question is how much work people are prepared to do to accomplish this. It has to be very simple and fast, with a good workflow. With this many alternatives, you probably have to know them all to make the reporting fast. People don’t like when you have to use many “clicks” to achieve something. Maybe a hierarchy of alternatives could appear in the report right-click-menu for each report section? I am very grateful when there is a possibility to create my own shortcuts and use the keyboard instead of the mouse. There should be a keyboard shortcut to produce a report for a standard exam, and the four or five most common report answers should also be made available by shortcuts. Can you enter free text in the sentence box? The technique sections may be added earlier in the reporting process, but it is nice to be able to report the image quality, so I can make a reservation against bad image quality. If I change from one type of finding to another, the other options should be reset. All these grading options should have numbers, so that you don’t have to read all the text, and so that you easily can think of it as a scale from one to five, for example. When things are to be done fast, you want to choose things in a specified order. All these options make the report very complete, but those options that are not applicable to all findings should be removed when they are inappropriate. The reporting process is going in this direction. Some find it troublesome, but I can not see how it can be made any simpler. When images are sent to another hospital, which is done for a variety of reasons, it is difficult to send the RIS information as well. A structured report, containing a complete set of information, could make this a lot easier. Observations:


VIII

It is hard to understand the hierarchy of terms when only one term was visible. The user had no problems with the understanding of tool drawers, how to create sentences or how sentences are added to the report. The prototype environment was somewhat confusing, because the butcons at the top had nothing to do with the plug-in. What, in the SR application, doesn’t work well? This prototype is very complete, which is good, but is extremely difficult to combine this completeness with simplicity. To limit the alternatives depending on type of examination, and previously selected options would be one way to make this simpler. It should be impossible to select inadequate or irrelevant options. Why? I believe the focus has to be on speed and ease of use to make this an alternative to speech recognition and dictation. You almost have to be able to know the location of each alternative in your sleep, so the options cannot move around. What do you think works well? I suppose this is what it is going to be like, and it is very complex. The concept is good. Space consuming lists are better than drop-down lists, because it is easier, faster, more clear, and gives a better overview. Are there functions available to do everything you would like to do? (Effectiveness)

I am missing the possibility to see the image while I am working. You also need to see the alternatives, and only get relevant options. It would also be good to have keyboard shortcuts to create the most common report. It has to feel easy to use, because a standard report might only take about 30 seconds to dictate. Is the application easy or hard to use? (Efficiency, learnability)

It feels quite easy. The created language might get bad at times, but I don’t think this is very important. A lot of energy can be spent on trying to create well formed text, but that is not as important for me personally. You know that the report has been created by selecting terms, and perhaps it could be presented like that. Solid process. Right lung. Reports could be created like that, because in many cases this would be easier to read than going through a lot of text. Are you worried about making errors? (User control, feedback, constraints) Yes, this will be an issue. Why? It is easier to make the mistake of clicking the wrong box, than it is to say the wrong term when dictating. Checking the wrong box might completely change the meaning of the report. You would have to read through the text, but these mistakes cannot be avoided. Do you think it is possible to use the application rapidly? Absolutely. As long as there is some logic to how it is built, and it is possible to learn where to find things. You get faster from training, and this also makes people more tolerant, because the first impression of a new system can be somewhat overwhelming. I can see myself using this kind of system, and I also believe this is the working model of the future. What do you think about the graphical aspects of the user interface? Lists work better than drop-down lists, because the latter need extra clicks, and don’t give an overview of the options. The tool drawers were a little difficult to understand at first sight, but now I understand completely.

The CMIV application – interview with CMIV Radiologist

What, in the SR application, doesn’t work well? You can not see measurements or annotations in the displayed image. It is like that last step is missing. Why is this bad?

IX Appendix

I have too look in three different places to get all the information. That takes time and requires me to move my eyes three times. The image series number is not visible in the image window. Why is this bad? It is difficult to relate the listed annotations to the images. What do you think works well? The listing of measurements work well. With only annotations and image links available, this is probably as good as it gets. Why? In the future, it will probably be common with many stacks and a lot of images in each stack. By setting measurements and annotations, work can be done that simplifies the task for those who come after. The information is more accessible this way. IDS5 does not support this today. If you want to make similar measurements the next time, it is easy to see how it has been done before. I can not guarantee that I will benefit from adding an extra element to the working procedure, but I feel safe using it, because I know then when I have mentioned all the measurements and really finished a report. I think it will be easier to finish a report, and I will have to memorize less things. It is also good to get key image information. Why? That way it is faster to find the image in a stack of images. When previously created reports are displayed, it is good to see time and date information both for the examination and for the report. Why? This way you can differ between them. Are there functions available to do everything you would like to do? (Effectiveness)

I would like to be able to print the thumbnail images as they are displayed in the drawer. These thumbnail images also need graphical elements. It is not possible to regret changes to the list. Is the functionality visible? (Visibility) “Save as new version” is not clear. More generally, it is not clear how the handling of documents work. You have to know about the context menu to find the functions for deleting, selecting and copying rows. Is any functionality missing? The possibility to label measurements in the image window. Why is this needed? It would make the information in the list unified, because the measurement and its label would be presented together, and not in separate rows. You can not see, in the image window, if there are key images in the stack. The naming of annotations in the list could be better. Perhaps exam number or request number could be used for this. Why is this needed? This way the annotation name could be used to search for the correct exam, and annotations could also refer all the way back to the examination room. It has to be made more clear what exam is the present exam, when the “open” tree is visible. Why?


X

The information about what exam is presently viewed is hidden, which makes it easy to pick the wrong report, specially when images are very similar. The links in the application should control the image viewer directly. Why? Bringing images into the plug-in is inefficient and space consuming. It should be made more clear which row in the list was just clicked. Perhaps the row could be colored, or marked with an arrow Why? It is sometimes hard to know what link was clicked. Is the application easy or hard to use? (Efficiency, learnability)

Easy. From the beginning, it takes some time to remember to start the image viewers with the right hangings, but otherwise it is easy. Is it similar to any other system you have been using? (Consistency, compatibility) It is similar to a feedback application I have seen, and to other IDS5 applications. Are you worried about making errors? (User control, feedback, constraints) No It does not feel like I can end up in a corner. Do you dare to try functionality? Yes Are there any difficulties to remember, from time to time, how the plug-in is used? No, I don’t think so How many times have you used the application? Twice Do you think it is possible to use the application rapidly? Yes. Other things will speed up too, such as finding errors. And sometimes, if measurements are summarized in stead of mentioned separately, the report has to be redone, because the clinician does not think it is clear enough. What do you think about the graphical aspects of the user interface? It takes up a lot of space, but I believe it is necessary because both surfaces are needed. You can not get away from it. The design is similar to IDS5.

Documents

Application design supporting structured radiology reports · 2007. 2. 13. · Administrators (AHRA) ... The standardized presentation also provides another advantage, as referring