An IDE for C# Script Development

Charles University in Prague

Faculty of Mathematics and Physics

MASTER THESIS

Jan Pelc

An IDE for C# Script Development

Department of Distributed and Dependable Systems

Supervisor: Mgr. Pavel Jezek, Ph.D.

Consultant: Ing. Tomas Novotny

Study program: Computer science

Specialization: Software systems

Prague 2015

I would like to thank everyone who helped me with this thesis, especially mysupervisor Pavel Jezek for his help with the text of this thesis and my consultantTomas Novotny for his support, encouragement, testing and patience. I wouldalso like to thank Josef Zavisek for his input and encouragement. Last but notleast, I would like to thank my family and colleagues for their support.

I declare that I carried out this master thesis independently, and only with thecited sources, literature and other professional sources.

I understand that my work relates to the rights and obligations under the ActNo. 121/2000 Coll., the Copyright Act, as amended, in particular the fact thatthe Charles University in Prague has the right to conclude a license agreementon the use of this work as a school work pursuant to Section 60 paragraph 1 ofthe Copyright Act.

In Prague, April 28, 2015 Jan Pelc

Title: An IDE for C# Script Development

Author: Bc. Jan Pelc

Department: Department of Distributed and Dependable Systems

Supervisor: Mgr. Pavel Jezek, Ph.D.

Consultant: Ing. Tomas Novotny

Abstract: The goal of this thesis is to explore tools for using the C# programminglanguage to create scripts – short programs for quick and easy solving of small,usually one-time temporary tasks that usually arise during the work on largerprojects. In the thesis we analyze existing tools and identify their advantagesand disadvantages, formulate requirements for our own tool, and develop ourown tool. The result of the thesis is a small integrated development environment(IDE) for quick and easy authoring of scripts in the C# language. The IDE offerssufficient features to allow easy authoring and debugging of programs consistingprimarily of a single C# source code file. In the work we make heavy use of theNRefactory library for syntactic and semantic analysis of the C# source code.

Keywords: IDE, C#, scripting, NRefactory

Nazev prace: Vyvojove prostredı pro vyvoj skriptu v jazyce C#

Autor: Bc. Jan Pelc

Katedra: Katedra distribuovanych a spolehlivych systemu

Vedoucı: Mgr. Pavel Jezek, Ph.D.

Konzultant: Ing. Tomas Novotny

Abstrakt: Tato prace se zabyva nastroji umoznujıcımi pouzitı jazyka C# k tvorbeskriptu – kratkych programu urcenych pro rychle vyresenı malych, typicky jedno-razovych uloh, ktere obvykle vyvstavajı pri praci na vetsıch projektech. V pracizanalyzujeme existujıcı nastroje, urcıme jejich vyhody a nevyhody, zformulujemepozadavky na nas vlastnı nastroj a tento nastroj vytvorıme. Vysledkem praceje male vyvojove prostredı (IDE) pro rychle a snadne psanı skriptu v jazyceC#. Prostredı nabızı dostatek funkcı pro snadnou tvorbu a ladenı programusestavajıcıch se prevazne z jedineho zdrojoveho souboru v jazyce C#. V praciintenzivne vyuzıvame knihovnu NRefactory pro syntaktickou a semantickouanalyzu zdrojoveho kodu v jazyce C#.

Klıcova slova: vyvojove prostredı, C#, skripty, NRefactory

Contents

1 Introduction 3

1.1 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . . . 41.2 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2 Problem analysis 7

2.1 Existing tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.1.1 Features of interest . . . . . . . . . . . . . . . . . . . . . . 72.1.2 Available tools . . . . . . . . . . . . . . . . . . . . . . . . 92.1.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.2 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152.2.1 Non-functional requirements . . . . . . . . . . . . . . . . . 152.2.2 Editor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162.2.3 Compilation . . . . . . . . . . . . . . . . . . . . . . . . . . 192.2.4 Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3 Implementation analysis 21

3.1 Tools and libraries . . . . . . . . . . . . . . . . . . . . . . . . . . 213.1.1 Basis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213.1.2 Programming environment . . . . . . . . . . . . . . . . . . 213.1.3 User interface . . . . . . . . . . . . . . . . . . . . . . . . . 223.1.4 Syntactic and semantic analysis . . . . . . . . . . . . . . . 223.1.5 Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . 233.1.6 Extensibility . . . . . . . . . . . . . . . . . . . . . . . . . . 24

3.2 Previous work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243.3 Application architecture . . . . . . . . . . . . . . . . . . . . . . . 26

3.3.1 Plugin infrastructure . . . . . . . . . . . . . . . . . . . . . 263.3.2 Temporary storage . . . . . . . . . . . . . . . . . . . . . . 303.3.3 Configuration and settings . . . . . . . . . . . . . . . . . . 31

3.4 Compilation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323.4.1 Header sections . . . . . . . . . . . . . . . . . . . . . . . . 323.4.2 Builder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.5 NRefactory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363.5.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . 363.5.2 Core . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393.5.3 Code completion . . . . . . . . . . . . . . . . . . . . . . . 41

4 Implementation 45

4.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454.1.1 Components . . . . . . . . . . . . . . . . . . . . . . . . . . 464.1.2 Plugins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

4.2 Application Core . . . . . . . . . . . . . . . . . . . . . . . . . . . 504.2.1 Plugin infrastructure . . . . . . . . . . . . . . . . . . . . . 504.2.2 Configuration and settings . . . . . . . . . . . . . . . . . . 544.2.3 Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . 564.2.4 Temporary storage . . . . . . . . . . . . . . . . . . . . . . 56

1

4.2.5 Application startup . . . . . . . . . . . . . . . . . . . . . . 564.2.6 Stand-alone application executable . . . . . . . . . . . . . 57

4.3 UI Core . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 574.3.1 Main window . . . . . . . . . . . . . . . . . . . . . . . . . 574.3.2 Commands . . . . . . . . . . . . . . . . . . . . . . . . . . 58

4.4 Editor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 604.4.1 Document infrastructure . . . . . . . . . . . . . . . . . . . 604.4.2 Extensibility . . . . . . . . . . . . . . . . . . . . . . . . . . 614.4.3 Highlights . . . . . . . . . . . . . . . . . . . . . . . . . . . 614.4.4 Code completion . . . . . . . . . . . . . . . . . . . . . . . 634.4.5 Other services . . . . . . . . . . . . . . . . . . . . . . . . . 66

4.5 Builder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 674.6 Compiler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 714.7 NRefactory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

4.7.1 Core . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 734.7.2 Code analysis . . . . . . . . . . . . . . . . . . . . . . . . . 784.7.3 Code analysis by NRefactory . . . . . . . . . . . . . . . . 824.7.4 Imports . . . . . . . . . . . . . . . . . . . . . . . . . . . . 844.7.5 Syntax highlighting . . . . . . . . . . . . . . . . . . . . . . 904.7.6 Code completion . . . . . . . . . . . . . . . . . . . . . . . 91

4.8 Debugger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

5 Conclusion 97

5.1 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 975.2 Comparison with other tools . . . . . . . . . . . . . . . . . . . . . 985.3 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

Bibliography 101

A Content of the CD 103

B User guide 105

B.1 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105B.2 Quickstart guide . . . . . . . . . . . . . . . . . . . . . . . . . . . 105B.3 Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107B.4 Source code editor . . . . . . . . . . . . . . . . . . . . . . . . . . 109B.5 Compiling and running . . . . . . . . . . . . . . . . . . . . . . . . 111B.6 Header sections . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

B.6.1 Available commands . . . . . . . . . . . . . . . . . . . . . 113B.6.2 Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . 116

B.7 Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117B.7.1 Execution control . . . . . . . . . . . . . . . . . . . . . . . 117B.7.2 State inspection . . . . . . . . . . . . . . . . . . . . . . . . 118B.7.3 Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

B.8 Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119B.8.1 Configuration sections . . . . . . . . . . . . . . . . . . . . 120

C Extension points 123

2

1. Introduction

The C# programming language is already over a decade a very popular tool forapplication development. Thanks to the Mono project [1] it can be used on otherplatforms than its native platform Windows and .NET, including increasinglypopular mobile operating systems. During its lifetime the C# language has beenextended with a lot of interesting features (e.g. lambda expressions, LINQ, orasynchronous programming support) and has attracted a broad user community,which has produced a lot of frameworks and libraries for various purposes. Fur-thermore, the C# language shields the developers from many low-level aspectsof computing, such as pointers or memory (de)allocation. Thanks to this, it al-lows for a greater level of abstraction, and is also more forgiving to beginners.All this combined with the presence of advanced development tools makes theC# language a unique choice for solving a wide range of common programmingproblems.

However, there is one domain where the available toolset for the C# languageis not sufficient. This domain is the usage of the C# language for quick and easysolving of small, often one-time temporary tasks. Because of the nature of theprograms that are written to solve these tasks, we will call them scripts, since theyare traditionally solved using specialized scripting languages (e.g. PowerShell,Bash, Python, Lua). Such specialized languages are often not very suitable fora given situation, mainly for these possible reasons:

• The nature of the problem may require users to work with the same tech-nologies or libraries as their larger C# projects.

• The specialized language may not be available in the target environment,or it may not be practical or even possible to deploy it there.

• Users may not have the necessary knowledge to solve the problem using thespecialized language, or they would be able to use the C# language to solvethe problem more effectively (e.g. by taking advantage of a known languageand familiar libraries, or static type checks, which are often not availablein specialized scripting languages).

By the term “script” we will from now on mean a small program (usually con-sisting of a single source code file) created ad hoc to quickly solve a temporarytask.

The situations where it is necessary to write scripts to solve such tasks arisevery often in the development practice. Most common are situations where usersare not necessarily required to use a specific language and there are other spe-cialized scripting languages designed to solve the problem, but they can benefitfrom using the C# language for the reasons mentioned above. These may includefor example:

• Small ad-hoc utilities (e.g. format conversions, downloading, extraction,data generation, password or key generation).

• Shell-like scripts (e.g. file and text manipulation, batch processing, addi-tional build steps).

3

• Prototyping of more complicated algorithms and ideas.

Then there are also situations where the user needs to make quick use of theC# language itself in order to find out some information. These situations mayinclude for example:

• API experiments – Users are not sure of the exact behavior of a certainlibrary class or method in a specific situation, or they need to try out a codesnippet, for example a non-trivial regular expression or a more complicatedLINQ expression.

• Microbenchmarks – Experimental performance comparison of two differentcode snippets (e.g. a question of, “How much is using the Activator classslower when compared to direct constructor calls?”).

• One-time testing (e.g. a test web service client).

Whenever such tasks need to be solved using C#, in most cases the only readilyavailable option is to launch a new instance of a classic large IDE (integrateddevelopment environment) like Visual Studio [2] or SharpDevelop [3] and createa new project in the IDE. This is often inconvenient for quick writing of smallscripts. Therefore, this work is focused on alternative options of using the C#

language to create scripts. We will have a look at the various tools that have beencreated for this purpose so far and assess their advantages and disadvantages.Then we will specify what our own tool should look like and create it. The resultof this work should be a small and easy-to-use tool for writing scripts in the C#

language.

1.1 Problem statement

In the introduction we have defined what C# scripts are and stated that our goalis to look for an easy way of working with them. In this section we will lookmore closely at the common problems that users face when they try to use theC# language for quick solving of small tasks in a classic large IDE, and betteroutline the motivation behind our work.

The situations where it is necessary or helpful to quickly create small tem-porary programs in C# come up very often. The simplest scenario is when theuser needs to immediately execute a single call, for example an API experiment.There are in general several options:

1) It is usually fastest to insert the given call into a suitable place in the currentlyopen project, possibly place a breakpoint on it, and run the project. Sucha solution is generally very quick and readily available, but it is not “clean”since it combines unrelated code and can easily lead to the pollution of theproject with commented-out code snippets. Moreover, this solution may notbe feasible for larger and more complicated projects whose execution flow isnot trivial.

4

2) Another option is to execute the call during a debugging session using a de-bugger console. But this requires the debugger to be active, it is not verycomfortable, and the debugger console usually allows only simple individualcalls (there is only a limited subset of the C# language available – for examplelambda expressions are usually not supported).

3) If a unit-testing framework is available, users can create and run a new testcontaining the given call. But this way they use something which was notmeant for such a purpose, and again they pollute unrelated code, this timethe unit tests.

4) The final option is to create a new project for our call and go through allthe steps that such an action requires. Even though this is in principle thecleanest solution, the overhead of creating a new project is significant – usersneed to launch a new instance of the IDE, create a new solution and a newproject, give it a meaningful name and save it into a new directory on thefilesystem. Because of this overhead, users usually resort to this only as thelast option.

If the code snippet is more complicated than a single call, it is often required tosave it for later reference. In this case, the first three options cannot be used andthe only practical and clean option is the last one – to put the code in a newproject of a classic large IDE. However, doing so has many disadvantages:

• It is usually necessary to launch a new instance of the IDE. Then theuser has to wait while the new instance initializes, and afterwards it usesa significant portion of system resources, especially memory. While it ispossible to share a single instance and a single solution between multiplescript projects, this in practice does not help much because it complicatesscript management and adds the inconvenience of having to switch betweenstartup projects.

• Although the code snippet itself usually consists of no more than a singlesource code file, the overhead of using a classic IDE requires naming andcreation of a new folder on the filesystem with a lot of additional files andsubfolders in it – the solution file (.sln), the project file (.csproj), theAssemblyInfo.cs file, the folders bin and obj with their own specific con-tent, the files for storing the user settings related to the environment (.suo,.user), etc.1 For the purpose of simple scripts, all these files are redundantand such overhead only complicates the manipulation and organization ofscripts, especially in larger projects where even the supporting scripts needto be organized in a repository for the purpose of version control, backups,and knowledge sharing.

1For example: creating, building, and running a simple console application in MicrosoftVisual Studio 2010 [2] requires creating 2 folders and 5 files (excluding the bin and obj sub-folders), or 8 folders and 13 files (including the bin and obj folders). SharpDevelop 4.3 [3]is only a little better in this respect, requiring 2 folders and 4 files, or 6 folders and 10 filesincluding the bin and obj folders.

5

• It can be difficult to modify scripts in an environment which has limited re-sources or in which the IDE is not present and its installation is complicatedor for any reasons (e.g. license-related, security-related or time-related) notpossible. This is often the case of a production or testing environment ifan immediate maintenance action is required.

Even though using a classic IDE for creating small helper programs has its dis-advantages, in our experience developers who extensively use the C# languagediscover in time that their workspace contains many projects that were createdad hoc, without any prior planning, when they wanted to immediately solvea small task, or when they wanted just to quickly test how a certain code snippetbehaved in a specific situation. Such projects are usually console applicationswith unpolished code (e.g. with no error handling or with hard-coded data), theyusually have a very specific or one-time function, and their solution file containsa single project file with a single user-modified source code file. These programsare often typical examples of scripts as we described them in the introduction.

1.2 Goals

We have defined scripts as small programs (usually consisting of a single sourcecode file) created ad hoc to quickly solve a temporary task. While it is possibleto use classic IDEs for script development, doing so has disadvantages describedin the previous section. On the other hand, when using classic IDEs for scriptdevelopment, users can take advantage of many features that make source codewriting easier, quicker and user-friendlier, such as syntax highlighting, code com-pletion, refactoring or debugging. What is missing is a C# tool which includesthese features, but which, as opposed to classic IDEs, is also light-weight anddoes not require using the classic organization of source code into solutions andprojects. Instead it should allow easy development of scripts.

Therefore, we will look for alternative tools (tools that are not classic IDEs)which allow using the C# language to author scripts. We will examine the varioustools that have been created for this purpose so far and assess their advantagesand disadvantages. Then we will specify what our own tool should look like andcreate it.

Summary

The primary goals of this work are:

a) Analyze the available tools that allow the usage the C# language for scriptdevelopment.

b) Based on this analysis, identify the advantages and disadvantages of the avail-able tools and formulate the requirements for a new tool.

c) Analyze the requirements for the new tool and develop it.

The result of this work should be a small and easy-to-use tool for writing scriptsin the C# language.

6

2. Problem analysis

In this chapter we analyze the relevant existing tools that allow using the C#

language to write scripts as we described them in the introduction. Based on thisanalysis, we formulate the requirements for our own tool, further specifying thegoals of our work.

2.1 Existing tools

What follows is an analysis of the existing tools that allow the usage of the C#

language for script development. First, we analyze what features are of interestto us and why with respect to the scenario described in the introduction. Then,each available tool is briefly described and its relevant features, advantages, anddisadvantages are stated. Finally, a conclusion based on our observation is made.

2.1.1 Features of interest

In this section we analyze the features that are of interest to us in the analysis ofexisting tools and explain why each feature is important and how we believe eachproblem should be ideally solved with respect to the aim of our work as statedin the introduction.

Ease of use

As we stated in the introduction, we are looking for a more convenient approach towriting scripts than using classic large IDEs. Therefore, we are interested in howeffectively and easily the given tool can be used for script development. An idealtool should be small, quick, easy to use, and should not strain system resourcestoo much. We also want to be able to use it quickly in other environments thanthe development environment, so an ideal tool should not require any specialinstallation and its license should allow users to deploy it to whatever machinethey need.

Compatibility with the standard C# language

When users are creating small ad hoc utilities, a common scenario is that someof these utilities need to be further evolved into programs that are no longersimple scripts. In this case, users want to be able to easily transform scripts intoprojects of classic IDEs. Therefore, we are interested if the scripts written in thegiven tool can be easily used as standard compatible C# code when inserted intoa project of classic IDEs, requiring only minimal changes (such as adding correctreferences). In an ideal tool the scripts should be compatible with the standardC# language.

Means of referencing external assemblies

The list of referenced assemblies is an information that is essential for successfulcompilation of any C# source code, but it is not part of the C# source code files

7

themselves. So the tools that we will be analyzing need to offer a way to pass thisinformation to the compiler. Because we are looking for an easy-to-use tool, wewant to avoid having to use the command line, and because the source code ofa script will usually fit into a single file, we want the script files to be self-contained(without any additional information needed to use them). Therefore, in an idealtool the additional references should be part of the source code itself, so thatthe script file is fully stand-alone (this should be ideally done in accordance withthe previous point, so that the compatibility with the standard C# language ispreserved).

Presence of a specialized C# source code editor

We are looking for an easy-to-use tool, so we want script development to becomfortable to users who are used to classic IDEs. Therefore, we are interestedif the given tool includes a source code editor with features that make codeauthoring easier, such as syntax highlighting, error highlighting, code completion,method parameter hints, basic refactoring, or quick code navigation. An idealtool should include an advanced C# source code editor with a rich set of featuresthat make code authoring easier.

Debugging options

Having a debugger available is an advantage because it allows users to quicklycheck if the program behaves correctly or to find out why it does not. Therefore,we are interested if the given tool allows debugging of the authored scripts in a waycommonly known from classic IDEs. An ideal tool should include a debugger thatsupports basic debugging options in a way commonly known from classic IDEs,which includes at minimum:

• An ability to pause the running program (either manually, or at a breakpointplaced into the source code, or when an exception is thrown).

• An ability to step into/over/out of called methods.

• An ability to inspect the state of the program (call stack, position withinthe source code, values of variables).

If the given tool does not contain dedicated debugging options, the scripts can beusually debugged by attaching a debugger from a classic IDE to the process run-ning the script, provided that correct symbol information was generated duringthe compilation of the script. However, this is not a very user-friendly solutionand it inherits the disadvantages of using a classic IDE for script development.

Support for code reuse

When scripts grow more numerous or complicated, it can happen that users needto share common code between them. In such a case it is useful to have an optionavailable to reuse a piece of code between scripts, even though using multiplesource code files is on the edge of our definition of scripts (it is usually better touse classic IDEs for more complicated programs that need multiple source code

8

files). Therefore, we are interested if the given tool allows some form of codereuse, i.e. using code from one script in another script. An ideal tool shouldsupport code reuse, for example by allowing compilation of multiple source codefiles into a single script, or by allowing one script to reference another. How thiscan be done technically is a similar question to “means of referencing externalassemblies” discussed above.

Support for executable generation

When users need to execute a script on a remote machine, during batch process-ing, or using a task scheduler, it is more convenient if the script exists as a stand-alone executable file that is independent of the tool that was used to create it.Therefore, we are interested if the given tool allows generating an executable filefrom the script, so that it could be distributed and executed independently ofthe tool itself. An ideal tool should allow some form of independent executablegeneration.

2.1.2 Available tools

If we want to use the C# language to quickly create small temporary programs,there are some tools available already. The most known tool is:

• LINQPad [15]

By searching the Internet for tools that support scripting in C#, we were able tofind:

• NScript [12]

• CSScript [13]

• ScriptCS [14]

We will also include two other tools that stand on the opposite ends of thespectrum of available tools. Even though these tools are not meant to be usedspecifically for scripting, they are universal and we will assess them as well forcompleteness:

• The C# Compiler

• A classic IDE

The last tool included in our comparison is a simple prototype of a specializedC# script editor:

• ScriptDevelop [11]

In this section we briefly describe each tool and state its relevant features, ad-vantages, and disadvantages with respect to the features of interest discussed inthe previous section.

9

The C# Compiler

The most primitive and bare-bones tool for compiling individual C# source filesis the direct invocation of the C# compiler, which is a part of the .NET runtimeenvironment. It allows building any application and running it. The obvious dis-advantage is the difficulty and inconvenience of such an approach – it is necessaryto know and enter the compiler options manually (for details see Command-lineBuilding With csc.exe [18]). While this task can be made easier by using somekind of batch processing or build automation, such a solution is cumbersome andlacks flexibility.

Advantages

✓ Available as a part of the .NET runtime environment.

Disadvantages

✗ Difficult, cumbersome and inconvenient to use.

✗ External tools are needed for both code authoring and debugging.

A classic IDE

On the other end of the spectrum of available tools are classic large IDEs, suchas Visual Studio [2], SharpDevelop [3] or MonoDevelop [4]. These IDEs areoptimized for large projects, but they can be of course used to develop smalltemporary programs as well, and they offer a full-featured editor, build systemand debugger. We have already stated the disadvantages of using a full-featuredIDE in the first chapter, so here we will only summarize them for completeness.

Advantages

✓ Full-featured source code editor, build system and debugger.

Disadvantages

✗ It is usually necessary to install the IDE (the installation of Visual Studiois especially time-consuming)

✗ The IDE is a huge application with a lot of features not necessary for scriptdevelopment, and as such its system resource requirements are high.

✗ It is necessary to start a new instance of the IDE for each script.1

✗ The IDEs do not support compilation and running of single source codefiles; it is necessary to include them into projects and solutions.

1While it is possible to share a single instance and a single solution between multiple scriptprojects, this in practice does not help much, complicates script management, and adds theinconvenience of having to switch between startup projects.

10

NScript

NScript [12] is arguably the oldest specialized tool that allows using the C#

language for simple scripting. It is a small tool that compiles and runs a singlestand-alone C# source code file either as a console application, or as a windowedapplication. It is possible to reference additional assemblies during compilation;these have to be listed in a separate accompanying file with the same name asthe compiled script but with a specialized extension.

Advantages

✓ Automatic compilation and running of single-file scripts without the needto invoke the compiler manually.

Disadvantages

✗ Another file is required to list additional references.


CS-Script

CS-Script [13] is another specialized tool that allows using the C# language forscripting. It is far more advanced than NScript, and can be used not only tocompile and run scripts, but also as a runtime library for dynamically loaded C#

code, treating the C# language as an embeddable scripting language for extensionof applications by allowing them to run any user-supplied code. It even offerssome interesting features in this respect, such as duck-typing (a class loaded froma script can be mapped on a new instance of the host-supplied interface withoutactually having to implement it formally in the code).

Another interesting feature is automatic guessing of referenced assembliesbased on namespaces imported with the using statement. This brings the ad-vantage of user-friendliness, but can also cause problems in case of conflicts orinability to guess the assembly correctly. Because of this, it is also possible tostate additional references either on the command line, or directly in the code inform of annotations enclosed in comments.

CS-Script compensates the absence of an editor or a debugger by offeringa special plugin for Visual Studio. This plugin allows using Visual Studio to editand run CS-Script scripts, but this approach also inherits most of the downsidesof using classic IDEs, except the ability to run single C# files. For debugging,CS-Script offers a tool that can convert scripts into projects and solutions thatcan be opened and debugged in Visual Studio.

Advantages

✓ Automatic compilation and running of single-file scripts.

✓ Reference guessing.

✓ Contains tools that make it easier to use Visual Studio as an editor anddebugger.

✓ Supports code reuse.

11

Disadvantages


ScriptCS

ScriptCS [14] is, much like CS-Script, another specialized tool that allows usingthe C# language for scripting. Unlike CS-Script it uses a different approach to theC# language itself, called REPL (Read-Evaluate-Print Loop). It allows process-ing of the C# language in a command-line-like manner, immediately evaluatingthe input statements and printing out the results, giving the language a morescript-like look and feel. Because of this, it does not use the stock C# compiler,but rather makes use of Microsoft Roslyn [5], an upcoming compiler-as-a-servicelibrary from Microsoft.2

For specifying additional references, ScriptCS uses special directives in thesource code that are not enclosed in comments. This, in addition to the REPLapproach to the C# language, makes script files written for ScriptCS incompat-ible with the standard C# language. However, the language is compatible withMicrosoft Roslyn.

For code authoring and debugging, Visual Studio with Roslyn CTP [5] canbe used.

Advantages


✓ The REPL approach gives the source code more script-like look and feel.

✓ Supports code reuse.

Disadvantages

✗ Incompatible C# code.

✗ Visual Studio (with Roslyn CTP) has to be used as an external source codeeditor and debugger.

LINQPad

LINQPad [15] is a popular tool which can, to a certain extent, replace applicationslike SQL Management Studio [19] which are used for configuring, managing,administering, developing and querying SQL databases. It is primarily focusedon querying databases using the C# language and LINQ, and allows writingand executing expressions, statements, methods and classes in the context ofa database connection.

Because of the presence of a code editor and some other features (e.g. the uni-versal “dump” command that allows complex output of any object into a specialwindow), LINQPad is often used as a small light-weight development environmentfor simple scripts.

2Microsoft Roslyn is further discussed in section 3.1.4.

12

One of the disadvantages of LINQPad is that it uses custom XML formatfor saving scripts. This format allows storing some additional information aboutthe script, such as its type (if it is an expression, statement, etc.), importednamespaces, and referenced assemblies.

Advantages

✓ Small and easy to use tool.

✓ Allows quick running of single expressions, statements, methods, or wholescripts.

✓ A full-featured C# editor is present (although most of the important featuresare available only in the paid version).

Disadvantages

✗ Incompatible C# code (XML script format, single expressions or state-ments).

✗ No executable generation (the scripts are executed in a special hosting en-vironment and cannot be used outside LINQPad).

✗ No support for code reuse.

✗ Most of the important code-editing functions are available only in the paidversion.

✗ External tools are needed for debugging.

ScriptDevelop

ScriptDevelop [11] is a simple prototype of a specialized C# script editor that wascreated as a bachelor thesis on Czech Technical University (CVUT). It is small,easy to use and supports editing, compiling and running of stand-alone single-fileconsole applications in C#.

While the editor supports simple syntax highlighting and code completion, itdoes not contain any further code-aware functions like method parameter hints,reference completion, or code navigation. The code completion which is supportedis very limited – it offers types and members only from a limited hard-coded setof assemblies. There is also no support for debugging or any form of code reuse(each source code file is strictly stand-alone).3

Advantages

✓ Small and easy to use tool.


3We will further analyze ScriptDevelop in section 3.2.

13

Disadvantages

✗ No support for code reuse.

✗ No support for debugging.

✗ The editor contains no code-aware functions besides the limited code com-pletion.

2.1.3 Summary

Table 2.1 summarizes the features of the existing tools. As can be seen from thealready available solutions, using the C# language as a scripting tool is a desiredfunctionality. During the lifetime of the C# language a variety of attempts hasbeen made to solve this problem, because using the command-line compiler ora full-blown IDE has many downsides that the specialized tools try to address.

Easyto

installanduse

Com

patiblewithstandard

C#

Referencesinthesource

file

Advancedsource

code

editor

Debugging

optionsavailable

Codereusesupported

Executablegeneration

Free

ofcharge

The C# Compiler ✓ ✓ ✓ ✓

A classic IDE ✓ ✓ ✓ ✓ ✓ 4

NScript ✓ ✓

CS-Script ✓ ✓ ✓ ✓ ✓

ScriptCS ✓ ✓ ✓ ✓

LINQPad ✓ ✓ ✓ 5

ScriptDevelop ✓ ✓ ✓ ✓ ✓

Table 2.1: Summary of the features of the existing tools.

None of the solutions we evaluated fully satisfy our requirements for an idealtool for developing C# scripts. If we are interested solely in compilation andrunning of stand-alone C# source files, the most interesting and advanced toolseems to be CS-Script, which even contains tools that allow using Visual Studioto author and debug scripts. However, the popularity of LINQPad for solvingsmall tasks unrelated to databases (which are the primary focus of LINQPad) isalso a sign that there is a desire for a small, quick and easy to use editor for C#

scripts as an alternative to classic large IDEs. Developing such an editor is theprimary goal of this work.

4The most common C# IDE, Visual Studio, is free of charge only for limited purposes in itsExpress edition. SharpDevelop and MonoDevelop are free or charge and open-source.

5Most of the important code-editing functions in LINQPad are available only in the paidversion.

14

2.2 Requirements

In the previous section, we have examined the existing tools that allow using theC# language for script development. We have studied their features and identifiedtheir advantages and disadvantages. In this section we look more closely at someof the points of interest stated in section 2.1.1, and based on our observations ofthe existing tools we formulate exact requirements for our application.

2.2.1 Non-functional requirements

We have stated several times that we aim for a “small, quick and easy to use”tool. Here we specify what exactly it means for our application in form of non-functional requirements.

Familiarity

Because the target user audience of our application is developers who routinelyuse classic IDEs, special effort will be made so that they would find the functionsthey want to use quickly and effortlessly. This includes menu structure, graphicalsymbols, command names, default keyboard shortcuts, etc.

• R1.1: When designing the user interface, we will attempt to make theapplication behave in an expected way for developers that are used to classicIDEs.

Installation

• R1.2: The application will not require any special installation. It will bedeployable simply by copying it, along with all of the files in its programdirectory, to any machine that has a Microsoft .NET Framework [16] of thecorrect version installed.

Startup

• R1.3.1: The application’s startup process will not add any extra overheadon top of what is expected of a typical .NET application. Any additionalstartup processing will be done in background once the user interface of theapplication is loaded and presented to the user.

To help better manage system resources usage in an environment where systemresources are limited:

• R1.3.2: The application will allow nonessential components to be loadedlater, after the core of the application is presented to the user and ready.Loading of nonessential components will be optional and potentially ableto be turned off. Unloading of already loaded components is not required.

15

Stability

The application will be used to perform creative work, and so it should be pro-tected from work loss in case of a malfunction. While malfunctions should ideallynot happen, such a scenario must be taken into account. There are two main stepsthat will be taken to protect the application:

• R1.4.1: If possible, a potential unexpected exception will be caught, loggedand if necessary presented to the user without terminating the entire app-lication.

• R1.4.2: The application will keep backup copies of any unsaved documents,and if it is terminated unexpectedly, it will offer to restore these backupsat next launch.

Reentrance

To better support organization of windows and documents:

• R1.5: The application will be able to run in multiple instances. Theseinstances will not interfere with each other.

Localization

Localization support is not needed because the target user audience are developerswho routinely use classic IDEs, where English is common.

• R1.6: The user interface of the application will be in English.

Extensibility

It is not expected that the application will be a complete, all-inclusive IDE, butrather a solid and well-usable basis that could be further extended and improvedby other students and developers based on their specific requirements, ideas andneeds.

• R1.7: The application will be designed in such a way so that it could beeasily extended by other developers.

2.2.2 Editor

The application will contain an advanced C# source code editor with a rich setof features that make code authoring easier. In this section we discuss what thismeans in greater detail and decide what features are important for us to include.

Syntax highlighting

Syntax highlighting is an important feature of any source code editor becauseit makes the displayed source code easier to read and understand. We want toinclude advanced syntax highlighting that is not based only on simple regularexpressions, but also on the real output from the syntactic and semantic analysis

16

of the C# source code. This way we can distinguish between identifier roles thatwe could not tell apart using only regular expressions, such as between type andnamespace names, or we can correctly highlight contextual keywords that are notreserved in the C# language, but recognized as keywords only in certain syntacticconstructs (e.g. keywords specific to LINQ expressions).

• R2.1.1: The editor will support syntax highlighting.

• R2.1.2: The syntax highlighting will use syntactic and semantic analysisof the C# source code where appropriate.

Code completion

While the user types identifier names, code completion offers a list of availabletypes, members, variables and other constructs. It allows to choose an identifierwithout the need to remember its exact name or without having to type out itsname in full.

• R2.2.1: Code completion will appear automatically when the user typesan identifier, but only in places where it is expected so that it would notinterfere when we want for example to name a new type, member, or vari-able.

• R2.2.2: In places where it is syntactically possible both to introduce a newidentifier or to use an existing one (such as when passing a lambda delegateas a real parameter), the code completion will appear in a different modeso that if the user introduces a new identifier, the code completion will notinterfere.

Because users rarely remember the exact namespace or assembly of a library typeby heart, these requirements are essential to allow quick and easy code authoring:

• R2.2.3: In addition to all entities that can be used at the current posi-tion, the code completion will also offer types and extension methods fromnamespaces that are not currently imported with a using statement andimport the namespaces accordingly when such an item is selected.

• R2.2.4: Code completion will also offer types and extension methods fromyet unreferenced but otherwise known (or precached) assemblies, and auto-matically reference the correct assembly upon selection.

• R2.2.5: Code completion will display the relevant XML documentation ofthe entities where available.

Parameter hints

Parameter hint is a pop-up window that appears when the user writes a methodinvocation. It displays the list of invoked method’s overloads, the list of param-eters for the currently selected overload, and the current parameter. It allowsto write a method call without the need to remember its exact overloads andparameters by heart.

17

• R2.3.1: The editor will support the parameter hint pop-up window.

• R2.3.2: The relevant XML documentation of the parameters will be dis-played in the parameter hint pop-up window where available.

Refactoring

Most of the refactoring transformations offered by classic IDEs and other ad-vanced tools are well suited for large projects where there is a significant benefitfrom using them to make the code cleaner, more organized, and easier to read.Most of these transformations are not very important in our scenario where weprimarily focus on single-file scripts. There are, however, some refactoring trans-formations that are important enough so that we will implement them as well:

• R2.4.1: Renaming of symbols (types, members, variables, namespaces).

• R2.4.2: Automatic assembly referencing and namespace importing duringcode completion when a type or an extension method from yet unimportednamespace or yet unreferenced assembly is selected.

• R2.4.3: Automatic assembly referencing and namespace importing whenthere is a type or an extension method already used in the source codewhich cannot be resolved, but is located in another namespace or assembly.In this case, a context action that will add the corresponding namespace orreference will be offered.

Navigation

• R2.5.1: The application will support the “Go To Definition” commandknown from classic IDEs. This command will navigate the user to thedefinition of the symbol at the cursor’s position in the source code.

• R2.5.2: The application will support the “Find References” commandknown from classic IDEs. This command will highlight or list all occur-rences of the symbol at the cursor’s position in the source code.

• R2.5.3: There will be a way to quickly navigate between types and mem-bers of the current source code file.

More advanced navigation commands are not necessary in our scenario.

Tooltips

• R2.6.1: When the mouse cursor is moved over a type, member, or variablereference in the source code, the application will display a tooltip with thedefinition of the symbol.

• R2.6.2: The summary of the symbol’s XML documentation will be dis-played in the tooltip where available.

18

2.2.3 Compilation

• R3.1: The application will be able to compile and run programs consistingof a single C# source code file. There will be no need to create and manageprojects or solutions as known from classic IDEs. Each source code file willbe self-contained and there will be no additional information required tocompile it.

Language compatibility

• R3.2.1: The source code written in the application will be compatible withthe standard C# language.

• R3.2.2: When inserted into a project of classic IDEs, the source codewritten in the application will require only changes to the configuration ofthe project itself (such as adding correct references).

Compilation instructions

A single C# source code file cannot be compiled on its own. The C# compilerneeds additional information, at least the list of referenced assemblies. Thisinformation is essential for successful compilation of C# source code, but it is notpart of the C# source code files themselves. In classic IDEs, this information isassociated with the project and stored in the project file. Since our applicationwill not have project files, we need another way to pass this information to thecompiler.

We want the script files for our application to be self-contained, so there mustnot be any additional information required to use them (see R3.1). Becauseof this, the additional compilation instructions will be part of the source codeitself. Since we do not want to introduce any custom syntax that would breakthe compatibility with the C# language (see R3.2), the additional compilationinstructions (e.g. external references) will be enclosed in C# comment blocks.

• R3.3.1: Any additional compilation instructions will be part of the script’ssource code itself.

• R3.3.2: Any additional compilation instructions will be enclosed in C#

comment blocks.

Code reuse

Although code reuse support is not an obvious requirement for an IDE focusedon single-file scripts, it is useful in case the scripts grow more numerous or com-plicated and we need to share common code between them. The application willsupport two mechanisms of code reuse:

• R3.4.1: One script will be able to include another. From the compiler’sperspective, this will behave as if there were multiple source code files con-tributing to a single output assembly. If needed, this can be used to emulatea project consisting of multiple source code files.

19

• R3.4.2: One script will be able to reference another. From the compiler’sperspective, this means that the referenced script will be compiled first andthe resulting assembly will be used as a reference of the referencing script.If needed, this can be used to emulate a solution consisting of multipleprojects.

These mechanisms will be implemented as compilation instructions the same wayas described in the previous section (see R3.3).

Executable generation

Having an independent executable is more convenient when we need to executethe script on a remote machine, during batch processing, or using a task scheduler.

• R3.5: The application will support generating an executable file from thescript, so the executable file could be distributed and executed indepen-dently of the application itself.

2.2.4 Debugging

The application will offer basic debugging options so that the users could inspectthe behavior of their scripts. Based on options available in classic IDEs, thismeans:

• R4.1: Manual pause and resume of the execution.

• R4.2: Ability to set a breakpoint on a line.

• R4.3: Pause when an exception is thrown.

• R4.4: Highlighted position within the source code.

• R4.5: Stepping (step into, step out, step over).

• R4.6: Call stack and thread list views.

• R4.7: Debugger console (a command line for executing statements andqueries).

• R4.8: Tooltips with values of variables when hovering the mouse cursorover the variables in the source code.

Complex visual object inspector is not needed; querying the current state (e.g.in the debugger console or tooltips) will use simple string representation to showthe value. More complex objects can be queried using the debugger console.

20

3. Implementation analysis

In the previous chapter we have analyzed the existing tools that allow the usagethe C# language for script development and formulated detailed requirements forour own tool. This chapter is focused on technical decisions regarding how ourown tool will be implemented.

3.1 Tools and libraries

In this section we choose what tools, libraries and frameworks will be used forthe development of our application. In general, we will prefer libraries that arenot “black boxes”, but instead:

1) We can look into their source code if the available documentation is not suf-ficient for us.

2) We can modify them if the library does not behave exactly as we need in someaspect or if it does not expose the exact information we want.

This usually means libraries written in C# with source code available under a per-missive license. However, this is only a preference, not a strict requirement.

3.1.1 Basis

As can be seen from the requirements specified in section 2.2, we are aiming tocreate a special kind of development environment for the C# language. The taskof creating a development environment is a very demanding one, especially whenwe want to include features such as an advanced source code editor or a debugger.Creating such an application from scratch would be a very large and complicatedendeavor. To make the project manageable, we can approach it from two differentdirections:

a) Create an add-on for an existing classic IDE or use an open-source classic IDEwith a permissive license and modify it to suit our goals.

b) Build our application from the ground up, but make heavy use of the existingcomponents and libraries, many of which were specially designed to be usedin IDE development.

Existing classic IDEs are large and complicated; as we stated in the introduction,we want our application to be a light-weight alternative to them. Because ofthis, we will use the second approach, which also gives us better control over ourapplication’s architecture.

3.1.2 Programming environment

Because we are developing a tool for the C# language and the C# languageis the language of choice of the author, we will use the C# language for thedevelopment. The target platform will be Microsoft .NET Framework 4.0 andwe will use Microsoft Visual Studio 2010, both being the latest versions when thework on this project started.

21

3.1.3 User interface

Microsoft .NET Framework offers two different sets of APIs for user inter-face development – Windows Forms [20] and Windows Presentation Foundation(WPF) [21].

WPF is more modern, offers greater flexibility and has many features and abil-ities that Windows Forms lacks, for example data binding and templates, relativelayout, greater customizability, or hardware acceleration. It is well establishedand widely supported by third-party controls and libraries. Because the authorof this work would like to gain more experience with WPF, we will use it in ourapplication.

Source code editor

A source code editor is an integral part of any IDE. It is the component the userinteracts with the most. Because we want to be able to highlight various portionsof the source code, it must support custom rendering.

Two most known and used libraries were considered – AvalonEdit [22] andScintillaNET [23]. Both support a wide range of additional features that areuseful for source code editors.

AvalonEdit is written in C# and WPF, whereas ScintillaNET is a managedWindows Forms wrapper around an unmanaged Scintilla [24] control. Because ofthis, we chose AvalonEdit. An overview of the AvalonEdit library can be foundin Using AvalonEdit [8].

Docking manager

The user interface of an IDE is usually quite complex and must support workingwith not only many open documents at once, but also windows and panels withvarious additional information (e.g. a filesystem explorer, an error list window,or a call stack panel).

A common approach of how to present this environment to the user is calleda docking manager. It is a high-level layer of an application’s user interface whichmanages windows with open documents and panels. It creates a tabbed interfacefor the open documents and allows the user to move the windows around, snapthem to the edges, combine them, split them, or hide them.

This docking behavior is an expected part of modern IDEs and we will alsouse it. It is not a standard part of the WPF framework, but since the dockingbehavior is quite common in applications with more complicated user interfaces,there are many available libraries that implement it. Most of the implementationsare commercial, but because we prefer open source, C# and we will use WPF, wechose AvalonDock [25] for this purpose.

3.1.4 Syntactic and semantic analysis

One of the most important goals when creating a C# code editor is the abilityto analyze and understand the source code that is being written. This ability iscalled syntactic and semantic analysis. There are various features of a source codeeditor that need source code understanding to work properly – for example code

22

completion, method call parameter hinting, navigation, or syntax highlighting(basically all the R2 requirements from section 2.2.2). We need not only a C#

parser which would provide us with the syntax tree of the code, but also a meansof semantically analyzing the code so that we would be able to resolve themeaningof the parsed code (e.g. know exactly what types, methods and variables are beingused in the code).

The syntactic and semantic analysis of C# source code is a very large anddifficult problem which requires complete understanding of the specification ofthe C# front-end. Therefore, we will look for a library that would help us withthis problem.

There are currently two libraries that can be used for this purpose –NRefactory and Microsoft Roslyn:

• NRefactory [6] is a C# library that allows parsing and semantic analysisof C# source code. It also contains support for refactoring, formatting,code issue analysis and code completion. NRefactory has been created asa part of SharpDevelop [3] specifically for the purpose of supporting itscode-aware functions, so it is well suited to be used in IDE development. Italso contains a large library of ready-made C# refactoring transformationsand code issue detectors which can be easily interfaced with. Its currentversion 5, which has been a complete rewrite, is stable.

• Microsoft Roslyn [5] is an open-source compiler and code analysis libraryfor C# and Visual Basic .NET languages. It allows syntactic and semanticanalysis of C# and Visual Basic source code, dynamic compilation to IL,and refactoring. It is currently in a beta stage and available as a previewversion.

Both libraries are capable of syntactic and semantic analysis of C# code,however, NRefactory is better suited to be used in an IDE development becauseit also supports code formatting and completion and contains various refactoringactions and code issue detectors. Roslyn has some extra features, such as ILgeneration and Visual Basic support, but these are not important with respectto our goals. Moreover, NRefactory is open source (Roslyn had not been open-sourced yet when work on this project began) and stable. Roslyn is still underdevelopment and only preview versions have been released so far.

Based on these arguments, we will use NRefactory in its current version 5 forour project. The library is introduced in depth in Using NRefactory for analyzingC# code [7].

3.1.5 Debugging

One of the features that we will need to implement is the support for debuggingthe code written in our application (requirements R4 from section 2.2.2). A run-ning CLR application can be debugged using a rather complicated ICorDebug

API, which is a set of public COM interfaces. However, the API is very largeand very little documented (some documentation can be found in Debugging In-terfaces [26]).

Luckily, there is an open source C# library built on top of this API. It is a partof SharpDevelop and called Debugger.Core. It shields us from the complexity of

23

the ICorDebug API and provides a set of classes that we can use to implementdebugging functionality. We will use this library in our application to implementthe debugger.

The overview of the library can be found in Internals of SharpDevelop’s De-bugger [9]. The library is also used to implement debugger visualizers in DebuggerVisualizers for the SharpDevelop IDE [10], which also describes how to work withthe library.

3.1.6 Extensibility

One of the goals of our application is to make it extensible (R1.7). Because ofthis the application will be implemented as a set of many plugins that use oneanother’s services. A plugin in this context will be a separate unit of functionalitythat extends the application or one of its parts in some way. Therefore, we willneed a means of discovering, loading, and initializing these plugins at runtime. Weare not looking for any inversion of control or dependency injection mechanisms,instead we need a naming and activation service. The required functionality isthis:

• Discover and load all assemblies with plugins in the application’s directory.

• In these assemblies, find all classes that implement a specific interface.

• Create instances of these classes and return them on demand.

There are many tools, libraries and frameworks that can be used to per-form these tasks. Microsoft .NET Framework itself contains two frameworks thatsupport extensibility – Managed Add-in Framework (MAF) [27] and ManagedExtensibility Framework (MEF) [28]. MAF does not suit our needs very well be-cause it is intended for large systems where the host application and add-ins arestrictly separated, run in different application domains and communicate acrossthese domains. MEF, on the other hand, is light-weight and covers the featureswe need, so we will use it in our application.

3.2 Previous work

ScriptDevelop [11] is one of the tools we described in section 2.1.2. It is a simpleprototype of a specialized C# script editor that was created as a bachelor thesis onCzech Technical University (CVUT). It supports editing, compiling and runningof stand-alone single-file console applications in C#. The editor supports simplesyntax highlighting and limited code completion.

The goals of ScriptDevelop are a subset of our own goals and it uses some ofthe libraries that we chose to use (WPF, AvalonEdit, AvalonDock), so we willconsider it as a starting point that we can rewrite and extend to meet our goals.

There are many ways in which ScriptDevelop can be improved to meet ourrequirements as stated in section 2.2:

• It contains only simple regular-expression-based syntax highlighting. Wewant to use syntax highlighting which takes into account the real output

24

from the syntactic and semantic analysis of the C# source code (R2.1.2).This way we can distinguish between identifier roles that we could not tellapart using only regular expressions, or we can correctly highlight contex-tual keywords that are not reserved in the C# language but recognized askeywords only in certain syntactic constructs.

• It contains only very limited code completion:

– Code completion is displayed only upon manual invocation. We wantcode completion to appear automatically when the user types an iden-tifier (R2.2.1). This is not as straightforward as it seems, because wecannot simply display it every time an identifier is typed. Code com-pletion must not interfere in places where a new identifier is expected,and must also support cases where it is syntactically possible both tointroduce a new identifier or to use an existing one (R2.2.2).

– Code completion offers types and members only from a limited hard-coded set of assemblies. We want code completion to automaticallyinclude all referenced assemblies.

– Code completion does not offer items from unimported namespacesand unreferenced assemblies. We want code completion to supporta mode where such items are offered and correct namespaces and ref-erences are added upon their selection (R2.2.3, R2.2.4).

– Code completion is based on older SharpDevelop components anda previous version of NRefactory. We want to use more powerfulNRefactory 5, which is a complete rewrite and has a different interface.

• It contains no debugging functionality. We want to offer basic debuggingoptions so that the users can inspect the behavior of their scripts (R4).

• It contains no code-aware functions besides code completion. We wantto offer many code-aware functions, including parameter hints, tooltips,refactoring, and navigation (R2).

• It does not allow code reuse. We want to support code reuse by allowingscripts to include or reference one another (R3.4).

We will use ScriptDevelop as a prototype starting point for our own application,but we will have to rewrite it significantly in order to meet the requirements forour application. The most important tasks that we will need to do include:

• Implement on-the-fly parsing of the source code using NRefactory 5.

• Add support for code reuse and make sure it cooperates correctly with theon-the-fly parsing from the previous point.

• Implement full code completion system according to our requirements.

• Implement other missing code-aware functions (proper syntax highlighting,parameter hints, tooltips, navigation, refactoring).

• Implement a debugger.

25

3.3 Application architecture

An integrated development environment is a complex application to build. Tosimplify its development, we can separate it into several primary components,each of which will be focused on a different functional aspect of the application.A primary component, in this sense, will be a large portion of the application,containing many classes or even assemblies. There will be these primary compo-nents:

• Application Core will contain the main entry point and infrastructure ofthe application.

• UI Core will implement the main window and basic user interface of theapplication.

• Editor will provide the concept of documents, their formats and editors.

• Compiler will contain the basic C#-related functionality that does not re-quire code-aware insight into the source code, including commands to buildand run the scripts.

• NRefactory will include all the code-aware features that require on-the-flysyntactic and semantic analysis of the C# source code.

• Debugger will implement the basic debugging functionality.

3.3.1 Plugin infrastructure

One of our requirements is to make the application extensible (R1.7). Becauseof this, we will implement the application as a set of many plugins that will useone another’s services. This will also simplify the development of the applica-tion, allowing us to gradually extend it by adding new plugins. The core of theapplication will contain only the basic infrastructure for application startup andplugin management. All other functionality will be implemented as plugins.

Plugin discovery

In order to simplify adding new functionality into the application and reduceboilerplate code, plugins will be initialized automatically. All that will be requiredis to include them into the application’s code and make sure they implementa specific interface. These interfaces will have MEF’s InheritedExport attributewhich will ensure that they will be discovered, or there will be a special Exportattribute if they also need additional metadata.

Plugin initialization

A common scenario is that a plugin will require some kind of initialization. Thisinitialization could happen in the plugin’s constructor, but plugin instances willbe created by MEF, and we will need finer control over when and where theinitialization takes place. Therefore, we will introduce a common base interfacefor plugins.

26

[InheritedExport]

public interface IPluginBase

{

void Load();

}

Any plugin that implements this interface will have its Load method called whenit is loaded into the application. Using the Load method for plugin initializationis preferred to its constructor for several reasons, especially if the plugin needs tointeract with other parts of the application:

• The Load method will be called after the plugin instance has been fullycomposed by MEF.

• The Load method will be guaranteed to run on the UI thread, so its imple-mentation can safely access UI elements.

• The Load method will help us better manage plugin initialization depen-dency.

Plugin initialization dependency

Plugins of our application will often depend on each other during initialization,which means that one plugin at some point inside the Load method will requirethat another plugin has already been loaded and is ready to be used, i.e. hadits Load method already called. This may not generally be the case because theorder in which the plugins are initialized is arbitrary. There are several ways thiscould be solved:

• Use dependency injection through constructor parameters. The instancespassed as constructor parameters must already have their own constructorscalled, so we can be sure any initialization done in their constructors hasalready happened. However, as we stated above, we prefer to use the Load

method for plugin initialization, so this is not a very useful solution.

• Have all public methods and properties of the plugin first check whetherthe plugin has been initialized (i.e. its Load method has been called) andif not, call it first. However, this approach introduces a lot of boilerplatecode into individual plugins and introduces thread-safety concerns.

• Have the plugin infrastructure itself keep track of which plugins had theirLoad method called and which had not. The infrastructure would then callthe Load method appropriately.

We will use the last approach. The plugin infrastructure will expose a methodsuch as this:

public static void EnsurePluginLoaded(IPluginBase plugin)

27

This method will ensure that the plugin instance passed as an argument hasbeen loaded. If it has not, it will be loaded synchronously before continuing. Themethod will also be implemented in such a way that it is thread-safe and executesthe Load method on the designated thread.

The EnsurePluginLoaded method could be then called either by the callingplugin, or the called plugin itself, as the first call of its public methods andproperties that require prior initialization (this still introduces boilerplate code,but not so much, and it also solves thread-safety problems).

Plugin initialization dependency cycles

The plugin initialization dependency resolution that will be done by the infras-tructure’s EnsurePluginLoaded method will need to correctly detect and preventdependency cycles. Such cycles can happen for example if two plugins in theirLoad methods depend on each other already having been initialized. Withoutcycle detection, the execution of Load methods would end up in an infinite loopand eventually overflow the call stack. Instead, we will have to detect this andthrow an exception with detailed description of the detected cycle so that wecould better debug these situations.

We will do this in the EnsurePluginLoaded method by remembering whichplugins are currently having their Load method called and detecting if it is enteredtwice for the same plugin. This will work trivially because the Load method willalways be called on the same thread.

Singleton plugins

The most common type of plugin in the application will be a singleton plugin,i.e. a plugin that exists as a single shared instance. MEF supports two ways ofaccessing the single instance of such plugins from other plugins:

• Injecting them into a property or a field using the Import attribute.

• Passing them as a parameter into the constructor.

This introduces a lot of boilerplate code and works only for objects that arethemselves being composed by MEF. In order to simplify the process of import-ing singleton plugins, allow them to be easily accessed from any class, and tomake sure they are only accessed after they had been properly initialized, we willintroduce a special generic base class with the following interface:

public abstract class SingletonPlugin<T> : IPluginBase

{

public static T Instance { get; }

}

This class will allow accessing the singleton instance of a plugin of type T. If the Tplugin implements IPluginBase, the Instance property will also ensure that theLoadmethod has run before returning the instance (using the EnsurePluginLoadedmethod described above). This way, accessing singleton plugins via their Instancestatic property ensures that the plugin instance we access has already been ini-tialized correctly. For example, if we define a plugin like this:

28

public class CustomPlugin : SingletonPlugin<CustomPlugin>

{

public void DoSomething();

}

We can then access this plugin from anywhere in the code by simply writing:

CustomPlugin.Instance.DoSomething();

Accessing a singleton plugin like this will ensure it has been properly initialized.

Plugin dependency

The requirement R1.3.2 states that we must allow nonessential components to beloaded later. Because of this, we will need plugins to be loadable incrementallyat an arbitrary point in time of the application’s life cycle. In order to do thiswith sensible granularity, we will allow loading of plugins one assembly at a time.However, this introduces a problem with plugin dependency.

Consider the situation shown in figure 3.1. There are two singleton pluginsA and B in separate assemblies. We have added plugin A into our MEF catalog,but not yet plugin B, on which plugin A depends. Since the assembly of pluginA references the assembly of plugin B, both assemblies have been loaded into theapplication, but only assembly of plugin A has been added to the MEF catalog.Now, if we try to access plugin B from A using the Instance syntax, we encounteran error, because MEF does not know about plugin B.

Figure 3.1: Example of the plugin dependency problem.

This could be solved when adding an assembly into the MEF catalog bygoing over its references and recursively adding them into the catalog as well.However, this approach would fill the catalog with a lot of assemblies withoutany plugins (framework assemblies, external libraries). Alternatively, we couldscan each reference for plugins and add only references with plugins into thecatalog, however such scanning would introduce additional overhead comparableto that of adding the assemblies into the catalog outright, since MEF internallyperforms a similar scan.

Instead, we will solve this simply by introducing a convention that only assem-blies with a certain name (e.g. Plugin.*.dll) can contain plugins. When addingan assembly into the MEF catalog we will still recursively scan all its references,but we will add a specific reference to the catalog only if its name conforms tothe convention.

29

3.3.2 Temporary storage

The application will make heavy use of temporary storage. We will need to useit for example to:

• Save new documents, so that they can be compiled and run without theneed to be named and saved first by the user.

• Save results of compilation (the .exe, .dll and .pdb files) so that scriptscan be compiled and run without cluttering the directory of their sourcecode.

• Save modified but yet unsaved documents, so that they can be restored incase the application is unexpectedly terminated.

There are three primary requirements on the temporary storage management ofour application:

1) When it is starting or exiting, the application must clean up the temporarystorage so that it would not remain cluttered with files that are no longerneeded.

2) Because the application must be able to run in multiple instances (require-ment R1.5), special effort needs to be made to make the temporary storagemanagement behave correctly with multiple application instances running con-currently in the same environment.

3) Because we will need to implement features like restoring lost work aftera crash (requirement R1.4.2), the temporary storage management must beable to access temporary storage of application instances that were termi-nated unexpectedly.

This problem cannot be properly solved by simply having the multiple instancesof the application use different locations for temporary storage. This is becausefor features like restoring lost work after a crash, the different instances have toaccess a common temporary storage location. On the other hand, if a commontemporary storage location were used and the user launched a second instanceof the application, this second instance would see the temporary storage of thefirst instance, which is still running, and incorrectly assume that it had beenterminated unexpectedly.

Solution

We will solve the problem using the following algorithm. There will be a commontemporary storage location. Each new instance of the application will:

1) Generate a unique identifier in the form of a random UUID.

2) Acquire a global mutex from the operating system with this UUID as a name.

3) Create a directory in the common temporary storage with this UUID asa name.

30

When the instance exits normally, it releases its global mutex and deletes itsdirectory. If the instance in terminated unexpectedly, the directory remains, butthe mutex is marked by the operating system as abandoned.

When a new instance of the application is launched, it performs a temporarystorage cleanup step during its initialization. This cleanup step is protected byanother (separate) global mutex to ensure that no two instances of the applicationperform it at the same time. During this initial cleanup step, the application looksinto the common temporary storage and if it sees any directory there, it checksthe global mutex with the same name as the directory. This mutex can be either:

a) Already acquired, which means that the instance the directory belongs to isstill running.

b) Free or abandoned, which means that the instance the directory belongs towas terminated unexpectedly and the directory is abandoned.

If we find that the directory is abandoned, we can examine it to see if it containsanything that should be taken care of (e.g. unsaved documents to be restored)and then delete it. If the directory is not abandoned, the associated applicationinstance is still running and we leave the directory intact.

3.3.3 Configuration and settings

The application will need to be configurable so that the user could customizesome of its aspects. There are a few basic points to consider when designing howthe application will be configured:

• Where the configuration will be stored and in what format.

• How the user will configure the application.

• How the configuration will be accessed in code.

User interaction

There are two opposite approaches to how the user can configure the application:

a) The application can provide detailed UI for configuration.

b) The application can provide human-readable configuration file that the usercan edit.

In order to simplify initial UI design of the application, we chose the secondapproach, i.e. to have minimal configuration-related UI and instead use a human-readable configuration file for customizing the application. This will allow us toadd the configuration UI later if needed. To make configuration a little moreuser-friendly, we will also:

• Allow the configuration file to be quickly opened within the applicationitself.

31

• Apply the new configuration the moment that the edited configuration fileis saved.

The configuration file will use the XML format because it is both human-readableand easily manipulable in code. The file’s default location will be in the samedirectory as the application’s executable. We will keep the default configurationfile embedded within the application itself and create a copy if the user decidesto modify it.

Configuration vs. settings

For the purpose of implementing configuration in code, we will use two differentterms:

• Configuration will allow the user to configure the behavior of various com-ponents of the application. It will be user-defined.

• Settings will allow various components of the application to persist theirstate between sessions. Users will not be able change this state directly.

For example, family and size of the editor’s font or mapping of keyboard shortcutsto commands will be read from configuration. On the other hand, settings willpersist information such as the window position and state (so that it can berestored next time), list of currently open documents (so that they can be openagain next time) or list of recently open documents available from the main menu.

Access in code

In order to make manipulation with configuration and settings simple when writ-ing the application, we will serialize and deserialize it transparently. Each config-uration section will have its own XML subtree in the configuration file and willbe implemented by a separate singleton plugin which will have its properties de-serialized automatically upon initialization. The plugin will also contain an eventthat will be triggered each time the configuration file has changed so that theapplication can reload the modified configuration.

3.4 Compilation

We have stated our requirements for the compilation of C# scripts in section 2.2.3.In this section we discuss how we will satisfy these requirements.

3.4.1 Header sections

Since we want the script files for our application to be self-contained (requirementR3.1), we need some way to pass additional information (the information thatis part of the project files in classic IDEs) to the C# compiler. In order not tointroduce any custom syntax that would break the compatibility with the C#

language (requirement R3.2), we have decided to put the additional compilationinstructions (e.g. external references) to special C# comment blocks. We will callthese special C# comment blocks with additional instructions header sections.

32

Header sections will be enclosed in block comments because block commentsare usually used less in the C# language. They will need to have some kind ofopening and closing tags, so that they could be distinguished from other com-ments. They will also need to have some kind of structure. Therefore, a headersection will look like this:

/*HEADER**

...

**HEADER*/

To simplify parsing and make the header sections stand out in the source code,the opening and closing tags will have to be the only non-whitespace content ontheir respective lines in order to be recognized as header sections.

The body of a header section will contain one or more header commands.Again, to simplify the structure of header sections, each header command willbe placed on a separate line and will have a fixed structure of a command wordfollowed by an optional argument. Empty lines or lines beginning with doubleslashes will be ignored. For example, a header section could look like this:

/*HEADER**

// Command without parameter

PAUSE

// Commands with parameter

OUT Hello world.exe

REF System.Core.dll

**HEADER*/

Available commands

We will need to offer header commands for most of the options that can bepassed to the C# compiler. This includes options such as referenced assemblies,embedded resources, output assembly name and type, etc. We will also offera command to pass any additional options to the compiler as a fallback in casesome required option is not available in the form of a header command.

Because of requirement R3.4.1 (one script will be able to include another),we will also need a command to include other source code files into the resultingassembly. In this case, we will also recursively scan the included files for their ownheader sections, which may for example contain references or resources requiredby the included file.

Because of requirement R3.4.2 (one script will be able to reference another),the command for referencing assemblies will also be able to reference anotherscript file. In this case, the referenced script file will be recursively compiled firstand the resulting assembly will be used as the reference instead.

33

File parameters

Many header commands will expect a path to a file as a parameter. In order tomake this parameter as versatile as possible, we will accept both absolute paths,or paths relative to the directory with the current source code file.

Most commands that expect a path to a file as a parameter can be appliedmultiple times to different files. To simplify this, we will introduce wildcards.Paths will be able to contain a wildcard character * in their file name. In thiscase, all the files found in the target directory that match the wildcard will bepassed to the command. The wildcard character * will be also usable as the nameof the last directory in the command’s parameter. In this case, all subdirectorieswill be also recursively searched for the specified file. For example, the followingcommand will include all files with the csx extension in the Common directory andalso all its subdirectories recursively:

INC Common\*\*.csx

Sometimes it is useful to use a wildcard to include a lot of files except a selectfew. For this purpose, a hyphen before the parameter will indicate that the fileshould be excluded. For example, the following commands will include all fileswith the csx extension in the Common directory, except other.csx:

INC Common\*.csx

INC - Common\other.csx

Application-specific commands

When debugging a script, it can be useful to specify what working directory itshould be run from or with what command-line arguments. Since such informa-tion is tied to individual scripts, it makes sense to include it somehow into thescript files themselves. Therefore, we will also support header commands that donot control the build process itself, but instead provide information that is passedto the application, which can then use it to control how the script is launched.Such header commands will only affect scripts launched from the application itselfand not scripts launched externally via the generated executable.

3.4.2 Builder

The process of compiling script files into executable binaries based on the in-formation contained within the script files in the form of header commands isspecial, because it may be useful to use it outside the application itself in orderto build script files externally, for example as a part of a batch script. In suchcases it would be preferable to invoke just some kind of a compact command-lineutility instead of using the full application. Because of this, will implement thebuild process in a separate library, which we will call Builder. This library willprovide an interface that the application will call to compile script files.

34

Project structure

Since script files will be able to include or reference one another, we will need tohandle this carefully with respect to all functionality requiring on-the-fly analysisof the source code. This means that we will need to maintain what we call theproject structure of the currently open scripts:

• A project will be a single stand-alone compilation unit that consists of oneor more source code files, their references, and its compilation results intoa single output assembly.

• The project structure will then be information about what projects areopen within the application, what are their external references, and howthey reference one another. This information will be essential later whenwe implement code-aware functionality.

Since the project structure depends solely on header sections within the scriptfiles which will be parsed by the Builder library, its interface will need a meansto report the project structure back to the application.

Error highlighting

Error highlighting is a very useful feature of any source code editor because itallows users to immediately see the errors within the source code without havingto look them up manually. In order to provide error highlighting, the interface ofthe Builder library will need to be able to report errors back to the applicationso that they could be presented to the user.

Real-time information

The code-aware functionality of the application, and thus the on-the-fly analysisof the source code, will need to be done in real time and reflect the current stateof the source code, which, as the user modifies it, will very often differ from thestate saved on the filesystem. Such local unsaved modifications can also affectthe project structure because the user can modify header commands as well.Therefore, the Builder library will need to:

• Provide a way to invoke it and supply replacement content for script filesthat are currently open and modified within the application.

• Provide a way to invoke it in a limited “parse” mode, which will only scanthe source code files for header commands and return the project structureinformation. In this mode, there will be no need to invoke the compileritself.

• Provide a way to invoke it in a limited “check” mode, which will invoke thecompiler, but only in order to report any compilation errors and withoutactually generating the output assemblies.

35

Code completion

It would be convenient if code completion, which will primarily work for theC# source code itself, also worked in header sections and header commands.This would simplify usage of header commands, which could then be offeredautomatically and users would not need to remember them or look them up.Also, completion of various arguments of header commands would be useful, forexample when specifying file paths. Therefore, the Builder library will provideinterface to code completion of header commands.

3.5 NRefactory

NRefactory [6] is a C# library that allows parsing and semantic analysis of theC# source code. It also contains support for refactoring, formatting, code is-sue analysis and code completion. Our application will make heavy use of theNRefactory library to implement its code-aware functions.

3.5.1 Overview

NRefactory is introduced in depth in Using NRefactory for analyzing C# code [7].It differs from the other libraries used in our application in the fact that it is noteasy to understand and work with. Therefore, here we will outline how it is usedto analyze C# source code. We will explain only its most basic principles andnomenclature so that we could later describe how it will be integrated into ourapplication.

The basic steps of syntactic and semantic analysis of the source code areshown in figure 3.2. When C# source code needs to be analyzed, the NRefactoryworkflow looks like this:

Figure 3.2: A simplified flow of source code analysis in NRefactory (from [7]).

1) A parser is invoked on the C# source code file. The result is an abstractsyntax tree, or AST, representing the syntactic structure of the original file.The AST also contains additional information about comments, whitespaceand exact position of individual syntactic elements within the original file, sothat refactoring transformations can alter the AST and render it back intosource code text with minimum changes. The AST is represented by theSyntaxTree class and its nodes by the AstNode abstract base class.

2) The AST of a single C# source code file is then converted into an unresolvedtype system. This process analyzes the AST and extracts information aboutthe types and members contained within the file. The type system is called“unresolved” because it only contains basic information about the types and

36

members as they appear in the source code itself, without being put into con-text of neighbouring types, other source files, or referenced assemblies. Theunresolved type system is represented by a complex hierarchy of classes (sim-ilar to the hierarchy shown in figure 3.4). The output of converting an ASTof a single C# source file into an unresolved type system is represented by theIUnresolvedFile interface implemented by the CSharpUnresolvedFile class.

It is rarely needed to analyze a single C# source file on its own. The source file isalmost always a part of a larger project that contains other source files, referencesto extrenal assemblies, or references to other projects. How this is handled byNRefactory is shown in figure 3.3. The workflow continues like this:

Figure 3.3: A detailed flow of source code analysis in NRefactory (from [7]).

3) We need to load information from the referenced assemblies. For this purposethe NRefactory library contains the CecilLoader class that uses the Cecillibrary [29], which is a dependency of NRefactory, to read an assembly file andcreate an unresolved type system representing the types and members withinthe loaded assembly. The result is represented by the IUnresolvedAssembly

interface.

4) Now we need to put all the source files (IUnresolvedFile) and referencedassemblies (IUnresolvedAssembly) together so that they would representa single project. This is done by the CSharpProjectContent class imple-menting IProjectContent. This class acts as a container for source filesand references and represents the resulting assembly. It also implementsIUnresolvedAssembly, so other instances of CSharpProjectContent can be alsoreferenced if we have multiple projects that reference one another.

5) When we have put together an IProjectContent instance representing theproject within which we want to analyze the source code, we call itsCreateCompilation method. This method creates an ICompilation instancewhich represents the project as a resolved type system. This is a type system

37

that takes into account all contextual information, such as assembly refer-ences and relations that span across multiple members, types and files. Theresolved type system forms a hierarchy reminiscent of the System.Reflection

API. This hierarchy is shown in figure 3.4.

Figure 3.4: A resolved type system class hierarchy in NRefactory (from [7]).

Now that we have the ICompilation instance, which represents full informationabout the context of the C# source code files contained within the project, we canfinally analyze their semantics. For this purpose we create a CSharpAstResolver

instance, which takes the ICompilation instance (the context) and the SyntaxTreeinstance (the parsed source file to be analyzed). Then we can call its Resolve

method with any node of the syntax tree as a parameter in order to find out itssemantic meaning.

The semantic meaning is returned in form of a ResolveResult instance.For example, a single identifier (a node of the type IdentifierExpression)can be resolved to a local variable (LocalResolveResult), property(MemberResolveResult), type (TypeResolveResult) etc. Sometimes it may evenmean nothing (UnknownIdentifierResolveResult) or several things at once(AmbiguousTypeResolveResult). Individual subclasses of ResolveResult carry theinformation about the entities that were identified by the resolver. The resolvingprocess can be seen in figure 3.3 as well.

The CSharpAstResolver class also allows retrieving the state of the resolverat a certain position within the source file. The state is represented by aCSharpResolver instance and can be queried for a lot of interesting information,e.g. the current type and member declaration, the current namespace and usingscopes, or name lookups.

It should also be noted that most classes in NRefactory are immutable (orat least freezable), which means that they can be safely accessed from mul-tiple threads. This is quite surprising in case of the CSharpProjectContent

class that acts as a container for C# source files and references. Each time aCSharpProjectContent needs to be modified (this happens often because it repre-sents a project that continually evolves as the source code is edited by the user),a new instance containing the changes is created and the original instance is leftunchanged.

38

3.5.2 Core

There are a lot of features in the application that will need the analysis providedby NRefactory (see requirements R2 in section 2.2.2). However, if every functionparsed and analyzed the source code on its own, the application would be unnec-essarily slow and resource intensive. Because of this, a central way of managingsource code parsing and analysis is needed. We need a system that would monitorchanges to the source code and keep its representation provided by NRefactory upto date, so that if any function asked for the data from NRefactory (e.g. an AST,a compilation, a resolver, etc.), it would get it right away.

The major responsibilities of this system, which we will call the NRefactorycore, will be:

• Monitor the project structure of C# scripts that are open in the application.

• Keep an up-to-date IProjectContent instance for each project.

• Load external assemblies and source code files.

• Reload external assemblies and source code files if they change.

• Whenever a source code file changes (either externally or locally within theapplication), reparse it.

• Upon request, return the current SyntaxTree for a source code file

• Upon request, return the current ICompilation for a project.

Volatile resources

The syntactic and semantic representation of the source code in our application,as provided by the NRefactory library, depends on three kinds of resources:

• A source code file.

• An external assembly.

• A project assembly (an assembly representing a script built by the applica-tion itself).

Such resources have an important property – they change over time. A sourcecode file can be edited. An external assembly can be replaced. A project assemblycan be recompiled. Therefore, we will introduce a concept we shall call a volatileresource. A volatile resource will be a resource that changes over time and allowsus to retrieve its current value and check it for updates. Thus, we will need twooperations upon each volatile resource:

• A method to retrieve the resource’s current state (value).

• A method to check if the resource’s state (value) we retrieved is still up todate.

39

From the point of view of the NRefactory library, a project will consist of a setof volatile resources representing its source code files, its external references toother assemblies, and its internal references to other projects. Whenever wedetect a change in any of these resources, we will know that any NRefactorystructures we have parsed and analyzed for the project are out of date and needto be produced again. This will also mean that the project itself has changed,which is important because the project itself can serve as a volatile resource ofother projects that reference it.

Resource loaders

Now that we have defined the concept of volatile resources, we will need a meansto create them and check if they are up to date. Therefore, we will introduceresource loaders, which will be the central points for acquiring volatile resourcesfor given source code files or assembly files. The loader for external assemblieswill have many responsibilities:

• Because external assembly files may change, we need to check them for anychanges. There are two ways this could be done – use a FileSystemWatcher

class from the .NET framework which monitors changes to files and direc-tories using the means of the operating system, or simply check the file’slast write time.

Since we do not expect external assembly files to change much (if at all), wewill use the second approach, which is simpler and does not use operatingsystem resources. Moreover, we will only explicitly check for update whenthe application returns into foreground from being inactive because changesto external files are expected to happen mostly when the user is away fromthe application.

• There can be XML documentation associated with the external assembly,which the NRefactory library allows to be loaded as well and associatedwith the assembly. Therefore, the loader will need to check for the presenceof the XML documentation alongside the loaded assemblies and load it ifpresent.

• Different projects can request the same assembly. Therefore, to avoid havingto load an external assembly multiple times and thus to reduce resourceusage, we will keep a weakly referenced cache of the loaded assemblies sothat a single assembly is not loaded twice unnecessarily. The cache willhave to be implemented in such a way that if an assembly is asked to beloaded while another thread is already in the process of loading the sameassembly, we wait for it to finish and use its result.

The loader for source code files will have other responsibilities:

• Because source code files may also change externally, we need to check themfor changes. We will do this in a similar way to external assemblies.

• Source code files can be opened in the application, modified by the user,and not yet saved. In this case, the user will expect all code-aware function-ality of the application to reflect this current (modified) state of the source

40

code, as opposed to the state stored on the disk. Therefore, we will need totreat this as a change of the source code file resource and use this modifiedcontent of the source code file for NRefactory processing instead of its orig-inal content on the filesystem. This way, internal source code modificationswill be included seamlessly into the version checking mechanism of volatileresources.

Project representation

One of the most important responsibilities of the NRefactory core subsystem willbe monitoring the structure of the C# scripts that are open in the application andkeeping an up-to-date IProjectContent instance for each project. A project willbe identified by its path and will consist of paths to its source code files, referencedassemblies, and other referenced projects. In order to keep the IProjectContent

instance up to date, each project also needs to have:

• For each source file its current parsed representation (IUnresolvedFile).

• For each external assembly reference its current representation loaded intothe application (IUnresolvedAssembly).

• For each internal project reference its current representation (the instanceof IProjectContent managed by the referenced project).

As the structure of a single project changes over time, modified source codefiles need to be parsed, external assemblies need to be loaded, and the project’sIProjectContent needs to updated to include the current representations of theseresources.

This is where we will make heavy use of the volatile resource concept de-scribed above. Each of the resources that make up a project (source code file,external assembly, another referenced project) can change over time – sourcecode files are modified regularly, external assemblies can potentially change, andother referenced projects change every time their own source files and referencesare modified. Because of this, every resource that makes up a project will beimplemented as a volatile resource, and so allow us to retrieve its current repre-sentation and check it for modifications. This way, when a project is queried forits NRefactory representation, we will know if we can return cached data or wemust create it anew.

3.5.3 Code completion

While the user types identifier names, code completion offers a list of availabletypes, members, variables and other constructs. It allows to choose an identifierwithout the need to remember its exact name or without having to type outits name in full, because names are traditionally long and descriptive in the C#

language. It is also often used for quick overview of available types and theirmembers, and it can also display snippets from the documentation. The require-ments for code completion in the application (R2.2) are specified in section 2.2.2.

41

User interface

The editor control that is used in the application, AvalonEdit, contains basicimplementation of a completion window. When we want to display it, we needto pass it the following information:

• The text segment (offset and length) of the area where the code completionhappens.

– The window is displayed beneath this area.

– The window is automatically closed when the caret leaves this area.

– The window filters its content by the text from the beginning of thisarea to the current caret position.

• A list of items to offer in the completion window. Each item has:

– The icon that is displayed in the first column of the completion list.

– The content of the second column of the completion list.

– The string that is used to filter the items in the completion list.

– The content of a tooltip that is displayed when the item is highlighted.

– The method that is invoked when the item is selected. This methodusually replaces the text in the completion segment with another text.

The example of such a completion window can be seen in figure 3.5. The textsegment for completion is the word “int”, its part “in” is used to filter the com-pletion items, tooltip is shown. When any item is selected, its content replacesthe word “int”.

Figure 3.5: An example of the code completion window.

We will use the user interface implementation of the code completion windowin AvalonEdit as the basis for our own code completion. However, we will needto extend it:

• In order to support extensibility (R1.7), we will create an extensible in-frastructure for code completion data sources, so that items in the codecompletion list could come from plugins exported via MEF.

• According to requirements R2.2.3 and R2.2.4, code completion will haveto offer types and extension methods from namespaces that are not yetimported via the using statement and from assemblies that are not yet ref-erenced, and automatically add the correct using statement or the correct

42

reference accordingly upon selection. Instead of combining all the sugges-tions into one long list, we will instead introduce three different levels ofcode completion:

1) Basic completion – Offers all known symbols that can be used at thecurrent position.

2) Import completion – Offers types and extension methods from names-paces that are not currently imported with a using statement andimports the namespaces accordingly when an item is selected.

3) Reference completion – Offers types, extension methods and names-paces from yet unreferenced but known assemblies, and automaticallyreferences the correct assembly when an item is selected.

The first level will be the default. Users will be able to switch to the nextlevel by invoking the code completion again using the keyboard shortcut,which is traditionally Ctrl+Space.

Therefore, we will have to add support for multiple levels of code comple-tion. If users do not find what they were looking for in the code completionlist, they will be able to invoke the code completion window again and itwill display a different set of items from the next level.

• To satisfy requirement R2.2.2, the code completion window will have tosupport a special mode so that if the user introduces a new identifier insteadof selecting an item from the list, the code completion will not interfere byinserting the item’s text instead.

We will do this by introducing the concept of active selection and inactiveselection. The selected (highlighted) item in the code completion list can beeither active or inactive. If a character that cannot be a part of an identifieris typed (e.g. a space, comma, etc.) and the current selection is active, theselected item gets inserted. On the other hand, if the current selection isinactive, no insertion takes place.1 The visual distinction of these two casescan be seen in figure 3.6. Optionally, the inactive selection can becomeactive after the first character that can be a part of an identifier is typed.2

Figure 3.6: An example of active and inactive code completion window.

1Inactive selection is used for example in cases where both a known and a new identifier canbe used, e.g. lambda method parameters in C# (see figure 3.6).

2This is used for example when the user typed the new keyword, after which a type namecan follow, or a square bracket (array construction), or a curly bracket (anonymous objectconstruction). If the user types a bracket, no completion takes place. If the user types a letter,the selection becomes active and normal completion takes place.

43

• We will also improve the user interface of the completion window by addingsupport for additional content in the right column, which can be used fordisplaying extra information, for example to indicate the return type ofmethods, as can be seen in figure 3.6. Because of this, we will also addsupport for automatic widening of the completion window, so that if thecompletion window is too narrow to fit the content of its visible items, itwill be automatically widened.

Data source

Deciding what exactly to offer in the completion window at a given location inthe source code is a complex task that requires syntactic and semantic analysisof the source code both preceding and following the location. Fortunately, theNRefactory library contains good support for code completion, which we willintegrate with our application. We will also need a separate data source for thehigher levels of code completion (import and reference completion), which meansa list of all potential types in all potential assemblies.

Code completion mode

The requirements R2.2.1 and R2.2.2 require code completion to:

• Appear automatically when the user types an identifier.

• Do not appear in places where the identifier denotes a name of a new symbolinstead of a reference to a previously known symbol.

• Appear in a different mode in places where it is syntactically possible bothto introduce a new symbol or to use an existing one (such as when passinga lambda delegate as a real parameter). This different mode is the inactiveselection mode described above.

Therefore, when the user starts typing an identifier, we must first look at theposition in the source code where the typed identifier is located and decide ifa new symbol can be introduced there or not:

• If a new symbol cannot be introduced at the location of the identifier, weopen code completion in normal (active selection) mode.

• If a new symbol can be introduced at the location of the identifier, thereare two further scenarios:

– A known symbol can be referenced at the location of the identifier.Therefore, we open code completion in inactive selection mode.

– No known symbol can be referenced at the location of the identifier.Therefore, there is no need to open code completion at all.

The decision process will be based on the same tools that the code completionengine from the NRefactory library uses to determine what items to offer in thecompletion list.

44

4. Implementation

The source code of the implemented application can be found on the enclosedCD in the src directory. To open it in Visual Studio, at least Visual Studio 2010and .NET Framework 4.0 or newer are required. Documentation generated fromthe the source code is also available on the CD, in the doc directory.

Libraries

The application is implemented with the help of a number of external libraries.During the development of the application, it was sometimes necessary to modifysome of these libraries. The changes were only minor (e.g. build configurationwas tweaked or additional members were exposed), but even so, we include theirfull modified source code as well. Therefore, there are two subdirectories in thesrc directory:

• ExtBrain.ScriptDevelop – The source code of the application itself.

• Libraries – The source code of the libraries that the application uses.

The readme.txt file in the Libraries directory lists the modified libraries and foreach library it contains the following information:

• The URL and revision of the library’s Subversion repository, from whichthe original source code was downloaded.

• A summary of custom modifications to the library.

For each library there is also a .patch file generated by the Subversion clientwhich describes what exactly has been modified in the source code of the library.

The libraries are set up to output the resulting binaries to the ExtBrain.

ScriptDevelop/Externals directory, from which the application references them.This directory also contains two other external libraries which there was no needto modify, so only their binaries are included:

• log4net.dll (version 1.2.13) – A helper library for logging.

• JetBrains.Annotations.dll (version 6.1.37.86) – Helper attributes for theResharper tool (which is not required to open or build the source code).

Previous work

As stated in section 3.2, we used ScriptDevelop [11] as the starting point for ourapplication. However, we have modified it heavily to meet our goals, and verylittle is left of the original source code. For reference, the original source code ofScriptDevelop is included on the enclosed CD in the previous directory.

4.1 Overview

This section provides a high-level overview of the application’s implementation.It details what components the application is composed of, what is their functionand how they cooperate and depend on each other.

45

4.1.1 Components

The application is separated into several primary components, each of which isfocused on a different functional aspect of the application. Such primary com-ponents are large portions of the application, containing many classes, plugins,or even assemblies. Each of these components will be further described in thissection.

There are six primary components of the application – Application Core, UICore, Editor, Compiler, NRefactory, and Debugger. Each of these componentsuses services provided by the previous ones. The six primary components can beseen in figure 4.1 as the central column.

Figure 4.1: Overall structure of the application.

The purpose of the primary components is as follows:

• Application Core contains the main entry point of the application and of-fers the basic supporting services such as plugin loading, configuration orlogging. It does not reference any other components – the rest of the app-lication is loaded dynamically at runtime.

• UI Core contains the main window of the application and implements basicuser interface – a single window with menu, toolbar, status bar, dockablepanels and document tabs. This interface is universal, is not tied to anyspecific application type, and could be used for any other application witha similar layout.

46

• Editor provides the concept of documents, their formats and editors. Itmanages currently open documents and their lifecycle. It also defines a textdocument and its editor and offers a lot of services for them, for examplehighlighting, tooltips, context menus, additional margin panels, or locationtracking.

• Compiler defines the C# document format and basic functionality that doesnot require code-aware insight into the source code. It contains commandsto build and run scripts. It also monitors the currently open scripts andruns the build process in order to display errors and to keep the projectstructure information up to date.

• NRefactory1 is the largest component of the application. It encompasses allcode-aware features of our C# source code editor that require on-the-fly syn-tactic and semantic analysis of the source code, such as syntax highlighting,code completion, code issue analysis, or navigation.

• Debugger implements the basic debugging functionality. It defines com-mands for starting, pausing and stopping the debugger, for stepping throughthe debugged code, or for breakpoint management. It also contains panelsthat display call stack, exception information, or command console.

The components that stand apart from the central column in figure 4.1 havespecial purpose:

• Common is a utility library with shared code that all components use. Itis a place for helper classes and extension methods that are not specific toany single part of the application.

• Builder is a library that the Compiler component calls to actually compileC# scripts into binary assemblies. It parses the header sections in the sourcecode and runs the actual C# compiler accordingly. It does not depend onthe rest of the application and it can be used on its own, for example itcould be easily made into a separate command-line tool for building C#

scripts.

• Explorer is an example of a third-party plugin that extends the applicationwith additional functionality (in this case a filesystem tree explorer). It waspart of the original ScriptDevelop project (see section 3.2) and it was leftmostly intact during the development of our application, only with minimalchanges to ensure its compatibility.

The first three primary components (Application Core, UI Core, Editor) areunrelated to C#. Application Core on its own could be used for any WPF appli-cation and provide it with the supporting infrastructure (plugins, configuration,logging). Application Core and UI Core together make up a generic WPF app-lication with a basic user interface consisting of main menu, toolbar, status bar,dockable panels, and document tabs. Editor adds the ability to open and edit text

1Please note that this component is named after the underlying external library it uses,which is NRefactory 5.

47

files. With only these three components the application is not tied to C# in anyway. The other three components (Compiler, NRefactory, Debugger) graduallyextend the application into a C# IDE.

Every component of the application consists of one or more projects in thesource code of the application, as described in table 4.1.

Component Project

Application Core ExtBrain.ScriptDevelop.ApplicationCore

UI Core ExtBrain.ScriptDevelop.Plugin.MainWindow

Editor ExtBrain.ScriptDevelop.Plugin.Editor

ExtBrain.ScriptDevelop.Plugin.Editor.Text

Compiler ExtBrain.ScriptDevelop.Plugin.CSharp

ExtBrain.ScriptDevelop.Plugin.Compiler

NRefactory ExtBrain.ScriptDevelop.Plugin.NRefactory.Core

ExtBrain.ScriptDevelop.Plugin.NRefactory.Import

ExtBrain.ScriptDevelop.Plugin.NRefactory.CodeCompletion

ExtBrain.ScriptDevelop.Plugin.NRefactory.Analysis

ExtBrain.ScriptDevelop.Plugin.NRefactory.Analysis.Import

ExtBrain.ScriptDevelop.Plugin.NRefactory.Analysis.Adapter

ExtBrain.ScriptDevelop.Plugin.NRefactory.Navigation

ExtBrain.ScriptDevelop.Plugin.NRefactory

Debugger ExtBrain.ScriptDevelop.Plugin.Debugger

Common ExtBrain.ScriptDevelop.Common

Builder ExtBrain.ScriptDevelop.Builder

Explorer ExtBrain.ScriptDevelop.Plugin.Explorer

Table 4.1: Components and their corresponding projects.

The interaction between the components of the application is shown in fig-ure 4.2 and could be summarized like this:

Figure 4.2: Interaction between the components of the application.

48

• The user types the source code in the Editor.

• The source code is processed by the Compiler, which invokes the Builder.

• The Builder parses the header sections in the source code, determines howto call the actual C# compiler, and calls it.

• If there were errors, the Builder returns their list to the Compiler, whichpresents it to the user and highlights the errors in the Editor.

• If there were no errors, a binary assembly is created, optionally with a sym-bol file for debugging.

• The Builder returns the information it parsed from the header sections inthe source code back to the Compiler.

• The Compiler uses this information the keep the project structure up todate. The project structure is information about what source files make upwhat assemblies and how the assemblies reference each other.

• The Compiler invokes this process either in full upon a manual requestfrom the user, or only partially whenever there has been a change in thesource code in order to reparse the header sections and update the projectstructure.

• NRefactory processes the project structure information, parses the sourcecode into syntax trees, loads external referenced assemblies, and buildsa type system representing the source code. Syntax trees and the typesystem are updated whenever there has been a change in the source code.

• Subcomponents of the NRefactory component consume the syntax treesand the type system information in order to provide various code-awarefunctionality to the Editor.

• The Debugger utilizes the syntax trees and the type system information forevaluation and state inspection.

4.1.2 Plugins

The application is implemented as a set of many plugins that use oneanother’s services. The main executable project, ExtBrain.ScriptDevelop.

ApplicationCore, contains only the basic infrastructure for application startup,plugin management, and a few supporting services like configuration and logging.Without any additional plugins to load, the application would start, but it wouldexit immediately because there would not even be any window to be displayed.Every separate function or feature of the application, no matter how small, isimplemented as a plugin and could be easily separated into an assembly of itsown2 and loaded later, or not loaded at all.

A lot of plugins can be themselves extended by other plugins. For example, theplugin that defines the main window of the application can be extended by plugins

2For practical reasons, a single assembly of the application usually contains many plugins.

49

defining individual components of the main window (main menu, toolbar, statusbar). The status bar plugin can be further extended by a plugin displaying currenttext cursor position or the status icon. Some plugins can even extend multipleother plugins. For example, every command in the application is a plugin thatcan have its representation in the main menu, in the toolbar, and in the keyboardshortcut manager. An example of such a plugin structure is shown in figure 4.3.

Figure 4.3: An example of a more complex plugin structure.

There is also a special kind of plugins called extension plugins. An extensionplugin attaches an extension to every instance of its target type to modify itsbehavior or to provide additional services. Extension plugins are heavily usedthroughout the application to attach behavior to individual open documents andto extend highlighted text segments with additional functionality.

4.2 Application Core

Application Core is the main entry point of the application – it contains theProgram.Main method and is reponsible for initialization of the entire application.It does not reference any other assemblies of the application, but instead loadsthe rest of the application dynamically at runtime. Therefore, the primary partof this component is the plugin infrastructure. The component also providessupporting services: logging, configuration, settings, and temporary storage.

4.2.1 Plugin infrastructure

The plugin infrastructure of the application is built on top of MEF, which is usedto manage discovery and instantiation of individual plugins through its mecha-nism of exports and imports. In addition to this the ExtBrain.ScriptDevelop.

ApplicationCore project contains classes and interfaces to further support pluginmanagement in the ExtBrain.ScriptDevelop.ApplicationCore.Infrastructure

namespace, which is illustrated in figure 4.4 and described in this section.

50

Figure 4.4: Plugin infrastructure.

Plugin loading

The loading of plugins into the application is handled by the PluginLoader staticclass using the method:

public static void LoadAssembly(Assembly assembly)

The PluginLoader class maintains a MEF catalog and a composition container.This method adds the given assembly into the catalog, updates the compositioncontainer, and initializes the newly added plugins. It also recursively loads andadds any assembly that was referenced by the original assembly and also containsplugins. This is to ensure that if a plugin references another plugin from a differentassembly, the referenced plugin is loaded along with the referencing plugin.

Plugin initialization

A plugin that needs to perform some kind of initialization upon startup canimplement the following interface:

[InheritedExport]

public interface IPluginBase

{

void Load();

}

Any plugin that implements this interface will have its Load method called whenit is loaded into the application. The Load method is guaranteed to run onthe UI thread, so its implementation can safely access UI elements. However,plugins that are exported to MEF using other contracts do not necessarily needto implement this interface.

Singleton plugins

The most common type of plugin in the application is a singleton plugin, i.e.a plugin that exists as a single shared instance. To simplify accessing singleton

51

plugins in code and to make sure they are only accessed after they had beenproperly initialized, there is a special generic base class:

public abstract class SingletonPlugin<T> : PluginBase

{

public static T Instance { get; }

}

This class allows accessing the singleton instance of the plugin of type T.

Extension plugins

The plugin infrastructure of the application also supports the concept of extensionplugins. An extension plugin is a special kind of plugin that attaches an extensionto an object. The object is then extended with additional functionality.

When an extensible object is created, extensions that extend this object arealso created and attached to it. The extensions of the object can be accessedin code. When the lifetime of the object ends, the extensions that have beenattached to it are detached and disposed. If a new extension plugin is loadedwhile the application is already running, the new extensions are created andattached to all extensible objects of the correct type that currently exist withinthe application.

The extension system consists of several interfaces3 and classes, as shown infigure 4.5.

Figure 4.5: Extension infrastructure.

The IExtension interface marks a class the instances of which serve as ex-tensions to an extensible object. It is only a marker interface, i.e. it contains nomembers.

3The actual definition of the interfaces is more complicated than described here for technicaland type safety reasons, but the core princliple is still the same.

52

public interface IExtension

{

}

The IExtensible interface marks an extensible class, i.e. a class that can haveextensions attached:

public interface IExtensible

{

ExtensionContainer ExtensionContainer { get; }

T GetExtension<T>() where T : IExtension;

}

The GetExtension method accesses the extension of type T associated with theIExtensible object. The ExtensionContainer property returns the extensioncontainer of the extensible object; each instance of IExtensible must provideits own ExtensionContainer, which is an internal class whose purpose is to keeptrack of extensions associated with a single extensible object.

The IExtensionPlugin interface marks a plugin that serves as a factory forextensions. It creates the corresponding IExtension for the given IExtensible

object:

[InheritedExport]

public interface IExtensionPlugin<TExtensible, TExtension>

where TExtensible : IExtensible

where TExtension : IExtension

{

TExtension InstallExtension(TExtensible target);

}

The lifecycle of extensions is managed by ExtensionManagerPlugin. This plu-gin exposes two methods:

public void NotifyExtensibleObjectCreated(IExtensible target)

public void NotifyExtensibleObjectDisposed(IExtensible target)

The first method is used to register a new extensible object. Upon registration,the InstallExtension factory method is called on all known extension pluginsto create new instances of available extensions, which are then attached to theobject. The second method is used to unregister the object when it is no longeralive. All extensions attached to the object are removed and the extensions thatimplement the IDisposable interface have their Dispose method called.

Sometimes the extensions depend on each other, i.e. the InstallExtensionme-thod assumes that the object being extended already contains another extensionit can access using the GetExtension method. This situation is correctly handledby the extension manager – if GetExtension method is called for an extensionthat is not yet installed, but already in the installation queue, it is installed firstand then returned to the InstallExtension method. This way the installationorder of extensions is corrected automatically. When an extensible object is un-registered, its extensions are uninstalled in the reverse order than the order inwhich they were installed. This ensures that dependent extensions can accesseach other correctly even in their Dispose method.

53

4.2.2 Configuration and settings

Configuration and settings are one of the supporting services of the ApplicationCore component. We described configuration and settings in section 3.3.3. Con-figuration allows the user to configure the behavior of various components of theapplication. It is user-defined and cannot be written to by the application. Set-tings allow various components of the application to persist their state betweensessions. Users cannot change this state directly.

The classes that participate in configuration and settings management can beseen in figure 4.6.

Figure 4.6: Configuration and settings infrastructure.

Configuration

The application contains no specialized configuration GUI. Instead, it is config-ured using an external XML file. This file can be edited directly in the applicationand whenever it is saved, it is reparsed and the components that can update theirconfiguration on the fly are notified that an updated configuration is available.

The default configuration file is embedded inside the application and is usedif no configuration file is found in the application’s directory. If the user choosesto edit the configuration file using an option in the application, it is opened inthe application’s editor. If the file does not exist yet, it is created as a copy ofthe embedded configuration file.

The configuration file is formatted as an XML document whose root nodecontains one element for each separate configuration section. This is an exampleof a simple configuration file with two sections:

<Configuration>

<KeyBindings>

<KeyBinding Command="GoToTypeCommand" Shortcut="Ctrl+T" />

</KeyBindings>

54

<Logging>

<Path>ScriptDevelop.txt</Path>

<Filters>

<Filter Prefix="*" Value="all" />

</Filters>

</Logging>

</Configuration>

In the code, each section of the configuration file is represented as a pluginderived from ConfigurationPlugin. When such a plugin is loaded, its public fieldsand properties are read from the appropriate section of the configuration file usingthe standard XmlSerializer of .NET. The name of the section by default equalsto the name of the plugin’s class, but can be defined separately using the XmlRootattribute. If the corresponding section is not found in the configuration file orcannot be properly deserialized, the state of the plugin is left unchanged. Becauseof this, the constructor of the plugin should initialize it to a correct default state.

To illustrate this, the KeyBindings section from the above example is repre-sented in the code like this:

[XmlRoot("KeyBindings")]

public sealed class KeyBindingConfiguration

: ConfigurationPlugin<KeyBindingConfiguration>

{

public class KeyBinding

{

[XmlAttribute]

public string Command { get; set; }

[XmlAttribute]

public string Shortcut { get; set; }

}

[XmlElement(ElementName = "KeyBinding")]

public List<KeyBinding> KeyBindingList { get; set; }

public KeyBindingConfiguration()

{

KeyBindingList = new List<KeyBinding>();

}

}

Since the ConfigurationPlugin class derives from SingletonPlugin, it can beeasily accessed using the Instance static property, for example:

var keys = KeyBindingConfiguration.Instance.KeyBindingList;

The ConfigurationPlugin class also provides the ConfigurationUpdated event,which is fired every time the configuration file has been reloaded and the publicfields and properties of the class have been updated with newly parsed values.

55

Settings

In a similar way to configuration, any plugin that derives from the SettingsPluginclass has its state automatically saved and restored. Saving is done at the appli-cation exit, restoring in the SettingsPlugin.Load method. All public fields andproperties of the plugin are serialized using XmlSerializer and persisted usingthe application settings mechanism of .NET described in [30].

4.2.3 Logging

The logging infrastructure can be seen in figure 4.7. Logging is implemented asthe LoggerPlugin singleton plugin, which contains Get methods for retrieving anILogger instance for a given type. The ILogger interface contains methods forlogging messages and exceptions of three different severity levels: Debug, Warning,and Error.

Figure 4.7: Logging infrastructure.

The logger writes messages into a text file whose path can be configured. Min-imal severity levels for individual types (filters) can also be configured. Logginguses the log4net library [31].

4.2.4 Temporary storage

Temporary storage is described in section 3.3.2. It is implemented by theTemporaryDirectoryPlugin, which contains methods for accessing the sharedor instance-specific temporary directories and for creating subdirectories insidethem.

4.2.5 Application startup

When the application is launched, these steps are performed:

• PluginLoader.LoadAssembly is called on the executing assembly (ExtBrain.ScriptDevelop.ApplicationCore). This ensures that the plugins for thecore infrastructure (described above) are initialized and ready.

56

• The Loader.Initialize method is initialized. The Loader class reads theconfiguration file to see what additional plugin assemblies are to be loadedand the Initialize method loads them using PluginLoader.LoadAssembly.Assemblies can be also configured to be loaded later, after a delay, which isalso handled by the Loader class.

• The MEF composition container is queried for a plugin of typeIMainWindowPlugin, and the window exposed by this plugin is displayed.

4.2.6 Stand-alone application executable

For easier manipulation and deployment, the application can package itself intoa single stand-alone executable file that contains all the currently loaded plugins,the current configuration file, and all the necessary third-party libraries. Theresulting single executable file can then be copied to any system which could runthe original application and launched directly from the executable file, withoutany installation or unpacking process.

This functionality is implemented in the Embedder class, which contains me-thods to build the stand-alone executable, to query what assemblies are packagedinto the current executable, and to load them.

The packaging process invokes the C# compiler in order to build the stand-alone executable. The resulting assembly contains only a short bootstrappingcode that sets up the AssemblyResolve event of the application’s AppDomain tolook for assemblies among the resources embedded into the executable itself, andthen calls the original Program.Main method to start the application. Duringthe compilation, all the currently loaded and referenced assemblies are insertedinto the resulting executable as embedded resource files, as well as the currentconfiguration file, which replaces the default configuration file of the originalapplication.

4.3 UI Core

UI Core is the primary user interface component of the application. It definesthe main window and its structure – menu, toolbar, status bar, dockable pan-els and document tabs. It also manages commands, which are individual ac-tions the user can perform by selecting them in the menu or toolbar or by usingthe respective keyboard shortcut. The UI Core component is implemented inthe ExtBrain.ScriptDevelop.Plugin.MainWindow project and the structure of itsplugins is shown in figure 4.8.

The user interface provided by this component is not tied to any specificapplication type and could be used for any other application with a similar layout.

4.3.1 Main window

The main window is composed and exposed by the MainWindowPlugin class. Itimports plugins with the IMainWindowComponentPlugin interface. These plug-ins make up the main window’s content. The window itself contains a single

57

Figure 4.8: Plugin structure of the UI Core component.

DockPanel control that is populated with the controls exposed by the individualIMainWindowComponentPlugin plugins:

• MainMenuPlugin – Imports commands and creates the main menu composedof the items defined by the imported commands.

• ToolbarPlugin – Imports commands and creates the toolbar composed ofthe buttons defined by the imported commands.

• StatusBarPlugin – Imports IStatusBarComponentPlugin plugins and createsthe status bar composed of the controls defined by the imported plugins.Also exposes methods to display textual status information.

• AvalonDockPlugin – Fills the central area of the window with aDockingManager control from the AvalonDock library and exposes it so thatother components of the application can use it to display dockable panelsand tabbed documents.

4.3.2 Commands

Commands are actions that the user can explicitly perform in the application.They can be represented by items in the main menu or by buttons in the toolbar.They can also have a keyboard shortcut assigned.

Definition

Individual commands are defined by classes exported via the ExportCommand at-tribute. These classes must implement the IExtendedCommand interface, whichoffers the following methods:

58

• Execute – Performs the action represented by the command.

• CanExecute – Returns if the command can be executed in the current stateof the application. Commands that cannot are displayed as grayed-out inthe menu and toolbar.

• IsVisible – Returns if the command is visible in the current state of theapplication. Commands that are not cannot be executed and are not dis-played in the menu and toolbar. This is used for commands that makesense only in certain contexts (e.g. while debugging).

• IsChecked – Returns if the command is activated in the current state of theapplication. This only affects how the corresponding menu item and tool-bar button are rendered. Activated commands are displayed with a checksymbol or with a highlight. This is used for commands that toggle variousoptions to offer visual feedback to the user.

The ExportCommandAttribute that is used to export individual commands alsohas several properties that define how the command is represented in the userinterface:

• Name – The name that is displayed in the menu and toolbar.

• Icon – The icon that is displayed in the menu and toolbar.

• Description – The description that is displayed in the tooltip and statusbar.

• Shortcut – The default keyboard shortcut that can be used to invoke thecommand.

Menu

In order for the command to have its representation in the menu, it must beannotated with the MenuCommand attribute, which is used to put the command tothe correct place within the menu. The attribute has three properties:

• Parent – Indicates in which menu the command will be placed. It can beany class exported via the ExportMenuCategory attribute.

• Group – Indicates the name of the group in which the command will beplaced. Different groups within the same parent are separated visually.

• Order – Indicates the order of items within the same group.

Additional entries not corresponding to any commands can be also placed inthe menu. Such entries inherit from FrameworkElement and are exported via theExportMenuItem attribute, which is used in the same way as the MenuCommand

attribute.

59

Toolbar

In order for the command to have its representation in the toolbar, it must beannotated with the ToolbarCommand attribute, which is used to put the commandto the correct place within the toolbar. The attribute has two properties:

• Group – Indicates the name of the group in which the command will beplaced. Different groups are separated visually.

• Order – Indicates the order of items within the same group.

Additional entries not corresponding to any commands can be also placed inthe toolbar. Such entries inherit from FrameworkElement and are exportedvia the ExportToolbarItem attribute, which is used in the same way as theToolbarCommand attribute.

4.4 Editor

Editor is a primary component of the application that adds the ability to openand edit text files. The component is implemented in two projects:

• ExtBrain.ScriptDevelop.Plugin.Editor – Contains basic document man-agement infrastructure. Provides the concept of documents, their formatsand editors and it manages currently open documents.

• ExtBrain.ScriptDevelop.Plugin.Editor.Text – Contains functionality spe-cific for text editors. Defines a text document and its editor and offers a lotof additional services for them, for example highlighting, tooltips, or contextmenus.

The user interface provided by this component is not tied to C# in any way andcould be used as a generic text editor or as an editor for other programminglanguages as well.

4.4.1 Document infrastructure

The singleton EditorPlugin is the central point of the Editor component. Itcontains methods for creating, opening, saving and closing documents. It man-ages the list of currently open documents and keeps track of which document iscurrently active (focused) in the application.

Documents open in the application are represented by instances of theIEditorDocument interface. This interface provides a lot of read-only propertiesthat expose various information about the document, for example its ID, name,path, format, version, and flags (e.g. if it is visible, active or modified). It alsocontains events that are invoked when some of this information is changed.

60

4.4.2 Extensibility

Documents in the application can be extended using the extension plugin mech-anism described in section 4.2.1. The IEditorDocument interface inherits fromIExtension and contains the GetExtension method that is used to access variousextensions of the document.

There are two helper interfaces for exporting plugins that register extensionsto specific document types:

• IEditorExtensionPlugin<TFormat, TDocument, TExtension>

This will add an extension of type TExtension to all documents of typeTDocument and format TFormat. The extension is created using the followingfactory method:

TExtension InstallExtension(TDocument target);

This method is called every time a document of the correct type has beenadded into the application. The extension returned by this method canthen be accessed using the GetExtension method of the IEditorDocument

interface. If TExtension implements IDisposable, its Dispose method iscalled when the document is closed.

• IEditorExtensionPlugin<TFormat, TDocument>

This is a similar interface, but it is used if we only want to be notified whena document of type TDocument and format TFormat has been added into theapplication and we do not want to add any new extension to the document.Every time a document of the correct type is created, the following methodis called:

void InstallExtension(TDocument target);

This extension mechanism for documents is heavily used throughout the applica-tion to add functionality to individual open documents.

4.4.3 Highlights

The highlight mechanism is the most important extension of text documents. Itallows highlighting certain segments of text and giving them a different look (e.g.altering its background color) or functionality (e.g. assigning it a tooltip text).This mechanism is used for example to highlight errors and issues in the editedsource code.

The highlighting is provided by the HighlightManager document extension,which contains methods for creating, deleting and retrieving highlights. The po-sition of highlights is automatically adjusted as the text of the document changes.If the text is changed in such a way that the highlight disappears (its length isreduced to zero), the highlight is removed from the document.

A single highlight is represented by the IHighlight interface, which containsproperties for accessing the highlight’s position, events that signal when thehighlight’s position is changed, and a method for deleting the highlight. The

61

IHighlight interface also inherits from the IExtensible interface, which meansthat highlights themselves can be also extended and contain the GetExtension

method that is used to access their extensions.

Rendering

The most important function of highlights is that they can change the way thetext is rendered. This is achieved using the HighlightRenderingInfo extension,which allows setting the highlight’s renderer. The renderer is an instance ofthe IHighlightRenderer interface, which contains methods that alter the way thehighlighted portion of the text is rendered. There are several IHighlightRendererimplementations:

• SimpleHighlightRenderer – Allows simple overriding of the text’s color ordrawing a rectangle around the highlighted text.

• GrayOutHighlightRenderer – Adds transparency to the highlighted text,making it appear grayed-out.

• DottedUnderscoreHighlightRenderer – Adds a dotted line under the text.

• WavedUnderscoreHighlightRenderer – Adds a waved line under the text.

• HintHighlightRenderer – Adds a short line under the beginning of the text.

Highlights that have no renderer associated will not be visible in the text, butcan have other useful functions. The actual rendering is implemented in theHighlightRenderingManager class, which is a document extension that overrideshow the text is rendered based on existing highlights.

Tooltip

Highlights can have a tooltip associated. This tooltip is then displayed when theuser hovers the mouse cursor over the highlighted text. The tooltip of a highlightcan be set using the HighlightTooltipInfo extension in form of an ITooltipData

instance, which defines the content and priority of the tooltip.

Shortcut

The shortcut column is the column on the editor’s right side, beyond the scroll bar.It displays small colored horizontal bars that represent highlights. The column’swhole height represents the entire file, so bars at the top indicate highlights atthe beginning of the file and bars at the bottom indicate highlights at the endof the file. Hovering the mouse cursor over the bar will display the highlight’stooltip, if any, and clicking the bar will scroll the text cursor to the highlight’slocation. The color of the highlight’s bar in the shortcut column can be set usingthe ShortcutBarInfo extension.

62

Icon

Highlights can have an icon associated. This icon is then displayed to the left ofthe line on which the highlighted text begins, in the column on the editor’s leftside. The icon is a UI element with which the user can interact. This is used forexample for breakpoint highlights that show a red circle which can be clicked.The icon can be set using the IconBarInfo extension.

Context actions

Highlights can have one or more context actions associated. When the text cursoris placed on a location where highlights with context actions are available, an iconis displayed at the beginning of the line. Clicking this icon (or pressing Alt +

Enter) will open a menu with the available actions. Context actions of a highlightcan be set using the ContextActionInfo extension in form of IContextAction

instances, which provide the appearance and callbacks of the context actions.


The Editor component also implements the core user interface and the underlyinginfrastructure required for code completion. The code completion UI is based onthe completion window contained in the AvalonEdit library we use for the editorcontrol, as described in section 3.5.3.

The ExtBrain.ScriptDevelop.Plugin.Editor.Text.Completion namespacecontains classes and interfaces that allow various levels of access to the codecompletion functionality:

• The CompletionWindowExtensions class contains extension methods forAvalonEdit’s TextEditor component. These extension methods help keeptrack of the code completion window associated with the editor compo-nent. At this lowest level, the completion window can be any subclass ofAvalonEdit’s CompletionWindow.

• The CustomCompletionWindow derives from AvalonEdit’s CompletionWindowand adds some of the features not present in the original completion window(inactive selection, right column content, automatic widening).

• The CompletionWindowManager can be attached to any TextEditor and con-tains methods to open the completion window over the given text segmentwith the given list of items. It also allows to open the completion windowasynchronously, i.e. to run a delegate which returns the list of items in thebackground, open the window when it is done, and ensure that the previousasynchronous task, if still running, is canceled when a new one is started.

• The CompletionManager is the highest level of access to the code completionwindow that our core implementation provides. It can be attached to anyTextEditor and implements a pluggable infrastructure for item sources,their order (sorting), and the completion window invocation mechanisms.

63

This implementation of the code completion functionality is universal and canbe potentially used for other purposes than C# code completion. We use it forexample to implement code completion also for header sections described in 3.4.1.It can be also used with any TextEditor instance, not just the one that is thepart of the ITextEditorDocument, so for example a debugger console can also offercode completion, even though it does not represent a source code document openwithin the application.

List items

To represent individual items for the code completion list, the originalCompletionWindow class uses instances of the ICompletionData interface. TheCustomCompletionWindow subclass uses its own ICompletionEntry interface andinternally uses an adapter class to make it compatible with AvalonEdit’sICompletionData. Our own ICompletionEntry looks like this:

public interface ICompletionEntry

{

string Identity { get; }

string Text { get; }

object Icon { get; }

object LeftColumnContent { get; }

object RightColumnContent { get; }

object Description { get; }

bool Visible { get; }

bool Preselected { get; }

bool TriggerCompletionUpon(string text,

char ch);

void Complete(TextArea textArea,

ISegment completionSegment,

EventArgs insertionRequestEventArgs);

}

The Identity property uniquely identifies a completion item and is used to pre-select previously used items in a new completion window. The Text property isused for filtering entries during typing.

The Icon, LeftColumnContent, RightColumnContent, and Description proper-ties define the visual aspects of the completion item. Icon should be a WPF’sImageSource instance. Other three can be either a string, a WPF’s UIElement,or a HighlightedString, which is our implementation of a string with additionalformatting (e.g. font color or weight). Description is displayed in a tooltip nextto the completion window when the item is selected.

The TriggerCompletionUpon method is called when a new character is typedto determine if we should complete the selected item first (e.g. in C# completion,this returns true for all characters that cannot be a part of an identifier). TheComplete method is then called to perform the actual completion action.

64

Data source

The code completion window can be invoked using the ShowCompletion methodof the CompletionManager class. When this method is called, an instance of theCompletionInvocationContext class is created. This instance contains variousinformation about the circumstances of the invocation:

• The parent CompletionManager instance.

• The document and position within it where the completion was invoked.

• The level of the completion.

• The context object provided by the CompletionManager instance.

• The trigger object passed into the ShowCompletion method.

Then all implementations of the ICompletionSourceProvider instance that wereimported by MEF have their GetCompletionSources method called:

[InheritedExport]

public interface ICompletionSourceProvider

{

IEnumerable<ICompletionSource> GetCompletionSources(

CompletionInvocationContext context);

}

The result is a list of ICompletionSource instances, which are defined like this:

public interface ICompletionSource

{

IEnumerable<ICompletionEntry> GetEntries(IDocument document,

int offset,

out bool inactive);

ISegment GetCompletionSegment(IDocument document, int offset);

}

Upon each such instance, the GetEntries method is called in a backgroundtask to gather items for the code completion list. When this call finishes, theGetCompletionSegment method is called to obtain the suggested completion textsegment upon which the code completion window should be invoked. Finally, thecode completion window is displayed.

Order

The order of the items in the code completion list can be modified by implement-ing the ICompletionComparisonProvider interface:

[InheritedExport]

public interface ICompletionComparisonProvider

{

IEnumerable<Tuple<double, Comparison<ICompletionEntry>>>

GetCompletionComparisons(

CompletionInvocationContext context);

}

65

The GetCompletionComparers method is called on each such instance known viaMEF. The result is a list of comparisons (Comparison<ICompletionEntry> dele-gates) where each comparison has a double priority attached to it. The compar-ison with the highest priority is used first when sorting, and a comparison withlower priority is used only if all the higher-priority comparisons returned zero (i.e.found the compared items equal for the purposes or sorting).

Triggers

The last aspect of the code completion that the core infrastructure takes careof is the mechanism of invoking the code completion window via so-called trig-gers. When the CompletionManager is created, it calls the GetCompletionTriggersmethod of every known implementation of the ICompletionTriggerProvider in-terface:

[InheritedExport]

public interface ICompletionTriggerProvider

{

IEnumerable<IDisposable> GetCompletionTriggers(

CompletionManager manager);

}

This call is intended to register event handlers that will trigger the code com-pletion window. The handlers can be unregistered in the Dispose method of thereturned objects, which is called when the CompletionManager itself is disposed.

The only universal trigger for code completion is the CodeCompletionCommand,which is by default mapped to the usual Ctrl+Space keyboard shortcut. If thecompletion window is already displayed, this command invokes it again, increas-ing the current code completion level. Other triggers can be implemented to in-voke the code completion window upon special circumstances, for example whentyping an identifier or after a dot character has been typed.

Triggers can pass additional information in a custom “trigger” object wheninvoking the window. This object is then available in the Triggers property of theCompletionInvocationContext instance. If multiple triggers invoke the windowas a reaction to a single event, the window is invoked only once, and the Triggersproperty contains multiple objects.

4.4.5 Other services

There are other services that the Editor component provides, most importantly:

• LocationTrackerPlugin – A plugin that allows defining locations in textdocuments and notifies subscribers how these locations change when doc-uments are opened, modified and closed. It is used for example in “findreferences” results window so that individual results would point to correctlocations even when the source code has been modified after the search wasperformed.

• TooltipManager – A text document extension that allows registeringITooltipProvider instances to the document. It detects when the mouse

66

cursor is hovering over a portion of the text and then requests a tooltipfrom the registered providers. This is used for example in:

– ResolverTooltipPlugin – Detects what symbol is under the mousecursor and displays its declaration and summary.

– DebuggerTooltipPlugin – Detects what variable is under the mousecursor and displays its value when debugging.

• MarginManager – A text document extension that allows creating additionalmargins (rows and columns) around the editor control. It is used for ex-ample to create the icon column for highlights to the left of the editor, theshortcut column for highlights to the right of the editor, or the navigationbar at the top of the editor.

4.5 Builder

Builder is a library that the Compiler component calls to actually compile C#

scripts into binary assemblies. It parses the header commands (see section 3.4.1)in the source code and runs the actual C# compiler accordingly. It does notdepend on the rest of the application and it can be used on its own, for exampleit could be easily made into a separate command-line tool for building C# scripts.We have discussed the Builder library in section 3.4.2. It is implemented in theExtBrain.ScriptDevelop.Builder project.

Interface

The central point of the Builder library is the BuildService class, which containsthe following method:

BuildOutput Build(BuildInput input);

The BuildInput and BuildOutput classes represent input and output data of thebuild process. The BuildInput class has an interface like this:

public class BuildInput

{

public BuildMode BuildMode { get; set; }

public IList<ProjectInput> Projects { get; }

public IDictionary<string, byte[]> ReplacementFiles { get; }

}

The input properties have the following purpose:

• BuildMode – One of the following values which indicate what exactly willbe performed and what information will be returned or generated (see ta-ble 4.2):

– Parse – Only header sections will be parsed and the project structureinformation will be returned without actually invoking the compiler.

67

– Check – The project will be checked for compilation errors withoutactually generating full output.

– Normal – Normal compilation will be performed.

– Debug – Normal compilation will be performed, but debugging infor-mation will be included to the output.

Projectstructure

Listoferrors

Outputassembly

Debug

inform

ation

BuildMode.Parse ✓

BuildMode.Check ✓ ✓

BuildMode.Normal ✓ ✓ ✓

BuildMode.Debug ✓ ✓ ✓ ✓

Table 4.2: Information returned by the Builder based on BuildMode.

• Projects – Specifies projects to be built. An input project is identified bythe path to the script file.

• ReplacementFiles – Allows specifying replacement content for selected files.This is useful when we want to perform Parse or Check builds with files thathave been modified locally but not yet saved on the filesystem.

The BuildOutput classes look like this:

public class BuildOutput

{

public IList<BuildError> Errors { get; }

public IList<ProjectOutput> Projects { get; }

}

public class BuildError

{

public string Text { get; }

public ProjectOutput Project { get; }

public Location Location { get; }

}

public class ProjectOutput

{

public string Path { get; }

public ProjectInput ProjectInput { get; }

public IList<BuildError> Errors { get; }

public IList<string> SourceFiles { get; }

public IList<string> ReferencedAssemblies { get; }

public IList<string> NeighbouringAssemblies { get; }

68

public IList<ProjectOutput> ReferencedProjects { get; }

public IList<ProjectOutput> DependentProjects { get; }

public IList<ProjectOutput> AdditionalProjects { get; }

public string OutputAssemblyPath { get; }

public string OutputAssemblyName { get; }

public ITarget Target { get; }

}

The most interesting of the output classes is ProjectOutput containing infor-mation about the project structure, which is essential later when performingsyntactical and semantical analysis of the source code using NRefactory. Theinteresting information contained in ProjectOutput includes:

• SourceFiles – A list of source code files that this project consists of.

• ReferencedAssemblies – A list of external assemblies referenced by thisproject.

• NeighbouringAssemblies – A list of assemblies referenced either directly bythis project, or transitively by any of its dependencies. This list indicateswhat assemblies must be copied to the output directory with this project’soutput assembly so that all dependencies are present.

• ReferencedProjects – A list of other projects that this project references.

• DependentProjects – A list of other projects referencing this project.

• AdditionalProjects – A list of other projects whose build process was in-duced by this project via a dedicated BUILD header command.

• Target – Information about this project’s compiler and framework version.

Implementation

The actual build process of a single project is implemented as a series of individualsteps that can be found in the ExtBrain.ScriptDevelop.Builder.Processing.

BuildSteps namespace. These steps have a predefined order and gradually collectdata or perform operations necessary to build the project. The most importantsteps include (in this order):

1) ParseStep – Iterates over all source code files of the current project andscans them for header sections. Parsing of header sections is done by theHeaderParserService class, which can parse a single source code file and re-turn the result in form of a list of IParseResult instances. The IParseResult

instance serves to separate the result of parsing for the purposes of bettererror messaging and potential in-memory caching (we can cache the list ofIParseResult instances if the source code file has not been modified). Parsingcontinues recursively on all newly added source code files (which can be addedby header commands) until no more such files appear.

69

2) BuildReferencesStep – Iterates over all referenced projects and recursivelybuilds them first before continuing. This step also detects potential projectdependency cycles.

3) LocateReferencesStep – Tries to locate the full path of assemblies that werereferenced only by their name but the corresponding file could not be found.This is mostly the case of framework assemblies, e.g. System.Core.dll. The ex-act location of these assemblies depends on the current framework representedby the project’s ITarget, which provides an instance of the IAssemblyLocator

interface:

public interface IAssemblyLocator

{

string TryLocateAssembly(string name);

}

Its implementation, the AssemblyLocator class, locates the assemblies bysearching the installation directory of the .NET framework of the correspond-ing version.

4) ReplaceSourcesStep – Writes the source code files that are modified in theeditor into a temporary location so that they could be handed over to the C#

compiler instead of the original unmodified source code files.

5) CheckResourcesAgeStep – Checks last write times of the input files and com-pares them to the last write time of the output assembly, if it already exists,in order to find out if it is necessary to recompile.

6) DetectNeighbouringReferencesStep – Checks which assembly files the currentproject depends on (assemblies referenced either directly by this project, ortransitively by any of its dependencies). This list indicates what assembliesmust be copied to the output directory with this project’s output assemblyso that all dependencies are present. To find out what assemblies anotherassembly references, we use this call:

Assembly.ReflectionOnlyLoadFrom(path).GetReferencedAssemblies();

However, since this loads the assembly into our application domain and thusprecludes doing the same again when the assembly changes, we must do itin a separate application domain via remoting. This is implemented in theAssemblyReflector class.

7) CompileStep – Runs the actual C# compiler. Since the .NET framework con-tains wrappers for the compiler, we use those wrappers instead of callingcsc.exe manually. The compiler is invoked by a call to CSharpCodeProvider.

CompileAssemblyFromFile.

8) CopyNeighbouringReferencesStep – Copies the assemblies detected during theprevious DetectNeighbouringReferencesStep to the output directory. Onlynon-framework assemblies are copied (i.e. assemblies that did not have to beresolved in the LocateReferencesStep using IAssemblyLocator).

9) OutputStep – Prepares the ProjectOutput structure.

70

4.6 Compiler

Compiler is a primary component whose core responsibility is to integrate therest of the application with the Builder library. It implements commands to buildand run scripts, monitors the currently open scripts and runs the build processautomatically in order to display errors and keep the project structure informationup to date. It also defines the C# document format and implements basic C#-related functionality that does not require code-aware insight into the source code.It is implemented in the ExtBrain.ScriptDevelop.Plugin.Compiler project. Thecore interface to the Builder library is encapsulated in the BuildPlugin, whichlooks like this:

public class BuildPlugin

{

event EventHandler<BuildEventArgs> BuildStarted;

event EventHandler<BuildEventArgs> BuildStopped;

void Parse(string path);

void Check(string path);

void Build(string path, bool debug, bool full,

Action<BuildOutput> callback);

void Cancel();

}

Errors

The application’s error highlighting and reporting functionality is implemented inthe ExtBrain.ScriptDevelop.Plugin.Compiler.Errors namespace. This names-pace contains plugins which:

• Run the build process in the Check mode (see section 4.5) on the scriptsopen in the application in order to update the list of errors. If enabledby the user, this process is run automatically each time the source code ismodified (ErrorCheckRunningPlugin, ErrorCheckStatusPlugin).

• Manage a repository of currently present errors (ErrorRegistryPlugin,ActiveErrorListPlugin).

• Present the current error check status and the list of errors to the user(ErrorPanelPlugin, ErrorStatusBarIndicatorPlugin).

• Display errors as red underlines in the editor (ErrorHighlightManager).

Project structure

The project structure is information about what projects are open within the app-lication, what are their external references, and how they reference one another.This information is essential for code-aware functionality because it defines whatmust be included in the syntactic and semantic analysis done by NRefactory. The

71

project structure depends on header sections within the script files. These headersections are parsed by the Builder library and the project structure informationis returned in the ProjectOutput objects described in section 4.5.

The project structure repository is managed by classes in the ExtBrain.

ScriptDevelop.Plugin.Compiler.ProjectStructure namespace. It is representedby the following two interfaces:

public interface ISourceInfo

{

string Path { get; }

IProjectInfo Project { get; }

IProjectInfo TopLevelProject { get; }

}

public interface IProjectInfo

{

string Path { get; }

IProjectInfo TopLevelProject { get; }

IList<string> GetSources();

IList<string> GetReferences();

IList<IProjectInfo> GetReferencedProjects();

IList<IProjectInfo> GetDependentProjects();

...

}

ProjectStructurePlugin is the plugin that implements the project structurerepository. It has the following interface:

public class ProjectStructurePlugin

{

event EventHandler<ProjectEventArgs> ProjectAdded;

event EventHandler<ProjectEventArgs> ProjectRemoved;

event EventHandler<ProjectUpdatedEventArgs> ProjectUpdated;

IEnumerable<IProjectInfo> Projects { get; }

void Update(string path, BuildOutput output);

IProjectInfo GetProjectInfo(string projectPath);

ISourceInfo GetSourceInfo(string sourcePath);

}

The ProjectParsingPlugin is a plugin that monitors document changes and runsthe build process in the Parse mode to keep ProjectStructurePlugin up to date.

72

4.7 NRefactory

NRefactory [6] is a C# library that allows parsing and semantic analysis of theC# source code. It is also the name of one of the primary components of ourapplication. The NRefactory component is responsible for all of the application’scode-aware functionality. It bears the same name as the library because it usesit heavily.

4.7.1 Core

The NRefactory core is a subsystem of our application that monitors changes tothe source code and keeps its representation provided by the NRefactory libraryup to date, so that if any other part of the application requests the data fromNRefactory (e.g. an AST, a compilation, a resolver, etc.), it gets it right away.

We have analyzed the requirements for the NRefactory core subsystem insection 3.5.2. The subsystem is implemented in the ExtBrain.ScriptDevelop.

Plugin.NRefactory.Core project. This section describes how it is implemented.

Volatile resources

A volatile resource is a resource that changes over time and allows us to retrieveits current value and check its current version. It is represented by the followinginterface:

public interface IVolatileResource<T>

{

bool CheckIsCurrent(object version);

Tuple<T, object> LoadCurrent();

}

The LoadCurrent method retrieves the current value and version of the resourceas a Tuple where the first item is the value of the resource and the second item isa version token representing the version associated with the returned value. Theversion token can be passed to the CheckIsCurrent method in order to check ifthe corresponding value is still up to date. If this method returns false, we canuse LoadCurrent again to retrieve the updated value.

It is possible for a resource to become invalid (for example if it represents a filethat has been deleted). In this case, the LoadCurrent method returns always nulland CheckIsCurrent returns always false.

There is a helper class VolatileResourceSnapshot<T> that represents a snap-shot of the volatile resource passed to its constructor. It allows easy access tothe value of the resource at the time of the snapshot’s creation, and containsa method to check if the snapshot is still up to date. Because NRefactory struc-tures are immutable, this class is internally used as a base class for objects thatrepresent the output of NRefactory based on the value of the snapshot.

73

Resource loaders

Resource loaders are singleton plugins that act as a factory for IVolatileResourceinstances representing source code files and assembly files.

• AssemblyLoaderPlugin is a single point to efficiently load external assembliesinto NRefactory. It contains a single public method GetAssemblyResource

(string assemblyPath) returning an instance of the IVolatileResource

<IUnresolvedAssembly> interface representing the given assembly.

• SourceLoaderPlugin is a similar plugin for source code file loading. Itcontains a public method GetSourceResource(string sourcePath) that re-turns an IVolatileResource<IDocument> instance representing the givenfile. However, source code files can be opened in the application, mod-ified by the user and not yet saved. In this case, we want to usethis modified content of the source code file for NRefactory processingrather than its original content on the filesystem. For this purpose,the SourceLoaderPlugin contains a method NotifySourceDocumentChanged

(string sourcePath, IDocument document) that allows substituting thecontent of individual files. This method is called each time a source code fileis modified in the application and the updated text of the source code fileis passed into it. This way, internal source code modifications are includedseamlessly into the version checking mechanism of IVolatileResource.

Project representation

The NRefactory core subsystem monitors the structure of the C# scripts thatare open in the application and keeps an up-to-date IProjectContent instancefor each project. We have described what we need for project representation insection 3.5.2. For the purpose of the NRefactory library, a project consists of:

• A path that acts as an identifier of the project.

• A list of source code files represented as paths.

• A list of references. A reference may be either a path to an external assem-bly, or a path to another project.

Each of the resources that make up a project (source code file, external as-sembly, another referenced project) can change over time, so we use theIVolatileResource and VolatileResourceSnapshot concepts described above.

A project is represented by the Project class. Because IProjectContent isimmutable and represents the state of a project at a certain time, the Project classimplements IVolatileResource<IProjectContent>. The three types of resourcesthat make up a project are represented like this:

• A source code file is identified by its path, represented as IVolatileResource<IDocument>, and its specific version is represented as an immutableSourceSnapshot class (subclass of VolatileResourceSnapshot<IDocument>).Upon construction of a SourceSnapshot, the file content is parsed and con-verted into an IUnresolvedFile, which is then used to represent the file in-side the IProjectContent of the current project. The SourceSnapshot class

74

also exposes the source file’s SyntaxTree (the parsed AST) and IDocument

(the source code text), which are useful to the rest of the application.

• An assembly is identified by its path, represented as IVolatileResource

<IUnresolvedAssembly>, and its specific version is represented as an im-mutable AssemblyReference class (subclass of VolatileResourceSnapshot

<IUnresolvedAssembly>). The AssemblyReference class implements theIAssemblyReference interface and because of this it can be used to rep-resent a reference inside the IProjectContent of the current project.

• An internal project reference is identified by its path, represented as itsown Project class (implementing IVolatileResource<IProjectContent>),and its specific version is represented as an immutable ProjectReference

class (subclass of VolatileResourceSnapshot<IProjectContent>). TheProjectReference class also implements the IAssemblyReference interfaceand because of this it can be used to represent a reference inside theIProjectContent of the current project.

The Project class keeps the original IVolatileResource instances for its re-sources, checks is they are up to date, and if they are found to be out of date, itloads their current version and updates the IProjectContent accordingly.

The loading process can be expensive – source code files need to be loaded andparsed, assembly files need to be loaded, and then in both cases they need to beconverted into an NRefactory type system. Because of this, the loading processis done asynchronously. The Project class keeps track of which of its resourcesare scheduled for loading, and if the same resource is updated or removed whileit is already scheduled for loading, the corresponding loading task is canceled.

The IVolatileResource does not support notifications (events) to signal thatthe resource has been changed. Instead it needs to be polled explicitly. Whenthis is done depends on the type of the resource:

• If the application has been deactivated, all resources of the project arechecked for modifications when the application becomes active again.

• A source code file is checked for modifications after it has been modifiedin the application’s editor. This check always results in reparsing of thesource code file, so it is performed after a slight delay which is reset each timeanother modification occurs. This way, the parser does not run continuouslyafter each keystroke, but rather only when there is a pause in the user’styping.

• Each time the current project’s IProjectContent is modified, this changeneeds to be reflected in all other projects that referenced the current project.Because of this, the Project class keeps track of other projects that referenceit and notifies them whenever the current IProjectContent changes. Thereferencing projects then update their own IProjectContent accordingly,potentially cascading the update further down the dependency graph.

75

Interface

The purpose of the NRefactory core subsystem is to keep up-to-date informationabout the syntax trees and the type system of projects active in the application.Other components of the application access this information in order to performtheir specific functionality.

When another component requests information about a certain source codefile, it is retrieved in the form of the ISourceSnapshot instance:

public interface ISourceSnapshot

{

string SourcePath { get; }

IDocument Document { get; }

SyntaxTree SyntaxTree { get; }

CSharpUnresolvedFile UnresolvedFile { get; }

}

The instance and all its parts are immutable and represent the state of the sourcecode file as it had been registered within the project at the time of retrieval.Internally, this interface is implemented by the SourceSnapshot class that hasbeen described above. The SyntaxTree property is the most interesting for othercomponents because it gets the source code file’s parsed AST.

Often we also need to access the type system of the entire project, not justa single source code file. This is required for creating a resolved type system andfor running the resolver on the AST in order to find out its semantic meaning.The type system for the entire project is represented by the IProjectSnapshot

interface:

public interface IProjectSnapshot

{

string ProjectPath { get; }

IProjectContent ProjectContent { get; }

ICompilation Compilation { get; }

}

As with ISourceSnapshot, the instance and all parts of IProjectSnapshot areimmutable and represent the state of the project at the time of retrieval.

Very often, both the information about a source code file and its projectis required. We allow retrieval of both at the same time in the form of anICombinedSnapshot interface, which inherits from both ISourceSnapshot andIProjectSnapshot. It is ensured that the returned version of IProjectContent

contains exactly the returned version of IUnresolvedFile.Sometimes we also need to make sure that the returned snapshot corresponds

exactly with the current state of the project. That may not always be the case –the subsystem may be waiting for the user to finish typing, or the parsing may bein progress but not yet complete. In case we need to use the most recent informa-tion, the methods for snapshot retrieval contain an optional boolean parameterforceCurrent that will ensure the most up-to-date information is returned. Itis achieved by checking the given source code file’s version first and potentiallywaiting until it is parsed. That means the call may be blocking if forceCurrentis used.

76

It is also useful to know when the parsed information about a certain sourcecode file or project has been updated. This is signaled by the ProjectUpdated

event, which offers detailed information about the update in its argument objectProjectUpdatedEventArgs.

The rest of the application has two options how to access the NRefactory coresubsystem:

• The first option is to use the NRefactoryPlugin singleton, which containsall the necessary methods and events to access the information from theparser:

– GetSourceSnapshot, GetProjectSnapshot and GetCombinedSnapshot

methods return for the given path the snapshot interfaces describedabove.

– GetProjectSnapshots method returns an enumerable of snapshots forall tracked projects.

– ProjectUpdated event is triggered each time the information abouta project is updated.

• Another option is via the NRefactoryExtension, which is an extension plu-gin of all documents the format of which is of the CSharpDocumentFormat

type. Because this extension is always linked with a specific document, itsmethods do not need to be passed the path to the source code file. TheNRefactoryExtension can be accessed like this:

var extension = document.GetExtension<NRefactoryExtension>();

The NRefactoryExtension exposes the following members:

– GetSourceSnapshot, GetProjectSnapshot and GetCombinedSnapshot

methods that return the snapshot interfaces for the associated doc-ument.

– SourceUpdated event that is triggered whenever the information aboutthe associated document is updated.

– ProjectUpdated event that is triggered whenever the information aboutthe project of the associated document is updated.

The two events of this extension are particularly useful for other documentextensions that depend on the parsing information. For example:

– Code folding depends only on the AST of the given document, so it isupdated in the SourceUpdated event.

– Syntax highlighting depends on the whole project (because it needsto resolve types, for instance), so it is updated in the ProjectUpdated

event.

77

The NRefactoryPlugin singleton contains a number of methods that notifyit about changes in the project structure. This is because the core functional-ity of the NRefactory core subsystem is designed to be stand-alone and not todepend on the rest of the application. Therefore, the subsystem needs to benotified whenever there are changes that affect it. This is done by a numberof simple additional plugins that observe the state of the application and no-tify the NRefactoryPlugin accordingly. These plugins reside in the ExtBrain.

ScriptDevelop.Plugin.NRefactory.Core.Connectors namespace.

4.7.2 Code analysis

The code analysis subsystem is a component that uses the NRefactory library toanalyze source code files that are open in the application, to scan them for variousissues, and to offer context-sensitive actions and refactoring transformations. Thediscovered issues and actions are then presented to the user, who can reviewthe issues and invoke the actions, including actions that are aimed to fix thediscovered issues.

• Code issues are presented as text highlights of various intensity (e.g. thecode that is detected to be unreachable or redundant is displayed in gray).

• Code actions pertaining to the current caret position are indicated byan icon to the left of the current line. When this icon is clicked, a popupmenu that lists the available actions is opened (or it can be opened by press-ing Alt+Enter, to avoid having to use the mouse). When an action fromthis menu is selected, it is executed. The actions in the menu can comefrom two different sources:

a) Code action analysis, which specifically looks for available code actionsrelevant to the current caret position and text selection.

b) Code issue analysis, where each issue can have a list of actions associ-ated with it. These actions are usually meant to fix the issue is someway (e.g. by removing the redundant code), and they are offered to theuser when the caret position is located inside the text segment markedby the issue.

In fact, any text highlight can have context actions associated with it thatcontribute to the context action menu (see section 4.4.3), but here we areinterested only in actions resulting from code analysis.

Code issue analysis and code action analysis are run only on open documents.They are both triggered on a document when the type system information aboutthe project that contains the document is updated (i.e. when any file inside theproject or dependent projects is reparsed). Because code action analysis dependsalso on the current caret position and text selection, it is additionally triggeredwhenever the caret position or text selection within the document changes.

The analysis is run on a document only when the document is visible in theapplication. If the document is not visible and the circumstances occur that trig-ger the analysis, it is postponed until the document becomes visible again. Thisis to avoid unnecessary overhead because there is no need to analyze documentsthat the user cannot see.

78

Implementation

The core functionality of the code analysis subsystem is implemented in theExtBrain.ScriptDevelop.Plugin.NRefactory.Analysis project. The main classthat manages analysis on a single document is CodeAnalysisManager, which isan extension of C# documents. This class monitors circumstances that triggerthe analysis, performs it and processes its results.

Individual code issues and code actions are detected by classes implementingthe ICodeIssueProvider or ICodeActionProvider interfaces. These interfaces areexported using MEF and queried by the CodeAnalysisManager. They are definedas follows:

public interface ICodeAnalysisProvider

{

ICodeAnalysisProviderMetadata Metadata { get; }

}

[InheritedExport]

public interface ICodeActionProvider : ICodeAnalysisProvider

{

IEnumerable<ICodeAction> GetActions(

CustomRefactoringContext context

);

}

[InheritedExport]

public interface ICodeIssueProvider : ICodeAnalysisProvider

{

IEnumerable<ICodeIssue> GetIssues(

CustomRefactoringContext context

);

}

The parent interface ICodeAnalysisProvider exposes various metadata aboutthe analysis provider (title, description, category). The only properties of theICodeAnalysisProviderMetadata interface that are currently important are:

• Identity – Gets unique string identifier of the analysis provider (for internalusage).

• ConfigurationNames – Gets string identifiers that can be used to refer tothis analysis provider in the configuration.

When code analysis is triggered, all currently imported instances of theICodeActionProvider and ICodeIssueProvider interfaces are gathered and theirGetActions and GetIssues methods are called in order to find code issues andcode actions that are present in the current version of the document. The me-thods are executed in parallel, but with a limited degree of concurrency so thatthe task scheduler would not be overloaded with too many tasks at once. Whena single method call finishes, previous code actions and issues of the correspond-ing provider (identified by the Identity metadata property) are replaced by thenewly discovered code actions and issues, if there were any.

79

The CustomRefactoringContext object passed into these methods is a subclassof NRefactory’s RefactoringContext class and encapsulates all information re-quired to analyze the document and the type system of the corresponding project.It also provides contextual information for the correct proposal of context codeactions (i.e. the current caret position and text selection).

Individual code actions are represented by the ICodeAction interface:

public interface ICodeAction

{

ICodeActionMetadata Metadata { get; }

string Description { get; }

TextLocation Start { get; }

TextLocation End { get; }

Action<CustomScript> Run { get; }

}

Discovered code actions are transformed into invisible text highlights that onlyhave the corresponding context action assigned. This means that if the userplaces the editor’s caret in the text segment denoted by the code action’s Start

and End properties, the corresponding context action is offered in the contextaction menu and can be selected by the user.

When the user selects the action from the context action menu, we check thatthe current version of the document is the same as the version upon which the codeanalysis generated the action (the versions should be in most cases equal becauseany modification of the document triggers new analysis, but it can happen thatthe action is selected before the new analysis completes and replaces the availableactions). If the versions are equal, we call the Run delegate to actually performthe code action.

The Run delegate receives a CustomScript instance that represents the envi-ronment for refactoring the current document. It is a subclass of the Script

and DocumentScript classes defined in NRefactory. These classes offer variousmethods that allow and aid the refactoring of the current document.

Code issues are represented by a similar interface, ICodeIssue:

public interface ICodeIssue

{

ICodeIssueMetadata Metadata { get; }

string Description { get; }

TextLocation Start { get; }

TextLocation End { get; }

IList<ICodeAction> Actions { get; }

}

The Actions property contains a list of code actions associated with the issue.These actions are usually meant to fix the issue in some way. If the user placesthe editor’s caret in the text segment denoted by the code issue’s Start and

80

End properties, the actions from the code issue are added to the list of currentlyavailable context actions.

Discovered code issues are transformed into text highlights (see section 4.4.3)of various intensity, so that the user would be visually notified that the issues exist.How these highlights look visually is determined by the Marker and Severity

properties of the code issue’s Metadata property.There are two highlight types (two different ways a code issue can be high-

lighted in the editor):

• By rendering the affected code with a color underscore.

• By graying-out the affected code (this usually denotes unreachable or re-dundant code).

The intensity of the highlight is determined by the code issue’s severity:

• Error – The code issue is displayed in red and counted as an error. If thehighlight is of the gray-out type, the affected code is displayed in a red fontinstead.

• Warning – The code issue is displayed in orange and counted as a warning.

• Suggestion – The code issue is displayed in green.

• Hint – The code issue is displayed only as a subtle, short underscore at thebeginning of the affected code, so that it would not distract but was stillnoticeable.

• None – The code issue is ignored and is not displayed in any way.

The highlights that are created to represent discovered code issues also havea tooltip and a shortcut assigned. The Description of the code issue is used asthe tooltip for the highlight, and a shortcut marker is inserted to the right marginof the editor with the color corresponding to the code issue’s severity. This way,the icon to the top right of the editor (above the shortcut margin) can be eitherred, orange or green according to if there are any errors or warnings among thecode issues discovered by the code analysis.

Configuration

Code issue analysis highlights places in the code that are potentially dangerousand is meant to improve code quality. However, not every user considers all issuesequally relevant to his or her coding style or to the given situation. Sometimes theuser may want to ignore a particular issue, either globally or locally in a particularcase, file or region, or lower the severity of a certain issue so that it would forexample not stand out and distract so much.

Local code issue suppression is usually done directly in the source code usinga special directive inside a comment that disables a particular occurrence of theissue, either between a pair of comment directives, or directly after a commentdirective. However, this method of code issue suppression is not supported in thecurrent version of the application.

81

Instead, global code issue suppression is supported via configuration. Usinga special section of the configuration file, the user can change the severity of codeissues, for example like this:

<CodeAnalysis>

<OverrideSeverity Provider="RedundantUsingDirective"

Severity="Suggestion" />

<OverrideSeverity Provider="RedundantThisQualifier"

Severity="None" />

</CodeAnalysis>

Such configuration elements override the severity of the specified code issues.Specifying the severity of None disables the corresponding analysis provider com-pletely so that it is not even run.

The name of the provider must correspond to one of the names availablethrough the ConfigurationNames property of the provider’s metadata. Since thesenames are not used in the application’s user interface, they are specially madeavailable through the context action menu as the last item, if there are currentlyany code issues or code actions available.

Both code issue and code action providers can be configured in this way,however changing severities of code action providers to other values than None inorder to turn them off does not have any useful effect.

The configuration of the code analysis is implemented in the classCodeAnalysisConfiguration that contains a single public method:

Severity? GetOverriddenSeverity(ICodeAnalysisProvider provider)

This method is used by the CodeAnalysisManager class and returns if the givenprovider has any severity configured.

4.7.3 Code analysis by NRefactory

The NRefactory library itself contains a vast number of ready-made code issue andcode action providers. This number still grows as the library is actively developed.We have designed our code analysis subsystem specifically to be compatible withthe code analysis providers of NRefactory, however, we use our own, simpler andmore compact versions of the interfaces for providers, issues and actions. Ourapplication can still use all of the code analysis providers from NRefactory byimplementing an adapter that allows them to be used by our own code analysissubsystem.

The code analysis providers in the NRefactory library use two different ab-stract base classes, CodeIssueProvider and CodeActionProvider. Metadata forthe providers are defined using special attributes, IssueDescriptionAttribute

and ContextActionAttribute, that annotate the non-abstract subclasses imple-menting a particular code issue or action provider. These classes and attributesexpose all the important information and functionality that we need in orderto implement our ICodeIssueProvider and ICodeActionProvider interfaces forthem.

82

Implementation

The adapters are implemented in the project ExtBrain.ScriptDevelop.Plugin.

NRefactory.Analysis.Adapter:

• NRefactoryCodeIssueProviderAdapter – Adapts the CodeIssueProvider ofNRefactory to use our ICodeIssueProvider interface.

• NRefactoryCodeActionProviderAdapter – Adapts the CodeActionProvider

of NRefactory to use our ICodeActionProvider interface.

The project also contains the classes NRefactoryCodeIssueProviderSource

and NRefactoryCodeActionProviderSource. These two classes implement theICodeIssueProviderSource and ICodeActionProviderSource interfaces, which area variant of ICodeIssueProvider and ICodeActionProvider and allow exposingmultiple instances of these interfaces through MEF:

[InheritedExport]

public interface ICodeIssueProviderSource

{

IEnumerable<ICodeIssueProvider> GetProviders();

}

[InheritedExport]

public interface ICodeActionProviderSource

{

IEnumerable<ICodeActionProvider> GetProviders();

}

All the code analysis providers offered by the NRefactory library are con-tained in the ICSharpCode.NRefactory.CSharp.Refactoring assembly. TheNRefactoryCodeIssueProviderSource class searches this assembly for all non-abstract subclasses of CodeIssueProvider with the IssueDescription attribute,instantiates them, wraps them using the adapter, and exposes them via theGetProviders method. The NRefactoryCodeActionProviderSource does the samefor CodeActionProvider subclasses.

Limitations

Although the adapters technically allow us to use all of the code analysis providersfrom NRefactory, not all are supported. The Script object that is passed into thedelegate performing a code action contains additional helper methods for morecomplex refactoring operations (e.g. renaming, new type creation, and other ac-tions that potentially span multiple documents). Such methods are not imple-mented in the current version of the application and code actions that use themare not supported. This is because such operations are more useful for complexprojects and therefore are outside the scope of a C# script editor.

83

4.7.4 Imports

One of the most important code-aware functions of a C# code editor is the abilityto automatically add using statements and references for types that are unknownin the current context. This is mainly done by the code completion, which allowslisting types from yet-unimported namespaces and even from yet-unreferencedassemblies and adding the required using statement or reference upon selectionof the type. However, it is also useful to have an unknown type highlightedin red and offer context actions that will import the appropriate namespace orreference when invoked. This is especially important when copying and pastingcode snippets from another source, where we can immediately see what types werenot recognized in the new context and add correct namespaces and referencesaccordingly.

Please note that “import” is not a common word in the C# nomenclature.When we say “importing an entity”, we mean the process of adding the appro-priate using statement or assembly reference or referring to the entity in sucha way so that it would be correctly recognized. Similarly, by an “import” wemean an entity that requires a new using statement, a new assembly reference,or a special form in order to be recognized.

Not only types may need importing. Other entities that may lose their mean-ing if taken out of context of referenced assemblies or using statements are ex-tension methods and namespaces.

Data source

In order to offer importing of entities (no matter if done via code completion, oras context actions correcting an unknown identifier), we need to gather all knowntypes, extension methods and namespaces and determine if they could be usedat a given position in the source code and under what circumstances.

This process is implemented in the ExtBrain.ScriptDevelop.Plugin.

NRefactory.Import project by the ImportProvider class. Upon construction thisclass is passed the resolver context at the given position within the source codeand the syntax tree of the current file. The class contains only three publicmethods:

public IList<TypeImportResult> GetTypeImports(

bool standAlone,

Func<ITypeDefinition, bool> predicate = null

);

public IList<MethodImportResult> GetExtensionMethodImports(

IType targetType,

Func<IMethod, bool> predicate = null

);

public IList<NamespaceImportResult> GetNamespaceImports(

Func<INamespace, bool> predicate = null

);

84

All three methods return a list of found imports, and take an optional parame-ter predicate that can be used to filter the returned entities early before moreprocessing is performed on them. The targetType parameter denotes the type ofthe expression upon which extension methods are being invoked. The standAloneparameter denotes that the types are not being used after a dot operator (if theyare, we can skip some processing).

The results of these calls are represented by the ImportResult base abstractclass:

public abstract class ImportResult

{

// Assembly (full path) of the imported entity

public string Assembly { get; }

// Namespace (full name) of the imported entity

public string Namespace { get; }

// Is new using statement required to use the entity?

public bool NewUsingRequired { get; }

// Is new reference required to use the entity?

public bool NewReferenceRequired { get; }

// Text that should be used to reference this entity

public string CompletionText { get; }

...

}

If the property CompletionText is not null, it means that the entity cannot bereferred to simply by its name and instead the text in CompletionText needsto be used (this is used for example when we cannot add a new using statementbecause of conflicts or when the type’s name has a different meaning in the currentcontext and so we must refer to the type by its full name). For extension methods,a non-null CompletionText means that we cannot use the instance-member-likeinvocation syntax but instead we must call the extension method as a staticmethod using the string in CompletionText.

Three specific classes derive from this class: TypeImportResult for types,MethodImportResult for extension methods and NamespaceImportResult fornamespaces. NamespaceImportResult is the simplest because namespace usageis not related to using statements and thus only NewReferenceRequired is used.

Conflict detection

When adding new using statements and referring to types in external namespaces,conflicts may arise. The ImportProvider class tries to detect the most commoncases of such conflicts and return results with such parameters that their usagewill not cause new errors. For example, if a new using statement cannot beadded when importing a type because adding the using statement would causeambiguity in the source code, the import’s CompletionText is set to the type’sfull name instead.

85

There are two kinds of conflicts that are detected and prevented by the app-lication:

• When importing a type, ImportProvider tests if the type can be safelyaccessed by its simple name. It does so by looking up the type’s namein the current resolver context to see if the name already denotes anothertype. If yes, then we cannot use the simple name because it would be eitherovershadowed by the already known type, or type ambiguity would arise.In such cases, the full name needs to be used to refer to the imported type.

• When adding a new using statement, ImportProvider tests if such an actionwould not result in ambiguity in the source code.

1) The source code is scanned for constructs that refer to a type by itssimple name. If the referred type is known at that point because ofa using statement, then we need to be careful, because if a namespacewas imported that also contained a visible type with the same signa-ture (simple name and type parameter count), adding such a usingstatement would cause type ambiguity.

2) The source code is scanned for extension method invocations. Whenadding a new using statement, we resolve each of these invocationsagain, but using a resolver state that has been modified to contain thenew namespace import. If any of these invocations result in an am-biguous call, then adding the using statement would cause extensionmethod ambiguity.

Successfully detecting a conflict when adding a new using statement meansit cannot be done and instead the entity in question needs to be referred towithout it:

– A type needs to be referred to by its full name.

– An extension method needs to be invoked as a static method of itscontaining type, which needs to be referred to by its full name.

Import assembly sources

One of the important features when considering imports and code completion isthe ability to offer entities from yet unreferenced assemblies and add the referencesupon request. Because there can be many potential assemblies to offer entitiesfrom, we need a way to find out what assemblies to include in the selection.

For this purpose, there is an interface IImportAssemblySource:

[InheritedExport]

public interface IImportAssemblySource

{

IEnumerable<IUnresolvedAssembly> GetImportAssemblies();

}

86

When imports are gathered by the ImportProvider class, all plugins implementingthis interface are queried and the relevant entities from the assemblies returnedby these plugins are included in the results.

There are currently three implementations of IImportAssemblySource:

• ProjectAssemblySource – Returns assemblies that represent the currentlyopen C# script files.

• UsedAssemblySource – Returns assemblies that are referenced by the cur-rently open C# script files.

• PrecachedAssemblySource – Returns assemblies that were explicitly markedto be precached for the purpose of importing new references. The usercan explicitly choose assemblies that are precached upon the applicationstartup and remain in the cache until the application is closed. This can beconfigured in the PrecachedAssemblies section of the configuration file.

Because ProjectAssemblySource can lead to importing other C# script files, theImportProvider class also checks if the unreferenced assembly from which it con-siders offering entities is not dependent on the current assembly. If yes, suchimports are not offered, because adding a new reference would in this case leadto a dependency cycle.

Import analysis

In addition to offering imports in code completion, we want to have unknowntypes highlighted and offer context actions that will import the correspond-ing types by performing the appropriate refactoring when invoked. We usethe code analysis engine to achieve this by implementing code issue and codeaction providers. The ExtBrain.ScriptDevelop.Plugin.NRefactory.Analysis.

Import component offers two plugins for the code analysis engine:

• UnknownIdentifierCodeIssueProvider, which gathers all identifiers thatcould not be resolved and displays them in red color.

• UnknownIdentifierCodeActionProvider, which offers context actions to cor-rect unknown identifiers in case they can be corrected.

The issues do not contain the actions directly. This is so for performance reasons,because detecting if an unknown identifier can be fixed includes iterating over allaccessible types from all known assemblies. We only do this for a single identifieronce the user moves the caret over it.

Detecting unknown identifiers

Detecting unknown identifiers cannot be easily done by looking at the errorsreturned from the compiler, mainly because:

a) The error list from the compiler is meant for user reference and is not suitedfor parsing.

87

b) We need additional information about the context of unknown identifiers (forexample the type upon which the unknown extension method is invoked).

Instead, whenever the source code is parsed by the NRefactory core, we look ateach identifier and try to resolve it in order to find out what the identifier denotessemantically. This resolving process can return errors in case the identifier couldnot be resolved to any entity, which is exactly what we need in order to mark theidentifier as unknown.

This way of detecting unknown identifiers has one disadvantage – the ana-lyzed code snippet needs to be syntactically correct, or else we could not resolveit semantically. This means, for example, that we cannot immediately markan unknown type while the user writes a member declaration because the state-ment is not yet complete and cannot be properly parsed. The unknown typegets recognized later, the moment the parser recognizes the declaration state-ment syntax and knows it should resolve the identifier as a type. However, thisis not a problem since the import analysis is meant for code taken out of context(e.g. by copy & paste) and such code is usually syntactically correct. Importingentities as the user writes code is instead handled by the code completion.

Unknown identifier representation

There are two kinds of situations where an unknown identifier may be encountered– stand-alone or after a dot as an unknown member.

1) Stand-alone identifiers. These are represented in the AST eitheras IdentifierExpression when part of an expression, or as SimpleType

when outside an expression. If they are unknown, both are resolved toUnknownIdentifierResolveResult. Such identifiers are represented in our codeas UnknownSingleIdentifier, which does not carry any additional information.

2) Member identifiers. These are represented in the AST either asMemberReferenceExpression when part of an expression, or as MemberType

when outside an expression. Their resolving is more complicated:

• UnknownMemberResolveResult in case of simple member expressions.

• MethodGroupResolveResult with zero eligible methods in case of memberexpressions where a method group is expected.

• ErrorResolveResult in case the expression before the dot was resolvedto a type or a namespace.

Unknown member identifiers are represented in our application by three sepa-rate classes, based on how the identifiers are used in the code (i.e. what kind ofexpression the dot before the unknown identifier expands):

• UnknownInstanceMember – There is an expression before the dot. Carries thetype entity of the expression and the MemberReferenceExpression in whichit was detected (which is needed when refactoring an extension method callinto a static method call).

88

• UnknownStaticMember – There is a type before the dot. Carries the typeentity.

• UnknownNamespaceMember – There is a namespace before the dot. Carriesthe namespace entity.

UnknownIdentifier is the base abstract class for all our unknown identifiers. Itcarries the identifier node itself, its name and its type argument count. Itsproperty IsAttributeTypeName denotes that the unknown identifier is used inan attribute declaration (in which case we must take into account the possibleAttribute suffix).

The UnknownIdentifier class contains the GetUnknownIdentifierFromNode

static method which analyses the given AST node according to the rules de-scribed above and returns the correct instance of UnknownIdentifier, or null ifthe given node does not represent an unknown identifier. The implementation ofUnknownIdentifierCodeIssueProvider is then a simple case of traversing the syn-tax tree, calling GetUnknownIdentifierFromNode on its every node and returninga code issue for every UnknownIdentifier returned.

Correcting unknown identifiers

Whenever the user moves the caret over an unknown identifier that was detectedby the UnknownIdentifierCodeIssueProvider, we want to offer corrective contextactions. That means we need to find out what the identifier could possibly mean,offer the guessed meanings in the form of context actions, and refactor the codeaccordingly if the context action is chosen.

The actual process of determining what imports to offer for an un-known identifier is implemented in UnknownIdentifierCodeActionProvider. TheImportProvider class described above is used to search for the entities to offer.What exactly is offered depends on the type of the unknown identifier:

• UnknownSingleIdentifier – We offer all accessible types and all root names-paces with the correct name. This also includes nested types because it isa common scenario to copy code referencing public nested types out of theparent class.

• UnknownInstanceMember – We offer all accessible extension methods thathave the correct name and can be applied to the target expression basedon its type.

• UnknownNamespaceMember – We offer all accessible types and all child names-paces that have the correct name and reside in the correct target namespace.

• UnknownStaticMember – We do not offer anything. Unknown static memberscannot be corrected by a new using statement or a new reference becausethe parent type is already recognized and fully known.

The refactoring that happens when the context action is selected is implementedin the ImportRefactoring class and includes these actions:

89

• If the import has NewUsingRequired set to true, a new using statement isadded to the top of the current source code file. This is implemented byNRefactory itself, in the UsingHelper.InsertUsing method.

• If the import has NewReferenceRequired set to true, a new referenceis added to the current project. This is implemented by our ownHeaderRefactoring class because it involves header sections that are specificto our application.

• If the import has a non-null CompletionText property, we cannot refer tothe imported entity by its short name, but instead we need to use the textin this property.

– For types, this simply means that we must replace the identifier withthe content of CompletionText.

– For extension methods, this means that we must refactor the call fromthe instance method form to the static method form, and use the con-tent of CompletionText to identify the static method. For example, ifthe original code snippet was items.Contains(x) and CompletionText

was System.Linq.Enumerable.Contains, we need to refactor the snip-pet to System.Linq.Enumerable.Contains(items, x).

To aid with such refactoring, the UnknownInstanceMember class,which represents the unknown identifier that can be an ex-tension method, carries in its Expression property the originalMemberReferenceExpression node. This way we can use NRefactory’srefactoring infrastructure that allows easy AST node construction andreplacement.

4.7.5 Syntax highlighting

Requirement R2.1 (advanced syntax highlighting) means we need to implementsyntax highlighting that is based not solely on regular expressions, which theAvalonEdit library contains extensive support for, but also on the real outputfrom the NRefactory processing. Such syntax highlighting is implemented in theExtBrain.ScriptDevelop.Plugin.NRefactory.SyntaxHighlighting namespace.

Syntax highlighting is implemented by the SyntaxHighlightingManager doc-ument extension plugin. When a new C# document is opened, the plugin assignsthe corresponding editor control the correct syntax highlighting definition (de-fined in the CSharp-Mode-VS-Base.xshd file). However, this would only performregular-expression-based highlighting. So, in addition to this, the plugin alsomonitors the NRefactory core and each time the source code is reparsed, it tra-verses the document’s AST and recolors certain identifiers:

• If the identifier is represented in the AST as a token node, it is a keyword andwill be recolored accordingly. This gives the keyword color even to identifierswhich are not reserved in the C# language, but recognized as keywords onlyin certain syntactic constructs (e.g. keywords specific to LINQ expressions).

90

• If not, we try to resolve the identifier in order to find its semantic meaning:

– If it resolves to a type, it is recolored as a type (unless it is var ordynamic, which are recolored as keywords). This allows us to distin-guish between type and namespace names.

– If it resolves to a local variable named value and we are in a propertyaccessor body, it is recolored as a keyword.

This gives us syntax highlighting similar to that in Visual Studio. The process canbe potentially easily expanded to give different colors to other language constructsbased on semantics.


We have analyzed how we plan to implement code completion in sec-tion 3.5.3. The general user interface for code completion is implementedas a part of the Editor component and described in section 4.4.4. In thissection we describe the implementation of code completion for the C# lan-guage. The implementation is located in the ExtBrain.ScriptDevelop.Plugin.

NRefactory.CodeCompletion project, primarily in the ExtBrain.ScriptDevelop.

Plugin.NRefactory.CodeCompletion.CSharp namespace.

Engine

The NRefactory library exposes its code completion functionality through theCSharpCompletionEngine class. We use this class as the base and inherit from itour own CustomCompletionEngine, which exposes the following two methods:

public IEnumerable<ICompletionEntry> GetStandardCompletionData();

public IEnumerable<IImportCompletionEntry> GetImportCompletionData(

bool includeReferencedAssemblies,

bool includeUnreferencedAssemblies

);

These methods are used to implement all three levels (basic, import, reference)of C# code completion.

• GetStandardCompletionData uses the GetCompletionData method of thebase CSharpCompletionEngine.

• GetImportCompletionData uses the ImportProvider class described in sec-tion 4.7.4 to gather ImportResult instances for possible unimported or un-referenced entities and returs corresponding IImportCompletionEntry in-stances for them.

There are a lot of different types of entries implementing ICompletionEntry thatcan be returned from CustomCompletionEngine. They form a complex hierarchylocated in the ExtBrain.ScriptDevelop.Plugin.NRefactory.CodeCompletion.

Entries namespace.

91

Mode detection

As we have stated in section 3.5.3, we need a way to detect if a new symbolcan be named at a particular location in the source code. If yes, we cannotautomatically open the completion window at that location, or we must open itin a special mode called inactive selection.

The decision process is implemented in the CustomCompletionEngine by theGetNameDefinitionStatus method, which can return three different values:

• NameDefinitionStatus.None – There cannot be a new name introduced atthe completion location. We can display the completion window with activeselection.

• NameDefinitionStatus.NonExclusive – There can be a new name intro-duced at the completion location, but also there can be another symbolreferenced at the location as well. We must display the completion windowwith inactive selection.

• NameDefinitionStatus.Exclusive – There must be a new name introducedat the completion location. We cannot display the completion window.

The GetNameDefinitionStatus method uses the same tools and mechanisms asthe CSharpCompletionEngine to detect what syntactic constructs are possible atthe given location.

Header sections

So far, we have been concerned only with code completion for the C# lan-guage itself. However, we have introduced special comment blocks calledheader sections (see section 3.4.1), which are complex enough to benefitfrom code completion as well. It is implemented in the Builder library inthe ExtBrain.ScriptDevelop.Builder.HeaderCompletion namespace and inte-grated with our application in the ExtBrain.ScriptDevelop.Plugin.NRefactory.

CodeCompletion.Header namespace.

4.8 Debugger

Debugger is a primary component of the application that implements the basicdebugging functionality. It provides commands for starting, pausing and stoppingthe debugger, for stepping through the debugged code, or for breakpoint man-agement. It also contains panels that display call stack, exception information,or command console. It is implemented in the ExtBrain.ScriptDevelop.Plugin.

Debugger project.

DebuggerPlugin

The basic debugger functionality is implemented in the singleton DebuggerPlugin,which uses the Debugger.Core library (see section 3.1.5) to debug running CLRprocesses. Its interface contains a lot of properties, methods and events thatare used to control the instance of the NDebugger class from the Debugger.Corelibrary.

92

public class DebuggerPlugin

{

public NDebugger Debugger { get; }

public Process Process { get; }

public bool IsDebugging { get; }

public bool IsRunning { get; }

public bool IsPaused { get; }

public event EventHandler DebuggingStarted;

public event EventHandler DebuggingStopped;

public event EventHandler DebuggingPaused;

public event EventHandler DebuggingResumed;

public void Start(ProcessStartInfo processStartInfo,

IEnumerable<string> sources,

bool breakAtBeginning);

public void Continue();

public void Stop();

public void Break();

public void StepOver();

public void StepInto();

public void StepOut();

}

Context

When the debugged process is paused, we use the concept of current thread,current stack frame, and current statement to define the context in which stateinspection takes place:

• The debugged process may contain multiple threads, each with its own callstack. The DebuggerPlugin allows selecting one thread and then operatesin the context of this selected (current) thread.

• The call stack of the current thread may contain multiple stack frames(invoked methods), each with its own set of local variables and currentstatement position. The DebuggerPlugin allows selecting one stack frameand then operates in the context of this selected (current) stack frame.

• Current statement is the statement in the source code that is just beingexecuted or just about to be executed. It depends on the current stackframe (each stack frame has its own current statement) and thus also onthe current thread (each thread has its own call stack).

The described concepts of current thread, current stack frame, and current nextstatement are exposed by the following properties and events of DebuggerPlugin.

93

public Thread CurrentThread { get; set; }

public StackFrame CurrentStackFrame { get; set; }

public DomRegion CurrentNextStatement { get; }

public event EventHandler CurrentThreadChanged;

public event EventHandler CurrentStackFrameChanged;

Evaluation

When the execution is paused, it is possible inspect the current values of variablesor invoke custom commands. This can be done using DebuggerPlugin.GetValue

method, which is used to implement the console window, the watch window,tooltips showing values of variables, and conditional breakpoints. Evaluationtakes place in the context of current thread and current stack frame.

Breakpoints

Breakpoints are places in the source code where execution will be paused auto-matically. DebuggerPlugin contains the following methods for breakpoint man-agement:

public Breakpoint AddBreakpoint(string path, int line);

public void RemoveBreakpoint(Breakpoint breakpoint);

The Breakpoint class extends the underlying breakpoint handle of the debuggerlibrary by implementing the following interface:

public class Breakpoint

{

public bool IsEnabled { get; set; }

public string Path { get; }

public int Line { get; }

public int Column { get; }

public bool IsSet { get; }

public DomRegion Region { get; }

public event EventHandler IsSetChanged;

public event EventHandler<CancelEventArgs> Hit;

}

The IsSet property is set to true once the debugger recognizes the position withinthe source code (i.e. loads the corresponding symbol files) and then the Region

property returns the proper text segment covered by the breakpoint.The Hit event is invoked when the breakpoint has been hit by the debugged

process, and we can use its CancelEventArgs to cancel the pause if we are not inter-ested in this particular occurrence. This is used to implement conditional break-points, which evaluate their condition in the Hit event (using DebuggerPlugin.

GetValue) and cancel the pause if the condition is not met.

94

All breakpoints across all documents are managed by the BreakpointPlugin

class. Breakpoints for a particular document, including their visual representa-tion, are managed by the BreakpointManager class.

Visualizers

Several features of the debugger (value tooltips, watch panel, console panel) needto display to the user values read from the debugged process. The are manyways such values can be displayed, ranging from simple text representations tocomplex object inspectors that use a tree view. Because of this, the debuggeroffers the IVisualizer interface:

public interface IVisualizer

{

object Visualize(Value value, Thread thread);

}

Classes implementing this interface inspect the given value and decide if theywant to visualize it. If yes, the Visualize method returns an object that canbe a simple string or a complex UI element presenting the value. IVisualizer

implementations are plugged into the application using the ExportVisualizer

attribute, which also allows specifying the priority of the visualizer (lower-priorityvisualizer is only used when all higher-priority visualizers return null).

Visualizers are managed by the singleton VisualizerPlugin plugin, which canbe asked for a visualization of the given value using the Visualize method. Thismethod iterates over all known visualizers ordered by their priority from thehighest to the lowest and returns the first non-null visualization.

There are currently two simple visualizers implemented in the application:

• PrimitiveValueVisualizer – Converts any primitive value to string in sucha way so that it would look like C# literal, using TextWriterTokenWriter.

PrintPrimitiveValue method from NRefactory.

• FallbackToStringVisualizer – Converts any value to string by invoking itsToString method in the debugged process.

Tooltips

When the debugged process is paused, users can inspect current values of variablesdirectly in the source code. When the mouse cursor is hovered over a variablein the source code, its current value is displayed in a tooltip. This is imple-mented in the DebuggerTooltipPlugin. First, the identifier under the mouse cur-sor is detected and resolved using NRefactory. Then, DebuggerPlugin.GetValueis called on the result to get the Value instance, which is then visualized by theVisualizerPlugin and shown in a tooltip.

Panels

The debugger contains several informational panels that the user can displaywhile debugging. These panels are implemented in the ExtBrain.ScriptDevelop.Plugin.Debugger.Panels namespace.

95

• ThreadsPanel – Lists all threads of the currently debugged process, indicateswhich thread is currently selected, and allows switching of the currentlyselected thread (DebuggerPlugin.CurrentThread).

• CallStackPanel – Lists stack frames in the call stack of the currentlyselected thread, indicates which stack frame is currently selected, andallows switching of the currently selected stack frame (DebuggerPlugin.CurrentStackFrame).

• ExceptionPanel – Displays details of the current exception, if any. It isautomatically displayed if the process has been paused because an exceptionhas been thrown. In this case, the thread where the exception has occurredwill have it associated and available for us to retrieve using DebuggerPlugin.

CurrentThreadException.

• WatchPanel – Allows creating one or more watch expressions. Thesewatch expressions are then evaluated each time the process is paused andtheir values are displayed in a table using DebuggerPlugin.GetValue andVisualizerPlugin.

• ConsolePanel – Offers a command line interface that allows evaluation ofcustom expressions. When an expression is entered, it is immediately eval-uated in the context of the current statement. The resulting value of theexpression is then printed out as text. This can be used for deeper inspec-tion of variable values, and also to modify the state of the process sincemethod calls and assignments can also be executed. Entered statementsare evaluated using DebuggerPlugin.GetValue and VisualizerPlugin.

Code completion

The console panel, the watch panel and the breakpoint condition dialog allowentering custom expressions for evaluation. Since these expressions are C# sourcecode fragments that should be valid in their respective contexts, we offer codecompletion functionality for them. This functionality is based on the general C#

code completion described in section 4.7.6, but modified for specific aspects ofthe debugger evaluator:

• Text boxes for the expressions do not contain complete code but insteadonly the expression itself which is evaluated in the context of the currentstatement or breakpoint position. Therefore, we must provide the correctcontext (position in the source code) to the code completion engine.

• Users may want to use types the namespaces of which are not imported inthe source code. Therefore, the code completion offers all known types andinserts their full name upon selection.

Code completion for the debugger evaluator is implemented in the ExtBrain.

ScriptDevelop.Plugin.NRefactory.CodeCompletion.Debug namespace.

96

5. Conclusion

The goal of this work was to create a small and easy-to-use tool for using the C#

programming language to write scripts. In the work we have analyzed the existingtools and identified their advantages and disadvantages, formulated requirementsfor our own tool, and developed our own tool.

5.1 Evaluation

We have created an application which satisfies all the requirements R1.1 – R4.8 wedefined in section 2.2. The result is a small integrated development environmentfor quick and easy writing of scripts in the C# language.

The IDE offers sufficient features to allow easy authoring and debugging ofprograms consisting primarily of a single C# source code file. It makes heavy useof the NRefactory library for syntactic and semantic analysis of the C# sourcecode, and thus offers many code-aware features that simple editors lack, suchas correct syntax highlighting, code completion, navigation, or refactoring (mostimportantly reference and using statement completion). The build system of theIDE is capable of processing stand-alone source code files that include informa-tion describing additional compiler dependencies such as external references, andtherefore small programs can be developed as single self-contained script files.The IDE also contains basic debugging options which can be used to inspect theexecution of the scripts.

The resulting application also offers some additional features which were notoriginally specified, but are interesting and can be helpful to users:

• A new script file created in the application does not need to be saved firstin order to run it or even debug it. This is very convenient when the useronly wants to quickly test a code snippet.

• The application is capable of packaging itself into a single stand-alone exe-cutable file that contains all the currently loaded plugins, the current con-figuration file, and all the necessary third-party libraries. The resultingsingle executable file can then be copied to any system which could run theoriginal application and launched directly from the executable file, withoutany prior installation or unpacking process.

• The entire application can be built in itself, which means the build systemis versatile enough to handle large projects with a lot of source code files,assemblies and dependencies. While this capability was not necessary, andindeed the usage of the application for such a large project is not convenientat all, it provides a useful testing scenario.

• The application features a code analysis subsystem that scans the sourcecode for various issues and offers context-sensitive actions and simple refac-toring transformations. This capability was not present among the speci-fied requirements because it was not very important for writing small ad-hocprograms, but since the NRefactory library offered it in an easy-to-integrateway, we have decided to include it.

97

5.2 Comparison with other tools

In section 2.1.2 we have analyzed the existing tools that allow the usage of theC# language for script development. None of them fully satisfied our ideas foran ideal tool. The tool that came the closest to what we were looking for wasLINQPad, which is popular for solving small tasks unrelated to databases (theprimary focus of LINQPad). How does our application compare to it?

• Both applications are small and easy to use.

• Both applications allow quick running of simple code snippets or scripts,but LINQPad also allows execution of stand-alone expressions and state-ments, whereas our application only allows execution of well-formed C#

programs. However, this makes sure the scripts in our application are validC# programs and our application offers the necessary code template withthe Main method automatically when creating a new script.

• LINQPad uses custom XML format for saving scripts. This format allowsstoring some additional information about the script, such as its type (ifit is an expression, statement, etc.), imported namespaces, and referencedassemblies. Our application saves scripts as valid C# source code and storesthe additional information in comments.

• LINQPad’s primary focus is on querying databases using the C# languageand LINQ. Our application has no such functionality.

• LINQPad executes code in a custom hosting environment and offers addi-tional runtime features, such as the universal “dump” command that allowscomplex output of any object into a special window. Our application exe-cutes scripts as pure stand-alone processes.

• Both applications offer a full-featured C# editor (although in LINQPadmost of the important features are available only in the paid version).

• LINQPad does not support code reuse. Our application allows one scriptto include or reference another.

• LINQPad has no executable generation (the scripts are executed in a specialhosting environment and cannot be used outside LINQPad). Our applica-tion compiles scripts as pure stand-alone executables.

• LINQPad needs external tools for debugging. Our application containsbasic debugging options.

However, during the work on our application, a new version of LINQPad has beenreleased, which added some of the previously missing features:

• A stand-alone command-line tool for execution of LINQPad scripts hasbeen released. This tool can be used instead of executable generation torun scripts outside of LINQPad itself.

• Debugging options similar to our own were introduced, but in the highestpaid variant of LINQPad.

98

5.3 Future work

Our aim was not to create a complete, all-inclusive IDE, but rather a solid andwell-usable basis that could be further extended and improved by other studentsand developers based on their specific requirements, ideas and needs. The primaryareas in which the application can be improved include for example:

• Additional editor features. The application contains only basic text-editing options. What is missing are features common from advanced texteditors such as for example search and replace, navigation between cursorpositions, reformatting, etc.

• Configuration UI. The application can be configured using a special XMLconfiguration file. To make this process a little more user friendly, the con-figuration file can be edited from the application itself and is automaticallyreloaded each time it is saved. However, for most important configurationoptions a dedicated user interface would be much more convenient.

• Debugger inspector. The application offers basic debugging options sothat the users could inspect the behavior of their scripts. When inspectinga value of a variable, only simple string representation is shown. Morecomplex objects need to be queried using the debugger console, which isnot very convenient. What is missing is a visual object inspector thatwould make inspecting complex values of objects and collections easier.

During the development of our application, the authors of SharpDevelop an-nounced the final version 5.0, stating that the core team would be shifting theirfocus from SharpDevelop itself to the universal components that it consists of(e.g. NRefactory, AvalonEdit, ILSpy, etc.) [32]. This news impacts the futuredevelopment our application since we use a lot of these components, which maynow see a faster development pace.

Also, the current development around the C# language (e.g. Microsoft open-sourcing a large portion of its previously proprietary implementation of the C#

runtime and libraries [33]) suggests that the popularity of the C# language, andthus tools associated with it, may be further growing in the future.

99

100

Bibliography

[1] Xamarin: The Mono Project, http://mono-project.com/, online 28. 4.2015

[2] Microsoft: Visual Studio, http://www.visualstudio.com/, online 28. 4.2015

[3] IC#Code: SharpDevelop, http://www.icsharpcode.net/OpenSource/SD/Default.aspx, online 28. 4. 2015

[4] Xamarin: MonoDevelop, http://monodevelop.com/, online 28. 4. 2015

[5] Microsoft: .NET Compiler Platform (“Roslyn”), MSDN, http://msdn.microsoft.com/en-us/vstudio/roslyn.aspx, online 28. 4. 2015

[6] IC#Code: Repository for NRefactory 5, GitHub, http://github.com/icsharpcode/NRefactory, online 28. 4. 2015

[7] Grunwald D.: Using NRefactory for analyzing C# code, CodeProject 2012,http://www.codeproject.com/Articles/408663/Using-NRefactory-

for-analyzing-Csharp-code, online 28. 4. 2015

[8] Grunwald D.: Using AvalonEdit (WPF Text Editor), CodeProject 2013,http://www.codeproject.com/Articles/42490/Using-AvalonEdit-

WPF-Text-Editor, online 28. 4. 2015

[9] Srbecky D.: Internals of SharpDevelop’s Debugger, http://community.sharpdevelop.net/blogs/dsrbecky/archive/2010/07/29/debugger.

aspx, online 28. 4. 2015

[10] Konıcek M.: Debugger Visualizers for the SharpDevelop IDE, 2011, Masterthesis, Charles University in Prague, Faculty of Mathematics and Physics,Department of Software Engineering, http://artax.karlin.mff.cuni.cz/~konim5am/thesis/MartinKonicek_DebuggerVisualizers.pdf,online 28. 4. 2015

[11] Karasek J.: Prostredı pro snadny vyvoj skriptu, 2012, Bachelor thesis,Czech Technical University in Prague, Department of Cybernetics, http://www.extbrain.net/theses/bp-jiri-karasek.pdf, online 28. 4. 2015

[12] Vavilala, R. K.: NScript – A script host for C#/VB.NET/JScript.NET,CodeProject 2002, http://www.codeproject.com/Articles/3207/NScript-A-script-host-for-C-VB-NET-JScript-NET, online 28. 4. 2015

[13] Shilo O.: CS-Script – The C# Script Engine, http://www.csscript.net,online 28. 4. 2015

[14] Block G., Rusbatch J., Wojcieszyn F.: scriptcs, http://scriptcs.net/,online 28. 4. 2015

[15] Albahari J.: LINQPad, http://www.linqpad.net/, online 28. 4. 2015

101

http://mono-project.com/

http://www.visualstudio.com/

http://www.icsharpcode.net/OpenSource/SD/Default.aspx

http://www.icsharpcode.net/OpenSource/SD/Default.aspx

http://monodevelop.com/

http://msdn.microsoft.com/en-us/vstudio/roslyn.aspx

http://msdn.microsoft.com/en-us/vstudio/roslyn.aspx

http://github.com/icsharpcode/NRefactory

http://github.com/icsharpcode/NRefactory

http://www.codeproject.com/Articles/408663/Using-NRefactory-for-analyzing-Csharp-code

http://www.codeproject.com/Articles/408663/Using-NRefactory-for-analyzing-Csharp-code

http://www.codeproject.com/Articles/42490/Using-AvalonEdit-WPF-Text-Editor

http://www.codeproject.com/Articles/42490/Using-AvalonEdit-WPF-Text-Editor

http://community.sharpdevelop.net/blogs/dsrbecky/archive/2010/07/29/debugger.aspx



http://artax.karlin.mff.cuni.cz/~konim5am/thesis/MartinKonicek_DebuggerVisualizers.pdf

http://artax.karlin.mff.cuni.cz/~konim5am/thesis/MartinKonicek_DebuggerVisualizers.pdf

http://www.extbrain.net/theses/bp-jiri-karasek.pdf

http://www.extbrain.net/theses/bp-jiri-karasek.pdf

http://www.codeproject.com/Articles/3207/NScript-A-script-host-for-C-VB-NET-JScript-NET

http://www.codeproject.com/Articles/3207/NScript-A-script-host-for-C-VB-NET-JScript-NET

http://www.csscript.net

http://scriptcs.net/

http://www.linqpad.net/

[16] Microsoft: Microsoft .NET Framework 4 (Standalone Installer), http://www.microsoft.com/en-us/download/details.aspx?id=17718, online28. 4. 2015

[17] IC#Code: ILSpy – .NET Decompiler, http://ilspy.net/, online 28. 4.2015

[18] Microsoft: Command-line Building With csc.exe, MSDN, http://msdn.microsoft.com/en-us/library/78f4aasd.aspx, online 28. 4. 2015

[19] Microsoft: SQL Server Management Studio, MSDN, http://msdn.microsoft.com/en-us/library/hh213248.aspx, online 28. 4. 2015

[20] Microsoft: Windows Forms, MSDN, http://msdn.microsoft.com/en-us/library/dd30h2yb.aspx, online 28. 4. 2015

[21] Microsoft: Windows Presentation Foundation, MSDN, http://msdn.microsoft.com/en-us/library/ms754130.aspx, online 28. 4. 2015

[22] IC#Code: AvalonEdit, http://avalonedit.net/, online 28. 4. 2015

[23] Community: ScintillaNET, Codeplex, http://scintillanet.codeplex.com/, online 28. 4. 2015

[24] Community: Scintilla: A free source code editing component for Win32,GTK+, and OS X, http://scintilla.org/, online 28. 4. 2015

[25] Community: AvalonDock, Codeplex, http://avalondock.codeplex.com/,online 28. 4. 2015

[26] Microsoft: Debugging Interfaces, MSDN, http://msdn.microsoft.com/en-us/library/ms404484.aspx, online 28. 4. 2015

[27] Microsoft: Add-ins and Extensibility, MSDN, https://msdn.microsoft.com/en-us/library/bb384241.aspx, online 28. 4. 2015

[28] Community: Managed Extensibility Framework, Codeplex, http://mef.codeplex.com/, online 28. 4. 2015

[29] Xamarin: Cecil, http://mono-project.com/Cecil, online 28. 4. 2015

[30] Microsoft: Application Settings for Windows Forms, MSDN, http://msdn.microsoft.com/en-us/library/0zszyc6e.aspx, online 28. 4. 2015

[31] Apache: Apache log4net, http://logging.apache.org/log4net/, online28. 4. 2015

[32] Wille, C.: SharpDevelop 5.0 Final, SharpDevelop Community Blog,http://community.sharpdevelop.net/blogs/christophwille/

archive/2014/10/28/sharpdevelop-5-0-final.aspx, online 28. 4. 2015

[33] Landwerth, I.: .NET Core is Open Source, .NET Blog, MSDN, http://blogs.msdn.com/b/dotnet/archive/2014/11/12/net-core-is-open-

source.aspx, online 28. 4. 2015

102

http://www.microsoft.com/en-us/download/details.aspx?id=17718

http://www.microsoft.com/en-us/download/details.aspx?id=17718

http://ilspy.net/

http://msdn.microsoft.com/en-us/library/78f4aasd.aspx

http://msdn.microsoft.com/en-us/library/78f4aasd.aspx

http://msdn.microsoft.com/en-us/library/hh213248.aspx

http://msdn.microsoft.com/en-us/library/hh213248.aspx

http://msdn.microsoft.com/en-us/library/dd30h2yb.aspx

http://msdn.microsoft.com/en-us/library/dd30h2yb.aspx

http://msdn.microsoft.com/en-us/library/ms754130.aspx


http://avalonedit.net/

http://scintillanet.codeplex.com/

http://scintillanet.codeplex.com/

http://scintilla.org/

http://avalondock.codeplex.com/



https://msdn.microsoft.com/en-us/library/bb384241.aspx

https://msdn.microsoft.com/en-us/library/bb384241.aspx

http://mef.codeplex.com/

http://mef.codeplex.com/

http://mono-project.com/Cecil

http://msdn.microsoft.com/en-us/library/0zszyc6e.aspx

http://msdn.microsoft.com/en-us/library/0zszyc6e.aspx

http://logging.apache.org/log4net/

http://community.sharpdevelop.net/blogs/christophwille/archive/2014/10/28/sharpdevelop-5-0-final.aspx

http://community.sharpdevelop.net/blogs/christophwille/archive/2014/10/28/sharpdevelop-5-0-final.aspx

http://blogs.msdn.com/b/dotnet/archive/2014/11/12/net-core-is-open-source.aspx



A. Content of the CD

The CD enclosed to this thesis contains the following directories and files:

• bin – Directory with the compiled executable binaries of the application(for installation instructions, see section B.1).

• doc – Directory with the documentation generated from the source code ofthe application.

• redist – Directory containing Microsoft .NET Framework 4 [16], which isa prerequisite for running the application.

• src – Directory with the source code of the application.

– src/Libraries – Source code of the modified libraries that the appli-cation uses.

– src/Libraries/readme.txt – Text document describing the modifiedlibraries (for details, see beginning of chapter 4).

– src/ExtBrain.ScriptDevelop – Source code of the application itself.

– src/ExtBrain.ScriptDevelop/ExtBrain.ScriptDevelop.csx – Scriptfile that will build the application from inside the applica-tion itself, placing the resulting binaries into the src/ExtBrain.

ScriptDevelop/Bin directory.

• example – Directory with a few example scripts that can be opened and runin the application.

• previous – Directory containing the source code of the original ScriptDe-velop [11], upon which our application has been based (see section 3.2).

• thesis – Directory with the text of this thesis in the PDF format.

• readme.txt – Text document describing the content of the enclosed CD.

103

104

B. User guide

This appendix describes the application from the user’s perspective. It detailshow to install and use the application.

B.1 Installation

The application does not require any special installation. It can be copied, alongwith all of the other files in its program directory, to any machine that hasa Microsoft .NET Framework 4 [16] installed. On the CD attached to this thesis,you can find the binaries of the application in the bin directory and Microsoft.NET Framework 4 in the redist directory. Also please note that the applicationshould have write access to the directory it is launched from (because of logand configuration files), so it is not recommended to launch it directly from theenclosed CD.

Aside from the executable file itself, the application contains a lot of assemblyfiles in its directory. All these files are required for the application to functionproperly and must be copied when installing the application to a new location.

To simplify manipulation and deployment, the application can package itselfinto a single stand-alone executable file that contains all the currently loaded plu-gins, the current configuration file, and all the necessary third-party libraries. Theresulting single executable file can then be copied to any system that could runthe original application and launched directly from the executable file, withoutany installation or unpacking process.

This packaging process can be initiated from the main menu by selectingFile → Export Application. After the application is packaged, the user will beprompted to save the resulting executable to a new location.

B.2 Quickstart guide

In order to quickly start working on a new script, select File → New → C#

Script from the main menu. An editor with a new script appears, its sourcecode already containing the following template:

using System;

/*HEADER**

PAUSE

**HEADER*/

class Program

{

static void Main(string[] args)

{

}

}

105

This is a standard C# console application with the Main method into which youcan start writing your own code. You can immediately compile and run yourscript using Build → Run (or Ctrl+F5) without having to name and save it first.You can even debug your untitled script right away (Debug → Run, or F5).

Header sections

In the script template shown above, please note the comment block above theclass definition. It looks like this:

/*HEADER**

PAUSE

**HEADER*/

This comment block is called a header section and its purpose is to control theway the script is compiled and executed. It consists of special header commands(described further in this chapter), one command per line. For example, the PAUSEcommand, which is already present in the template, causes the console windowwith the script to pause and display “Press any key to continue...” just beforeit exits, provided it has been launched from the application (it does not pause ifit has been launched separately via the compiled executable file). There are alsosimilar commands for specifying working directory and command line arguments,among others.

References and using statements

The most common and useful header command you will encounter is the REF

command, which is used to reference another assembly, including library assem-blies of the framework. However, in most cases you do not need to specify itmanually. For example, if you want to use methods from the LINQ library, youcan just start typing the name of the method, e.g. Any. At this point, the wordAny will be shown in red and code completion will offer no options, as can be seenin figure B.1.

Figure B.1: Unknown identifier example.

There are now two options:

a) You can press Ctrl+Space two more times to open reference completion, whichwill include items from unreferenced assemblies (see figure B.2), and select thecorrect method from it.

b) You can click the red socket wrench icon at the beginning of the line. Thisicon indicates that there are some issues with possible automatic fixes on theline. Clicking it (or pressing Alt+Enter) will open a menu with possible fixes(see figure B.3). You can select the correct method from it.

106

Figure B.2: Reference completion example.

Figure B.3: Reference context action example.

Both actions will result in the correct reference and using statement being addedto the source code, which will now become like this (please note that “usingSystem.Linq” and “REF System.Core.dll” have been inserted):

using System;

using System.Linq;

/*HEADER**

PAUSE

REF System.Core.dll

**HEADER*/

class Program

{

static void Main(string[] args)

{

if(args.Any

}

}

This way you can quickly use types and extension methods that reside in names-paces which have not been imported yet with a using statement, or whose as-sembly is not yet referenced. Also please note that the application must knowsomehow which assemblies to offer for referencing. The list of such assembliescan be configured, as described further in this chapter.

B.3 Basics

The application is laid out like a standard text editor with a tabbed interface.The main window, which can be seen in figure B.4, consists of the main menu andtoolbar at the top, informational status bar at the bottom, and a document viewin the central area. Multiple documents can be open at the same time and the

107

user can switch between them using their tab headers at the top. The documentview contains a text editor, whose exact functionality depends on the type ofdocument that is currently open.

Figure B.4: The main window of the application.

There are many commands available from the main menu. If a command hasa keyboard shortcut, it is displayed next to the command in the main menu. Somecommands are available also as buttons in the toolbar. These commands use thesame icon for the button as in the main menu and their name and keyboardshortcut are displayed in a tooltip when the mouse cursor is hovered over thebutton.

Not all commands are always visible. Some commands appear only in certaincontexts, for example when debugging is active or when a C# document is active.

The Filemenu contains common commands for creating, opening, saving, andclosing documents. Documents can be additionally opened by dragging them tothe application’s window or by specifying their paths in the application’s com-mand line. If the application is closed and there are documents that are still open,they will be opened again when the application is next started. If the applica-tion is terminated unexpectedly and there are documents that contain unsavedmodifications, they will be restored when the application is next started.

Supported formats

The application recognizes three types of documents based on their extension:

1) Text document – A generic plain text document format that is used as a fall-back when no other known format could be detected. The editor offers noadditional functionality to text documents besides simple editing.

2) XML document – For .xml, .xaml, .cfg and .config extensions. The editoroffers syntax highlighting and code folding (the ability to expand and collapseindividual elements) to XML documents.

108

3) C# document – For .cs and .csx extensions. The editor offers full functionalityto C# documents.

The application is primarily designed to work with C# source code files. Otherdocument types are supported only for convenience. The .csx extension is usedby default to distinguish stand-alone C# scripts created in the application fromother C# source code files.

Encoding

The application uses UTF-8 encoding with byte order mark (BOM) to save doc-uments. When opening documents, ASCII, other Unicode encodings, or the sys-tem’s default encoding may also be used.

B.4 Source code editor

The application is primarily designed to work with C# source code files, for whichthe source code editor supports a lot of additional functionality. This sectiondescribes how to use the features specific to C#.

Basics

The Edit menu contains common commands for history and clipboard manage-ment. Commands specific to C# include:

• Comment Selection, Uncomment Selection, Toggle Comment – Comments oruncomments the selected lines using line comments (double slashes).

• Rename – If the text cursor is placed over a named symbol (variable, type,member, namespace), opens a window that allows specifying a new namefor the symbol and renames all references of the symbol accordingly.

Code completion

While the user types identifier names, code completion offers a list of availabletypes, members, variables and other constructs. It allows to choose an identifierwithout the need to remember its exact name or without having to type out itsname in full.

Code completion appears automatically when the user types an identifier, orit can be invoked manually by pressing Ctrl+Space. It operates on three levels:

1) Basic completion – Offers all known symbols that can be used at the currentposition.

2) Import completion – Offers types and extension methods from namespacesthat are not currently imported with a using statement and imports thenamespaces accordingly when an item is selected.

3) Reference completion – Offers types, extension methods and namespaces fromyet unreferenced but otherwise known assemblies, and automatically referencesthe correct assembly when an item is selected.

109

The first level is the default. The user can switch to the next level by pressingCtrl+Space again.

Parameter hints

Parameter hint is a popup window that appears when the user writes a methodinvocation. It displays the list of invoked method’s overloads, the list of parame-ters for the currently selected overload, and the current parameter.

Parameter hint appears automatically when the user types a method invo-cation, or can be invoked manually by pressing Ctrl+Shift+Space. When thewindow is displayed, the user can switch between available overloads by pressingUp and Down keys.

Navigation

The View menu contains commands that allow quick navigation through the C#

source code:

• Go To Type – Displays a list of all known types and when a type is selected,navigates to its definition.

• Go To Member – Displays a list of all known members and when a memberis selected, navigates to its definition.

• Go To Definition – If the text cursor is placed over a named symbol (vari-able, type, member), navigates to its definition. If it is a library entity(an entity defined in an external assembly) and the path to ILSpy [17] isproperly configured, displays the entity’s decompiled source code in ILSpy.

Navigation bar

The navigation bar is the row on the editor’s top side. It contains two selectionboxes. The first one lists all types defined in the current source code file, thesecond one lists all members defined in the currently selected type. When thetext cursor is moved within the source code, the selected values in the boxesare updated to indicate what type and member the text cursor is currently at.Selecting different values from the lists will move the text cursor to the beginningof the selected type and member in the source code, so the navigation bar can beused to quickly locate types and members in longer files.

Code issues

As the user types source code, the application continuously analyses it and detectsissues (errors, warnings, suggestions). The detected issues are presented to theuser by underlining or coloring the related source code. For example, source codethat is detected to be redundant is displayed in gray, and unknown identifiers aredisplayed in red. Hovering the mouse cursor over such highlighted source codewill display the issue’s description in a tooltip.

110

Context actions

For many places in the source code, the application can offer various small refac-toring transformations, and for many issues, the application can offer automaticactions to fix them. For example, source code that was marked as redundant canbe quickly removed. When the text cursor is placed on a location where suchactions are available, an icon is displayed at the beginning of the line. Clickingthis icon (or pressing Alt + Enter) will open a menu with the available actions.

Shortcut column

The shortcut column is the column on the editor’s right side, beyond the scroll bar.It displays small colored horizontal bars that represent issues (errors, warnings,suggestions) found in the current source code file. The column’s whole heightrepresents the entire file, so bars at the top indicate issues at the beginning ofthe file and bars at the bottom indicate issues at the end of the file. Hovering themouse cursor over the bar will display the issue’s description in a tooltip, clickingthe bar will scroll the text cursor to the issue’s location.

The small colored square at the top of the shortcut column indicates if thereare any errors or warnings in the current source code file. It is red if there areerrors, orange if there are warnings but no errors, and green if there are no errorsor warnings. Clicking the square will cycle the text cursor over the most severeissues found in the current source code file.

Reference highlighting

When the text cursor is placed over a named symbol (variable, type, member,namespace) in the source code, all references of the same symbol are highlightedin the document for easier orientation. Alternatively, the user can manuallyhighlight all references of the symbol under the text cursor using the View →

Reference Highlighting → Highlight References command. There is also theView → Find References command that will list the symbol’s references acrossmultiple documents in a separate window.

B.5 Compiling and running

Scripts open in the application can be compiled using commands in the Build

menu:

• Run – Compiles the script and launches it.

• Build – Compiles the script without launching.

• Rebuild – Compiles the script without launching, forcing compilation evenwhen the output assemblies are up to date.

• Save Assembly As – Compiles the script and prompts the user to save theoutput assembly to a new location.

111

These commands will save the script first if it contains any unsaved modifications.The only exception to this are newly created unnamed scripts that were neversaved yet. Such scripts can be run without having to save them first.

By default, scripts are compiled into a temporary directory, from which theoutput executable is launched. This can be changed using header commandsdescribed in the next section. Header commands can also be used to specifyworking directory and command line arguments of the script launched from theapplication.

Errors

If the compilation process fails due to errors in the source code, a window listingthe errors is displayed and the errors are highlighted in the source code usinga red underline. The application also offers automatic error checking which canbe toggled in the Build → Automatic Error Checking menu:

• Continuous – Errors are checked each time the source code has been changedin the application, even when the changes were not saved yet.

• After Save – Errors are checked each time the script has been saved.

• Off – Errors are not checked automatically. A check can be invoked manu-ally using the Build → Check For Errors command, which works even onunsaved scripts.

The colored square in the bottom-right corner of the application’s main windowindicates if there are any errors in the current script. It can be green (no errors),red (at least one error), or gray (error checking is in progress).

Root file

Each script open in the application is considered stand-alone by default. Thismeans that all build-inducing commands such as Run are by default executedupon the currently active (focused) script. However, if an open script includes orreferences another open script, the application tries to detect their dependencyhierarchy and act upon the top-level script (the referencing one) instead evenwhen a lower-level script (the referenced one) is currently active.

If necessary, users can override this behavior by explicitly specifying the rootfile. All the build-inducing commands are then executed upon this root file nomatter what document is currently active. The root file is specified globallyfor the whole application and can be set or unset using the Build → Root File

menu, the selection box in the toolbar, or by toggling Root File in the document’scontext menu available by right-clicking the document’s tab header.

Command line compilation

Scripts can be also compiled using the command line. Any command line argu-ment of the application in form of /build:path will cause the application not todisplay its user interface, but instead to compile the script file denoted by path . Ifthe compilation is successful, the application exits immediately with an exit code

112

0. If there were any errors, the application prints the errors to stdout and exitswith an exit code 1. In batch files it is recommended to run the application usingstart /wait command so that the execution would pause until the compilationis finished.

B.6 Header sections

In order to correctly compile a source code file, the C# compiler needs additionalinformation, such as the list of referenced assemblies. In classic IDEs, this in-formation is associated with the project and stored in the project file. In ourapplication, this information is stored in the source code files themselves usingspecial comment blocks called header sections. A header section looks like this:

/*HEADER**

...

**HEADER*/

There can be multiple header sections in a single file. A single header sectionconsists of any number of header commands. Each header command must beplaced on a separate line. The first word of the line identifies the command, therest of the line is the command’s optional parameter. If the line is empty orbegins with double slashes, it is ignored. For example, a header section can looklike this:

/*HEADER**

PAUSE

OUT Hello world.exe

// References

REF System.dll

REF System.Core.dll

**HEADER*/

B.6.1 Available commands

The available header commands are listed in table B.1 and described in thissection.

OUT

The OUT command specifies the path of the output assembly. If omitted, theoutput assembly will be generated in a temporary directory when run from theapplication.

113

Command Parameter Summary

OUT Path Path of the output assembly.TARGET Target Type of the output assembly.VER Version Compiler version.MAIN Type Main class.OPT Option Additional compiler option.INC File(s) Includes additional source files.REF Reference(s) References assemblies.IMP Reference(s) References and includes assemblies.BUILD File(s) Compiles additional scripts.EMB File(s) Includes embedded resources.LNK File(s) Includes linked resources.RES File(s) Includes WPF resources.DIR Directory Working directory of the script.ARGS Arguments Command line arguments of the script.PAUSE None Pauses the script before exiting.

Table B.1: Summary of the available header commands.

TARGET

The TARGET command specifies the type of the output assembly (i.e. the “/target”compiler option). Expected values are:

• EXE – a console application (default)

• WINEXE – a windows application

• LIBRARY – a class library (dll)

VER

The VER command specifies which compiler version to use. Default is “v4.0”.

MAIN

The MAIN command specifies the full name of the class with the Main method inthe output executable. It needs to be used if there is more than one such class.

OPT

The OPT command adds its parameter as a separate option into the compiler’scommand line. It can be used to pass additional options to the compiler.

INC

The INC command specifies another source code file to be included when compilingthe output assembly. From the compiler’s perspective, this behaves as if therewere multiple source code files contributing to a single output assembly. Theincluded source code files are also scanned for header sections and the commandsinside them are processed too.

114

REF

The REF command references an assembly. It can also be used to reference anotherscript file. From the compiler’s perspective, this means that the referenced scriptwill be compiled first and the resulting assembly will be used as a reference ofthe current assembly.

IMP

The IMP command references an assembly in the same way as the REF command,but then embeds the referenced assembly file into the output executable andincludes code that will load it correctly when needed. The referenced assemblydoes not need to be distributed with the output executable separately. Thisfeature is experimental and can only be used when compiling executable binaries.

BUILD

The BUILD command compiles the specified script file as a post-processing step,after building of the current script is complete.

EMB

The EMB command includes the specified file into the output assembly as an em-bedded resource.

LNK

The LNK command includes the specified file into the output assembly as a linkedresource.

RES

The RES command includes the specified file into the output assembly as a WPFresource.

DIR

The DIR command specifies the working directory in which the script will belaunched. The specified directory is only used when the script is launched fromthe application.

ARGS

The ARGS command specifies the command line arguments with which the scriptwill be launched. The specified arguments are only used when the script islaunched from the application.

PAUSE

The PAUSE command specifies that the script should pause and display “Press anykey to continue...” just before it exits. The script will be paused like this only ifit is running in a console window and has been launched from the application.

115

B.6.2 Parameters

Many header commands expect a path to a file as a parameter. The path to thefile can be expressed as an absolute path, for example:

INC C:\Work\Scripts\Common.csx

It can also be a path relative to the directory of the current file (the file with thecommand), for example:

INC Common.csx

Wildcards

Most commands can be applied multiple times to different files. Paths passed tothese commands can contain a wildcard character * in their file name. In thiscase, all the files found in the target directory that match the mask will be passedto the command. For example, the following command will include all files withthe csx extension in the Common directory:

INC Common\*.csx

The wildcard character * can be also used as the name of the last directory in thecommand’s parameter. In this case, all subdirectories are also recursively searchedfor the specified file. For example, the following command will include all fileswith the csx extension in the Common directory and also all its subdirectories:

INC Common\*\*.csx

Excluding files

Sometimes it is useful to use a wildcard to include a lot of files except a select few.For this purpose, a hyphen before the parameter indicates that the file should beexcluded. For example, the following commands will include all files with the csxextension in the Common directory, except other.csx:

INC Common\*.csx

INC - Common\other.csx

The decision process takes place after all header commands have been parsed, sothe order of the commands has no significance.

References

The header commands REF and IMP expect a reference as a parameter. By default,the parameter is processed in the same way as file parameters described aboveand can result either in an assembly file which is used directly, or a script filewhich is compiled first and then the resulting assembly is used as the reference.However, if no file is found using the processing described above, the parametervalue is passed to the compiler as a reference directly. This way, standard librariescan be referenced without knowing their full path.

116

B.7 Debugging

The application contains a debugger that behaves similarly to debuggers in clas-sic IDEs. The debugger in the application supports breakpoints, stepping, andexpression evaluation.

B.7.1 Execution control

Debugging is controlled using the commands in the Debug menu. The currentscript is compiled and started in the debugger by the Run command, paused bythe Break command, resumed by the Continue command and terminated by theStop command. Debugging can also be started by the Step Into command, inwhich case the script will be paused at the beginning, or Run To Cursor command,in which case the script is paused when the control reaches the statement at thetext cursor’s position.

Stepping

The debugger contains several commands that resume the execution in a limitedway:

• Step Over – Executes the current statement and pauses again at the nextstatement in the same method.

• Step Into – Executes the current statement and pauses again at the nextstatement in the same method, or at the beginning of a method invokedwhile executing the current statement, if possible.

• Step Out – Continues execution of the current method and pauses again atthe next statement in the method that invoked the current method.

• Run To Cursor – Continues execution until a statement at the text cursor’sposition is reached.

Breakpoints

Breakpoints are places in the source code where execution will be paused au-tomatically. They can be placed on the current line using the Debug → Toggle

Breakpoint command or by clicking in the left margin of the editor. If a break-point already exists at the selected line, it will be removed. The breakpoint’scontext menu (available by right-clicking on the breakpoint in the source code oron its icon in the left margin of the editor) contains commands to delete, disableor enable the breakpoint, and allows to set the breakpoint’s condition, which isa boolean expression that is evaluated each time the breakpoint is reached to seeif the script should be paused there or not.

117

B.7.2 State inspection

When the script is paused, its current state can be inspected as described in thissection.

Current thread and stack frame

The debugged script process may contain multiple threads, each with its own callstack. The debugger allows the user to select one thread and then operates inthe context of this selected thread. Similarly, the current thread’s call stack maycontain multiple stack frames (invoked methods), each with its own set of localvariables and current statement position. The debugger allows the user to selectone stack frame and then operates in the context of this selected stack frame.

Current statement

The current statement is the statement in the source code that is just beingexecuted or just about to be executed. It depends on the current stack frame(each stack frame has its own current statement) and it is highlighted in thesource code in different colors:

• Yellow – The highlighted statement has not been executed yet, but itsexecution is just about to start. There is no stack frame above the currentone in the call stack of the current thread.

• Green – The highlighted statement is just being executed, but its executionhas not finished yet. There is at least one more stack frame above thecurrent one in the call stack of the current thread.

Variable values

When the script is paused, users can inspect current values of variables. Whenthe mouse cursor is hovered over a variable in the source code, its current valueis displayed in a tooltip. More complex inspections can be performed using thewatch and console windows described in the following section.

B.7.3 Windows

There are several windows in the application that are available only when thedebugger is active. These windows can be displayed using respective commandsin the Debug menu or using the corresponding toolbar buttons.

Threads

The Threads window lists all threads of the currently debugged script, indicateswhich thread is currently selected, and allows switching of the currently selectedthread.

118

Call Stack

The Call Stack window lists stack frames (method calls) in the call stack of thecurrently selected thread, indicates which stack frame is currently selected, andallows switching of the currently selected stack frame.

Exception

The Exception window contains details of the current exception. It is automati-cally displayed if the script was paused because an exception has been thrown.

Watch

The Watch window allows creating one or more watch expressions. These watchexpressions are then evaluated each time the script is paused and their values aredisplayed in a table.

Console

The Console window offers a command line interface that allows evaluation ofcustom expressions. When an expression is entered, it is immediately evaluatedin the context of the current statement. The resulting value of the expression isthen printed out as text. This can be used for deeper inspection of variable values,and also to modify the state of the script since method calls and assignments canalso be executed.

B.8 Configuration

The application is configured using an external XML configuration file. Theconfiguration file must be placed in the same directory as the application’s exe-cutable file and it must have the same name, but the .cfg extension instead of.exe. If there is no configuration file, the application uses the default configu-ration file stored internally. This internal configuration file contains the defaultconfiguration of the application, and so users are not required to create or editthe configuration file if they are satisfied with the current configuration.

The application contains no specialized user interface for modifying config-uration. However, the configuration file can be edited in the application itselfthe same way as any other XML file. The application detects changes to theconfiguration file and applies them as soon as the modified file is saved.

The current configuration file can be opened in the editor using the Edit →

Preferences command. If the file does not exist yet, it is created as a copy ofthe default configuration file that is stored internally.

Also note that the current configuration file will be embedded into the app-lication if the user packages the application into a stand-alone executable, asdescribed in section B.1.

119

B.8.1 Configuration sections

The configuration file is formatted as an XML document whose root node containsone element for each separate configuration section. The exact format of theseelements depends on the section and is described in the following text.

Logging configuration

The Logging configuration section allows configuring the path where the applica-tion writes its log file. It looks like this:

<Logging>

<Path>log.txt</Path>

</Logging>

By default, the application writes the log to a file with the same path as theapplication’s executable file, but the .log extension instead of .exe. Alternatively,the Path element can be left empty, in which case the log is not written anywhere.

Text editor configuration

The TextEditor configuration section allows configuring the editor’s font. It lookslike this:

<TextEditor>

<FontFamily>Consolas</FontFamily>

<FontSize>14</FontSize>

</TextEditor>

Plugin configuration

The Plugins configuration section specifies what plugins will be loaded at theapplication’s startup. It looks like this:

<Plugins>

<Plugin Name="ExtBrain.ScriptDevelop.Plugin.*.dll" Delay="0" />

</Plugins>

The Delay attribute specifies time in milliseconds after the application’s startupthat the plugins specified in the element will be loaded.

Precached assemblies configuration

The PrecachedAssemblies configuration section specifies assemblies that will beloaded in advance at the application’s startup. It looks like this:

<PrecachedAssemblies>

<Assembly>System.dll</Assembly>

<Assembly>System.Core.dll</Assembly>

<Assembly>System.Xml.dll</Assembly>

<Assembly>System.Xml.Linq.dll</Assembly>

<Assembly>System.Drawing.dll</Assembly>

</PrecachedAssemblies>

120

Types, extension methods and namespaces from these assemblies will be offeredin code completion and context actions even when their respective assemblies arenot referenced yet in the script.

Keyboard shortcuts configuration

The KeyBindings configuration section allows overriding the keyboard shortcutof any command in the application. It looks like this:

<KeyBindings>

<KeyBinding Command="GoToTypeCommand" Shortcut="Ctrl+T" />

</KeyBindings>

The Command attribute specifies the short or full name of the class that implementsthe command. The Shortcut attribute specifies the overridden keyboard shortcutfor the command. The shortcut is parsed using the KeyGestureConverter classof WPF, so it follows the same rules. Multiple shortcuts can be specified andseparated by semicolons.

Code analysis configuration

The CodeAnalysis configuration section allows overriding the default severity ofany code analysis provider in the application. It looks like this:

<CodeAnalysis>

<OverrideSeverity Provider="RedundantUsingDirective"

Severity="Suggestion" />

<OverrideSeverity Provider="InvokeAsExtensionMethod"

Severity="None" />

<OverrideSeverity Provider="InconsistentNamingIssue"

Severity="None" />

</CodeAnalysis>

The Severity attribute can be set to one of the following values:

• None – The provider will be disabled.

• Hint – The issues found by the provider will be indicated only by a subtleshort underline and will not be shown in the shortcut column.

• Suggestion – The issues found by the provider will be indicated by a greencolor.

• Warning – The issues found by the provider will be indicated by an orangecolor and will count as warnings in the shortcut column.

• Error – The issues found by the provider will be indicated by a red colorand will count as errors in the shortcut column.

The Provider attribute specifies the name of the code analysis provider. Theprovider names can be found out in the application by placing the text cursorupon an issue and choosing Configure Severity from the context actions menu.This will open the configuration file and copy the respective OverrideSeverity

XML element into the clipboard.

121

Code formatting configuration

The CSharp configuration section allows overriding the default formatting of theC# source code that is used when performing refactoring transformations. Itlooks like this:

<CSharp>

<CSharpFormattingOptionsPreset>

Allman

</CSharpFormattingOptionsPreset>

</CSharp>

The value of the CSharpFormattingOptionsPreset element can be oneof the following values: Allman, GNU, KRStyle, Mono, SharpDevelop, orWhitesmiths. These values correspond with the presets defined in theFormattingOptionsFactory class of the NRefactory library. Alternatively,TextEditorOptions and CSharpFormattingOptions elements can be used in-stead of the CSharpFormattingOptionsPreset element to specify custom format-ting. The structure of these elements is defined by the TextEditorOptions

and CSharpFormattingOptions classes of the NRefactory library (the standardXmlSerializer is used to deserialize them).

ILSpy configuration

The ILSpy configuration section allows specifying the path to the ILSpy [17]executable. It looks like this:

<ILSpy>

<Path>ILSpy\ILSpy.exe<Path>

</ILSpy>

The path can be either absolute, or relative to the application’s executable. IfILSpy is properly configured, the Go To Definition command can be used tonavigate to an external entity’s decompiled source code in ILSpy.

122

C. Extension points

This appendix provides an overview of the most useful extension points in theapplication. These extension points allow quickly extending the functionality ofthe application without having to modify the existing source code base.

Plugin

To add a new plugin that needs to be initialized upon startup to the application,implement the IPluginBase interface. For details, see section 4.2.1.

Command

To add a new command into the application, implement the IExtendedCommand

interface and export it using the ExportCommand attribute. Commands can bealso added to the main menu, to the toolbar, and to various context menus usingthe MenuCommand and ToolbarCommand attributes. For details, see section 4.3.2.

There are several helper IExtendedCommand implementations that can be de-rived from, for example:

• BaseCommand – Basic implementation, also implements IPluginBase for ini-tialization.

• SimpleCommand – Basic implementation, exposes methods without theparameter parameter.

• DockableContentCommand<T> – Implements a command for showing and hid-ing a dockable panel.

• DocumentCommand<TFormat, TDocument> – Implements a command that actsupon a document of a specific format and type.

Document extension

A new functionality can be added to every document open in the applicationby implementing the IEditorExtensionPlugin interface. For details, see sec-tion 4.4.2.

Dockable panel

A new dockable panel can be added to the application by inheriting from theDockableContentPlugin class.

Configuration section

A new section can be added to the configuration file by inheriting from theConfigurationPlugin class. For details, see section 4.2.2.

123

Persisted settings

A new persisted settings object can be added to the application by inheritingfrom the SettingsPlugin class. For details, see section 4.2.2.

Main window component

A new component can be added to the main window by implementing theIMainWindowComponentPlugin interface.

Status bar component

A new component can be added to the status bar by implementing theIStatusBarComponentPlugin interface.

Code analysis provider

A new code issue or code action provider can be added to the application byimplementing the ICodeIssueProvider or ICodeActionProvider interface. Fordetails, see section 4.7.2.

Code completion provider

New entries to the code completion window can be added by implementing theICompletionSourceProvider interface. For details, see section 4.4.4.

Debugger visualizer

New debugger visualizers can be added by implementing the IVisualizer in-terface and exporting it using the ExportVisualizer attribute. For details, seesection 4.8.

124