The Roslyn Compiler: Look at Your Code from a Different Perspective (Raffaele Rialdi)

Premium community conference on Microsoft technologies itcampro@ itcamp14#

Look at your code from a different perspective

twitter: @raffaeler

email: [email protected]

blog: http://www.iamraf.net

The Roslyn compiler:


Huge thanks to our sponsors & partners!


• .NET C# and VB compilers totally re-written

–Using C# and VB

• Open source under the Apache 2.0 license

–https://roslyn.codeplex.com/

• Will be shipped with VS.NET vNext

Roslyn aka ".NET Compiler Platform"

git repo

Usually a new compiler is important because of new language features …

https://roslyn.codeplex.com/


1. Auto-properties initializers

2. Primary constructors

3. Using static

4. Declaration Expressions

Upcoming C#6.0 new features

public string Name { get; set; } = “Raf”;

public class Dog(string name) {public string Name { get; } = name;

}

using System.Console;using System.Math;

if(double.TryParse(text, out double value)) { … }

WriteLine("Hello, world {0}", Max(a, b));ReadKey();


5. Exception filters

6. Binary literals and digit separators

7. Indexed members and element initializers

8. Await in catch and finally blocks

Upcoming C# 6.0 new features

var bits1 = 0b00010001;var bits2 = 0b0001_0001;

var hex = 0x00_C6;var dec = 1_234_000;

try { … }catch(SomeException e) if(myFilter(e)) { … }

customer["first“] = “Raf“; customer.$first = “Raf“;

catch(…) { await … }finally { await … }


Why Roslyn is relevant?

• Microsoft perspective– Only one compiler engine for both VB and C#– Future evolutions of C# and VB will be easier to

implement– Better tooling inside Visual Studio

• Visual Studio plugin developers– Better refactoring, coloring, helpers, static analysis, etc.

• All the other developers– Using C# and VB as scripting languages– Documentation and code-search via syntactical analysis– Enforcing business rules with custom semantical

analysis


Where to start

I want to just try the new compiler– Nuget package: Microsoft.Net.ToolsetCompilers

– Will use the compiler inside the package itself

Tools for developers– Nuget package:

Microsoft.CodeAnalysis.CSharp.Workspaces

Solutions using Compilers and tools– Nuget package:

Microsoft.CodeAnalysis (contains all)

Visual Studio templates


THE PIPELINE


The pipeline workflow

• The source is tokenized– Syntax analysis using the grammar rules

• Recognizing the declarations– Symbols creation to name variables, classes, methods, …

• Binding the identifiers with symbols– Retrieve symbolic information from nodes

• The code is generated to form the assembly– Generation of the PE file, manifest, IL code, resources, etc.– Note: the APIs for reading/writing these files is marked 'internal'

Parsing sources

Creating symbols

Semantic model

IL Emitter


• Transformation from text to nodes is reversible– Full fidelity (comments and alignments as well)

• Every syntax tree is immutable and thread-safe– Allow multiple consumers to access concurrently

a single tree

Parsing the sources

… ……


• Syntax API is represented with these classes:– SyntaxTree is the binary form of the source with full

fidelity– SyntaxNode represents declarations, statements,

clauses and expressions– SyntaxToken identifies the special language tokens like

keywords, identifiers, operators or punctuation• They never have children

– SyntaxTrivia everything not impacting on the generated IL code such as whitespaces, comments, etc.

From sources to syntax tree

SyntaxTree tree = CSharpSyntaxTree.ParseText(source);SyntaxNode node = tree.GetRoot();Debug.Assert(node.CSharpKind() == SyntaxKind.CompilationUnit);var root = (CompilationUnitSyntax)node;


Visual Studio Syntax Tree tool


Applying the language rules

• The syntax tree represent only the basic info

• The compilation applies the language rules to the syntax tree and extracts the symbols– Symbols are declared namespaces, types,

members, variables, etc.

• Compilations are immutable too– therefore they are thread-safe

SyntaxTree tree = CSharpSyntaxTree.ParseText(source);var compilation = CSharpCompilation.Create("CSharpColorizer",

new SyntaxTree[] { tree },_metadataFileReferences,new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary));

var result = compilation.Emit(ms); // optional


Semantics

• «Semantics is the study of the meaning»

– This translates in "understanding the code"

• Semantic info correlates the symbols derived from the syntax tree and accessing:

– The exact location in the sources

– Warnings and errors

– The types returned from an expression

– Much more!

• SemanticModel is immutable and thread-safe


Working with projects and solutions

• Symbolic dependencies may affect even an entire VS Solution

– The compiler needs to know about solutions, projects, files

• Workspace API entry-points

–Workspaces resemble solutions

–Projects are, well, projects

–Documents are the files subject to compilation


Workspaces

var workspace = new CustomWorkspace();var solution = workspace.CurrentSolution; // the only solutionvar project = solution.AddProject(

"projectName", "assemblyName", LanguageNames.CSharp);var mainDocument = project.AddDocument("fake.cs", source);

• Workspaces fire events– On opening/closing documents– When something change in the solution– If an error occurs

• Flowchart:– IDE opens & closes the documents, refactors the code, …– Tools receive the events and reacts appropriately

• CustomWorkspace allows working on the fly (no IDE)

Workspace(Solution)

Project

Document


Model hierarchy

Host (IDE)

Workspace

Solution v1

Project1 Project2 Project3

Document1 Document2

Text Syntax SemanticModel

Tools

Solution v2


• Roslyn compiler is deterministic

– Same sources identical binary

– Super-important for debugging!

• Immutable and thread-safe APIs

– Syntax trees, Compilation, SemanticModel, Solution, Project, Document

• http://referencesource.microsoft.com/

–Roslyn powered

Tips


Questions?

Technology

The Roslyn Compiler: Look at Your Code from a Different Perspective (Raffaele Rialdi)