Extending Type Systems in a Library

Yuriy Solodkyy, Jaakko Järvi, Esam MlaihDepartment of Computer Science and Engineering

Texas A&M UniversityMarch 23, 2010

http:://parasol.tamu.edu/~yuriys/

2

Motivation: Trends

Programs grow in size and complexityo Use multiple librarieso Depend on third party components (APIs, protocols)o Have to account for OS, hardware & compiler

differenceso Are written by many people with different skill sets

Languages grow in abstraction levelo Crave for more featureso Have to deal with backward compatibilityo Provide larger standard librarieso Domain specialization

3

Motivation: Bugs

Cost U.S. economy $60 billion each yearo users incurred 64% of the cost o developers and vendors – 36%

Improvements in testingo can reduce it by about a third ($23 billion)o won't eliminate all software errors

Collection of Software Bugs by Thomas Huckle

4

Motivation: Bugs

Collection of Software Bugs by Thomas Huckle

Ariane 5 Explosionnarrowing conversion6/4/1996, $500 million

NASA Mars Climate Orbitermixing metric and imperial units8/23/1999, $125 million

LA Air-Traffic Controlcounter underflow9/14/2004

Patriot Missile Failurefloat imprecision rounding02/25/1991, 28 dead 100 injured

Zune's New Year Freezeinfinite loop12/31/2008

Pentium FDIV bugincomplete entries in a look-up-table1994, $400 million

5

Motivation: Solutions

Testing is not a panaceao Does not prove absence of errorso Specific to a project and cannot be reusedo Has to be maintained in sync with evolution of projecto Improvements in it will reduce the cost of bugs by about a

third, but won't eliminate all software errors Many errors can be detected without running the

program through the use of:o Type systemo Static analysis

Sometimes absence of specific run-time errors can be proven

6

The problem with Type System

Interesting ones are domain-specifico e.g units, qualifiers

Impossible to account for all interesting ones in general-purpose programming languageo covariance typing, full static typing and subtype

substitutability: pick twoo can be designed and approached differently

Composability of multiple type systems in a single program is hard to achieveo e.g. units for measurements and regular expression

types for patterns

Solodkyy et al. LCSD’06 7

XTL: eXtensible Typing Library Type systems

– prevent certain kinds of bugs from happening

– not easily extensible– domain specific

Observation– many interesting domain

specific type-systems can be implemented as libraries

Objective– explore how far a pure

library solution suffices in extending a type system

Solodkyy et al. LCSD’06 8

Use Cases Tracking physical quantities/units

– converting between compatible units– rejecting operations that do not agree on

units Tracking semantic properties

– nullability, sign, oddity of a number– security vulnerability to format strings– usage of user pointers in kernel space– deadlocks and data races

Regular Expression Types– typing XML documents– ordering patterns

Variant Parametric Types– defining relations between instances of a

parameterized type based on relations of its argument types

units– kg = lbs – ok, convert– kg = km/h – error

semantic properties– even+odd=odd– sprintf(buf,…);– T* krnl = usr;: error

regular expression types– T*U* <: (T|U)*

variant parametric types– vector<B*> <: vector<D*>

Solodkyy et al. LCSD’06 9

XTL Contributions

We report on implementing simple type qualifiers as a C++ library

We implement regular expression types (in a limited form) to check XML data

We provide a framework to help others in extending the C++ type system for their abstractions

10

Example

// a is a positive valuedouble a = current_height();

// b is a positive valuedouble b = max_allowed_height();

// b-a may be negative though! Result is always positivedouble c = std::sqrt(b-a);

// b+a is assumed to be positive. Unless there is bug!double d = std::sqrt(b+a);

returns only positive numbers

upper bound for values from previous

call

difference is positive, but not for type system

argument and result are always positive

Solodkyy et al. LCSD’06 11

Refining Type Systems

What do we need? How do we achieve that in C++?

Useful building blocks

Typing rules Type construction via class templates

tuple, variant, optional

Evaluation rules Function templates and overloading

enable_if

Subtyping rules A dedicated meta-function

MPL

Composability Library-only solution Interoperability of the above libraries

Objective: emulate domain-specific type system as a library

Solodkyy et al. LCSD’06 12

Type System 1: Type qualifiers

Allow tracking of semantic properties like:– immutability of a certain value (const)– sign of a number (pos, neg)– assumptions about pointers (optional, nonnull)– trustworthiness of a certain value (tainted, untainted)– oddity of a number (odd, even)– origin of a pointer (user, kernel)

Easily defined in terms of:– direction (positive/negative qualifier)– abstract operations on qualifiers

But– cannot handle flow-sensitive qualifiers– cannot handle arbitrary reference qualifiers

Objective: providing support for type qualifiers in programs

Solodkyy et al. LCSD’06 13

Example: Qualifiers’ Hello World#include <xtl/qualdecl.hpp>DECLARE_NEGATIVE_QUALIFIER(pos);DECLARE_POSITIVE_QUALIFIER(tainted);// ... Other qualifiers ...namespace xtl {

template <> struct minus<pos, neg> { typedef qual<pos> type; };template <> struct minus<neg, pos> { typedef qual<neg> type; };template <> struct mul<pos, pos> { typedef qual<pos> type; };template <> struct mul<pos, neg> { typedef qual<neg> type; };template <> struct mul<nonnull, nonnull>{ typedef qual<nonnull> type;};template <> struct div<nonnull, nonnull>{ typedef qual<nonnull> type;};

} // of namespace xtlint main(){

untainted<nonnull<neg<int> > > a(-42);pos<untainted<nonnull<int> > > b(7);neg<nonnull<long> > c = a * b; // OK: drop negative qualifier untainted//nonnull<pos<double> > d = b - a; // Error: nonnull isn’t carried by -pos<tainted<double> > e = b + a*c; // OK to add positive qualifier//pos<double> f = e; // Error: ... but not OK to drop it!

}

Declare few qualifiers:Q is positive if T <: Q TQ is negative if Q T <: T

Define how different operations transfer

properties

Declare your variables with appropriate

properties

Multiplication carries nonnull & negativeness

on pos & neg arguments

Subtraction does not carry nonnull

Positive qualifiers can be added to

result type...... but not

dropped once they are there!

14

Example// Example definition that accepts only positive doublestemplate<class U>typename enable_if< typename is_subtype<U, pos<double> >::type, void>::type descend(const U& altitude){ pos<double> a = subtype_cast<pos<double> >(altitude); //...};

// Data coming from measurements is marked untaintedextern pos<untainted<int> > get_corridor_height();

untainted<pos<int> > a = get_corridor_height();descend(a); // no negative altitudes here!

function descend accepts only positive

numbers

we achieve this by restricting its argument type to be a subtype of

pos<double>

we convert value of a subtype into a value of

a supertype

returns a value that is both: positive and

untainted

a can hold positive, untainted values. order

of qualifiers is not important!

no negative altitudes can appear here!

Solodkyy et al. LCSD’06 15

Type System 2: XML Typing

Types can describe XML elements with certain structure– sequences, alternatives, elements etc.

Subtyping describes structurally more powerful types– types that can hold all the values of their subtype

Compile-time assurance that only valid XML documents are produced– when document schema changes, are we backward

compatible with the old one? Value-preserving type conversions

– logically the same entities can be represented by different XML elements

Objective: providing support for typing XML snippets in programs

16

typedef element<name, string> XMLname;typedef element<email,string> XMLemail;typedef element<icq, int> XMLicq;

typedef element<contact, boost::variant< XMLemail, XMLicq >> XMLcontact;

typedef element<person, fusion::tuple< XMLname, XMLcontact >> XMLperson;

<xsd:element name="name" type="xsd:string"/><xsd:element name="email" type="xsd:string"/><xsd:element name="icq" type="xsd:decimal"/>

<xsd:element name="contact"> <xsd:complexType> <xsd:choice> <xsd:element ref="email"/> <xsd:element ref="icq"/> </xsd:choice> </xsd:complexType></xsd:element>

<xsd:element name="person"> <xsd:complexType> <xsd:sequence> <xsd:element ref="name"/> <xsd:element ref="contact"/> </xsd:sequence> </xsd:complexType></xsd:element>

XML Schema’s choice is mapped to Boost

variant

back references are mapped to previous

typedefs

we map XML data types into C++ types

for each tag we create a dedicated tag-typeExample

XML Schema C++

for each tag we create a dedicated tag-type

we map XML data types into C++ typesback references are

mapped to previous typedefs

back references are mapped to previous

typedefs

XML Schema’s sequence is mapped to

Fusion’s tuple

XML Schema’s sequence is mapped to

Fusion’s tuple

XML Schema’s choice is mapped to Boost

variant

we use a dedicated type element to represent

XML elements

we use a dedicated type element to represent

XML elements

17

Example// ...typedef variant<Email,Tel,ICQ> AnyContact;typedef element<person, tuple<Name, Tel, ICQ> > Person;typedef element<person, tuple<Name, AnyContact, AnyContact> > PersonEx;

int main(){ Person p(make_tuple(Name("Yuriy"), Tel("555-4321"), ICQ(1234))); PersonEx x = p; // OK: Subtyping conversion // p = x; // ERROR: Not in subtyping relation

ifstream xml("old-person.xml"); xml >> x; // read data from XML file. assumes file exist cout << x << endl; // show XML source on the screen}

Tel <: AnyContactICQ <: AnyContactName, Tel, ICQ <: Name, AnyContact,

AnyContactPerson <: PersonEx

Instantiate an XML snippet that

corresponds to Person type

Person <: PersonExAssignment involves subtype conversionPersonEx is not a subtype of Person

Parses only XML files that correspond to PersonEx schemaProduces XML source

on the screen

18

XDuce Type

– set of sequences over a certain domain Regular Expression Types

– concatenation : A,B– alternation : A|B– repetition : A*– optional : A?– type construction : l[A]– recursion : X = A,X | ø

Subtyping– inclusion between the sets defined by types

19

C++ Type

– set of sequences over a certain domain Regular Expression Types

– concatenation : A,B tuple<A,B>– alternation : A|B variant<A,B>– repetition : A* vector<A>– optional : A? optional<A>– type construction : l[A] element<l,A>– recursion : X = A,X | ø –

Subtyping– is_subtype and subtype_cast

Objective: similar representation and the same semantics as in XDuce

Solodkyy et al. LCSD’06 20

XTL’s Strengths Simplicity

– reasonably powerful type systems can be built without creating a language tool

Genericity– common interface for defining custom subtyping relation– common interface for defining conversion from a subtype

to a supertype Reusability

– ready definitions to be used in other type systems subtyping of array types subtyping of function types subtyping of sequences and union types subtyping of qualified types

Solodkyy et al. LCSD’06 21

XTL’s Limitations Unable to take information about control-flow into

account– if (x>0) … does not make the type of x - pos<int>

Slows down compilation on complex type systems– e.g. XML types

Scaling problems on complex type systems– XML typing is exponential

No implicit transitivity of subtyping relation– library has no access to all available types

Meta-function join may return an arbitrary upper bound– same reason – no global information on all types

22

THANK YOU!

Abstract Questions deserve

Abstract Answers

23

Compilation Times

n/k 1 2 3 4 5 6 7 8 91 1.48 1.66 1.73 1.81 2.03 2.14 2.35 2.54 2.79

2 1.52 1.77 2.29 3.67 8.28 27.38

3 2.43 2.00 2.66 6.00 31.43

4 2.62 2.03 3.63 25.57

5 2.32 2.28 7.15

6 2.73 4.13 19.30

7 2.48 2.86 56.78

8 1.74 3.65

9 1.76 4.93

N 0 1 2 3 4 5 6 7 8 9 10

Time 1.12 1.72 2.61 2.81 3.18 3.94 4.97 5.55 6.28 15.40 19.22

XML Typing

Type Qualifiers