18
A Type System for Object Initialization In the Java TM Bytecode Language * Stephen N. Freund John C. Mitchell Department of Computer Science Stanford University Stanford, CA 94305-9045 {freunds, mitchell}@cs.stanford.edu Abstract In the standard Java implementation, a Java language program is compiled to Java bytecode. This bytecode may be sent across the network to another site, where it is then interpreted by the Java Virtual Machine. Since bytecode may be written by hand, or corrupted during network transmission, the Java Virtual Machine con- tains a bytecode verifier that performs a number of con- sistency checks before code is interpreted. As illustrated by previous attacks on the Java Virtual Machine, these tests, which include type correctness, are critical for sys- tem security. In order to analyze existing bytecode ver- ifiers and to understand the properties that should be verified, we develop a precise specification of statically- correct Java bytecode, in the form of a type system. Our focus in this paper is a subset of the bytecode lan- guage dealing with object creation and initialization. For this subset, we prove that for every Java bytecode program that satisfies our typing constraints, every ob- ject is initialized before it is used. The type system is easily combined with a previous system developed by Stata and Abadi for bytecode subroutines. Our anal- ysis of subroutines and object initialization reveals a previously unpublished bug in the Sun JDK bytecode verifier. 1 Introduction The Java programming language is a statically-typed general-purpose programming language with an imple- mentation architecture that is designed to facilitate transmission of compiled code across a network. In * Supported in part by NSF grants CCR-9303099 and CCR- 9629754, ONR MURI Award N00014-97-1-0505, and a NSF Graduate Research Fellowship. To appear in OOPSLA ’98. the standard implementation, a Java language pro- gram is compiled to Java bytecode and this bytecode is then interpreted by the Java Virtual Machine. While many previous programming languages have been im- plemented using a bytecode interpreter, the Java archi- tecture differs in that programs are commonly transmit- ted between users across a network in compiled form. Since bytecode may be written by hand, or corrupted during network transmission, the Java Virtual Machine contains a bytecode verifier that performs a number of consistency checks before code is interpreted. Fig- ure 1 shows the point at which the verifier checks a program during the compilation, transmission, and ex- ecution process. After a class file containing Java byte- codes is loaded by the Java Virtual Machine, it must pass through the bytecode verifier before being linked into the execution environment and interpreted. This protects the receiver from certain security risks and var- ious forms of attack. The verifier checks to make sure that every opcode is valid, all jumps lead to legal instructions, methods have structurally correct signatures, and that type con- straints are satisfied. Conservative static analysis tech- niques are used to check these conditions. As a result, many programs that would never execute an erroneous instruction are rejected. However, any bytecode pro- gram generated by a conventional compiler is accepted. The need for conservative analysis stems from the un- decidability of the halting problem, as well as efficiency considerations. Specifically, since most bytecode is the result of compilation, there is very little benefit in de- veloping complex analysis techniques to recognize pat- terns that could be considered legal but do not occur in compiler output. The intermediate bytecode language, which we re- fer to as JVML, is a typed, machine-independent form with some low-level instructions that reflect specific high-level Java source language constructs. For exam- ple, classes are a basic notion in JVML, and there is a form of “local subroutine” call and return designed

A T yp e System for Ob ject ... - Williams College

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: A T yp e System for Ob ject ... - Williams College

A Type System for Object InitializationIn the JavaTM Bytecode Language∗

Stephen N. Freund John C. MitchellDepartment of Computer Science

Stanford UniversityStanford, CA 94305-9045

{freunds, mitchell}@cs.stanford.edu

Abstract

In the standard Java implementation, a Java languageprogram is compiled to Java bytecode. This bytecodemay be sent across the network to another site, where itis then interpreted by the Java Virtual Machine. Sincebytecode may be written by hand, or corrupted duringnetwork transmission, the Java Virtual Machine con-tains a bytecode verifier that performs a number of con-sistency checks before code is interpreted. As illustratedby previous attacks on the Java Virtual Machine, thesetests, which include type correctness, are critical for sys-tem security. In order to analyze existing bytecode ver-ifiers and to understand the properties that should beverified, we develop a precise specification of statically-correct Java bytecode, in the form of a type system.Our focus in this paper is a subset of the bytecode lan-guage dealing with object creation and initialization.For this subset, we prove that for every Java bytecodeprogram that satisfies our typing constraints, every ob-ject is initialized before it is used. The type system iseasily combined with a previous system developed byStata and Abadi for bytecode subroutines. Our anal-ysis of subroutines and object initialization reveals apreviously unpublished bug in the Sun JDK bytecodeverifier.

1 Introduction

The Java programming language is a statically-typedgeneral-purpose programming language with an imple-mentation architecture that is designed to facilitatetransmission of compiled code across a network. In

∗Supported in part by NSF grants CCR-9303099 and CCR-9629754, ONR MURI Award N00014-97-1-0505, and a NSF GraduateResearch Fellowship.

To appear in OOPSLA ’98.

the standard implementation, a Java language pro-gram is compiled to Java bytecode and this bytecodeis then interpreted by the Java Virtual Machine. Whilemany previous programming languages have been im-plemented using a bytecode interpreter, the Java archi-tecture differs in that programs are commonly transmit-ted between users across a network in compiled form.

Since bytecode may be written by hand, or corruptedduring network transmission, the Java Virtual Machinecontains a bytecode verifier that performs a numberof consistency checks before code is interpreted. Fig-ure 1 shows the point at which the verifier checks aprogram during the compilation, transmission, and ex-ecution process. After a class file containing Java byte-codes is loaded by the Java Virtual Machine, it mustpass through the bytecode verifier before being linkedinto the execution environment and interpreted. Thisprotects the receiver from certain security risks and var-ious forms of attack.

The verifier checks to make sure that every opcodeis valid, all jumps lead to legal instructions, methodshave structurally correct signatures, and that type con-straints are satisfied. Conservative static analysis tech-niques are used to check these conditions. As a result,many programs that would never execute an erroneousinstruction are rejected. However, any bytecode pro-gram generated by a conventional compiler is accepted.The need for conservative analysis stems from the un-decidability of the halting problem, as well as efficiencyconsiderations. Specifically, since most bytecode is theresult of compilation, there is very little benefit in de-veloping complex analysis techniques to recognize pat-terns that could be considered legal but do not occur incompiler output.

The intermediate bytecode language, which we re-fer to as JVML, is a typed, machine-independent formwith some low-level instructions that reflect specifichigh-level Java source language constructs. For exam-ple, classes are a basic notion in JVML, and there isa form of “local subroutine” call and return designed

Page 2: A T yp e System for Ob ject ... - Williams College

Java

CompilerLoader

Bytecode

Interpreter

class file

B.class

! ! !

"

"

"

#

. .. .

. .. .

. .. .

. .. .

. .. .

. ..}

}

class A {void f() {...

A.java A.class

class file

Java Virtual Machine

Verifier

Linker

network

Trusted CodeUntrusted Code

Figure 1: The Java Virtual Machine

to allow efficient implementation of the source languagetry-finally construct. While some amount of type in-formation is included in JVML to make type-checkingpossible, there are some high-level properties of Javasource code that are not easy to detect in the resultingbytecode. One example is the last-called first-returnedproperty of the local subroutines. While this propertywill hold for every JVML program generated by compil-ing Java source, some effort is required to confirm thisproperty in bytecode programs [SA98].

Another example is the initialization of objects be-fore use. While it is clear from the Java source languagestatement

A x = new A(〈parameters〉)

that the A class constructor will be called before anymethods can be invoked through the pointer x, this isnot obvious from a simple scan of the resulting JVMLprogram. One reason is that many bytecode instruc-tions may be needed to evaluate the parameters for thecall to the constructor. In the bytecode, these will beexecuted after space has been allocated for the objectand before the object is initialized. Another reason, dis-cussed in more detail in Section 2, is that the structureof the Java Virtual Machine requires copying of point-ers to uninitialized objects. Therefore, some form ofaliasing analysis is needed to make sure that an objectis initialized before it is used.

Several published attacks on early forms of the JavaVirtual Machine illustrate the importance of the byte-code verifier for system security. To cite one specific

example, a bug in an early version of Sun’s bytecodeverifier allowed applets to create certain system objectswhich they should not have been able to create, suchas class loaders [DFW96]. The problem was caused byan error in how constructors were verified and resultedin the ability to potentially compromise the security ofthe entire system. Clearly, problems like this give riseto the need for a correct and formal specification ofthe bytecode verifier. However, for a variety of reasons,there is no established formal specification; the primaryspecification is an informal English description that isoccasionally at odds with current verifier implementa-tions.

Building on a prior study of the bytecodes for localsubroutine call and return [SA98], this paper developsa specification of statically-correct bytecode for a frag-ment of JVML that includes object creation (alloca-tion of memory) and initialization. This specificationhas the form of a type system, although there are sev-eral technical ways in which a type system for low-levelcode with jumps and type-varying use of stack locations(or registers) differs from conventional high-level typesystems. We prove soundness of the type system bya traditional method using operational semantics. Itfollows from the soundness theorem that any bytecodeprogram that passes the static checks will initialize ev-ery object before it is used. We have examined a broadrange of alternatives for specifying type systems capa-ble of identifying that kind of error. In some cases, wefound it possible to simplify our specification by being

2

Page 3: A T yp e System for Ob ject ... - Williams College

more or less conservative than current verifiers. How-ever, we generally resisted the temptation to do so sincewe hoped to gain some understanding of the strengthand limitations of existing verifier implementations.

In addition to proving soundness for the simple lan-guage, we have structured the main lemmas and proofsso that they apply to any additional bytecode com-mands that satisfy certain general conditions. Thismakes it relatively straightforward to combine our anal-ysis with the prior work of Abadi and Stata, showingtype soundness for bytecode programs that combine ob-ject creation with subroutines. In analyzing the interac-tion between object creation and subroutines, we haveidentified a previously unpublished bug in the Sun im-plementation of the bytecode verifier. This bug allows aprogram to use an object before it has been initialized;details appear in Section 7. Our type-based frameworkalso made it possible to evaluate various repairs to fixthis error and prove correctness for a modified system.

Section 2 describes the problem of object initializa-tion in more detail, and Section 3 presents JVMLi,the language which we formally study in this paper.The operational semantics and type system for this lan-guage is presented in Section 4. Some sound extensionsto JVMLi, including subroutines, are discussed in Sec-tion 6, and Section 7 describes how this work relates toSun’s implementation. Section 8 discusses some otherprojects dealing with bytecode verification, and Sec-tion 9 gives directions for future work and concludes.

2 Object Initialization

As in many other object-oriented languages, the Javaimplementation creates new objects in two steps. Thefirst step is to allocate space for the object. This usuallyrequires some environment-specific operation to obtainan appropriate region of memory. In the second step,user-defined code is executed to initialize the object. InJava, the initialization code is provided by a constructordefined in the class of the object. Only after both ofthese steps are completed can a method be invoked onan object.

In the Java source language, allocation and initial-ization are combined into a single statement. This isillustrated in the following code fragment.

Point p = new Point(3);p.Print();

The first line indicates that a new Point object shouldbe created and calls the Point constructor to initializethis object. The second line invokes a method on thisobject and therefore can be allowed only if the objecthas been initialized. Since every Java object is createdby a statement like the one in the first line here, it does

not seem difficult to prevent Java source language pro-grams from invoking methods on objects that have notbeen initialized. While there are a few subtle situationsto consider, such as when a constructor throws an ex-ception, the issue is essentially clear cut.

It is much more difficult to recognize initialization-before-use in bytecode. This can be seen by looking atthe five lines of bytecode that are produced by compilingthe preceding two lines of source code:

1: new #1 <Class Point>2: dup3: iconst_34: invokespecial #4 <Method Point(int)>5: invokevirtual #5 <Method void Print()>

The most striking difference is that memory allocation(line 1) is separated from the constructor invocation(line 4) by two lines of code. The first intervening line,dup, duplicates the pointer to the uninitialized object.The reason for this instruction is that a pointer to theobject must be passed to the constructor. A conventionof parameter passing for the stack-based architecture isthat parameters to a function are popped off the stackbefore the function returns. Therefore, if the addresswere not duplicated, there would be no way for the codecreating the object to access it after it is initialized. Thesecond line, iconst 3 pushes the constructor argument3 onto the stack. If p were used again after line 5 ofthe bytecode program, another dup would have beenneeded prior to line 5.

Depending on the number and type of constructorarguments, many different instruction sequences mayappear between object allocation and initialization. Forexample, suppose that several new objects are passed asarguments to a constructor. In this case, it is necessaryto create each of the argument objects and initializethem before passing them to the constructor. In gen-eral, the code fragment between allocation and initial-ization may involve substantial computation, includingallocation of new objects, duplication of object point-ers, and jumps to or branches from other locations inthe code.

Since pointers may be duplicated, some form of alias-ing analysis must be used. More specifically, when aconstructor is called, there may be several pointers tothe object that is initialized as a result, as well as point-ers to other uninitialized objects. In order to verify codethat uses pointers to initialized objects, it is thereforenecessary to keep track of which pointers are aliases(name the same object). Some hint for this is given bythe following bytecode sequence:

3

Page 4: A T yp e System for Ob ject ... - Williams College

1: new #1 <Class Point>2: new #1 <Class Point>3: dup4: iconst_35: invokespecial #4 <Method Point(int)>6: invokevirtual #5 <Method void Print()>

When line 5 is reached during execution, there will betwo different uninitialized Point objects. If the byte-code verifier is to check object initialization statically,it must be able to determine which references point tothe object that is initialized at line 5 and which pointto the remaining uninitialized object. Otherwise, theverifier would either prevent use of an initialized objector allow use of an uninitialized one. (The bytecode pro-gram above is valid and accepted by verifiers using thestatic analysis described below.)

Sun’s Java Virtual Machine Specification [LY96] de-scribes the alias analysis used by the Sun JDK verifier.For each line of the bytecode program, some status in-formation is recorded for every local variable and stacklocation. When a location points to an object that isknown not to be initialized in all executions reachingthis statement, the status will include not only the prop-erty uninitialized, but also the line number on which theuninitialized object would have been created. As refer-ences are duplicated on the stack and stored and loadedin the local variables, the analysis also duplicates theseline numbers, and all references having the same linenumber are assumed to refer to the same object.

When an object is initialized, all pointers that referto objects created at the same line number are set toinitialized. In other words, all references to uninitializedobjects of a certain type are partitioned into equivalenceclasses according to what is statically known about eachreference, and all references that point to uninitializedobjects created on the same line are assumed to bealiases. This is a very simple and highly conservativeform of aliasing analysis; far more sophisticated meth-ods might be considered. However, the approach can beimplemented efficiently and it is sufficiently accurate toaccept bytecode produced by standard compilers.

Our specification of statically-correct Java bytecodein Section 4 uses the same form of aliasing analysis asthe Sun JDK verifier. Since our approach is type based,the status information associated with each reference isrecorded as part of its type.

One limitation of aliasing analysis based on linenumbers is that no verifiable program can ever be ableto reference two objects allocated on the same line,without first initializing at least one of them. If this sit-uation were to occur, references would exist to two dif-ferent objects from the same static aliasing-equivalenceclass. Unfortunately, there was an oversight in this re-gard in the development of the Sun verifier, which al-

lowed such a case to exist (as of version 1.1.4). Asdiscussed in Section 7, aliasing based on line num-bers makes it problematic for a subroutine to returnan uninitialized object.

3 JVMLi

This section describes the JVMLi language, a subset ofJVML encompassing basic constructs and object initial-ization. Although this language is much smaller thanJVML, it is sufficient to study object initialization andformulate a sound type system encompassing the staticanalysis described above. The run-time environmentfor JVMLi consists only of an operand stack and a fi-nite set of local variables. A JVMLi program will be asequence of instructions drawn from the following list:

instruction ::= push 0 | inc | pop| if L| store x | load x| new σ | init σ | use σ| halt

where x is a local variable name, σ is an object type, andL is an address of another instruction in the program.Informally, these instructions have the following effects:

push 0: pushes integer 0 onto the stack.

inc: adds one to the value on the top of the stack, ifthat value is an integer.

pop: removes the top element from the stack, providedthat the stack is not empty.

if L: if the top element on the stack is not 0, execu-tion jumps to instruction L. Otherwise, executionsteps to the next sequential instruction. This as-sumes that the top element is an integer.

store x: removes a value from the top of the stack andstores it into local variable x.

load x: loads the value from local variable x and placesit on the top of the stack.

halt: terminates program execution.

new σ: allocates a new, uninitialized object of type σand places it on the stack.

init σ: initializes the object on top of the operandstack, which must be a previously uninitialized ob-ject obtained by a call to new σ. This representscalling the constructor of an object. In this model,we assume that constructors always properly ini-tialize their argument and return. However, asdescribed in Section 6, there are several additional

4

Page 5: A T yp e System for Ob ject ... - Williams College

properties which must be checked to verify thatconstructors do in fact behave correctly.

use σ: performs an operation on an initialized objectof type σ. This is an abstraction of several op-erations in JVML, including method invocation(invokevirtual) and accessing an instance field(putfield/getfield).

In each case, all explicit or implicit assumptionsmust be satisfied in order for the statement to be ex-ecuted. For example, a pop instruction cannot be ex-ecuted if the stack is empty. The exact conditions re-quired to execute each instruction are specified in thedefinition of the operational semantics, in Section 4.3.Although dup does not appear in JVMLi for simplicity,aliasing may arise by storing and loading object refer-ences from the local variables.

4 Operational and Static Semantics

4.1 Notation

This section briefly reviews the framework developedby Stata and Abadi in [SA98] for studying JVML. Webegin with a set of instruction addresses Addr. Al-though we shall use integers to represent elements ofthis set, we will distinguish elements of Addr from in-tegers. A program is formally represented as a partialfunction from addresses to instructions. Dom(P ) is theset of addresses used in program P , and P [i] is the ith

instruction in program P . Dom(P ) will always includeaddress 1 and is usually a range {1, . . . , n} for some n.

Equality on partial maps is defined as

f = g iff Dom(f) = Dom(g)∧∀y ∈ Dom(f). f [y] = g[y]

Update and substitution operations are also defined.∀y ∈ Dom(f):

(f [x '→ v])[y] ={

v if x = yf [y] otherwise

([b/a]f)[y] = [b/a](f [y]) ={

b if f [y] = af [y] otherwise

where a, b, and v range over the codomain of f . Thisnotation for partial maps will be used throughout thispaper.

Sequences will also be used. The empty sequence isε, and v · s represents placing v on the front of sequences. A sequence of one element, v · ε, will sometimes beabbreviated to v. When convenient, we shall also treatsequences as partial maps from positions to elements ofthe sequence. For a sequence s, Dom(s) is the set ofindices into s, and s[i] is the ith element in s from theright. Substitution is also defined on sequences, in thesame manner as substitution on partial maps.

4.2 Values and Types

The types will be integers and object types. For objects,we assume there is some set T of possible object types.These types include all class names to which a programmay refer. In addition, there is a set T of types foruninitialized objects. The contents of this set is definedin terms of T :

σi ∈ T iff σ ∈ T ∧ i ∈ Addr

The type σi is used for an object of type σ allocated online i of a program, until it has been initialized. Giventhese definitions, JVMLi types are generated by thegrammar:

τ ::= Int | σ | σi | Top

where σ ∈ T and σi ∈ T . The type Int will be used forintegers. We discuss the addition of other basic types inSection 6. The type Top is the supertype of all types,with any value of any type also having type Top. Thistype will represent unusable values in our static analy-sis. In general, a type meta-variable τ may refer to anytype, including any object type σ ∈ T or uninitialized-object type σi ∈ T . In the case that a type meta-variable is known to refer to some uninitialized objecttype, we will write it as τ , for example.

Each object type and uninitialized object type has acorresponding infinite set of values which can be distin-guished from values of any other type. For any objecttype σ, this set of values is Aσ. Likewise, there is a setof values Aσi for all uninitialized object types σi. Inour model, we only need to know one piece of informa-tion for each object, namely, whether or not it has beeninitialized. Therefore, drawing uninitialized and initial-ized object “references” from different sets is sufficientfor our purposes, and we do not need to model an ob-ject store. As we will see below, this representation hassome impact on how object initialization is modeled inour operational semantics. Values of the form a or bwill refer to values known to be of some uninitializedobject type.

The basic type rules for values are:

v is a valuev : Top

n is an integern : Int

a ∈ Aτ , τ ∈ T ∪ Ta : τ

We also extend values and types to sequences:

ε : εa : τ s : αa · s : τ · α

4.3 Operational Semantics

The bytecode interpreter for JVMLi is modeled usingthe standard framework of operational semantics. Each

5

Page 6: A T yp e System for Ob ject ... - Williams College

P [pc] = incP * 〈pc, f, n · s〉 → 〈pc + 1, f, (n + 1) · s〉

P [pc] = popP * 〈pc, f, v · s〉 → 〈pc + 1, f, s〉

P [pc] = push 0P * 〈pc, f, s〉 → 〈pc + 1, f, 0 · s〉

P [pc] = load xP * 〈pc, f, s〉 → 〈pc + 1, f, f [x] · s〉

P [pc] = store xP * 〈pc, f, v · s〉 → 〈pc + 1, f [x '→ v], s〉

P [pc] = if LP * 〈pc, f, 0 · s〉 → 〈pc + 1, f, s〉

P [pc] = if Ln += 0

P * 〈pc, f, n · s〉 → 〈L, f, s〉

P [pc] = new σa ∈ Aσpc ,Unused(a, f, s)

P * 〈pc, f, s〉 → 〈pc + 1, f, a · s〉

P [pc] = init σa ∈ Aσj

a ∈ Aσ,Unused(a, f, s)P * 〈pc, f, a · s〉 → 〈pc + 1, [a/a]f, [a/a]s〉

P [pc] = use σa ∈ Aσ

P * 〈pc, f, a · s〉 → 〈pc + 1, f, s〉

Figure 2: JVMLi operational semantics.

instruction is characterized by a transformation of ma-chine states, where a machine state is a tuple 〈pc, f, s〉with the following meaning:

• pc is a program counter, indicating the address ofthe instruction that is about to be executed.

• f is a total map from Var, the set of local vari-ables, to the values stored in the local variables inthe current state.

• s is a stack of values representing the operandstack for the current state in execution.

The machine begins execution in state 〈1, f0, ε〉. In thisstate, the first instruction in the program is about tobe executed, the operand stack is empty, and the localvariables may contain any values. This means that f0

may map the local variables to any values.Each bytecode instruction has one or more rules in

the operational semantics. These rules use the judg-ment

P * 〈pc, f, s〉 → 〈pc′, f ′, s′〉

to indicate that a program P in state 〈pc, f, s〉 canmove to state 〈pc′, f ′, s′〉 in one step. The completeone-step operational semantics for JVMLi is shown inFigure 2. In that figure, n is any integer, v is any value,L and j are any addresses, and x is any local variable.

These operational semantic rules, with the exception ofthose added to study object initialization, are discussedin detail in [SA98]. The rules have been designed sothat a step cannot be made from an illegal state. Forexample, it is not possible to execute a pop instructionwhen there is an empty stack.

The rules for allocating and initializing objects needto generate object values not in use by the program.The only values in use are those which appear on theoperand stack or in the local variables. Therefore, thenotion of being unused may be characterized by

(unused)

a +∈ s∀y ∈ Var. f [y] += a

Unused(a, f, s)

When a new object is created, a currently unusedvalue of an uninitialized object type is placed on thestack. The type of that value is determined by the ob-ject type named in the instruction and the line numberof the instruction. When the value for an uninitial-ized object is initialized by an init σ instruction, alloccurrences of that value are replaced by a new valuecorresponding to an initialized object. In some sense,initialization may be thought of as a substitution of anew, initialized object for an uninitialized object. Thenew value is required to be unused. This allows theprogram to distinguish between different objects of the

6

Page 7: A T yp e System for Ob ject ... - Williams College

(inc)

P [i] = incF i+1 = F i

Si+1 = Si = Int · αi + 1 ∈ Dom(P )

F , S, i * P(if)

P [i] = if LF i+1 = FL = F i

Si = Int · Si+1 = Int · SL

i + 1 ∈ Dom(P )L ∈ Dom(P )F , S, i * P

(pop)

P [i] = popF i+1 = F i

Si = τ · Si+1

i + 1 ∈ Dom(P )F , S, i * P

(push 0)

P [i] = push 0F i+1 = F i

Si+1 = Int · Si

i + 1 ∈ Dom(P )F , S, i * P

(load)

P [i] = load xx ∈ Dom(Fi)F i+1 = F i

Si+1 = F i[x] · Si

i + 1 ∈ Dom(P )F , S, i * P

(store)

P [i] = store xx ∈ Dom(Fi)

F i+1 = F i[x '→ τ ]Si = τ · Si+1

i + 1 ∈ Dom(P )F , S, i * P

(halt)P [i] = haltF , S, i * P

(new)

P [i] = new σF i+1 = F i

Si+1 = σi · Si

σi +∈ Si

∀y ∈ Dom(F i). F i[y] += σi

i + 1 ∈ Dom(P )F , S, i * P

(init)

P [i] = init σF i+1 = [σ/σj ]F i

Si = σj · αSi+1 = [σ/σj ]αj ∈ Dom(P )

i + 1 ∈ Dom(P )F , S, i * P

(use)

P [i] = use σF i+1 = F i

Si = σ · Si+1

i + 1 ∈ Dom(P )F , S, i * P

Figure 3: Static semantics.

7

Page 8: A T yp e System for Ob ject ... - Williams College

same type after they have been initialized, but this factis not necessarily needed to study the properties ad-dressed by this paper.

4.4 Static Semantics

A program P is well typed if there exist F and S suchthat

F , S * P ,

where F is a map from Addr to functions mappinglocal variables to types, and S is a map from Addr tostack types such that Si is the type of the operand stackat location i of the program. As described in [SA98],elements in a map over Addr are accessed as F i insteadof F [i]. Thus, F i[y] is the type of local variable y at linei of a program.

The Java Virtual Machine Specification [LY96] de-scribes the verifier as both computing the type informa-tion stored in F and S and checking it. However, weassume that the information stored in F and S has al-ready been computed prior to the type checking stage.This simplifies matters since it separates the two tasksand prevents the type synthesis from complicating thestatic semantics. In other words, we only need to trustthe implementation of the type checker, and not the im-plementation of the type inferencing part of the anal-ysis. If the two stages are combined, as they are incurrent implementations, a bad program could be ac-cepted due to an error in the process of computing typeinformation. However, separating the two tasks pre-vents the type checker from accepting a bad programdue to such an error.

The judgment which allows us to conclude that aprogram P is well typed by F and S is

(wt prog)

F 1 = FTop

S1 = ε∀i ∈ Dom(P ). F , S, i * P

F , S * P

where FTop is a function mapping all variables in Varto Top. The first two lines of (wt prog) constrain theinitial conditions for the program’s execution to matchthe type of the values given to the initial state in theoperational semantics. The third line requires that eachinstruction in the program is well typed according to thelocal judgments presented in Figure 3.

The (new) rule in Figure 3 requires that the typeof the object allocated by the new instruction is left ontop of the stack. Note that this rule is only applicableif the uninitialized object type about to be placed ontop of the type stack does not appear anywhere in F i

or Si. This restriction is crucial to ensure that we donot create a situation in which a running program may

have two different values mapping to the same staticallycomputed uninitialized object type.

The rule for (use) requires an initialized object typeon top of the stack. The (init) rule implements thestatic analysis method described in Section 2. The rulespecifies that all occurrences of the type on the top ofthe stack are replaced by an initialized type. This willchange the types of all references to the object that isbeing initialized since all those references will be in thesame static equivalence class, and, therefore, have thesame type.

Figure 4 shows a JVMLi program and the type infor-mation demonstrating that it is a well-typed programaccording to the rules in this section.

5 Soundness

This section outlines the soundness proof for JVMLi.The main soundness theorem states that no well-typedprogram will cause a run-time type error. Before stat-ing the main soundness theorem, a one-step soundnesstheorem is presented. One-step soundness implies thatany valid transition from a well-formed state leads toanother well-formed state.

Theorem 1 (One-step Soundness) Given P , F ,and S such that F , S * P :

∀pc, f, s, pc′, f ′, s′.P * 〈pc, f, s〉 → 〈pc′, f ′, s′〉∧ s : Spc

∧ ∀y ∈ Var. f [y] : Fpc [y]∧ ConsistentInit(F pc , Spc , f, s)

⇒ s′ : Spc′

∧ ∀y ∈ Var. f ′[y] : F pc′ [y]∧ ConsistentInit(F pc′ , Spc′ , f ′, s′)∧ pc′ ∈ Dom(P )

This theorem lists the four factors which dictatewhether or not a state is well formed. The values onthe operand stack must have the types expected by thestatic type rules, and the local variable contents mustmatch the types in F . In addition, the program countermust always be in the domain of the program. Thiscan be assumed on the left-hand side of the implicationsince the operational semantics guarantee that transi-tions can only be made if P [pc] is defined. If the pro-gram counter were not in the domain of the program,no step could be made.

The final requirement for a state to be well formed isthat it has the ConsistentInit property. Informally, thisproperty means that the machine cannot access two dif-ferent uninitialized objects created on the same line ofcode. As mentioned in Section 2, this invariant is criti-cal for the soundness of this static analysis. The Consis-

8

Page 9: A T yp e System for Ob ject ... - Williams College

i P [i] Fi[0] Si

1: new C Top ε2: new C Top C1 · ε3: store 0 Top C2 · C1 · ε4: load 0 C2 C1 · ε5: load 0 C2 C2 · C1 · ε6: init C C2 C2 · C2 · C1 · ε7: use C C C · C1 · ε8: halt C C1 · ε

Figure 4: A JVMLi program and its static type information.

(cons init)∀τ ∈ T . ∃b : τ . Corresponds(F i, Si, f, s, b, τ)

ConsistentInit(F i, Si, f, s)

(corr)

∀x ∈ Dom(F i). F i[x] = τ =⇒ f [x] = bStackCorresponds(Si, s, b, τ)Corresponds(F i, Si, f, s, b, τ)

(sc 0)StackCorresponds(ε, ε, b, τ)

(sc 1)StackCorresponds(Si, s, b, τ)

StackCorresponds(τ · Si, b · s, b, τ)

(sc 2)

τ += τStackCorresponds(Si, s, b, τ)

StackCorresponds(τ · Si, v · s, b, τ)

Figure 5: The ConsistentInit judgement.

9

Page 10: A T yp e System for Ob ject ... - Williams College

tentInit property requires a unique correspondence be-tween uninitialized object types and run-time values.

Figure 5 presents the formal definition of Consistent-Init. In that figure, F i is a map from local variables totypes, Si is a stack type. The judgment (cons init) issatisfied only if every uninitialized object type τ hassome value b that Corresponds to it. The first line ofrule (corr) guarantees that every occurrence of τ in thestatic types of the local variables is matched by b in therun-time state. The second line of that rule uses an aux-iliary judgment to assert the same property about thestack type and operand stack inductively. Given thisinvariant, we are able to assume that when an init in-struction is executed, all stack slots and local variablesaffected by the type substitution in rule (init) appliedto that instruction contain the object that is being ini-tialized.

The proof of Theorem 1 is by case analysis on allpossible instructions at P [pc]. The proof of this theoremand those that follow appear in the extended version ofthis paper.

A complementary theorem is that a step can alwaysbe made from a well-formed state, unless the programhas reached a halt instruction. This progress theoremcan be stated as:

Theorem 2 (Progress) Given P , F , and S such thatF , S * P :

∀pc, f, s.s : Spc

∧ ∀y ∈ Var. f [y] : Fpc [y]∧ ConsistentInit(F pc , Spc , f, s)∧ pc ∈ Dom(P )∧ P [pc] += halt

⇒ ∃pc′, f ′, s′. P * 〈pc, f, s〉 → 〈pc′, f ′, s′〉

Theorem 1 and Theorem 2 can be used to prove in-ductively that a program beginning in a valid initialstate will always be in a well-formed state, regardless ofhow many steps are made. In addition, a program willnever get stuck unless it reaches a halt in instruction.When it does reach a halt instruction, the stack willhave the correct type, which is important since the re-turn value for a program, or method in the full JVML,is returned as the top value on the stack. The followingtheorem captures this soundness property:

Theorem 3 (Soundness) Given P , F , and S suchthat F , S * P :

∀pc, f0, f, s.P * 〈1, f0, ε〉 →∗ 〈pc, f, s〉∧ ¬∃pc′, f ′, s′. P * 〈pc, f, s〉 → 〈pc′, f ′, s′〉

⇒ P [pc] = halt∧ s : Spc

If a program executing in our machine model attemptsto perform an operation leading to a type error, such asusing an uninitialized object, it would get stuck sincethose operations are not defined by our operational se-mantics. By proving that well-typed programs only getstuck when a halt instruction is reached, we know thatwell-typed programs will not attempt to perform anyillegal operations. Thus, this theorem implies that ourstatic analysis is correct by showing that no erroneousprograms are accepted. Therefore, no accepted programuses an uninitialized object.

One technical point of interest is the asymmetry ofthe checks in rule (corr). That rule requires that alllocations sharing type τ contain the same value b, butit does not require that all occurrences of b map to thetype τ in the static type information. We do not need tocheck the other direction because the rule is used only inthe hypothesis of rule (cons init), where the conditionon the existential quantification of b requires that b : τ .Therefore, b ∈ Aτ , and the only types which we mayassign to b are τ and Top.

This allows us to assume that, as long as the stackand local variables are well typed when rule (cons init)is used, any occurrences of b are matched by either τ orTop. Thus, with the exception of occurrences of Top,the correspondence between b and τ holds in both di-rections. The situation for Top introduces a specialcase in the proofs but does not affect soundness, andthe asymmetric checks are sufficient to prove the sound-ness of the system. If we were to change our model sothat object values could potentially have more than oneuninitialized object type, i.e. all uninitialized object ref-erences are drawn from a single set, then we would needto check both directions for the correspondence and ex-plicitly deal with the special case for Top in the (corr)judgment.

6 Extensions

Several extensions to the JVMLi framework describedin the previous sections have been studied. First, thereare additional static checks which must be performed onconstructors in order to guarantee that they do prop-erly initialize objects. Section 6.1 presents JVMLc, anextension of JVMLi modeling constructors. Anotherextension, JVMLs, combining object initialization andsubroutines, is described in Section 6.2. Section 6.3shows how any of these languages may be easily ex-tended with other basic operations and primitive types.The combination of these features yields a sound typesystem covering the most complex pieces of the JVMLlanguage.

10

Page 11: A T yp e System for Ob ject ... - Williams College

6.1 JVMLc

The typing rules in Section 4 are adequate to checkcode which creates, initializes, and uses objects, assum-ing that calls to init σ do in fact properly initializeobjects. However, since initialization is performed byuser-defined constructors, the verifier must check thatthese constructors do correctly initialize objects whencalled. This section studies verification of JVML con-structors using JVMLc, an extension of JVMLi.

The rules for checking constructors are definedin [LY96] and can be summarized by three basic points:

• When a constructor is invoked, local variable 0contains a reference to the object that is beinginitialized.

• A constructor must apply either a different con-structor of the same class or a constructor fromthe parent class to the object that is being initial-ized before the constructor exits. For simplicity,we may refer to either of these actions as invokingthe super class constructor.

• The only deviation from this requirement is forconstructors of class Object. Since, by the Javalanguage definition, Object is the only class with-out a superclass, constructors for Object need notcall any other constructor. This one special casehas not been modeled by our rules, but would betrivial to add.

Note that these rules do not imply that constructors willalways return. For example, they do not prevent non-termination due to an infinite loop within a constructor.A more interesting case is when two constructors fromthe same class call each other recursively and, therefore,never fully construct an object passed to them. Whileprograms potentially exhibiting this behavior could bedetected by intra-procedural analysis, this type of anal-ysis falls outside of the bounds of the current verifier.

JVMLc programs are sequences of instructions con-taining any instructions from JVMLi plus one new in-struction, super σ. This instruction represents callinga constructor of the parent class of class σ (or a differentconstructor of the class σ).

For simplicity, the rest of this section assumes thatwe are describing a constructor for object type ϕ, forsome ϕ in T . To model the initial state of a constructorinvocation for class ϕ, a JVMLc program is assumedto begin in a state in which local variable 0 containsan uninitialized reference. This corresponds to the ar-gument of the constructor. Prior to halting, the pro-gram must call super ϕ on that object reference. Asmentioned above, this represents calling the super classconstructor.

We will use ϕ0 as the type of the object stored inlocal variable 0 at the start of execution. The value inlocal variable 0 must be drawn from the set Aϕ0 . Wenow assume Addr includes 0, although 0 will not be inthe domain of any program. Also, machine state in theoperational semantics is augmented with a fourth ele-ment, z, which indicates whether or not a super classconstructor has been called on the object that is be-ing initialized. The rules for all instructions other thansuper do not affect z, and are derived directly from therules in Figure 2. For example, the rule for inc is:

P [pc] = incP *c 〈pc, f, n · s, z〉 → 〈pc + 1, f, (n + 1) · s, z〉

As demonstrated in Theorem 4 below, the initial statefor execution of a constructor for ϕ is 〈1, f0[0 '→aϕ], ε, false〉 where aϕ ∈ Aϕ0 .

For super, the operational semantics rule is:

P [pc] = super σa ∈ Aσ0

a ∈ Aσ,Unused(a, f, s)P *c 〈pc, f, a · s, z〉 → 〈pc + 1, [a/a]f, [a/a]s, true〉

The typing rule for super is very similar to the rulefor init, and is shown below with the judgment fordetermining whether a program is a valid constructorfor objects of type ϕ. All the other typing rules are thesame as those appearing in Figure 3.

(super)

P [i] = super σF i+1 = [σ/σ0]F i

Si = σ0 · αSi+1 = [σ/σ0]αi + 1 ∈ Dom(P )

F , S, i * P

(wt cst)

F 1 = FTop[0 '→ ϕ0]S1 = ε

Z1 = falseϕ ∈ T

∀i ∈ Dom(P ). F , S, i * P∀i ∈ Dom(P ). Z, i * P constructs ϕ

F , S * P constructs ϕ

The (wt cst) rule is analogous to (wt prog) from Sec-tion 4. However, this rule places an additional restric-tion on the structure of well-typed programs. The judg-ment

Z, i * P constructs ϕ

is a local judgment which gives Zi the value true orfalse depending on whether or not all possible execu-tion sequences reaching instruction i would have calledsuper ϕ or not. The local judgments are defined in Fig-ure 6. As seen by those rules, one can only conclude that

11

Page 12: A T yp e System for Ob ject ... - Williams College

P [i] ∈ {inc, pop, push 0, load x, store x, new σ, init σ, use σ}Zi+1 = Zi

Z, i * P constructs ϕ

P [i] = if LZi+1 = ZL = Zi

Z, i * P constructs ϕ

P [i] = super ϕZi+1 = true

Z, i * P constructs ϕ

P [i] = haltZi = true

Z, i * P constructs ϕ

Figure 6: Rules checking that a super class constructor will always be called prior to reaching a halt instruction.

a program is a valid constructor for ϕ if every path toeach halt instruction has called super ϕ. These judg-ments also reject any programs which call super for aclass other than ϕ. The existence of unreachable codemay cause more than one value of Z to conform to therules in Figure 6. To make Z unique for any given pro-gram, we assume that, for program P , there is a uniquecanonical form ZP . Thus, ZP,i will be a unique valuefor instruction i.

The main soundness theorem for constructors in-cludes a guarantee that constructors do call super onthe uninitialized object:

Theorem 4 (Constructor Soundness) Given P ,F , S, ϕ, and aϕ such that F , S * P constructs ϕ andaϕ : ϕ0 :

∀pc, f0, f, s, z.P *c 〈1, f0[0 '→ aϕ], ε, false〉 →∗ 〈pc, f, s, z〉∧ ¬∃pc′, f ′, s′, z′.

P *c 〈pc, f, s, z〉 → 〈pc′, f ′, s′, z′〉⇒ P [pc] = halt∧ z = true

The main difference in the proof of Theorem 4, in com-parison with Theorem 3, is that the corresponding one-step soundness theorem requires an additional invari-ant. The invariant states that when program P is instate 〈pc, f, s, z〉, z = ZP,pc . The proof of this theo-rem appears in the extended version of this paper.

This analysis for constructors is combined with theanalysis of normal methods in a more complete JVMLmodel currently being developed.

6.2 JVMLs

The JVML bytecodes for subroutines have also beenadded to JVMLi and are presented in another extendedlanguage, JVMLs. While this section will not go intoall the details of subroutines, detailed discussions ofbytecode subroutines can be found in several otherworks [SA98, LY96]. Subroutines are used to compilethe finally clauses of exception handlers in the Javalanguage. Subroutines share the same activation recordas the method which uses them, and they can be calledfrom different locations in the same method, enablingall locations where finally code must be executed tojump to a single subroutine containing that code. Theflexibility of this mechanism makes bytecode verifica-tion difficult for two main reasons:

• Subroutines are polymorphic over local variableswhich they do not use.

• Subroutines may call other subroutines, as longas a call stack discipline is preserved. In otherwords, the most recently called subroutine must bethe first one to return. This is a slight simplifica-tion of the rules for subroutines defined in [LY96],which do allow a subroutine to return more thanone level up in the implicit subroutine call stackin certain cases, but does match the definitionspresented in [SA98].

JVMLs programs contain the same set of instructionsas JVMLi programs and, also, the following:

jsr L: jumps to instruction L, and pushes the returnaddress onto the stack. The return address is theinstruction immediately after the jsr instruction.

12

Page 13: A T yp e System for Ob ject ... - Williams College

P [pc] = jsr LP * 〈pc, f, s〉 → 〈L, f, (pc + 1) · s〉

P [pc] = ret xP * 〈pc, f, s〉 → 〈f [x], f, s〉

Figure 7: Operational semantics for jsr and ret.

(jsr)

P [i] = jsr LDom(F i+1) = Dom(F i)Dom(FL) ⊆ Dom(F i)

∀y ∈ Dom(F i). F i[y] +∈ T∀y ∈ Dom(Si). Si[y] +∈ T

∀y ∈ Dom(F i)\Dom(FL). F i+1[y] = F i[y]∀y ∈ Dom(FL). FL[y] = F i[y]

SL = (ret-from L) · Si

(ret-from L) +∈ Si

∀y ∈ Dom(FL). FL[y] += (ret-from L)i + 1 ∈ Dom(P )

L ∈ Dom(P )F , S, i * P

(ret)

P [i] = ret xRP,i = {L}

x ∈ Dom(F i)F i[x] = (ret-from L)∀y ∈ Dom(F i). F i[y] +∈ T∀y ∈ Dom(Si). Si[y] +∈ T

∀j. P [j] = jsr L ⇒(∀y ∈ Dom(F i). F j+1[y] = F i[y]∧ Sj+1 = Si

)

F , S, i * P

Figure 8: Type rules for jsr and ret.

13

Page 14: A T yp e System for Ob ject ... - Williams College

ret x: jumps to the instruction address stored in localvariable x.

The operational semantics and typing rules for theseinstructions are shown in Figure 7 and Figure 8.These rules are based on the rules used by Stata andAbadi [SA98]. The type (ret-from L) is introduced toindicate the type of an address to which subroutine Lmay return. The meaning of RP,i = {L} in (ret) is de-fined in their paper and basically means that instructioni is an instruction belonging to the subroutine startingat address L. All other rules are the same as those forJVMLi.

The main issue concerning initialization which mustbe addressed in the typing rules for jsr and ret isthe preservation of the ConsistentInit invariant. A typeloophole could be created by allowing a subroutine andthe caller of that subroutine to exchange references touninitialized objects in certain situations. An exampleof this behavior is described in Section 7.

When subroutines are used to compile finallyblocks by a Java compiler, uninitialized object refer-ences will never be passed into or out of a subroutine.The Java language prevents a program from splitting al-location and initialization of an object between code in-side and outside of a finally clause since both are partof the same Java operation, as described in Section 2.Either both steps occur outside of the subroutine, orboth steps occur inside the subroutine. We restrict pro-grams not to have uninitialized objects accessible whencalling or returning from a subroutine. For (ret), thefollowing two lines are added. These prevent the sub-routine from allocating a new object without initializingit:

∀y ∈ Dom(F i). F i[y] +∈ T∀y ∈ Dom(Si). Si[y] +∈ T

Similar lines are added to (jsr). The discussion of theinteraction between subroutines and uninitialized ob-jects in the Java Virtual Machine specification is vagueand inconsistent with current implementations, but therules we have developed seem to fit the general strategydescribed in the specification.

This is certainly not the only way to prevent sub-routines and object initialization from causing prob-lems. For example, slightly less restrictive rules couldbe added to (jsr):

∀y ∈ Dom(FL). FL[y] +∈ T∀y ∈ Dom(Si). Si[y] +∈ T

These conditions still allow uninitialized objects to bepresent when a subroutine is called, but those objectscannot be touched since they are stored in local vari-ables which are not accessed in the body of the subrou-tine. This would allow the typing rules to accept more

programs, but these programs could not have been cre-ated by a compiler from any valid Java program.

The main soundness theorem, Theorem 3, has beenproved for JVMLs, and for JVMLc with subroutines,by combining the proof of JVMLi soundness with thework of Stata and Abadi. These proofs appear in theextended version of this paper.

6.3 Other Basic Types and Instructions

Many JVML instructions are variants of operations fordifferent basic types. For example, there are four addinstructions corresponding to addition on values of typeInt, Float, Long, and Double. Likewise, manyother simple operations have several different forms.These instructions and other basic types can be addedto JVMLi, or any of the extended languages, easily.These instructions do not complicate any of the sound-ness proofs since they only operate on basic types anddo not interfere with object initialization or subroutineanalysis.

The only tricky case is that Long and Double val-ues take up two local variables or two stack slots sincethey are stored as two-word values. Although this re-quires an additional check in the rules for load andstore to prevent the program from accessing a par-tially over-written two-word value, this does not poseany serious difficulty.

Of the 200 bytecode instructions in JVML, all butapproximately 40 fall into this category and may beadded to JVMLi without trouble, although a full pre-sentation of the operational and type rules for these in-structions is beyond the scope of this paper. With theseadditions, and the methods described in the previoussubsections, the JVMLi framework can be extended tocover the whole bytecode language, except for a full ob-ject system, exceptions, arrays, and concurrency. Con-sidering objects and classes requires the addition of anobject heap and a method call stack, as well as a typingenvironment containing class declarations. We are cur-rently developing an extended system covering all thesetopics except concurrency.

7 The Sun Verifier

This section describes the relationship between the ruleswe have developed for object initialization and subrou-tines and the rules implicitly used to verify programsin Sun’s implementation. We first describe a mistakewe have found in Sun’s rules and then compare theircorrected rules with our rules for JVMLs.

14

Page 15: A T yp e System for Ob ject ... - Williams College

1: jsr 10 // jump to subroutine2: store 1 // store uninitialized object3: jsr 10 // jump to subroutine4: store 2 // store another uninitialized object5: load 2 // load one of them6: init P // initialize it7: load 1 // load the other8: use P // use uninitialized object!!!9: halt

10: store 0 // store return address11: new P // allocate new object12: ret 0 // return from subroutine

Figure 9: A program that uses an uninitialized object but is accepted by Sun’s verifier.

7.1 The Sun JDK 1.1.4 Verifier

As a direct result of the insight gained by carrying outthe soundness proof for JVMLs, a previously unpub-lished bug was discovered in Sun’s JDK 1.1.4 imple-mentation of the bytecode verifier. A simple programexhibiting the incorrect behavior is shown in Figure 9.Line 8 of the program uses an uninitialized object, butthis code is accepted by this specific implementation ofthe verifier. Basically, the program is able to allocatetwo different uninitialized objects on the same line ofcode without initializing either one, violating the Con-sistentInit invariant. The program accomplishes this byallocating space for the first new object inside the sub-routine and then storing the reference to that object ina local variable over which the subroutine is polymor-phic before calling it again. After initializing only oneof the objects, it can use either one.

The bug can be attributed to the verifier not placingany restrictions on the presence of uninitialized objectsat calls to jsr L or ret x. The checks made by Sun’sverifier are analogous to the (jsr) and (ret) rule in Fig-ure 8 as they originally appeared in [SA98], without theadditions described in the previous section. Removingthese lines allows subroutines to return uninitialized ob-jects to the caller and to store uninitialized values acrosssubroutine calls, which clearly leads to problems.

Although this bug does not immediately create anysecurity loopholes in the Sun Java Virtual Machine, itdoes demonstrate the need for a more formal specifi-cation of the verifier. It also shows that even a fairlyabstract model of the bytecode language is extremelyuseful at examining the complex relationships betweendifferent parts of the language, such as initialization andsubroutines.

7.2 The Corrected Sun Verifier

After describing this bug to the Sun development team,they have taken steps to repair their verifier implemen-tation. While they did not use the exact rules we havepresented in this paper, they have changed their im-plementation to close the potential type loophole. Thissection briefly describes the difference between their ap-proach and ours. The Sun implementation may be sum-marized as follows [Lia97]:

• Uninitialized objects may appear anywhere in thelocal variables or on the operand stack at jsr L orret x instructions, but they can not be used afterthe instruction has executed. In other words, theirstatic type is made Top in the post-instructionstate. This difference does not affect the abilityof either Sun’s rules or our rules to accept codecreated for valid Java language programs.

• The static types assigned to uninitialized objectspassed into constructors are treated differentlyfrom other uninitialized object types in the Sunverifier. Values with these types may still be usedafter being present at a call to or an exit from asubroutine. Also, the superclass constructor maybe called anywhere, including inside a subroutine.

Treating the uninitialized object types for constructorarguments differently than other uninitialized types al-lows the verifier to accept programs where a subroutinemust be called prior to invoking the super class con-structor. This is demonstrated by Figure 10. That fig-ure shows a constructor for class C, as well as a roughtranslation of it into JVMLc with subroutines (we ig-nore the code required for the exception handler).

The bytecode translation of the constructor will berejected by our analysis because control jumps to a sub-routine when a local variable contains the uninitializedobject passed into the constructor. It is accepted by the

15

Page 16: A T yp e System for Ob ject ... - Williams College

class C extends Object {C() {int i;try {i = 0;

} finally {}super();

}. . .

}

1: push 0 // put 0 on stack2: store 1 // store it in ‘‘i’’3: jsr 7 // jump to subroutine4: load 0 // load constructor arg.5: super C // call superclass cnstr.6: halt

7: store 2 // store return address8: ret 2 // return from subroutine

Figure 10: A constructor which will always call a subroutine before invoking the super class constructor, and itstranslation into JVMLc with subroutines (ignoring the exception handler). Note that this is not a valid Java program.

Sun verifier due to their special treatment of the typeof the uninitialized object passed into the constructor.However, the Java language specification requires thatthe superclass constructor be called prior to the startof any code protected by an exception handler. There-fore, the Java program in Figure 10 is not valid. Theadded flexibility of their method is not required to ver-ify valid Java programs, but it does make the analysismuch more difficult. In fact, several published attacks,including the one described in Section 1, may be at-tributed to errors in this part of the verifier. Otherverifiers, such as the Microsoft verifier, currently rejectthe bytecode translation of this class.

In summary, the differences in the two verificationtechniques would only become apparent in handwrittenbytecodes using uninitialized object types in unusualways, and both systems are sufficient to type checktranslations of valid Java programs. Since our method,while slightly more restrictive, makes both verificationand our soundness proofs much simpler, we believe thatour method is reasonable.

8 Related Work

There are several other projects currently examiningbytecode verification and the creation of correct byte-code verifiers. This section describes some of theseprojects, as well as related work in contexts other thanJava. There have also been many studies of the Javalanguage type system [Sym97, DE97, NvO98], but wewill mostly focus on bytecode level projects. Althoughthe other studies are certainly useful, and closely re-lated to this work in some respects, they do not addressthe unique way in which the bytecode language is usedand the special structures in JVML.

In addition to the framework developed by Stata andAbadi [SA98] and used in this paper, there are otherstrategies being developed to describe the JVML type

system and bytecode verification formally. The mostclosely related work is [Qia98], which presents a statictype system for a larger fragment of JVML than is pre-sented here. While that system uses the same generalapproach as we do, we have attempted to present asimpler type system by abstracting away some of theunnecessary details left in Qian’s framework, such asdifferent forms of name resolution in the constant pooland varying instruction lengths. Also, our model of sub-routines, based on the work of Stata and Abadi, is verydifferent. The rules for object initialization used in theoriginal version of Qian’s paper were similar to Sun’sfaulty rules, and they incorrectly accepted the programin Figure 9. After announcing our discovery of Sun’sbug, a revised version of Qian’s paper containing rulesmore similar to our rules was released.

Hagiya and Tozawa present a type system for thefragment of JVML concerning subroutines [HT98]. Weare currently examining ways in which ideas from thattype system may used to eliminate some of the simpli-fications to the subroutine mechanism in the work ofStata and Abadi.

Another approach using concurrent constraint pro-gramming is also being developed [Sar97]. This ap-proach is based on transforming a JVML program intoa concurrent constraint program. While this approachmust also deal with the difficulties in analyzing sub-routines and object initialization statically, it remainsto be seen whether it will yield a better framework forstudying JVML, and whether the results can be easilytranslated into a verifier specification.

Other avenues toward a formal description of theverifier, including model checking [PV98] and data flowanalysis techniques [Gol98], are also currently beingpursued.

A completely different approach has been taken byCohen, who is developing a formal execution modelfor JVML which does not require bytecode verifica-tion [Coh97]. Instead, safety checks are built into the

16

Page 17: A T yp e System for Ob ject ... - Williams College

interpreter. Although these run-time checks make theperformance of his defensive JVM too slow to use inpractice, this method is useful for studying JVML exe-cution and understanding the checks required to safelyexecute a program.

The Kimera project has developed a more experi-mental method to determine the correctness of exist-ing bytecode verifiers [SMB97]. After implementing averifier from scratch, programs with randomly insertederrors were fed into that verifier, as well as several com-mercially produced verifiers. Any differences among im-plementations meant a potential flaw. While this ap-proach is fairly good at tracking down certain classesof implementation mistakes and is effective from a soft-ware engineering perspective, it does not lead to thesame concise, formal model like some of the other ap-proaches, including the approach presented in this pa-per. It also may not find JVML specification errors ormore complex bugs, such as the one described in Sec-tion 7.

Other recent work has studied type systems for low-level languages other than JVML. These studies includethe TIL intermediate languages for ML [TMC+96],and the more recent work on typed assembly lan-guage [MCGW98]. The studies touch on some of thesame issues as this study, and the type system for typedassembly language does contain a distinction betweentypes for initialized and uninitialized values. However,these languages do not contain some of the constructsfound in JVML, and they do not require aspects ofthe static analysis required for JVML, such as the aliasanalysis required for object initialization.

9 Conclusions and Future Work

Given the need to guarantee type safety for mobileJava code, developing correct type checking and anal-ysis techniques for JVML is crucial. However, there isno existing specification which fully captures how Javabytecodes must be type checked. We have built on theprevious work of Stata and Abadi to develop such aspecification by formulating a sound type system for afairly complex subset of JVML which covers both sub-routines and object initialization. This is one step to-wards developing a sound type system for the wholebytecode language. Once this type system for JVML iscomplete, we can describe a formal specification of theverifier and better understand what safety and securityguarantees can be made by it.

Although our model is still rather abstract, it hasalready proved effective as a foundation for examiningboth JVML and existing bytecode verifiers. Even with-out a complete object model or notion of an object heap,we have been able to study initialization and the inter-

action between it and subroutines. In fact, a previouslyunpublished bug in Sun’s verifier implementation wasfound as a result of the analysis performed while study-ing the soundness proofs for this paper.

The work described in this paper opens severalpromising directions. One major task, which we arecurrently undertaking, is to extend the specificationand correctness proof to the entire JVML, includ-ing the method call stack and a full object system.The methods described in Section 6 allow most vari-ants of simple instructions to be added in a stan-dard, straightforward way, and we are also examiningmethods to factor JVML into a complete, yet mini-mal, set of instructions. In addition, the Java objectsystem has been studied and discussed in other con-texts [AG96, Sym97, DE97, Qia98], and these previousresults can be used as a basis for objects in our JVMLmodel. We are in the process of finishing a soundnessproof for a language encompassing the JVML elementspresented in this paper plus objects, interfaces, classes,arrays, and exceptions. Other issues that have not beenaddressed to date are concurrency and dynamic load-ing, both of which are key concepts in the Java VirtualMachine.

We also believe it will be feasible to generate an im-plementation of a bytecode verifier from a specificationproven to be correct. This specification could be ex-pressed in the kind of typing rules we use here, or somevariant of this notation.

Finally, we expect that in the long run, it will be use-ful to incorporate additional properties into the staticanalysis of Java programs. If Java is to become a popu-lar and satisfactory general-purpose programming lan-guage, then for efficiency reasons alone, it will be nec-essary to replace some of the current run-time tests byconservative static analysis, perhaps reverting to run-time tests when static analysis fails. For example, wemay be able to eliminate some run-time checks for ar-ray bounds and pointer casts. Other safety properties,such as the use of certain locking conventions in a con-current JVML model, could also be added to our staticanalysis.

Acknowledgments: Thanks to Martın Abadi andRaymie Stata (DEC SRC) for their assistance on thisproject. We thank Li Gong for his encouragement, andFrank Yellin and Sheng Liang of JavaSoft for severaluseful discussions.

References[AG96] Ken Arnold and James Gosling. The Java Program-

ming Language. Addison-Wesley, 1996.

[Coh97] Rich Cohen. Defensive Java Virtual MachineVersion 0.5 alpha Release. Available from

17

Page 18: A T yp e System for Ob ject ... - Williams College

http://www.cli.com/software/djvm/index.html,November 1997.

[DE97] S. Drossopoulou and S. Eisenbach. Java is type safe— probably. In European Conference On Object Ori-ented Programming, pages 389–418, 1997.

[DFW96] Drew Dean, Edward W. Felten, and Dan S. Wallach.Java security: from HotJava to netscape and beyond.In Proceedings of the IEEE Computer Society Sym-posium on Research in Security and Privacy, pages190–200, 1996.

[Gol98] Allen Goldberg. A specification of java load-ing and bytecode verification. In Proceedingsof the Fifth ACM Conference on Computer andCommunications Security, 1998. Available fromhttp://www.kestrel.edu/˜goldberg.

[HT98] Masami Hagiya and Akihiko Tozawa.On a new method for dataflow anal-ysis of Java Virtual Machine subrou-tines. Available from http://nicosia.is.s.u-tokyo.ac.jp/members/hagiya.html. A prelimi-nary version appeared in SIG-Notes, PRO-17-3,Information Processing Society of Japan, 1998.

[Lia97] Sheng Liang. personal communication, November1997.

[LY96] Tim Lindholm and Frank Yellin. The Java VirtualMachine Specification. Addison-Wesley, 1996.

[MCGW98] Greg Morrisett, Karl Crary, Neal Glew, and DavidWalker. From system F to typed assembly language.In Proc. 25th ACM Symposium on Principles of Pro-gramming Languages, January 1998.

[NvO98] Tobias Nipkow and David von Oheimb. Javalight isType-Safe - Definitely. In Proc. 25th ACM Sympo-sium on Principles of Programming Languages, Jan-uary 1998.

[PV98] Joachim Posegga and Harald Vogt. Byte code veri-fication for java smart cards based on model check-ing. In 5th European Symposium on Research inComputer Security (ESORICS), Louvain-la-Neuve,Belgium, 1998. Springer LNCS.

[Qia98] Zhenyu Qian. A formal specification of Java VirtualMachine instructions for objects, methods and sub-routines. In Jim Alves-Foss, editor, Formal Syntaxand Semantics of Java. Springer-Verlag, 1998.

[SA98] Raymie Stata and Martın Abadi. A type system forJava bytecode subroutines. In Proc. 25th ACM Sym-posium on Principles of Programming Languages,January 1998.

[Sar97] Vijay Saraswat. The Java bytecode verifica-tion problem. Available from http://www.re-search.att.com/˜vj, November 1997.

[SMB97] Emin Gun Sirer, Sean McDirmid, and Brian Ber-shad. Kimera: A Java system architecture. Avail-able from http://kimera.cs.washington.edu, Novem-ber 1997.

[Sym97] Don Syme. Proving Java type soundness. Techni-cal Report 427, University of Cambridge ComputerLaboratory Technical Report, 1997.

[TMC+96] D. Tarditi, G. Morrisett, P. Cheng, C. Stone,R. Harper, and P. Lee. TIL: A type-directed op-timizing compiler for ML. ACM SIGPLAN Notices,31(5):181–192, May 1996.

Sun, Sun Microsystems, and Java are trademarks or registered trade-marks of Sun Microsystems, Inc. in the United States and othercountries.

18