The programming language standards scene, ten years on Paper 11: C

ELSEVIER Computer Standards & Interfaces 16 (1994) 495-5t)3

SIAHIAIm INI[R[AC[S

The programming language standards scene, ten years on Paper 11" C

Derek Jones

Knowledge Software, Ltd, Farnborough, Hants GU14 9RZ. UK

Abstract

This paper, one of a simultaneously published set, describes the establishment in 1984/5 of the standards project for the programming language C, and the progress of the project to the end of 1993.

Key words: Programming languages; Standardization; C

1. Introduction 2. The early history

This paper is one of a set published simultaneously in a special issue of Computer Standards & Interfaces under the general title The programming language standards scene, ten years on. This set of papers updates The current programming languages scene published in 1983 in what was then the journal Computers and Standards [2]. That in its turn updated Part 1 of the book Programming language standardisation published in 1980 [4]. This set of papers covers the period from 1 May 1993 (the reference date of the 1983 set) to 1 January 1994. However, C is a programming language which had not been subject to standardization activity at the time of the 1983 set. It was mentioned in Paper XIVB of that set [8] as a language which might potentially be standardized, and a project was in fact started later in that year. This paper therefore essentially covers the years 1984-1993.

2.1. Motivation

Work on the ANS1 standard for C started in the summer of 1983. This standardization effort grew out of the desire for a complete and precise definition of the C language. The book The C programming language, by Kernighan and Ritchie [6], had in many areas been supplanted by Unix C compilers, which provided extensions and interpretations to the definition in the book.

The goal of the ANSI C committee, X3Jl l , was to develop a consistent and unambiguous definition of the C language. The committee aimed to 'codify common existing practice'. Thus attention was focused on what existing C programs did; an important consideration was that existing code should continue to work.

Although existing code was considered inlpor- tant, existing implementations were not. In prac-

0920-5489/94/$l)7.00 © 1994 Elsevier Science B.V. All rights reserved SSDI 0920-5489(94)00025-C

496 D. Jones/Computer Standards & Interfaces 16 (1994) 495-503

tice it is difficult to separate the two. The existing code was written using existing compilers. Also vendors, with existing products, made up a large portion of the committee membership.

Another major goal was to preserve the tradi- tional spirit of C. This was later summarized [9] as:

Trust the programmer. Don't prevent the programmer from doing what

needs to be done. Keep the language small and simple. Provide only one way to do an operation. Make it fast, even if it is not guaranteed to be

portable. Note that, though many users new to C think

that it is a language designed for portability, the above points show that this is not the case. Both the standard and its rationale talk about portable and non-portable language features on an equal footing.

2.2. The ISO connection

In line with international agreements on language standardization (see Paper 1 of this series), the US national body formally notified the responsible international committee, I S O / I E C JTC1 /S C 22 of the launch of this project and asked if there was any interest elsewhere in creating an international standard. The widespread use of C ensured that the response was over- whelmingly positive, and at its 1985 plenary meeting SC22 formally constituted WG14, with a US convenor, to work with X3J l l on the international version which, it was agreed by all, needed to be identical - in technical content at least - with the eventual ANSI standard. There was some unease within some national bodies, the UK in particular, that the US clearly intended the project to be US-run in the same way as some earlier projects, and indeed had laid down a resolution to that effect. In the case of C the US view prevailed. (For more about the question of ANSI- ISO cooperation and the 1985 SC22 plenary meeting, see Paper 1 in this set.)

In the event, WG14-X3Jl l collaboration, apart from the inevitable occasional frisson, worked reasonably well, certainly in comparison with what

was happening elsewhere, though some tensions did come to the surface in the later stages, as we shall see.

3. Technical issues

During the five years which it took to produce the final draft standard, the committee received many proposals for extensions and changes. Mostly these were turned down, and the number of additions in the final standard language was small.

In creating the C standard the committee broke it down into three areas: the environment, the language and the library. The environment con- sists of the translation and execution environment. The translation environment is where the compiler operates to produce code from the in- put source text. The C language includes a preprocessor, and the translation environment for this preprocessor could be different from the translation environment for the compiler code generator. The execution environment is where the final program actually runs. The standard specifies eight phases of translation, and require- ments are specified for each.

The language is broken down into: declarations, expressions, statements and the preprocessor. Statements were virtually unchanged. There was some tightening up of the semantics of expressions, but no new operators. There were additions and changes to declarations and the preprocessor. New style declarations in the standard use abstract declarators, which the original Kernighan and Ritchie version did not, allowing the type of parameters to be specified. Also, it became possible to specify that a function takes a variable number of arguments. Many of these new ideas were taken from the recently created language C + +.

As for the standard C library, that was based on a standard issued by the U n i x / u s r / g r o u p in 1984, the facilities specific to Unix being dropped. The library was one area where portability was given a high priority. It was considered important that library functions should be implementable in a wide variety of environments. It was recognized

D. Jones / Computer Standards & Interfaces 16 (1994) 495-503 497

that computer manufacturers were not going to change their operating systems to accommo- date C.

In some cases where a feature was considered to be part of C, but not necessarily implementable on all machines (signal handling for example) it is made optional. A definition is given for what the functions implementing that feature must do, but they need not be implemented.

FinaLly, the standard introduced some new preprocessor operators and two new preprocessor tokens. Also a rule was introduced to break the chain on recursive invocations of macros - something which had given trouble to C users in the past.

4. Formal or informal definition?

One issue that arose in this project, as with other languages, concerned whether the defini- tional methods used should be formal or informal. This is something in which I was personally interested. As a compiler writer, I find the definition of programming languages in standards documents generally to be ambiguous. The problem arises because all languages have their own culture. Users within this culture have a particular way of looking at the world. Thus to understand the meaning of a language standard it is necessary to be a user of that language. Furthermore, when a standard is defined some points are always overlooked.

A formal definition overcomes both of these problems. It provides a complete definition of a language based on mathematical principles, without reference to any other culture. Also, the exercise of writing the definition forces the au- thors to address all of the issues involved in the language. Thus it is rare to find undefined questions in a formal definition.

X3J11 did form an ad hoc committee to look into the formal specification of programming languages, which recommended that the C standard be written both in prose and formally (the suggested method was denotational semantics). Un- fortunately, their recommendat ion was not implemented.

5. The final stages

After five years and two previous public reviews, the draft proposed ANSI standard went out for its third public review in August 1988. By this time there was strong pressure, especially from vendors, to leave the draft standard unchanged. Having a standard now being considered more important than having a better standard later. Also, after five years, committee members wanted an end to it.

At the WG14 meeting in June 1988, the question was put by the convertor, P.J. Plauger: is there any issue that would cause you to recom- mend that your country vote No at the 1SO level to the current ANSI draft standard? Two countries indicated that they would: UK and Den- mark.

Following a final review of comments, in De- cember 1988 X 3 J l l submitted its final draft of the standard to ANSI for ratification, with the expectation that approval would follow within a few weeks. However, it took another year before it finally happened.

By the time of the joint WG14-X3J l l meeting in Seattle in April 1989, final publication was still held up by various administrative details - or so they were considered at the time - and there were now the two international No votes to try to resolve, both UK and Denmark having, as fore- cast, voted against the ANSI draft standard going forward as an ISO standard. The Danes wanted a 'work around' for the Danish character set, while the UK were unhappy at the number of implicitly undefined areas of the language and wanted a more precisely defined standards document. However, the overriding aim of the US remained that of getting the current draft published unal- tered as an ANSI standard, and all parties wanted the ISO C Standard to be the same as the ANSI C Standard.

Three No votes were required to stop the ANSI draft going through as an ISO document, and so far there were only two, but quite a few countries had not registered a vote on the ISO ballot and it was just possible that another No vote would come forward. There had already been three public reviews of the draft standard


and another - which a substantive change would necessitate - would probably hold up the final publication date by at least a further six months. Hence the resistance to any substantive changes, however well intended, was enormous.

The Danish proposal would have been a substantive change and failed to receive any support. Various technical problems were also found in the proposal. However, there was quite a bit of sympathy with the UK's position. Many members of X3J l l felt that the wording of the draft standard should be more precise. Eventually, the com- promise reached was that an addendum should be produced, which would address those areas of the language currently not explicitly specified. X3J l l voted unanimously to act to adopt the addendum produced for the ISO C Standard.

After the Seattle meeting, final acceptance of the draft by ANSI as a standard was held up by an appeal to X3, the X3J l l ' s parent body. One of the respondents to the public review felt that his comments had not been fully addressed by the committee. This view was also shared by others concerning their own comments, but in this particular case the individual concerned was not willing to give up easily. He appealed through successive layers of the ANSI hierarchy, losing at every level. He finally gave up his initial appeal in November 1989. (There was a fear that he would also appeal against the publication of the final document, which he did in fact do. However, even excluding public holidays his submission did not reach ANSI by the closing date, and the standard went forward for final publication. The person responsible for this delay has not since been seen in the C standards world.)

Meanwhile the status of the ISO addendum was enhanced at the SC22 meeting in Berlin in September 1989, where it was agreed that the proposed C addendum would be normative, that is, changes would be made to the text of the standard itself, and the standards document reis- sued, rather than being issued as a separate document. Work on the addendum would be international, rather than being run by one country like the original standard.

The ANSI C standard [1] was published in December 1989 (after six years of elapsed time

and an estimated 50 man years of work) and became an ISO standard the following year. In due course, ANSI withdrew its own standard in favour of the ISO standard, so now there is just the one document [5].

6. Since the standard

The proposed Normative Addendum meant that, though the standard had been published, there was work still to do, and in fact three areas of activity emerged of which the Addendum was one, defect report processing and the Numerical C Extensions report (being produced by X3JI1) being the others.

6.1. The normative addendum

Though the original purpose of the addendum to the C standard was to address 'insecurities' in the text of the document, it was soon agreed to add the other topic which had led to a No vote, to address the issue of the creation of more readable source code for those users without certain characters on their keyboards (for instance square and curly brackets are not available on Danish keyboards). These two points were soon extended further, to include a Japanese proposal for additional library routines to support the processing of multibyte characters (an issue which has arisen with several languages, as can be seen from other papers in this set).

As work on the addendum progressed, an alternative and better method of addressing the original UK concerns became available, the defect reporting machinery. So work in this area - for which the addendum was originally intended! - was moved out, and is described below under Defect Reports. This left the addendum dealing with character set issues.

The existing C standard provides a mechanism for handling those situations where characters are not available on some national keyboards. Trigraphs are sequences of three characters that may be used instead of those characters that might not be available. There are nine trigraphs, some examples being ?? = (representing # ) and

D. Jones/Computer Standards & Interfaces 16 (1994) 4~)5 503 499

'??(representing <). The Danes argued that such usage did not produce readable source code, a point that most people would agree with. Two different methods were proposed to solve the problem, each addressing different characters or operator tokens. One was that digraphs, sequences of two rather than three characters, could be used as alternatives to some punctuators, for example %: (%% in C + + , this difference arising because the SC22 C + + working group, WG21, adopted an early draft of the C Addendum) as a (proposed) replacement for #; the other was that macros would be used to provide alternative for operators, for example and eq is a proposed replacement for &.

The exact wording of the Danish proposal has had a chequered history. This has recently (autumn 1993) led the UK to vote No on the basis that things are not sufficiently stable. The Netherlands also continued to vote No, on the grounds that the intended additions have insuffi- cient utility.

At the time of writing the situation is still unstable, with the C + + committee having adopted an early WG14 proposal that is incom- patible with the Addendum, the UK and Nether- lands continuing to vote No, and growing unease being felt by X3J l l . One point in favour of the Danish proposal is its relatively small size and the fact that many people want the other part of the Addendum (multibyte character support) adopted without any further delay: and splitting or remov- ing the Danish proposal would be likely to cause delay.

Turning now to the Japanese proposal, C was the first language standard to recognize that it is not always possible to represent a character within a byte. To this end the standard defines a wide character type and a small set of library routines for manipulating objects of this type. The Japanese proposal extends the existing functionality to provide a richer set of multibyte manipu- lation routines.

This new functionality includes the reading and writing of characters outside of the ASCII character set, comparison routines and routines to convert between different representations. Representat ion is an important issue. A lot of

problems would be solved if it was not necessary. to deal with character representation, both in memory, within files and within source text. How- ever, the Multibyte Support Extensions (MSE, to give it its working name, equivalent to MOCS in Cobol) is intended as a set of primitives upon which developers can build. So the question of possible representation and how best to manipu- late such objects has to be dealt with.

The Japanese have done a very good job at producing a workable proposal and have also addressed the political needs of the international community. At the time of writing there is univer- sal support for the proposal, and several implementations have been created.

6.2. Defect reports

Even with the huge effort that goes into the creation of a standard, problems are still found after publication. In the US the C standards committee, X3JI 1, is required to provide answers to any questions that have been formally submitted through CBEMA (Computer Business Equip- ment Manufacturers Association), who provide the secretariat for X3. It was agreed between X3J l l and WG14 that the answers to these interpretation requests would also be adopted at the ISO level and that national standards committees would funnel questions through CBEMA.

There were two reasons for this agreement. Firstly, X3J11 were the recognized experts on the C standard and had the manpower to provide answers. Secondly, there seemed to be no proce- dures available at the ISO level to handle interpretation requests. Other ISO standards committees had been asking how interpretations of standards were to be handled, and ISO set up a task force to look into it. This group found that machinery already existed to handle things called "defect reports ' . These defect reports bore a close similarity to X3's interpretation requests. Rather than go through the tortuous |SO process of creating the machinery needed to handle ' interpretation requests' (in the precise X3 sense), it was decided to adopt the existing defect reporting machinery.

In the case of C this defect reporting machin-


ery was adopted at the WG14 meeting held in Washington during December 1992. Also, given that ANSI had by then formally dropped its separate standard and adopted the ISO C standard, it was felt that now was the time for WG14 to start giving answers to any questions raised concerning the meaning of the C standard. The July 1993 meeting of WG14 in London was the first at which defect reports against the C standard were considered.

What of the interpretation requests them- selves? Do they indicate that the standard is no good?

Someone who was told that over fifty interpretation requests had been submitted against the C standard might well think that the original au- thors had done a sloppy job; the more so, per- haps, given that interpretation requests often contain multiple questions. What does all this really amount to?

In fact, questions can be broken down into several categories: • The questioner does not understand the stand-

ard. The answer to these usually involves citing the relevant subclauses (like C programs, answers to interpretation requests are short and to the point).

• The questioner does not like what the standard says and is asking for a change. The answer to these usually involves saying that the suggested change will be considered next time the standard is revised (this happens about once a decade). In one case somebody read about the answer to an interpretation request in a maga- zine, did not like it, and promptly submitted a request asking if that really was the intended answer!

• The standard is silent on a particular topic. Strictly speaking the answer to such questions should be that the behaviour is undefined. Sometimes 'a detailed reading of the standard will show' (committees never admit to making mistakes) ' that the behaviour is not undefined'.

• The wording in different parts of the standard appears to conflict. Of course, it does not actually conflict, because 'a detailed reading. . . '. In practice, the majority of questions have

little impact on programs in the real world. My

own view is that C programmers should take comfort from the fact that a group of people is actually trying to create a rigorous language definition. Those interested in rigorous languages might like to know that there have been over 2,000 interpretation requests against the Ada standard. (It is true that only one bug was found in the Pascal standard, but 85 have been found in the Extended Pascal standard to date).

6.3. Numerical extensions

The Numerical C Extensions Group, whose work is mainly driven by the needs of the super- computing community, is a subcommittee of X3J l l , its work currently having no ISO status. Its charter is to produce a technical report (for ANSI). This document, when published, will provide a strong indicator of what might be added in this area to a future revision of the C standard. Among the topics being considered are variable length arrays, compound literals, and complex datatypes.

7. Validation

7.1. Conformity checking

Having a standard is one thing, making sure that vendors conform to it is another. Vendors claim conformity - some started even before the standard existed - but some independent means of checking these claims of conformity is obvi- ously needed if potential purchasers of compilers want to make sure.

Two main bodies provide validation services, using a test suite to check claims of conformity, and issuing certificates to those that pass. In the US this body is the National Institute of Stan- dards and Technology (NIST), while in Europe there is a consortium of national standards bodies with the British Standards Institution's Quality Assurance Division ( B S I / Q A ) taking the lead role.

The validation suite selected is very influential. Any compiler failing to process the validation suite correctly is not considered to conform to the standard, and its vendor does not obtain a certificate. The lack of a certificate can impact sales.

D. Jones / Computer Standards & Interfaces 16 (1994) 495-503 501

In the US, certification of conformity is a requirement for government contracts, and Fed- eral Information Processing Standards are specified accordingly. The FIPS for C [3] references the ANSI (now ISO) standard, and it is the FIPS that NIST check for conformity to, rather than the A N S I / I S O standard directly. As a result of a competitive procurement conducted by NIST in 1988, Perennial, Inc of Santa Clara, California, was awarded the contract as the exclusive provider of the C validation suite to be used to demon- strate conformity to the C FIPS. Within the Eu- ropean Community, a similar exercise carried out by B S I / Q A a few years earlier resulted in Plum Hall, Inc being chosen as the exclusive supplier of validation suites for C.

Both organizations are now actively offering compiler validation services. Over 15 companies have now formally validated their products. Like the standards document itself, validation suites are also subject to interpretation requests. Both BSI and NIST have created review panels whose job it is to answer such questions and resolve disputes.

As part of their XPG branding programme, X / O p e n have accredited various testing labora- tories to perform C compiler testing. In the UK this is performed by the National Computing Centre Ltd in Manchester, using the Perennial validation suite.

7.2. Ecaluation

B S I / Q A ' s service is a validation 'and evaluation' service. The purpose of validation is to test that the language implemented by a compiler follows the rules given in the standard. A validation certificate does not give any indication of the usability of a compiler, which is the domain of evaluation. B S I / Q A has both a validation and an evaluation suite.

What evaluation covers somewhat depends on what the validation service covers. Various ideas were put forward as to what should be in a validation suite, the two extremes being: (1) only those constructs explicitly specified as constraints on C programs, or syntax errors should be tested; (2) all sentences containing a 'shall ' should be

tested. The difference is that many of the conditions covered by (2) are specified as being undefined and thus potentially legal by the standard. These undefined 'shall 's occur because of a tech- nicality in the structure of the standard: only those 'shall 's that occur in sections with the header 'Constraints ' are considered errors. A program that breaks any other 'shall ' condition is not in error, it is classified as undefined behaviour.

The proponents of (1) argue that these are the only conditions mandated by the standard. All other conditions are undefined behaviour and cannot, reliably be tested. The proponents of (2) argue that the set of conditions in (1) is too small and restrictive. There are many conditions that a p rogrammer would consider to be errors, that by the letter of the standard are not errors. multiply-defined labels in the same function being an example. They also argue that validation, to be worthwhile, should check as many elements of the compiler as possible.

Areas that can be covered by evaluation include floating point arithmetic, capacity to handle large numbers of declarations and statements, detection of undefined features, and handling of optionally implemented library functions. In this regard, an evaluation suite for floating point arithmetic and the trigonometric functions was available, through the work of Cody and Waite in the US, and there was experience in testing Pas- cal floating point arithmetic at the National Phys- ical Laboratory in the UK. Part of the Plum Hall test suite includes an expression generator. Scripts are available to drive this generator to produce self checking expression tests that cover all com- binations of C operators and operand types. An- other section of the Plum Hall suite includes a collection of C programs, each containing one instance of an undefined construct. Some of these undefined constructs can be detected at compile time, while others can only be detected at link or runtime.

8. ' C 9 X ' or C + + ?

C + + (see Paper 16 in this set) is widely viewed as the successor to C. Many programmers hold

502 D. Jones/Computer Standards & Interfaces" 16 (1994) 495-503

the view that once C + + has settled down (the language having just gone through a major revision), they can easily move their code away from C to C + + . However, things might not be that simple. Most people assume, incorrectly, that C + + is a pure superset of C. This view is partly created by the fact that many C + + compilers operate by first translating to C. The ability to be translated to C is no guarantee of compatibility. Translators are also available to convert Fortran, Cobol, Pascal, Basic, Ada, Lisp and many other languages to C - but nobody would claim that these languages were similar to C. Thus the connection between C and C + + is not as strong as most users of either language think.

Potential problems arise because the creators of C + + , and the I S O / A N S 1 committees for C + + standardization, regard it as a separate language, distinct from C, although the designers did aim for compatibility with 'classic' C [7] - see Paper 16 for further discussion of this point. The standards committees for C + + are charged with documenting any differences in behaviour of any construct which is also in C.

WG14 was asked for its views on the establishment of a standards project for C + + , and the issue was discussed at the April 1989 meeting. It came out in favour, but stated that it was not the committee to do the job - i.e. a separate working group should be set up for C + + . The vote in favour of the project was close, with many people arguing that C + + was not yet sufficiently mature to warrant standardization. Those wanting a standard felt that such a body would stop Bjarne Stroustrup, the language creator, from making further changes to the specification. In fact Stroustrup simply joined the committee and continues to make the changes through the committee, one result being that C + + continues to evolve and grow. Some say that the language is getting too complicated and will fall apart under its own weight.

The question that remains for the C standards world is how C + + affects its future plans. C itself is not permanent ly set in stone, with a revision of the standard for the language, natu-

rally dubbed 'C9X', due to appear some time after the intended appearance of the C + + standard. Hence the possibility is there for C then to move somewhat towards C + + , in addi- tion to C + + being constrained by user pressures on its standards committee not to do too much of its own thing.

Much may depend on whether the user com- munities of these two closely related but never- theless distinct languages grow further apart in future, or closer together. This will clearly have some influence on the way they both develop. The next revision of C could react against the perceived complexity of C + + by producing a small, efficient object-oriented language, but would this be convergent or divergent are far as C + + is concerned? As Paper 16 also records (see section 5 of that paper), the arguments continue. At its meeting in December 1993, WG14 had its first formal discussion about the revision of the C standard. It was decided that the July 1994 meeting in Tokyo would at tempt to define the spirit of C for the next century. Much may depend on the outcome of that meeting.

References

[1] ANS X3.159: 1989, Programming language C, American National Standards Institute, New York.

[2] The current programming language standards scene, special issue of Computers and Standards, Vol 2 Nos 2-3, 1983.

[3] FIPS PUB 160, Programming language C, National Insti- tute for Standards and Technology, Gaithersburg Virginia, 1991.

[4] I.D. Hill and B.L. Meek, Programming language standardisation, Ellis Horwood 1980.

[5] ISO/IEC 9899: 1990, Programming language C, Interna- tional Organization for Standardization and International Electrotechnical Commission, Geneva.

[6] B.W. Kernighan and D.M. Ritchie, The C programming language, Prentice-Hall, Englewood Cliffs N J, 1978.

[7] A. Koenig and B. Stroustrup, C++ : as close as possible to C - but no closer, The C++ Report, Vol 1 No 7, July 1989.

[8] B.L. Meek, Paper XIVB: Other potential standards, pp 143-146 of [2].

[9] Rationale for Draft Proposed ANS programming language C, X3Jl l document 88-003, 1988.

D. Jones/Computer Standards & hzterfaees 16 (1994) 495 503 5(13

Derek Jones has been a director of Knowledge Software for eleven years, working in the area of optimizing compilers. Languages with which he has been involved are Pascal, Cobol, Fortran, C, CHILL, Coral 66, Snobol- 4 and C + + . During the last three years his main interests have been in the checking of applications for conformity to the C, Posix and XPG standards. He has been convenor of the UK C panel for four years, has represented the UK at WG14 meet-

ings for five years, and was appointed project editor for the C Normative Addendum in 1990.

Documents

The programming language standards scene, ten years on Paper 11: C