1 Muhammed Al-MulhemVisual Languages Visual Programming Languages ICS 519 Part 1 ICS Department KFUPM Sept. 15, 2002

1Muhammed Al-Mulhem Visual Languages

Visual Programming Languages

ICS 519Part 1

ICS Department

KFUPM

Sept. 15, 2002


• Learning programming is a difficult task.

• Writing and testing a program is a difficult and time consuming task.

• The majority of computer users are non programmers.

• Programming languages are designed for professional programmers and computer scientists.

1Introduction


• There are historical reasons for this lack of emphasis on human convenience in programming languages (Shu, 1988).

• The computer scientists, for the last fifty years, have concentrated on machine efficiency.

• Computing costs were high and human costs were relatively low.

• In the early years of computing, it was justifiable that efficiency took precedence over ease of programming.


• With the decline in computing costs it is realized that it is the human resource that must be optimized.

• The challenge is to bring computer capabilities to people without special computer training.

• To meet the challenge a number of facilities have been introduced to ease the pain of programming and remembering commands.

• These facilities include icons, pointing devices, and menus.


• Programming languages have evolved over the last fifty years from first generation programming languages to fourth generation programming languages.

• One characteristic of most of these languages is that they use one-dimensional and textual formats to represent programs.

• These formats are suitable for sequential machines but not for the multi-dimensional and visual human mind (Chang, 1986).


• Visual programming is an attempt to simplify programming and to make better use of the human mind's capabilities

• George Raeder states "... human mind is strongly visually oriented and that people acquire information at a significantly higher rate by discovering graphical relationships in complex pictures than by reading text. ... When we scan paragraph, headlines, and material in bold type to speed up text search, we are really acting in the pictorial mode to overcome this limitation" (Raeder, 1985).


• Visual programming represents a conceptually revolutionary departure from the traditional one-dimensional and textual programming languages.

It is stimulated by the following premises:

1) Pictures are more powerful than words as a means of communication. They can convey more meaning in a more concise unit of expression.

2) Pictures aid understanding and remembering.

3) Pictures may provide an incentives for learning program.


4) Pictures do not have language barriers. When properly designed, they are understood by people regardless of what language they speak.


• Visual programming has gained momentum in recent years.

• The falling costs of graphics-related hardware and software has made it feasible to use pictures as a means of communicating with the computers.

• Visual programming is a very young but growing field.

• There is no common agreement on the definition of visual programming.


2Definitions

• Visual programming languages is a new paradigm.

• There is no single formal definition for VPL. For this reason we will present a number of definitions that appear in the literature.


1) Chang, 1987

• Visual programming languages, usually deals with objects that do not have an inherent visual representation. This includes traditional data types such as arrays, stacks, and queues and application data types such as forms, documents, and databases. Presenting these objects visually is helpful to the user.

• For the same reason, the languages themselves should be represented visually.

• In other words, both programming constructs and the rules to combine these constructs should be presented visually.


2) Shu, 1988

• A visual programming language can be informally defined as a language which uses some visual representations ( in addition to or in place of words and numbers) to accomplish what would otherwise have to be written in a traditional one-dimensional programming language.

• Note that this definition imposes no restrictions on the type of data or information.

– It is immaterial whether the object being operated on or being displayed to a user by a visual language is textual, numeric, pictorial, or even audio.

– What is important is that the language itself must employ some meaningful visual expressions as a means of programming.


3) Tortora, 1990

• A visual language models an icon system. In the icon system, a program consists of a spatial arrangement of pictorial symbols ( elementary icons ).

• Note that, the spatial arrangement is the two-dimensional counterpart of the sequential statements in the traditional programming languages.

– In these languages, a program consists of a string in which terminals are concatenated.

– Concatenation is the only construction rule, therefore it doesn’t appear in the language grammar.


• In the case of visual languages, three construction rules are used to spatially arrange icons:

– Horizontal concatenation.

– Vertical concatenation.

– Spatial overlay.

• For this reason, there is a need to include both spatial operators and elementary icons in the vocabulary of terminal symbols of the visual language grammar.

• Elementary icons are partitioned into two categories:

– Object icons which identify objects.

– Process icons which express computation.


4) Al-Mulhem, 1993

• Visual programming languages is a language which uses a combination of text and meaningful graphical symbols to write programs. These graphical/textual symbols represent the following:

a) Data types such as numbers, characters, pictures, and sound.

b) Data structures such as arrays, records, and pointers.

c) Programming constructs such as iteration, selection, and conditional looping.

d) Rules to combine the programming constructs.


5) Burnett (1999)

• Visual programming is programming in which more than one dimension is used to convey semantics.

• Examples of such additional dimensions are the use of multi-dimensional objects, the use of spatial relationships, or the use of the time dimension to specify “before-after” semantic relationships.

• Each potentially-significant multi-dimensional object or relationship is a token (just as in traditional textual programming languages each word is a token) and the collection of one or more such tokens is a visual expression.


• Examples of visual expressions used in visual programming include diagrams, free-hand sketches, icons, or demonstrations of actions performed by graphical objects.

• When a programming language’s (semantically-significant) syntax includes visual expressions, the programming language is a visual programming language (VPL).


VPL Research

• Directions of research in this area are:

1. Visual approaches to traditional programming languages (flowchart).

2. Visual approaches that deviated from traditional programming (programming by demonstration).


How good are they?

• Good for toy programs.

• Bad for realistically-sized programs.


More approaches

• Incorporating visual techniques into programming environments:

• To support textual specification of GUI layout

• To support electronic forms of software engineering diagrams.

• To create/visualize relationships among data structures.

• To visually combine textually-programmed units to build new programs.


Examples

• Visual Basic (for BASIC)

• VisualWork (for SmallTalk)

• CASE tools that support visual specificatio of relationships among program modules, automatic code generation, …etc.



More approaches

• Domain-specific visual programming systems

• Examples:

1. LabVIEW – programming laboratory data acquestion.

2. AVS – programming Scientific Visualization

3. PhonePro – programming telephone and voice-mail behavior.

4. Cocoa – programming graphical simulations and games.


What is next?

• VPLs for general-purpose programming.

• This can be reached by:

1. Improving the ways VP can be used.

2. Improving domain-specific programming


Strategies used in VPLs

• Concrete: specify some aspect of semantics on a specific object or value.

• Direct: manipulate a specific object or value directly to specify semantics.

• Explicit: describe dataflow relationships by drawing directed edges among related variables.

• Immediate visual feedback: automatic display of effects of program edits.


Classification

1. Pure visual programming

2. Hybrid (textual / visual) programming

3. Programming-by-example

4. Constraint-based programming

5. Form-based programming


Pure visual programming

• Programmer manipulate icons to create a program.

• A program is debugged and executed in the same visual environment.

• A program is compiled directly from its visual representation.

• A program is never translated into intermediate text-based language.

• Examples: Prograph, VIPR, and PICT.


Hybrid (textual / visual) programming

• Two forms:

1. Systems in which programs are created visually and then translated into an underlying high-level textual languages.

• Example: Rehearsal World – the user trains the system to solve a particular problem by manipulating graphical actors (icons) and then the system generates a Smalltalk program to implement the solution.


Hybrid (continue)

2. Systems which involve the use of graphical elements in a textual language.

• Example: Work of Erwig et. al. – developing extensions to languages like C and C++ which allow programmers to mix their text code with diagrams.

• For instance, one can define a linked list data structure textually and then perform an operation like deletion of a node by drawing the steps in the process.


Programming by example

• Allowing the user to create and manipulate graphical objects with which to teach the system how to perform a particular task.

• Examples: Rehearsal World and Pygmalion


Constraint-oriented programming

• A programmer models physical objects as objects in the visual environment which are subject to constraints designed to mimic the behavior of natural laws, like gravity.

• Popular for simulation design.

• Examples: ThingLab, ARK


Form-based programming

• Borrowed their visualizations and programming metaphors from a spreadsheets.

• Represent programming as altering a group of interconnected cells over time and often allow the programmer to visualize the execution of a program as a sequence of different cell states which progress through time.

• Example: Forms/3


VPL Orphans

• Algorithm animation systems

• Provides interactive graphical displays of executing programs

• Example: BALSA

• user-interface design systems

• Provided with many modern compilers.

• Example: Visual Basic, Visual C++


Theory• Based on work by Chang• Definitions• Icon (generalized icon): An object with the dual

representation of: – logical part (the meaning)– Physical part (the image)

• iconic system: structured set of related icons

• iconic sentence (visual sentence): spatial arrangement of icons from iconic system


Theory (con.)

• visual language: set of iconic sentences constructed with given syntax and semantics

• syntactic analysis (spatial parsing): analysis of an iconic sentence to determine the underlying structure

• semantic analysis (spatial interpretation): analysis of an iconic sentence to determine the underlying meaning


Theory (con.)

• A visual Language Models an Icon System.

• In the Icon System, a program (Visual Sentence) consists of a spatial arrangement of pictorial symbols (elementary icons).

• The spatial arrangement is the two-dimensional counterpart of the standard sequentialization in the construction of programs in the case of traditional programming languages.


• In traditional languages, a program is expressed by a string in which terminals are concatenated.

• As this operation (concatenation) is the only construction rule, it does not appear explicitly in the vocabulary of the language grammar.

• In the case of visual languages, three construction rules are used to spatially arrange icons:

– Horizontal concatenation &

– Vertical concatenation ^

– Spatial overlay +


• For this reason, there is a need to include the spatial operators and elementary icons in the vocabulary of terminals of the visual language grammar.

• The elementary icons are partitioned into:– object icons

– process icons.

• Elementary object icons identify objects while process icons express computations.

• The meaning depends on the specific visual sentence in which it appears.


Example

• Consider the following primitive icons from the Heidelberg icon set.

where

• the square denotes a character, and its interpretation is unique in the system.

• the arrow can be used to express insertion or moving operations in different visual sentences.


• A Visual Language (VL) is specified by the triple

» (ID, Go, B)

where

• ID is the icon dictionary

• Go is a context-free grammar

• B is a domain-specific knowledge base.


The icon dictionary ID

• It is a set of triples of the form

» i = (il , ip , type(i))

• Each triple i describes a primitive icon where

– il is the icon name (logical part or meaning)

– ip is the icon sketch (physical part)

– type(i) is the icon type (process or object)


The grammar Go

• It is a context-free grammar (N, T, S, P).

Where

• N is the set of nonterminals

• T = Tl U Tsp

• Tl = { il | i = ( il, ip, type(i)) ID}

• Tsp = { &, ^, + }

• S is the start symbol of the grammar

• P is the set of productions of Go


• The grammar Go specifies how more complex icons can be constructed by spatially arranging elementary icons.

• Usually, nonterminals in Go & the start symbol represent composite object icons.

• Composite object icons can be derived in one or more steps starting from a given nonterminals, N, which is different from S.

• In other words, composite object icons are obtained by spatial arrangement of elementary object icons only and are assumed to be of type object.


• Visual Sentences contain at least one process icon.

• Visual Sentences have no type.

• Here, object icons denote both elementary object icons & composite object icons.

• It is easy to prove that all the icons involved in the icon system (elementary object, composite object, and visual sentences) are objects with dual representation of a logical and a physical part as follows:


• Elementary object --- by definition in ID.

• Composite object and visual sentences ---– The physical part is their image

– The logical part (the meaning) can be derived from the meaning of the elementary objects occurring in them.

• The meaning of a visual sentence is expressed in terms of a conceptual tree, which represents an intermediate code for later execution.


Conceptual Trees

• The meaning of a visual sentence is expressed in terms of a conceptual tree (CT), which represents an intermediate code for later execution.

• In CT notations a concept is denoted by [ ], and a relation is denoted by ( ).


The concepts are expressed in the form

[ OBJECT : object-name]

[ EVENT : event-name]

where

• object-name represents the logical part (meaning) of an object icon (elementary icon or composite object icon)

• event-name are event names, i.e., procedure names, which accomplish the task expressed by the visual sentence


• The following CTs are basic substructures occuring in all the CTs which represent the logical part of a visual sentence.

[EVENT: event-name] (rel1) [OBJECT: object-name1]

(rel2) [OBJECT: object-name2] (1)

[EVENT: event-name] (rel) [OBJECT: object-name] (2)

[OBJECT: object-name] (rel) [OBJECT: object-name] (3)

The meanings of these CTs are as follows:


• 1) A process icon has two object icons as arguments

Example:

consider the following icon from the Heidelberg icon set


• It is composed of the object icons

• Selected-string

• String

• and the process icon Up-arrow


• Its meaning is expressed by the CT

[EVENT: replace] (object) [OBJECT: String]

(place) [OBJECT: Selected-string]


2) A process icon requires only one argument.

Example:


• It is obtained by composing in spatial overlay:

• the object icon Selected-string

• and the process icon cross


• Its meaning is described by the CT

[EVENT: delete] (object) [OBJECT: Selected-string]


3) Here we have two combined object icons. This is the case in which one of the two objects involved in the visual sentence represents a qualitative or quantitative specification of the other object.

Example:

The object "name” identifies the object "file"

File nameFile name


The meaning is

[OBJECT: file] (identifier) [OBJECT: name]


• Complex structures can be obtained by composing the basic substructures as shown below:

1 2 3

This visual sentence could mean: append file1 to file2 and copy the result into file3


• The sentence could be decomposed into two subsentences:

1 2

and

2 3


• These subsentences are described, respectively, by the following CTs

[EVENT: append] (object) [OBJECT: file1]

(place) [OBJECT: file2]

[EVENT: copy] (source) [OBJECT: file2]

(destination) [OBJECT: file3]


The whole visual sentence is described by

[EVENT: copy] (destination) [OBJECT: file3]

(source) [EVENT: append]

(object) [OBJECT: file1]



The Knowledge Base B• The Knowledge Base B contains domain-specific

information necessary for constructing the meaning of a given visual sentence.

• It contains information regarding» Event names

» Conceptual relations

» Names of the resulting objects

» References to the resulting objects.


It is structured in the following seven tables:1) EVENT-NAME [proc-name, obj-name1, obj-name2, op1, op2] This

table returns the name of the event (procedure) to be associated to the process icon proc-name when the objects obj-name1, obj-name2 are spatially arranged (via op1, op2) to proc-name.

2)LEFT-REL [proc-name, obj-name1, obj-name2, op1, op2]

3)RIGHT-REL [proc-name, obj-name1, obj-name2, op1, op2]

These two tables hold the conceptual relations existing between EVENT-NAME [Proc-name, obj-name, obj-name2, op1, op2] and obj-name1 or obj-name2 respectively.


4)RESULT-NAME [Proc-name, obj-name1, obj-name2, op1, op2]

This table returns the name of the object resulting from the execution of the event.

EVENT-NAME [Proc-name, obj-name1, obj-name2, op1, op2]. With arguments obj-name1 and obj-name2.

5)RESULT-REF [Proc-name, obj-name1, obj-name2, op1, op2].

This table returns a reference to the object representing the result.


The following two tables take into account cases in which object icons are composed (composite object icons).

6) RELATION [obj-name1, obj-name2, op].

This returns the conceptual relation existing between obj-name1 and obj-name2 when they are spatially combined by means of the operator op.

7)RESULT-OBJECT [obj-name1, obj-name2, op].

This returns the name of the object resulting by the spatial combination (via op) of obj-name1 and obj-name2.

In the previous tables, obj-namei refer to the logical part of the icons.


• Besides domain-specific relations the tables can also contain three special relations:

<new> :This relation means that the combination

obj-name1 op obj-name2

gives rise to a new object, different from obj-name1 and obj-name2.

<null>:This relation means that the combination

obj-name1 op obj-name2

generates an object that has the same logical part of the obj-name1 or obj-name2.

In other words, it points out that no additional information is derived from such a combination.


<l-obj> , <r-obj>:

These two relations mean that the combination

obj-name1 op1 proc-name op2 obj-name2

generates an object that has the same reference of the icon whose name is obj-name1 or obj-name2 respectively.


file2 file3

Obj-name1Obj-name1

Op1=&Op1=&

Proc-nameProc-name Op2=&Op2=& Obj-name2Obj-name2


Table 1: returns the name of the event “copy”

Table 2, 3:

LEFT-REL

[EVENT: copy] (source) [OBJECT: file2]

(destination) [OBJECT: file3]

RIGHT-REL


Table 4: Return file 3

Table 5: Return a reference (pointer) file3

Table 6:

[OBJECT: file] (identifier) [OBJECT: file3]

returns this

Table 7: Return the name of the resulting object in 6


The Attribute Grammar • An attribute grammar consists of a semantics

rules for synthesizing the meaning of a visual sentence

• AG consists of an underlying context-free grammar where

i) each nonterminal in the context-free grammar has associated two attribute sets, namely, synthesized and inherited attributes.


ii) each production has associated a set of semantics rules, used to determine the values of the attributes of the nonterminals depending on the values of the attributes of the remaining nonterminals appearing in the same production.

• The initial nonterminal has only synthesized attributes, and one of them is designated to hold the meaning of the derivation tree.


The production rules in Go are of the following form:

a) X t t T

b) X Y where type (X) = type (Y)

c) X M op L | L op M | M1 op M2

d) X M1 op1 L op2 M2

where

• type (L) = PROCESS

• type (M) = type (M1) = type (M2) = OBJECT.

• op, op1, op2 = nonterminals for spatial operators ( &, * , +)


• The type for a given nonterminal X is determined as follows:

Let

X t be a derivation in Go and t T*

Then type (X) = spatial operator if t Tsp

type (X) = type (t) otherwise.

• The set of inherited attributes is empty for each nonterminal in Go.


• The set of synthesized attributes associated to each nonterminal X is defined as follows.

• For a given non-terminal X

1. if type (X) = OBJECT :

NAME (X) is the name of the object icon represented by X.

CG(X) is the CT associated to the subtree rooted in X.

REF(X) is the reference number. The reference number is an index associated with each elementary object icon occurring in a visual sentence. It allows one to distinguish between different occurrences of the same elementary object icon in a visual sentence.


2. if type (X) = PROCESS:

NAME(X) is the name of the process icon represented by X.

3. if type (X) : "spatial operator" :

OP(X) is the spatial operator derived from X.


• The semantic rules associated with the four production rules are:

a) X t

a1) If t = icon-id

where

( icon-id, icon-sk, type( i ) ) ID and

type ( i ) = OBJECT (i.e. t is a terminal for an elementary icon).

Then

NAME(X) icon-id

REF (X) ref-set

CG (X) [OBJECT : NAME(X), REF(X)]


a2) If t = icon-id

where

( icon-id, icon-sk, type( i ) ) ID and

type ( i ) = PROCESS

Then

NAME(X) t

a3) If t {&, L , +}

Then

OP(X) t


b) X Y

b1) If type (Y) = OBJECT

Then

NAME(X) NAME (Y)

REF (X) REF (Y)

CG (X) CG (Y)

b2) If type (Y) = PROCESS

Then

NAME(X) NAME (Y)

b3) If Y op and op {&, L , +}

Then

OP(X) OP(Y)


c) X M op L

c1) If type (M) = OBJECT and type (L) = PROCESS

Then

NAME(X) RESULT-NAME [NAME(L), NAME (M), ‘null-obj’, op(op),

‘null-op’]

REF(X) SET-REF (L, M, ‘null-obj’, op, ‘null-op’)

CG (X) SELECTIVE-MERGE (L, M, ‘null-obj’, op, ‘null-op’)


c2) If type (M) = PROCESS and type (L) = OBJECT

Then

NAME (X) RESULT-NAME [NAME(M), ‘null-obj’, NAME(L), ‘null-op’, OP(op)]

REF(X) SET-REF (M, ‘null-obj’, L, ‘null-op’ , op)

CG (X) SELECTIVE-MERGE (M, ‘null-obj’, L, ‘null-op’, op)


c3) If type (M) = type (L) = OBJECT

Then

NAME (X) RESULT-OBJECT [NAME (M), NAME(L), OP(op)]

REF(X) REF (M) U REF (L)

CG (X) LINK (X, M, L, op)


d) X M1 op1 L op2 M2

If type (L) = PROCESS and

type (M1) = type (M2) = OBJECT

Then

NAME (X) RESULT-NAME [NAME (L), NAME(M1), NAME (M2), OP(op1), OP (op2)]

REF(X) SET-REF (L, M1, M2, op1, op2)

CG (X) SELECTIVE-MERGE (L, M1, M2, op1, op2)


• Semantic rules c and d make use of the following procedures:

Procedure LINK (X, M1, M2, op)

rel RELATION [NAME(M1), NAME (M2), OP(op)]

case rel of

<new> : if NAME (X) ID then

- add the new composite object NAME(X) to ID

- return [OBJECT: NAME (X), REF(X)]

endif

<null> : return CG(M1)

else : return APPEND (CG(M1), CG(M2), rel)

end case

end LINK


Procedure SELECTIVE-MERGE (L, M1, M2, op1, op2)

C [EVENT:EVENT-NAME [NAME(L),

NAME(M1), NAME(M2), OP(op1), OP(op2)]

if OP(op1) = ‘null-op’ then G1 C

else rel1 LEFT-REL [NAME(L), NAME(M1),

NAME (M2), OP(op1), OP(op2)]

G1 APPEND (C, CG(M1), rel1)

endif

if OP(op2) = ‘null-op’ then G2 C

else rel2 RIGHT-REL [NAME(L), NAME(M1),

NAME (M2), OP(op1), OP(op2)]

G2 APPEND (C, CG(M2), rel2)

endif

return HEADER-MERGE (G1, G2)

end SELECTIVE-MERGE


Procedure SET-REF (L, M1, M2, op1, op2)

ref RESULT-REF [NAME(L), NAME(M1), NAME (M2), OP(op1), OP(op2)]

case ref of

<l-obj> : return REF(M1)

<r-obj> : return REF(M2)

<new-obj>: return NEXT-REF( )

end case

end SET-REF


The Icon Interpreter:

• The system diagram of the icon interpreter is as follows:

G0 BID

PatternAnalysis

SyntaxAnalysis

Construction of theLogical part

Visual Sentencce S

Pattern String of SParse Tree of S

Meaning of SCG


• The module ”Pattern Analysis" transforms the visual sentence into pattern string .

Example

• The visual sentence

• is represented by the pattern string

(char & char & char) + cross


• The icon interpreter is logically divided into two phases:

1.The syntax analysis is accomplished by a parsing algorithm for context-free grammars.

2.Constructing the meaning of the given visual sentence by evaluating the attributes of the nonterminal on the root of the parse tree using the ATTRIBUTE_EVALUATOR procedure

• The attribute evaluation is accomplished by means of a postorder visit of the parse tree t of the visual sentence S


Procedure ATTRIBUTE-EVALUATOR (X).

if X is a nonterminal in Go

then /* let p be the production applied to the node X in t, and let 1, ..., np be the indices of nonterminals in the right-hand side of p*/

for j = 1 to np do

call ATTRIBUTE-EVALUATOR (jth nonterminal son of X in t)

end for

apply the semantic rules associated to p so as to evaluate the attributes of X

endif

end ATTRIBUTE-EVALUATOR


Example

• This example shows an iconic command language for simple operating system.

• The set of elementary icons ID and the grammar G0, are shown in the following slides.




• The knowledge base B is given next.

• Each entry in the tables, which constitute the knowledge base, contains five items, which correspond, from top to bottom, to the tables:

– EVENT_NAME

– LEFT_REL

– RIGHT_REL

– RESULT_NAME

– RESULT_REF



• Consider the following visual sentence:

• Whose corresponding pattern string is

[FILE,1] & PLUS & [FILE,2] & ARROW&[FILE,3]

• The parse tree for this sentence is shown next.



• The construction of the conceptual tree is given in the next table. At level 1, the resulting conceptual tree is given by:

[EVENT: copy] (destination) [OBJECT: file3]

(source) [EVENT: append]




Examples of VPLs

• Prograph (1989) – Dataflow programming

• DataLab (1990) – Programming by example

• Form/3 (1994) – Form-based programming

• Clarity (2001) – Functional programming


Prograph

• The Prograph language is an iconic language.

• Program is drawings.

• Prograph interpreter and compiler execute those drawings.

• Prograph is object-oriented

• Prograph supports dataflow specification of program execution


Simple program

• A simple Prograph code which sorts, possibly in parallel, three database indices and updates the database.


Paragraph’s icons

• Prograph uses a set of icons:

• classes are represented by hexagons,

• data elements by triangles,

• pieces of code by rectangles with a small picture of data flow code inside, and

• inheritance by lines or downward pointing arrows.



More Icons

• The representations can then be combined and used in a variety of language elements.

• For example, initialization methods - which are associated with an individual class - are depicted as hexagonally shaped icons with a small picture of data flow code inside.


Icon (C0nt.)

• Triangles represent instance variables in a class’ Data Window to show that they’re data.

• Hexagons drawn with edges and interiors similar to the instance variable triangles represent class variables. This associates them with the class as a whole while also associating them with data.

• Next Figure shows three more complex representations: initialization methods , instance variables, and class variables.


Icon (Cont.)


Object-Oriented

• The Prograph language is object-oriented.

• Class-based,

• Single inheritance (a subclass can only inherit from one parent),

• Dynamic typing

• With garbage collection mechanism based on reference counting.


Cont.

• The programmer can design and implement new classes,

• He can visually specifying:

– the new class’ inheritance,

– additional instance or class variables,

– modified default values for inherited instance or class variables,

– and new or overridden methods.


Cont.

• Polymorphism allows each object to respond to a method call with its own method appropriate to its class.

• Binding of a polymorphic operation to a method in a particular class happens at run-time, and depends upon the class of the object.

• The syntax for this ‘message send’ is one of the more unusual aspects of Prograph’s syntax.

• The concept of ‘message sending’ per se is not used in Prograph.


Annotation

• Rather, the idea of an annotation on the name of an operation is used. There are four cases:

1. No annotation means that the operation is not polymorphic.

2. ‘/’ denotes that the operation is polymorphic and is to be resolved by method lookup starting with the class of the object flowing into the operation on its first input


Annotation (Cont.)

3. ‘//’ denotes that the operation is polymorphic and is to be resolved by method lookup starting with the class in which this method is defined.

4. ‘//’ plus super annotation means that the operation is polymorphic and resolves by method lookup starting with the superclass of the class in which this method is defined.

• In Prograph there is no SELF construct (or a ‘this’ construct, to use the C++ terminology).


• Prograph has an almost completely iconic syntax, and this iconic syntax is the only representation of Prograph code.

• Next Figure shows a small code fragment typical of the Prograph language.

• There is no textual equivalent of a piece of a

Prograph program - the Prograph interpreter and compiler translate directly from this graphical syntax into code.


Example


Icons

• Next Figure shows most of the lexicon of the language (icons).

• The inputs and outputs to an operation are specified by small circles at the top and bottom, respectively, of the icons.

• The number of inputs and outputs - the operation’s arity - is enforced only at run-time.

• In addition, there are a variety of annotations that can be applied to the inputs and outputs, or to the operation as a whole to implement looping or control flow.



Spaghetti Code

• Visual languages like Prograph sometimes suffer from the spaghetti code problem: there are so many lines running all over that the code for any meaningful piece of work actually ends up looking like spaghetti.

• Prograph deals with this issue by enabling the programmer to ‘iconify’ any portion of any piece of code at any time during the development process.


Local

• This iconified code is called a ‘local’.

• Effectively, locals are nested pieces of code.

• There is no limit to the level of nesting possible with locals, and there is no execution penalty to their use.

• In addition, locals can be named and this naming, if done well, can provide a useful documentation for the code.


Example

• Next Figure is a code to build a dialog box containing a scrolling list of the installed fonts without the use of Prograph locals to factor the code.



Same Example

• Next Figure is the same code with the use of Prograph locals to factor the code.



Local (Cont.)

• Note that long names, often containing punctuation or other special characters can be used for the names of locals.

• The proper use of locals can dramatically improve the comprehensibility of a piece of code.

• The one negative aspect of their use is the so-

called “rat hole” phenomena - every piece of code in its own rat hole. This can sometimes make finding code difficult.


Syntax

• Three of the most interesting aspects of the Prograph syntax are:

– list annotation,

– injects, and – control annotation.


List Annotation

• Any elementary operation can be made to loop by list annotating any of its inputs.

• When this is done, the operation is invoked multiple times - one time for each element of the list that is passed on this input.

• Thus, list annotation on an input causes the compiler to construct a loop for the programmer. Since this annotation can be made on a local as well as a primitive operation, this looping construct is quite powerful.


List Annotation (Cont.)

• In addition, list annotation can be done on a output.

• On an output terminal, list annotation causes the system to put together all the outputs at this terminal for every iteration of the operation and to pass as the ‘final’ output of the terminal this collection of results as an output list.



• List annotation, then, enables the programmer to easily make an operation into a loop that either breaks down a list into its component elements and runs that operation on the elements, or to builds up a list from the multiple executions of an operation.

• An individual operation can have any number of its inputs or outputs with list annotations.



• Next Figure shows several examples of list annotation, an efficient mechanism for iterating over a list.

• The operation will automatically be called repeatedly for each element of a list, or its outputs will be packaged together into a list.





Inject

• Inject lets you pass a name at run-time for an input which expects a function.

• This is similar to Smalltalk’s perform, procedure variables in Pascal or function pointers in C.

• Suppose, for example, that you want to implement a function FooMe that takes two arguments: a list of objects, and a reference to a method to be applied to each of those objects.


Inject (Cont.)

• In Object Pascal pseudo-code, this function might look something like this:

Function FooMe(theList: TList; Procedure DoThis(object: TObject));

{ Iterates through the list of objects applying the DoThis procedure }

BEGIN

theList.Each(DoThis);

END;


• Next Figure shows a Prograph implementation of FooMe.

• Note that just the name - as a string - of the method to be applied to each object is passed to FooMe.

• This string is turned into a method invocation by the inject.



Inject (Cont.)

• The Prograph representation of inject - a nameless operation with one terminal descending into the operation’s icon - is a particularly well-designed graphic representation for this programming concept.

• When properly used, inject can result in extremely powerful and compact Prograph implementations of complex functions.

• However, when used improperly, it can result in code that is very difficult to read or debug, like the computed GOTO of FORTRAN.


Control Flow

• Control flow is a combination of annotations on operations and a case structure.

• A Prograph method can consist of a number of mutually exclusive cases. These cases are somewhat similar to the cases in a Pascal CASE statement or a C switch expression.

• The main difference is that unlike Pascal or C, the code to determine which case will be executed is inside the case, not outside it.


Example

• Suppose that we want to implement a small function that has one integer input, and

• if that input is between 1 and 10, the value of the function is computed one way;

• if it is between 11 and 100, another way, and

• if it is anything else, yet a third way.

• In Pascal-like pseudo-code, this function would look something like this:


Function Foo( i: Integer): Integer;

Begin

Case i of

1..10: Foo := Calculation_A(i);

11..100: Foo := Calculation_B(i);

OtherwiseFoo := Calculation_C(i);

End; {Case}

End; {Foo}


Control Flow (Cont.)

• The implementation of Foo in Prograph would also involve three cases, as shown in the next Figure.

• The control annotations are the small boxes on the right of the match primitives at the top of the first two cases. (Note that the window titles show the total number of cases in a method, and which case this window is, as in “2:3” the second of three cases.)




• In this simple example, the same annotation - a check mark in an otherwise empty rectangle - is used.

• The semantics of this annotation are: “If this operation succeeds, then go to the next case.” The check mark is the iconic representation of success, and the empty rectangle represents the ‘go to the next case’ notion. If the check mark were replaced by an ‘x’, then the semantics would have been: “If this operation fails, then go to the next case”.



• In addition to the otherwise empty rectangle, there are four other possibilities, and the five different semantics of control annotations are shown together in the next Figure.

• It is a run-time error and a compile-time warning to have a next-case control annotation in a location where it cannot execute, for example, in the last case of a method.











Classification of Visual Programming

• Looking at the recent work reported in the literature, we see visual programming progressing in number of directions.

1) Visual Environments for different applications.

2) Visual Languages.

3) Theory of Visual Languages.


1) Visualization of Data

• Typically, the information or data is stored internally in traditional databases, but expressed in graphical form and presented to the user in a special framework.

• Users can traverse the graphical surface or zoom into it to obtain greater detail with a joystick or a pointing device.


Visualization of Data (Cont.)

• The user can use the mouse to answer many questions without the need of a keyboard.

• These systems are devoted primarily to using “direct manipulation” as a means of information retrieval, using a graphical view of a database for visualization of the information retrieved.


Example

• A visual environment can be built for the retrieval of information from a database of ships.

• Each ship will be represented by an icon.

• Icon shape, color and size represent type, status, and size of the ship.


2) Visualization of Programs

• The goal of this is to provide programmers and users with an understanding of :

– What the program can do

– How they work

– Why they work

– What are the effects


This is beneficial for

• beginners learning to program.

• professionals dealing with various aspects of programming tasks:

– Designing,

– Debugging,

– Testing, and

– Modification.


BALSA

• BALSA ( Brown Algorithm Simulator and Animator ) developed by Sedgewick and his colleages in Brown University. [ Sedgwick, 1983 ]

• It enables users :– to control the speed of an animated algorithm

– to decide which view to look at.

– to specify the input data to be processed.


• BALSA include the following animated algorithms:

• Mathematical algorithms such as– Euclid’s GCD algorithm.

– Random numbers.

– Curve fitting.

• Sorting algorithms such as– Insertion sort.

– Quick sort.

– Merge sort.

– External sort.


• Searching algorithms such as– Sequential search.

– Balanced trees.

– Hashing.

– Radix searching.

• Selecting an icon from the iconic table with a mouse causes a 10 - 15 minute dynamic simulation of the selected topic to be run, with pauses at key images, after which the user can interact with the algorithms and images.



3) Visualization of Software Design• It is very important throughout the software life

cycle to understand:– specifications

– design

– system structure

– dependencies among data and components

– etc.

• This observation, together with the success of using graphical techniques has led to the development of a visual environment for software life cycle.


The PEGASYS• The main purpose of PegaSys is to provide a

visual environment for the development and explanation of large program designs (as opposed to detailed algorithms and data structures in a program)

• PegaSys supports the specification and analysis of data and control dependencies among the components of large programs.


• A program design is described in PegaSys by a hierarchy of interrelated pictures.

• Icons denote predefined or user-defined concepts about dependencies in programs.


• The predefined primitives denote:

– objects (such as subprograms, modules, processes, and data structures),

– data dependencies (involving the declaration, manipulation, and sharing of data objects), and

– control dependencies (dealing with asynchronous and synchronous activities).

• A PegaSys user can define new concepts in terms of the primitives.


Example

• A network of 4 hosts can be represented in PegaSys as follows: –Each of the four hosts in the network is

indicated by an ellipse,

–the communication line indicated by a dashed rectangle.

–a packet of data indicated by a label on arcs.


• Dependencies among hosts, packets, and the line are described by the write relation (denoted by the letter W on arrows) and the read relation (denoted by R)

The system says that:• the broadcast network consists of four hosts that

communicate by means of a line.

• More precisely, processes named Host1,... Host4 write values of type pkt into a module called Line and read values of the same type from the Line module.



Visualization of Parallel Systems

• A number of display facilities have been associated with:

» debuggers

» performance evaluation tools

» program visualization systems


Debugger displays

Debugger displays attempt to provide some view of program state such as:

• the pattern of access to shared memory

• the interprocessor communication of a particular distributed memory architecture

• the order in which subroutines are called


Displays for Performance Visualization Tools

Performance visualization tools generally provide a graphical representation of standard metrics such as:

• Processor utilization

• Communication load


Displays for Program Visualization Systems

Program visualization systems provide highly application specific views of:

• the program's data structures

• the operations which update these data structures

• or some more abstract representation of the computation and its progress.


Program Graphs

Many systems display graphs in which:

• the vertices represent program entities and

• the arcs represent call relations or temporal orderings, similar to the graph shown in Fig. 1.


Fig. 1. An animated call graphMAIN

SELECT

PRUNE EXPAND UPDATE

PROCESS

COUNT

BROADCAST


Concurrency of the Computation

• A number of systems use a graph to represent the concurrency of the computation.

1) The concurrency map

• The concurrency map displays process histories as event streams on a time grid.


• Each column of the display represents the sequential stream of events for a single process.

• Each row of the grid represents an interval of time during which the events in different columns may occur concurrently.

• Every event in one row must occur before any event in the next row.


Communication Graphs

• A number of systems present displays that represent communication among processors through the use of boxes, arrows, and lines.

– Boxes represent processors,

– Lines represent connections. and

– Arrows represent communication events, as shown in Fig. 2.


Fig. 2: Communication Graph

P6P7

P3 P2

P4 P5

P0 P1


VPL

• Here we will show some Visual Programming Languages.


BLOX

• The basic elements in BLOX system are icons similar in shape to jigsaw puzzle pieces.

• Programs are composed by

1) Juxtaposing icons in appropriate ways, so that neighbors interlock.

2) Encapsulating substructure


Pascal-BLOX

WHILE

• The “lock and key” metaphor enforces proper syntax.


Programs in Pascal - BLOX

• It uses top-down program design.

• The code segments could be encapsulated in the appropriate tiles. This include codes corresponding to:

– the alternative branches

– loop bodies

– Boolean expressions associated with IF and WHILE statements.


ExampleBegin

S1;If not I1 Then

BeginS2;S3;

EndElse

While L1 DoBegin

If I2 Then S4Else S5;

S6;End;

While L2 Do S7;S8;

End.


Pascal - BLOX Program


Advantages

• Top-down specification of programs

• Extension of a familiar metaphors from children's toys.

• Automatic elimination of the most source of possible syntax errors in user programs, by the use of the “Lock and Key” metaphor.

• The representation can be applied to diverse and varied activities in a uniform manner.


Our VPLs

• Our efforts to design and implement visual programming languages include:

1) DataLab (with T. Lewis, and H. Kim, 1989)

2) VISO: A Visual Languages for Concurrent Programming (with S. Ali, 1994)


VISO

• VISO is a visual programming language for concurrent programming.

• It has a graphical syntax based on the language OCCAM.

• It uses a modular approach of visual programming




DataLab

• DataLab is a general-purpose system for the specification and synthesis of abstract data types (ADTs).

• DataLab uses a combination of graphical and textual forms as a specification model for ADTs.

• An imperative semantics is used to automatically generate an encapsulated executable code (Pascal modules) from such a specification.


Contribution

• DataLab extends earlier work by

1) allowing data structures to be created and displayed in their classical notation,

2) generating encapsulated imperative code (Pascal), and

3) incorporating composition of simple structures into more complex structures.


Example - Linked List• The first step is to define the interface part

unit LINKED_LIST;

interface

type

LL_list = ^LL_node;

LL_node = record

data : string;

next : LL_list;

end;

procedure LL_insert(var list : LL_list; item : string);

implementation

end.


• Next, the procedure "LL_insert" is specified by two Conditional transformations

0

node^.data := item;

list

listlist^

list^

1

2:list

node^ 4

3

9:next

8

10:list

11

1

node 5 6:node

7


0

2:list 3 1

8:next

7

list^.data := item;

list

list

list^

4

5

6:list

9

2


What is next for VPLs?

• More effective use of color and sound.

• 3-D instead of 2-D graphics.

• Animated graphics.

• Languages for concurrent and parallel programming.

Documents

1 Muhammed Al-MulhemVisual Languages Visual Programming Languages ICS 519 Part 1 ICS Department KFUPM Sept. 15, 2002