programmer's editor

8/8/2019 programmer's editor

1/34

INDIRA INSTITUTE OF

ENGINEERING & TECHNOLOGY

N0.1, VGR GARDEN,

VGR NAGAR, PANDUR-631 203

THIRUVALLUR TALUK & DISTRICT

JAVA MINI PROJECT

PROGRAMMERS EDITOR

DEPARTMENT OF COMPUTER SCIENCE AND

ENGINEERING

2008-2012


2/34

ABOUT THE E TOR

A simple text editor that demonstrates the integration of the Syntax Highlighting

package with a text editor.

THE BASICS

Syntax Highlighting works like this: you give the text to the lexer, it goes through it and gives

it back to you piece by piece and tells you what colour to make each piece.

STREAMS AND READERS

The syntax lexers accept your document through Streams and Readers. Fortunately it is very

easy to turn just about anything into a Stream or a Reader. Java comes with many prebuilt

classes for this purpose. A FileReader or a StringReader could be used. The demo uses a

custom DocumentReader.

TOKENS

The lexer returns Tokens. Tokens don't tell you the actual colour that the text should be, but

they do give you enough information to figure it out. The token contains such useful

information as the type of text, a description of the text, and the position of the text in the file.

COLOURING PARTS OF THE DOCUMENT

The entire document is coloured and it looks good in the editor. You might think that this isthe end of the story. Sadly, its not. Editors are meant to edit documents. The documents

change. The obvious approach is to re-colour the document when the text changes. This maywork for small documents, but as the document size gets larger it will quickly become

unwieldy. For a 1000 line document, it could take as much as a few seconds to colour the

entire document. Waiting a few seconds each time a character is typed does not make for a

good text editor.

The trick is that not the entire document needs to be re-coloured when something changes.

But how much really needs to be re-coloured? Not many editors do this part right. I have seen


3/34

editors that re-colour the previous three lines and the next three lines. That approach worksmost of the time, but it is pretty easy to fool.

INITIAL STATE

Every so often the syntax lexer returns to what are known as the initial state. At these times,

the lexer returns a token and continues lexing as if it were at the beginning of the documentagain. Since the lexer acts as if it were at the beginning of the document from an initial state,

the lexer could be restarted from this point without affecting the colouring of what comes

afterwords. It can be determined from the last token returned if the lexer is in the initial state

after returning that token.

So that solves half the problem. Just re-colour the document from the last initial state. If the

user is only going to append to the end of the document, this solves the problem. We can just

keep track of the last initial state and re-colour from there to the end of the document. But

what if something in the middle of the document changes? We really need to keep track ofall

initial states so that we can restart the lexer from near anywhere in the document. Then we

won't need to colour the entire rest of the document either. If the lexer returns to an initial

state at the same point that it returned to an initial state the last time, the rest of the document

is already coloured correctly.

Example

The demo keeps the list of initial states in a balanced tree. If desired the list could be included

with the Meta data of the document or stored in some other fashion. Care must be taken when

looking at initial positions after start position. If text has been added or removed from the

document, the positions after the addition/removal will have changed.

ONLY WHAT IS VISIBLE

The initial colouring time for a document may become an issue. One way around this would

be to only colour what is visible on the screen. If the user scrolls, then more of the document

will have to be coloured as the user scrolls.

Another approach that is used by the demo is to start a separate thread to do the colouring. In

this case, the document colouring happens in the background and the user may modify the

document while that happens.


4/34

FEATURES

The package supports several programming languages.

ySupports java, c, html, and java properties. (More can be added with relative ease)

y True lexer support - language syntax specified using JFlex.y Language specification compliant:

o C/C++ (Test Case) Keywords Pre-compiler statements (even those with comments) Comments Literals

Integers, Floats, and Doubles Characters and Strings (with escape sequences)

Identifiers Operands Tri-graph escape sequence support Error reporting

o Java (Test Case) Keywords Comments (including documentation comments) Literals

Integers, Floats and Doubles (with bounds checking) Characters and Strings (with escape sequences including

unicode)

true, false, and null Identifiers Operands Error reporting

o HTML (Simple Test Case) Text Character entity references Comments Scripts


5/34

Tags Error reporting

o HTML (Complex Test Case) Text Character entity references Comments Scripts Tag parts Name/value pairs Error reporting

o Java Properties (Test Case) Name/value pairs Values split across multiple lines. Comments

PACKAGES USED IN THIS EDITOR

1. Syntax

2. Syntax.Lexer


6/34

SYNTAX PACKAGE

CLASS SUMMARY

HighlightedDocument ProgrammerEditorDemo ToHTML ToHTMLAntTask

SYNTAX.LEXER PACKAGE

CLASS SUMMARY

CLexer CToken HTMLLexer HTMLLexer1 HTMLToken HTMLToken1 JavaLexer JavaScriptLexer JavaScriptToken JavaToken LatexLexer LatexToken PlainLexer PlainToken PropertiesLexer PropertiesToken SQLLexer SQLToken Token

ProgrammerEditorDemo


7/34

package com.Ostermiller.Syntax; importjava.awt.*; importjava.awt.event.*; importjavax.swing.*;

public class ProgrammerEditorDemo extends JFrame { /** The document holding the text being edited. */ private HighlightedDocument document = new

HighlightedDocument(); /** The text pane displaying the document. */ private JTextPanetextPane = new JTextPane(document); /** * Create a new Demo */ public ProgrammerEditorDemo() { // initial set up that sets the title super("Programmer's Editor Demonstration"); setLocation(50, 50); // Create a scroll pane wrapped around the text pane JScrollPanescrollPane = new JScrollP ane(textPane); scrollPane.setPreferredSize(new Dimension(620, 460)); // Add the components to the frame. JPanelcontentPane = new JPanel(new BorderLayout()); contentPane.add(scrollPane, BorderLayout.CENTER); setContentPane(contentPane); // Set up the menu bar. JMenustyleMenu = createStyleMenu(); JMenuBarmb = new JMenuBar(); mb.add(styleMenu); setJMenuBar(mb); // Make the window so that it can close the

application addWindowListener(new WindowAdapter() { public void windowClosing(WindowEvent e) { System.exit(0); }

public void windowActivated(WindowEvent e) { // focus magic textPane.requestFocus(); } }); // Put the initial text into the text pane and // set it's initial colouring style. initDocument();


8/34

// put it all together and show it. pack(); setVisible(true); } private class StyleMenuItem extends JRadioButtonMenuItem

implements ActionListener { Object style; StyleMenuItem(String name, Object style) { super(name); this.style = style; addActionListener(this); } public void actionPerformed(ActionEvent e) { document.setHighlightStyle(style); } } /** * Create the style menu. * * @return the style menu. */ private JMenucreateStyleMenu() { JRadioButtonMenuItem[] items = { new StyleMenuItem("Java",

HighlightedDocument.JAVA_STYLE), new StyleMenuItem("C/C++",

HighlightedDocument.C_STYLE),

new StyleMenuItem("HTML (Simple)",HighlightedDocument.HTML_STYLE), new StyleMenuItem("HTML (Complex)",

HighlightedDocument.HTML_KEY_STYLE), new StyleMenuItem("LaTeX",

HighlightedDocument.LATEX_STYLE), new StyleMenuItem("SQL",

HighlightedDocument.SQL_STYLE),

new StyleMenuItem("Java Properties",HighlightedDocument.PROPERTIES_STYLE),

new StyleMenuItem("Plain",HighlightedDocument.PLAIN_STYLE),

new StyleMenuItem("Grayed Out",HighlightedDocument.GRAYED_OUT_STYLE),

}; JMenu menu = new JMenu("Style"); ButtonGroup group = new ButtonGroup(); for(int i = 0; i


9/34

return menu; } /** * Initialize the document with some default text and set * they initial type of syntax highlighting.

*/ private void initDocument() { String initString = ( "/**\n" + " * Simple common test program.\n" + " */\n" + "public class HelloWorld { \n" + " public static void main(String[] args) { \n" + " // Display the greeting. \n" + " System.out.println( \"Hello World!\");\n"

+ " }\n" + "}\n" ); textPane.setText(initString); } /** * Run the demo. * * @paramargs ignored */ public static void main(String[] args) { // create the demo ProgrammerEditorDemo frame = new ProgrammerEditorDemo(); } }

HighlightedDocument

package com.Ostermiller.Syntax; importjavax.swing.text.*; import java.io.*; import java.lang.reflect.Constructor; import java.lang.reflect.InvocationTargetException; importcom.Ostermiller.Syntax.Lexer.*; /**


10/34

* A demonstrationtext

* editor that uses syntax highlighting. */ public class HighlightedDocument extends DefaultStyledDocument

{ public static final Object C_STYLE = CLexer.class; public static final Object HTML_STYLE = HTMLLexer.class; public static final Object HTML_KEY_STYLE =

HTMLLexer1.class; public static final Object JAVA_STYLE = JavaLexer.class; public static final Object JAVASCRIPT_STYLE =

JavaScriptLexer.class; public static final Object LATEX_STYLE =

LatexLexer.class; public static final Object PLAIN_STYLE =

PlainLexer.class; public static final Object PROPERTIES_STYLE =

PropertiesLexer.class; public static final Object SQL_STYLE = SQLLexer.class; public static final Object GRAYED_OUT_STYLE = new

Object(); /** * A reader wrapped around the document so that the

document can be fed into

* the lexer. */ private DocumentReaderdocumentReader; /** If non-null, all is drawn with this style (no

lexing). */ private AttributeSetglobalStyle = null; /** * The lexer that tells us what colours different words

should be.

*/ private Lexer syntaxLexer; /** * A thread that handles the actual colouring. */ private Colourercolourer; /** * A lock for modifying the document, or for actions that

depend on the

* document not being modifie d. */ private Object docLock = new Object();


11/34

/** * Create a new Demo */ public HighlightedDocument() { // Start the thread that does the colouring

colourer = new Colourer(this); colourer.start(); // create the new document. documentReader = new DocumentReader(this); syntaxLexer = new JavaLexer(documentReader); } /** * Colour or recolour the entire document */ public void colourAll() { colour(0, getLength()); } /** * Colour a section of the document. The actua l colouring

will start somewhere * before the requested position and continue as long as

needed. * * @param position * the starting point for the colouring. * @param adjustment * amount of text inserted or removed at the

starting point. */ public void colour(int position, int adjustment) { colourer.colour(position, adjustment); } public void setGlobalStyle(AttributeSet value) { globalStyle = value; colourAll(); } public void setHighlightStyle(Object value) {

if (value == HighlightedDocument.GRAYED_OUT_STYLE) {

setGlobalStyle(TokenStyles.getStyle("grayedOut")); return; } if (!(value instanceof Class)) value = HighlightedDocument.PLAIN_STYLE; Class source = (Class) value; Class[] parms = { Reader.class };


12/34

Object[] args = { documentReader }; try { Constructor cons =

source.getConstructor(parms); syntaxLexer = (Lexer) cons.newInstance(args); globalStyle = null;

colourAll(); } catch (SecurityException e) {

System.err.println("HighlightEditor.SecurityException"); } catch (NoSuchMethodException e) {

System.err.println("HighlightEditor.NoSuchMethod"); } catch (InstantiationException e) {

System.err.println("HighlightEditor.InstantiationException");

} catch (InvocationTargetException e) {

System.err.println("HighlightEditor.InvocationTargetException");

} catch (IllegalAccessException e) {

System.err.println("HighlightEditor.IllegalAccessException");

} } // // Intercept inserts and removes to colour them. //

public void insertString(int offs, String str,AttributeSet a) throws BadLocationException { synchronized (docLock) { super.insertString(offs, str, a); colour(offs, str.length()); documentReader.update(offs, str.length()); } } public void remove(int offs, intlen) throws

BadLocationException { synchronized (docLock) { super.remove(offs, len); colour(offs, -len); documentReader.update(offs, -len); } } // methods for Colourer to retrieve information DocumentReadergetDocumentReader() { return

documentReader; }


13/34

Object getDocumentLock() { return docLock; } Lexer getSyntaxLexer() { return syntaxLexer; } AttributeSetgetGlobalStyle() { return globalStyle; } }


14/34

JavaLexer

package com.Ostermiller.Syntax.Lexer; /** * A lexer should implement this interface.

*/ public interface Lexer { /** * Returns the next token. * * @return the next token */ public Token getNextToken() throws java.io.IOException ; /** * Closes the current input stream, and resets the scanner

to read from a new input stream. * All internal variables are reset, the old input stream

cannot be reused * (content of the internal buffer is discarded and

lost). * The lexical state is set to the initial state. * Subsequent tokens read from the lexer will start with

the line, char, and column * values given here. * * @param reader The new input. * @paramyylineThe line number of the first token. * @paramyycharThe position (relative to the start of the

stream) of the first token. * @paramyycolumnThe position (relative to the line) of

the first token. * @throws IOException if an IOExecption occurs while

switching readers. */ public void reset(java.io.Reader reader, intyyline,

intyychar, intyycolumn) throws java.io.IOException; }


15/34

JavaScriptToken

package com.Ostermiller.Syntax.Lexer; /** * A JavaScriptToken is a token that is returned by a lexer

that is lexing a javascript * source file. It has several attributes describing the

token: * The type of token, the text of the token, the line number

on which it * occurred, the number of characters into the input at which

it started, and * similarly, the number of characters into the input at which

it ended.
* The tokens should comply with the *

* Java Script Reference. */ public class JavaScriptToken extends Token { public final static int RESERVED_WORD_ABSTRACT = 0x101; public final static int RESERVED_WORD_BOOLEAN = 0x102; public final static int RESERVED_WO RD_BREAK = 0x103; public final static int RESERVED_WORD_BYTE = 0x104; public final static int RESERVED_WORD_CASE = 0x105; public final static int RESERVED_WORD_CATCH = 0x106; public final static int RESERVED_WORD_CHAR = 0x107; public final static int RESERVED_WORD_CLASS = 0x108; public final static int RESERVED_WORD_CONST = 0x109;

public final static int RESERVED_WORD_CONTINUE = 0x10A; public final static int RESERVED_WORD_DEFAULT = 0x10B; public final static int RESERVED_WORD_DO = 0x10C; public final static int RESERVED_WORD_DOUBLE = 0x10D; public final static int RESERVED_WORD_ELSE = 0x10E; public final static int RESERVED_WORD_EXTENDS = 0x10F; public final static int RESERVED_WORD_FALSE = 0x110; public final static int RESERVED _WORD_FINAL = 0x111; public final static int RESERVED_WORD_FINALLY = 0x112; public final static int RESERVED_WORD_FLOAT = 0x113; public final static int RESERVED_WORD_FOR = 0x114; public final static int RESERVED_WORD_FUNCTION = 0x115; public final static int RESERVED_WORD_GOTO = 0x116; public final static int RESERVED_WORD_IF = 0x117; public final static int RESERVED_WORD_IMPLEMENTS = 0x118; public final static int RESERVED_WORD_IMPORT = 0x119; public final static int RESERVED_WORD_IN = 0x11A; public final static int RESERVED_WORD_INSTANCEOF = 0x11B; public final static int RESERVED_WORD_INT = 0x11C; public final static int RESERVED_WORD_INTERFACE = 0x11D; public final static int RESERVED_WORD_LONG = 0x11E; public final static int RESERVED_WORD_NATIVE = 0x11F;


16/34

public final static int RESERVED_WORD_NEW = 0x120; public final static int RESERVED_WORD_NULL = 0x121; public final static int RESERVED_WORD_PACKAGE = 0x122; public final static int RESERVED_WORD _PRIVATE = 0x123; public final static int RESERVED_WORD_PROTECTED = 0x124; public final static int RESERVED_WORD_PUBLIC = 0x125;

public final static int RESERVED_WORD_RETURN = 0x126; public final static int RESERVED_WORD_SHORT = 0x127; public final static int RESERVED_WORD_STATIC = 0x128; public final static int RESERVED_WORD_SUPER = 0x129; public final static int RESERVED_WORD_SWITCH = 0x12A; public final static int RESERVED_WORD_SYNCHRONIZED = 0x12B; public final static int RESERVED_WORD _THIS = 0x12C; public final static int RESERVED_WORD_THROW = 0x12D; public final static int RESERVED_WORD_THROWS = 0x12E; public final static int RESERVED_WORD_TRANSIENT = 0x12F; public final static int RESERVED_WORD_TRUE = 0x130; public final static int RESERVED_WORD_TRY = 0x131; public final static int RESERVED_WORD_VAR = 0x132; public final static int RESERVED_WORD_VOID = 0x133; public final static int RESERVED_WORD_WHILE = 0x134; public final static int RESERVED_WORD_W ITH = 0x135; public final static int IDENTIFIER = 0x200; public final static int LITERAL_BOOLEAN = 0x300; public final static int LITERAL_INTEGER_DECIMAL = 0x310; public final static int LITERAL_INTEGER_OCTAL = 0x311; public final static int LITERAL_INTEGER_HEXIDECIMAL = 0x312; public final static int LITERAL_LONG_DECIMAL = 0x320; public final static int LITERAL_LONG_OCTAL = 0x321; public final static int LITERAL_LONG_HEXIDECIMAL = 0x322; public final static int LITERAL_FLOATING_POINT = 0x330; public final static int LITERAL_DOUBLE = 0x340; public final static int LITERAL_CHARACTER = 0x350; public final static int LITERAL_STRING = 0x360; public final static int LITERAL_NULL = 0x370; public final static int SEPARATOR_LPAREN = 0x 400; public final static int SEPARATOR_RPAREN = 0x401; public final static int SEPARATOR_LBRACE = 0x410; public final static int SEPARATOR_RBRACE = 0x411; public final static int SEPARATOR_LBRACKET = 0x420;

public final static int SEPARATOR_RBRACKET = 0x421; public final static int SEPARATOR_SEMICOLON = 0x430; public final static int SEPARATOR_COMMA = 0x440; public final static int SEPARATOR_PERIOD = 0x450; public final static int OPERATOR_GREATER_THAN = 0x500; public final static int OPERATOR_LESS_THAN = 0x501; public final static int OPERATOR_LESS_THAN_OR_EQUAL = 0x502;


17/34

public final static int OPERATOR_GREATER_THAN_OR_EQUAL =

0x503; public final static int OPERATOR_EQUAL = 0x504; public final static int OPERATOR _NOT_EQUAL = 0x505; public final static int OPERATOR_LOGICAL_NOT = 0x510; public final static int OPERATOR_LOGICAL_AND = 0x511;

public final static int OPERATOR_LOGICAL_OR = 0x512; public final static int OPERATOR_ADD = 0x520; public final static int OPERATOR_SUBTRACT = 0x521; public final static int OPERATOR_MULTIPLY = 0x522; public final static int OPERATOR_DIVIDE = 0x523; public final static int OPERATOR_MOD = 0x524; public final static int OPERATOR_BITWISE_COMPLIMENT = 0x530; public final static int OPERATOR_BITWISE_AND = 0x531; public final static int OPERATOR_BITWISE_OR = 0x532; public final static int OPERATOR_BITWISE_XOR = 0x533; public final static int OPERATOR_SHIFT_LEFT = 0x540; public final static int OPERATOR_SHIF T_RIGHT = 0x541; public final static int OPERATOR_SHIFT_RIGHT_UNSIGNED =

0x542; public final static int OPERATOR_ASSIGN = 0x550; public final static int OPERATOR_ADD_ASSIGN = 0x560; public final static int OPERATOR_SUBTRACT_ASSIGN = 0x561; public final static int OPERATOR_MULTIPLY_ASSIGN = 0x562; public final static int OPERATOR_DIVIDE_ASSIGN = 0x563; public final static int OPERATOR_MOD_ASSIGN = 0x564; public final static int OPERATOR_BITWISE_AND_ASSIGN = 0x571; public final static int OPERATOR_BITWISE_OR_ASSIGN = 0x572; public final static int OPERATOR_BITWISE_XOR_ASSIGN = 0x573; public final static int OPERATOR_SHIFT_LEFT_ASSIGN = 0x580; public final static int OPERATOR_SHIFT_RIGHT_ASSIGN = 0x581; public final static int OPERATOR_SHIFT_RIGHT_UNSIGNED_ASSIGN= 0x582; public final static int OPERATOR_INCREMENT = 0x590; public final static int OPERATOR_DECREMENT = 0x591; public final static int OPERATOR_QUESTION = 0x5A0; public final static int OPERATOR_COLON = 0x5A1; public final static int COMMENT_TRADITIONAL = 0xD00; public final static int COMMENT_END_OF_LINE = 0xD10; public final static int COMMENT_DOCUMENTATION = 0xD20; public final static int WHITE_SPACE = 0xE00; public final static int ERROR_IDENTI FIER = 0xF00; public final static int ERROR_UNCLOSED_STRING = 0xF10; public final static int ERROR_MALFORMED_STRING = 0xF11; public final static int ERROR_MALFORMED_UNCLOSED_STRING =

0xF12; public final static int ERROR_UNCLOSED_CHARACTER = 0xF20; public final static int ERROR_MALFORMED_CHARACTER = 0xF21; public final static int ERROR_MALFORMED_UNCLOSED_CHARACTER =

0xF22;


18/34

public final static int ERROR_INTEGER_DECIMIAL_SIZE = 0xF30; public final static int ERROR_INTEGER_OCTAL_SIZE = 0xF31; public final static int ERROR_INTEGER_HEXIDECIMAL_SIZE =

0xF32; public final static int ERROR_LONG_DECIMIAL_SIZE = 0xF33; public final static int ERROR_LONG_OCTAL_SIZE = 0xF34;

public final static int ERROR_LONG_HEXIDECIMAL_SIZE = 0xF35; public final static int ERROR_FLOAT_SIZE = 0xF36; public final static int ERROR_DOUBLE_SIZE = 0xF37; public final static int ERROR_FLOAT = 0xF38; public final static int ERROR_UNCLOSED_COMMENT = 0xF40; private int ID; private String contents; private intlineNumber; private intcharBegin; private intcharEnd; private int state; /** * Create a new token. * The constructor is typically called by the lexer * * @param ID the id number of the token * @param contents A string representing the text of the

token * @paramlineNumber the line number of the input on which

this token started * @paramcharBegin the offset into the input in characters

at which this token started * @paramcharEnd the offset into the input in characters at

which this token ended */ public JavaScriptToken(int ID, String contents,

intlineNumber, intcharBegin, intcharEnd){ this (ID, contents, lineNumber, charBegin, charEnd,

Token.UNDEFINED_STATE); } /** * Create a new token. * The constructor is typically called by the lexer * * @param ID the id number of the token * @param contents A string representing the text of the




which this token ended


19/34

* @param state the state the tokenizer is in after

returning this token. */ public JavaScriptToken(int ID, String contents,

intlineNumber, intcharBegin, intcharEnd, int state){ this.ID = ID;

this.contents = new String(contents); this.lineNumber = lineNumber; this.charBegin = charBegin; this.charEnd = charEnd; this.state = state; } /** * Get an integer representing the state the tokenizer is

in after

* returning this token. * Those who are interested in incremental tokenizing for

performance

* reasons will want to use this method to figure outwhere the tokenizer

* may be restarted. The tokenizer starts inToken.INITIAL_STATE, so

* any time that it reports that it has returned to thisstate, the

* tokenizer may be restarte d from there. */ public intgetState(){ return state; } /**

* get the ID number of this token * * @return the id number of the token */ public intgetID(){ return ID; } /** * get the contents of this token * * @return A string representing the text of the token */ public String getContents(){ return (new String(contents)); } /** * get the line number of the input on which this token

started *


20/34

* @return the line number of the input on which this token

started */ public intgetLineNumber(){ return lineNumber; } /** * get the offset into the input in characters at which this

token started * * @return the offset into the input in characters at which

this token started */ public intgetCharBegin(){ return charBegin; } /** * get the offset into the input in characters at which this

token ended * * @return the offset into the input in characters at which

this token ended */ public intgetCharEnd(){ return charEnd; } /** * Checks this token to see if it is a reserved word. * Reserved words are explained in Java * Language Specification. * * @return true if this token is a reserved word, false

otherwise */ public booleanisReservedWord(){ return((ID >> 8) == 0x1); } /** * Checks this token to see if it is an identifier. * Identifiers are explained in Java * Language Specification. * * @return true if this token is an identifier, false

otherwise */ public booleanisIdentifier(){ return((ID >> 8) == 0x2);


21/34

} /** * Checks this token to see if it is a literal. * Literals are explained in Java

* Language Specification. * * @return true if this token is a literal, false otherwise */ public booleanisLiteral(){ return((ID >> 8) == 0x3); } /** * Checks this token to see if it is a Separator. * Separators are explained in Java * Language Specification. * * @return true if this token is a Separator, false

otherwise */ public booleanisSeparator(){ return((ID >> 8) == 0x4); } /** * Checks this token to see if it is a Operator. * Operators are explained in Java * Language Specification. * * @return true if this token is a Operator, false otherwise */ public booleanisOperator(){ return((ID >> 8) == 0x5); } /** * Checks this token to see if it is a comment. * * @return true if this token is a comment, false otherwise */ public booleanisComment(){ return((ID >> 8) == 0xD); } /** * Checks this token to see if it is White Space. * Usually tabs, line breaks, form feed, spaces, etc. *


22/34

* @return true if this token is White Space, false

otherwise */ public booleanisWhiteSpace(){ return((ID >> 8) == 0xE); } /** * Checks this token to see if it is an Error. * Unfinished comments, numbers that are too big, unclosed

strings, etc.

* * @return true if this token is an Error, false otherwise */ public booleanisError(){ return((ID >> 8) == 0xF); } /** * A description of this token. The descri ption should * be appropriate for syntax highlighting. For example * "comment" is returned for a comment. * * @return a description of this token. */ public String getDescription(){ if (isReservedWord()){ return("reservedWord"); } else if (isIdentifier()){ return("identifier"); } else if (isLiteral()){ return("literal"); } else if (isSeparator()){ return("separator"); } else if (isOperator()){ return("operator"); } else if (isComment()){ return("comment"); } else if (isWhiteSpace()){ return("whitespace"); } else if (isError()){ return("error"); } else {

return("unknown"); } } /** * get a String that explains the error, if this token is an

error. * * @return a String that explains the error, if this token

is an error, null otherwise.


23/34

*/ public String errorString(){ String s; if (isError()){ s = "Error on line " + lineNumber + ": "; switch (ID){

case ERROR_IDENTIFIER: s += "Unrecognized Identifier: " + contents; break; case ERROR_UNCLOSED_STRING: s += "'\"' expected after " + contents; break; case ERROR_MALFORMED_STRING: case ERROR_MALFORMED_UNCLOSED_STRING: s += "Illegal character in " + contents; break; case ERROR_UNCLOSED_CHARACTER: s += "\"'\" expected after " + contents; break; case ERROR_MALFORMED_CHARACTER: case ERROR_MALFORMED_UNCLOSED_CHARACTER: s += "Illegal character in " + contents; break; case ERROR_INTEGER_DECIMIAL_SIZE: case ERROR_INTEGER_OCTAL_SIZE: case ERROR_FLOAT: s += "Illegal character in " + contents; break; case ERROR_INTEGER_HEXIDECIMAL_SIZE: case ERROR_LONG_DECIMIAL_SIZE: case ERROR_LONG_OCTAL_SIZE: case ERROR_LONG_HEXIDECIMAL_SIZE: case ERROR_FLOAT_SIZE: case ERROR_DOUBLE_SIZE: s += "Literal out of bounds: " + contents; break; case ERROR_UNCLOSED_COMMENT: s += "*/ expected after " + contents; break; } } else { s = null;

} return (s); } /** * get a representation of this token as a human readable

string. * The format of this string is subject to change and should

only be used

* for debugging purposes.


24/34

* * @return a string representation of this token */ public String toString() { return ("Token #" + Integer.toHexString(ID) + ": " +

getDescription() + " Line " +

lineNumber + " from " +charBegin + " to " + charEnd + " :" + contents); } }


25/34

PlainToken

package com.Ostermiller.Syntax.Lexer; /** * A PlainToken is a token that is returned by a lexer that is

lexing a plain * text file. It has several attributes describing the token: * The type of token, the text of the token, the line number

on which it * occurred, the number of characters into the input at which

it started, and * similarly, the number of characters into the input at which

it ended. */ public class PlainToken extends Token { public final static int TEXT = 0x200; public final static int WHITE_SPACE = 0xE00; private int ID; private String contents; private intlineNumber; private intcharBegin; private intcharEnd; private int state; /** * Create a new token. * The constructor is typically called by the lexer * * @param ID the id number of the token * @param contents A string representing the text of the




which this token ended */ public PlainToken(int ID, String contents, intlineNumber,

intcharBegin, intcharEnd){ this (ID, contents, lineNumber, charBegin, charEnd,

Token.UNDEFINED_STATE); } /** * Create a new token. * The constructor is typically called by the lexer * * @param ID the id number of the token


26/34

* @param contents A string representing the text of the



at which this token started

* @paramcharEnd the offset into the input in characters atwhich this token ended * @param state the state the tokenizer is in after

returning this token. */ public PlainToken(int ID, String contents, intlineNumber,

intcharBegin, intcharEnd, int state){ this.ID = ID; this.contents = new String(contents); this.lineNumber = lineNumber; this.charBegin = charBegin; this.charEnd = charEnd; this.state = state; } /** * Get an integer representing the state the tokenizer is in

after

* returning this token. * Those who are interested in incremental tokenizing for

performance * reasons will want to use this method to figure out where

the tokenizer * may be restarted. The tokenizer starts in

Token.INITIAL_STATE, so * any time that it reports that it has returned to this

state, the * tokenizer may be restarted from there. */ public intgetState(){ return state; } /** * get the ID number of this token * * @return the id number of the token */ public intgetID(){ return ID; } /** * get the contents of this token * * @return A string representing the text of the token */


27/34

public String getContents(){ return (new String(contents)); } /** * get the line number of the input on which this token

started * * @return the line number of the input on which this token

started */ public intgetLineNumber(){ return lineNumber; } /** * get the offset into the input in character s at which this

token started * * @return the offset into the input in characters at which

this token started */ public intgetCharBegin(){ return charBegin; } /** * get the offset into the input in characters at which this

token ended * * @return the offset into the input in characters at which

this token ended */ public intgetCharEnd(){ return charEnd; } /** * Checks this token to see if it is text. * * @return true if this token is an text, false otherwise */ public booleanisText(){ return((ID >> 8) == 0x2); } /** * Checks this token to see if it is White Space. * Usually tabs, line breaks, form feed, spaces, etc. * * @return true if this token is White Space, false

otherwise */


28/34

public booleanisWhiteSpace(){ return((ID >> 8) == 0xE); } /** * Checks this token to see if it is an Error.

* Unfinished comments, numbers that are too big, unclosedstrings, etc. * * @return true if this token is an Error, false otherwise */ public booleanisError(){ return false; } /** * Checks this token to see if it is a comment. * * @return true if this token is a comment, false otherwise */ public booleanisComment(){ return false; } /** * A description of this token. The description should * be appropriate for syntax highlighting. For example * "comment" is returned for a comment. * * @return a description of this token. */ public String getDescription(){ if (isText()){ return("text"); } else if (isWhiteSpace()){ return("whitespace"); } else { return("unknown"); } } /**

* get a String that explains the error, if this token is anerror. * * @return a String that explains the error, if this token

is an error, null otherwise. */ public String errorString(){ return ""; }


29/34

/** * get a representation of this token as a human readable

string. * The format of this string is subject to change and should

only be used * for debugging purposes.

* * @return a string representation of this token */ public String toString() { return ("Token #" + Integer.toHexString(ID) + ": " +

getDescription() + " Line " + lineNumber + " from " +charBegin + " to " + charEnd + " :

" + contents); } }


30/34

WORKING OF PROGRAMMERS EDITOR

STEP 1: DOUBL CL CKTHE SYNTAX.JAR FILE.


31/34

STEP 2:CLICKON THE STYLEBUTTON WHICH WILL O EN A DRO DOWN

WINDOW.BY DEFAULT JA A LANGUAGE IS SELECTED


32/34

STEP 3: SELECT ANY ONE OF THE LANGUAGES. For example, select

HTML SIMPLE).


33/34

STEP 4: NOW, TYPE THE CODE.YOU WILL FIND THE SYNTAX IN COLOUR.


34/34

CONCLUSION ABOUTTHEEDITOR

The PROGRAMMERS EDITOR can be enhanced further with some

additional features and functionalities.

In addition to the features like syntax highlighting,colouring selection of

languages. We can include the following features

y MANIP LATION OF CONTENT(CUT,COPY,PASTE,SAVE,..)y WE CAN EMPOWER WITH ADDITIONALLANGUAGESy ERROR HIGHLIGHTINGy WE CAN PROVIDE THE HELP FACILITYy PARTITON OF WINDOW

Documents

programmer's editor