How to Present your Data on Internet? A guide for beginners Martin Molhanec, M.Sc., Ph.D

How to Present your Data on Internet?

A guide for beginners

Martin Molhanec, M.Sc., Ph.D.

Abstract

How to get data. How to save data. How to manipulate with data. How to present data on Internet.

How to get data

Directly

MeasuringApparatusUser

PROBLEM:How to communicate with measuring apparatus?

Usually low level communication!

How to get data

Indirectly

MeasuringApparatus

Fileor

Database

User

Two steps:1) Save data with the use of supplier program.2) Present data saved in file or database.

How to get data

Directly

Indirectly

MeasuringApparatus

Fileor

Database

User

Two ways!

How to get data directly

We must wrote a program! We must understand the API! Program language?

Visual Basic – Microsoft C, C++ - Microsoft, Borland, etc. Pascal (Delphi) – Borland Assembler – Microsoft, GNU


APIApplicatio

nProgramInterface

How to get data directly

We must wrote a program Program language

Visual Basic – Microsoft C, C++ - Microsoft, Borland, etc. Pascal (Delphi) – Borland Assembler – Microsoft, GNU


APIApplicationProgramInterface

Not easy task!Very good programming

practice!

Structure of program

Initialization HW configuration and setup



While not end Get data from measuring apparatus Put data to database (or write to file)

End while



While not end Get data from measuring apparatus Put data to database (or write to file)

End while

Finishing Closing of files, disconnection from

database

Problems

Low level API Assembler

Complicated communication Handshake

Unknown programming language You must to learn

Bad documentation As usual

Close API No information

Low level API Assembler

Complicated communication Handshake

Unknown programming language You must to learn

Bad documentation As usual

Close API No information

Problems

Not possible to advice!Too many ways!

How to get data indirectly

1. phase: put data to database (or to file)

Suppliers program 2. phase:

show data on Internet Our program

MeasuringApparatus

Fileor

Database

User

Formatof

data

We must know!

Format of data

CSV Comma Separated Values

Easy to understand Simple

XML eXtensible Markup Language

Complicated Contemporary standard Many tools to work with it

The CSV File Format

Each record is one line ...butA record separator may consist of a line feed (ASCII/LF=0x0A), or a carriage return and line feed pair (ASCII/CRLF=0x0D 0x0A)....but: fields may contain embedded line-breaks so a record may span more than one line.

Fields are separated with commas.Example John,Doe,120 any st.,"Anytown, WW",08123

Leading and trailing space-characters adjacent to comma field separators are ignored. So John , Doe ,resolves to "John" and "Doe"

Space characters can be spaces, or tabs.

The CSV File Format

Fields with embedded commas must be delimited with double-quote characters.

In the above example. "Anytown, WW" had to be delimited in

double quotes because it had an embedded comma. Fields that contain double quote characters

must be surrounded by double-quotes, and the embedded double-quotes must each be represented by a pair of consecutive double quotes. So,

John "Da Man" Doewould convert to

"John ""Da Man"" Doe"

The CSV File Format

A field that contains embedded line-breaks must be surounded by double-quotesSo: Field 1: Conference room 1 Field 2: John, Please bring the M. Mathers file for review -J.L. Field 3: 10/18/2002 ... would convert to: " Conference room 1" ,"John, Please bring the M. Mathers file for review -J.L.", 10/18/2002

Note that this is a single CSV record, even though it takes up more than one line in the CSV file. This works because the line breaks are embedded inside the double quotes of the field.

The CSV File Format

Fields with leading or trailing spaces must be delimited with double-quote characters.So to preserve the leading and trailing spaces around the last name above: John ," Doe ",...

Fields may always be delimited with double quotes.The delimiters will always be discarded.

The first record in a CSV file may be a header record containing column (field) namesThere is no mechanism for automatically discerning if the first record is a header row, so in the general case, this will have to be provided by an outside process (such as prompting the user). The header row is encoded just like any other CSV record in accordance with the rules above. A header row for the multi-line example above, might be: Location, Notes, "Start Date", ...

The CSV File Format

Example Data: Here is a small set of records that demonstrate some of the constructs discussed above.

John,Doe,120 jefferson st.,Riverside, NJ, 08075

Jack,McGinnis,220 hobo Av.,Phila, PA,09119

"John ""Da Man""",Repici,120 Jefferson St.,Riverside, NJ,08075

Stephen,Tyler,"7452 Terrace ""At the Plaza"" road",SomeTown,SD, 91234

,Blankman,,SomeTown, SD, 00298

"Joan ""the bone"", Anne",Jet,"9th, at Terrace plc",Desert City,CO,00123

Example 1.

How to read CSV file format in Visual Basic.

Then open your CSV file for input as you would do for any other txt file. Read a line and store it in a string, like OneLine. Then do this:

data = Split(OneLine,",")

The Split commands takes the string, cuts it ups according to what delimiter you chose (in this case comma) and stores the pieces in the variable as an array. So now data has become an array and contains following:

OneLine contains "12/04/06,John Doe,36,176"

data(0) will contain 12/04/06 data(1) will contain John Doedata(2) will contain 36 etc.

Example 1.


Dim delim as String = "," 'define a delimiterDim record as String ' record storageDim recsplit() as string ' splited record storage 'open the file....'while not end of the file

record = stringReader.ReadLine() 'read a record recsplit() = Split(record, delim) 'split the record 'make your calculations..... 'each position of recsplit can be accesed normally

‘While end

Example 1.


Using MyReader As New Microsoft.VisualBasic.FileIO.TextFieldParser("tm.CSV")

MyReader.TextFieldType = FileIO.FieldType.DelimitedMyReader.SetDelimiters(",")

Dim currentRow As String()

While Not MyReader.EndOfDatacurrentRow = MyReader.ReadFields()

End While

End Using Will be explained in more details!

Example 1.


Create a new TextFieldParser. The following code creates the TextFieldParser named MyReader and opens the file test.txt.

Using MyReader As New _ Microsoft.VisualBasic.FileIO.TextFieldParser _ ("C:\TestFolder\test.txt")

Example 1.


Define the TextField type and delimiter. The following code defines the TextFieldType property as Delimited and the delimiter as ",".

MyReader.TextFieldType = FileIO.FieldType.Delimited MyReader.SetDelimiters(",")

Example 1.

How to read CSV file format in Visual Basic.Loop through the fields in the file. If any lines are corrupt, report an error and

continue parsing. The following code loops through the file, displaying each field in turn and reporting any fields that are formatted incorrectly.

Dim currentRow As String()While Not MyReader.EndOfData

TrycurrentRow = MyReader.ReadFields() ‘!!! Here is CVS reading !!!Dim currentField As StringFor Each currentField In currentRow

MsgBox(currentField)NextCatch ex As Microsoft.VisualBasic.FileIO.MalformedLineExceptionMsgBox("Line " & ex.Message & "is not valid and will be skipped.")

End Try

Example 1.


Close the While and Using blocks with End While and End Using.

End WhileEnd Using

Example 2.

How to read CSV file format in PHP.<?php

// reads a csv file and returns a two-dimensional array of lines/fields function read_csv($file,$delimiter) {

$data_array = file($file); // read whole file to array for ( $i = 0; $i < count($data_array); $i++ ) // process rows {

$parts_array[$i] = explode($delimiter,$data_array[$i]); } return $parts_array;

}

?>

Example 2.

How to read CSV file format in PHP.// this willl display all records in the csv file $data = read_csv('read_csv.txt','|'); for ( $i = 0; $i < count($data); $i++ ) {

for ( $u = 0; $u < count($data[$i]); $u++ ) {

echo $data[$i][$u].' '; if($data[$i][$u] == end($data[$i])) { echo '<br>'; }

} } echo '<p>';

The XML file format

Features of XML XML provides a text-based means to describe

and apply a tree-based structure to information.

At its base level, all information manifests as text, interspersed with markup that indicates the information's separation into a hierarchy of character data, container-like elements, and attributes of those elements.

In this respect, it is similar to the LISP programming language's S-expressions, which describe tree structures wherein each node may have its own property list.

The XML - Quick syntax tour

The basic syntax for one element in XML is

<name attribute="value">content</name>

TAG STAR

T

ATTRIBUTE

VALUE

CONTENT

TAG END

ATTRIBUTE


Here is an example of a simple recipe expressed using XML: <?xml version="1.0" encoding="UTF-8"?> <recipe name="bread" prep_time="5 mins" cook_time="3

hours"> <title>Basic bread</title> <ingredient amount="3" unit="cups">Flour</ingredient> <ingredient amount="0.25" unit="ounce">Yeast</ingredient>

<ingredient amount="1.5" unit="cups“ state="warm">Water</ingredient>

<ingredient amount="1" unit="teaspoon">Salt</ingredient> <instructions>

<step>Mix all ingredients together, and knead thoroughly.</step> <step>Cover with a cloth, and leave for one hour in warm room.</step> <step>Knead again, place in a tin, and then bake in the oven.</step>

</instructions> </recipe>

Too complex example?








The first line is the XML declaration: it is an optional line stating what version of XML is in use (normally

version 1.0), and may also contain information about character encoding and external dependencies.








The remainder of this document consists of nested elements, some of which have attributes and

content. An element typically consists of two tags, a start tag and an end tag, possibly surrounding text

and other elements.


Here is an example of a simple recipe expressed using XML:

<title>Basic bread</title> <ingredient amount="3" unit="cups">Flour</ingredient>



and other elements.






and other elements.






and other elements.






and other elements.



<instructions>

<step>Mix all ingredients together, and knead thoroughly.</step> <step>Cover with a cloth, and leave for one hour in warm

room.</step> <step>Knead again, place in a tin, and then bake in the

oven.</step> </instructions>



and other elements.

Outer element

Inner element


XML provides special syntax for representing an element with empty content. Instead of writing a start tag followed immediately by an end tag, a document may contain the empty element tag where a slash follows the element name. The following two examples are functionally equivalent:

<foo></foo> <foo />


Correctness in an XML document For an XML document to be correct, it must be:

Well-formed. A well-formed document conforms to all of XML's syntax rules. For example, if a non-empty element has an opening tag with no closing tag, it is not well-formed. A document that is not well-formed is not considered to be XML; a parser is required to refuse to process it.

Valid. A valid document has data that conforms to a particular set of user-defined content rules, or XML schemas, that describe correct data values and locations. For example, if an element in a document is required to contain text that can be interpreted as being an integer numeric value, and it instead has the text "hello", is empty, or has other elements in its content, then the document is not valid.

How to read the XML file format

Processing XML files SAX and DOM are object oriented programming APIs widely used to

process XML data. The first XML parsers exposed the contents of XML documents to applications as SAX events or DOM objects.

SAX is a lexical, event-driven interface in which a document is read serially and its contents are reported as "callbacks" to various methods on a handler object of the user's design. SAX is fast and efficient to implement, but difficult to use for extracting information at random from the XML, since it tends to burden the application author with keeping track of what part of the document is being processed. It is better suited to situations in which certain types of information are always handled the same way, no matter where they occur in the document.

DOM is an interface-oriented API that allows for navigation of the entire document as if it were a tree of "Node" objects representing the document's contents. A DOM document can be created by a parser, or can be generated manually by users (with limitations). Data types in DOM Nodes are abstract; implementations provide their own programming language-specific bindings. DOM implementations tend to be memory intensive, as they generally require the entire document to be loaded into memory and constructed as a tree of objects before access is allowed.

Where to save data

In text file CSV XML

Easy way, but not good for next processing!

Where to save data

In text file CSV XML

In database Relational database Object oriented database

Better way for next processing!

Which database?

Commercional MS SQL Server MS Access Oracle …

Free MySQL Postgress …

Example 3.

We show, how to: Managing MS Access. Configuring ODBC. Programming in Visual Basic.

Example 3.

<% ' Old ASP syntax.

Dim MyConn

Set MyConn = Server.CreateObject("ADODB.Connection")

' New ASP.NET syntax.

Dim MyConn MyConn = Server.CreateObject("ADODB.Connection")

%>

Example 3.

<%

Dim RS As RecordSet

' Old ASP syntax (retrieving recordset column value).Set MyConn = Server.CreateObject("ADODB.Connection")MyConn.Open("TestDB")Set RS = MyConn.Execute("Select * from Products")Response.Write RS("Name")

' New ASP.NET syntax (retrieving recordset column value).MyConn = Server.CreateObject("ADODB.Connection") MyConn.Open("TestDB")RS = MyConn.Execute("Select * from Products")Response.Write RS("Name").Value

%>

Example 4.

We show, how to: Managing MySQL. Configuring PHP. Programming in PHP.

Example 4.

<?php// Connecting, selecting database$link = mysql_connect('mysql_host', 'mysql_user', 'mysql_password') or die('Could not connect: ' . mysql_error());echo 'Connected successfully';mysql_select_db('my_database') or die('Could not select database');

How to manipulate with data?

SQL Structured Query Language.

Standard language for Defining Manipulation Selecting

data in relational databases.

SQL

CREATE TABLE Command for creating a database table

CREATE TABLE test (

Intenzity INTEGER PRIMARY KEY,

Name VARCHAR(10)

);

Table name

SQL


CREATE TABLE test (


Name VARCHAR(10)

);

Columns names

SQL


CREATE TABLE test (


Name VARCHAR(10)

);

Columns types

SQL


CREATE TABLE test (


Name VARCHAR(10)

); Indicate main unique column

SQL

INSERT INTO Is used to insert a new data to table

INSERT INTO test VALUES (120, 'Andrew');

INSERT INTO test VALUES (200, 'Gordon');

intenzity name

SQL

SELECT … FROM … WHERE Is used to get data from database table!!!

SELECT intenzity, name FROM test WHERE intenzity > 150;

What I want

SQL

SELECT FROM WHERE Is used to get data from database table!!!

SELECT intenzity, name FROM test WHERE intenzity > 150;

From which table

SQL

SELECT FROM WHERE Is used to get data from database table!!!

SELECT intenzity, name FROM test WHERE intenzity > 150; Condition

Example 5.

We show, how to: Write a program in Visual Basic which get data

from MS Access database through the use of ODBC.

Example 3 ! Remember!

Example 5.

<%

Dim RS As RecordSet

' Old ASP syntax (retrieving recordset column value).Set MyConn = Server.CreateObject("ADODB.Connection")MyConn.Open("TestDB")Set RS = MyConn.Execute("Select * from Products")Response.Write RS("Name")

' New ASP.NET syntax (retrieving recordset column value).MyConn = Server.CreateObject("ADODB.Connection") MyConn.Open("TestDB")RS = MyConn.Execute("Select * from Products")Response.Write RS("Name").Value

%>

Example 6.

We show, how to: Write a program in PHP which get data from

MySQL database. Example 4! Remember!

Example 4.

<?php// Connecting, selecting database$link = mysql_connect('mysql_host', 'mysql_user', 'mysql_password') or die('Could not connect: ' . mysql_error());echo 'Connected successfully';mysql_select_db('my_database') or die('Could not select database');

Example 4.

// Performing SQL query$query = 'SELECT * FROM my_table';$result = mysql_query($query) or die('Query failed: ' . mysql_error());

Example 4.

// Printing results in HTMLecho "<table>\n";while ($line = mysql_fetch_array($result, MYSQL_ASSOC)) { echo "\t<tr>\n"; foreach ($line as $col_value) { echo "\t\t<td>$col_value</td>\n"; } echo "\t</tr>\n";}echo "</table>\n";

Example 4.

// Free resultsetmysql_free_result($result);

// Closing connectionmysql_close($link);?>

Data presentation

HTML HyperText Markup Language.

Standard language for writing web pages. Textual. Graphic. Scripting.

HTML – basic elements

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">

<html>

<head> <title> replace with your document's title </title>

</head>

<body> replace with your document's content

</body>

</html>

Too complex example?



<html>

<head> <title> replace with your document's title </title>

</head>


</body>

</html>

Standard header



<html>

<head> <title>replace with your document's title </title>

</head>


</body>

</html>

HTMLdocument

hastwo

parts:

1] HEAD

2] BODY

HTML – table elements

<h4>Two rows and three columns:</h4><table border="1"> <tr> <td>100</td> <td>200</td> <td>300</td> </tr> <tr> <td>400</td> <td>500</td> <td>600</td> </tr></table>

Dynamic pages

.ASP Microsoft Visual Basic or Javascript

.PHP Free PHP language

We need dynamic pages because we want to get data

from database (or text file) and put that data onto the web.

It needs a little bit programming!

Resume

We show how to present our data from database to Internet.

We get basic knowledges about: File text formats: CSV and XML Databases: MS Access and MySQL HTML ASP and PHP Thank you for

your attention!

Documents

How to Present your Data on Internet? A guide for beginners Martin Molhanec, M.Sc., Ph.D