Week 7 - Data Formats

Data formats

xml

json

JavaScript Object Notation

lightweight, text-based open standard

designed for human-readable data interchange

derived from the JavaScript

for representing simple data structures and associative arrays

despite its relationship to JavaScript it is language-independent

with parsers available for many languages

csv

comma-separated values

stores tabular data (numbers and text)

plain-text form

data from spreadsheets etc

JavaScript Object Notation

JavaScript Object Notation data format, or JSON

derived from the literals of the JavaScript programming language.

JSON a subset of the JavaScript language.

As a subset, JSON does not possess any additional features that the JavaScript language itself does not already possess.

Although JSON is a subset of a programming language,

it itself is not a programming language

it is a data interchange format.

JSON is an increasingly popular standard,

it can be used as the data format wherever the exchange of data occurs.

JSON

json is built on two structures:

collections of name/value pairs (objects)

an object begins with {and ends with }

each name is followed by :

name/value pairs are separated by , (comma).

ordered lists of values (arrays)

an array begins with [ and ends with ]

values are separated by , (comma).

{ 
	"firstName": "John", 
	"lastName" : "Smith", 
	"age" : 25, 
	"address" : 
	{ 
		"streetAddress": "21 2nd Street", 
		"city" : "New York", "state" : "NY", 
		"postalCode" : "10021" 
	}, 
	"phoneNumber": 
	[ 
		{ 
			"type" : "home", 
			"number": "212 555-1234" 
		},
		{ 
			"type" : "fax",
			 "number": "646 555-4567" 
		} 
	]
 }

object that describes a person

string fields for first name and last name

a number field for age

contains an object representing the person's address

and contains a list (an array) of phone number objects.

chartData = [
    { x : "1", y: "19021"},
    { x : "1", y: "12837"},
    { x : "1", y: "12378"},
    { x : "1", y: "21882"},
    { x : "1", y: "17654"},
    { x : "1", y: "15833"},
    { x : "1", y: "16122"}
];

{items: [
   { name: 'Adobo', aisle: 'Mexican', price: 3.01},
   { name: 'Balsamic vinegar', aisle: 'Condiments', price: 4.01},
   { name: 'Basil', aisle: 'Spices', price: 3.59},
   { name: 'Bay leaf', aisle: 'Spices', price: 2.01},
   { name: 'Beef Bouillon Granules', aisle: 'soup', price: 5.01},
   { name: 'Vinegar', aisle: 'Condiments', price: 1.99},
   { name: 'White cooking wine', aisle: 'Condiments', price: 2.01},
   { name: 'Worcestershire Sauce', aisle: 'Condiments', price: 3.99},
   { name: 'Pepper', aisle: 'Spices', price: 1.01}
]}

Keys in JSON

keys in a JSON object have to have double quotes around them in JavaScript, all of the following are valid
```
var jack = { "age": 20 }; 
var jack = { 'age': 20 }; 
var jack = { age: 20 };
```
However, only the first line would be valid in JSON.

Values in JSON

Values in JSON can be of the following types:

String

Number

Array

Boolean true/false

null

Items in arrays can be any of these types.

Strings need to be double quoted, but the rest don’t
```
{ "num":2, "decimal":2.5, "boolean":true, "array":[ 1, 2, 3, true, null ] }
```
key value pairs need a comma after them, unless it’s the last pair in the object

JSON basic types

Number

type is not specified in specification

but in practice double precision floating-point format, as this is how JavaScript in Web browsers treats it

String

double-quote

UTF-8 by default

Boolean

true or false

null

empty

Non-significant white space may be added freely around the "structural characters" (i.e. the brackets "[{]}", colon ":" and comma ",").

🔗 http://www.json.org/

🔗 http://www.w3schools.com/json/json_syntax.asp
Ajax with jQuery

jQuery comes with jQuery.ajax(),

a powerful method to handle Ajax requests

makes it very easy to work with external json files
```
$(document).ready(function() {
    $.ajax({
      "url": “externalJson.js",
      "type": "get",
      "dataType": "json"
    });
 });
 
```
Ajax calls are asynchronous

Ajax calls are asynchronous.

the run in the background

do not stop the rest of thecode from executing.

when making an Ajax request

the code immediately after the Ajax request code is run, regardless of whether or not the Ajax call has finished.

can’t just return data within the function, because the Ajax method will take an arbitrary amount of time to respond and it run while other code is executing.

use callback functions which will only run once the data is returned
```
var req = $.ajax({
   "url": "/someUrl" 
});

 req.done(function() {
    //do something 
});
```
the done function takes three arguements

.done(response, status, jqXHR):

response is the response from the server; typically, the JSON the server has responded to.

status is a string denoting the status it is nearly always “success”.

jqXHR returns the jqXHR object.

Cross server issues

. $ajax() works well for files stored on the same server

however if you try to load files from external sourse you can find that the data does not load

an example error message could be

“XMLHttpRequest cannot load http://api.dribbble.com/shots/626625. Origin http://localhost:8000 is not allowed by Access-Control-Allow-Origin.”

Be aware that not all browsers will explain why the Ajax request failed, but will only inform you that it did.

Chrome’s console. or Firebug are really useful to show what is happening

The problem is due to security.

By default, the browser won’t allow one domain to make an Ajax call to a URL on another domain to pull in data, because this could be potentially dangerous.
JSONP

Mozilla Developer Network explains (🔗 https://developer.mozilla.org/en/docs/HTTP_access_control)

Cross-site HTTP requests initiated from within scripts have been subject to well-known restrictions, for well-understood security reasons. For example, HTTP Requests made using the XMLHttpRequest object were subject to the same-origin policy. In particular, this meant that a web application using XMLHttpRequest could only make HTTP requests to the domain it was loaded from, and not to other domains.

this is not practical when it comes to using third-party APIs

many workarounds exist.

One of the newer workarounds is Cross-Origin Resource Sharing (CORS), which allows the server to include a header in its response, stating that a cross-origin request is valid.

Another solution, the one commonly used today, is to use JSONP, which stands for JSON Padded.

JSONP works by putting the URL of the data you want into a script tag and getting the server’s response as a JavaScript script, thus allowing you to get at the data

It wraps the response from the server in a function.

For example, the server might respond with: someFunction(data);

data is the JSON data.

That is then included as regular JavaScript in your page

it is then possible to write a function to process the data.

jQuery does automatically

using jsonp with jQuery
```
$(document).ready(function(){
    var req = $.ajax({
        url: "http://api.lmiforall.org.uk/api/v1/soc/search?q=computer", dataType: "jsonp"
});

req.done(function(data) {
    console.log(data);
    }); 
});
```
displayAjax.html
```
<!DOCTYPE html>
<html>
  <head>
  <title>Exercise</title>
  <script src="jquery-1.12.1.min.js"></script>
  <script src="app1.js"></script>
  <link rel="stylesheet" href="styles.css">
  </head>
  <body>
  </body>
</html>
```
app1.js
```
$(function() {
  var req = $.ajax({
  url: "http://api.lmiforall.org.uk/api/v1/soc/search?q=computer",
  dataType: "jsonp"
  });
  req.done(function(data) {
  console.log(data);
  });
});
```

xml

<?xml version="1.0" encoding="utf-8"?>
<foods>
   <food>
    <name>Adobo </name>
    <aisle>Mexican </aisle>
    <price>3.01 </price>
   </food>
   <food>
    <name>Balsamic vinegar </name>
    <aisle>condiments </aisle>
    <price>4.01 </price>
   </food>
   <food>
    <name>Basil </name>
    <aisle>Spices </aisle>
    <price>3.59 </price>
   </food>
   <food>
    <name>Bay leaf </name>
    <aisle>Spices </aisle>
    <price>2.01 </price>
   </food>
   <food>
    <name>Beef Bouillon Granules </name>
    <aisle>Soup </aisle>
    <price>5.01 </price>
   </food>
   <food>
    <name>Vinegar </name>
    <aisle>Condiment </aisle>
    <price>1.99 </price>
   </food>
   <food>
    <name>White cooking wine </name>
    <aisle>Condiments </aisle>
    <price>2.01 </price>
   </food>
   <food>
    <name>Worcestershire Sauce </name>
    <aisle>Condiments </aisle>
    <price>3.99 </price>
   </food>
   <food>
    <name>Pepper </name>
    <aisle>Spices </aisle>
    <price>1.01 </price>
   </food>
</foods>

XML (eXtensible Markup Language)

Simplified version of SGML

enabling users to define their own language

XML is not a tag set but a meta language

Tags not concerned with how to render data, but instead define content

(HTML) <b> indicates that text is bold

(XML) <employee> indicates that data is about employees

Definition is either

implicit (deduced from document structure)

or is explicit (defined in Document Type Definition or schema)

Some uses of xml

xhtml

rss feeds

written in xml

metadata

describes resources

used be search engines

often written in RDF XML

databases

templates

layout for content management systems

WAP and WML

markup languages for handheld devices

Benefits of XML

Openness

XML is a W3C standard

endorsed by software industry market leaders.

Extensibility

There is no fixed set of tags.

New tags can be created as they are needed.

Self-description

XML documents can be stored without definitions, because they contain meta data in the form of tags and attributes.

XML provides a basis for author identification and versioning at the element level.

Benefits of XML

Contains machine- readable context information

Tags, attributes and element structure provide context information that can be used to interpret the meaning of content, opening up new possibilities for highly efficient search engines, intelligent data mining, agents, etc.

Separates content from presentation

XML tags describe meaning not presentation.

The look and feel of an XML document can be controlled by XSL style sheets, allowing the look of a document (or of a complete Web site) to be changed without touching the content of the document.

Multiple views or presentations of the same content are easily rendered.

Supports multilingual documents and Unicode

This is important for the internationalization of applications.

Facilitates the comparison and aggregation of data

The tree structure of XML documents allows documents to be compared and aggregated efficiently element by element.

Can embed multiple data types

XML documents can contain any possible data type

from multimedia data (image, sound, video)

to active components (Java applets, ActiveX).

Can embed existing data

Mapping existing data structures like file systems or relational databases to XML is straightforward.

Provides a 'one-server view' for distributed data

XML documents can consist of nested elements that are distributed over multiple remote servers.

XML is currently the most sophisticated format for distributed data - the World Wide Web can be seen as one huge XML database.

Rapid adoption by industry

Software AG, IBM, Sun, Microsoft, Netscape, DataChannel, SAP and many others have already announced support for XML.

Microsoft use XML as the exchange format for its Office product line,

while both Microsoft's and Mozzila's Web browsers support XML.

SAP has announced support of XML through the SAP Business Connector with R/3.

Software AG supports XML in its Bolero and Natural product lines and provides Tamino, a native XML database.

students.html

Specifies presentation

<html>
<head></head>
<body>
<h2> Student List</h2>
<ul>
  <li> 200506789 </li>
  <li> Adam </li>
  <li> adam@unl.ac.uk</li>
  <li> yes - final </li>
</ul>
<ul>
  <li> 200706791 </li>
  <li> Adrian </li>
  <li> adrian@gcu.ac.uk</li>
  <li> no </li>
</ul>
<ul>
  <li> 200506739 </li>
  <li> Brian </li>
  <li>brian@gcu.ac.uk</li>
  <li> no </li>
</ul>
</body>
</html>

students.xml

Specifies data structure

<?xml-stylesheet href=“example.xsl" type="text/css"?>
<studentList>
   <student>
     <id>200506789</id>
      <name>Adam</name>
      <email>adam@unl.ac.uk</email>
      <bsc level="final">yes</bsc>
 </student>
 <student>
      <id>200706791</id>
      <name>Adrian</name>
      <email>adrian@gcu.ac.uk</email>
      <bsc level="one">no</bsc>
 </student>
 <student>
     <id>200506739</id>
     <name>Brian</name>
     <email> brian@gcu.ac.uk </email>
     <bsc level="final">yes</bsc>
 </student>
</studentList>