Pracowania Programowania

JSON & XML

Co-author: Tomasz Ziętkiewicz*

Attention! The code for this classes can be found on the branch JSONandXMLStart in the https://github.com/WitMar/PRA2018-2019 repository. The ending code in the branch JSONandXMLEnd.

JSON

JSON (JavaScript Object Notation) http://www.json.org/ is a lightweight, text-based data exchange format.

It is based on a subset of JavaScript.

JSON is widely used to store and transmit structured data in text form its features are:

it is human-readable
it is widely distributed - you can find libraries for almost any language (list at http://www.json.org)

Example JSON

{
    "artist": "Pink Floyd",
    "title": "Dark Side of the moon",
    "year": 1973,
    "tracks": [
        {
            "track#": 1,
            "title": "Speak to Me/Breathe",
            "length": "3:57",
            "music": ["Mason"]
        },
        {
            "track#": 2,
            "title": "On the run",
            "length": "3:50",
            "music": ["Waters", "Gilmour"]
        }
    ]
}

Everyday life example: API rowerów miejskich

JSON Syntax

In JSON we have two main data structures:

object (object, dictionary, map)

set of key-value pairs

{
"title": "Dark Side of the Moon",
"year": 1973,
"tracks#": 9
}

array (table, list)

an ordered set of values

{
    "name":"John",
    "age":30,
    "cars":[ "Ford", "BMW", "Fiat" ]
}

Seven types of values

XML

XML (Extensible Markup Language) - markup language, which, like JSON, enables serialization and exchange of structural data in text form.

XML syntax

In an XML document, we can separate content and markup (special characters).

The tags are between "<" and ">" or "&" and ";".

The content of the document is all characters that are not markers.

Tagi

element beginning tag:

<album>

element end tag:

</album>

empty tag:

<album />

Element

The element starts with the start tag, ends with the end tag, or is an empty tag.

Between the tags there is the content of the element, which can be either plain text or nested elements.

The start and empty tags can contain attributes, or key-value pairs.

We write the name of the key without quotes and values between quotes.

<track number="3" title="Time" length="3:57">
    Ticking away the moments that make up a dull day
    You fritter and waste the hours in an offhand way
    Kicking around on a piece of ground in your home town
    Waiting for someone or something to show you the way
</track>

Comments

Comments are placed between the "<! -" and "->" tags.

Example of an XML

<?xml version="1.0" encoding="UTF-8"?>
<album title="Dark Side of the Moon" year="1973">
    <track number="1" title="Speak to Me/Breathe">
        Breathe, breathe in the air
        Don't be afraid to care
        Leave but don't leave me
        Look around and choose your own ground
        For long you live and high you fly
        Smiles you'll give and tears you'll cry
        And all you touch and all you see
        Is all your life will ever be
    </track>
    <track number="2" title="On the run" />
    <track number="3" title="Time" length="3:57">
        Ticking away the moments that make up a dull day
        You fritter and waste the hours in an offhand way
        Kicking around on a piece of ground in your home town
        Waiting for someone or something to show you the way
    </track>
</album>

Everyday life example: API rowerów miejskich, XML

Serialization / Deserialization

Serialization - a process involving the transformation of data structures or the state of an object into a sequential form that allows saving or transferring this data and potentially reconstructing data structures or objects at a later time / by another process / computer (deserialization).

For example, serialization may consist in saving objects generated by our program to a JSON file for later loading of these objects back into the program in order to continue calculations.

JSON and XML are examples of formats well suited for data serialization in a human-readable way. You can also serialize data in a binary form, which is incomprehensible to humans.

Jackson

Jackson - zestaw narzędzi do przetwarzania danych dla Javy ("suite of data-processing tools for Java").

Głównym komponentem jest generator/parser JSON, pozwalający m.in. na deserializację/serializację do/z JSON z/do Javy.

Posiada liczne moduły dodające obsługę innych formatów danych, m.in. XML, YAML czy CSV.

Strona domowa projektu nie działa, ale projekt jest aktywnie rozwijany na GitHub. Zarchiwizowana wersja strony domowej.

Jackson

Jackson - a set of data processing tools for Java ("suite of data-processing tools for Java").

The main component is the JSON generator/parser, allowing, among others, to deserialize/ serialize to and from JSON in Java.

It has numerous modules that add support for other data formats, including XML, YAML or CSV.

The home page of the project does not work, but the project is actively developed on GitHub.

Archived version of the home page.

Task 1: Serialization to JSON by Jackson (already done in repo)

Add the Jackson library dependencies to pom.xml

Hint: You can find appriopriate dependency on the website repozytorium GitHub modułu.

Create two employees (objects of the Employee class). Let one employee be the superior and the other his subordinate.

Give an address to each employee.

Use om.fasterxml.jackson.databind.ObjectMapper to save (serialize) the supervisor object to the json file.

Example:

https://www.mkyong.com/java/how-to-convert-java-object-to-from-json-jackson/

http://www.baeldung.com/jackson-object-mapper-tutorial

Task 2: De-serialization with JSON (already done in repo)

Restore the supervisor from the JSON file created in task 1. Change salary to your manager and save (serialize) to the json file.

Task 3: Annotations

In Java, the notation of lowerCamelCase is used to write class field names.

In JSON there is no accepted standard of notation (discussion on StackOverflow).

By default, the fields in JSON generated by Jackson have the same names as the fields in the class we serialize.

Add annotations to the modeled classes that change the name of the salary to "money" and annotations to ignore the "pesel" field in serialization.

Hint: Find proper tags in documentation dokumentacji.

Task 4: Deserialization of the generic types

Create a json file containing a list of several employees.

Load this list into the ArrayList <Employee> collection.

Hint: Please use the 3 minute tutorial.

Jackson XML

Jackson has a module extending it to support XML serialization.

To use XML instead of JSON, just change the "ObjectMapper" to XmlMapper ":

ObjectMapper xmlMapper = new XmlMapper ();

More info:

http://www.baeldung.com/jackson-xml-serialization-and-deserialization

Task 5: XML

Using an existing JacksonSerialization class, modify it or create a new class to allow serialization / deserialization to / from XML format.

Add the xml files to the root/resources directory corresponding to existing json files.

Hint: You do not have to create the contents of xml files yourself, you can generate them using the appropriate methods.

Hint: Remember to add a library to pom.xml. The appropriate entry can be found in the XML module GitHub repository.

Task 6: Joda Time

Add a field

DateTime birthDate

to the Employee class containing the date of birth of the employee.

Add annotations on it

@JsonFormat(shape = JsonFormat.Shape.STRING, pattern = "yyyy-MM-dd HH:mm:ss.SSSZ")

Try to serialize and then deserialize the object of the modified class.

Hint: You may need the jackson-datatype-ioda module.

You need to register the Joda module in the mapper.

https://stackoverflow.com/questions/29523169/register-jodamodule-in-jax-rs-application

Task 7: Recursive references

Note that in JSON, entire objects with dependencies are printed. What would happen if employee X had a subordinate Y whose subordinate would be X again? We would get infinite recursion and error in serialization. To avoid this, we can use the annotation

Uncomment 65 line in ModeObjectCreator.java

@JsonIdentityInfo(generator=ObjectIdGenerators.IntSequenceGenerator.class,
        property="refId", scope=Employee.class)
public class Employee { ... }

That will cause objects to be printed only once, and multiple references will use Id as a reference.

Lambda Expressions

Traditionally in Java we use the iterators explicitly specified in the code to navigate the collections:

List <String> names = new ArrayList <>();
for (Student student: students) {
    if (student.getName ().startsWith("A")) {
        names.add (student.getName ());
    }
}

Using streams, we use the so-called internal iterations, that is, the logic of iteration after elements is hidden for us and we focus on their processing.

List <string> names = students.stream()
    .map (student :: getName)
    .filter (Name-> name.startsWith("A"))
    .collect (Collectors.toList());

Operations on streams

The operation on the streams are divided into intermediate and final (terminal).

Intermediate operations return the stream as a result of the action. In the above example, such operations are map ans filter. Terminal operations terminate processing, usually they aggregate results, count values, or do nothing (eg. foreach operation may be terminal), in the above example such operation is collect.

Special streams have been created for numerical operations IntStream, DoubleStream, and LongStream.

IntStream.rangeClosed(1, 10).forEach(num -> System.out.print(num));
// ->12345678910
IntStream.range(1, 10).forEach(num -> System.out.print(num));
// ->123456789

Structure construction

Stream of constants and array elements:

Stream.of("This", "is", "Java8", "Stream").forEach(System.out::println);

String[] stringArray = new String[]{"Streams", "can", "be", "created", "from", "arrays"};

stringArray.forEach(System.out::println);

Stream methods

Map

It changes the processed element to something else, i.e. it takes an element of one type and returns another type of element. In practice, it is usually used to extract some attributes from objects.

students.stream()
        .map(Student::getName)
        .forEach(System.out::println);

The Student::getName is a short reference to the method from the class. So on a student object that comes from stream, we call the getName() method.

It can be written also as:

students.stream()
        .map(student -> student.getName())
        .forEach(System.out::println);

Filter

Selects elements from the stream relative to a given condition

students.stream()
                .filter(student -> student.getScore() >= 60)
                .collect(Collectors.toList());

Distinct

Removes repetitions from collection

students.stream()
                .map(Student::getName)
                .distinct()
                .collect(Collectors.toList());

Limit

Limits the number of elements in the stream to the given number.

Sorted

Sorts elements in a natural order.

students.stream()
                .map(Student::getName)
                .sorted()
                .collect(Collectors.toList());

With the help of the Comparator class, you can define sorts of different types

//Sorting names if the Students in descending order
students.stream()
                .map(Student::getName)
                .sorted(Comparator.reverseOrder())
                .collect(Collectors.toList());
//Sorting names if the Students in descending order
students.stream()
                .map(Student::getName)
                .sorted(Comparator.reverseOrder())
                .collect(Collectors.toList());
//Sorting students by First Name and Last Name both
students.stream()
                .sorted(Comparator.comparing(Student::getFirstName).
                                thenComparing(Student::getLastName))
                .map(Student::getName)
                .collect(Collectors.toList());
//Sorting students by First Name Descending and Last Name Ascending
students.stream()
                .sorted(Comparator.comparing(Student::getFirstName)
                                .reversed()
                                .thenComparing(Student::getLastName))
                .map(Student::getName)
                .collect(Collectors.toList());

FlatMap

It works like a map, only as a result it returns a stream of processed items.

List<Integer> numbers = Arrays.asList(1, 2, 3, 4);
List<List<Integer>> mapped =
                                numbers.stream()
                                .map(number -> Arrays.asList(number -1, number, number +1))
                                .collect(Collectors.toList());

System.out.println(mapped); //:> [[0, 1, 2], [1, 2, 3], [2, 3, 4], [3, 4, 5]]

List<Integer> flattened =
                                numbers.stream()
                                .flatMap(number -> Arrays.asList(number -1, number, number +1).stream())
                                .collect(Collectors.toList());

System.out.println(flattened);  //:> [0, 1, 2, 1, 2, 3, 2, 3, 4, 3, 4, 5]

Terminals

Match

Match is used when we are interested in appearing in the stream of an element with given properties.

//Check if at least one student has got distinction
Boolean hasStudentWithDistinction = students.stream()
                .anyMatch(student -> student.getScore() > 80);

//Check if All of the students have distinction
Boolean hasAllStudentsWithDistinction = students.stream()
                .allMatch(student -> student.getScore() > 80);

//Return true if None of the students are over distinction
Boolean hasAllStudentsBelowDistinction = students.stream()
                .noneMatch(student -> student.getScore() > 80);

Find

Allows you to find an object in the stream with given properties, we can choose any matching object or the first one matched.

//Returns any student that matches to the given condition
students.stream().filter(student -> student.getAge() > 20)
                        .findAny();
//Returns first student that matches to the given condition
students.stream().filter(student -> student.getAge() > 20)
                        .findFirst();

Reduce

It allows you to reduce the data stream to a single value.

//Summing without passing an identity
Optional<integer> sum = numbers.stream()
                .reduce((x, y) -> x + y);
//Product without passing an identity
Optional<integer> product = numbers.stream()
                .reduce((x, y) -> x * y);

Collect

Collects stream elements in the list.

students.stream()
    .filter(student -> student.getScore() >= 60)
            .collect(Collectors.toList());

Hint: Collect return a new object! New list, map, set etc. But the values are the same as in the processed collection.

RemoveIf

Removing elements from list base on filter condition.

students.removeIf(student -> student.getScore() >= 60)

Recommended! At home, try to complete tasks from the repository:

https://github.com/vfarcic/java-8-exercises