2. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | JSON Pointer and JSON Patch Updates to Java API for JSON Processing Kin-man Chung Oracle Corporation September 30, 2014

3. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. 3

4. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Program Agenda JSON-P API Overview JSON Pointer JSON Patch JSON Editing/Transformation JSON Query with Java SE 8 Big JSON data 1 2 3 4 5 4 6

5. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Program Agenda JSON-P Overview JSON Pointer JSON Patch JSON Editing/Transformation JSON Query with Java SE 8 Big JSON data 1 2 3 4 5 5 6

6. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | JSON Object Model Overview An immutable representation of a JSON value Represented by: – JsonObject – JsonArray – JsonNumber – JsonString – Literals TRUE, FALSE, and NULL 6 JsonValue

7. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | JSON Object Model Overview Represents a JSON object java.util.Map Created with – JsonObjectBuilder – JsonObjectReader 7 JsonObject

8. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | JSON Object Model Overview Represents a JSON array java.util.List Created with – JsonArrayBuilder – JsonArrayReader 8 JsonArray

9. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | JSON Object Model Overview Factory class for creating JSON data Plugin factories with service provider API includes – createObjectBuilder() – createArrayBuilder() 9 Json

10. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |10 [ { "name": "Duke", "gender": "M", "phone": { "areacode": "650", "number": "234-5678"} }, { "name": "Jane", ”gender": "F", "phone": { "areacode": "777", "number": "999-5555"} }, { "name": "Amy", "gender": "F", "phone": { "areacode": "505" "number": "333-4444"} } ] Example: contacts in JSON

11. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |11 JsonArray contacts = Json.createArrayBuilder().add(Json.createObjectBuilder().add("name", "Duke").add("gender", "M").add("phone", Json.createObjectBuilder().add("areacode", "650").add("number", "234-5678"))).add(Json.createObjectBuilder().add("name", "Jane").add("gender", "F").add("phone", Json.createObjectBuilder().add("areacode", ”777").add("number": "999-5555"))).add(Json.createObjectBuilder().add("name", “Amy").add("gender", "F").add("phone", Json.createObjectBuilder().add("areacode", "505").add("number", "333-4444"))).build(); Example: contacts in JSON object model

12. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |12 JsonArray contacts = Json.createArrayBuilder().add(Json.createObjectBuilder().add("name", "Duke").add("gender", "M").add("phone", Json.createObjectBuilder().add("areacode", "650").add("number", "234-5678"))).add(Json.createObjectBuilder().add("name", "Jane").add("gender", "F").add("phone", Json.createObjectBuilder().add("areacode", ”777").add("number": "999-5555"))).add(Json.createObjectBuilder().add("name", “Amy").add("gender", "F").add("phone", Json.createObjectBuilder().add("areacode", "505").add("number", "333-4444"))).build(); Example: contacts in JSON object model

13. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |13 JsonArray contacts = Json.createArrayBuilder().add(Json.createObjectBuilder().add("name", "Duke").add("gender", "M").add("phone", Json.createObjectBuilder().add("areacode", "650").add("number", "234-5678"))).add(Json.createObjectBuilder().add("name", "Jane").add("gender", "F").add("phone", Json.createObjectBuilder().add("areacode", ”777").add("number": "999-5555"))).add(Json.createObjectBuilder().add("name", “Amy").add("gender", "F").add("phone", Json.createObjectBuilder().add("areacode", "505").add("number", "333-4444"))).build(); Example: contacts in JSON object model

14. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | JSON Streaming Model Overview Produces stream of parser events Parser events – START_OBJECT, END_OBJECT, START_ARRAY, END_ARRAY – KEY_NAME, VALUE_STRING, VALUE_NUMBER – VALUE_TRUE, VALUE_FALSE, VALUE_NULL Low level Pull – Application controls accessing and advancing parser events Suitable for processing big JSON data 14 JsonParser

15. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |15 JsonArrayBuilder arrayBuilder; JsonParser parser = Json.createParser(...); while (parser.hasNext()) { JsonParser.Event event = parser.next(); switch(event) { case START_ARRAY: arrayBuilder = Json.createArrayBuilder(); break; case VALUE_STRING: arrayBuilder.add(parser.getString()); break; case END_ARRAY: // do something with the array process(arrayBuilder.build()); break; case START_OBJECT:... } JSON Streaming API Example: create a JsonArray

16. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Program Agenda JSON-P API Overview JSON Pointer JSON Patch JSON Editing/Transformation JSON Query with Java SE 8 Big JSON data 1 2 3 4 5 16 6

17. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | JSON Pointer Specified in RFC 6901 A string syntax for referencing a JSON value Example – /0/phone/number refers to the phone number of first person in the contacts 17

18. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | JSON Pointer 18 Proposed API JsonArray contacts = Json.createArrayBuilder().add(…).build(); // Create a JsonPointer JsonPointer p = Json.createPointer("/0/phone/number"); // Get the value at the referenced location in the target JsonValue v = p.getValue(contacts); // Replace a value at the referenced location, returning a new array // with the change JsonArray result = p.replace(contacts, "123-4567");

19. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | JSON Pointer getValue – Get the value at the referenced location add – Add/insert a value at the referenced location replace – Replace a value at the referenced location remove – Remove a value at the referenced location 19 Methods in JsonPointer

20. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | JSON Pointer An operation is applied to a JsonArray or JsonObject Operations do not modify the target JsonArray or JsonObject Operation add, replace, or remove returns a new JsonArray or JsonObject containing the result – Transforms the target into the result 20 Operations in JsonPointer

21. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Program Agenda JSON-P API Overview JSON Pointer JSON Patch JSON Editing/Transformation JSON Query with Java SE 8 Big JSON data 1 2 3 4 5 21 6

22. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | JSON Patch Specified in RFC 6902 Sequence of operations for modifying a JSON document – Operations specified in a JSON array Suitable for use in http PATCH method 22

23. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | JSON Patch [ {"op":"replace", "path":"/1/phone", "value": { "areacode": "111", "number": "222-3333"} } {"from":"/1/phone", "path": "/2", "op":"copy"} ] 23 Example

24. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | JSON Patch A JSON patch is an array of operations An patch operation is a JSON object Each operation must have an “op” field with a value of – “add”, “replace”, “remove”, “move”, “copy”, or “test” Each operation must have a “path” field – A JSON Pointer specifying the target location Other fields depending on “op” 24 Operations

25. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | JSON Patch // Create the target and the patch JsonArray target = Json.createArrayBuilder().add(…).build(); JsonArray patch = Json.createArrayBuilder()….build(); // Create JsonPatch from patch JsonPatch jsonpatch = Json.createPatch(patch); // Apply the patch to the target and return the result JsonArray result = jsonpatch.apply(target); 25 Proposed API

26. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Program Agenda JSON-P API Overview JSON Pointer JSON Patch JSON Editing/Transformation JSON Query with Java SE 8 Big JSON data 1 2 3 4 5 26 6

27. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | JSON Editing/Transformation A JSON Patch transforms a JSON target to a JSON result Propose adding capability to edit a JsonArray or JsonObject Use builder pattern: – Create builders with initial JsonArray or JsonObject – Add to ObjectBuilder remove(name) – Add to ArrayBuilder add(index, value), set(index, value), remove(index) – Builder returns immutable JsonArray or JsonObject from build() 27

28. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | JSON Editing // Create the target JsonArray target = Json.createArrayBuilder().add(…).build(); // Create a builder initialized with the target JsonArrayBuilder builder = Json.createArrayBuilder(target); // Creates a new object and insert it into the array JsonObject john = Json.createObjectBuilder()….build(); JsonArray result = builder.add(1, john).build(); 28 Proposed API

29. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Program Agenda JSON=P API Overview JSON Pointer JSON Patch JSON Editing/Transformation JSON Query with Java SE 8 Big JSON data 1 2 3 4 5 29 6

30. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | JSON Query with Java SE 8 A JsonObject is a Map, and a JsonArray is a List, so queries can be implemented with Java’s stream operations, using Lambda expressions Get concurrent processing for free Example: Output names of contacts whose gender is F 30 JsonArray contacts; contacts.getValuesAs(JsonObject.class).stream().filter(x->"F".equals(x.getString("gender"))).map(x->x.getString("name")).forEach(System.out::println);

31. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | JSON Query with Java SE 8 Example: Collect query results in a List 31 JsonArray contacts; List names = contacts.getValuesAs(JsonObject.class).stream().filter(x->"F".equals(x.getString("gender"))).map(x->x.getString(“name”)).collect(Collectors.toList());

32. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | JSON Query with Java SE 8 Java Collectors return Lists or Maps. We need collectors that return JsonArrays or JsonObjects 32 Problem

33. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | JSON Query with Java SE 8 New helper class JsonCollectors that construct Collectors for JSON objects or arrays toJsonArray: – Accumulates values in a JsonArray toJsonObject: – Accumulates values in a JsonObject groupBy – Implements “group by” operations on the values 33 Proposed API

34. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | JSON Query with Java SE 8 Collect the query result in a JsonArray 34 Example JsonArray contacts; JsonArray names = contacts.getValuesAs(JsonObject.class).stream().filter(x->"F".equals(x.getString("gender"))).map(x->x.getString("name")).collect(JsonCollectors.toJsonArray());

35. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | JSON Query with Java SE 8 Collect the query result in a JsonObject 35 Example JsonArray contacts; JsonObject nameToPhones = contacts.getValuesAs(JsonObject.class).stream().filter(x->"F".equals(x.getString("gender"))).collect(JsonCollectors.toJsonObject( x->x.getString("name"), x->x.getJsonObject("phone"));

36. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | JSON Query with Java SE 8 Group the contacts by gender 36 Example JsonArray contacts; JsonObject groups = contacts.getValuesAs(JsonObject.class).stream().collect(JsonCollectors.groupBy(x->getString("gender")));

37. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | JSON Query with Java SE 8 Change the phone areacodes from “415” to “650” 37 Example: Putting it all together Static int index = -1; JsonArray patch = contacts.getValuesAs(JsonObject.class).stream().peek(x->index++).filter(x->x.getObject("phone").getString("areacode").equals("415")).map(Json.createObjectBuilder().add("op", "replace").add("path", "/"+index+"/phone/areacode").add("value", “650”).build()).collect(JsonCollectors.toJsonArray()); JsonArray result = Json.createPatch(patch).apply(contacts);

38. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Program Agenda JSON-P API Overview JSON Pointer JSON Patch JSON Editing/Transformation JSON Query with Java SE 8 Big JSON data 1 2 3 4 5 38 6

39. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Support for big JSON data Big JSON data – Cannot fit in memory – Generated dynamically – Potentially infinite We already have a stream model, suitable for processing big data – Add API to make it more useful 39

40. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Support for big JSON data Creating partial JSON data in JsonParser – Create an object at START_OBJECT – Create an array at START_ARRAY Skipping values in JsonParser – Skip enclosing array – Skip enclosing object – Skip long string 40

41. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Support for big JSON data Creating java.util.stream in JsonParser – For processing or querying of the created object or array – Create stream when creating objects or arrays Lazy evaluation, as needed – Get optimization hints from the operation Filter or short-circuit operations trigger skipping – Seamless integration with JsonParser event stream 41

42. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Summary Support for JSON Pointer and JSON Patch Add editing/transformations to JsonObject and JsonArray Add helper classes/methods for JSON query Add support for big JSON data WARNING: The API changes are just proposals, are not final 42

43. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Status Proposed JSR to be submitted – Watch https://www.jcp.org for announcementhttps://www.jcp.org Roadmap – Expert Group formed: 2014 – Public Review Draft: 2015 – Proposed Final Draft: 2015 – Final Release: 2016 43

44. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Reference Info Questions and comments – users@json-processing-spec.java.net users@json-processing-spec.java.net Interested to join Expert Group – Nominate yourself at new JSR page (when approved) Personal contact – kinman.chung@oracle.com 44

45. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Q & A 45

46. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |46