spec

Software for Diffraction

encode()/decode()

data stream manipulation

DESCRIPTION

The encode() and decode() built-in functions convert between spec data objects and standard data-interchange formats. Such a capability may be useful, for example, to pass data over sockets between spec and other processes. Currently, JSON (JavaScript Object Notation) and Base64 (RFC 2045) formats are supported.

BUILT-IN FUNCTIONS

In the encode() and decode() functions, the first argument is the name of the format. Currently, "json" and "base64" are recognized formats. The argument is case insensitive.

encode(format, obj [, ...])
Returns a string representation of the spec data object or objects in the specified format.
decode(format, str [, arr])
If two arguments, returns a spec data object obtained from the string representation in format. If three arguments, the result will be returned in the array arr and the function will return 0 or -1, to indicate success or failure. The type of array for arr (associative or data) depends on the decoding format. JSON requires an associative array. Base64 requires a data array.

JSON FORMAT

The JSON format is documented at json.org. A JSON value can be a string, a number, an object, an array or the words true, false or null. Arbitrary white space is allowed between the various elements.

Strings are always delimited by double quotes. Within strings, octal string escape sequences are not allowed. Instead, \u followed by four-hex-digits is used. Since spec only supports eight-bit characters, the two high-order hex digits are ignored when decoding such sequences.

Numbers can only be in decimal format (no octal or hexadecimal), including exponential format. Leading zeroes are not allowed.

A JSON object is a comma-delimited list of name/value pairs (separated by a colon), enclosed in curly brackets. The name is always a string.

The array object is a comma-delimited list of values enclosed in square brackets.

spec encodes one- and two-dimensional associative arrays as JSON objects. One- and two-dimensional data arrays are encoded as JSON arrays. If multiple spec objects are included as arguments to encode() spec encodes them as a JSON array.

The strings ".true.", ".false." and ".null." are encoded as the JSON booleans true, false and null.

Since spec does not have a composite data type, the decode() function for JSON is limited on what it can return. There are two version of the decode() function. Both take a string argument which contains the encoded data. The first version returns a spec data object appropriate to the data. Only one type of thing can be returned, such as a number, a string, a one- or two-dimensional associative array, a data array or a value corresponding to true (1), false (0) or null (""). It is an error if the string argument to decode() has more than one kind of value.

The second version of decode() takes an additional associative array argument and allows any valid JSON string as input argument. The decoded values are placed in the passed array and the function returns 0 for success and -1 on a failure to decode.

If the input string is a JSON object, a one-dimensional associative array is created indexed by the object name. The object value is converted to a string. If the value is a JSON object or array, it can be decoded on a subsequent call to decode(). The first character of the returned object can be used as a check on whether further decoding is needed. If the first character is { the string is a JSON object. If the first character is [ the string is a JSON array.

When decoding JSON arrays with the three-argument version of decode, the associative array is indexed by consecutive integers starting at zero, and the element values are always converted to strings. JSON values true, false and null are returned as strings with values ".true.", ".false." and ".null.".

EXAMPLES

The following examples illustrate JSON encoding.

spec associative arrays are encoded as a JSON object, which is a set of name/value pairs, enclosed in curly brackets. The name is always a string. spec's encoded objects will only include string and number value types. To encode a spec associative array such as the A[] motor position array:

1.SPEC> s = encode("json", A)

2.SPEC> print s
{"0": 120, "1": 60, "2": -35.2645, "3": -45}

spec number-valued data arrays can contain signed or unsigned 8-, 16- or 32-bit integers, 32-bit floats or 64-bit doubles and can have one or two dimensions. To encode a data array:

3.SPEC> ulong array data[10], data[2][5]

4.SPEC> array_op("fill", data)

5.SPEC> s = encode("json", data)

6.SPEC> print s
[0,1,2,3,4,5,6,7,8,9]

7.SPEC> array_op("fill", data2)

8.SPEC> print encode("json", data2)
[[0,0,0,0,0],[1,1,1,1,1],[2,2,2,2,2]]

A spec string data array is similar to an 8-bit array with respect to storage, but the contents will be encoded as JSON strings. A one-dimensional string array will be encoded as a single string. A two-dimensional string array will be encoded as one-dimensional array of strings:

9.SPEC> string array sdata[20], sdata2[3][20]

10.SPEC> sdata = "\033[7mtest\033[0m"

11.SPEC> sdata2[0] = "first"

12.SPEC> sdata2[1] = "middle"

13.SPEC> sdata2[2] = "last"

14.SPEC> print encode("json", sdata)
"\u001B[7mtest\u001B[0m"

15.SPEC> print encode("json", sdata2)
["first","middle","last"]

Note the octal escapes in the first example have been encoded using \u followed by four hexadecimal digits.

A list of mixed data objects can also be encoded:

16.SPEC> s = encode("json", PI, "hello world", data, A)

17.SPEC> print s
[3.14159265358979, "hello world", [0,1,2,3,4,5,6,7,8,9],
{"0": 120, "1": 60, "2": -35.2645, "3": -45}]

Here, the list is encoded as an array of values consisting of a number, a string, an array and an object.

The decode() function with two arguments will only accept JSON encoded strings that can be decoded to one of the recognized spec data types. Mixed data types, such as combinations of scalers and arrays in the last example above, require the three-argument version of decode():

18.SPEC> global result[]

19.SPEC> decode("json", s, result)

20.SPEC> print result
result["0"] = 3.14159265358979
result["1"] = "hello world"
result["2"] = "[0,1,2,3,4,5,6,7,8,9]"
result["3"] = "{"0": 120, "1": 60, "2": -35.2645, "3": -45}"

21.SPEC> p decode("json", result[2])
{0, 1, 2, 3, 4, 5, 6, 7, 8, 9}

22.SPEC> p decode("json", result[3])
tmp["0"] = 120
tmp["1"] = 60
tmp["2"] = -35.2645
tmp["3"] = -45

In command 21, the two-argument form of decode() returns a data array, while command 22 decodes the JSON object in the last element of result[] and returns an associative array.

BASE64 FORMAT

Base64 is an encoding scheme for arbitrary text or, more usefully, binary data that uses an ASCII character set where each ASCII character contains 6 bits of the original data. Thus, four bytes of an encoded Base64 string represent 24 bits of data. The character set for Base64 are the letters A-Z and a-z, the digits 0-9 and the special symbols + and /. The = symbol is used for padding. The carriage return and newline characters can be used for readability when encoding and are ignored when decoding. spec encodes to a 76 character line length.

The spec encode() function for Base64 encodes only data arrays or strings. Only one object at a time can be encoded. The function returns the resulting Base64 string.

With two arguments, fmt and str, the decode() function will return the decoded string as a string, which may be unprintable, depending on the nature of the encoded object. When called with a data array as the optional third argument, the decoded object will be placed in the array. One generally would need to know what type and size of object the str argument represents in order to allocate a data array of the appropriate type and size.