Midi Device Schema Specifications

This document proposes a solution for describing the midi System Exclusive implementation of any midi capable device.

 

Document status: draft 0.1

 

Content:

Abstract. 1

Overview.. 1

Requirements. 2

Schema. 3

Declarations 4

The Type System. 8

Record Base Types 10

 

Abstract

The goal of the Midi Device Schema is to unambiguously describe the midi capabilities of a midi enabled hardware (or virtual) device focusing on the System Exclusive implementation of that device. This description can then be interpreted by software and used to provide structured System Exclusive parsing and compiling of Midi messages. Such software would relieve an (midi) application developer from having to understand and implement all the details of the devices he or she wishes to support in their software.

 

Over the years the System Exclusive part of the midi protocol has been proven to be a pain. Manufacturers implement their System Exclusive messages in what is in their view the best possible way (best meaning cheap in most cases). Because the System Exclusive messages has no predefined structure overall (other than start and markers), they need to be handled specifically and separately in the application code of midi software. This leads to application developers having to write specialized code for each device and not being able to support all devices. If, as a user, your midi device was not on the supported devices list, you could not use the software.

 

The Midi Device Schema aims to change that. In order to tackle this problem, the proposed solution uses metadata to describe the Midi System Exclusive implementation of a specific midi device in a schema.

 

Overview

Although this specification only deals with the textual Midi Device Schema declarations I like to give you a baseline-architecture to position the explanation of schema content in.

 

The following figure displays this architecture using a receive scenario.

The Data Object Model component is instantiated for one specific device for which to encipher the midi data (both send and receive). The Midi Device Schema (file or otherwise) is located and parsed into the Schema Object Model. This object model is a read-only, in-memory representation of the Midi Device Schema. The Midi Device Schema is expressed using the Midi Type System and optional custom types. The Data Object Model component will create the necessary Data Converters for this particular Midi Device Schema. Next the application code routes the incoming Midi Data Stream through the Data Object Model which in turn call into the Data Converters component to convert the physical midi data into a logical in-memory representation that is passed to an Application Component for further processing in the application.

 

Figure 1: Basic architecture

 

Sending logical midi data from the application to the midi device would require the application to pass this data into the Data Object Model which in turn will call into the Data Converter component to transform the data into physical midi data according to the Midi Device Schema.

 

All behavior of transforming midi data from physical to logical and visa versa is driven by the declarations in the Midi Device Schema.

 

Requirements

The requirements laid out for the Midi Device Specification are as follows:

 

1.x

Global Requirements

2.x

Transformation Requirements

3.x

Extensibility Requirements

 

[TODO: bring requirements up to date]

 

#

Requirement

Solution

1.0

All oddities of packaging 8-bit data in 7-bit Midi System Exclusive must be abstracted from the developer.

Logical midi data is semantically equal to the physical midi data but presents the data in a natural way to the application using “native” data types.

1.1

The physical-to-logical and visa versa transformation of data should be extensible.

A factory pattern is used to create the data converters. All types from the Base Type System are provided. Custom types can be added.

2.0

An addressable index into the schema must be provided.

- no solution yet.

2.0.1

Repeatable records must be supported (addressable index)

- no solution yet.

2.1

Multiple field transformations must be supported.

The Converter

2.1.1

 

 

2.2

Multiple record (=multiple fields) transformations must be supported.

The GroupConverter

2.2.1

 

 

 

 

 

 

Schema

The term Schema represents a description of a digital data structure; midi messages in this case. The proposed solution sets down a type system as a basis for describing the device’s System Exclusive implementation. The schema describes all aspects of a System Exclusive message; from message exchange patterns to field data types.

 

The choice was made to express the Midi Device Schema as Xml Schema xml. Because almost all of the constructs needed are readily available in Xml Schema and the parsing technology is free.

 

There are some liberal interpretations of Xml Schema in the Midi Device Schema language.

 

Xml Schema

Midi Device Schema

simpleType

A data type to be used in a field

complexType

A logical field record; grouping of fields.

element

A logical field declaration

union

Mask, shift and bitwise-OR a data byte

fixed

A fixed midi protocol or device specific value.

abstract

Used on a field type to indicate a place holder to be overridden by derived schema types.

nillable

Specified on a field to make that field optional.

facet

A constraint on the value of the field.

annotation

Used for documentation.

midi:name

A custom name attribute used to name facets (enumerations).

 

Declarations

The following sections show how to declare Midi Device Schema constructs in Xml Schema.

 

The examples use “mt” as the namespace alias for the midi base type’s namespace.

 

Midi Device Schema declaration

A Midi Device Schema declaration is exactly the same as an Xml Schema declaration. The following example illustrates a basic schema declaration.

<?xml version="1.0" encoding="utf-8"?>

<xs:schema targetNamespace="[name of schema]" xmlns:xs="http://www.w3.org/2001/XMLSchema">

Note that the targetNamespace is used to identify the Midi Device Schema. This name should be chosen such that it uniquely identifies the midi device and its System Exclusive implementation. Schemas with the same targetNamespace are considered the same schemas.

 

Declaring a data type

Data types are the basis for converting physical midi data into logical midi data and visa versa. The data converters are created based on these data types.

 

This example shows the mother of all data types, the midiByte.

<xs:simpleType name="midiByte">

  <xs:restriction base="xs:unsignedByte" />

</xs:simpleType>

 

Note that typically custom data types are derived from existing data types. See also Inheriting a data type.

 

Declaring a data record

A data record is a collection (sequence) of fields that logically make up a unit. How a device’s System Exclusive data fields are grouped into records is a design issue for the developer that creates the Midi Device Schema.

 

This example shows the template for a System Exclusive message. Note the begin- and end-markers of the midi SysEx and an abstract type as a place holder for the message body content.

<xs:complexType name="midiSysExMsg">

  <xs:sequence>

    <xs:element name="SOX" type="mt:midiSysEx" fixed="240" />

    <xs:element name="Body" type="midiSysExMsgAbstractBody" />

    <xs:element name="EOX" type="mt:midiSysEx" fixed="247" />

  </xs:sequence>

</xs:complexType>

 

Declaring a field

A field represents a logical value in a SysEx message and is always declared as part of a data record.

 

This example show the declaration of the Universal SysEx Id field (using a contrained data type).

<xs:complexType name="…">

  <xs:sequence>

    <xs:element name="UniversalId" type="mt:midiUniversalSysExID" />

    …

  </xs:sequence>

</xs:complexType>

 

Note that a field’s type can be either a data type (representing a concrete value) or another record type (representing a sub-record of fields).

 

Inheriting a data type

Currently only restriction-inheritance is supported for data types.

 

The example shows the declaration of the midiSysExData data type that is constrained to bits 0-6 (not shown).

<xs:simpleType name="midiSysExData">

    <xs:restriction base="midiByte">

      …

    </xs:restriction>

  </xs:simpleType>

Declaring custom data types is a matter of picking the most appropriate base type and optionally adding constraints.

 

[Implementation:] In order to overrule data converter functionality one could declare a new data type and inject their custom data converter for that data type by providing a converter factory.

 

Inheriting a data record

A data record (complex type) can be inherited in order to override abstract fields in the base type. See also Overriding a field.

 

The following example shows the declaration of a universal SysEx message based on the midiSysExMsg template presented earlier.

<xs:complexType name="midiUniversalSysExMsg">

    <xs:complexContent>

      <xs:extension base="midiSysExMsg">

        <xs:sequence>

          …

        </xs:sequence>

      </xs:extension>

    </xs:complexContent>

  </xs:complexType>

Note that deriving data records (complex types) is by extension, not restriction as with data types.

 

Overriding a field

In order to override an existing (abstract) field on a base type the names of the field should match exactly.

 

Note that only fields whose type is another data record type can be overridden.

In the following example the Body field of the midiSysExMsg base type is overridden with a new type. The midiUniversalSysExBody is a data record type that declares all the fields for a universal System Exclusive message.

<xs:complexType name="midiUniversalSysExMsg">

    <xs:complexContent>

      <xs:extension base="midiSysExMsg">

        <xs:sequence>

          <xs:element name="Body" type="midiUniversalSysExBody" />

        </xs:sequence>

      </xs:extension>

    </xs:complexContent>

  </xs:complexType>

 

Field overrides are name-based and the type of the field in the base does not have to be abstract in order to be overridden. So whenever a derived data record specifies a field who’s name is present in its base record, it will override that field with the (data) type specified. The field (data) type in its base will not be used, unless the new field (data) type derives from it.

 

Note that it is permitted to specify fields with the same name in different (complex) types for a Midi Device Schema (as is the case when overriding a field). Xml Schema parsers will generate an error for this condition which can be ignored.

 

Data type constraints

An Xml facet can be used to constrain the valid value range of a data type.

 

The following example shows the constraints placed on a data byte in a SysEx message.

<xs:simpleType name="midiSysExData">

    <xs:restriction base="midiByte">

      <xs:minInclusive value="0" />

      <xs:maxInclusive value="127" />

    </xs:restriction>

  </xs:simpleType>

 

The following xml facets are supported:

Xml Facet

Description

minInclusive

Specifies the minimal value for the data type. The value specified lies inside the valid range.

maxInclusive

Specifies the maximal value for the data type. The value specified lies inside the valid range.

minExclusive

Specifies the minimal value for the data type. The value specified lies outside the valid range.

maxExclusive

Specifies the maximal value for the data type. The value specified lies outside the valid range.

enumeration

Specifies an exact value that is valid for the data type. Multiple enumeration facets can be used fro one data type.

length

Specifies the fixed length of a data type. Only valid for data types deriving from midiString.

 

 

Constant values

There are two ways to declare constant values.

 

  1. Constraint a custom data type
    Constraint a custom data type with one enumeration facet that specifies the constant value. Use this data type for all fields that require the constant value.
  2. Specify the fixed attribute for a field.
    For each field that requires the constant value specify the fixed attribute and the constant value.

 

The following example shows the fixed values for the SysEx begin- and end-markers in a midi SysEx message.

<xs:complexType name="midiSysExMsg">

  <xs:sequence>

    <xs:element name="SOX" type="mt:midiSysEx" fixed="240" />

    <xs:element name="Body" type="midiSysExMsgAbstractBody" />

    <xs:element name="EOX" type="mt:midiSysEx" fixed="247" />

  </xs:sequence>

</xs:complexType>

 

Documenting a schema

You can use the standard Xml Schema annotation tags to document the Midi Device Schema declarations.

 

<xs:schema targetNamespace="…" xmlns:xs="http://www.w3.org/2001/XMLSchema">

  <xs:annotation>

    <xs:documentation>The midiTypes schema represents the base type used in the midi device schema framework.</xs:documentation>

  </xs:annotation>

  <xs:simpleType name="midiByte">

    <xs:annotation>

      <xs:documentation>midiByte is the base type for all (simple) midi types. It represents an unsigned 8-bit byte.</xs:documentation>

    </xs:annotation>

    <xs:restriction base="xs:unsignedByte" />

  </xs:simpleType>

  …

</xs:schema>

 

The Type System

[Note: The type system is still under development!]

 

The Midi Device Schema type system is designed to allow a schema author to express the logical midi System Exclusive fields of a device. All types inherit from the midiByte data type which represents an unsigned byte (8-bits).

 

The following figure displays the type hierarchy for the (simple) data types declared in the type system.

 

 

 

midiByte

Represent the base data type for all other types but is not declared abstract and thus can be used as any data type. The value represents the full 8 bits of an unsigned byte.

 

midiBit0-midiBit7

The midiBit0 till midiBit7 data types each represent one bit of the 8-bit byte. midiBit0 represents the least significant bit and midiBit7 represents the most significant bit.

 

midiNull

The midiNull data type can be used for fillers or dummy fields. The value defaults to 0 (zero) but can be set at a per field bases using the fixed attribute on the field declaration. Fields that use this data type are not visible at a logical level. When logical data is converted to physical midi data, these fields are inserted automatically at their designated positions within the record.

 

midiSysExData

The midiSysExData represents a data byte inside a midi System Exclusive message. This data type enforces that bit7 (most significant bit) is always clear (not set) to comply with the midi specifications for System Exclusive messages.

 

midiNibble

The midiNibble data type is abstract and cannot be used in a field declaration. The data type is a logical indication that the data is 4-bits in length (a nibble) and is mainly used by the nibble-data converter implementation.

 

midiLSN

The midiLSN represents the Least Significant Nibble (LSN) of a data byte (lower 4 bits).

 

midiMSN

The midiMSN represents the Most Significant Nibble (MSN) of a data byte (upper 4 bits).

 

Note that a contradiction is hidden in this data type: The midiMSN derives from midiSysExData. That type ensures that bit7 is always cleared. This means that bit7 of a midiMSN field is also always cleared.

 

midiComposite

The midiComposite is an abstract type that is used as a base for logical values that span multiple physical bytes.

 

This type is not intended to be derived by schema authors.

 

midiString

The midiString (derives from midiComposite) represent multiple physical data bytes that make up one logical text string.

 

In order to use this in a schema declaration one should derive a custom data type and specify the length facet for that custom type. The string-data converter will use the specified length to read the text string.

 

Record Base Types

The Type System also contains (complex) record types (field structures). Checksum calculations for instance can be implemented this way. These types are still under development and too volatile to document here. These will be added as soon as the design is somewhat more stable.

 

The current functionality for these (complex) record base types include: