Metastorage Documentation

Metastorage is a persistence layer generator application based on the persistence module of the MetaL compiler engine. Metastorage is capable of generating the necessary software components to implement a persistence layer API from a description in a format based on XML named Component Persistence Markup Language (CPML). The generated code is self-contained and does not depend on MetaL or any libraries of code that are not supplied.

What is MetaL?

MetaL is a meta-programming language compiler engine. Meta-programming is a method to develop programs in high-level language that can be translated into one or more programming languages.

MetaL compiler is able to process source code in XML and generate code in virtually any target language: PHP, Java, Perl, etc..

What is a persistence layer?

Many applications need to manage information that contain the state of the processes and the entities that are involved in the applications.

The information is generated and stored in some container so it can be retrieved later is known as persistent because it can outlive the process that generated it. Common examples of persistence containers are: relational databases, flat file databases, XML files, LDAP accessible servers, etc..

A persistence layer is an applications programming interface (API) that is used to abstract the operations of storing and retrieving information from a persistence container.

Motivation and goals

Managing information that is stored in a persistence container is a common need of many applications. However, the effort to implement a persistence layer often takes a significant amount of time and is tedious.

Reduce the development effort

The main goal of this generator is to drastically reduce the development effort necessary to implement persistence layers without loosing the flexibility to customize the persistence API according to the application needs.

The component description should be simple enough to not take too much time to write by hand.

Produce optimized code

Another goal is that the generator should produce optimized code according to hints specified in the component description.

The generator should analyze the component description to avoid producing generic code that is not needed by the application.

The code should always be generated according to the best strategy to execute in less time and if possible take less hardware resources.

Persistence container independent

As any abstraction, the component description and the generated API should be independent of the type of container where the persistent data is stored. Therefore, the component description should not contain details that are specific to a particular type of persistence container.

This provides the flexibility to switch between container types if and when the component design developer finds it convenient.

Features list

XML definition format

Easy to write and understand component definition format in XML

Database schemas

Automatic generation of (database) schema definition of the persistence container

Automatic generation of all classes

Generates all the code for the data classes, factory class and (database) schema installation class

Self-contained code

The generated code is self-contained and does not depend on MetaL or libraries of code that are not supplied

Object Query Language

Provides an Object Query Language (OQL) to define expressions to filter objects in search queries

No SQL

All code is generated without the need for the developer to write SQL code manually

Forms

Generation of classes to create, validate and process Web forms to manage objects of the data classes.

Reports

Generation of classes to execute queries to retrieve data from many objects to elaborate reports or to execute another kind of bulk processing

UML Diagrams

Automatic generation of Entity-Relationship graphs in UML of the component class diagram

Design concepts

Data classes

There are several approaches to implement persistence layers. The approach that was used here is object oriented. This persistence layer generator creates a set of data classes that implement the persistence API based on a CPML specification.

The data classes model entities are manipulated by the application. For instance, a content management application would have entities like: article, category, author, etc..

Variables

The objects of the data classes have variables hold the information of the entities that is meant to be stored and retrieved from the persistence container. For instance, an article class managed by a content management application would have variables like: title, lead, body, date, etc..

Data types

Variables may be of several basic data types like: text, integer, boolean, float, decimal, date, time and timestamp.

Object IDentifiers (OID)

Each object of a class is unique. Although different objects of the same class may have variables with the same values, there must be a way to distinguish which object is which, so applications can locate and operate on each object individually.

The way to distinguish each object is by the means of special class variables that are known as Object IDentifiers (OID).

Usually, object identifiers are integer variables that are assigned with unique (sequential) values. Their existence is implicit, so they usually do not need to be explicitly declared and initialized by applications.

In particular situations, object identifiers may be defined explicitly by multiple class variables that have unique combinations of values across all objects of the class.

Relations

Each component may have defined many data classes. Classes may be related to each other. For instance, in a content management system, an article may be published by an author, the article may belong to one or more categories, a category may contain many articles, etc..

The relations between classes may have different levels of multiplicity: one to one, one to many and many to many.

References

Variables may also be of a special type of data that is a reference to an object of the same or other classes. A reference denotes a relation of multiplicity one to one.

As a special case, references may be undefined, meaning that an object may or may not reference another object, denoting a multiplicity of one to zero or one.

Collections

When an object of a class may be related to many objects of the same or other classes, the group of related objects is known as a collection.

Collections are defined within the scope of a class. The definition of a collection specifies the class of the related objects of the collection.

The collection definition also specifies a reference which is either a variable or another collection of the related class. The reference variable or collection also references back to the specified collection.

If the reference specifies a variable of the related class, that denotes a relation of multiplicity one to many. If the reference specifies another collection of the related class, that denotes a relation of multiplicity many to many.

Validation

The data stored in object may be subject of application specific rules that determine if it is consistent. For instance, in a content management system, the title of an article may not be empty, there may only be one author with given e-mail address, the name of each article category must be unique, etc..

The persistence API may also be able to verify the consistency rules defined by the component design developer when needed. This meant to help to prevent that inconsistent data be stored in the persistence container.

Functions

Along with the variables, the data classes also contain functions that implement the actions that are important for the applications that use the generated persistence layer.

The generated functions can be of many types like: creating a new data object, validating the object data consistency, storing and retrieving from the persistence container, etc..

The generator produces the necessary code to implement functions specified by the component design developer according to the needs of the application. The definition of some types of functions may contain additional parameters to customize their behavior.

The generator may implicitly generate additional class functions and variables to support the operations that are needed.

Object factory

Most actions performed using the persistence layer start by creating new data objects by either retrieving them from the persistence container or by creating a new object from scratch.

Data object creation is performed by a special type of class known as the factory class. This class manages the life cycle of all persistent objects. It can create individual objects or retrieve whole collections of objects from the persistence container.

Like the data classes, the functions that are provided by the factory class are specified by the component design developer.

The factory class also provides centralized services like error handling or transaction management.

Object search filter language

The data object classes and the factory class can have several types of functions that access the persistence storage container and retrieve one or more objects.

Frequently applications need to search and retrieve only the objects that satisfy a given condition. For this purpose, a search filter expression can be associated with any of the functions that retrieve objects of a given class or objects that belong to a collection of related classes.

The CPML also includes the definition an object search filter language to be used in the definition of functions that are meant to return objects that satisfy given search conditions.

Container schema setup

Before starting to store data class objects, the persistence container may need to be initialized. The initialization usually consists in the installation of a schema to accommodate the data objects.

For this purpose, the generator may also create a special class that can provide services to install and manage the schema of data to be stored in the persistence container.

Component Persistence Markup Language - CPML

XML based component description format

The component description format used by Metastorage is based on XML. The idea is to use a format that is simple to write by hand while it is verbose enough to understand even by developers that are not yet familiar with Metastorage.

Metastorage XML format follows the Simplified XML (AKA Minimal XML) restrictions, i.e. it does use character entities and XML tags have no attributes. This makes the format easier to read while it honors the meaning of the letter X in XML: be eXtensible. If the format XML tags used attributes, it could not be as extensible because attributes may not contain any tags inside them.

Format structure

The definition of components with CPML starts with the root tag that is always named component. Below you may find a complete example of a component definition using CPML.

Inside the component tag comes the definition of some component attributes and the definition of each of the classes that compose the component.

Component name

(Required) Components must have a name. This the base name that will be given to the component factory and schema classes that are generated.

Component description

(Required) Components must have a short description text that will be used for documentation purposes.

External components

An application is often made of several components that may reference each other classes. A component that references classes of other components must declare them first in a special section that starts with the tag component.

External component file

The specification of an external component must indicate the name of file that contains its definition. The file name may contain the file path that may be absolute or relative to the current component file path.

External component name

An external component is defined by its name. This name must be exactly the same that is specified in the external component definition file.

Data object classes

All data object classes are defined in sections starting with tag class. Alternatively, you may define individual classes in separate XML files. For that you need to use the tag includeclass specifying the class definition file name as value, like in the following example. The contents of the class definition file are same as for classes defined within the component file. The root tag of the class definition files is also class.

<includeclass>edition.class</includeclass>

Element identifiers

Each class and any of its elements may have associated a custom identifier defined by the designer. This identifier is optional but once it is defined it should not be changed for all the component development lifetime.

An element identifier is defined by the tag id. Its value may be of any type but it must be unique across all types of elements that define their identifiers. Besides the class definitions, the types of elements that may define their unique identifiers are: variables, collections, validation rules and functions.

Class name

(Required) Each class must have a name. This the same name that will be given to each class that is generated.

Renamed class previous name

(Required when renaming a class) By default the name of a class is mapped to the name of the table on which the persistent objects are stored when using a database as persistent storage container.

When a class is renamed, the database table must also be renamed. The schema management class generated by Metastorage must know the new table name and the previous name, so it can rename the table without loosing previously stored objects.

The was tag must be used to specify the previous name of a class when it is renamed.

WARNING: when a class, a variable or a collection is renamed, make sure the was tag is set with the correct previous name that was used when the database schema was installed for the first time or updated for the last time. Failure to do so may prevent that the schema is upgraded correctly.

Class variables

(Required) Each class usually contains several variables that are defined using separate variable sections for each.

Variable name

(Required) Each variable must have a name. This is the same name that will be given to each variable member in the generated class.

Renamed variable previous name

(Required when renaming a variable) The was tag must be used to specify the previous name of a variable when it is renamed.

Please read the section about "Renamed class previous name" to learn about important details about renaming classes, variables and collections.

Variable basic data type

(Required alternative) Each variable must be of a type. The type must be either a basic data type or a reference to a object of the same or another class. The type tag is used to specify the basic data type of a variable. Supported basic data types are: text, integer, boolean, float, decimal, date, time and timestamp.

Variable large data type

Besides the normal scalar data types, variables may store large data types usually known as BLOBs - Binary Large OBjects. Large data types are suitable for storing binary or text data with length that may exceed 255 bytes. A large data variable is defined by setting the type tag to the value largedata.

Large data variables cannot be assigned or retrieved directly like other type of variables. Instead, the functions of type setlargedata and getlargedata must be used.

Variable large data binary and text types

Large data variables may be either binary (BLOB - Binary LOB) or text (CLOB - Character LOB). The distinction is made by setting the binary tag to 1 for binary and 0 for text large data variables. By default, large data variables are binary.

Variable initial value

A variable of a basic data type may have an initial value. That value is automatically assigned to the variable when the object is created.

Also, when a new basic data type variable is added to a class, the initial value is automatically stored in the variable for all the previously persisted objects. This is not the case for large data variables.

The initialvalue tag is used to specify the initial value of a variable.

Variable creation and updating automatic values

A variable may be set to be initialized automatically with a special value using the autocreate attribute tag. This attribute determines that the variable will be created with an automatic initial value and will never be changed in subsequent updates of the object done using a class persist type function.

Alternatively, a variable may be set to be initialized and updated automatically with a special value using the autoupdate attribute tag. This attribute determines that the variable will be set with an automatic value every time the object is updated using a class persist type function, including when the object is created and stored for the first time.

Either autocreate and autoupdate variables may not be changed directly by the applications access the objects of their classes.

Currently, only the variables of the types date, time and timestamp may be defined as autocreate or autoupdate. The values of either of these attributes may be either localdate and utcdate for date variables, and localtime and utctime for time or timestamp variables. localdate and localtime represent the current date and time in the local time zone and utcdate and utctime represent the current date and time in the UTC (Universal Time) time zone.

Class object reference variable

(Required alternative) Variables may contain references to objects of the same or other classes. The class tag is used to specify the class of object reference variable. This attribute must contain the name of a class specified in the same component definition or in from another component specified by the component attribute.

Reference variable of an external component class

A variable may contain a reference to an object of a class of another component. The component tag is used to specify the external component that defines the reference class. The specified external component must be previously declared in the current component definition.

Optional variable

Setting the value of a variable can be made optional. An optional variable that is not initialized it is said to be set to null. The optional tag is used to specify a boolean value that determines whether it is intended to make a variable optional.

By default, all variables are required (not optional). Attempting to store an object with required variables that are not initialiazed makes the calls to the class persist type function fail.

Length limited text variable

The length of a text type variable may be limited. The length tag is used to specify an integer value that specifies the maximum length that the variable value may have.

Multiline text variable

A variable of type text may store multiple lines. The multiline tag is used to specify a boolean value that specifies whether the variable value may spawn multiple lines of text.

Collections of objects

(Optional) The relation between the objects of a class with a group of objects of the same or other class is specified as a collection. A class may specify one or more collections using separate sections for each.

Collection name

(Required) Each collection must have a unique name that may not be the same as of any variables of the class.

Renamed collection previous name

(Required when renaming a collection) The was tag must be used to specify the previous name of a collection when it is renamed.

Please read the section about "Renamed class previous name" to learn about important details about renaming classes, variables and collections.

Collection class

(Required) A collection references objects a of a specified class.

Collection of an external component class

A collection may contain objects of a class of another component. The component tag is used to specify the external component that defines the collection class. The specified external component must be previously declared in the current component definition.

Collection class reference

(Required) A collection reference specifies a variable or another collection of the referenced class.

Validation rules

(Optional) Each class may have defined a set of validation rules to determine the consistency of the data hold in its objects.

Validation rule type

(Required) Each validation must be of a specific type. Currently the types of validation that are supported are: notempty and unique.

Validation error code

(Optional) A class may verify several types of validation rules at once. If one validation rule does not verify, applications may need to know which rule has failed. An errorcode maybe associated with each validation rule to let applications act upon the rule that failed.

Validation rule specific parameters

(Required depending on rule type) Each type of validation may need additional parameters that define the details of the scope of the validation.

notempty rule

The notempty validation verifies that one or more text specified class variables are set to a non empty string.

The validation specific parameters consist of the list of variables specified using the variable tag for each variable to be checked.

unique rule

The unique validation verifies that there is no other stored object with the same combination of values of a group of class variables of a given object.

The validation specific parameters consist of the list of variables specified using the variable tag with the combination of values to be verified as unique.

Class functions

(Optional) Each data class usually contains several functions that execute actions based on the data stored on the class objects. The class functions are defined using separate function sections for each function.

Function name

(Required) Each function must have a name. This is the same name that will be given to each function member in the generated class.

Function type

(Required) Each data class function is of a specified type defined according to the function purpose. Currently the types of functions that are supported are: addtocollection, cloneobject, custom, delete, getcollection, getlargedata, getreference, persist, removefromcollection, setlargedata, setreference and validate.

Custom function arguments

(Optional) Some types of function may take additional arguments that can be used to configure details of the function behavior using values that are only be defined at run time by the application that calls the function.

This is the case for instance of functions that search for objects of a given class may take arguments that define values that can be used in a filter condition parameters.

A class function may have multiple custom arguments. If there are arguments that were defined but actually are not used in the function, probably because they were forgotten or for some other type of mistake, Metastorage fails specifying which argument was left unused.

Each argument is defined separately in sections that start with the tag argument. The argument details are defined inside the respective section.

name

Each argument must have a unique name.

type

The type of the argument is also a required parameter when the argument type is scalar or an array or an hash table. Currently supported scalar argument types are: integer, text, float, decimal, boolean, time, date, timestamp, array and hashtable.

class

An argument may also be a reference to an object. The class parameter specifies the name of a class of the object argument.

oid

An object argument may also be passed by using its object identifier value (oid). For object arguments of classes with an implicit oid, the argument is an integer number usually stored in the id class variable. An object identifier arguments is specified using an empty oid parameter.

component

An argument may be an object of a class defined in an external component definition. The component parameter specifies the name of the external component on which the object class is defined.

Class function specific parameters

(Required depending on function type) Each type of function may need additional parameters that define the details of the scope of the function.

addtocollection

A addtocollection type function adds to a collection defined in a class an object of the class specified in the collection definition.

Notice that if the collection establishes a one-to-many type of relationship, the reference variable of the object added to the collection, is changed. So, the added object needs to be persisted to commit the change after calling the addtocollection function.

The generated function is of type BOOLEAN. It returns true if it succeeded or false if there was an execution error. The function takes as argument a reference to the object to be added to the collection.

The function specific parameters are the collection that specifies the class collection name and the object section. This section must define the name of the object argument.

cloneobject

A cloneobject type function creates a copy of the current object with the same variable values. The object copy is unsaved. Applications must call a persist type function to store the new object.

The generated function is of type OBJECT. It returns an object reference if it succeeded or null if there was an execution error. This function does not take any additional arguments.

custom

A custom type function may be used to implement customizations of a class using arbitrary code in the target language written by the component designer.

The custom code is defined with the function specific do parameter.

The language parameter defines on which programming language the custom code is written. Only the current target language is supported, so this parameter may be omitted.

The type of the function return value may be specified within the return parameter section. If the function does not return a value, this section should be omitted.

The return type definition may include either a type name or a class name. The supported return types are the same as for the custom function arguments.

The class return type parameter may be used to specify that the return value is an object of a given class. Additionally, the oid return type parameter may be used to specify that function returns an object identifier value of that class, rather than an object reference.

If the specified class belongs to a different component, it must be specified also the component name as a separate return type parameter with the name component.

delete

A delete type function deletes the object from persistent storage if it was already persisted.

If the object has any collections, it is also removed from those collections. For collections of the deleted object that establish one to many relationships, this means that all objects that belong to such collections will have the respective reference variable set to null before the object is deleted. This meant to assure referential integrity.

The generated function is of type BOOLEAN. It returns true if it succeeded or false if there was an execution error. This function does not take any additional arguments.

getcollection

A getcollection type function retrieves all the objects that belong to a class collection of a given object.

The generated function is of type ARRAY. It returns an array with a collection of objects if it succeeded or null if there was an execution error.

The only function specific parameter is the collection that specifies the name of the collection of the class to be retrieved.

getlargedata

A getlargedata type function retrieves data stored in a large data class variable.

The generated function is of type BOOLEAN. It returns true if it succeeded or false if there was an execution error.

The main function specific parameter is the name of the large data variable of the class that should be retrieved.

Currently, a large data variable may be retrieved all at once either to a data string variable, or to a file, or as the output of the current script. The parameter tags data, file and output specify respectively which retrieval mode is used. These tags define sections that specify the name of the argument that will be used to return by reference the retrieved data or the name of the file on which the data will be stored. The output parameter does not take additional arguments.

If a variable is optional, its value may be undefined when it is retrieved. In this case the large data variable contents may not be retrieved and a getlargedata function may fail.

To avoid this problem it may be specified the parameter tag undefined. This tag defines a section that is used to specify the name of an additional boolean argument that is passed to the function by reference to return the information of whether the specified large data variable is undefined.

getreference

A getreference type function retrieves an object referenced by class variable of a given object.

The generated function is of type OBJECT. It returns an object reference if it succeeded or null if there was an execution error.

The main function specific parameter is the name of the variable of the class that references the object to be retrieved.

Optionally, the oid parameter may be specified to tell that the function should return the object identifier stored in the reference variable, instead of the actual referenced object. The oid parameter tag must be empty.

Usually, a getreference type function may not be used to retrieve an object pointed by a reference variable of a class defined in an external component. However, it can be used to retrieve just the object identifier stored in the reference variable, when the function is defined using the oid parameter. Then that object identifier can be passed to a getobject function of the external component factory class to retrieve the actual object.

persist

A persist type function stores in the persistence container the changes made to a given object.

The generated function is of type BOOLEAN. It returns true if it succeeded or false if there was an execution error. This function has no specific parameters

removefromcollection

A removefromcollection type function removes from a collection defined in a class an object of the class specified in the collection definition.

Notice that if the collection establishes a one-to-many type of relationship, the reference variable of the object removed from the collection, is changed. So, the removed object needs to be persisted or permanently deleted to commit the change after calling the removefromcollection function.

The generated function is of type BOOLEAN. It returns true if it succeeded or false if there was an execution error. The function takes as argument a reference to the object to be removed from the collection.

The function specific parameters are the collection that specifies the class collection name and the object section. This section must define the name of the object argument.

setlargedata

A setlargedata type function sets the value of a large data class variable.

The generated function is of type BOOLEAN. It returns true if it succeeded or false if there was an execution error.

The main function specific parameter is the name of the large data variable of the class that should be changed.

Currently, a large data variable can be set to the contents of a data string variable, or to the contents of a file, or be set to an undefined value if the large data variable is optional.

The parameter tags data, file and undefined specify respectively how to set the large data variable value. These tags define sections that specify the name of the argument that will be used to pass the data string, or the name of the file from which the data will be retrieved. The undefined parameter does not take additional arguments.

setreference

A setreference type function stores a reference to an object in the class variable of a given object. If a null object is passed by argument the function will store a null value in the object class variable. Changes are only effective when a function of type persist is called on the specified object.

The main function specific parameters consist of the definition of the reference variable and a section with the argument in which it is stored the reference.

Optionally, the oid parameter may be specified to tell that the function reference argument is actually the object identifier to be stored in the reference variable, instead of an object reference. The oid parameter tag must be empty.

Usually, a setreference type function may not be used to store a reference to an object of a class defined in an external component. However, it can be used to store the identifier of the external object in the variable, when the function is defined using the oid parameter.

The generated function is of type BOOLEAN. It returns true if it succeeded or false if there was an execution error.

The only situation on which this type of function may fail is when it is passed a reference to an object that is not yet persisted. This situation exposes a bug in the calling application that must be fixed by invoking the persist type function of the reference object class before passing it to a setreference type function.

validate

A validate type function validates the consistency of the data of an object verifying all the validation rules defined for the class of the object. Validation rules are verified by the order that they are specified in the component definition.

The generated function is of type BOOLEAN. It returns true if it succeeded or false if there was an execution error. If it succeeded but one validation rule was not verified, the function returns the respective error code in a out argument. The error code value 0 means that all validation rules were verified.

The validation specific parameters consist of a section with the definition of the argument in which it is stored the errorcode. This section must define the name of the error code argument.

Indexes

(Optional) Each class may have defined a set of indexes to accelerate object queries with conditions that involve one or more class variables.

Index name

(Required) Each index must have a name that must be unique among all indexes defined in a component or related components.

Unique index

(Optional) An index may be defined as unique if for all objects of the class there will be at most one object with the same combination of values in the index variables.

Index variables

Each index must be associated to one or more variables. A variable may only be specified once for each index and may not be of type largedata.

Variable name

(Required) The definition of each index variable must specify the class variable name.

Variable sorting

(Optional) The definition of each index variable may specify the sorting order by which a variable is expected to be used in object queries that sort the returned results.

Object factory class

The object factory class is a special purpose class that is meant to create new objects from scratch or retrieve them from the persistence container.

For each component, there should be only one factory class. Like the data classes, the factory class may also have customizable functions defined in a similar way. The only function definition attributes that are different are the function types and their parameters.

The currently supported factory class function types are: createobject, custom, finishtransaction, getallobjects, getobject and starttransaction.

createobject

A createobject type function creates a new object of a given class from scratch. The object is only effectively stored in the persistence container when a class function of type persist is called on the created object.

The generated function is of type OBJECT. It returns an object reference if it succeeded or null if there was an execution error.

The only function specific parameter is the class that specifies the name of the class of the object to be created.

custom

A custom type function may be used to implement customizations of the factory class using arbitrary code written by the component designer in the target language.

The factory class custom functions follows the same definition rules as of data class custom functions.

finishtransaction

A finishtransaction type function terminates a pending transaction started with a function of type starttransaction.

The generated function is of type BOOLEAN. It returns true if it could finish a transaction, or false if there was an execution error.

The commit parameter must be defined to specify how the transaction should be finish. This parameter may have three possible values: yes to make the function commit the changes made during the transaction, no to rollback any changes, and argument to take a boolean argument that will tell whether the changes should be commited in case the argument is true, or to rollback the changes otherwise.

If the commit parameter value is argument, it must also defined a section starting with the tag argument to specify the name of the argument.

getallobjects

A getallobjects type function retrieves all objects of a given class from the persistence container.

The generated function is of type ARRAY. It returns an array of object references if it succeeded or null if there was an execution error.

The only function specific parameter is the class that specifies the name of the class of the objects to be retrieved.

getobject

A getobject type function retrieves an object of a given class from the persistence container specifying as argument the primary key value.

The generated function is of type OBJECT. It returns an object reference if it succeeded or null if there was an execution error.

The function specific parameters are the class that specifies the name of the class of the objects to be retrieved and a id section that specifies name of the argument that will be used to pass the object identifier to the function.

starttransaction

A starttransaction type function initiates a transaction. Transactions must be ended by calling a function of type finishtransaction.

The generated function is of type BOOLEAN. It returns true if it can start a transaction, or false if there was an execution error. This function does not take any additional arguments.

Object search filters

Metastorage provides a means to search for objects using special filter parameters. These parameters are actually search condition expressions that are associated with functions that retrieve one or more objects. These expressions are defined with the syntax of the object search filter language that is part of the CPML definition.

The object search functions are actually functions of the type getobject and getallobjects of the factory class and getcollection of the data object classes.

Filter expressions

Each object search function may be associated with only one filter expression, but there may be many object retrieval functions in one class and each may satisfy different search conditions.

A filter condition is associated to an object retrieval function by adding a filter section inside the function specific parameters section.

A filter section must define a boolean expression composed by a sequence of operand expressions and operators. When the filter expression is evaluated for each of the objects being queried, it determines which objects are selected to be returned as result of the execution of the function.

Filter operands

Filter operands are expressions that are evaluated when the function is executed.

Each operand expression is represented by a tag with a name that specifies the type of operand. Some types of operands take additional parameters that are specified inside the section defined by the operand tag.

The result of the evaluation of each filter operand expression is a value of a data type that depends on the type of expression.

A considerable list of operand expression types is supported for several distinct purposes.

Variable access

variable

isnull

variable represents an expression that evaluates the value of a given class variable. The type of the expression is the same of the specified variable.

isnull represents a boolean expression that evaluates whether a given class reference variable is set to a null object.

This operand takes as required argument the name of the reference variable to be accessed. As an optional parameter it may also be specified the object of the variable to be accessed. The default object is of the class or collection that is queried by the function to switch the filter is being applied.

Examples:


 <variable>
  <name>title</name>
  <object>article</object>
 </variable>


 <isnull>
  <variable>article</variable>
  <object>author</object>
 </isnull>

Object query joins

object

The object operand expression type is meant to declare an object of a given class involved in a filter expression that refers to multiple objects of related classes. This type of filter expression is also known as joins, exactly in the same sense of SQL query joins.

The object operand expression itself does not represent a value but rather an alias to an object that is assigned implicitly to another object type expression using for instance the equalto operator.

This operand type takes the name of the object as required parameter. This name will be used by other types of expressions to refer to this object.

The actual class of the object may be defined as an optional parameter. If the class parameter is omitted, it is assumed that the name of class is the same defined by the name parameter.

If the object class is defined in an different component, the component tag should be used to specify the name of that external component.

Example:


  <object>
   <name>author</name>
   <class>writer</class>
  </object>

 <equalto />

  <variable>
   <object>article</object>
   <name>author</name>
  </variable>

Object collections

collection

The collection operand expression type is meant to specify a collection of objects related to an object of a given class. It should be used in expressions that check the presence of objects in class collections, for instance using the in operator.

This operand type takes the name of the object and the name of the collection as required parameters.

Example:


  <object>
   <name>article</name>
  </object>

 <in />

  <collection>
   <object>author</object>
   <name>articles</name>
  </collection>

Literal values

text

integer

float

decimal

boolean

date

time

timestamp

A filter expression may use literal values to represent constants of every data type.

A literal value can be specified with an operand expression defined by tags with the same name as the data type that they represent. The actual constant value should be defined inside the literal expression tag.

Examples:


 <text>John</text>


 <timestamp>2004-05-28 05:03:04</timestamp>

argument

The argument operand expression type represents a literal value that is passed as argument to the function when it is called.

This expression type should specifies the name of the argument that it represents. The type of the expression is the same as the type of the argument specified in the argument declaration.

Example:


 <argument>title</argument>

Date and time constants

The date and time expressions allow the use of constant values in expressions that are relative to the current date and time. These constants are computed at runtime, when the query statements are being prepared to be executed.

Each of the supported expressions have variants to express dates and times in two main time zones: the local time zone and the Coordinated Universal Time (UTC) time zone. The UTC expressions have the same names, except that they have the utc suffix.

now

nowutc

Current timestamp

todaytime

todaytimeutc

Current time of the day

today

todayutc

Today's date

time

timeutc

Current time

weekday

weekdayutc

Current day of the week: 1 - Sunday, 2 - Monday, 3 - Tuesday, 4 - Wednesday, 5 - Thursday, 6 - Friday, 7 - Saturday

Logical expressions

not

The not operand expression type is meant to negate the logical value of an boolean expression defined inside its tags. The type of this expression value is also boolean.


 <not>

  <isnull>
   <variable>article</variable>
   <object>author</object>
  </isnull>

 </not>

Expression grouping

group

The group operand expression type is meant to group a sub-expression assuring that it is evaluated with priority relatively to other expressions that follow the group.


  <group>

    <variable>
     <name>article</name>
    </variable>

   <minus />

    <integer>18</integer>

  </group>

 <multiplyby />

  <argument>factor</argument>

Row grouping

Row grouping expressions are functions that can be used in column value expressions of reports that group rows with the same object variable values.

count

The count expression is a grouping function that returns the number of group rows with the same values that were merged into a single row.

It may take optional parameters that specify an object and an object variable. In this case, the function returns only the number of objects that have that variable set.


    <column>

      <name>count</name>

      <value><count>
        <variable>author</variable>
        <object>article</object>
      </count></value>

    </column>

minimum

maximum

The mimimum and maximum expressions are grouping functions that return respectively the lowest and highest value of a given object variable in the grouped rows.

These functions can be used with object variables of the types integer, decimal, float, date, time and timestamp.


    <column>

      <name>middle</name>

      <value>
          <group>
              <maximum>
                <variable>
                  <name>distance</name>
                  <object>office</object>
                </variable>
              </maximum>

            <plus />

              <minimum>
                <variable>
                  <name>distance</name>
                  <object>office</object>
                </variable>
              </minimum>
          </group>

        <divideby />

          <integer>2</integer>
      </value>

    </column>

sum

average

The sum and average expressions are grouping functions that return respectively the total of the sum and the average of the values of a given object variable in the grouped rows.

These functions can be used with object variables of the types integer, decimal and float.


    <column>

      <name>total</name>

      <value>
        <sum>
          <variable>
            <name>price</name>
            <object>item</object>
          </variable>
        </sum>
      </value>

    </column>


    <column>

      <name>average</name>

      <value>
        <average>
          <variable>
            <name>price</name>
            <object>item</object>
          </variable>
        </average>
      </value>

    </column>

Filter operators

The filter operators specify operations that are be used to combine the expression operands and successively evaluate the whole expression.

Usually an operator is placed between two operands: the left operand and the right operand. An expression is evaluated performing the operation with the left and right operands. The resulting value is combined with the operand on the right of a second operator if it exists, and so on until it remains only one result value.

This order of evaluation may not always be exactly like this. Each operation may have a different level of priority. For instance, multiplications and divisions must be evaluated before additions and subtractions.

If the following operator has a greater priority than the current operator, then the whole expression on the right of the following operator it is evaluated before evaluating the current operation.

If it is necessary to assure that an operation between operands is evaluated before the following operation that has higher priority, use the group operand to enclose the sub-expression defined by the operands and operators that need to be evaluated first.

Metastorage filter expression operators are specified using tags with names that clearly identify the operation that they represent.

The supported operators are divided in several groups marked with the respective relative priority level in the front of the operator names.

Comparison operators

equalto (2)

differentfrom (2)

morethan (2)

lessthan (2)

moreorequalthan (2)

lessorequalthan (2)

The comparison operations compare the left and right operand expressions. The result of comparison operations is a boolean value that indicates whether the comparison condition is true or false.

The comparison operators take left and right operands of the same type. Those can be of any of the supported types except boolean.

Arithmetic operators

plus (3)

minus (3)

multiplyby (4)

divideby (4)

The arithmetic operators perform calculations with the operand expressions. The result of arithmetic operations is another value of the same type of the operands.

The arithmetic operators take left and right operands of the same type that must be either of type integer, float or decimal.

Logical operators

and (1)

or (1)

The logical operators perform boolean operations only with boolean the operand expressions. The result of logical operations is another boolean value.

Collection operators

The collection operators perform operations over a collection of scalar values or objects.

in (5)

The in operator can check the presence of a given object in a collection of objects.


  <object>
   <name>article</name>
  </object>

 <in />

  <collection>
   <object>author</object>
   <name>articles</name>
  </collection>

not in (5)

The not in operator can check the absence of a given object in a collection of objects.


  <object>
   <name>article</name>
  </object>

 <not /> <in />

  <collection>
   <object>author</object>
   <name>articles</name>
  </collection>

Object declarations

In certain circumstances a filter expression may involve objects that need to be declared before they are used.

When it is necessary to make a previous declaration of an object to be used in a filter expression is associated to an object retrieval function, the object declaration may be done by adding an object section inside the function specific parameters section.

The parameters of an object declaration are the same of the object query join operands that may be used within filter expressions.

Schema setup class

The schema setup class is also a special purpose class that is meant to provide services to setup a data schema to store the objects in the persistence container.

For each component, there should be only one schema setup class. Like the data classes and the factory class, the schema setup class may also have customizable functions defined in a similar way.

The only currently supported schema setup class function types is installschema.

installschema

A installschema type function does all that is necessary to install the data schema in the persistence container. The meaning of this action depends on the type of persistence container being used.

Currently the only type persistence container that is supported are SQL based relational databases. For this type of persistence container, this function installs the database schema.

The generated function is of type BOOLEAN. It returns true if it succeeded or false if there was an execution error. This function does not take any parameters.

Form classes

A data object form class is a type of class that may be generated to provide a user interface for creating or editing objects of the data classes defined for a component.

All form classes are defined in sections starting with the tag form. Alternatively, you may define individual forms in separate XML files. For that you need to use the tag includeform specifying the form definition file name as value. The contents of the form definition file are same as for forms defined within the component file. The root tag of the form definition files is also form.

Form templates

Before describing how the form output is generated, it is important to understand the concepts of the template system that is used to build the form output that is presented to the user.

Template clips

The form output is generated from data taken from template files. The template data is extracted from portions of the template files named clips.

A clip is a section from the template file that is within special delimiter marks. The clip start mark starts with the character { followed by the clip name and then the character }. The clip end mark is similar, except that it starts with the characters {/.

Template marks

Inside a template clip, there may be other named marks that also start with the character { followed by the mark name and then the end character }.

Template clip marks serve as place holders that determine where it will be placed certain types of information that is evaluated by the template processing engine. For instance, template clip place holder marks are used to determine where each field of the form is going to appear within the template data.

Conditional template sections

Templates may also have conditional sections. These sections are only processed depending on whether a specific mark is defined. If the conditional mark is not defined, the whole section defined by the conditional marks is skipped as if that template section was not there.

Conditional sections are delimited by two equal marks that start with {? followed by the conditional mark name and then the end character }.

Input fields, labels and marks

Any type of elements may be inserted in the template place holder positions, but the most important type of elements is the form input fields.

Along with input fields, it may be inserted field labels and marks. Input labels are meant to tell the user what is in each field. Input marks are meant hint the user about whether the field is required, and so needs to be entered, or the field is invalid, and so it needs to be verified and eventually corrected.

The place holder mark name for input fields is the input name followed by the suffix input. For instance, if the input name is somefield, the place holder mark is {somefieldinput}.

Similarly, field label and mark place holder names should have the suffixes label and mark respectively, when they are used in the template.

Template example

Here is an example of a template clip data section named form:


 {form}
 <p>{title}<br/>
 {?errormessage}<b>{errormessage}</b><br/>{?errormessage}
 {somefieldlabel}: {somefieldinput} {somefieldmark}<br/>
 {submitinput}</p>
 {/form}

Form expressions

Before describing the different aspects of the definition of a form, it is important to understand the concept of form expressions.

Form expressions define values that are evaluated when the form class and data structures are generated. These may represent things like variables of the classes or return values of functions that are called at run time to retrieve the values to be used in the forms.

Form expressions may be string values or boolean conditions. Each expression is specified by a tag with a unique name.

String expressions

String expressions may be concatenated with other string expressions or interleaved with character data, forming a longer string expression.

Here is the list of the currently supported string expressions:

errormessage

errormessage represents the message that describes the validation error associated to the first field of the form determined to be invalid. It is usually used to present to the user to explain why the form was not accepted. Error messages may be stored in an automatic form class variable defined in the validation error messages theme section.

Example:


 <errormessage/>

formvariable

formvariable represents the value of string custom form variable from those specified in the theme definitions. A form variable expression takes as data argument the name of the variable that is meant to be represented.

Example:


 <formvariable>images</formvariable>

formtext

formtext represents one message to present to the user. That message is eventually in an idiom set according to the current user locale.

Form text messages are defined by the application that uses the form class. These messages are stored in an hash array form class variable named text.

A form text expression takes as data argument the name of the text message that it represents. That name is used as index to retrieve the message from the text hash variable.

Example:


 <formtext>title</formtext>

skinclip

skinclip represents a text data segment extracted from the template file defined in the form skin.

A skin clip expression takes as data argument the name of the skin template file clip that is meant to be extracted.

Example:


 <skinclip>backgroundcolor</skinclip>

conditional

conditional represents a string expression that is evaluated at run time choosing between two possible expressions. The choice depends on the value of a boolean condition also evaluated at run time.

A conditional expression takes as arguments a boolean expression and two string expressions. The boolean expression may be specified within either the tags istrue or isfalse, depending on whether it is intended to evaluate if condition is true or false.

The string expressions are defined within the tags then and else. Each defines the values that will be used in case the condition is true or false. The expression correspondent to else case may be omitted if corresponds to an empty string.

Example:


 <conditional>
  <istrue><errormessageset/></istrue>
  <then>Error: <errormessage /></then>
  <else>No validation error</else>
 </conditional>

objectvariable

objectvariable represents the value of text variable of a specified object.

A object variable expression takes as arguments another expression that identifies the object that is meant to be accessed and the name of the text variable to be retrieved at run time.

The object and the variable name are specified within the tags object and variable respectively.

Example:


 <objectvariable>
  <object><formvariable>parent</formvariable></object>
  <variable>name</variable>
 </objectvariable>

roleimage

roleimage represents the file name of an image of a given role from those specified in the theme definitions. A role image expression takes as data argument the name of the role of the image that is meant to be used.

Example:


 <roleimage>cancel</roleimage>

Boolean expressions

Currently, boolean expressions may not combined with any other type of expressions or data.

Here is the list of the currently supported boolean expressions:

errormessageset

errormessageset represents a condition that determines whether the error message variable was set due to an form field with an invalid value. The evaluation of this condition is done at run time by verifying if the error message variable is set to a non-empty string.

Example:


 <errormessageset/>

verifyvariable

verifyvariable represents a condition that determines whether it is invalid a field of the form associated to the specified variable. A verify variable expression takes as data argument the name of the variable that is meant to be verified.

Example:


 <verifyvariable>description</verifyvariable>

formvariableisnotnull

formvariableisnotnull represents a condition that determines whether it is not null (is defined) a custom form variable from those specified in the theme definitions. A form variable is not null expression takes as data argument the name of the variable that is meant to be checked.

Example:


 <formvariableisnotnull>parent</formvariableisnotnull>

Form name

(Required) Each form must have a name. This the prefix of the name that is given to the form handling class that is generated.

Form type

(Required) Each form is of a type. Currently, only forms of type createobject are supported. See the section on form type specific parameters for more details on the supported form types.

Form theme definitions

Themes are groups of settings that define details of presentation and behavior of the forms that are generated.

Each form may have multiple theme definition sections. However, some theme settings may not be defined more than once in all the theme sections.

A form theme definition section starts with the tag theme.

You may also create individual theme definitions in separate XML files. For that you need to use the tag themefile specifying the theme definition file name as value, like in the following example. The contents of the theme definition file are same as for theme sections defined within the component file. The root tag of the theme definition files is also theme.


 <themefile>newsflash.theme</themefile>

Form variables

The generated form classes may have custom variables. These additional variables may be set by the applications to define values to be used by the form classes for customization purposes.

Form class variables should not be confused with the class variables of the data objects that the forms are intended to manipulate.

A form variable is defined in a theme section starting with the tag formvariable.

Variable name

(Required) Each form variable must have a unique name.

Variable type

(Required) Each form is of a type. The variable type is defined by the tag type except for object variables that have their types defined with the class tag.

Variable default value

A form variable may have a default value. This is the initial value that the variable will have when the form class object is created. If this value is not defined, the variable will be undefined.

Presentation template

Although the definition of the values to insert in the template place holders is done in a separate section named page, a theme may contain additional definitions for certain template place holder values.

Template definition sections start with the tag template.

Template variables

Template variables define string expressions that are used in replacement of the template marks of specified names.

Template variables definition sections start with the tag variables.

Template conditional variables

Template conditional variables define a condition boolean expression and one or two conditional string expressions values. These values are used in replacement of the template marks of specified names, depending on the value of the condition.

If there is a conditional section with the name of a given conditional variable, the content of the conditional section is only outputted when the condition is true.

Template conditional variables definition sections start with the tag conditional.

Condition

The condition boolean expression may be specified within either the tags istrue or isfalse, depending on whether it is intended to evaluate if condition is true or false.

Conditional values

The conditional string expression values have to be specified within the tags then and else, for defining the values either for when condition is true or false respectively.

The else value may be omitted. In that case, an empty string value is assumed as default.

Automatic layout

The layout of the form elements may be designed manually by a developer or a presentation designer, but to gain time, there is also the possibility to have the form templates generated automatically from simple layout hints.

The definition of automatic layout hints is done in a theme section that starts with the tag layout.

Layout type

(Required) Currently, the layout type is the only automatic layout hint that is supported.

The only type of layout that is supported for now is vertical. In this type of layout, the fields of the form are displayed vertical column of field rows. Each row has a form field with labels on the right side and field marks on the left, if any.

The type of automatic layout is specified with the tag type.

Presentation skin template

Skins are special purpose templates that involve the body of the form when they are used. Skin templates are like other templates but they have a special mark that determines the place where the form template body is inserted.

The definition of skin templates is done in a theme section that starts with the tag skin.

Skin template file

(Required) The definition of the skin template file is done with the tag file.

Skin template clip

The definition of the skin template clip name is done with the tag clip. This definition is optional. The default clip name is form.

Skin body template variable

The definition of the skin template body variable is done with the tag bodyvariable. This definition is optional. The default body variable name is body.

Validation error messages

When a form is validated, there may be fields with invalid values. For each invalid field there is an associated error message that may be displayed, so the user can understand why the field value was not accepted as valid.

For that purpose, the generated form class may have a special variable that is used to store the error message associated to the first invalid field.

The definition of error message variables is done in a theme section that starts with the tag errormessage.

Error message form class variable

(Required) The definition of the validation error message form class variable is done with the tag formvariable.

Template conditional variable

An error message may be displayed when the form was submitted to the server and it has some invalid fields.

The error message template conditional variable specifies the name of the template mark that defines where the error message will appear. It also defines the name of conditional section of the template that will be displayed only when the validation error message is set.

The definition of the template conditional variable name is done with the tag templateconditional. When this definition is omitted, the error message is not displayed in the form.

Field marks

To help the user navigation in the generated forms, each field may be displayed along with special marks that denote whether it is a field that requires user input, or it is a field that needs to be verified and corrected by the user, as it was submitted with invalid values.

Field marks are defined with template data that is processed when the form output is composed.

The definition of field marks is done in a theme section that starts with the tag marks.

Required field mark

The required field mark data is usually displayed near by every field that needs to be entered by the user.

Usually this mark data is defined using a template clip. The definition of the required field mark data is done with the tag required.

Invalid field verification mark

The invalid field verification mark data is usually displayed near by every field that did not pass the validation checks when the form was submitted. If an invalid field is also a required field, only the field verification mark is displayed.

Usually this mark data is also defined using a template clip. The definition of the field verification mark data is done with the tag verify.

Presentation default styles and style sheet classes

Since the output of the form elements is only composed when the template is processed, there is no way to define the elements presentation style details directly in the form template clip. To overcome this limitation, the elements fields presentation details may be defined separately in theme sections.

Presentation styles may be defined either as individual style properties or style sheet classes defined according the CSS (Cascaded Style Sheets) standards.

Individual style properties are more appropriate to generate form output that is independent of the style sheets used in the whole page. Style sheet classes are more appropriate to generate more compact form output.

There may be different theme sections to define either style or style sheet classes. These sections start respectively with the tags style and class.

Input styles

Currently, the only form elements that may have their presentation styles defined separately are the input fields.

The definition of input styles is done in a section that starts with the tag input.

Inside an input styles section it must be included the definition of the styles for each type of form input. Such definition should be in the form of a tag with the name of type of form input. That tag data should be the definition of the input style or name of the style sheet class. Currently, only text input fields are supported.

Graphical images

A theme may specify use multiple graphical images to better compose the presentation of the generated forms. Each image file that is used must be declared so it is copied to the installation directory when the form classes are generated.

Each image file that is used must be declared in a individual theme section that starts with the tag image.

Image file

(Required) The definition of the image file name is done with the tag file.

Image role

Each declared file may have an optional role identification. This identification may serve to determine automatically the name of an image file included for a specific purpose. The image role is usually referenced when using roleimage string expressions.

Currently, the only supported image role is cancel. Images with this role are meant to be used in form canceling submit buttons.

The definition of the image role is done with the tag role.

Images directory path

To use an image file in Web pages, it is necessary to specify its URL. Usually the URL consists of the image file name preceeded by the URL of its directory path. That may be either relative to the current page or to the current document root directory.

The URL of the images directory path may be hard-coded in the generated form classes, but usually it is more convenient to be defined at run time, for instance in a form class variable.

Regardless what may be more convenient for your application, you may define the image directory path URL in a theme section using the tag imagespath.

Pre-defined themes

To simplify the development, Metastorage already provides a few pre-defined themes that give you a quick choice between alternative options of look and feel for the generated forms.

Use the tag themename to specify the use a pre-defined theme in your form definitions. Currently there are only two predefined themes available: 9x and xp.

The list of pre-defined themes is in a file named themes.themes that is located in the themes sub-directory of the Metastorage installation directory.

To add more pre-defined themes, you need to add a new theme sections to themes.themes file starting with the tag theme.

Inside each new theme section there should be only two tags: name and file. name should be a unique name for the pre-defined theme. file should the path of the theme definitions file relatively to the themes directory. The new theme definitions file should follow rules for separate theme files. All the files that belong to a new pre-defined theme should be located in a sub-directory of the themes directory with the same name of the new theme.

Here is an example of a new pre-defined theme section named bold. It needs to be added to the themes.themes file to be usable by its name in form definitions.


 <theme>
  <name>bold</name>
  <file>bold/bold.theme</file>
 </theme>

Form pages

A form may present its fields in one or more Web pages. Currently, only single page forms are supported, but in future versions of Metastorage, it will be possible to split the fields of a form between multiple pages, giving to the user the ability to navigate between them.

The definition of the form page structure is done with the tag page.

Page template

The layout of each form page is defined using individual templates. The definition of page form templates follows the same structure and has the same default values as the presentation templates defined the theme sections.

The only required parameter is the page template file for forms with custom layout definitions. In this case, the file parameter specifies the path of the file that contains the form template clip to be used. For forms with automatic layout, Metastorage generates a template file automatically, so it is not necessary to specify the page template file path.

Form type specific parameters

Currently, Metastorage only generates one type of form. However, it will support multiple types of form later, each one with a specific set of parameters.

The definition of the form type specific parameters is done in a section that starts with the tag parameters.

`createobject`

The createobject type of forms are meant for creating new objects of a given class from user defined data.

Form object

(Required) The object definition section specifies details associated with object of the class that is meant to be created.

The definition of the form object details is done in a section that starts with the tag object.

Object class

(Required) The definition of the form object class is done with the tag class.

Object variables

(Required) For each variable of the class of the object that is meant to be created, there must be a definition of the details that specify how the respective fields will be presented in the form.

The name of the template marks that correspond to each object variable form field is based on the variable name. For instance, if you have an object variable named description, the field mark for the input is {descriptioninput}, for the label is {descriptionlabel} and for the marks is {descriptionmark}.

The definition of the form object variables is done in sections that start with the tag variable.

Variable name

(Required) The definition of the form object variable name is done using the tag name.

Variable default value

A variable field may have a default value set by the application at run time. This can be achieved by specifying a form class variable that should be used to specify the default value. The form class variable has necessarily to be of a compatible type.

(Required) The definition of the form variable default value is done using the tag defaultvalueformvariable.

Variable input label

Form fields may have optional labels that are displayed near them. The label text is a string expression, possibly defined in the current local

Metastorage Documentation

Contents

XML definition format

Database schemas

Automatic generation of all classes

Self-contained code

Object Query Language

No SQL

Forms

Reports

UML Diagrams