Page version
Contents
- What is Metastorage?
- What does Metastorage generate?
- What are the advantages of using Metastorage?
- Does Metastorage completely replace the work of a software developer?
- How does Metastorage work?
- How good is the code generated by Metastorage?
- Does Metastorage generate bloated code?
- Is it possible to tune the performance of object queries?
- What languages are supported by Metastorage?
- When will my favorite programming language be supported by Metastorage?
- Do I have to know the MetaL language to use Metastorage?
- Can I customize the code generated by Metastorage with my own handwritten PHP code, in a way that my customizations are not lost when Metastorage generates the code again?
- How can I try to use Metastorage?
- Where can I see an example of how to use the code generated with Metastorage?
- Do I have to use the generated forms handling classes to manipulate my class objects?
- What is current state of development of Metastorage?
- What features are planned for future versions of Metastorage?
- Where can find more about code generation from high level models?
- Who has created and developed Metastorage?
- When Metastorage started to be developed?
- How much does Metastorage cost?
- What motivates the development of Metastorage?
- How can I obtain support to develop with Metastorage?
- What can I do to contribute to Metastorage development?
- What is Metastorage?
Metastorage is an application that automatically generates code
for an Object Oriented API to store, retrieve and manipulate the
persistent objects of classes described in a high level data model
definition.
Metastorage provides a much more efficient development process for medium
or large size applications that store and retrieve data from SQL based
databases or other types of storage persistence containers, like XML files,
flat file databases or LDAP servers.
When storing objects in SQL databases, Metastorage generated code performs
efficient object-relational mapping. Metastorage antecipates static
storage optimization decisions at compile time, so applications gain in
performance, code size and memory usage by avoiding evaluating static
decisions at run time. |
- What does Metastorage generate?
Metastorage generates data object access classes. Currently it generates
code in PHP. The data objects are stored
in SQL based relational databases. In the future, other languages and
storage containers will be supported.
Metastorage takes a description of the information of the entities of
your application data model defined in a XML based format named
CPML (Component Persistence Markup
Language).
The data model information consists of the definition of a set of
classes of objects with variables, validation rules, relationships and
functions that are needed to access the data to be stored and retrieved.
Metastorage generates classes of objects in your programming language
of choice. The generated classes make an API that you can use
immediately to build your application.
Metastorage also generates classes that can install or upgrade your
database, as well classes to create, validate and process
forms that a provide
Web based interface to access your data objects, ready to use in your
applications. |
|
Metastorage can also generate
class diagrams using UML
notation for the generated classes. The generated diagrams can be
rendered in many image formats.
You can use the UML diagram images in the documentation of your
project. The diagrams also help to explain the implementation of your
project to everybody involved in it, from the software developers to
the clients or sponsors the project. |
|
|
- What are the advantages of using Metastorage?
There are three main advantages of using Metastorage over traditional
development methods that are less automated:
- Development speed - Reduce development time drastically
Metastorage is capable of generating code from your component model
definition in just a few seconds. It would take days or weeks to write,
test and debug equivalent code manually by a human developer.
This is very important to reduce the time to market your
software products, especially if they are medium or large size projects.
- Software quality - Produce optimized bug free code
Metastorage analyzes all the hints that are provided in your component
class definitions and makes conceptual optimizations to generate code
that is fast and compact.
The generated code is as bug free as Metastorage itself. Since
Metastorage has been thoroughly tested and it will be even more tested over
time, this means that the generated code is virtually bug free.
Since it is programmed generator, Metastorage will generate bug free
code consistently every day. It is reliable because it is not
susceptible to moods or distractions that affect human programmers and make
them produce code with bugs inadvertently.
- Flexibility - Persistence container and language independence
The definition of the data model used by Metastorage is independent of
the type of container that is used to store the data persistently.
This means that Metastorage can generate code that stores and retrieves
the data objects from different types of storage, like SQL based database,
XML files, flat file databases, LDAP servers, etc..
The generated API to access the different types containers is the same,
making it possible to switch the storage of part or all data object
classes, without the need to change your application.
This also means that the generated API is independent of the database
type that is used to store the persistent objects when a SQL database
container is used.
|
- Does Metastorage completely replace the work of a software developer?
Absolutely not! Metastorage is meant to automate the tedious task
of generating code to implement a design specification. The generated code
follows well known design patterns. That leaves the software developer with
more time to concentrate on software development tasks that require an high
level of knowledge and creativity, which are proper of human beings.
It would be hard, if possible at all, to replicate with any advantage
the human intelligence. That is definitely not the focus of Metastorage.
With a tool like Metastorage, the software developer will be more
concentrated on designing the software models and implementing the
business rules that address the requirements of the projects.
Metastorage only takes care of generating code that implements an API
based on a specification defined by the software developer. This way,
the software developer can be more productive.
|
- How does Metastorage work?
In reality, Metastorage is just a wrapper script around the
MetaL compiler engine. It takes the parameters
passed in script command line and forwards them to the MetaL compiler. The
parameters are used by specific MetaL compiler modules that take care of
generating code and data files based on the specified component definition.
Most of the work done by Metastorage is performed by the MetaL compiler
persistence module. The whole Metastorage build process can be divided in
two stages: generation and compilation.
- Generation
In the first stage, the MetaL persistence module parses the component
definition file to build the definition data structure in memory. Then it
validates it by verifying the consistency of the interdependent parts.
Finally, it generates all the code of the classes the MetaL language
source files. In this stage it also generates database schema definition
files, UML schema diagram graph definition files, and eventually, output
template files when automatic layout forms are requested.
- Compilation
In the second stage, the class source files generated earlier in the
MetaL language are compiled into source files of the chosen target
language.
In this stage, other modules of the MetaL compiler engine are used, such
as the modules for generation of object oriented programming classes,
database access code, forms generation and validation, output templates
processing, execution flow, variable and expression processing, etc..
During the actual compilation, these modules implicitly validate the
MetaL language source code files that was generated in the first stage. If
there is a bug that leads to the production of code with incorrect syntax
during the generation stage, the whole Metastorage build process is halted
throwing an error.
This circumstance has been helping the meta-meta-programming driven
code generation process to self-validate very early during the development
of Metastorage. This fact contributes to generate highly reliable code that
can be trusted.
|
- How good is the code generated by Metastorage?
The code generated by Metastorage can be as good as code written by hand
by a human developer because the generator was also hand written by a human
developer.
The generator is not based on static templates, XSLT (XML Style sheet
Transformations) or any other method that is not self-validating. The
generator produces MetaL language source code that is validated and
compiled before it is outputted in the target language. Therefore it
reflects the best of the knowledge of the Metastorage developer to write
code in the supported target languages.
Over time the generated code will be reviewed iteratively. This makes
the developers evaluate the code over and over again to think about better
strategies to implement the tasks meant to be executed by each fragment of
generated code.
If a better strategy is found, it can be either be implemented
permanently in a new version of Metastorage or be made available as
alternative code generation option. In the former situation, the users of
Metastorage can automatically benefit of the newer approach just by
rebuilding the code generated by Metastorage. In the later situation, the
previous generation option is made the default and the users of Metastorage
may explicitly switch to the new generation alternative approach if they
find it convenient.
The users of Metastorage are encouraged to question and propose new
generation approaches for any parts of the code produced by Metastorage.
This way, Metastorage may evolve faster leading to a greater level of
satisfaction.
|
- Does Metastorage generate bloated code?
No, Metastorage performs extensive analysis to determine which segments
of code are needed considering the types of operations that the developer
that designed the component model wants the data object classes to
perform.
This way, Metastorage does not generate code that is not necessary. The
resulting classes are very compact.
|
- Is it possible to tune the performance of object queries?
Yes. Metastorage provides an
Object Query
Language (OQL) that lets the developer define filters that
specify search conditions that can be associated to the actions that
retrieve objects.
The OQL expression conditions are pre-compiled and embedded in the
generated code to maximize the object queries execution performance.
Metastorage OQL is very similar to SQL. It just has a XML syntax.
Developers that are familiar with SQL are able to tune their object
queries in the same way they tune SQL queries.
|
- What languages are supported by Metastorage?
Although Metastorage can generate code in any languages supported by
MetaL, currently, Metastorage only supports the generation of code in PHP.
The reason for this is that the code generated by the database and forms
modules of the MetaL compiler relies on components that are only available
in PHP.
|
- When will my favorite programming language be supported by Metastorage?
There are plans to evolve the MetaL modules so they support other
languages besides PHP. Adding support to generate code in other languages
will happen when the relevant MetaL modules supports such languages. There
are no short term plans to support other languages unless somebody
contributes with the necessary code.
|
- Do I have to know the MetaL language to use Metastorage?
No. MetaL is just a means to generate code in a target language
of your choice. The generated code can be used in your applications like
any other code of the programming language that you use.
Metastorage uses a XML based format named
CPML to define the components that it
generates, but this format has nothing to do with normal programs in the
MetaL language.
|
- Can I customize the code generated by Metastorage with my own handwritten PHP code, in a way that my customizations are not lost when Metastorage generates the code again?
Yes. You should never need to change the code generated by
Metastorage, though.
Metastorage lets you define customized functions with your handwritten
PHP code that is integrated in the component definition. This way, every
time Metastorage generates the code, the classes always include your
custom functions.
|
- How can I try to use Metastorage?
Metastorage is available for download in the
MetaL download page. It comes with
documentation and examples of use. You also need to download and install
the MetaL project archives to use Metastorage.
You are recommended to read the
Metastorage tutorial to learn how
to install it, design components, generate code and UML class diagrams and
use the generated code in your applications.
|
- Where can I see an example of how to use the code generated with Metastorage?
- Do I have to use the generated forms handling classes to manipulate my class objects?
No, if you need something different than what the form handling classes
generated by Metastorage can provide, you can write your own user interface
code and still use the data object classes generated by Metastorage to
retrieve, validate, store or delete your persistent objects.
The current support for generating Web forms to access your class objects
is limited. It is meant to implement the most common types of user
interfaces based in Web forms. So, it is not unusual to need to develop
special user interfaces to address specific needs of your projects.
|
- What is current state of development of Metastorage?
The current list of features of Metastorage is available in its
documentation.
The progress of the Metastorage project is also recorded in a XML/RSS
based feed available on the URL below. This feed includes not only the
newly added features, but also the features that were added to the to do
list.
http://www.meta-language.net/metastorage-progress.xml
If you use a browser capable of rendering XML documents with XSL, you
can see a list of projects items sorted by the date when they were
updated. The items that are marked as done as those that are
already implemented.
|
- What features are planned for future versions of Metastorage?
The features that are planed for future versions are listed in the feed
that makes available the progress of Metastorage. The
items of the feed that are marked as todo as those that are
planned to be implemented in the near future.
Apart from those that are listed, there are some major features that are
more relevant because they constitute significant enhancements to current
version. Here follow some of the planned major features:
- Object editing forms
Currently, form handling classes only support one type of forms that is
just meant for creating new objects. Another important type to be supported
will allow the creation of forms for editing and deleting previously
created objects.
- Multi-page forms
The classes of objects may have as many variables as needed. For large
classes it may not convenient to display all the form fields for all the
variables at once. Splitting the variables in groups of fields that are
displayed in different pages may be more convenient. In that case there
will be additional buttons to allow the navigation between the different
form pages.
- Commenting and documenting the generated code
Some developers only use code that they fully understand. In the future,
each block of code that is generated will have inline comments that provide
detailed explanations about the purpose of the relevant lines of code.
Currently, MetaL already has the capability of generating documentation
in printable formats from the source of the classes. It only requires that
some special purpose documentation sections be present in the declaration
of variables, functions and arguments. Multiple documentation sections may
be defined to generate documentation of the same class in different idioms.
In the future, Metastorage will generate classes already built with
documentation sections, so MetaL can automatically generate documentation
from the classes' source.
- Report generation classes
The object model is useful to provide an interface to manage data
entities. However, when you need to generate reports or do a batch
processing of large amounts of data that may eventually need to be combined
from multiple classes of related data objects, the object oriented mappings
are usually inefficient.
A better solution for this purpose would be to retrieve data in tabular
format, consisting of multiple rows of data returned in a predefined
number of columns, just like it is stored in database tables and query
result sets.
In the future, Metastorage will generate classes that retrieve data in
tabular format composed from objects of related classes just like in SQL
queries. The retrieved data may be formatted to compose reports that are
suitable for presenting to the users of your applications.
- Web services provider and consumer classes
The standardization of the protocols for providing access to remote
services over the Internet, turned the Web services more appealing to share
information or provide programmatic access and control of resources that
are based on the manipulation of data distributed over remote servers.
Web services that use popular protocols like SOAP and XML-RPC work based
on the exchange of request and response messages between the servers that
provide the services and the clients that consume them.
In the future Metastorage will be able to generate classes that
encapsulate the functionality that is necessary to process and handle Web
service requests that manipulate data objects. Additionally, it will also
generate classes that will be able to send the requests and collect the
responses to implement Web service consumer applications.
|
- Where can find more about code generation from high level models?
Code generation is not a new technique. There are many other tools that
resort to code generation to achieve goals similar to those of Metastorage.
One site where you may find a lot of information about code generation
techniques and tools is the Code
Generation Network. Here you will find a database of existing database
generation tools for various purposes that support many different languages.
The approach that Metastorage takes of generating code from high level
specification is known as meta-modeling. One site that is dedicated
exclusively to this approach is
Metamodel.
|
- Who has created and developed Metastorage?
- When Metastorage started to be developed?
Metastorage development started in the September 2002. However, a
significant part of Metastorage is based on MetaL compiler modules that
started to be developed around middle of 1999.
|
- How much does Metastorage cost?
- What motivates the development of Metastorage?
Despite Metastorage is Open Source, it requires a great investment of
time and skill to research and develop. Making it Open Source was just a
means to make it subject of as much testing and criticism as possible. That
would help to find any design or implementation flaws as soon as possible.
However, opening the source of Metastorage was not an absolutely
necessary step as that was not the main motivation of its development. The
main motivation to develop Metastorage was to have a tool that would
drastically reduce the time to market of medium or large size applications.
While it is being developed, Metastorage generates code that is being
used in the implementation of services that will be offered to the users of
the PHP Classes site. Such
services will constitute a package that will be part of the paid
subscription offer that has been planned in 2002.
The current free services that the PHP Classes site provides to the
users will always remain free. The paid subscription services will generate
the necessary income to keep the site open, while more free services will
be provided for the joy of its already large community.
The main challenge of implementing the services that were planned is
that they require significant development effort. The idea of
Metastorage is to help reducing drastically the development time, so the
site can survive until it starts generating enough income to keep it open
with many new developments that were planned but there was never time to
make them happen.
This will also foster further development of Metastorage itself,
incorporating more and better capabilities that will address the needs of
more and more developers.
|
- How can I obtain support to develop with Metastorage?
Metastorage is mostly a derivative part of the MetaL project. So, for
now you can obtain support by placing your queries in the
MetaL mailing list.
|
- What can I do to contribute to Metastorage development?
One of the advantages of opening the source of a project is that you
provide opportunities that let others fix any bugs and improve the
project features. Sending bug fixes or improvement patches is one of the
ways to contribute to the Metastorage project.
If you do not have the time or you do not feel capable to contribute
with any code, you will also be contributing if you just report any bugs
you find or suggest any features. In any case, use the
mailing list to start contributing to the
project.
Another way to contribute is to write articles, tutorials or use case
presentations. Then you could share them with other users, so they can
learn faster about how to take advantage of the capabilities provided by
Metastorage.
Metastorage has a lot of potential but to take advantage of such
potential there are several types of complementary projects that could be
implemented in parallel by independent developers.
Here is a list of suggestions of possible independent projects that can
improve the potential of Metastorage:
- Metastorage designer GUI tool
A GUI based tool that could make the creation of CPML component files
more intuitive and more productive would be a very interesting project for
the Metastorage users. Since CPML files are written in XML, such tool could
be a standalone GUI application written in any language, or a plug-in
component for an existing IDE like for instance
Eclipse.
- Port Metabase and forms packages to other languages
Metabase is the API
that is currently used by MetaL database interface module to generate code
that accesses databases using SQL. Metabase API provides a way to write
database independent applications. It also provides a way to create and
maintain database schemas without the need to write any SQL code by hand.
The forms generation and
validation package is a class used to compose and validate HTML forms.
It generates automatically Javascript code with the forms HTML output to
perform several types of validation on the client side. The class itself
can perform the same types of validation on the server side.
Currently, Metabase and the forms packages are only available in PHP.
Porting these packages to other languages would enable Metastorage to
support more target languages, making it useful to more developers.
- Reverse engineer database schemas
Some developers will have the need to migrate existing database
applications to start using Metastorage and develop those applications with
greater productivity.
For such developers it would be useful to provide a tool that analyzes
the schema of the application database and performs reverse engineering to
generate an equivalent CPML definition of classes and relationships from
the correspondent database tables and keys.
Additionally, it could be provided a tool that converts from Metabase
schema definition format to CPML. Metabase API also provides database
schema management functions that make it possible to generate a schema
definition from a database that may have not been created by Metabase
schema installation functions. This way, the migration from an arbitrary
database to CPML could be simplified.
Conversion between UML standard formats to CPML Already done by Peter Howard
UML is a standard visual language
that is often used to design object oriented software systems. There are
many applications that provide a graphical user interface to model software
systems using UML, like for instance
Umbrello and
Argo UML, which are Open Source
programs.
UML modeling tools are usually able to export models in
XMI
format. This is a XML based format meant for interchanging model
meta-information.
A conversion tool could extract class model data from XMI files and
convert it to the CPML format. That would allow the users of Metastorage to
design their classes with UML modeling tools.
Ideally, such tool would be able to convert new models from scratch or
merge a newly edited model with a CPML file generated from merge previously
converted model that was possible edited manually by a developer.
- Forms presentation themes
Metastorage can generate forms to provide a Web interface to manage
objects of the generated data classes. The output of the forms is based on
re-definable themes that specify HTML templates, graphics, frame skins,
colors, fonts, styles, among other aspects of the forms presentation.
Presentation themes provide great flexibility to improve the forms
usability and make them visually more appealing to the users of the
applications developed with Metastorage.
The number of predefined themes distributed with Metastorage is
limited. Each developer may prefer use other themes. Anybody can design new
themes following the specification detailed in the
Metastorage documentation.
New themes may be contributed to Metastorage, so other developers can reuse
them in their applications.
|
|