Mnemonic Mnemonic Core
General Info

Introduction
Screenshots
Mailing Lists and IRC
Alternative Browsers
Special Thanks

FAQ
Understanding Mnemonic
TODO list and ideas
Bug Reports


User Info

Download binaries
Platforms
Compiling Mnemonic
Other useful software


Developer Info

Core
Message modules
Library modules
Object modules
Coding Guidelines
Browse Source
Using CVS


View with any browser

Website questions to:
webmaster@mnemonic.org

Mnemonic questions to:
disc@mnemonic.org

 

Overview

The mnemonic core is a small and efficient (one shared library of less than 200 Kb) yet flexible messaging system. It provides a very clean way to separate implementation from interface. In contrast to virtual members, messages can deal with multiple receivers and support dynamic loading of receiving objects. The result is an extremely modular programming environment. There are some similarities with the OpenSTEP messaging system.

The message system is not tied to the C++ language per se, although at present only a C++ implementation exists. Stubs to extend it to use CORBA for inter-process and inter-machine communication have been on the drawing board and will probably be implemented at a later time.

The mnemonic core is a spin-off project of the mnemonic browser project but it can be used separately without any problems (although, of course, the modules being written for the browser project will likely be of use in other projects as well).

This page is a reference guide for users of the mnemonic core. In short, it describes

  1. Introduction
  2. Libraries, Messages and Message handlers

  3. Sending messages
  4. Receiving messages
  5. Establishing message channels

  6. Exporting objects for use by others
  7. Creating new message types
  8. Reference counting pointers
  9. Threaded execution

  10. Debug output and performance timers

  11. Loading additional objects
  12. Using standard makefiles

Introduction

Using distributed objects is a difficult issue. While on-the-wire protocols and interface description languages are readily available (ie. CORBA), there remain several problems on a higher level which need treatment. How does one find an appropriate object from which to request services? How do they `come to life' and when is it safe to shut them down again? How is it possible to use these techniques for applications that are confined to the local machine, eg. embedding of one graphical user interface in another one? And once communication between objects is set up, how can one extend or filter it? These are some of the main issues that the mnemonic core tries to address. Programs using the mnemonic core consist of various small modules, object implementation libraries (or OILs for short), which communicate with each other by sending and receiving messages. In CORBA-speak, one could say that the messages provide the interface and contain the data and return values (which is always passed by value) while the senders/receivers are the objects that talk to eachother. There is really not all that much difference with CORBA conceptually at this point. The message system is the backbone of the dynamic extensibility of Mnemonic. The normal virtual function / abstract interface approach in C++ is too limited for true extensibility. Thus, the mnemonic core uses message passing which provides for "multiple" receivers while still preserving the benefits of virtual functions and abstract interfaces.

The data flow in a mnemonic program is a bit unusual, but specially tailored to enable easy extension of the program's capabilities. The usual procedure is to setup an interest in certain `data' messages first, and then send another message which starts the data flow. Using this mechanism, other objects can listen to the same data messages.

Upon receipt of a message, an recipient has three options. The first is to handle it and afterwards inform the message sender to pass the message on to the next recipient. The second is to handle it and signal the sender to stop sending it to other recipients on the list. The third one is to refuse the message, in which case it will automatically be passed on to the next recipient (this is somewhat like the NextSTEP mechanism).

At any moment, multiple threads can be sending messages to any given recipient. These calls are always made through the interests of the recipient. Similarly, a dynamic object may export parts of itself (or objects created by it) to the external world by putting a pointer in a message. All this `external' use of an object is tracked through reference counting. A dlobject is supposed to use reference counted pointers (available in the core) to keep track of the number of references to its interests and its exported objects. The destructor of a dlobject should wait for the reference count to drop to one before destroying itself.

Libraries, Messages and Message handlers

The way I think about this distinction is that OILs get activated by receiving messages while libraries provide C++ classes that can be used by one or more OILs directly. For instance, oil-gtkrenderer is a module that gets activated when a new in-memory representation of a CSS-styled XML/HTML document is announced, and it uses lib-gtklayout classes as a sort of data containers (plus algorithms, of course) to store the layout-tree (and passes these containers to other OILs, for instance oil-gtkclient, through messages).

So think of libraries as providing objects that can be shared between OILs, while OILs themselves are the stand-alone building blocks that pass around these objects through messages.

For historical reasons, message handlers are called Object Implementation Libraries (OILs).

A common way to implement a message system is to have one global message manager object to which messages can be sent, and which is responsible for forwarding them to the interested receivers. This however makes it necessary for all objects to have carry a label, eg. implemented through a `isA()' member which returns an integer or enum. This is what is called a brittle construction in the C++ FAQ; you can read about the disadvantages there. The bottom line is that mnemonic core does not have one global message manager. Instead, each message type contains a static member object (ie. this object is shared among all instances of the message) which is templated over the message type, and which handles all interest registration and performs the actual distributing of messages.

Sending messages

Sending messages is very easy, though one has to take proper care of handling any exceptions that might be thrown by the message system. A short code excerpt could be

msg_foo foomessage(/* parameters of the message */);
try {
   foomessage.send();
   }
catch(messagebase_error &ex) {
   switch(ex.status()) {
      case messagebase_error::no_recipients:
      case messagebase_error::no_handling_recipients:
   }
}
(we have left out the code to handle the exception).

Receiving messages

To make your object `foo' listen to messages of a given type, let us say `msg_foo', your class should create an interest in this message type. This interest is represented by a `interest<...>' object, which is usually created in your constructor. As you need to know how many other threads are using code of your object at any moment, this pointer has to be a reference counting pointer. An example:

#include <mnemonic/objectmanager.hh>
#include <mnemonic/ptr.hh>
#include <msg/foo.hh>

class foo {
   public:
      foo()
         {
         interest_msg_foo_= 
           new interest<foo, msg_foo>
             ::create(this, foo::accept_msg_foo);
         }
      ~foo()
         {
         interest_msg_foo_->unregister();
         interest_msg_foo_.wait();
         }
      untyped_interest::callresult accept_msg_foo(msg_foo &m)
         {
         /* handle the message */
         if(/* we are done and want to unload */) {
            object_manager::instance().unload(this);
            }
         return untyped_interest::call_next;
         }
   private:
      ptr<interest<foo, msg_foo> > interest_msg_foo_;
};
Let us go through this step-by-step. In the constructor, an interest in messages of the type msg_foo is created. We keep track of it using a reference counting pointer `interest_msg_foo_'. From that point on, msg_foo messages sent by other objects will be communicated to us by the message sender calling `accept_msg_foo'.

Some data in msg_foo might tell us to stop listening to this particular message type. If such a situation occurs, we can ask the object_manager to unload the object. The object_manager will start a separate thread and call our destructor in order to do this. In the destructor, we first unregister all our interests (effectively disabling further calls to `accept_msg_foo' to be made). After that, we still have to wait for any ongoing calls (in different threads) to accept_msg_foo which could currently be in progress. This is done by calling the `wait()' member of the reference counting pointer, which will wait until the count drops to zero.

Creating new message types

The separation of interface from implementation is achieved in mnemonic by putting the interface in the message and the implementation in the message receivers. The messages are physically located in shared libraries separate from the OILs. For an example on how to create new message types, refer to the module `mnemonic-msg-demo'.

Establishing message channels

Often, one wants to make sure that a series of messages is handled by one given receiver (ie. when the messages represent the content of one document broken up in smaller pieces). For this purpose, messages inherit from the group_identifiers class, which can store one or more identifiers as defined in midentifier.hh. Listeners can ask for all messages, or only the messages for which the full set of midentifiers of the message is contained in the set of midentifiers of the listener.

Exporting objects for use by others

Apart from communicating by using messages, it is also possible to export a pointer to an object, enabling direct access. The way to do this is to put this pointer in a message requesting the functionality. As we have to keep track of all threads accessing a given object, it is necessary to use reference counting pointers for this.

Reference counting

The ptr<...> is a reference counting pointer. You do not have to make your class inherit from a specific base class in order to be able to use them, as the counter object is kept separate. Note that this does mean that the following piece of code will not work:

#include <mnemonic/ptr.hh>

A *a;
ptr p1;
ptr p2;
a=new A;
p1=a;
p2=a;   // INVALID
p1=0;
p2=0;
This initialises two counting objects. You should always initialise pointers from other pointers except the very first time. So, in the above example, p2=p1 is the correct way of handling this.

Threaded execution

Mnemonic core uses threads heavily. A number of classes wrapping posix threads is available. For detailed information, see threads.

To keep track of the number of threads accessing a given objects, reference counted pointers should be used. These can be found in `ptr.hh' and examples of their use are available in mnemonic-msg-demo and mnemonic-oil-demo.

Debug output and performance timers

The mnemonic core contains an efficient mechanism to output debugging information, which can be completely removed from the source with a single compilation option. To use it, your module should contain something like

#define DEBUG_CLASS "mylibrary"
#include <mnemonic/debug.hh>

int variable;
debug("this is debugging output " << variable);
You will have to compile this piece of code with the -DDEBUG symbol turned on (this is normally handled automatically when you configure the software with --enable-debug). If this symbol is not defined, the C++ preprocessor will reduce the debug statement to an empty line.

At runtime, the output of the various modules can be turned on or off based on the DEBUG_CLASS defined in the modules. To turn debugging on, your program has to do something like

#include <mnemonic/debugger.hh>
debugger::instance().turn_on("mylibrary");
The argument of turn_on determines which module's output is turned on (there is a corresponding turn_off call as well). This argument can be ALL in which case the debug statements of all modules become active.

The core also contains a stopwatch class which can be used to determine the time elapsed between two points in the code. In order to use this, do something like

#include <mnemonic/stopwatch.hh>

stopwatch sw;
sw.start();
...  // your code
sw.stop();
cerr << "this took " << sw << endl;
Of course, instead of using cerr, you would probably want to wrap this in a debug call.

Using standard makefiles

The mnemonic core provides lots of makefile rules for common tasks like creating shared libraries, creating rpm and debian packages and setting correct compilation options for templates. They can be found in `Mnemonic.Rules.make' and `Mnemonic.Make.vars', usually installed in the `$(prefix)/share' subdirectory. Refer to the modules mnemonic-msg-demo and mnemonic-oil-demo for examples.

Loading additional objects

All the objects with extension .oil get loaded upon startup of the mnemonic objectmanager (actually, upon calling objectmanager::instance().load_interests). There is however always the possibility to load additional modules at runtime. This is done by calling

object_manager::instance().
    new_object_by_filename("objectname.oil", NULL);
If you do this in response to the acceptance of a message, and if you want the message to be re-broadcast to the group as to make the newly loaded object see it, you can in addition do
return untyped_interest::pass_on;