Mnemonic Layout library (lib-layout)
General Info

Introduction
Screenshots
Mailing Lists and IRC
Alternative Browsers
Special Thanks

FAQ
Understanding Mnemonic
TODO list and ideas
Bug Reports


User Info

Download binaries
Platforms
Compiling Mnemonic
Other useful software


Developer Info

Core
Message modules
Library modules
Object modules
Coding Guidelines
Browse Source
Using CVS


View with any browser

Website questions to:
webmaster@mnemonic.org

Mnemonic questions to:
disc@mnemonic.org

 

Device independent layout library

The lib-layout module contains the algorithms for the computation of the location and size of boxes that make up a page description. It is independent of any GUI toolkit; it is only concerned with the dimensions of the boxes, not with the content. The algorithms largely follow TeX (based on hbox, vbox, parbox and friends), with a few extensions motivated by the XSL/CSS box model (in particular, there are primitives for tables, text wrapping around floats and multicolumn typesetting, which are usually only accomplished through very bulky macro packages in TeX, if at all possible).

There are also classes for fonts and font metrics with full support for the extended metric information available in TeX tfm files. There will be a separate tree-type for boxes that form mathematical formulae.

As of yet, only one GUI dependent layer has been written on top of lib-layout: the gtk-- based lib-gtklayout.

  1. box hierarchies
  2. coordinate systems and numbers
  3. how a box determines its size and position
  4. drawing and re-drawing
  5. viewports
  6. incremental rendering and reflow

  7. horizontal boxes
  8. vertical boxes
  9. paragraph boxes
  10. table boxes
  11. fonts and text boxes
  12. mathematics subtrees
  13. event boxes

  14. performance considerations

Box types and hierarchies

The main layout tree is built from objects derived from box. These can all be found in the layout.hh header file. This tree is also called the geometrical tree as it determines the way in which

In addition, there is the so-called property tree. This tree only contains property_boxes and stores information about color and font settings. All boxes in the geometrical tree have a pointer to a box in the property tree, but this pointer does not have to point to a unique property box; instead, multiple geometrical boxes can share the same properties. Properties can be inherited from one property_box to the other, ie. it is possible for one box to take the properties of another and only modify eg. the font information. This is achieved by having each property_box have a pointer to a parent box (ie. no multiple inheritance). As far as memory management is concerned, there is a global property_manager object in any geometrical tree which keeps a list of all property boxes and is responsible for deleting them when the tree is deleted.

Mathematical expressions are considered to be very much like normal text strings, but instead of having just a string of symbols, the symbols are linked together in another tree-like structure. We could have put every single character of a mathematical expression in its own box, but it turns out to be far more efficient to use a separate box mechanism.

The normal tree type and the mathematical tree type can be mixed at will. See the picture below.

In this example, the root points of normal trees are blue, while the root of mathematical trees are red. The fraction appearing in the mathematical expression has a bit of text as the numerator, which is sitting in a parbox. Whereas the text boxes contain strings of text characters, the math box has a tree of mathematical objects as its contents.

Containers which have their `flat_hierarchy' flag set are special in two ways. Their children are supposed to be interpreted as boxes on the same level as the container. Moreover, they do not do any actual drawing on the screen. Instead, they can implement the `draw_before_children' and `draw_after_children' members to set and unset the graphics drawing parameters. All `flat_hierarchy' boxes must also have their `ignore_geometry' flag set.

Coordinate systems and numbers

All boxes either have a horizontal or a vertical orientation, which is given by the direction of the baseline. The distinction determines which direction has the capability to `stretch' or `shrink'. For example, a bit of space between words has a horizontal direction. The `shift' parameter of a box determines the location of the baseline itself. Boxes cannot stretch or shrink in the direction of the shift parameter.

Boxes can have a preferred width or height that is either a fixed number, a fixed percentage of the parent size, or they can decide to exactly wrap the children.

The sizes are summarised in the picture below:

How a box determines its size and position

The width and height of a box is determined either by the height and width of the parent box, by the height and/or width of the child boxes or by an explicit specification.

For example, the width of a box in TeX is determined as follows:

\hbox{...}take width from children
\hbox to 200pt{...}width specified to be 200pt
\vbox to 200pt{...}width taken from parent
\parbox to .7\hsize{...}width is a percentage of parent box width

In other words, horizontal(vertical) boxes with no specified width(height) take it from the children(parent) box and vice versa. Moreover, once a width or height specification is encountered, this is kept for children as well, unless they override it explicitly. Overriding the size for children can lead to an overfull or underfull enclosing parent box. The stretch and shrink settings are fixed in a local way. That is, the actual values that they are fixed at are determined by the parent box but they do not depend on the parent box of the parent box. Algorithm: if the parent box width is not fixed, all children take their natural width. If the parent box width is fixed, the stretch/shrink of the children is used to make them fit.

All boxes set the location of their child boxes with respect to their own origin. Moving a container box can then be done without any recalculation of the child boxes.

CSS has the concept of `collapsing margins'; when two boxes with collapsing margins adjoin, the space between the boxes is the maximum of the two margins. This is a bad concept to put in the base library. Instead, we should have a generic method to determine the position and size of a box given the specifications of the previous box. In other words, fixing a box is not completely local, we might want to query the previous and next boxes as well.

Alignment. Boxes can set their horizontal and/or vertical position equal to the position of another box. This is done using many-to-one bidirectional maps; each box can figure out which other boxes take their position from it. Before a box is being fixed, it first fixes all those positions of the other boxes. This alignment relation is stronger than the normal positioning rules, but those normal rules still apply for the coordinate that is not fixed (if there is one).

Size hiding. Boxes can hide their width and height for other boxes. This comes in two flavours. Either the hiding box determines which other boxes do not see its size, or the other boxes determine which sizes they do not want to see.

Position dependent parent size. Right now, a parent reports its size as one given number to the child (ie. a vbox reports its width to a child parbox). With the introduction of size hiding, some hboxes present in a vbox can have an invisible vertical size for the following parbox, yet have to show their horizontal size so that the parbox can make things flow properly.

Drawing and re-drawing

A distinction is made between two different box types. The first one are the automatically redrawing boxes. These are boxes that have to be positioned on the canvas once and are automatically scrolled by the GUI toolkit. The second type needs a redraw upon every expose.

The base classes in lib-layout will determine the position of the boxes and the moment when they have to be repositioned/redrawn. The actual drawing is done using two member functions which are declared virtual abstract in the base classes. An excerpt from the class box declaration:

class box {
   public:
      ...
      virtual void position(int childnum, 
                            long x_loc, 
                            long y_loc, 
                            draw_info& inf)=0;
      virtual void draw(long x_loc, 
                        long y_loc,
                        draw_info& inf)=0;
      ...
};

Viewports

Boxes have both an external and an internal size. The external size is the size used to layout the box inside the container. The internal size is the size of the material that the box contains. These two sizes typically differ for boxes that have scrollbars around them. The main viewer window of a browser is such a box, but these boxes can appear as children as well.

Incremental reflow and forced redraw

The concepts of incremental reflow (re-calculating the layout parameters after new boxes have been added to the tree or existing boxes have changed size) and forced redraw (re-drawing the part of the tree that is inside the viewport whenever the content of a box has changed) are strongly related. In general the procedure works as follows.

Horizontal boxes

Horizontal boxes, or hboxes for short.

Vertical boxes

Vertical boxes, or vboxes for short, layout their child elements below each other. Child boxes that have their float property set are treated separately. They align to the left or right side of the vbox. Vertical boxes have the additional property that they can split (for page breaks), though that feature is not implemented yet.

If the vbox has a specified width, it freezes children to this width. Otherwise, children are frozen to their natural width. The vertical position of all children is initially determined by their natural height. At the end, when all children have been processed and when the vbox has a specified height, the positions and heights of all children are frozen so as to fit inside the vbox's height. If the vbox does not have a specified height, the children are frozen at their natural height and the resulting positions.

While iterating over the children, floating boxes can be encountered. All floats are moved as far left or right as possible, but downward if there is not enough space available. Child boxes with the float_right property set are only allowed when the vbox has a fixed width, otherwise they would float to infinity (this is true in left-to-right typesetting mode; for right-to-left mode it is float_left that is only possible when the width of the vbox is specified).

Floating boxes influence the width that is made available to the subsequent children. They are frozen at either their natural width minus the accumulated float's width, or at the vbox's width minus the float's width if the vbox has a specified width.

Paragraph boxes

Paragraph boxes, or parboxes for short, are responsible for putting children next to each other and continuing on the next line when there is not enough space. Construction of individual lines out of paragraphs of text largely follows the algorithms in TeX; some information may be found in `The TeXbook', chapter 14, and `Digital Typography', both by Donald Knuth.

Although those algorithms can wrap text in such a way that the line length depends on the line number, they do not handle wrapping where the line length depends on the actual distance in pixels from the start of the paragraph. Mnemonic's lib-layout has been extended to cover this as well, as this is a requirement for wrapping around arbitrary external boxes (For instance, a floating image might be 5 cm high. The number of lines that actually have to be shortened in order for the text to flow around it clearly depends on the line height, which again depends on the material present on each line; there might be an inlined image that is 3.5 times the line height if there were no image present).

There are three different ways to make the parbox determine its size:

  1. Layout for the maximum available width but use the actual width of the content after line breaking to determine the parbox size (there may not always be a line which fills the entire available width).
  2. Layout for a percentage of the available width and take this width as the parbox width as well.
  3. Layout for a fixed size.

Wrapping around floats. This will be done by a combination of alignment (with a specific element in the parbox) and size hiding (from the parbox). The float has to be inserted as a hbox _before_ the parbox in which it logically belongs. This makes the horizontal position fixed. The vertical position is then fixed as `below the given element in the parbox'. This makes the vertical position available as soon as it is needed (the line following the element). - We introduce one more box type, called a `floatbox'. This box can sit in a vbox, hbox or parbox. Eg \vbox{\floatboxfloatleft{...} \hboxanchor=a{normal stuff} \floatboxfloatright{...} \hboxanchor=b{more normal stuff}\hbox{...}} The floatbox is always top-aligned with the subsequent hbox. A similar pattern holds for hboxes. When parboxes are involved, we start from \vbox{\parbox{...}} The parbox creates a few lines, \vbox{\hbox{...} \hbox{...} \parbox{...} // remains of} and then stumbles upon a child floatbox. It throws it out, which means that the enclosing vbox will now know about the influence of the floatbox on the available line width, and it will also know where to position it (no alignment needed).

Discretionary breaks (again, see `The TeXbook' for explanation) will probably be implemented at a later stage. breakpoint: _if_ you break here, then - line number of just broken line is - its natural height is - best to break via previous point - total demerits

Property boxes

Property boxes are boxes that store information about the subtree which they contain. Think of, for instance, information about font size, or color.

Text boxes and font information

The layout library does not know about the precise form of character glyphs that make up a text string (in this respect, it is very similar to TeX). It does however want to know about the metric properties (ie. the width and height of characters). The font classes in lib-layout contain abstract virtual members returning this metric information. These members will have to be implemented by each and every (gui dependent) descendant of lib-layout.

For mathematical fonts, additional parameters are needed, which are not available in any of the common font formats (type 1, truetype) except for TeX fonts. See the separate page about font encodings for more details.

Another issue: precision. The math engine needs to position boxes at sub-pixel locations, and have them rendered there as well. Does FreeType support rendering at sub-pixel positions into a bitmap? YES. Note that the most efficient way to handle equations is probably to render an entire equation in one bitmap and then keep that one buffered on the server. We need some technique for that too. FreeType uses a subpixel resolution of 1/64 pixel.

Mathematics subtrees

Event boxes

These are ignore_geometry boxes which can therefore contain any sort of collection of boxes inside. Once its children have their width/height frozen (by calling freeze_children) and once the locations are frozen too (which is true after the parent geometry box has done that) it can determine whether a given coordinate is inside the hitbox.

All event boxes are initialised with a pointer to an event_generator box. This is the box that will be responsible for the collection of events from the GUI and distribution over event boxes. The event_generator box has to be a parent of the event box (maybe use factory?). When the event_generator has frozen all its children, it can sort the event boxes by vertical and horizontal position for speed (to be implemented later).

GUI dependent: the event_generator contains a callback member which is called by the GUI main loop when a mouse event occurs. If necessary, a callback to the owner of the event-box is made. In all cases, the event_generator expects to be able to return to the GUI main loop again.

An event like this never leads to self-destruction. If the event is a link-clicked type thing, a new page can be fetched by relying on the network layer to spawn a new thread. In other words, the callback routine sends off a msg_get_url, which returns as soon as the new network thread is running. The GUI event loop is entered again, and the entire widget is cleaned up by the newly created

- Click events and click-drag events.

Glyph blitting

While standard X11 supports drawing of arbitrary strings, we have to have more control over individual characters for professional quality typesetting. The level of support in various font libraries is not the same, so we will want to write a uniform API. In particular, support for drawing entire strings including kerning and ligatures is practically absent, even in FreeType.

Drawing many individual bitmaps ourselves is however going to be slow when either the overhead for such a draw call is high (a test which draws every character of a word separately using gdk_draw_string with the present layout code shows that it is visibley slower than drawing words in one shot) or when the bitmaps have to be transferred over the network (which will kill performance).

So we will probably want to use our own glyph buffering system on the server. Which techniques (apart from giving each glyph its own GdkWindow) are available?

We also have to find out how much information TeX actually needs and how expensive rendering can become. Ideally the layout library would not care about how you actually draw the characters (ie. it should be possible to switch to a fast mode where we draw using X11 fonts).

One particularly nice feature of storing all characters in a separate box is the fact that selection becomes a lot easier to program (and conceptually cleaner). This does however require that we get smart boxes, which do not store the position but are just stacked next to eachother. The parent box could then even draw them in one shot if the inter-character space is zero. We definitely don't want such boxes to be more than a few bytes in size. Thus, the abstract_box has to be further minimalised, maybe making all data members go away.

Performance considerations

When the layout tree gets _really_ big (like pointing it at ftp:///somedirwiththousandsoffilesinit ) we should think of a better way to organise data flow. On the other hand: this is a _page_ viewer. It does not make sense to have 100000 lines on one page since the scrollbar will be zero pixels high anyway. The entire idea of web browsers is that you do not put a gigabyte database in one page but split it up in smaller hyperlinked blocks.