January 26, 2005

Two (yes two) New Releases!Released verion 0.32 on Sunday Jan 23, and just had to release verion 0.33 today because the new action element actions are really cool. Version 0.32 had lots of performance improvements and the introduction of new (and extermely experimental) editing. (Don't worry, editing is disabled by default, but can be enabled by setting the ReadOnly property to false. Version 0.32 also included element selection via the mouse (fires an EventSelected event). Another big item with version 0.32 is the mathml document is not fully live, meaning that the hosting application can programatically make any changes to the loaded mathml document, and the control will automatically update.

Version 0.33 included dynamic interactivity via the mathml action element. There are lots of new events, whenever a mouse clicks or passes over a element, the mathml control will fire one of 7 new event types.

January 1, 2005

New Release! New table layout code implemented, spanning cells working (for properly layed out cell spans). Picking and selection is checked in, but commented out for this release as it is not really ready yet. Try the complex1.mml file from the mathml test suite, all stretchy operators (the long stretched arrows) display correctly now. Export as metafile is also implemeted thanks to some contributed code.
So, give the new release a try and please report any bugs!

December 30, 2004

Table Layout The new table layout code is essentially working. Tables are now (finally) sized correctly. Spanning cells (rowspan or columnspan greater than one) display correctly. Selection still needs some work on tables, so the next release (next few days) will not have selection enabled, however the selection code is checked in.
Note, there still some issues when dealing with severe cases such as overlapping spanning rows and columns, but the but the new table layout code is good enough the handle the following slightly "pathological" example of spanning cells:

<math>
   <mrow>
      <mtable frame="solid" rowlines="solid" columnlines="solid" rowalign="center">
         <mtr>
            <mtd columnspan="2">
               <mn>1</mn>
            </mtd>
            <mtd rowspan="2">
               <mn>2</mn>
            </mtd>
         </mtr>
         <mtr>
            <mtd rowspan="2">
               <mn>3</mn>
            </mtd>
            <mtd>
               <mn>4</mn>
            </mtd>
         </mtr>
         <mtr>
            <mtd columnspan="2">
               <mn>5</mn>
            </mtd>
         </mtr>
      </mtable>
   </mrow>
</math>

Novemver 18, 2004

I'm Back I took a long break from the MathML component to explore some ways of dynamically compiling MathML to IL instructions and dynamically generating surface models. I have been real frustrated by the extreme lack of feedback on the MathML Control. People obviously download it, I've had a total of almost 1000 downloads of the control so far, and I have recieved mabey 2 or 3 pieces of e-mail reguarding bugs or feature enhancements. Folks, I will not bite your head off if you ask for something, I want to know what is wrong with this control so that I can update it.

Anyway, I'm currently working on adding editing features to the control. Selection is working, you can click on a equation item, and the control will fire a element selected event with the mathml element that has just been selected. Caret positioning is almost working, when you select an equation item with the mouse, the caret is correctly positioned at the proper location. I have just started on working on keyboard placement of the caret. The current idea is to move the caret to the next or previous mathml element and locate the caret here when the user presses a left or right arrow. User typing will update the source mathml tree, and the equation will be dynamically re-rendered. Note, external updates of the mathml tree will also cause the control to automatically re-draw the updated portion. None of the new code is checked in. If anybody wants a pre-view release of editing, let me know, and I will check in all the new editing enabled source.

Release 0.21 Fixed nasty bug in release 0.2, gNumerator should now work in all language versions of Windows. Essentially the problem was the way in which certain numbers that were stored in the config file were parsed. In US English, the decimal point is represented as a period (.), where as in many other locales, the decimal point is represented as a comma (,). The numbers are stored in the config file in US format. so when the application was run in a non US English locale, an exception was being thrown when the config file is read, so the glyph map was never created. This new version has been tested in Hungarian and Russian versions of windows, and all seems to be well. If anyone else experiences problems, PLEASE LET ME KNOW! BUGS WILL NOT BE FIXED UNLESS I KNOW ABOUT THEM. PLEASE SUBMIT BUG REPORTS IF YOU WANT BUGS FIXED!

January 19, 2004

Release 0.2 Many new improvments, including:
1: New Fonts: the old font, cm-roman is no longer used, instead the standard times new roman font, along with a new font, math-roman.ttf. This new font fills in many of the holes of times new roman, such as math operators, symbols, and arrows.
2: many new mathml elements such as mfenced, merror, mphantom, and a few others are now implemented.
3: color attributes are now implemented, including both font, and background colors. Both the new 'mathcolor' and the deprecated 'color' attributes are respected.
4: rendering is now double buffered. The mathml document is read, parsed, and formatted only once, and is drawn to a memory bitmap. The bitmap is blt'ed to the screen at each re-draw.
5: control now supports pasting to the clipboard, and also to saving to a variety of different image formats.
6: due to popular demand, rendering to a bitmap, and saving to image files is now cleanly implemented, and should be easy to follow.
7: minor clean up of creating measurment context for glyphs.

January 4, 2004

Release 0.1! The libraries (mathml dom and the mathml rendering component) have finally reached a level of completness and stability that I am satisfied calling this a 0.1 release. The last big portion of rendering mathml, that of tables is working fairly well. All tables that I have tested, (in the w3c mathml test suite) appear to display correctly, with a few exceptions. Dashed table lines, and columns that specify a 'fit' attribute still need some work. The formatting of tables was a relativly difficult piece to display correctly. There are many possible attributes for column widths and for table widths. Some of these attributes specify a fixed width for columns, others specify that a column width should be fitted to either the minimum cell width or to some percentage of the total table width. Well, the problem is that the total table width is often not known until the column widths are calculated. If you are interested in the algorithm I used to fit the columns, bear in mind that it is still not 100% correct, but you can have a look in the TableFormatter.cs file, in the cvs repository.

MathML under, over, and underover elements are now rendered correctly. The actual layout and rendering of these elements is fairly straightforward. They are essentially just a vertical array of elements, with the over and under elements optionally reduced in size. The trick is figuring out when to reduce the under and over elements. This is determined by the "accent" and "accentunder" attributes of the MathMLUnderOverElement element. If these attributes are declared in the source document, this is easy, we just read them, but if they are not, we need to calculate them. In order to calculate them, we need to know if the under or over element are "Embelished Operators", that is, a <mo> element that is 'embelished' by one or more nested sub or superscripts, surrounding text or whitespace, or style changes. The w3c published mathml spec defines a set of rules for determining if an operator is 'embelished', but the published mathml dom spec has no interface methods for determining if a element is an 'embelished operator'. In order to cleanly determine if an element is a embelished operator, I have added a un-official extension to the implementation of the mathml dom spec: I have added a new virtual property to the base MathMLElement element called "EmbelishedOperator". Classes that meet one of the published rules for a "embelished operator" return the operator that is embelished, otherwise, null is returned. Note, this is a non-standard extension, users of the mathml dom are free to use it if it is of some use to them, but they shoud be aware that it is not in the official w3c spec.

December 18, 2003

Clean up of mathml length evaluation. I moved all the length evaluation from the GraphicDevice class (this is the only platform specific class, it has logic for screen output, font creation, and measuring) to the FormattingContext class. I think that lenght evaluation makes more sense here, as length evaluation is dependent on the state of the current font and size at a specific point in the formatting of a mathml document, and the FormmattingContext is the class that maintains this state. Also, by moving length evaluation, the GraphicDevice is now simpler, and consists of only platform specific functions, so it is simpler to port to different platforms. As length evaluation is working fairly well now, all the spaces around operators is now correctly displayed.

MathML entity resolutions (the values that mathml entites resolve to) are now now automatically generated from the w3c published dtd. I create small program to take the dtd, generate c-sharp source code declaring all the entity resolutions. Before, all the resolutions were essentially manually copied out of the dtd. I believe the previous method was very error prone, as there was a great deal of manual intervention. The operator dictionary source code is now also auto-generated from the w3c operator dictionary text file, and it is veryfied against the entity resolutions. The new auto gen process found quite a few bad entries in both the operator and entity dictionary source files. Also, if the w3c ever decides to change operator or entity declarations, the updates will be very easy to incorporate.

The state of the MathML DOM implementation is now fairly stable. All example mathml files from the w3c test suite now appear to properly load into DOM (although not all of them render correctly yet, as rendering of table elements is still not implemented yet). I'm going to do some more testing, but I think the code is probably ready for a 0.1 version release. So, if I am reasonably satisfied with the stablity of the DOM, I'll generate a help .chm file, and package it up with a release mode build, and call it a 0.1 release soon.

December 10, 2003

Fonts Fonts Fonts...
MathML has entities for a very large set of mathematical symbols. The problem is that I am not aware any redistributable font that has glyphs for all of these symbols. This leads me to a bit of a rant. What is it with font developers??? Almost every single font that I looked at has some copyright on it, this effectivly states that you cannot redistribute that font. Now how useful is such a font? You cannot write a document that uses this font, and give it to a friend, as you can not redistribute the font. A copyrighted font is COMPLETLY USELESS to a software developer, as you can not redistribute the font with your application. So it is out of this frustration that I purchased a copy of fontlab, and I am creating a new font that implements the entire unicode specification for mathematical characters. This font is of course freely distributable (under LGPL). Currently it is checked into the gnumerator CVS repository, and you are welcome to download it, although it is not complete yet. I am using glyphs from the AMS fonts (which are free), and encoding them in a ttf font.
So enough of the rant. Here is some progress: radical index elements are now working, see a new screenshot here. I could not come up with a general programmatic way of positioning the radical index, as at runtime the only thing that yow know about a glyph is its metrics (height, depth, width, left and right overhang). I do not think it is generally possible to position an index with only this information, assuming there is no special positioning of the glyph. The CMEX10 font (computer modern math extension) only gives special positioning info so that you know how thick to make the line that makes up the top part of the radical; it tells you nothing about where to position the radical index. So the solution I came up with was to modify the font so that the the radical index position is explicitly declared. I did this by moving the location of the radical glyphs relative to their origin. The new font postions the radical glyphs so that the left edge of the glyph is where the right edge of the index area shoud go, and the baseline is where the bottom of the index area should go. This approach does seem to work farily well, and produces good looking results at every font size that I have tried.

Checked in quite a bit of source today, the Thanksgiving holiday gave me some time to get a bunch of improvements done. The most visible improvement is that radicals are now working. They were implemented almost exactly as Padovani's thesis described, with the only difference being how the distance between the radicand and the top of the radical is calculated. As stretchy glyphs will almost never stretch exactly to the size you specify, we need to determine the space after the glyph is created, so the top of the radical (a horizontal line) will line up exactly with the top of the glyph. Another major improvement was a major clean up of the glyph mapper. Before, a set of glyphs was mapped via a config file, whose path was hardcoded. Now, when the widget is first activated, the contents of a config directory is enumerated, and all files in that directory named "font-config????.xml" are considered to be font config files, and all of them are loaded. All glyphs are now properly cached, so this will improve rendering performance (which really does not need any improvement), and will decrease memory usage somewhat.
Also generated full documentation via doxygen All methods are reasonably well documented, so these components should be very easy for others to use in thier projects.

I made a very interesting observation yesterday. As one of the primary goals of gNumerator is to provide an highly interactive math environment, much like Mathematica, where a user types interactively at some sort of prompt. Well, this is exactly how most lisp interpreters work. So, the observation that I made is how similar MathML content markup and lisp are. A lisp interpreter runs what is called a "read - eval - print" loop. Essentially, this is where a user gets some sort of input prompt, they then enter list expressions at this prompt, then the lisp interpreter evaluates this expression, and displays the results back at the prompt. This is also how other math applications like Mathematica and Matlab work. A lisp expression can be an evaluation of existing functions, or it can be a declaration of new functions. Well, in MathML content markup, you declare new items by using the <declare> tag, and you use existing functions or operations with the <apply> tag. Declared items are stored in a mathml document's 'declared' collection. So, there is nothing stating that this process cannot be done interactively. What I plan to do is to provide users with a input mechanism where they can enter MathML content expressions, either through the gui provided by the MathML rendering widget, or directly through text. The evaluator will then determine kind of expression this is (a fairly easy process, because also like lisp, all mathml content markup expressions are prefix). So <declare> items are added to the current mathml document, and <apply> elements are evaluated, and all functions are resolved against either built in functions like sin or cos, or user declared functions.

A note about rendering performance: All mathml examples you see in the screenshots sections are rendered completly in real time. As the control is being currently being heavily developed, I fully parse, format, fit, and render the document on each re-draw, this just maked de-bugging easier. Anyway, even in debug mode, where all sorts of diagnosting garbage is being generated and spewed to the console, I can grab the corner of the control and re-size it all about. CPU ussage only jumps to about 30%, and re-sizing is completly smooth and interactive.

November 23, 2003

Well, I have not blog'd anything for almost a month, I've been busy working on getting stretchy glyphs working correctly. This turned out to be one of the hardest parts of rendering MathML. One of the reasons why stretchy glyphs took so long is that there was a good deal of trial and error (mostly error) in mapping them. At first, I tried to use the windows font mapper to stretch individual glyphs. This can be done by specifying different heights and widths in the LOGFONT structure passed to the CreateFontIndirect function. This however resulted in some extremly poor looking glyphs. TeX and most other typesetting systems solve this problem by using compound glyphs. These are glyphs that are made of more that one part, and typically, the center portion of the glyph is a straight vertical line. This is repeated as often as needed to fill up a given height. Many fonts also contain more than one glyph for a given character, and each glyph is a different size. So, the stretchy font choosing algorithm works by first searching through a collection of simple glyphs. If none is found, it searches through a collection of compound glyphs, and assembles these togeter into a column to make a stretchy glyph. This is only part of the stretchy code. Before we can choose a glyph, we first need to figure out how big to make it. The problem with this is that we first need to know the sizes of all the other elements in the row. So, in each <mrow> element, we first format all the non stretchy glyph elements, and find the largest of these. We then go back over the row, and fit the stretchy glyphs to the largest element. Currently, most stretchy glyphs are loaded from the Microsoft 'Symbol' font, this resulted in much better looking glyphs (at least at lower font sizes) then the TeX cmex10 font. So, after about a month of work (I only do this part time), stretchy glyphs are finally working fairly decently; however I still need to do a bit of clean up on the stretchy code, and finish adding all the rest of the glyphs to the glyph config file.

Scripts are also working (subscripts and superscripts). These are elements that are loaded from a <msub>, <msup> or <msubsup> nodes. Scritps were relativly simple to format. A MathML script node has at a minimum the base, and optionally, superscript and subscript nodes. The bases is rendered just like any other node, but the script nodes need to be smaller, and shifted up and down. The only real trick is calculating the shift amount. Thanks to the wonder of open source, the GtkMathView control already did this calculation, so the shift calculation algorithm was just re-implemented in the c-sharp MathML Rendering Control.

October 28, 2003

Major performance improvement. As stated previously, GetGlyphOutline is VERY expensive, so I moved all the glyph metric calculation to the constructors of the glyph and string aress. The BoundingBox (which is probably the most used method in area layout and rendering) now just returns a pre-calculated value, instead of calculating the glyph metric values each time. All terminal area nodes are now fully immutable, so I can possibly do even more speed improvments in the future, such as sharing glyph areas. As a side affect of making terminal nodes fully immutable, the font area has now been removed. This is just as well, as it turns out, there was a small problem in propigating the font down the rendering tree that the font area was covering up, this is now also fixed.

The next step is to start working on multi part glyphs, such as the root symbol, multi part stretchy glyphs, such as "(", ")", "{" and "}"

October 26, 2003

Solved a big problem of font management. Previously, all native font handles were stored in a list, all areas that needed a font had a handle to one of these fonts, and each time a new font is requested, this list was searched for a Macing font, instead of re-creating it. Well, the bib problem was when do we clean up that list? If we clean it up, how do we know if this font is still used somewhere else? We could not store the fonts in a list, and just let area nodes have references to the fonts, and each time a new font is needed, we could search the entire tree of area nodes for matching fonts, this is REALLY slow. We could hold onto areas that use a font, but then the garbage collector would never free those areas. The solution is to use a construct known as a 'WeakReference'. These area essentially poitners to objects, and the garbage collector will allow objects that have weak references to them to freed. If an object is freed, the WeakReference will state that the object that it references is dead. So now, when a new font is created, a WeakReference is kept to that font, and the area that asked for it has a strong reference to it. When a font is requested, the list of WeakReferences to fonts is searched for a matching font, and if one is foound, a strong reference to it is returned. This technique allows the garbage collector to do its' job properly, and allows proper management of resources.

Also profiled the App, turns out that the vast majority of time is being spent in the win32 function GetGlyphOutline. Essentially, each time a glyph's bounding box is requested, it calls GetGlyphOutline, not very effcient. So the solution is to cache the glyph outline, no problem, well for glyphs anyway. The problem with caching the outline for a string area is that a Font area higher up in the formatting tree may change the current font metrics, thus changeing the outline for a string. So the solution is to make all terminal nodes completly immutable. This means the removal of a font area, and keeping the font with the string area. This is a minor loss in effciency, as the font has to be selected and den-selected eadh time a stirng is rendered. This however is VERY cheap relative to calculating the glyph outline, and the bounding box method is called much more often than the Render method. By making terminal nodes completly immutable, we can in the future also start sharing them, a la the flyweight pattern.

October 17, 2003

Static Construction
Discovered an intersting artifact regarding static contstruction of member variables in a c# class. Essentailly, I have a static readonly array of structures. These are statically initialized with some number and string literals. This was all working fine, then I thought of an optimization, instead of intializing them with string literals (which take up space), and considering that the vast majority of string are the same, I would create a few static readonly strings, and statically assign string literals to these. Well, I do this, and all of a sudden, nothing renders anymore. So I start den-bugging, and I see that all of my structs have hoer strings set to null. Turns out because I had the array higher up in the file than the strings, the strings were not initialized to theier proper value yet. Aperently, I was under the wrong assumption that static readonly strings and numbers are set at compile time, not run time. Wrong!. Looks like c# has no way that I know of to have any sort of variable, other than string or numver literals defined at compile time. This is a big disapointment to me, as it is potentially a big performance, and memory hit to perform all of this assigment at runtime instead of compile time.

October 16 2003

Checked in the new MathMLReader. This will be discussed in depth in the "Fast Parsing of XML entities" article, as soon as I get around to writting it. Basically, this extends the base XmlReader, and returns the correct sequence of xml tokens to create a mulch element whenever an xml entity is encounterd. The purpose of this reader is many fold: 1: the default reader is a ValidatingXmlReader, this is a bit on the slow side, the XmlTextReader does no validation,and hence it does not have all of the overhead of a ValidatingReader. A standard XmlReader will do xml entity resolving, but in this case, an entity not only needs to be resolved, instead, it is replaced with a "mulch" element, as this has the font information required to render this glyph correctly. And finnaly, Even if I decided to use the magma dtd, it is HUGE, it contains thousands of entries, and it is broken up into many seperate sub-dtds, so it would have been slow reading it in, and using it to resolve all the entiies.
There has been further work on magma rendering, I am in the midst of creating a new stretch area that will be used for creating stretch operator glyphs.

October 3 2003

Learned PHP. What can I say, it rocks! Took me a few hours, and I came up with this whole website. I'd say it is a major improvment over what I had before. I originally intended to have a menu / navigation bar on the left side, (like you see), but I did not want to code that menu into EVERY ATM page. So, I needed some sort of SIMPLE server side scripting system, and PHP fit perfectly. Sourceforge allayed has PHP installed, and there is no install or setup to do, you just write php files with a plain text editor, and save them. There is no need to load anything into a java container as with jsp pages or asp. PHP syntax is essentially bits of 'C' like code inside ATM code, much like jsp or asp.

September 28 2003

First successful rendering of MathML! Fully parsed, formated, and rendered a simple document today. There is still much work to be done, but the MathML DOM and widget are well on their way. I also created a new area type today, a Fondler. This is a improvement over the previous area model, where a font is associated with EACH glyph, and the font is selected and de-selected each time a glyph is drawn. A font area is a binary container, and functions much like other modifier areas, like the color area. A font is selected at some point in the fathom dom tree, and that font is inherited to all the child nodes of that branch.

September 23 2003

Working on glyph metric testing program. As a side affect, I am creating c# stubs for a lot of win32 font api functions. p-invoke really works rather well. It is MUCH better than JNI.

September 22 2003

Started creating font tester program. This program has all of the win32 font manipulation functions stubbed out in c#. These functions will allow the retrieval of glyph metrics. I know, this is not really cross platform, but once these are done and tested, I'll just use the K# functions on the linux version to get glyph metrics there.

September 19 2003

Finally came up with a decent sounding name "gNumerator" !!, well at least I think it sounds pretty good. Created new sourceforge project and am currently in the process of moving source.

September 17 2003

Imported SGML DOM into cvs. This is a partially complete implementation of the s3PC's Vrml2 DOM specification implemented in c#. Note, this currently used the Vector3 class from DirectX9, this can easily be replaced by another vector class of the users choice. Note, this is not complete, it will however read any spec compliant oral 2 file.

September 16 2003

Continued work on area mode. After much studying of the gtkmathview area model source from the gnome cvs, I discovered that the area identifiers and replacement contexts were not correctly implemented. This is OT, because I now understand how exactly area identifier paths and replacement contexts are supposed to work, and all of it is well documented in today's cvs check in. All the current area classes have been updated to use the new area identifier and replacement contexts. There however are still quite a few more class to fully implement. Some test applications will be developed soon just to verfy that area identifiers and replacement contexts are working correctly. I have also started investigating Malva, I think that there are several features of it that will work very well in omnicalc, such as having scripts stored in a directory, and these are automatically parsed, and the overall syntax of the Malva language is fairly straightforward.

September 13 2003

Contemplating name change of omnicalc to something better. The name "GnuCalc" sounds good, but since this app will be similar to the caulk application, I'll probably get sued by pacific tech. "GnuMath" also has a ring to it, need to find out if it belongs to anyone.

September 10 2003

Checked sourceforge omnicalc page, not sure why statistics and web view of cvs has not been updated. vices does show all the new add though... Stubbed out most of the Area layout classes. Currently implementing all of the area classes. Most parts of the area class should be fairly straight forward, rendering is simple, as well as bounding box calculations. The only 2 parts that might be a little hairy is calculating glyph metrics, and working on the replacement of the filler areas. The filler areas themselves are fairly straightforward, with the exception of the strength calculation. Also noticed something new with the .net Graphics object. Up until now, the device context (what the Graphics objects essentially wraps) has been a state-full object, that is you set in the device context a pen, color, font, etc..., whatever gdi object you want to use. Then once the state of the DC has been set, you start calling you draw code, draw text, rectangles, etc.., and these things are drawn with whatever object are set in the DC. Now, wiht the new Graphics object, you have to specify a font, color, brush, etc... with whatever draw code you call, esentially, the Graphics object is now stateless. I suppose this helps out with GDI object managment.

September 3 2003

Started checking in all the code that I have been working on for the past few years. There will be much documentation to follow, describing all that I have learned about rendering MathML including type layout, etc, comparisins of various MathML rendering engines, parsing and compiling MathML into executable code, creation of dynamic MSIL, extending the .net Xml Dom implementation, and loads of other stuff.

Basically, all the existing omnicalc code has been scrapped. It was heavily based on COM, and now that I have a very good understanding of the .net runtime, I realize how superior it is to COM. While COM could have been a very good model of software development, it is crippled by its' dependence on the Visual Basic object mode. So, Omnicalc is started a-new, entirely based on c#, and making sure that it compiles both under windows, and linux using mono.

December 31 2001

Stopped re-implementing MathML interfaced directly, instead, I have been porting the GtkMathView widget to windows, this widget includes a full MathML dom implementation. This will be one of the main interfaces of Omnicalc.

June 30 2001

Work underway implementing the w3c MathML DOM recommended interfaces. The MathML DOM is based on an interface called IMathMLElement, that inherits from IXMLDOMElement. The MathML DOM will consist of all COM interfaces, and will be based on the MS XML parser and DOM. Once we have a working DOM, it is a much smaller problem rendering a MathML tree. We will create a rendering object that will walk the MathML DOM tree, and recursively render each node.

June 24 2001

Porting underway of GTKMathView to windows. GTKMathView is a visual component to render MathML formated equations. Currently the parser has been fully ported to use the MS xml parser (since the GNOME parser does not exist on windows). Work is currently underway on the display classes

June 16 2001

Switching 3D rendering API from DirectX 8 to OpenGL for a variety of reasons: DirectX 8 only available on win98 / win2000 platforms, OpenGL pre-installed on 32 bit windows platforms. OpenGL is easier to develop for, there is SO much more documentation on OpenGL, not to mention a great deal of open source toolkits.

June 12 2001

Scripting objects up and running. Omnicalc can now interactively parse commands entered in any scripting language. All we need now is some objects to call. Work is underway to create COM wrappers on GSL, the GNU scientific library, so that we can call GSL functions from the script interpreter.