Extracting, Analyzing and Optimizing a Vector Shape from a BitmapData Shape

Tuesday, April 28th, 2009

This post has been a long time coming, as I've re-written the code more times than I care to recall. There are just too many ways to skin this damn cat! What I'm sharing now is not a final solution by any means. I just need to stop obsessing for a moment, move on and actually use the code for the cool things I've been planning to do with it! Then obsess further at a more suitable time :D

Also, retrospectively, this post is gonna take some courage and dedication to get through without falling asleep. The code was thrilling to conceive, but less so to write about. Apologies.

Anyway, the challenge is : Given extracted "shape outline points" from a BitmapData shape, I want to optimize these points into the minimum amount which correctly define that shapes perimeter.

Rather than going over the ifs, buts, successes and failiures, here's how the process currently works:

1) Remove redundant points from the "shape outline points"

Since the shape is extracted from a bitmap, the "outline" is composed of rows and columns. Any row or column longer than 2 contains "redudant" points (or pixels), between the start and the end of the line. These are easily removed by looping through the points, keeping track of the current row or column direction.

In the case of the F, the process is done, however, much more work is required for those pesky diagonals and curves...

2) Create "Line" Objects which connect the "non redundant" points

The optimization routines use net.sakri.flash.vector.VectorLine, which has the following properties:

  • start_point:Point
  • end_point:Point
  • type:uint VERTICAL, HORIZONTAL or DIAGONAL
  • direction:Point UP, DOWN, LEFT or RIGHT. Only relevant to VERTICAL and HORIZONTAL lines

The code loops through the "non redundant points", creating Lines. At this point, only VERTICAL and HORIZONTAL lines are present (again, due to the "rows and columns" nature of bitmaps). The loop sets each "line direction", based on the lines start and end positions. See the little red arrows below:

3) Use the directions of the "Line" Objects to discover "break points" or "turning points"

Essentially the goal is to isolate Lines, Diagonals and Curves. I have tried approaches ad infinitum, and so far the following delivers the most Bang For my Buck:

  • Loop through the lines, minding a "vertical anchor line" and a "horizontal anchor line".
  • Whenever the direction of lines changes from the current anchors, this can be seen as a "break" or a "turning point".
  • Store these "break points" in a list.
  • Use the "break point" as the current anchor, and continue looping.

The image below is a "single clockwise pass", the "turning points" are represented as blue pixels accentuated by totally awesome green circles.

Following the lines starting from the top left corner, you should "discover" the same "break points".

The first "horizontal direction change" takes place at the bottom right corner of the M. Up until this point all the horizontal lines go "RIGHT".

The first "vertical direction change" occurs at the bottom of the first "V" shape between the M's legs. Up until this point, all the verticals point DOWN, and after the turn point UP.

This first "pass" uncovers 10 break points, which could be used to analyze the "sub components" (deal with diagonals and curves). However, it's painfully clear that more break points are needed. In the "M", there are 13 which define the shape. So, a "second" pass sounds like a good idea, this time counterclockwise (or in reverse order). This results in different "break points" than the clockwise pass, (again just follow the lines with your eyes...).

Below is a "single counter-clockwise pass", the "turning points" are again visualized as blue pixels surrounded by gratuitous neon green pixels:

Putting these two together, running clockwise and counter-clockwise gives us:

Now we've got 16 break points for "M"! This is good enough, the main lines and diagonals are isolated. There was much rejoicement!

Here's the same for a character with some curves:

Nifty as this might sound, the approach isn't without it's shortcomings. "Staircases" and "Curve Staircases" are a thorn in my side. In such cases, there are "clear" changes in visual direction, yet, neither the HORIZONTAL nor the VERTICAL direction changes in a "staircase". Experience the phenomena in characters including T,F,4,S,a, etc. etc. Witness the F below:

At this point, we say "F it", and move on.

4) Use the "break points" to cut the shape up into segments, and analyze

Loop again. This time looking at the sets of VectorLines separated by "break points":

  • Any set of 1 must be a Vertical or a Horizontal line.
  • Detect Diagonals
  • Handle Curves and "complex shapes"

Here's my "CPU Cheap" approach to detecting a Diagonal in a set:

Actionscript:
  1. protected var _diagonal_buffer:Number=.15;
  2.         protected function isDiagonal(lines_vect:Vector.<VectorLine>,index1:uint,index2:uint):Boolean{
  3.             var first:Point=VectorLine(lines_vect[index1]).start_point;
  4.             var middle:Point=getMiddlePointInVLineSegment(lines_vect,index1,index2);
  5.             var last:Point=VectorLine(lines_vect[index2]).start_point;
  6.             var diff:Number=Point.distance(first,last)-Point.distance(first,middle)-Point.distance(middle,last);
  7.             return Math.abs(diff)<_diagonal_buffer;
  8.         }

The code grabs the "middle point" within a set of Lines. Then it compares the distance from the "sets start point to finish point", with the added distance of "start TO midpoint+midpoint TO endpoint". If the difference is less than "_diagonal_buffer", it's a diagonal. Accept it.

The remaining shapes fall under the categories of "Curves" and "Complex Shapes". Both can actually be treated the same : Based on a parametrized "Curve Accuracy", the remaining shapes can just be divided into "sub segments".

Let's say a Curve has 200 pixels, which might translate to 50 "Lines" once "redundant points" are removed. If "Curve Accuracy" is set to 15, it's relatively easy to loop through the 50 lines, and isolate them into "Sub Lines" grouped by this "Curve Accuracy". Maybe an easier way to picture this is just to chop up a curve by choosing every X points along it's path. If that makes no sense, just check out the code.

There's a few more optimizations, but I can't be arsed to go into it right now. This post is long enough as it is.


Click here for the demo (Comic Sans pictured below):

Download the Source (Requires FlexBuilder, with sdk 3.2+)

Congrats, give yourself a hearty pat on the back for making it all the way through! You have our Gratitude! I personally guarantee future posts with better entertainment value using this code base and it's oncoming descendants :D

Mouse interactions on Text Layout Framework InlineGraphicElements in EditMode

Thursday, March 26th, 2009

One slightly frustrating (but totally logical) feature of using editmode in TLF is that inserted InlineGraphicElements lose their "interactivity". Instead, they are treated the same as text characters, that is, the user can select them as if they were selecting text:

There was a comment In my previous entry about the possibility of mouse interaction with inserted graphics.

I have a way to do this, a bit hackish, but it does the job. Every added graphic needs to be stored in an array. When a user clicks, the code loops through all the graphics and checks if one of them happens to be under the mouse. This is achieved using the trusty old localToGlobal() method:

Actionscript:
  1. protected function handleMouseDown(e:MouseEvent):void{
  2.     var hitrect:Rectangle;
  3.     var bm:Bitmap;
  4.     for(var i:uint=0;i<_images.length;i++){
  5.         bm=Bitmap(_images[i]);
  6.         var ltg:Point=bm.localToGlobal(new Point(0,0));
  7.         hitrect=new Rectangle(ltg.x,ltg.y,bm.width,bm.height);
  8.         if(hitrect.contains(mouseX,mouseY)){
  9.             mx.controls.Alert.show("so, I see you have clicked on an image eh?");
  10.             break;
  11.         }
  12.     }
  13. }

This is very useful if you want to edit the graphic somehow, for instance, in the RTE that I'm currently working on I use this for "table support". In the TextFlow, my tables are represented as bitmaps, when clicked, the actual DataGrid is opened and ready to edit. When finished, a new BitmapData is grabbed and the InlineGraphic in the TextFlow is updated.

I have found one "bug" (or a feature?!), if an image is taller than (somewhere around 1000 pixels), the image is no longer properly "wrapped", and actually starts to cover some text.

It's also possible to listen to MouseMove, and set some rollover/out states to the graphics. You can see a demo here and download the source mxml file here.

Working on 3d text generator with papervision3d

Tuesday, October 14th, 2008

It's no secret I have a bit of a fetish for playing with text and making text effects in flash. Back in May I decided to create dynamic 3d TriangleMeshes from BitmapData captures of text in textfields. Like the majority of my experiments, I never finished it.

I'm now posting this for three reasons:

  • Aral Balkans Grab the low hanging fruit Presentation which I saw last month in Ghent. I didn't agree with everything he said, but what stuck with me was : "Talk about stuff you're working on". The idea is, if you wait forever for your "perfect finalized grand hobby project" release, it'll never happen. You need peers to egg you on. So, I've been doing this on and off for a while and this post is an experiment to see if I'll actually get a finished product out someday
  • I saw a post about 3d text with Away3d a few weeks back.
  • Just the other day, Mark Barcinski posted this on the Papervision3d mailing list which is pretty much exactly the process I had come up with. Doh. Somebody always does it before you... I'm just waiting for some Zupko or Ralph to see this and say : "oh, I typed that with my toes back when the beta of player8 came out..." ;)


Click here to try out the current state of my 3d text test tool

screen shot of 3d text testing tool

I won't post any code for now as it's in an embarrassing state... just getting shit to work eh. The current state of the algorithm only handles Pixel fonts with no diagonals or curves... The flow is:

  1. Take a BitmapData snapshot of a textfield with one character.
  2. Create "monochrome" bitmaps of the character, and any "holes" or shapes within it (That would be the black bitmaps at the bottom right of the screenshot above).
  3. "Scan" these monochrome images for a list of all the "shape edge" pixels.
  4. Optimize the "shape edge" points into lines, diagonals and curves. These are the "outlines" at the top left of the screenshot.
  5. "Triangulate" those points (this is not quite perfect yet, just punch in "B" into the input field for a fix of teh fail). This will generate the bluish patchwork letter to the right of the outlines.
  6. Use the triangles and the outline points to generate faces and vertices, UVMap them, pass a material and render.
  7. Feel intimidated by the bugs :D

Next steps are obviously to create some code that's actually usable. Get the diagonals and curves to work. Try this out with some shaders, the 3d render is crap now. Get full words out. Animate the individual letters. I'll need some help with optimizing some of the "mathier" parts, but that's what the mailing list and friends like David are for ;)

Damn you Aral... I'm never gonna get this done :D