an approach to triangulating polygons with holes

June 12th, 2009

I know this has been done before, probably much more efficiently, but I had an idea wednesday and had to pursue it to its conclusion :)

The other day Katopz pointed me to this snippet which triangulates polygons. I’ve also seen an approach to ear cutting by nicoptere.

I figured that by converting an “O” into a “C” leaves me with two polygons which are easy to triangulate. Allow me to elaborate:

So the idea is just to cut out a rectangle (two triangles), which connects the polygon and the hole. Triangulate the “cut shape” and then concat the extra two triangles to it. But which rectangle to choose?

I wrote a class called PolygonWithHoles.as . This contains a polygon (Vector.<Point>), and holes (Vector.<Vector.<Point>>). The idea is too loop through the “holes” one at a time (cue beavis and butthead). The safest bet is to find the shortest “connection” between a point on the main polygon and the hole (done by looping and using Point.dist()).

This too has one stipulation. In order to have a rectangle, the code has to find the “shortest connection” either before or after our initially picked “shortest connection”.

Once the “cut out rectangle” has been discovered, the perimeter of the main polygon and the hole are merged through a few loops. If there are more holes, the now modified main shape is chopped again and again, eventually resulting in a “hole free” polygon. This can then be triangulated, and the “removed rectangles” are appended as sets of triangles.


Click image for demo


This approach seems to work well over 90% of the time. A few characters in some fonts do throw an error… not sure why. The triangulation is pretty fast, you can see this in the tracebox in the demo. The vectorization of the text is the slow bit.

The next step, of course, is to extrude the characters using textures :) Maybe even add shading some day…

The code is more or less legible this time, eventually I hope to clean this up into some sort of an API… Once again, the download is a zip of my always expanding “FindBitmapPixelOutline” project… download it’s latest incarnation here. The project you want to look at is “TriangulationTest.mxml”. Warning, it’s 6 megs with tons of crap you probably don’t want :D

.

Dynamic text with phong shading using Stok3d

June 9th, 2009

Yesterday David DerSchmale Lenaerts released Stok3d into the wild. When he showed me the demos, I immediately felt (you guessed it) STOKED!

Naturally my first question was : "does it support transparency?!", and with a resounding yes, I just had to try this with text :D

I wanted to generate the texture maps dynamically. It's not quite there yet, but not bad for a quick experiment :)

This took three steps:

  1. Create the diffuse map by generating a text field, taking a bitmap data snapshot of it and adding some bevel to it.
  2. Create the normal map by repeating the previous step, only using Red and Blue colors, with slightly different bevel settings.
  3. Create the specular map by extracting "outlines" using convolution filter.

These three bitmaps look like this:

Here's the code for generating them:

Actionscript:
  1. var font_size:uint=140;
  2. var tf:TextFormat=new TextFormat("Arial",font_size,0xac8ea9,true);
  3. var diffuse:BitmapData=createTextLineBitmapDataFromTextFormat("STOK3D",tf);
  4. diffuse.applyFilter(diffuse,new Rectangle(0,0,diffuse.width,diffuse.height),new Point(),new BevelFilter(5,45,0xFFFFFF,1,0x00,1,10,10,3,3));
  5. _diffuse_map=new Bitmap(diffuse);
  6.  
  7. tf=new TextFormat("Arial",font_size,0xbf6cde,true);
  8. var normal:BitmapData=createTextLineBitmapDataFromTextFormat("STOK3D",tf);
  9. normal.applyFilter(normal,new Rectangle(0,0,normal.width,normal.height),new Point(),new BevelFilter(5,45,0x89a1f9,1,0xFF00FF,1,12,12,3,3));
  10. _normal_map=new Bitmap(normal);
  11.  
  12. tf=new TextFormat("Arial",font_size,0xFFFFFF,true);
  13. var specular:BitmapData=createTextLineBitmapDataFromTextFormat("STOK3D",tf);
  14. specular.applyFilter(specular,new Rectangle(0,0,normal.width,normal.height),new Point(),new BevelFilter(5,45,0x999999,1,0x00,1,0,0));
  15. specular.applyFilter(specular,new Rectangle(0,0,normal.width,normal.height),new Point(),
  16.                                             new ConvolutionFilter(3,3,new Array(0,20,0,20,-80,20,0,20,0),10));
  17. specular.applyFilter(specular,new Rectangle(0,0,normal.width,normal.height),new Point(),new BlurFilter(2,2,2));
  18. _specular_map=new Bitmap(specular);

Then I just grabbed Davids demo code and replaced his textures with mine... and voila:

click image for demo:


storm image from http://www.flickr.com/photos/digitaltool/2569963337/

For some reason the demo doesn't start everytime. Just refresh a few times and you should be STOKED! ;) Also, the specular lights go black sometimes. I'm convinced this is just due to my happy go lucky texture generation, but David claims he needs to look at his code.

check the demo

download the demo as project. This is just my demo file, you'll need to fetch the Stok3d code from google (see link above);

I have to say, Stok3d is pretty damn cool! Sweet work herr schmale :thumbsup: :pirate:

.

Verlet Text Effect

June 4th, 2009

Here's a demo I made for my presentation at multi-mania. I played around with verlet integration before and have long wanted to try this technique on text. Once I got the vectorization going, it was just a matter of connecting all the points to "Verlet Sticks".

This turned out to be a little tougher than expected. The tricky thing about Verlet Sticks, is that if you connect the same Verlet Point to multiple Verlet Points, the calculations go berzerk, and your shape quickly vanishes from the canvas into the ether. So the challenge was to find a way to connect all the points of a vector shape, with a minimum number of "individual connections", yet in a way that maintains the integrity of the shape. Keyboard cat played me off multiple times in my attempts :D .

Finally, the solution I landed on, was to create a grid out of all the points. The idea is to create a boundary rectangle, which maintains it's shape via "cross sticks":

Then, create a grid using all the points of your vector shape(s), in such a way that each point connects to a subdivision of the outlines of that "boundary Rectangle":



In the image above you can see the "boundary rectangle" surrounding the vector shape. Then, each point is connected with a vertical and horizontal line. These verticals and horizontals are then connected to a "subdivided" boundary rectangle.



In the demo you can turn "render sticks" on and off to visualize the trick.

Strengths:

  • Can handle pretty much any set of shapes and subshapes

Weaknesses:

  • Due to the "Boundary Rectangle" the shapes "bouncing" is limited, a "T" for instance can't lie diagonally, it always ends up on one of the four sides of the rectangle.
  • The vertical and horizontal lines shouldn't have more than two subdivisions. To fix this, the code first checks for all points along the same x and y, then shifts any instances above 2 by .5 pixels each. It's practically invisible, but it's there.
  • Doesn't support transparency, I guess this could be done with a blendmode or so...
  • More complex shapes can still spazz out and disappear :(

Click to try out the demo here . Ignore the collision physics, it's CRAP! The effect works best with "pixel fonts". The controls are :

  • RENDER : renders the input text with the selected font
  • UPDATE : update one frame of animation (when stopped)
  • PLAY : updates the animation on an enter frame
  • BOUNCE : when the characters are just skidding the floor, spice things up by clicking this button

The code is DIRTY, but does the trick. I've uploaded the source for the brave (or desperate ;) ). It's the same project that I've been using for my last 5 or so blog posts, so there's a lot of redundant code in there. Download it here.

ext steps:

  • compose a clean "all purpose" class for generating such a grid
  • Render shapes with curves, using the drawing api
  • Use nicopteres vectorization code to do this with photos...
  • Improve the curves and optimization, again I think I can use some code from nicopetre

I just wanted to put this out there because, well, it's kind of nifty... More to come, enjoy!

.

Detecting edge pixels with Marching Squares Algorithm

May 28th, 2009

Last week a kind individual called "Rothrock" commented on an older blog entry of mine Extract shape outline points from BitmapData and pointed me to the Marching Squares Algorithm.

(Image ruthlessly stolen from wikipedia without remorse)

This does more or less what my "outline point extraction" algorithm was doing, except, with a grid of 4 pixels instead of 9. I nearly choked at the thought of how wasteful my previous approach was. So, I promptly implemented the algorithm in as3, and tested out the speed difference.

Indeed, Marching Squares pwns and humiliates my older implementation, it's 2-3 times faster. This due to the fact that the source bitmap data does not need to be doubled in size, and naturally less pixels require inspection.

However, there are two gaping problems with this 4 pixel grid approach.

1) 'Single pixel vertical and horizontal lines' result in duplicate points

Although the 4grid handles such "single pixel lines" like a champion, it "scans" those points twice (see the line at the bottom of the wikipedia image above). If the goal is to (let's say) extrude this shape, problems just might ensue. So my implementation allows for a parameter which returns only unique points.

2) There is a special case which causes an eternal march. Poor squares.

If you consider the grid variations on that wikipedia page image, the squares march right into our lonely pixel, and keep doing circles Ad infinitum.

The best solution I could come up with, and please don't hesitate to share a better one, was to set a MAX_POINTS value. If the number of discovered points exceeds this, then the algorithm stops, and hands the task over to... wait for it... my old approach :D I refactored this into a class called MarchingSquares9Grid.as . This one has no duplicate pixels by the way, thanks to the double sizing of the source bitmap.

Click the images below to see the algos in action:




On my machine the MarchingSquares runs around 50 fps, but dips whenever the above mentioned exception occurs. The point is that you'll always get the pixel outline, in most cases very fast, in fringe cases a bit slower.

I wrote a MarchingSquares class, which can be used as follows:

Actionscript:
  1. MarchingSquares.getBlobOutlinePointsClockwise(bmd);
  2. //or
  3. MarchingSquares.getBlobOutlinePointsCounterClockwise(bmd);

If speed isn't an issue, use :

Actionscript:
  1. MarchingSquares9Grid.getBlobOutlinePoints(bmd);

Just two considerations:

  • I couldn't be arsed to make the MarchingSquares9Grid run counter clockwise, so there's a method that reverses your Vector if that tickles your perversions.
  • Both of these will require at least a one pixel padding of transparent pixels in your source bitmap. This can be introduced with some bitmapdata magic.

download MarchingSquares.as and MarchingSquares9Grid.as here

Thanks again Rothrock!

It’s Multi-Mania Mayhem May 18-19 Man!

May 6th, 2009


May 18-19th in Kortrijk Belgium Europe’s biggest free multimedia event, Koen was kind enough to invite me to present alongside a cool list of speakers. This will be my second performance at the conference. :buttrock:

I'll be doing an upgraded version of my FITC presentation : "Text Addiction". I've renamed it to "Hot tips to improve your Text life". The "Text Addiction" presentation was almost "too well organized" and focused on one "avenue". This time I'll be covering a broader range of text related topics relating to Flashplayer10. This includes the "vectorizing" I've been doing lately, text-effects with 3d and the drawing api, and aspects of the Text Engine and the Text Layout Framework.

Hope to see you there!

3d Text by Extruding BitmapData Snapshots of Text

May 5th, 2009

Putting together a number of previous blog posts, here's the current state of my 3d text. In terms of speed and quality, it's a big improvement on the examples I was showing at FITC in Amsterdam, old example here.

So, the process is (links to blog posts with explanations and source code):

Here's some screen shots, click images for the demo :

The colors are random, use the widgets to choose a font, the text size and depth of the extrude.

On my machine the framerate is pretty steady in the 20's even with longer strings. The text box on the right displays the duration of the entire process described above.

Check out the demo

Download the source (I know, the project has the same name as the last 3, because it is, I've just added a bunch more code to it). Parts of it I'm quite happy with, others less so, but you know, there's always room for improvement ;)

Extracting, Analyzing and Optimizing a Vector Shape from a BitmapData Shape

April 28th, 2009

This post has been a long time coming, as I've re-written the code more times than I care to recall. There are just too many ways to skin this damn cat! What I'm sharing now is not a final solution by any means. I just need to stop obsessing for a moment, move on and actually use the code for the cool things I've been planning to do with it! Then obsess further at a more suitable time :D

Also, retrospectively, this post is gonna take some courage and dedication to get through without falling asleep. The code was thrilling to conceive, but less so to write about. Apologies.

Anyway, the challenge is : Given extracted "shape outline points" from a BitmapData shape, I want to optimize these points into the minimum amount which correctly define that shapes perimeter.

Rather than going over the ifs, buts, successes and failiures, here's how the process currently works:

1) Remove redundant points from the "shape outline points"

Since the shape is extracted from a bitmap, the "outline" is composed of rows and columns. Any row or column longer than 2 contains "redudant" points (or pixels), between the start and the end of the line. These are easily removed by looping through the points, keeping track of the current row or column direction.

In the case of the F, the process is done, however, much more work is required for those pesky diagonals and curves...

2) Create "Line" Objects which connect the "non redundant" points

The optimization routines use net.sakri.flash.vector.VectorLine, which has the following properties:

  • start_point:Point
  • end_point:Point
  • type:uint VERTICAL, HORIZONTAL or DIAGONAL
  • direction:Point UP, DOWN, LEFT or RIGHT. Only relevant to VERTICAL and HORIZONTAL lines

The code loops through the "non redundant points", creating Lines. At this point, only VERTICAL and HORIZONTAL lines are present (again, due to the "rows and columns" nature of bitmaps). The loop sets each "line direction", based on the lines start and end positions. See the little red arrows below:

3) Use the directions of the "Line" Objects to discover "break points" or "turning points"

Essentially the goal is to isolate Lines, Diagonals and Curves. I have tried approaches ad infinitum, and so far the following delivers the most Bang For my Buck:

  • Loop through the lines, minding a "vertical anchor line" and a "horizontal anchor line".
  • Whenever the direction of lines changes from the current anchors, this can be seen as a "break" or a "turning point".
  • Store these "break points" in a list.
  • Use the "break point" as the current anchor, and continue looping.

The image below is a "single clockwise pass", the "turning points" are represented as blue pixels accentuated by totally awesome green circles.

Following the lines starting from the top left corner, you should "discover" the same "break points".

The first "horizontal direction change" takes place at the bottom right corner of the M. Up until this point all the horizontal lines go "RIGHT".

The first "vertical direction change" occurs at the bottom of the first "V" shape between the M's legs. Up until this point, all the verticals point DOWN, and after the turn point UP.

This first "pass" uncovers 10 break points, which could be used to analyze the "sub components" (deal with diagonals and curves). However, it's painfully clear that more break points are needed. In the "M", there are 13 which define the shape. So, a "second" pass sounds like a good idea, this time counterclockwise (or in reverse order). This results in different "break points" than the clockwise pass, (again just follow the lines with your eyes...).

Below is a "single counter-clockwise pass", the "turning points" are again visualized as blue pixels surrounded by gratuitous neon green pixels:

Putting these two together, running clockwise and counter-clockwise gives us:

Now we've got 16 break points for "M"! This is good enough, the main lines and diagonals are isolated. There was much rejoicement!

Here's the same for a character with some curves:

Nifty as this might sound, the approach isn't without it's shortcomings. "Staircases" and "Curve Staircases" are a thorn in my side. In such cases, there are "clear" changes in visual direction, yet, neither the HORIZONTAL nor the VERTICAL direction changes in a "staircase". Experience the phenomena in characters including T,F,4,S,a, etc. etc. Witness the F below:

At this point, we say "F it", and move on.

4) Use the "break points" to cut the shape up into segments, and analyze

Loop again. This time looking at the sets of VectorLines separated by "break points":

  • Any set of 1 must be a Vertical or a Horizontal line.
  • Detect Diagonals
  • Handle Curves and "complex shapes"

Here's my "CPU Cheap" approach to detecting a Diagonal in a set:

Actionscript:
  1. protected var _diagonal_buffer:Number=.15;
  2.         protected function isDiagonal(lines_vect:Vector.<VectorLine>,index1:uint,index2:uint):Boolean{
  3.             var first:Point=VectorLine(lines_vect[index1]).start_point;
  4.             var middle:Point=getMiddlePointInVLineSegment(lines_vect,index1,index2);
  5.             var last:Point=VectorLine(lines_vect[index2]).start_point;
  6.             var diff:Number=Point.distance(first,last)-Point.distance(first,middle)-Point.distance(middle,last);
  7.             return Math.abs(diff)<_diagonal_buffer;
  8.         }

The code grabs the "middle point" within a set of Lines. Then it compares the distance from the "sets start point to finish point", with the added distance of "start TO midpoint+midpoint TO endpoint". If the difference is less than "_diagonal_buffer", it's a diagonal. Accept it.

The remaining shapes fall under the categories of "Curves" and "Complex Shapes". Both can actually be treated the same : Based on a parametrized "Curve Accuracy", the remaining shapes can just be divided into "sub segments".

Let's say a Curve has 200 pixels, which might translate to 50 "Lines" once "redundant points" are removed. If "Curve Accuracy" is set to 15, it's relatively easy to loop through the 50 lines, and isolate them into "Sub Lines" grouped by this "Curve Accuracy". Maybe an easier way to picture this is just to chop up a curve by choosing every X points along it's path. If that makes no sense, just check out the code.

There's a few more optimizations, but I can't be arsed to go into it right now. This post is long enough as it is.


Click here for the demo (Comic Sans pictured below):

Download the Source (Requires FlexBuilder, with sdk 3.2+)

Congrats, give yourself a hearty pat on the back for making it all the way through! You have our Gratitude! I personally guarantee future posts with better entertainment value using this code base and it's oncoming descendants :D

Extract shape outline points from BitmapData

April 13th, 2009

On my quest to generate 3D text from non-embedded fonts, after extracting positive and negative shapes from characters, the next step is to analyze and extract the points which define the pixel boundary surrounding the extracted shapes.

There is a "brute" way to do this, simply by looping through every pixel in a BitmapData, and checking weather any surrounding pixels are transparent. This works, but the discovered pixels are not in a meaningful order.

The goal is to create an algorithm which circumnavigates the extremities of shapes, storing the traversed points in sequence, eventually doing this:

My original approach was to grab the first transparent pixel, then loop around its 8 surrounding pixels, run through some rules, then determine the next pixel to move to, until the "3x3 grid" would arrive at its starting point. The "3x3" grid would move along the shape clockwise.

As a side note, it was a relief to see Marios presentation in Milan, where he had pretty much the same approach, except his was moving counter clockwise. One very important "innovation" that I learned from Mario was to double the size of the analyzed bitmap. This prevents "single pixel" lines, which, if you think about it, are very tricky for the "3x3 grid" to analyze.

My implementation was sluggish, and had a few bugs... So I came up with another approach. I guessed that the rules of moving clockwise never change, so I figured a lookup table approach might just work. The idea is to represent the "3x3 grid" surrounding a pixel as a String using zeros for transparent pixels, ones for non-transparent ones. The lookup table would then map the "next position" to the various "3x3 grids". Here's a short snippet of my lookup table:

Actionscript:
  1. protected var _variations:Object={
  2.     "000110110":new Point(0,1),
  3.     "111110000":new Point(-1,0),
  4.     "111011011":new Point(0,-1),
  5.     "111111110":new Point(0,1),
  6.     "100111111":new Point(1,0)}

It's still not the Usain Bolt of algorithms, but a big improvement on my previous attempt. So where did this magical lookup table come from? If you look at the animated gif above, it's actually a screenshot of a "outline pixels generator" tool which I wrote just for this purpose. This allowed me to move along the edge of a shape, as the eye would follow it, and store the "moves" as "rules". You can find the entire lookup table in the zip below.

To test and demo the algorithm, I wrote a quick flex app, click to try it out:



Choose any font, pick a character, then extract the outlines. There is one thorny topic of annoyance. Flash player adds a weird anti-alias to all text above a certain font size. Small non-embedded text appears as one would expect, but in bigger sizes (which is needed for better curve information), there is a "quasi-anti-alias". In the above image,

  • The left-most image is a non-embedded text field, with the "anti-alias".
  • The second image is a "mono-chrome" representation of it.
  • The third is the generated outline (of the mono-chrome)
  • The last one is the original (first image), with the generated outline superimposed.

The outline is not "ideal", all pixels with an alpha greater than 0 are represented as black, making the image "blockier" than necessary. In the "superimposed" image, particularly with curves, there is a clear difference between the original character and the curve. meh. There is a way to get the "aliased" text with embedded fonts, more here. But since I'm interested in non-embedded fonts...





The second "demo" within is an animated "blob", which extracts the outline on every frame. I made this mostly just to "test" the algorithm... So far it's not throwing any errors for me. The frame rate is dependent on the number of "separate shapes and negative shapes". If you have a slow machine I don't recommend this one :D

The next step is to analyze these outline paths and optimize them into "vector information", which I'll blog about shortly.

Click here to try out the demo, then click
here to download the zipped flex source folder. The relevant code is in plain as3 classes, in case flex isn't your thing.

The FlashBar miniconference summary

March 31st, 2009

The inaugural edition of the Flash Bar mini-conference was a raging success! After months of planning, we were treated to a stellar list of speakers:

Amongst the many highlights was this demonstration of liquid dynamics, a collaborated effort between David and Bartek:

The resident DJ treated us to a solid set, spinning a vast selection of styles ranging from Abba to Britney Spears. Unlike some of the sausage fests in previous conferences I've attended, the sound system was kept at an ideal volume, enabling a steady flow of conversation, and just enough backdrop for potential moments of silence.

During the award ceremony and closing raffle, we even managed to get a group shot in front of the main podium with all the organizers, sponsors, speakers and attendees:

I feel honored to have participated in this pioneering event, and look forward to a rich and colorful agenda of future dates!

Mouse interactions on Text Layout Framework InlineGraphicElements in EditMode

March 26th, 2009

One slightly frustrating (but totally logical) feature of using editmode in TLF is that inserted InlineGraphicElements lose their "interactivity". Instead, they are treated the same as text characters, that is, the user can select them as if they were selecting text:

There was a comment In my previous entry about the possibility of mouse interaction with inserted graphics.

I have a way to do this, a bit hackish, but it does the job. Every added graphic needs to be stored in an array. When a user clicks, the code loops through all the graphics and checks if one of them happens to be under the mouse. This is achieved using the trusty old localToGlobal() method:

Actionscript:
  1. protected function handleMouseDown(e:MouseEvent):void{
  2.     var hitrect:Rectangle;
  3.     var bm:Bitmap;
  4.     for(var i:uint=0;i<_images.length;i++){
  5.         bm=Bitmap(_images[i]);
  6.         var ltg:Point=bm.localToGlobal(new Point(0,0));
  7.         hitrect=new Rectangle(ltg.x,ltg.y,bm.width,bm.height);
  8.         if(hitrect.contains(mouseX,mouseY)){
  9.             mx.controls.Alert.show("so, I see you have clicked on an image eh?");
  10.             break;
  11.         }
  12.     }
  13. }

This is very useful if you want to edit the graphic somehow, for instance, in the RTE that I'm currently working on I use this for "table support". In the TextFlow, my tables are represented as bitmaps, when clicked, the actual DataGrid is opened and ready to edit. When finished, a new BitmapData is grabbed and the InlineGraphic in the TextFlow is updated.

I have found one "bug" (or a feature?!), if an image is taller than (somewhere around 1000 pixels), the image is no longer properly "wrapped", and actually starts to cover some text.

It's also possible to listen to MouseMove, and set some rollover/out states to the graphics. You can see a demo here and download the source mxml file here.