A few months ago I heard about this great app called Composite. This app lets you draw to the canvas using the video feed input as paint. It’s really quite cool and when I started learning about what Core Image could do I started to wonder if similar thing could be done easily with core image filters.
Well the answer is yes, but I’m not going to just copy what composite is doing. I wanted to do something more . . . useless. So, I’ve implemented Conway’s Game Of Life that uses the video feed as the cell images. I’ll go through how I did that in this post.
But, I’m getting a little ahead of myself, as I am wont to do. The process of drawing using the video input as canvas involves two technologies, Core Image and Core Graphics. We’ll be drawing a CGImage using core graphics, that CGImage can then be converted into a Core Image version, and used in whatever series of CIFilters we want to put them through. Depending on resources, we can use combinations of drawing and core image filters to perform all kinds of image and video manipulation.
We’re checking for for retina devices in this code block (create an instance float variable called ‘scl’ in our header). When we render to the screen using openGL we’re in pixels, so we need to scale everything up if we’re on a retina device.
We’re almost to some results. Next, we need to create the method that will draw a CGImage, and then convert it into a CIImage. Add the following method declaration to your header and implement the method in the .m file:
We’ll be modifying this method to draw our Game Of Life state, but for now we’ll just draw a great big oval. The last four lines of the method create a CGImage from our context, create a Core Image image from the CGImage, release the CGImage and return the Core Image. By using these steps we can draw anything with Core Graphics, convert it into a CIImage, and then use that image in a filter.
Now that we’ve set up the method to create our CIImage, we need to invoke it and use the CIImage in a filter. We’re also going to scale our incoming video up to match the size of the CIImage that we’ve created. Change the captureOutput method to the following:
The first filter we apply to our image (the incoming video still) is the CIAffineTransform that scales up (or down) the video to fit the size of the screen. The second one, the CIMinimumCompositing uses the image we created (white oval on black) and compares it to the scaled still. By pixel this filter uses the minimum color value of the two values. Since black is the absolute minimum and white is the absolute maximum in color values, we’ll get a black oval around the outside and whatever color from the video frame.
Importing the Game Of Life
Now it’s time to import our Game Of Life model. I’m not going to cover the implementation of the Game Of Life. I based it on a version created by Alan Quatermain. I added a couple extra things beyond the simple execution of the Game Of Life, a random seed program that populates some of the cells, and a method to add a specific pattern (crawlers) to the grid.
Copy the files to our project, add the import statement, and create a Game Of Life instance variable in our ViewController class.
Next, initialize the GOL by adding the following lines of code to the viewDidLoad method:
GOL = [[GOLModel alloc] initGameWidth:30 andHeight:20];
Now that we have an initialized GOL object, we’ll need to change our drawGameOfLife method to visualize the state of the game.
1) We’re getting the cell width and height by dividing the number of cells by the size of our screen.
2) Next, we are clearing the screen, but with an opacity of .4, this will make cells take several turns after they’ve died to completely dissappear. It makes the visual a little more interesting.
3) Then, iterating through each cell in our GOL model, we calculate the starting x and y coordinate for that cell.
4) We next draw a white circle at the appropriate position if we’ve found an active cell. We add +1 and use a width and height – 2 to leave a littel padding between cells.
5) This code is the same as before, converting the CGImage into a CIImage.
If you run now you’ll have the a bunch of little circles, randomly placed, instead of one big circle. You should be able to see the video feed through the circles.
Progressing the Game Of Life
Now let’s get the game going. We’ll need to add a button to the xib so we can turn on and off the game. Give the button an IBOutlet called playGOLButton and hook up a method called playGOL. Also, we’ll need a timer, add an NSTimer *refreshTimer instance variable as well.
Next we’ll implement the method playGOL and add another method, updateGOL that we’ll call with a timer. Add the updateGOL method to our header file. Here’s the implementation of those two methods:
In the first method, we’re just changing the buttons label, and then either creating a timer for the updateGOL method or turning the timer off.
In the updateGOL method we are calling the update method on our GOL object, which progresses the game one step. Then we redraw the maskImage based on the updated GOL model state.
We were calling that same line of code in the captureOutput method. Take the line out of the captureOutput method, we don’t need to run that code so often. We should get some performance benefit our of running the drawGameOfLife method only every half second, instead of every frame. Also, add it to the viewDidLoad at the very end, otherwise we’ll have a black screen until we push the play button.
Changing Cell Values in the Game Of Life
The next thing we want to do is add multitouch interaction to our GOL model. When we touch the screen, for each touch, we need to calculate the position of the cell in GOL and turn it on or off. The first thing we need to do is turn multitouch on for our view. Put this in the viewDidLoad method:
Next we’ll add a touchesBegan and a touchesEnded method, these are almost identical:
loc = CGPointMake(loc.x, screenSize.height - loc.y);
intcolWidth = screenSize.width / GOL.width;
introwHeight = screenSize.height / GOL.height;
intx = floor(loc.x / colWidth);
inty = floor(loc.y / rowHeight);
[GOL toggleCellX:x andY:y];
maskImage = [selfdrawGameOfLife];
The first thing we do is iterate through the touches set. For each touch we first flip the touch location, because the view uses one coordinate system and Core Graphics uses a vertically flipped system.
Once we’ve done that we can use the screen position and the knowledge of the cells size to calculate the coordinate of the cell we’ve touched in the GOL model. Then we call the GOL -toggleCellX:andY: method to turn it on or off.
Finally, after we’ve iterated through all our touches and flipped their respective cells, we update the drawing of the GOL in Core Graphics and return the CIImage for our CIFilter chain.
The only difference in the touchesBegan method is that if the game is currently active, instead of toggling an individual cell, we call spawnWalker. This method creates a specific Game Of Life pattern at the touch point. This self-sustaining pattern moves across the screen unless it encounters another set of active cell.
If you build and run you can manipulate the Game Of Life via touches, either while the game is running, or when it’s stopped.
Here are a couple of walkers in action:
A Few More Unnecessary Additions
Let’s put a few unnecessary visual enhancements to this app. First, let’s use a perlin noise generator to determine the color of the cell. Import the CZGPerlinGenerator class into our project. You’ll need to set up the generator object in the viewDidLoad block, like this (also, import the class into our header file and create an instance variable called perlin):
perlin = [[CZGPerlinGenerator alloc] init];
perlin.octaves = 1;
perlin.zoom = 50;
perlin.persistence = 0.5;
Make sure that block of code is executed before our maskImage = [self drawGameOfLife]; line. Now in our drawGameOfLife method change this line: