Note: Super sorry about the seriously shaky footage above. Next time I’ll make sure it doesn’t happen.

Yesterday, I spent about 12 hours making what I thought would take “a couple hours” — the Flow Receiver module. In the video above, you can see me demoing the system. Everything is up and running and working seemingly well. This means it’s time to start creating “art.”

I designed the UI for this thing to be as simple as I could make it while also accounting for the fact that I’d be using it on a touch-enabled display. It only has three different “views” which you can see below.

+ View larger on Flickr.

+ View larger on Flickr.

+ View larger on Flickr.

Everything for Flow Receiver is built using Adobe AIR. It communicates with Community Core Vision using CCV’s built-in UDP->TCP bridge which sends out TUIO data associated with the tracked blobs.

Flow Receiver intercepts this TUIO data and displays the incoming data in the right-side panel. I’ll also use this software to interface with the Flow Storage back-end module for creating projects and recording the TUIO data to those projects.

When in “recording mode”, Flow Receiver buffers the incoming TUIO points until its internal “cache” is full, then sends that info via HTTP POST request to Flow Storage which dumps it into a MySQL database. I’m also storing a timestamp (to the millisecond) with each TUIO point collected so that I can later determine velocity data between “frames” as needed.

When it’s time to export a project’s data, I’ll use Flow Receiver to export the data as either an XML or CSV text file.

Not much else to it than that; it’s a super-simple application.

On a side note, I’m pretty amazed at what we can do with technology these days, the number of options from which we have to choose, and the speed at which we can produce this stuff. I’ve somehow managed to create a fully-functioning toolset in a total of about 20 man-hours, cobbled together from open source libraries, applications, and various other technologies — all at pretty much zero cost (aside from the time put in).

Well, that itch has resurfaced, and I realized I was making things way too complicated. This time, I’ve turned to using some open source tools for the data capture portion of the project. Components include:

• Computer
• Webcam
• Community Core Vision (CCV)
• Color Tracking Patch for CCV
• Processing TUIO Client API

It’s actually really simple to get set up. Here’s a little demo of how it all comes together.

Next steps include making a client to listen for the TUIO data generated by CCV. I’m probably going to opt for an AS3 AIR client, since it’ll be fastest for me to create. Then create a web application that captures and stores this data. This I’ll build in CodeIgniter, again, since it’ll be fastest for me to create in this environment.

I’ve got a lot of stuff on my plate this month, and I want to participate in this month’s NaNoDrawMo on top of it. So, we’ll see where things net out.

I recently discovered a huge disconnect between text-based 3D graphics programming and my brain. It was quite an unpleasant discovery.

Out of frustration, I decided to turn to a more graphical, procedural approach. For some reason when it comes to 3D, I really need a UI and “objects”. It’s all too abstract when I’m dealing with lines of code.

So I’m trying out visual procedural programming. And I’m starting with Derivative’s TouchDesigner. I was introduced to the application long ago, back in 2005 at FITC. But it’s gone through a complete rewrite and UI overhaul.

A certain Steve Mason (former coworker at Odopod and most recently at Obscura Digital) re-introduced me to the application a couple months ago. He’s done some seriously amazing stuff with it already.

And tonight, I made a spherical 3D particle cloud. Nothing revolutionary, but there is some serious power and possibility embedded in this application. The UI is absolutely next level and intriguing. I highly recommend you download it and have a run through the video tutorial screencasts. It’s a Windows-only application (sadly). And it’s still buggy since it’s in very active development. I’d recommend you download the latest experimental release. I had more luck with it than the main release on their Downloads page from May of this year.

It definitely took some time. But it’s finally out. And just when I thought I had all my bases covered, Adobe has just released a new public beta of AIR along with a new public beta of Flex 3. This renders two very crucial tutorials that I put together obsolete. I’ll be working this week to update those two tutorials.

For now, if you’re willing to brave it, here are the links a few folks have been waiting for:

• visp-vj.com – The official site for visp.
• visp Google Code Project – This is the project’s home for source code, bug tracking, and other related nerdiness.
• Mailing List – This is the Google Groups mailing list for visp.

I do want to say that visp has definitely been created with programmers in mind. You build your custom media from the ground up. I’ve tried to simply encapsulate things into the flash.display.Sprite class as a base (for custom media) and provide an easy API for handling both software UI input along with MIDI input.

If you’ve got questions, problems, bugs, or simply need some help, please head over to the Mailing List. I won’t be answering anything in the comments of this post. On the list, I’ll do my best to answer in a timely manner. And if you’ve got feature requests, they’ll be duly noted. But remember, this is opensource software, so you’re more than welcome to build the features yourself.

I’ve got a long list of excuses and a short list of completed milestones. However, visp is still moving forward (believe it or not). Here’s what I’ve done since the first preview post:

• Revised functionality (by taking a lot of the more complicated and not-100%-solid features out)
• Revised interface size so that it’ll work with 1024×768 screens (since that’s likely to be the minimum resolution for most laptop VJs)
• Created a consistent API for creating media modules, filter modules, and transition modules
• Set up the proper destination site and domain along with the Subversion respository
• Combed through source code to keep formatting and variable naming consistent
• Fixed some bugs
• Updated codebase to use the latest public release of Apollo (beta 1) and Flex Builder 3 (beta 1)
• Rewrote the MIDI proxy to be usable with all MIDI devices and not dependent on Processing (to reduce memory and processing overhead).
• Rewrote the input interface so that media can be controlled via on-screen UI elements (sliders and buttons)

I’ve still got some more items to go. I want to make sure folks have proper documentation and tutorials for creating their own media, filters, and transitions. I also want to make sure that simply getting the projects set up in Flex Builder 3 is very straight forward (i.e. step-by-step instructions). There will be a 1.0 release in mid-September; I have an Adobe Edge Newsletter article being published on visp, so it has to be done.

Thanks for your continued patience, everyone.

I want visp to be useful (and usable) for anyone that decides to try it out. So, I’m deciding to hold off on releasing for just a bit longer until the next public alpha / beta of the Apollo runtime is released. I might do a small “preview release” of my own that supports the current public alpha version of the Apollo runtime, but if I do, then the release will not have all of the functionality in it that will make it a useful application.

I’ll try to work toward that this week, as I’d love to get it into peoples’ hands. Thanks for your patience.

Everyone, I’m caught up in a project at work that’s forcing me to put a hold on the release of this software. However, I realize I need to put out some proof that this thing exists. So, above is a screen cap from the software I’ve entitled visp.

And I want it to be no secret that the title comes from these words: visual performer. I also want to say that this software was designed as a prototype for a personal proof-of-concept; it isn’t intended to be software that’s supported or salable. And I actually haven’t taken the time to reflect on whether or not my concept was proved as a result of its development. Anyway, enough jibber jabber. Here’s why I built visp along with a general list of its features.

The Reason(s)

Too much UI management. That’s why I hate VJ software. More often than not, VJ software gets you sucked into managing the UI during your performance rather than putting the emphasis on performing. That’s something I never really understood. I always wanted to be gestural with my performance. I wanted to be able to physically say, “Now I want the visuals to blow up.” or “Now let’s start highlighting this sound.” Instead, I found myself using the keyboard and mouse to manage some UI element, forcing me to miss those precious cues. Granted, yes, I could use some sort of controller (MIDI or OSC or something hacked together). But that leads me to my next irk.

Too much focus on clip triggering. I don’t think like this. I’ve got an interactive media background. I think in dynamics. I think in variables and rule sets. I don’t think in terms of linear media (generally). Almost all the software out there has this huge focus on layering video clips and that’s just not for me. So, yes, while I can gesturally trigger clips using some sort of controller, adjusting layer opacities and filters using sliders, I’m still limited to the linear pre-rendered footage in my library. And it just doesn’t suit my tastes. So, you may ask, why not use something programmatic like Processing or Max/MSP/Jitter or Touch?

Maintaining visual continuity is so difficult. In a lot of these other software environments, you can build very beautiful, very dynamic, and very interactive visuals. However, moving from one visual construct to another is no simple task. There’s no “meta” application that really lets you string these pieces together in a unified environment (unless you build it yourself). I started off actually doing just that for Processing applets but was instantly thwarted due to a bug in the latest build. Fortunately, I had access to Adobe’s new desktop platform, codenamed Apollo.

The Solution

Apollo came at just the right time. I could leverage my abundant Flash / ActionScript kung-fu. I could develop a standalone desktop application (that gave me access to native OS windows). And I could leverage the very easy-to-use Flex paradigm of application development. The Flash 9 Player was miles ahead of the Flash 8 Player, so I thought I’d take her for a spin.

Also, I wanted as much of a hands-off-the-keyboard-and-mouse experience as possible. I need to evaluate still where I’ve succeeded and where I’ve failed. Frankly, right now if you don’t have a MIDI controller, this thing is useless.

The Features

I’m just going to list these off in no particular order and with no real explanation. That’ll all come when it gets released.

• Support for development of your own “Modules” that essentially give you a Flash canvas to play in and do whatever your heart desires
• Module browser using thumbnails you make yourself
• Full MIDI support using an external, standalone Java applet (also opensource)
• 6 slider-like MIDI inputs for use in your own custom “Modules”
• 4 velocity-aware button-like MIDI inputs for use in your own custom “Modules”
• Support for transitioning between Modules (using built-in Transitions). You choose the duration and the transition effect.
• Change the stage’s background color (wow)
• Two active filters can be going at a time, with up to three MIDI assignable sliders – and a UI interface that you build yourself
• Output your visuals in a separate, chrome-less window of 320×240 or 640×480 resolution (thank you Apollo team)
• Real-time 640×480 preview screen
• BPM Tempo Tapper for assigning some beat-driven automation to the input sliders and buttons. It’s not audio-driven but VJ-driven. You gotta tap the tempo yourself.
• FPS (frames per second) display
• OS X and Windows compatible — seriously — I’m actually able to jump back and forth with the same codebase and same modules without any hiccups or platform-specific code!
• It’s internet aware – that’s the beauty of Apollo.

That’s all I’m going to say about this thing for now. But feel free to post questions. I’ll be happy to answer them.