I had come across Texture Networks: Feed-forward Synthesis of Textures and Stylized Image from Cornell University a while back in my regular monitoring of the API space, so I was pleased to see Algorithmia building on this work with their Deep Filter project. The artistic styles and textures you can apply to images is fun to play with, and even more fun when you apply at scale using their API. While this was fun, what really caught my attention was their post on their open source AWS AMI for training style transfer models--where you can develop your own image filter.

I fired up an AWS instance, and within 48 hours I had my first image filter. For some reason, I went to bed that night thinking about drone video footage and began wondering if I could apply Algorithmia's filters, as well as the custom filters I create to drone footage. The next day I started playing with my prototype, to see what was possible with applying this type of image filtering to videos.

My approach has three distinct features:

While the videos I've been able to produce so far are interesting enough, I am intrigued by the process of defining and applying the filters. There are a handful of things going on here for me:

Algorithmia has enough filters to keep me interested for a while, but this is where I think the real value will exist--assembling well trained filters. If you can craft exactly the right filter, I think it can be used to define entirely new virtual landscape derived from the physical world around us. Some of the filters offer a pretty extreme shift in the video landscape, but some of them provide just enough polish to make you think you are in an alternate reality.

I'm not sure where I'm going with this work. I'm finding it oddly soothing to define new filters, and apply them to my drone videos from this summer. For now, I'm just thinking of interesting filters to build and apply the filters to the videos I already have. It also has me thinking about how I can take my drone out to get more shots--from both natural and urban landscape.

For near-term future, I'm just getting to know what the filters produce, and how certain drone shot(lighting, color, terrain) respond to each filter. It has me thinking a lot about what would make a good filter, and what types of images I can use to train image filter algorithms that can be applied in this way. If you want to learn more about what I'm doing you can submit a commit via the Github issues for this project, or you can contact me directly.