blog

Go With the Flow?

by | Aug 7, 2013 | Developer Blog | 3 comments

Yesterday, an announcement of a new Kickstarter project came through my feed reader. In and of itself, that’s not remarkable — seems like at least one Kickstarter announcement crosses my radar every day lately.
This one, though, is to fund the development of “NoFlo,” a new software development environment that uses Flow Based Programming to put development tools into the hands of non-coders.

Image from http://www.kickstarter.com/projects/noflo/noflo-development-environment

Image from http://www.kickstarter.com/projects/noflo/noflo-development-environment


They make some pretty bold claims:

If you’re a programmer, deep down, you know code is bad, it rots, and with each new line your software accrues complexity debt. New code means more bugs, more refactoring and more time needed to bring someone else up to speed. The inevitability of spaghetti-code is the result of one simple thing – text. It doesn’t matter if it’s JavaScript, Python or C++, text-based code is ineffective at describing the macro flow of software.

When we design software, we usually start by drawing boxes and arrows on a whiteboard. As the software evolves, people rarely go back to the original drawings, and the text-based source code becomes the lone key to understanding how different pieces of the system fit together. With NoFlo, the diagram is the software!

On the one hand, the idea of new tools excites me. On the other hand, I’ve been in this business long enough to have seen breathless claims like this over and over again. In the early 90s, expensive CASE tools were supposed to do this. Programming without programmers! In the late 90s, Visual Basic was supposed to do this (and then JavaBeans, and OLE/COM/DCOM to a different extent). The (now discontinued) Quartz Composer in OS X provided exactly the same kind of flow-based programming.
In the digital media world we already have a lot of experience using this kind of environment — the commercial Max and open source Pure Data have been in use for decades. As someone who learned audio synthesis on a huge modular analog system, the idea of plugging inputs to outputs to create complex systems is something that I’ve had some experience with. The one claim that I wouldn’t make about a system built on the same idea is that it prevents spaghetti code, especially since most of the Max and Pd patches I’ve seen actually look like literal spaghetti:

image from http://cs.smith.edu/~jfrankli/studentResearch/constance/pictures/camerahmm1.jpg

image from http://cs.smith.edu/~jfrankli/studentResearch/constance/pictures/camerahmm1.jpg


I also can’t quite wrap my head around the assertion that source code living in text files is the most significant problem in software development. I’ve written here before about my fascination with Knuth’s concept of Literate Programming, or Jon Bentley’s line in the preface to “Programming Pearls” that you should want to curl up in front of a fire with a snifter of brandy and some source code. Is the problem with software that we’re writing novels instead of IKEA assembly manuals?
On the other hand,  it’s easy to point to many artists, musicians and composers who are using Max and Pd to develop deep and complex interactive music or video processing systems by plugging stuff together and tweaking.
One of the NoFlo team members is also responsible for ‘meemoo‘, a browser-based flow environment for building hackable apps in the browser. It comes with a ton of useful components already built-in — I was able to quickly patch together components that read from my webcam, tiled the image and then rotated it:
meemo quickie

meemo quickie


It’s a lot of fun.
They also point to a book on the topic of Flow-Based Programming — the second edition is available from Amazon, or you can read the full text of the first edition at the author’s website.
It’s interesting that this book is focused on the concepts of flow at a very low level  — he describes each of the components being connected as separate processes or threads, and the issues related to concurrency rather than focusing on a click and drag UI to wire components together.
At what point does an application built like this become too unwieldy to be practical? Is writing source code really the hardest part of programming, or is it harder in practice to bring the kind of algorithmic thinking to a problem that you need to develop a solution? Will systems like this find the next 10 years of our careers doing nothing but fixing bizarre designs started by amateurs?
Or is this actually the holy grail of software development, and it’s finally time for me to open my gourmet ice cream place and earn an honest living?

+ more

Accurate Timing

Accurate Timing

In many tasks we need to do something at given intervals of time. The most obvious ways may not give you the best results. Time? Meh. The most basic tasks that don't have what you might call CPU-scale time requirements can be handled with the usual language and...

read more