This is a quick post (or should I say nanoblog?) to share my new home page. It’s at parumi.org (I did a big update on Des 2nd). After using so many web2.0-social networks I felt the need to have a boring, simple and static home page written in html. Actually, I happily failed at writing directly in html just because I had wiko at hand.

Wiko stands for wiki compiler, and is the project name we gave to some python scripts David and me have been lately developing for personal use. It basically convert wiki text files to html, LaTeX and blogs; as is easy to imagine it combines very well with your favorite version control system to create an actual wiki (it’s collaborative but only through committing to the version control system). Visit the wiko home page to learn more. And maybe use it.

I’ve been a long time longing for a N700 or N800 so this short message that just popped into my inbox made my day ;)

Assumpte: N810 maemo submission accepted
Data: Fri, 9 Nov 2007 18:17:31 +0200 (EET) (17:17 CET)

Congratulations! You have been accepted to the N810 maemo
device program. We will send your discount and instructions
as soon as the device is available in your selected shop (soon).

maemo team – http://maemo.org

The N810 maemo device program aims to offering a low price for the new Nokia N810 Internet Tablet (99€) to the active contributors of the maemo community, open source programmers, designers, bloggers and the like.
I’m eager to have it into my hands. We’ll see how hard it is to port Clam and other Linux audio apps to it.

Since the change of SVN for CVS in Clam (a year ago aprox) we do not explicitly tag the releases and that’s fine because SVN revision numbers are global. On the other hand, SVN do can create tags but it is dangerous because a tag is exactly the same thing as a branch. So SVN doesn’t prevent to commit to a tag.

Our tagging approach is very simple and proved useful: just write the revision number of each release in the CHANGES files. This simplifies the release process and also the way to make diffs –since you always use the same syntax.

Now an example: Let’s say we want to see changes in SMSTranspose files from last stable release (1.1.0):

1) Look for the svn revision corresponding to a stable version in the CHANGES file


NetworkEditor$ head CHANGES

2007-??-?? NetworkEditor 1.1.1 SVN $Revision: 10220 $
''
*

2007-06-08 NetworkEditor 1.1.0
'More eye-candy, please'
* SVN Revision: 10216
* Using CLAM 1.1.0
* New examples
* genderChange: fully working now and with an interface

So version 1.1.0 is revision 10216.
By the way, you maybe are curious on this $Revision: 10220 $ part of the first line. This is a SVN variable (or pattern). Each time you commit this CHANGE file the number gets updated to the current revision. That means that we actually never write the revision numbers in CHANGES files, we only have to remove the “$” when we decide to tag the release.

2) Now diff the files of interest on that version and head


NetworkEditor$ svn diff -r 10216:HEAD src/processing/SMSTranspose.*xx
Index: src/processing/SMSTranspose.cxx
===================================================================
--- src/processing/SMSTranspose.cxx (revision 10216)
+++ src/processing/SMSTranspose.cxx (revision 10281)
@@ -20,11 +20,15 @@
*/
#include "SMSTranspose.hxx"
-#include
+#include
#include
...

Last, a quick tip for the gvim users: pipe the diff result to gvim using -R – options:
$ svn diff | gvim -R -
And take advantage of vim syntax highlighting and quick navigation.

This post is just a quick note. I’ll extend on it later.

Imagine you have a data flow graph with nodes A, B, C, D, G. Of these nodes, A and B are a source of stream and G is a sink. Now, say, the static scheduler gives you this iterative list of firings: B, A, C, B, A, G, B, D, D, G, D, A, G, C, B, G

Now, your application is real-time and, as usually, it is call-back based (the hardware awakes a process in a regular basis providing and consuming data).

Data flow scheduling is very interesting so far, but a question arises: In practice how can you use this scheduling from the call-back process?

Actually the obvious solution I can think is not good at all: you can do call-back adaptation of a blocking interface. That implies leaving the call-back process to just do buffering, and have another thread that executes the data-flow. Where source and sink nodes (A, B and G) can be blocking if data is not available. But in most cases minimize buffers on the inputs and ouputs is desired. So you want the scheduler that collects input as regularly as possible. This is an admitted (SDF scheduling paper, section V.C) limitation of the model.

I haven’t found yet any research focusing on that problem (though I admit I still have a bunch of related papers to read). So any feedback on this will be greatly appreciated. However, I have some ideas on how the solution could be, I’ll post about it later.

In CLAM related papers we use to define the synchronous flow as a flow where messages get produced and consumed at predictable —if not fixed— rate. This definition relates very well with my intuitive idea of synchronicity. However, having seen examples of multi-rate data flows and having implemented dynamic schedules, I didn’t feel very comfortable with this definition. The main concern is that node’s firing does not necessarily need to be predictable.

Recently, reading some papers from Edward Lee and others I saw the light:

A synchronous data flow is a graph of synchronous nodes. Where a node is synchronous if we can specify a priori the number of input samples consumed on each input and the number of output samples produced on each output each time the node is fired.

This is a extremely simple property of the graph itself and it is independent of its scheduling strategy. But the point is that it is not so intuitively related with the synchronicity idea. This great paper from Lee and Messerschmitt shows (very formally) how to know if an iterative scheduling exists or not. The condition is simple: the rank of the graph’s incidence matrix must be the number of nodes minus one. Also, if such scheduling exist, there is an algorithm that finds it. The algorithm itself is simple to understand but not so to implement since it involves linear equations solving and exponential search.

To make it clear, an scheduling is a list of executions that gets repeated —being for one or many processors—, and necessarily needs to be iterative since an infinite stream is assumed. Of course, an iteration is a list that may include some or all the nodes many times.

I just discovered this article on Linux kernel management style written by Linus Torvalds. It’s amusing and ironic in the best Linus style. For example, his first recommendation:

First off, I’d suggest buying “Seven Habits of Highly Successful
People”, and NOT read it. Burn it, it’s a great symbolic gesture.

I think these paragraphs summarize quite well the core of the philosophy:

So the name of the game is to avoid decisions, at least the big and painful ones. Making small and non-consequential decisions is fine, and makes you look like you know what you’re doing, so what a kernel manager needs to do is to turn the big and painful ones into small things where nobody really cares.

It helps to realize that the key difference between a big decision and a small one is whether you can fix your decision afterwards. Any decision can be made small by just always making sure that if you were wrong (and you will be wrong), you can always undo the damage later by backtracking. Suddenly, you get to be doubly managerial for making two inconsequential decisions – the wrong one and the right one.

Later he talks about how to put blame and criticize. I do strongly agree on how sense humor plays an important role here.

Some humor can help pad both the bluntness and the moralizing. Going overboard to the point of being ridiculous can drive a point home without making it painful to the recipient, who just thinks you’re being silly. It can thus help get through the personal mental block we all have about criticism.

There is one topic I’d like to do a series of small articles: automatic releases. This is something we have been improving more and more in CLAM and now I feel we are getting it right. In a nutshell, managing a CLAM release used to be very time consuming (it could take months!) now it’s like pressing a button.

We’ve done a lot of things: keep the code stable in all platforms (automatic tests and testfarm is a great help here), automate the process of generating all release files which includes source tarballs, mac-osx and windows installers, debian package, and more. Of course, the apps must have the version embedded (available in the about box, for instance). And finally, all those files have to be published accessible and well organized in the web.

And there’s more complexity to manage here: we provide “subversion” or “development” releases on one hand and “stable” releases on the other. The first ones look like CLAM-NetworkEditor-0.4.3_setup.exe, the second ones look like CLAM-NetworkEditor-0.4.4~svn09719_setup.exe. This last long version means: this is a svn snapshot of NetworkEditor (revision 9719) in the way of the stable version 0.4.4.
And of course, we are managing versions of four applications and one set of libs. Thus, the complete version of one app includes the CLAM lib version.

On the lowest level of this infrastructure we have the scripts that automatically retrieve the correct version. Today I’ve finished a code clean-up of the python module we are using. The main functions of the versionInfo.py module is this:

def versionFromLocalInfo( product='CLAM', changesFile="CHANGES" )

Keep reading to learn how it works.
Read the rest of this entry »