Thursday, 30 July 2009

Do a bit of plumbing with Yahoo! Pipes

A demonstration of a simple Yahoo! Pipe.

Yahoo! Pipes has been around for a while now, but I would like to showcase this excellent tool by demonstrating the creation of an 'On This Day In History' type feed which I have recently created. It is a simple 'Pipe' - many fantastic and complicated uses are possible for mashing up feeds from all sorts of sources - but I hope this is a good 'starter' demo which will encourage people to build something useful for their business sites. You can put a 'Pipe' on any web page, and of course they are particularly suited for sucking into a Webadvantage® module, so businesses who use Glanton Solutions should be particularly interested. Why not create something relevant for your customers and/or employees to give your site enhanced individuality and 'stickiness'?

My demo comes from the website I manage for the Crimean War Research Society. Rather esoteric you say? Yes, just a bit! However, this particular 'Pipe' can be easily modified to fit any number of situations.

It all started about four years ago when I decided that some sort of 'On This Day' feature would be good on the society website - not just 'on this day in history', but specifically 'on this day during the Crimean War'. I was inspired by the fact we had a chronological list of main events during the war on our site, amounting to 103 dates from 1853 to 1856 inclusive.

Could I expand this and re-jig it so it was a list of 365 - plus one for Feb 29 1856, the day in that leap year when the armistice was signed - and make the right event(s) image appear at the top of our homepage on the right date...?

First I created a spreadsheet with my 366 rows and two columns, one for date and one containing the text. This required considerable off-and-on research and wasn't completed for a few months. Next I found a piece of free javascript which had been written to put a different image in an iframe on each of the seven days of a week. Hmm... surely I could play with the javascript and make it do the same, but for every day of the year, including leap years?

So I set about making 366 gifs of my daily texts. This was quite tedious! I then played with the javascript - again quite tedious as every day of the year had to be identified. But, however large and clumsy this appears (the javascript file is quite long and there are 366 gifs sitting in their own folder at a server), it worked; the resultant code for the iframe is deceptively neat and tidy and can still be seen in perfect operation today at

Below is a diagram of how it works [click on each image to view at full size]:

However, more recently I had become fascinated by the idea of providing the whole thing as an RSS feed so that people could put it on their own site, or suck it into a feed reader. How to do it? To start with I thought a combination of Feedity and Yahoo! Pipes might do the trick. Feedity is a very neat tool with which you can create an RSS feed from a webpage which doesn't have one. It is very cute, and I have used it elsewhere, but the free version only allows you up to 10 posts in the resulting feed. As I'm way too cheap to pay for one of the pro deals at Feedity, this clearly wasn't going to work for my 366 items. Yahoo! Pipes on its own would clearly have to be the answer - I would have to work out what was going to be the best format to suck in my data. I looked at XML, but ended up converting my original XLS spreadsheet to CSV (Comma Seperated Values) and adding a few columns to define the navigation and add a link to the final feed. Then I uploaded it to a server.

The whole Pipes interface and drawing board is wonderfully designed: you really feel as if you are doing the plumbing! The first thing to put on the drawing board was a 'Fetch CSV' module, which happens to be the first one in the 'Source' folder. It was then a simple matter to enter the URL of my CSV file and give names to its five columns. Now at this stage all you would get as output is the whole CSV file as 366 posts in an RSS feed.

So from the 'Operator' menu I selected the 'Filter' module, and the 'Date builder' from the 'Date' menu. See image below. The date for my purposes had to be 'today' and this had to be connected to the 'Day' and 'Month' columns of the CSV file, restricting the filter's output to the right day in the right month.

The resulting Yahoo! Pipe is here.

The great thing about Pipes is that there are hundreds published and you can clone ones that you think will work for you to your own free account. There you can modify them precisely to your specifications - or if no one has done something similar in the past, you can start from scratch. The potential for business websites is enormous: from mash-ups of news feeds about your particular industry, through comparative prices of raw materials to 'quotes of the day' or 'did you know that' - the latter two ideas could use a clone of my Pipe as a base from where to suck in you own CSV file. The output of your Pipe can be pumped into various social media sites, got as a 'badge', RSS, JSON, PHP or CSV, or sent by email or phone.

Our next post will be Glanton Solutions personnel giving their personal recommendations for free tools and apps they can't live without. Stay tuned!


  1. First, he was extremely dismissive of me, when I tried to ask questions about what he was doing he addressed me in the manner of "Look lady . . . " plumber sunshine coast

  2. For quite a while numerous pipes administrations have been making an awesome showing with regards to in giving crisis plumbing in Midlothian.

  3. Several professional plumbing services are available here to meet the plumbing needs of the people.
    Emergency plumbing

  4. Little breaks in the house can be settled all alone, if we know somewhat about pipes and we have the correct apparatuses with us. Plumbing services

  5. A neatly done plumbing job, can win a person the appreciation of the people who come over to his house. Plumber

  6. A plumber's work varies in a lot of aspects of commercial and residential plumbing service and maintenance to drain and sewer backup, he is nearly involved in all of the works which range in this arena. toilet repair

  7. كيف تحقق النجاح بالنسبة لخدمات كشف تسربات المياه بجدة حتي نكون علي اقتناع تام بأننا شركة كشف تسربات بجدة تقدم خدمات لا مثيل لها من خلال فنين يمتلكون القدرة علي اكتشاف جميع انواع التسريبات من خلال تدريبهم العالي علي الاجهزة الحديثة التي تساعد علي ذلك بدون هدم او تكسير كل ذلك سوف تجدونه عندما تتواصلوا مع شركة كشف تسربات المياه بجدة والتي تقدم كل ما لديها لمساعدتكم