It’s been quite a while since #MakeOpenData took place, the event where trainshare transformed from being purely an idea to something more tangible.

Since @AdrianKuendig, @visualcontext, @koma5 and i weren’t able to deliver a running prototype until the end of the event, we continued working on the App as well as the API whenever we had some spare time. While it’s neither done nor available in the AppStore yet, we just wanted to give you a sneak peak of what’s to come.

As you can see, from the video above, there are still things need to be figured out or refined. While we might think of some ourselves, we always welcome your suggestions, so don’t hesitate.

Well that’s it for this time around, thanks to my trainshare team members, the awesome guys behind #MakeOpenData who made this event possible, journalists and bloggers who wrote articles like this, this and this. You rock!

Socket.io

Abstract - It’s a Javascript wrapper library written for Node.js which makes the usage of websockets a lot easier. It also takes care of supporting older browsers with multiple fallbacks. By using socket.io, creating “realtime” applications is a matter of minutes.

Meteor

Abstract - Meteor is an opinionated toolset of common Node.js npm packets and some custom code. Among many things it enforces the use of Fibers, Socket.io and MongoDB both on the server aswell as on the client side.

Some explanation regarding what i said during the talk for those who weren’t present. I’m well aware that i did not sell Meteor as the next big project that everyone has to pick up. The reason is simply that it’s really early in it’s development process. Additionally i think it’s highly dangerous to sell Meteor as something for beginners, since it’s magic bits hide a huge amount of logic a beginner normally doesn’t need to use in order to build a hello world application. By embracing Meteor the novice programmer isn’t learning about those hidden areas which probably lead to unexpected results here and there. After all distributed systems are hard.

An example for such a use-case is the todos application which comes bundled with meteor. It’s nicely done and quite easy to understand, but what happens if let’s say 2 users, with a high latency connection to the server, update the same entry at about the same time? Your changes will be reflected immediately on your machine, with the syncing following a few seconds later, a few seconds later from there you’ll might be surprised that your original changes have been overwritten by the second user. With a traditional stack you’d do some locking or at least insert versioned entries, however with MongoDB this is not supported by default. You’d be able to implement versioning youself by using timestamps and inserts for every update but why the additional overhead?

Just to state it again, the words above do reflect my personal experience when playing with Meteor 2 weeks after it’s initial appearance in April 2012, i’m happy to give it another try once it’s some more mature.

As for the @JSZurich event, thanks to @ikr for hosting us and @seldaek for organizing it.

Sharing a train, something the swiss twitter community did for years via the hashtag #trainsharing should now be brought to the masses with a dedicated app.

My goal is to realize the backend as well as various mobile clients with the help of other hackers during the make.opendata.ch hackathon to show off what can be done by leveraging publicly accessible data. And just for the fun of jumping on the buzzword bandwagon, it’ll be SoLoMo (social-local-mobile) ;-).

What led to the idea

Day in and day out i’m commuting back and forth from where i live to Zürich. In order not to see those hours spent commuting as wasted time, you’ll have to spend it wisely which often times means doing some kind of work. As a programmer that’s cake ;-). However there are times you just want to talk to someone remotely familiar. You could obviously do some kind of creepy stalking and wait outside the train until the very last minute before departure to see if you see someone familiar. Chances are you’ll either find no one or they actually want/have to do work instead of chit-chat with you. So status quo sucks, right? Well could we improve it?

The idea

Obviously yes. Let’s create a mobile app that allows to checkin to a train and then find conversation-partner-matches via Twitter, Facebook and maybe even Foursquare.

The User Interface

When starting the app for the first time you’re using one of those buttons on the login screen to connect to your favourite network and start using the TrainsharingApp.

As a next window you’re seeing the home screen on which you can enter your commuting route. This will make a call to sbb.ch to get the timetable and save it locally for later reuse. If you’ve already used this route there’s a quick dial button to select your route and then select the departure time on the following view.

As soon as you’ve chosen a time, you’re automatically checked in to that train-route which will trigger a push notification to users on the same train or users which will share part of your route and with whom you are friends on one of the social networks offered at login. After sending those notifications to friends or receiving one you’re able to initiate a Meetup by selecting their name, selecting optional information and tipping the “Meetup” button.

The Technical Implementation

As for the backend, there exists an already scraped static dataset of train lines with their according numbers, eg. S8 18898 and each route (Station to Station, see below). All in all about 230’000 routes which will reside inside a MySQL DB. In case you want to play around with the dataset, here’s a routes table dump.

For matching friends there’ll be some kind of NoSQL DB instance since MySQL isn’t that good at it without creating a lot of table and row overhead. Still needs to be figured out, suggestions are welcome.

Additionally there’ll be a join table (routes_users, see below) for matching the routes with users. Since the routes_users table can fill up quite fast and past data doesn’t need to be held warm, it’ll be pushed to S3 for later analysis and then cleared every 24 hours.

So much for the storage part, now over to the API. There will be three endpoints at least, /login, /checkin and /read.

/login will be used to send the social network authentication tokens over to the server to allow us to do the bandwith heavy parts. The request should be one of type POST with the credentials in it’s body. Which will give a trainsharingID back as a response. This trainsharingID will then be stored on the client device and used for every single request as a querystring.

/checkin is, as the name suggests, the endpoint for when a user wants to check in to a train ride. For clarification, a train ride consists of multiple routes (Station-to-Station) and may also involve switching trains. Requests to /checkin should also be of type POST with the trainsharingID in the URL as follows /checkin?trainsharingID=your_trainsharing_id. As for the POST body, it’s the information getting delivered when clicking on the details section for the specific connection timetable on sbb.ch.

/read will be the only endpoint accessible via a GET request, though it still needs the trainsharingID in the querystring. /read is for manually asking if new overlaps have been found since checking in. The goal however should be that this endpoint is only used for development and new overlaps are sent to the user via PUSH notifications in a production setup.

Enough with the server stuff, what happens on the mobile? -Well, upon first launch of the app the user signs in with one of those social networks which will create an OAuth token and an OAuth token secret which will be sent to the /login endpoint mentioned above. The trainsharingID in the response then gets stored persistently in the application storage.

Next up is the home screen where the user can enter departure and arrival destination to look for available trains. This will trigger a POST request to sbb.ch from the phone, parsing the response and showing the available options to the user. If a user selects a specific time another GET request will be fired to fetch the details information for that connection whose response get’s parsed and sent to the /checkin endpoint.

If matches already exists the response to the /checkin POST request will contain users from various networks who are matching the friends criteria. In case none exist yet the response will be empty. In both cases the user will be notified via PUSH if additional matches are made later on.

Both a proper UI and API endpoints for the meetup functionality still need to be figured out. Maybe it even makes more sense to use their native networks eg. Facebook or Twitter for messaging instead of building yet another Whatsapp clone.

Well, that’s all there is to say about the TrainsharingApp at the moment. Have i missed something? Or do you have a specific suggestion what needs to be changed, added or removed?

In case you are interested in creating a native mobile client besides the one for WindowsPhone, which is already being covered by Adrian, let me know either via twitter or the comments below so I can get in touch with you. The same goes for designers too.

Below are the recordings of last weeks joint event by the IT geeks Zurich and GTUG Zurich. At this event Luuk van Dijk a staff software engineer at Google who’s currently working on the Go compiler and Johan Euphrosine a Developer Programs Engineer working on App Engine gave some insights on how Go can be leveraged and what it does internally to do so.

Also many thanks to the organizer Muharem Hrnjadovic for making this second amazing IT geeks event possible.

And without further ado, enjoy the talks below and excuse the Pizza distribution dilemma we luckily had to deal with.

While there are lively ecosystems of blogs around niche topics like visualizations, processing or data journalism there’s not enough interdisciplinary communication going on in my opinion.

Well, how about creating it by interviewing those experts, give them some airtime and maybe even connect previously unknown scientists, journalists, hackers, politicians and ideators to each other.

Since i’m still playing with the idea there’s nothing fixed just yet. Language might be english or german depending on who’s in front of the camera. Also if you’re interested in being a host or you’re having a name suggestion let me know.

Would you listen to or watch it?

John Perry:

One needs to be able to recognize and commit oneself to tasks with inflated importance and unreal deadlines, while making oneself feel that they are important and urgent.

Being a heavy procrastinator myself I agree with John Perry but I’d like to extend one point. While some work can be encouraging even slightly more can have devastating effects. From my personal experience a task overflow can turn everything into white noise at which stage you’re not caring about any of it anymore.

Positive side effect, you’re able to work off those items quite relaxed. On the other side you might also over-commit to incoming work because estimating white noise is kind of difficult.

On another note, have a look at the awesome copy in the article footer. Brutally honest.

Yesterday, on january 19th, Swiss OpenData enthusiasts and activists have founded the OpenData.ch Association in Bern. It’s goal is to bring together citizens, journalists, designers and developers to realize ideas based on publicly available OpenData and OpenGovernmentData.

More about the goals and mindset of OpenData.ch can be found in the German-Only - Open Government Data for Switzerland Manifesto.

If you’re up to create something with OD, whether your a developer or not, reserve march 30th and 31st when the next Make.opendata.ch-Hackathon will be held. Need, some inspiration of what the first one was like? Checkout the great summary by datavisualization.ch or read my hackathon review.

Thrilled to start this new chapter with an amazingly diversified group of people.

James Hague:

Be widely read. There are endless books about architecture, books by naturalists, both classic and popular modern novels, and most of them have absolutely nothing to do with computers or programming or science fiction.

Seems funny now, but had the most difficult time to let go of all those other things when started studying. No more philosophy or politics just algorithm runtimes and graph theory.

via the codeproject newsletter

Richard Minerich:

We work in an environment where hearsay and taste drive change instead of studies and models.

While VCs and influencers encourage us to jump on the emotional UX and viral social-network train to make our ideas succeed, we tend to not consult our logs first. After all, tapping in the dark is not science, but that’s sort of another topic.

Richard Minerich writes about the shift in programming languages away from proven ones to scripting languages. I tend to agree that dynamic languages should mainly be used as glue. Building a house out of porous cardboard could work if you’re an experienced professional, but it’ll probably fail for most of us. That said most of my code to date is dynamically typed because it’s just way to comfortable.

However there’s no test that’ll cover every single error case on the other side there always will be static and correct formulas.

Maciej Ceglowski:

I love free software and could not have built my site without it. But free web services are not like free software.

The reason I’m writing this is that while premium services are making money they’re not necessarily attracting enough users to actually accomplish something while at the same time, free, VC-backed startups are doing exactly that. The middle way is to design a so called freemium service where premium users have to pay for free ones, but they’re obviously not going to tell them that.

Now after having migrated this blog over to a static version I needed a replacement to enable visitors to send me e-mail while at the same time not opening the doors for spammers. PHP scripts can easily fulfill that job but I want something else.

While having used the free version of Wufoo in the past I thought it’s a no brainer to go back and leverage it again. Obviously paying for it this time. Then it hit me while checking the pricing page. The cheapest subscription is 15 dollars per month, while free plans are displaying ads to your visitor. What are they thinking! Paying 15 dollars, which is more than I pay for hosting, while only receiving about two messages during that time period.

That said, please startups and SaaS companies, remove the free model and make premium reasonably priced.

As for Wufoo, i’d guess replacing the free plan with one where you’d pay 2 dollars a month would make them more profit than showing ads on those confirmation pages.

When talking about showing ads to free users checkout the tweet by @romeroabelleira and give it some thought (translated):

Dear Advertisers on Spotify, I don’t even pay for Spotify, I’m therefore worth nothing to you too. Sorry, Juan

And don’t waste your time looking for the contact form, I’d just put the e-mail address into the footer for now.

Dslrnewsshooter video: Nikon D4 - video feature run through from Dan Chung on Vimeo.

The most innovative feature in my perspective however is the Ethernet connection and what comes with it, a camera management console completely built in HTML and JS so one’s finally able to leave all the proprietary, heavily bloated, vendor specific crap software behind and focus on realizing ideas.

Checkout the video WHY by Corey Rich below if you want to see what’s possible with a D4 presumed you know what you’re doing.

WHY - Nikon D4 Release Video from Corey Rich on Vimeo.

A great read if you’re working on the next big thing yourself, because you’ll might need a team to work on it later.

Once you are no longer alone working on your project, the code you have written sets an example. - Philip Hofstetter

Also, I think it’s totally fine to hack together an initial version, hackathon style, you can improve later on however the structure, needs to be reasonably stable.

Right now I’ve already broken most of my New Years resolutions, I admit it. Why? - Well, I thought of me as a machine, meaning that I could easily deliver one hundred percent without an issue. In fact, that’s extremely unlikely to happen and I obviously knew that but I still made that promise with myself, you need to aim high right?. However, the impact it has by breaking resolutions or not reaching your goals makes you feel terrible or worse even makes you feel sorry for yourself. That’s why I recommend you watch this excellent talk by Tony Schwartz and start correcting your self expectations by stop wanting to prove yourself to others or yourself. It’s good to have goals but a weekly to do list has nothing to do with dreams just doable tasks.

via swiss-miss

While I’m not sure yet if I’ll ever do the same, I started wondering on how to realize something like a Link-Blog with Octopress. Turns out it’s pretty easy.

First modify the article.html inside source/_includes and exchange the lower <h1> part which is responsible for the page view title with an if-else clause.

1
2
3
4
5
6
7

<h1 class="entry-title">
  {% if page.ref_url %}
    <a class="reference" href="{{ page.ref_url }}">{{ page.title }}</a>
  {% else %}
    {% if site.titlecase %}{{ page.title | titlecase }}{% else %}{{ page.title }}{% endif %}
  {% endif %}
</h1>

Then continue with adding an if-else clause to the atom.xml file too. Extend the <link> element inside the parent <entry> with the code below. Done.

1
2
3
4
5

{% if post.ref_url %}
  <link href="{{ post.ref_url }}"/>
{% else %}
  <link href="{{ site.url }}{{ post.url }}"/>
{% endif %}

If you want to create a Link-Blog post now, add ref_url to the markdown file header and Octopress takes care of the rest.

1
2
3
4
5
6
7
8

---
layout: post
title: "This Awesome Article"
date: 2012-01-13 21:20
comments: true
ref_url: http://somesite.com/thisawesomearticle.html
---
This one is really great, check it out.

Alternatives

If Jekyll is too nerdy for you, then please checkout tumblr whose philosophy has been based on link-blogging for ages.

On the otherside if Octopress and therefore Ruby is still to cool for you to use then give the newly released Second Crack by Marco Arment a try. It’s also baking your markdown files to flat html ones, but it’s written in PHP.

(Sidenote: PHP and I were never really friends)

Getting the “old data”

Log into your BlogEngine site and head over to Settings where you click on the Export button for BlogML. Now FTP or SSH into your server and get a complete dump of the App_Data/files folder onto your local machine.

Setup Octopress

Octopress? - Isn’t this tutorial for Jekyll? - you might ask. Well Octopress is built on top of Jekyll and brings some plugins and clever defaults to make your life easier. Start off by following the official Octopress Setup tutorial. If your working on an OS X machine you might want to consider installing POW since it’s making it much easier to work with Jekyll locally. Once POW is installed open your Terminal and go to ~/.pow and create a symlink to the octopress-repo root.

1

$ ln -s /Users/yourUsername/Documents/octopress myapp

Now open http://myapp.dev in your browser to check if it’s working so far.

Import the “old data”

Create a folder _import inside the octopress/source directory and put the blogml.rb conversion file in there.

1
2
3
4

$ cd octopress/source
$ mkdir _import
$ cd _import
$ wget https://github.com/philippkueng/philippkueng.github.com/blob/source/source/_import/blogml.rb --no-check-certificate

This import script was actually created by @derekmorrison to assist him by Moving to Jekyll. Since URLs have to be rewritten because of the .aspx extensions which are part of BlogEngine I slightly modified the script to additionally create an .htaccess file and to play nice with non-ASCII encoded post-titles.

Next, move the BlogML.xml file inside the source directory and also create two seperate folders called files and images inside source. Then copy the contents of the previously dumped App_Data/files folder to their respective folders (images or files) by keeping the path structure intact. Means, that App_Data/files/2010/2/file.zip will go to octopress/source/files/2010/2/file.zip.

Now to the fun part, open your Terminal and navigate inside your octopress/source directory. Then execute:

1

$ ruby -r './_import/blogml.rb' -e 'Jekyll::BlogML.process("BlogML.xml")'

This should have imported all your existing posts so that you can now generate your Jekyll blog from it by exiting the source directory and issuing the rake command.

1
2

$ cd ..
$ bundle exec rake generate

Deploying

There’s already documentation available on how to deploy Octopress to various hosting platforms. Since you’re migrating from BlogEngine I highly recommend running your own VM or getting a shared hosting account with .htaccess support in order to not let your visitors down with feeding them broken links. Actually since Heroku is offering their polyglot Cedar-stack you should be able to run .htaccess on there too. YMMV

If your hosting provider is only offering FTP, then deploy manually by uploading the contents of the public folder and adding the .htaccess file from inside the source folder.

Fixes

We’re not quite done yet. Log into your Google and Bing Webmaster account and replace the sitemap.axd entry with sitemap.xml. The same goes for your Feedburner account, log in and replace feeds.axd with atom.xml.

Finally fix the disqus comments. Log into their dashboard and select the account you want to migrate. Then click on the Tools tab and afterwards the Migrate Threads navigation entry. Hit the Start Crawling button for the Redirect Crawler Migration and your done.

After reading his idea I immediately responded via a comment that he should make it happen, after all aren’t those the successful startups that fullfill an itch that a founder has himself?

Throughout the rest of the day my brain was occupied by coming up with ways to turn this idea into reality, thinking about partners in the printing business who would send out the printed goods and maybe also pre-fill them. What might their pricing requirements look like? - after all their struggling themselves in the world of the internet. After the physical side was mentally figured out the question was how to build the website? - which technology stack to use? - hacking it together over a weekend? - with whom?

Later that day I told my father, a non computer savvy person, about that great idea I saw on the internet today. He was looking at me as if i was out of my mind. He totally disagreed and instead suggested that women, in the week prior to giving birth when being bored and at home, could potentially pre-write the letters so that the husband could then just print out the pictures stick them into the cards and then send them to friends and family.

After having the discussion with my father i think he’s totally right in terms of not outsourcing the bonding part with people you care about. It’s not about time you’d gain by using such a service it’s about openly investing yourself into the relationships to others by sending a handwritten card rather than wrapping your personal writing in an emotionenless off the shelf card with corporate branding on the back.

On a sidenote to all developers, I know it’s all too easy to get stuck in the optimization trap without knowing it. We tackle the market analysis with the identical mindset as writing revision one of the service. The approach might work in an emotionless business area but is certainly not suited at all for private relationships. As a hint, talking to people with different backgrounds helps enormously.

In an effort to make the world a better place, hackers all around Switzerland joined this years make.opendata.ch hackathon in Zürich and Lausanne. The goal was to use publicly available data, so-called OpenData and visualize it in a meaningful way so everyone is able to make sense of it.

Despite of having a french language deficit - it was a dark chapter in grammar school - i travelled all the way to EPF Lausanne which was absolutely worth it.

After the kickoff Florin, Frederic and I immediately started working on a project to improve the routes suggestion of traditional mapping applications such as Google Maps and OpenStreetMap by providing a way to add obstacles to a map and then calculate a new obstacle-free route to help people on tight schedules arrive on time. While we knew that large companies are working on that we were still looking for a hack to finish something usable until the deadline the day after.

The theoretical solution - The frontend would be a static website where the user can enter departure and destination location. This then would make an API call to the backend where the hacks are doing its’ part. The backend would consist of a webkit process running, controlled by PhantomJS. If a request gets issued PhantomJS loads Google Maps Directions with the start and end location, let the server-side webkit process render the page and then add all the obstacles via Lat/Long with a predefined icon which is easily distinguishable from the map itself. Next, PhantomJS creates a PNG-screenshot of the obstacles and route containing website. It starts the next process which parses the PNG pixel-by-pixel and looks if there is a blue line and an obstacle crossing. If so, make a circle of 300m radius around the obstacle and randomly add a Lat/Long coordinate, being part of the circle border, as a waypoint to the routes call of Google Maps. Then repeat the process above to check if the new route interferes with obstacles. If finally and obstacle-free route is found, return the details to the frontend.

During dinner, about three hours in, we came to the conclusion that it is just too much of a hack to spend valuable hacking time on, so we had to find something more suitable for the next day. Which we did in SwissMap.

Full of vim and vigor we then started on Saturday morning, Florin by collecting the interesting data from all over the bfs website, Frederic by merging and crunching the data into a format which is then processable via Javascript and I took care of the view and the comparing alogrithm. We aimed at building a web application where citizens and data-journalists can compare two different, unrelated datasets and visualize the outcome on a swiss heatmap. Luckily our very intense and focused hacking on day two was enough so that we were able to meet the presentation deadline in the afternoon. Epic Win.

[ live, wiki, source ]

I hereby thank the sponsors, hackers and especially the team behind the MAKE.OpenData project Andreas, Hannes, Oleg, Antoine, François, Frederic and Jeremy very much for making the event possible and inspiring us.

Amazing MAKE.OpenData projects others have built

• green-street [ live, wiki, source ]
• gesagt-im-parlament.ch [ live, wiki, source ]
• politnetz visualization [ live, wiki ]
• Swiss Army Contaminated Sites [ live, wiki ]
• Where Did My Taxes Go [ live, wiki, source ]
• openletten [ live, wiki, source ]

Travelling is supposed to be relaxing right? However there are many things you need to take care of for example writing postcards, sending SMSes to loved ones and keep notes of all the sites, actually this time RL ones, visited.

While I did exactely that for my last vacation in Canada, we were handling it differently this time.

During the flight from Frankfurt to San Francisco we came up with the photoblog called sfo2yvr, which includes the airport code of SF and Vancouver in its’ name, creative we know. It is based on Jekyll rather than Tumblr, which we actually wanted to use in the first place. However, we needed a platform to publish articles offline and did not want to rely on third party apps like MarsEdit, with git and github pages that is cake.

The idea was to update it once a day, however because US National Parks mostly do not yet offer Wifi, nor electricity or running water (rivers do not count), we sometimes only committed to the local repository and pushed to github when we were in a peopled area later.

This technique for blogging worked pretty well for us so far, however I am really curious what others, maybe you, have done in terms of solving the offline problem or finding the suitable picture for the day.

Changing habits is difficult. On December 31 you think that in 201x you will stop smoking, start eating healthier food or in my case plain and simple gain weight. (yes, I want to be a sumo wrestler) The only problem is that you are currently within your comfort zone and you are not taking immediate action. You are procastinating. I guess a lot of people, me included, never get out of it and already fail before having started. A tip, which might sound weird, but has worked for me so far, stop thinking about it and put yourself in a position where “The only way out is through.” — Winston Churchill.

Then secondly as with every habit you want to adapt, it needs to be included in your daily routine. That means if you can keep a daily routine up for about three weeks you are most certainly going to succeed with making it a habit.

About four weeks ago my friend Steve, with whom I co-founded bitfondue, sent out a tweet which started a discussion between the two of us on how to simplify life by leveraging agile practices with the outcome that we agreed on producing a podcast about that. Still having the two points above in the back of our heads we skipped being procrastinating wusses and started being agile right away. We successfully recorded the first iteration, set up a blog using Jekyll (we are nerds what do you expect? Wordpress.) and registered the domain within the first week after initiating the adventure.

Enough history, now to the meat, if you want to have 30 minutes of LOL I highly recommend listening to Iteration 1 otherwise give Iteration 2 or 3 (the one below) a try. For more information, future iterations or comments head on over to agilelivin.

It works as folling, prior to adding a task to your list go mentally through it and notice every single subtask you need to achieve in order to check that item of your list. That way I accomplished 4 tasks which consumed about 2h of my time without really noticing it and that said those tasks were sitting on my list for 3 weeks already.

Next there is the second and last rule, do not dare to put tasks on your list you know upfront you are not going to do anyway. For me such a thing is sorting my photo library, I used to have an entry for sorting pictures taken while being in Canada (2008) on the list until very recently. By the way decluttering it regularly helps a lot too, believe me.