Browsing Category

Tools

Announcements, Development, Tools

Tupalo.com on your Android phone

Android, Google’s highly popular and growing Operating system, is a success. Aside from the iPhone it’s market share, especially in the US, is strongly growing. Reasons enough to make Tupalo.com fully available for your Android phone.

As of today we’ve deployed and released our first Version of the Tupalo Android App. It’s a nifty, free app, that gives you all the tools to discover your neighbourhood.

Browsing your favorite spots, adding them as favorites and even checking in at your favorite venue is supported.

The beautiful code comes straight out of the hands from our mobile developer Ryan, who just moved from San Francisco to Vienna and now built one of the best local apps out there!

Download the App by photographing the QR Code below or search for it on your Android phone!

Code, Development, Tools

Generating XML diffs with awk and bash

Did you ever wonder where Tupalo.com gets all these spots from? Apart from the ones our awesome users add, we get them from our various partners, often in the form of XML. This generally works fine, but XML’s structured nature also means that you can’t just treat it like any old text file.

That’s something we recently had to work around when we wanted to generate a daily XML diff that only contains elements which changed since the previous day’s feed. Of course there are several open source tools for exactly this purpose (e.g. diffxml or xmldiff) but since we didn’t get them to do what we want in a reasonable amount of time, we just decided to roll our own.

The final solution is a 71 line bash script, which downloads a zip, extracts it, generates MD5 sums for every element and then creates a diff between this new file and the previous list of MD5 sums. Once we know which elements have changed we merge them into a new feed which then gets handed to our importer. The awesome xmlstarlet was a great help in this, as was battle-tested old awk.

Let’s look at an interesting snippet from the script:

 xmlstarlet sel -I -t -m "//item" -v "./guid" -o "|" -c "." -n - | 
  sed -e '...' |
  awk \
    'BEGIN { 
      FS="|" 
      RS="\n"
    }
    {
      id=$1
      command="printf \"%s\" \"" $2 "\" | md5sum | cut -d\" \" -f1"
      command | getline md5
      close(command)
      print id":"md5
    }' >> $MD5_DIR/vendor-md5-$TODAY

Here we use xmlstarlet to iterate over all the items in the feed (the XPath “//item”), print the value of the “guid” element (-v “./guid”), output a pipe character (-o “|”) and then copy the current element followed by a newline (-c “.” -n) . This then gets piped through sed for some cleaning up (which I omitted here for brevity’s sake) before awk takes the part after each “|”, generates an MD5 sum and finally produces a file that looks like this:

rKKTZ:4012fced7c4cd77da607d294fbb8b5b6
hC7Jr:39245a0f9a976e6d47c0e2d76abf6238
...

Now that we are able to create a daily list of MD5 sums, it’s easy to generate the diff feed:

if [ -e $MD5_DIR/vendor-md5-last ] ; then
  changed=`diff $MD5_DIR/vendor-md5-last $MD5_DIR/vendor-md5-$TODAY | 
	   grep "^>" | 
           cut -d":" -f 1 | 
           cut -b 1-2 --complement`
 
for record in $changed ; do
    f=`fgrep -l "<guid>$record</guid>" $FILE_PATTERN`
    xmlstarlet sel -I -t -c "/rss/channel/item[guid='$record']" $f >> vendor-import-$TODAY.xml
  done

Here we create an array with the id of the changed elements over which we then iterate. In the loop we once again use xmlstarlet to extract the current item from the feed which contains the right guid.

This is a good example of how familiar old Unix tools can be combined to create a fairly concise solution for a non-trivial problem.

Development, Tools

Google Analytics – visits for all visitors – comparison Ruby script

Hi. I stole this GA Visits for all visitors ruby script from our CTO. Hope you find it useful. It requires installed Ruby.

It displays

  • visits for all visitors
  • hourly view
  • yesterday – compared to – day before
  • current day – compared to – day before
  • side-by-side

Visits for all visitors - Google Analytics ruby script

Note: screenshot does not display Tupalo.com data, but data from another experimental domain.

here is the code, all you need to do is to paste your profile-id into it. (the stuff after &id= highlighted in yellow).

#!/usr/bin/ruby -wKU

class Fixnum
def day(v=self); v * 24 * 3600; end
def week(v=self); v * 7.day; end
end

def google_analytics_visitors_compared_to_last_week_url(time = Time.now)
date_now = time.strftime("%Y%m%d")
date_last_week = (time - 1.week).strftime("%Y%m%d")
"https://www.google.com/analytics/reporting/visits?id=your id here&seg0=-1&pdr=#{date_now}-#{date_now}&cdr=#{date_last_week}-#{date_last_week}&cmp=date_range&gdfmt=hour&rpt=VisitsReport&gdfmt=hour"
end

def open_in_browser(url)
system "open '#{url}'"
end

open_in_browser google_analytics_visitors_compared_to_last_week_url(Time.now)
open_in_browser google_analytics_visitors_compared_to_last_week_url(Time.now - 1.day)

facebook-logo-520
Announcements, Development, Tools, Tupalo

It’s all about “liking” your favorite local spot

Showing love for the restaurant of your choice, or for getting the best teeth ever made from your close by dentist is important. The web transforms how such local businesses attract new customers, and how they are being honored for doing a good job.

Besides from writing a review for such places, there is other ways to give them Kudos. Sharing it with your friends is probably one of the more efficient ones. And when it comes to “friends” there is no way around social networking giant Facebook. The controversial company that made worldwide friendship a phenomenon and made everybody friens with Kevin Bacon – by 6 degrees.

Tupalo.com has now also a nifty, simple and elegant way of letting your Facebook friends know, how much you like a certain place.

Just click on the infamous “like” button and share it with your Facebook friends instantly. If you are reading this blog, then the best place to start your “tour de like” is problaby here: LIKE IT

Now, you might ask, what about my not so positive experiences? The hair in the soup. The 45minute waiting time at your local hairdresser?

Well, since Facebook does not offer a “dislike” button (yet), you’ll just have to stick with your positive experiences until now :)

SEO, Tools

Greasemonkey script: General CTR in Google Webmaster Tools Search Queries

Ok, hi, it’s Franz Enzenhofer, SEO Manager at Tupalo.com.

As any other SEO I love Google Webmaster Tools, especially the “Search Query” function – a tool that not only shows you what queries your site gets traffic from (SERP clicks), but also what keywords your site shows up on Google in general (without the people necessarily clicking on the result = impressions). And even though it is a great tool, Google – due to some unknown reason – did not include a general CTR (clicks / impressions) value.

So some rainy afternoon (there were more than enough this s*%$y summer (in Austria at least)) I coded a little greasemonkey script.

(Note: that is not Tupalo.com data – it’s from another experimental domain)

The good news is: you now get the “general CTR” of all the search queries displayed below the overall impressions / clicks data.

The bad news is: The query / impressions / clicks data itself is pretty bad, the more you drill down on timespans, compare different markets or site segments you realize that Google is only showing a very biased sample of  query / impressions / clicks data, so the click through rate jumps up and down pretty randomly. And don’t even try to compare average clicks per query / impressions over different markets, thats the point when it gets really frustrating.

But find out for yourself:

1) start up firefox (yeah, you need firefox).

2) go to the greasemonkey install page (you need the newest 0.8+ version of greasemonkey, only available for firefox, the script does not work with google chrome greasemonkey version

3) install the script gwt-general-ctr.user.js

4) go to your google webmaster tools ->  your site on the web -> search queries page. [note: script works on the english version of webmaster tools, only]

5) look at the lovely CTR figures

p.s.: yeah, yeah I know I could have designed it better, but I always like to know what data comes from google, and what is the greasemonkey script, so I kept it ugly on purpose, and anyway I’m an SEO, not a designer.

Update: Hey, as Google introduced “Google Instant Search” this tool will now come on even more handy.