We hate spam as well, and we’re very sorry that this happened.

In a nut shell, our email delivery system got itself into a state that it shouldn’t have been able to get into, and was reporting that emails weren’t delivered when in fact they had been. Our email generator did what it was supposed to do and attempted to resend these ‘failed’ emails, causing a loop of retries. We discovered and fixed the problem quickly, but know that for those few affected users it would have been a huge inconvenience.

We care about our users and we’d like to say sorry. If you were one of the ~100 people who received multiple emails, please write to us at feedback@chomp.com and include your Chomp username by March 31 and we’ll send you a $10 iTunes gift card. We hope that you’ll keep using Chomp to find great apps to spend it on.

If you take your unhandled exceptions seriously (we do), analyzing them regularly is a top priority. To take the tedium out of manually paging and downloading all the individual CSV exports, you can use the Internet’s Swiss Army Knife™, curl.

If you’re on Mac OS X, curl is already installed. On Windows you will need to download and install.

First, use curl to log in to Flurry, obtaining the session cookie for future requests:

<youremail> and <yourpassword> needs to be URL encoded:

$ /usr/bin/php -r 'echo urlencode("rob@domain.com")."\n";'
  rob%40domain.com

$ curl --cookie-jar ./flurry.jar \
  -d "loginEmail=<youremail>&loginPassword=<yourpassword>&rememberMe=true&__checkbox_rememberMe=true" \
  -k https://dev.flurry.com/secure/loginAction.do

• –cookie-jar stores the response cookie for later use
• -d sends the POST data
• -k turns off certificate verification

Grab the Project ID for your App by looking at the CSV download link from the Flurry Error listing page.

Now ask curl to repeatedly download each page of your exception log CSV export:

$ curl --cookie ./flurry.jar \
'http://dev.flurry.com/exceptionLogsCsv.do?projectID=<yourproject>&versionCut=versionsAll&intervalCut=allTime&direction=1&offset=[000-100:15]&pageSize=15' \
--output "exception#1.csv"

You’re ready to read those into iWork Numbers or Microsoft Excel one at a time, or you can go another step further and combine them all for a single exception log:

$ grep -h -v -e "^Timestamp" exception???.csv | sort > AllExceptions.csv

Now get on with tracking down your reported errors and uncaught exceptions.

To encourage users to submit a rating it’s essential to reduce the overall complexity of the process. If they have to jump through hoops before they can tell the world about this app they’re using, they may not bother. Our aim is to keep that process quick and easy, helping to make sure a user will have their say.

One of the great features of Chomp Connect is the ability to find and use an existing user session. If a user has taken the time to signup for or sign into an account, we don’t want them to go through that again unless they really have to! That session may have been established using the Chomp app directly, or by Chomp Connect in another app.

To achieve this we need to share data between apps. These apps have no ability to know each other is installed, or even if the Chomp app is installed. If Chomp was installed, you could use a registered URL scheme to launch Chomp, have the user perform the rating and possibly return them to your app (if your App also registered a URL scheme). Although more and more people are downloading Chomp every day, it would still limit the potential for users to review the app on Chomp.

I searched for other methods of sharing small amounts of data between apps, and discovered that iPhone OS 3.0 and above provides great support for pasteboards. There are two system pasteboards, System and Finder, but also an application pasteboard that can be created with a unique name. These application pasteboards can be configured as persistent allowing them to be shared between applications that know that unique name. We configure Chomp Connect to use a specific pasteboard name, enabling any app that embeds Chomp Connect to seamlessly share any previous session on the device.

Sharing information between Apps with UIPasteboard

Implementation is very straightforward, using the UIPasteboard convenience methods that set and get a simple string pasteboard item.

/* ChompDialog.m */

+ (UIPasteboard *)getPasteboard {
    // Chomp Connect pasteboard will be created if none exists
    UIPasteboard *ccPasteboard = [UIPasteboard pasteboardWithName:@"com.chomp.connect.pboard" create:YES];
    if (ccPasteboard.changeCount < 1) [ccPasteboard setPersistent:YES];
    return ccPasteboard;
}

+ (NSString *)getChompUserToken {
    return [[ChompDialog getPasteboard] string];
}

+ (void)setChompUserToken:(NSString *)newToken {
    [[ChompDialog getPasteboard] setString:newToken];
}

The end result is reduced complexity for a user to perform a review, therefore a higher probability that they will.

We make it easy for your users to show their love!

In the meantime though it was just me pretty much by my lonesome, really wanting to get MySQL to be able to at least do a good enough job to buy me a few weeks worth of time. There’s an article in the MySQL docs about fine-tuning MySQL for full text searches, which was a great starting point. Turns out there were a few different issues contributing to the plain old vanilla out-of-the-box MySQL full text searches not performing the way we wanted them to.

The first major issue was the way in which MySQL tokenizes the field being used for the index. The really simple settings were things like tuning the stopword list (which actually we’ve removed completely now, we’re taking care of the problem by sanitizing the row we have the index on before we insert into it) and turning the minimum word length down to two characters (so that apps like “Oz” get indexed at all, and so that you can find “Oz Weather” without it getting buried in all the weather apps out there). The more complex bit however was getting punctuation treated as word matter instead of whitespace. There are a bunch of apps where the punctuation is significant, such as “Ping!” or “m:Vampire”. The easiest way to do so (without recompiling MySQL) is to define a new collation and modify the character set in the process. This post, linked from the bottom of the MySQL fine-tuning post, is a great practical example. Docs are hard to find, and the configuration is a bit raw, but once you understand what’s going on it’s pretty trivial to tell the system to use characters like :, $, and % as word characters.  One useful debugging tip there is to dump the index info using the myisam_ftdump utility to get a list of tokens from the index. Really helped when I thought I had the right set of characters in my modified character set but had made a mistake.

The second major issue was sorting by relevance coming out of the match. Because folks tend to use subwords pretty frequently (must be because of the small keyboard, we see lots of stuff like ‘facebo’, which I assume maps to Facebook) we use boolean mode match()es, but we still want to sort by relevance. The common suggestion there is to do a select from the table using a boolean match() and then select the match() of the text against the query to use as a column to sort by. The suggestion is in the second comment down on the MySQL full text search functions page. Not bad, but also far from perfect. I haven’t dug into the details of how that function operates, but it certainly wasn’t doing what we wanted it to. For instance if someone used to search for “facebook” they would get back a whole bunch of other apps ahead of the facebook app itself. Stuff like “Facebook It!” for example, which I certainly consider to match the term “Facebook” less well than the exact text. What we were looking for was a measure of how far the application name was from the input term and sort by that. What we wanted was the Levenshtein Distance, but unfortunately MySQL doesn’t have that. We could suck all the values out and then compute the distances on the web front end – too nasty for large sets though. Fortunately there’s a Levenshtein Distance User Defined Function for MySQL that did the trick quite nicely. Now the whole thing is fully offloaded to the database and we can hold off making additions to the infrastructure till we have some time to really think through which option we want to go with.

Add in a little bit of text munging before we push the text into the index and we have a version that works with prefix strings for both punctuation bearing and punctuation free version. So now if you search for ‘m:vampire’ or ‘mvampire’ or even ‘mvamp’ on chomp you get back the m:Vampire app as the first hit. The nice thing about having that relevance hack defined as a UDF is that we can retweek it as necessary to generally provide a different kind of relevance. For us it was LD, and that gets us far enough. But if you want to use boolean matches on a full text search and your relevance function is just about anything else that can be evaluated as a pairwise comparison between the query and each individual row this is a decent way to get MySQL sorting things the way you see fit. No recompiling MySQL core required.