About two months ago, one my favorite apps (BeyondPod) pushed out a significant update with a complete redesign. I considered this a welcome change given the previous version had several UX bugs and was just plain ugly. The new design took advantage of the some newer API calls offered by Google including many that were adopted by Youtube and Facebook which meant a more uniform interaction when using my phone. Earlier this week, the application required an update and being curious, I took a look at the app’s page to see what was changing. Nothing major, but before I swiped away, I noticed that a lot of the recent reviews were negative. In fact, almost all of the reviews I came across took major gripes with the new redesign.
So, what’s the point? Why the post? Well, the BeyondPod situation highlighted an issue with the Google Play application rating system (and most rating systems for that matter) — they lack any notion of time. Apps are not static and are always subject to change, so their ratings should account for that. In the case of BeyondPod, the major update had seemed to cause many existing users to re-rate the app much lower than they may have previously, but unless you read those reviews, it was not clear that the app was no longer well-received.
The first question I wanted to answer was if the update to BeyondPod had actually caused a drop in ratings. In order to answer this question, I needed the review data. Firing up the developer tools in Google, I could see that reviews were loaded up into multiple DOM elements and after enough paging, a request was made to the server to get more.
It’s worth noting one of Google’s techniques when loading ratings just because it’s different than what I have seen before. Instead of loading one or two “pages” of reviews, Google loads six identical containers complete with the proper columns and corresponding reviews. Hitting the next button simply toggles the opacity of the current container to 0 and the next container to 1. This method is pretty interesting as it requires very little changes or references to the DOM.
The secondary plan was to just pound away at the AJAX interface Google used within the site itself. This should have been my first route, but I wanted to avoid Google’s spam detection. Normally I would just whip up a script in vanilla python to handle the task, but figured this was best served for the Google App Engine. Within 15 minutes, I had a working application with the following flow:
- Issues N POST requests to the AJAX interface with sort order set to “newest first”.
- Extract reviews from the results, saving the time and rating
- Sort based on the time and then carve out a 30-day window
- Average all reviews across the window
- Return results
With the results in hand, I was able to answer my original question of if the review averages had fallen since the update.
As shown in the chart above, the past 2 months have shown a drop in average ratings for the application. Without analyzing the content of the reviews, one can’t be certain if the update has anything to do with the drop, but given the update was released 2 months ago, I think the chances are high. It’s worth noting that Google does not appear to present the “star” rating if there’s not a comment attached to the review. This makes the corpus a bit bias, but it’s better than nothing at all.
After getting the results I wanted, I felt it would be best to serve the data inside of the actual Play store website. To do that, I made a Chrome extension that called the Google App Engine application (GAE offers HTTPS for all services which means no conflicts with content security) and then adjusts the header color and rating background with the corresponding star color. This ends up looking like the following:
The only issue with the final solution is that it’s unlikely that I will use my computer to browse the app store before actually downloading an application. Either way, this was a fun exercise and who knows, maybe one day Chrome extensions will run on Android.