How to download your WordPress.com stats in CSV, JSON, or XML format

I wanted raw data about the popularity of my various posts on this blog to better determine what sort of topics I should post about. WordPress.com provides some nice aggregate stats, but I wanted more. After stumbling around the Internet for awhile, I cobbled together a way to download my blog data in either CSV, XML, or JSON format.

There are three steps:

1. Get an API key
2. Get your blog URL
3. Construct the URL to download the data

Get an API key

Akismet is WordPress.com’s anti-spam solution. Register for an Akismet API key at http://akismet.com/wordpress/ by clicking on “Get an Akismet API key”.

Sign up for an account. If you choose the personal blog option, you can drag the slider all the way to the left and register for free. If you value the service that Akismet provides, you can pay more. When you complete the signup flow, you will be provided with a 12 digit ID. Copy this down.

Get your blog URL

Copy the full URL of your blog, minus the leading https://. For me this is developmentality.wordpress.com.

Construct the URL

There is a limited API for downloading your data at the following URL:

http://stats.wordpress.com/csv.php

View this in a browser to see what the API parameters are.

Construct the url

http://stats.wordpress.com/csv.php?api_key=<api_key>&blog_uri=<blog_uri>

View this URL in the browser (or via wget / curl) and you should see the view data.

There are multiple data sources. From the documentation:

table String One of views, postviews, referrers, referrers_grouped, searchterms, clicks, videoplays

Here is some sample data from each table. Change the format param from csv to json or xml to get the data in different formats.

views

CSV

"date","views"
"2013-12-31",118

JSON

[{"date":"2010-02-05","views": 46}]

XML

<views>
    <day date="2014-01-01">112</day>
</views>

postviews

CSV

"date","post_id","post_title","post_permalink","views"
"2014-01-28",369876479,"Three ways of creating dictionaries in Python","https://developmentality.wordpress.com/2012/03/30/three-ways-of-creating-dictionaries-in-python/",46

JSON

[{"date":"2014-01-29","postviews":[{"post_id":369876479,"post_title":"Three ways of creating dictionaries in Python","permalink":"http:\/\/developmentality.wordpress.com\/2012\/03\/30\/three-ways-of-creating-dictionaries-in-python\/","views":22},{"post_id":369875635,"post_title":"R - Sorting a data frame by the contents of a column","permalink":"http:\/\/developmentality.wordpress.com\/2010\/02\/12\/r-sorting-a-data-frame-by-the-contents-of-a-column\/","views":16}]}]

XML

<postviews>
    <day date="2014-01-30"></day>
    <day date="2014-01-29">
        <post id="369876479" title="Three ways of creating dictionaries in Python" url="https://developmentality.wordpress.com/2012/03/30/three-ways-of-creating-dictionaries-in-python/">54</post>
    </day>
</postviews>

referrers

CSV

"date","referrer","views"
"2014-01-28","http://www.google.com/",63

JSON

[{"date":"2014-01-30","referrers":[]},{"date":"2014-01-29","referrers":[{"referrer":"http:\/\/www.google.com\/","views":66},{"referrer":"www.google.com\/search","views":27},{"referrer":"www.google.co.uk","views":10}]}]

XML

<referrers>
    <day date="2014-01-30"></day>
    <day date="2014-01-29">
        <referrer value="http://www.google.com/" count="" limit="100">66</referrer>
    </day>
</referrers>

referrers_grouped

CSV

"date","group","group_name","referrer","views"
"-","Search Engines","Search Engines","http://www.google.com/",1256

JSON

[{"date":"-","referrers_grouped":[{"referrers_grouped":"Search Engines","views":{"http:\/\/www.google.com\/":1305}}]}]

XML

<referrers_grouped>
    <day date="-">
        <group domain="Search Engines" name="Search Engines">
            <referrer value="http://www.google.com/">1305</referrer>
        </group>
    </day>
</referrers_grouped>

Dates aren’t included so it’s the sum over the past N days, defaulting to 30. To change this, set the days URL parameter:

http://stats.wordpress.com/csv.php?api_key=<api_key>&blog_uri=<blog_uri>&table=referrers_grouped&days=<num_days>

searchterms

CSV

"date","searchterm","views"
"2014-01-28","encrypted_search_terms",190

JSON

[{"date":"2014-01-30","searchterms":[]},{"date":"2014-01-29","searchterms":[{"searchterm":"encrypted_search_terms","views":159},{"searchterm":"dynamically load property file in mule","views":2}]}]

XML

<searchterms>
    <day date="2014-01-30"></day>
    <day date="2014-01-29">
        <searchterm value="encrypted_search_terms" count="" limit="100">159</searchterm>
        <searchterm value="dynamically load property file in mule" count="" limit="100">2</searchterm>
    </day>
</searchterms>

clicks

CSV

"date","click","views"
"2014-01-28","http://grab.by/grabs/b608b9c315119ca07a1f7083aabbb9c7.png",3

JSON

[{"date":"2014-01-30","clicks":[]},{"date":"2014-01-29","clicks":[{"click":"http:\/\/www.anddev.org\/extended_checkbox_list__extension_of_checkbox_text_list_tu-t5734.html","views":2},{"click":"http:\/\/android.amberfog.com\/?p=296","views":2}]}]

XML

<clicks>
    <day date="2014-01-30"></day>
    <day date="2014-01-29">
        <click value="http://www.anddev.org/extended_checkbox_list__extension_of_checkbox_text_list_tu-t5734.html" count="" limit="100">2</click>
    </day>
</clicks>

videoplays

I am not sure what this format is as I have no video plays on my blog.

Conclusion

I hope you find this useful. I’ll make another post later showing how to crunch some of this data and extract meaningful information from the raw data.

Why You’ll (Rarely) Catch Me With a Printed Book

kodumut via Flickr under Creative Commons license

I love books. I love book stores. I worked in a library for four years. But when I read Ali Wunderman’s post entitled Why You’ll Never Catch Me With An E-Reader, I was not convinced. In almost all cases I prefer to buy and read digitally. I’ll discuss her argument, my reasons for preferring digital in most cases, and cases where e-readers and digital books are worse than their paper alternatives.

The two main points of Ms. Wunderman’s argument are:

• She likes the sensations of reading a physical book (touch, smell, sight)
• She values the serendipity of meeting new friends who love the book that she’s reading. Had she used an e-reader, those people would not have been able to see what she was reading, and thus she would have missed out on such encounters

I can’t argue against the first point – it’s a matter of opinion whether or not holding a book feels good. Since e-readers are such a recent invention, I think this argument is rooted in nostalgia more than anything else. It would be interesting to see whether children who grow up with a choice between e-readers and physical books end up with such a physical attachment to books. I do like the touch and smell of books, but it’s not enough to make me buy paperbacks exclusively.

The second point is also subjective. I’ve never had strangers comment on what I’m reading, but I can imagine it would be a fun experience. I am willing to bet that it’s rare. On the other hand, I have heard that some women are more comfortable reading romance novels on their e-readers than physical copies. I don’t really care one way or another; I don’t read books with the intention of showing others what I’m reading.

I have had the experience of bonding with new friends over the contents of our respective bookshelves. If there were no books for us to look at, we would have missed out on some level of connection. While this makes more sense to me than the serendipity argument, I still don’t think this is a reason to stay slavishly attached to dead trees. Once you have established a relationship, it’s easy to converse about books you’ve read, no matter the medium.

Digital preference

The reasons I prefer digital books are price, convenience, ergonomics and lack of physical clutter.

Price

E-books are often cheaper than physical alternatives. This makes sense – the price of publishing and distribution is virtually zero.

Convenience

E-books are incredibly convenient. Let me count the ways:

• Instant gratification – purchase, download, and start reading a book in less than a minute. No need to wait for a book to be shipped to you, or to go to a store
• Instant definitions – no need to break the flow of reading to learn the meaning of a word. Tap and hold on the word to get a quick pop-up definition
• Read free samples of a book before committing to buying it
• Free lending library – check out one free book a month to read

There’s a saying that the best camera is the one you have with you. It’s the same with books. I rarely bring physical books with me, sometimes I have my Kindle Paperwhite, but I always have my phone with me, and that phone has all of my Kindle purchases on it. When I used to use an iPhone it was uncomfortable to read on such a small screen, but I recently switched to a Nexus 5 and have happily consumed entire books on it. The progress I make on one device is instantly synced with all of my other Kindle compatible devices.

Ergonomics

I find it more pleasant to read digitally. I can control how big the font is, I can turn pages one-handed, I can read in the dark with no external illumination, and my devices are light to hold. For example, I bought Cryptonomicon on Kindle to replace a hardcover version partly because I was tired of reading such a bulky book.

Clutter-free

Most importantly, buying digitally frees me from physical clutter. If you’ve ever moved, you know how heavy and unwieldy books are. If you’re traveling, they add weight and bulk to your luggage.

“Rarely”

Unlike Ms. Wunderman, I am not absolute in my preference. I often prefer digital, but I acknowledge that there are some real problems with digital books. These include ownership, longevity, batteries, screens, and the distractions of reading digitally.

Ownership

When you buy digital content, what do you actually own? In 2009, Amazon deleted unauthorized versions of Animal Farm and 1984 straight off of owners’ Kindles. As the article says,

Digital books bought for the Kindle are sent to it over a wireless network. Amazon can also use that network to synchronize electronic books between devices — and apparently to make them vanish.

When you deal with DRM (digital rights management) content, there are very strong restrictions placed on what you can and can’t do with the content. It’s more like a limited license to view the content rather than outright ownership.

For instance, let’s look at lending. With physical books, you can lend your book to whomever you want for as long as you want. With Amazon’s titles, not all publishers allow digital lending in the first place. Of those that do, there are Draconian limitations. From the Amazon Kindle help page:

You can lend a Kindle book to another reader for up to 14 days… A book can only be loaned one time.

Until you can freely loan or give away your digital copies of books, paper wins hands down.

Longevity

Even if you buy DRM-free content, you take the chance that you won’t be able to read that content in a few years or decades. There is a strong precedent of technologies dying and data being trapped on obsolete devices; see Lost Formats for examples. If Amazon goes out of business, what happens to all of the Kindle content you’ve amassed? Hopefully if that were to happen, Amazon would offer a service like Google Takeout to transfer the books to you. (Full disclosure: I work for Google.)

Trav1085 via Wikipedia

The other aspect of longevity is the e-readers themselves. I haven’t had the best of luck with my Kindles so far – I am on my fourth Kindle in about as many years. Three of them died quickly, but as of yet I’ve had no problems in the past 2.5 years with the Paperwhite. It makes me wonder how long these devices last. If you have to buy a new $100 device every 3-5 years, this changes the calculus of whether e-books are more affordable.

Batteries

Digital readers run out of batteries; books don’t. Standalone e-readers typically don’t need charging very often, but phones do. If I were traveling and didn’t have ready access to electricity, this would be a concern. In practice this isn’t a big problem for me.

Screens

Some people prefer the look of a book to an e-reader screen. The e-ink display on the Kindle has improved with each generation, both in resolution, sharpness, and refresh speed. I don’t think there’s an objective winner here. The display on my Nexus 5 is incredibly sharp and I can read it in the dark, just as with the Paperwhite. My only objection to the screen is that it contributes to my spending 90% of my time staring at a glowing rectangle. I mitigate this somewhat by changing to the white text on black background on my phone, and keeping the light low on the Paperwhite.

Distractions

It’s easy to get distracted if you are reading digital books on a multi-purpose device. Reading takes time and concentration. Sometimes it’s hard to stick with that when there’s the allure of games and an infinite expanse of Internet content that’s a few button presses away.

Standalone e-readers offer a more focused reading experience that’s closer to that of reading a book. Since there’s fewer things you can do on it, there’s less temptation to do something other than read. Depending on your level of willpower, this point could be completely moot.

Conclusion

I vividly remember seeing the first clunky version of the Kindle just a few years ago and wondering how its owner could enjoy reading on it. The technology has improved so much since then that I’m a happy convert. While I acknowledge the superiority of physical books in some ways, it often makes sense to buy digital. Doing so avoids physical clutter and is extremely convenient. Most of the technical problems with digital books and e-readers have been solved; the remaining hurdles of consumer-unfriendliness are sociological problems that we can combat. For example, Microsoft changed its restrictive DRM in the Xbox One due to overwhelming negative response. If consumers showed as much passion for their rights to the book publishers and Amazon, perhaps we’d see a loosening of the reins as well.

A lesson in overkill – using LaTeX to render pull quotes

A few years ago I conducted an interview with William Wilson, author of my favorite guitar software. I wanted to insert large pull quotes into the post to break up the monotony of the text and to catch the reader’s attention, especially those just skimming the post. While the “correct” way to do this would have been through HTML and CSS, I couldn’t get them to look the way I wanted with the limited options available to me using the standard WordPress.com templates. As an alternative, I decided to try to create the pull quotes as images (with appropriate alt text, of course).

Here’s an example:

I tried a few approaches with standalone graphics editors (such as creating text in Acorn) but my limited knowledge of those tool didn’t produce an acceptable result. I decided to try something I have a little more experience with – LaTeX.

The unfortunately-named LaTeX is a program for creating and typesetting documents. Proponents of LaTeX extol how it separates rendering from the content, just as web designers rely on the separation between HTML (content) and CSS (style). This separation makes it fundamentally different from editors like Microsoft Word and Google Drive which intermingle styles and content together (so-called What You See Is What You Get (WYSIWYG) editors).

I first learned about LaTeX in college, when all of the computer science professors typeset their problem sets and papers using it. The documents looked gorgeous, and I learned just enough to get by. I use the MacTeX distribution for Mac, with the TeXShop frontend for editing. If you’re at all interested in trying it out yourself, ScribTeX allows you to play around with the language without having to install anything.

Everyone I’ve ever talked to who uses LaTeX ends up copying and pasting styles and snippets of markup from existing templates; it’s an incredibly complex system and one that’s hard to keep in your head. If you need absolute precision of how things render, you will probably end up being very frustrated with LaTeX. I remember spending hours trying to move figures to a certain position on the page or align certain, before giving up in frustration. For the task of rendering pull quotes and others where you don’t need absolute control over the output, LaTeX does a great job.

In order to create large, arresting images, I needed to increase the font size and increase the margins from the default. The best solution I found was the following:

\documentclass[36pt]{amsart}
\usepackage[margin=2.5in]{geometry}
\geometry{letterpaper}

\begin{document}

{\Huge``I started designing apps for my guitar students as a way to compete with Guitar Hero.''}\\

{\Huge``I like to hand someone my phone and say, try this. You'd be amazed at how they struggle to find 
something you thought was obvious.''}\\

{\Huge``If my wife looks at me like I'm nuts (which happens frequently) I try to rethink things.''}\\

{\Huge``Only include features in an app that 80\% of users will use''}\\

{\Huge``If you look at Apple's success I think it's largely the result of keeping things simple and easy to use, and gearing products for the average user. Take the same approach in your apps.''}\\

\end{document}

Once you have that entered into a .tex file, you can hit Typeset and out pops a rendered PDF.

At that point I took a screenshot of the section of the text I wanted, uploaded that to my hosting provider, and linked the image as usual.

Conclusion

LaTeX is an incredibly powerful tool for document creation and typesetting. In a pinch you can use it to do as I did and render small snippets of text as pull quotes.

“Beautiful LEGO” review – inspiring and beautiful

Disclaimer: I received a free review copy from No Starch Press. The images I’m linking to do not come from the book itself; as I mention later in the review, the images in the book look better than the ones I am including in this review.

Beautiful LEGO, by Mike Doyle, asks the question “Can LEGO be art?” Given that Mr. Doyle is himself a LEGO artist, it’s no surprise that the answer presented in the work is a resounding yes. If you’re looking for an in-depth discussion of the subject with academic criticism, look somewhere else. If you’re already a fan, this book is an incredible collection of artistic talent, and one I recommend without reservation.

Contents

The book is divided into various themed sections, punctuated by interviews with the artists. The sections include

• Everyday Wonderful – Depicts everyday objects such as a Polaroid camera, the Nintendo Entertainment system, cameras, and telephones. These works are very realistic and not stylized.

• Attic Treasures – Works by Matt Armstrong, in the same vein as Everyday Wonderful, but focused more on older technologies such as Morse code, sewing machines, and telescope.

• CubeDudes™ – Angus Maclane creates famous characters and historical figures out of LEGO, with the heads at 45 degree angle (pointing towards you). These include Smokey the Bear, Abraham Lincoln, and Star Trek characters. Here we really start to see the art form come alive as more than just a representational medium. There is great style and care put into these small figures, instantly recognizable despite their size.

In all there are about 30 sections/interviews throughout the book. The usual suspects, such as cars, buildings, space ships, and mechs, are present, as well as less usual subjects, like Monty Python (Pythonscape) and mosaics.

Interviews

There are about 10 interviews with the artists, and they might be my favorite part of the book. The artists discuss how they got into using LEGO as an artistic medium as well as their design process.

Interview subjects include

• Ramón and Amador Alfaro Marcilla – two brothers who work together to create incredible sci-fi works, such as the chest burster from the movie Alien (p. 9). They also have the honor of having the most disturbing picture in the whole book, The Doll (p. 5).
• Jordan Schwartz – a professional LEGO designer working in Denmark.
• Nathan Sawaya – a builder who creates life-size sculptures, and who has an art show called “The Art of the Brick”.
• Mike Nieves – a builder who uses pieces I’ve never seen before, as well as using familar pieces in creative ways. For instance, Olaf the Bearded depicts a warrior with a long flowing beard. On closer inspection, the beard is an octopus figure.
• Arthur Gugick – a creator of incredibly detailed architectural creations, such as Big Ben and Salisbury Cathedral.
• Mike Doyle – the author of the book. Like Arthur Gugick, he creates large scale buildings. He is perhaps best known for his beautiful decaying Old Victorian mansions.
• Nannan Zhang – very short interview, but Mr. Zhang creates some of the darker pieces in the book, such as End of Days.
• Lino Martins – a builder best known for his series of car creations.
• Ian Heath – creates human characters with lots of personality, like Freddy Mercury and Stephen Hawking.

My favorite quote is from Lino Martins, whose work “Hidden Treasure – 1949 Buick Fastback” is shown below:

One LEGO piece, while an engineering marvel, is not very exciting on its own, but bins of thousands of pieces – that’s stored kinetic potential. That is a million works of art waiting to be made. That is life. And in the hands of another LEGO artist, the very same pieces can become a million things I have never fathomed myself. It’s like being in art school all over again. Even without a signature, our styles are diverse enough that we can tell one artist’s work from another. (p. 186)

Styles

I thought this comment on styles was hyperbole, but as I looked through the book a few times, I realized that it’s not. I started to recognize artists that appeared in multiple sections of the book. For instance, Tyler Clites packs a ton of personality into small spaces. “Sometimes It Sucks To Be a Ghost” (p. 105), shows a ghost being sucked into a vacuum cleaner, with toys strewn about the floor. Without reading the artist’s name, I guessed that another work, “Grandpa! You better not be using my loofah again!” (p.92) was by the same author.

The styles and scale of composition vary tremendously among the artists. Mike Doyle, for instance, creates enormous, intricate, realistic decaying Old Victorian mansions

Nathan Sawaya creates life size human sculptures, such as Frozen Figure (p. 48). Others create whimsical fantasy architecture on a medium scale, such a Sean and Steph Mayo’s Micro Fall’s Fortress (p. 142). MisaQa’s Little Town (p. 106-107) is even more microscopic.

At the most minimal level are Eric Constantino’s adorable Big Eyed LEGO Duck and Big Eyed LEGO Peacock, which are instantly recognizable yet weigh in at no more than a dozen pieces or so.

This book helped me appreciate the fact that there is real artistry and talent that goes into making very small scenes. It’s easy to appreciate the enormous scale of Nathan Sawaya and Mike Doyle’s work, but I am convinced that just as much skill goes into some of the smaller compositions featured in this book.

Quality

The production quality of this book is excellent. All of the pictures have had their backgrounds digitally erased and replaced with a pleasing gradient texture (look at the Mike Doyle pictures I linked to above to get a sense of what all the pictures look like). There are a few places where the photos submitted by the artist were clearly not high resolution enough to feature in the book, and they look very out of place. The two instances I found were “Mort” (p. 101) and “Pierre, Of Course” (p. 133). This is a minor nitpick of an otherwise excellent book. For some sample high-res images see MicroBots, or the page of horses.

Conclusion

I have read through this book about four times now, twice on my own, and twice with a 2- and 3-year-old on my lap. They would not let me put it down and were awe-struck on every page, oohing and awwing. As soon as I finished it once, they made me immediately start over again. It was wonderful to see such excitement and energy channeled towards these LEGO creations.

This book is a wonderful addition to my library, and I’m confident that anyone with a passing appreciation for LEGO will love it too.

2014 Resolution – 52 blog posts

I have barely blogged in the past two years:

The two main reasons are

• Some of my posting has moved to Google+
• I’ve gotten lazy

Writing is an important skill to hone, so I’m going to do better in 2014. Inspired by Jennifer Dewalt’s 180 websites in 180 days, and Brian Brushwood’s video Why Resolutions are DUMB, I publicly commit to publish 52 posts in 2014. If I’m successful, that will be over twice as many as I’ve managed to publish in the past 2 years combined.

To whet your appetite, here’s some of the things I’m planning on writing about:

• Meeting Edward Tufte, one of the foremost experts on data visualization
• A talk given by Don Norman, author of The Design of Everyday Things
• A walkthrough of a Go program that converts images into LEGO mosaics (see gallery on G+)
• Switching to Android after 4 years on iPhone – what took so long, what’s better, and what I miss

I hope you’ll join me.