Accidentally in Code

Archive for the ‘Visualization’ Category

After I gave my presentation the other week, someone asked a question. It was:

So, basically what you’re doing is data-mining?

And I said, no, well yes, but that’s not how I think about it. I see it as creating something that will help people understand their use of Twitter. The fact that I achieve this by data mining is by-the-by.

Maybe when we speak to other programmers it’s OK to say something like, “I’m data-mining social graphs in Twitter and visualizing them” but when we speak to our users, that may not mean very much to them. What’s more, I don’t think I would have come up the idea to do that if I’d gone to Twitter with the intention of data-mining. This didn’t come from me as a programmer with an interest in data-mining, or an interest in visualization (as an aside, I took a course in visualization at Edinburgh and hated it. Mostly because we were coding in Tcl). It came from me as a Twitter user, wanting a better way to measure engagement than followers/following.

Yesterday, I wrote a little bit about the journey that brought me to Ottawa. I think I’ve finally realized what I’m passionate about. It’s people. It’s users. This is why I’m so fascinated about what I’m working on right now – what’s more people than social networking? It’s also why I’m so interested in Usability. I’ve read every article on Don Norman’s website, I find usability so interesting, so important.

I’m passionate about giving users what they want – that’s usability, better ways to display data, etc. That’s creating the things they say they want.

Even more so, though, I’m passionate about giving user what they want, that they don’t realize they want yet. In small ways, that’s telling people who are emailing spreadsheets about Google Docs, or explaining to someone frustrated by their web designer about the simplicity and ease of use of WordPress. In bigger ways, it’s been taking a mess of spreadsheets and turning it into a database that can answer questions that users hadn’t even thought to ask. It’s been creating something that’s can make you really aware of your conversational network, and encourage you to talk to new people (the most rewarding feedback I got was from someone who told me they were now making an effort to speak to more people after seeing their graph). I hope these things are just the beginning.

So, what do I want to be when I grow up? I want to be a programmer who speaks fluent human. How about you?

I know, it’s been a long time coming. But finally – my new visualization which I’ve created using IBM’s Many Eyes (which is awesome, although very blue).

Since Twitter released it’s new “Lists” feature, there’s been talk about how Lists are a good way to measure influence – someone who is on more lists is likely to be more influential. Likewise, someone with lots of followers who’s not on any lists (or maybe one called “spammers”) is less influential.

However, what about measuring influence within lists? For instance, if you’re using lists to collect a group of people, such as the Girl Geek Dinners list, it might be nice to visualize something that indicates how influential people are within that network.

So, what I’m doing is displaying the scatter-graph of followers vs following, with the size of the bubble being proportional to the number of times the user has been mentioned by another user in the list.

See the dynamic one for my friends here. Static one below:

9117565e-d240-11de-be19-000255111976 Blog_this_caption

Here’s the one for GGDOttawa (dynamic version here):
B2507ef0-d249-11de-aa1c-000255111976 Blog_this_caption

More coming soon! Taking requests and suggestions!

This is kinda awesome – check out the demos here.

Screenshot for “Social Networking” from¬†Amazon:

 

"social networking" on Amazon - using TouchGraph

"social networking" on Amazon - created by TouchGraph

My Facebook network (zoomed out):

 

My Facebook Network (zoomed out) created by TouchGraph

My Facebook Network (zoomed out) - created by TouchGraph

Zoomed in:

 

Facebook Network (zoomed in) - created by TouchGraph

Facebook Network (zoomed in) - created by TouchGraph

And finally, this adds a whole new dimension to Vanity Searching!

 

Vanity Search - created by TouchGraph

Vanity Search - created by TouchGraph

 

 

 

 

 

I’ve created my slides for my presentation (abstract) following a Presentation Zen (Amazon) approach. So there are basically no words on any of my slides. I’m going to lay them out one by one, with narration. Let me know what you think!

Title Slide

Title Slide

I’ve gone for a blank template. I wonder if this is too plain and could use some background color? Let me know what you think. The color of the lettering matches the color scheme for my soon-to-be-launched website.

This is where I introduce myself and what I’m talking about. I don’t really like this bit of presenting, so I’ll keep it brief.

The World Wide Web

The World Wide Web

Image from iStockPhoto

I’m going to start off by talking about the “World Wide Web” and trying to provoke a bit of thought about it. It’s relatively recent, and yet it’s game changing. How many of us could live without it? I certainly couldn’t. Whilst the internet has been around since the 1960’s, the web as we know it (with hyper-linked web pages) has only been around since 1989. But within 20 years we’ve got to the point where the web is basically infrastructure…

Electricity Infrastructure

Electricity Infrastructure

Image from Flikr User BK59

… like electricity. For more on how I think the internet is just infrastructure like electricity, running water, or roads, see this post.

Clay Shirky - Why I Ignore 5 Year Plans

Clay Shirky - Why I Ignore 5 Year Plans

Clay Shirky – giving us some needed perspective. I think when we reflect on what’s changed, and how recent it is, we realize that this is really just the start of it. I remember life-before-Facebook but I can’t imagine living without it, even though I’m not always sure that the level connectivity it gives us is a good thing (for more on my mixed feelings about Facebook, read this post).

Near-Universal Authorship

Near-Universal Authorship

A couple of hundred years ago, all the information your average person had came from the Bible, if they could even read. Now, not only are rates of literacy very high but the internet gives everyone the ability to be a publisher of content too. There’s a fascinating article on this here.

How Web 2.0 is Changing the Way we Comminicate

How Web 2.0 is Changing the Way we Comminicate

I put this graphic up in this post. What I want to show here is that there’s a shift to newer technologies and that it’s a process. This diagram captures what I’m thinking and finding now, but no doubt in a year it will be different. Also important, is how many arrows are going into Twitter – it’s simplicity and flexibility mean that it’s a great way to do a whole variety of things. There’s definitely stuff that’s missing from the diagram – the thing is to balance what’s important with trying to include everything and making it impossible to follow.

Serendipitous Connections

Serendipitous Connections

Image from iStockPhoto

Something that isn’t represented in the previous diagram though, is the possibility of serendipitous connections. By lowering the bar to communication, and through ambient awareness we can have more “weak-tie” relationships. Think about how many people you’ve lost touch with but found (or had find you) on Facebook, or the number of people you “follow” on Twitter but have never met.

Hang on - Isn't Twitter Completely Pointless?

Hang on - Isn't Twitter Completely Pointless?

Image from http://geekandpoke.typepad.com/geekandpoke/2009/04/twitter-business-plan.html

I don’t think Twitter is a Pointless Waste of Time, but a lot of people do. If you do think Twitter’s pointless, why do you? Have you tried it? Over the next couple of slides I’m going to talk a bit about the impact that Twitter has had on events and the reasons why I think it’s useful.

Iranian Election, 2009

Iranian Election, 2009

Images from http://spinster.blogs.com/rak/2009/06/green.html, http://whatothersmissed.blogspot.com/2009/06/round-up.html, http://www.flickr.com/photos/ari/3644667355/, http://www.flickr.com/photos/ari/3625401150/

In the aftermath of the Iranian election, the US government intervened to change Twitter’s maintenance schedule so that it wouldn’t be down during the day in Iran. Twitter allowed Iranians to communicate with the outside world and express their distress at the rigged election. The governent’s reaction to this did not help their case.

I was working in Shanghai during the summer, and when the riots started they shut down Facebook and Twitter (at the time, I wrote this post). When I was there in 2007, you couldn’t access Wikipedia – but this time I could. This makes me think that the government is no longer afraid of information – it’s afraid of the conversation.

What's the Connection Between the British Parliament and an Irish Pop Star?

What's the Connection Between the British Parliament and an Irish Pop Star?

Images from http://commons.wikimedia.org/wiki/File:British_Houses_of_Parliament.jpg, http://www.flickr.com/photos/24960504@N06/3581090321

As a Brit, I tend to follow the news over there and the other week two things happened. The first one that a law firm got an injunction to prevent a newspaper reporting from a question to be asked in parliament regarding some toxic waste and a firm called Trafigura. The newspaper (the Guardian – read their story here) could only report that they couldn’t report anything about the MP asking the question or the question itself. However some smart people soon worked out what the question was and soon #trafigura was trending – causing the very awareness that they had sought to prevent.

The seond thing that happened was that this pop star died, of a heart condition. However he was gay, and with his husband at the time so it was enough to get some vile columnist in the Daily Mail to write a homophobic diatribe about him. Her name was soon trending as people expressed outrage (interestingly the Wikipedia article on Jan Moir consists of little more than the story of this).

I think what these two events show, is that Twitter provides a forum for people to express their frustration with all kinds of things – whether it’s concern over restrictions of reporting on parliament or just celebrity gossip. It also captures what people are getting angry, or excited about. Trending topics can answer what’s hot right now – Apple trends whenever it has an announcement, Windows 7 and even Ubuntu 9 were trending on their release.

Trend Graph for #trafigura and "Jan Moir"

Trend Graph for #trafigura and "Jan Moir"

Trendistic is a service that allows us to graph how often words are occurring in Tweets. We can see that #trafigura rapidly became popular and disappeared again quickly once the injunction had been lifted. “Jan Moir” has a wider curve, and there was another flurry of mentions when she released an apology a week later.

Information Gathering

Information Gathering

Image from iStockPhoto

Information gathering is the best use I get out of Twitter, and I feel it’s something that it would be hard to replicate anywhere else with as little effort. Follow leaders in your field, or just people who work in a similar area to you who tweet interesting stuff they found, or people who inspire you. I get so much information this way, and it doesn’t take very long to go through it. I’m literally crowd-sourcing my news! And if someone I’m following tweets too much or I lose interest I can un-follow them. Lists are going to make this even easier.

Ambient Awareness / Ambient Intimacy

Ambient Awareness / Ambient Intimacy

Image from iStockPhoto

Ambient Awareness is like the Facebook newsfeed. It’s passively letting information about your friends lives come to you. But on Twitter, people often update more often – so if it’s someone you’re really fond of it can be better than Facebook for staying in touch with their lives. Two of my good friends live in London and I love the little bits of their lives that I see on Twitter and we definitely use ambient awareness to stay in touch. I was having a lousy day last week and I tweeted about it and one of my friends sent me a message just saying “*hug*”. When another friends released his work project (Google Sync) and it hit the trending topics, it was really nice to share his excitement about it in real time.

Great article in the NY Times on Ambient Intimacy.

Customer Relationship Management

Customer Relationship Management

Image from iStockPhoto

Twitter is amazing for overhearing what your customers are saying about your business. There was actually a paper published earlier this year analyzing brand sentiment on Twitter. Businesses (and non-profits, like @kiva) that get it are on Twitter seeing what’s being said about them and taking part in the conversation.

I wrote more about customer relationship management in this post.

Conversations on Twitter

Conversations on Twitter

Image from iStockPhoto

I think conversations are a great way of measuring engagement on Twitter, and this is what I’m working on at the moment. If your a brand, how engaged are you with your community? If you’re an individual, how engaged are you with the people you’re following? Are you passively absorbing content or are you sharing, adding value? If you’re a spammer, no-one’s talking to you!

Who's Following You?

Who's Following You?

Image from iStockPhoto

It’s possible to have a large number of followers by either paying for them, or by following people in the hopes they’ll follow you back (and unfollowing those that don’t) until eventually you have several thousand people “following” you.

Spammers

Spammers

Image from http://geekandpoke.typepad.com/geekandpoke/2009/02/spam-followers.html

But are they listening?¬† Or are they just spammers? If they were listening you’d expect at least some of them to be talking to you.

Is Anyone There?

Is Anyone There?

So I decided that follower/following counts were basically meaningless, and wrote a program that will graph your conversation network (more on that in this post).

And it’s fascinating, because someone’s graph really says a lot about the kind of user they are. Are they a power-user? A regular user? A spammer? A light user?

My Conversation Network

My Conversation Network

Spammer Conversation Network

Spammer Conversation Network

Power User Conversation Network

Power User Conversation Network

Moderate User Conversation Network

Moderate User Conversation Network

Light User Conversation Network

Light User Conversation Network

It’s nice how you can see the different networks that he’s a part of here. I hope to use a clique finding approach to draw out this kind of information for bigger and busier networks.

Is This Pretentious?

Is This Pretentious?

@NonaCobbzea's Twitter Conversation Network

@NonaCobbzea's Twitter Conversation Network

This one is interesting – the network is fairly connected, if it wasn’t for the big ball of purple in the middle (@NonaCobbzea) it would look pretty normal. She’s sending unwanted @ messages with the same link in each one to a weird “blog” that’s a overly descriptive of certain aspects of her life I’d rather not know about. But is it working? Most likely it’s driving traffic, but no-one’s talking to her and hopefully Twitter will kick her off soon.

There’s a bit of an Ottawa focus, so I wonder if the connectedness in the graphs is a result of that.

I wonder how this will change with the new report as spam feature.

@spam on Twitter - October 22nd

@spam on Twitter - October 22nd

I got another irrelevant @ message. The content of this one was a link to … you knew where I was going with that. I don’t need to say it! Check out the graph for the originator – @BaileyWolfeaem:

@BaileyWolfeaem's Twitter Conversation Network

@BaileyWolfeaem's Twitter Conversation Network

Again, lots of one way links to “normal” Twitter uses (who have reciprocal conversations). the graph is huge (goes way off the screen) and quite sparsely connected.

Another spammer, this one better disguised, brought to my attention by @smiffy:

How amusing. I have just been followed by what looks like a ‘bot. The funny bit is that half of the ‘bot’s followers are ‘bots themselves.

So here is @tina_revers:

@tina_revers's Twitter Conversation Graph

@tina_revers's Twitter Conversation Graph

Again, the one way links and a fairly small graph. Obviously for this spammer what I really want to graph are who’s following who – I have a theory that there is a clique of bots all following one another and spewing out spam – but this is difficult because of the way the API works and the limitations on calls. I’m working on it, though!

Finally, below is the graph for @connectcreators – I don’t think they’re a spammer, in fact what they’re doing looks like a good initiative but I do think they need to “connect” more!

@connectcreators Twitter Conversation Network

@connectcreators Twitter Conversation Network

I used Twitter’s new “Report as Spam” feature today for the first time, after I got a bizarre @ message from @billy_Napper_ (Following 0, 0 Followers) about the top 10 hosting sites. I don’t pretend to know all the different kinds of spam on Twitter, but there are those spammers that follow you in order to get you to follow back and click on their links (I talked about them in this post) and then there are the kind that send you unwanted @ messages.

So – I graphed him! See below:

@billy_Napper_'s Twitter Conversation Network (spammer)

@billy_Napper_'s Twitter Conversation Network (spammer)

Notice the one way links (purple) and the disjointed network (although I’m surprised it’s not more disjointed). The graph is also large relative to the number of people he’s following/being followed by.

I don’t know if this is particularly harsh, but I consider anyone who regularly promotes the same website without declaring an affiliation to be a spammer. Sometimes a well-disguised spammer, kudos to that, but a spammer nevertheless. Note – this doesn’t include people promoting their blog on Twitter. I tweet posts from my blog that I consider to be particularly important, but by no means all. I figure if people were that interested, they’d be subscribing to my RSS feed. But I digress.

Yesterday I was followed by @emmily67. As of now is following 2,001 and has 1,222 followers. She’s tweeting amusing little things like “Marriage is a relationship in which one person is always right, and the other is a husband.“, and “Life would be easier if you could mark people as spam“. Four of her links are the same site as the one in her bio, though, and some of the amusing phrases seemed to be running on repeat. So I, ah, marked her as spam.

However I’m interested in these kinds of Twitter users because I want to graph them (if you have any examples of this, let me know). So here’s hers:

@emmily67's Twitter Conversation NetworkWhat do you think? Does this constitute spam? Or is it legitimate albeit self-promoting?

@RebekahHarriman:

@RebekahHarriman's Twitter Conversation Network

@RebekahHarriman's Twitter Conversation Network

@RossIGrant:

@RossIGrant's Twitter Conversation Network

@RossIGrant's Twitter Conversation Network

@ponkey_60 – this one is interesting because usually she protects her tweets. She unprotected them for me so I could create the graph. I wonder if people with protected tweets typically have smaller networks?

@ponkey_60's Twitter Conversation Network

@ponkey_60's Twitter Conversation Network

@douglasgresham:

@douglasgresham's Twitter Conversation Network

@douglasgresham's Twitter Conversation Network

@tgrevatt is having a lot of conversations!! With a lot of people who have a lot of conversations. This graph is fairly densely connected (next step is to be able to pull these networks out). There are a few purple links, which are likely to indicate things like ReTweets.

@tgrevatt's Twitter Conversation Network

@tgrevatt's Twitter Conversation Network

@matthewfelgate:

@matthewfelgate's Twitter Conversation Network

@matthewfelgate's Twitter Conversation Network

 


Archives