Linden Lab Labels Yankee Group's Report "Second Grade Math"

On Monday we reported on a Yankee Group study that pronounced that "the hype surrounding Second Life doesn’t match its actual marketplace impact." Senior Analyst Christopher Collins argued that Second Life was too PC-centric for modern consumers who are used to mobile content and, as a supporting point, rated the average user's time in world at 12 minutes per month. Those numbers didn't sound right to a range of writers. Now Linden Lab has weighed in on the matter, putting the average user time at 23.6 hours per month. "So, just what is that 12 minutes per month number??," blogged Ginsu Linden. "As near as we can tell, that might be the average time that users spent logged in on the Second Life website in a month. Or . . . could it be . . . the 6.2 million unique user registrations divided by 23 million hours is close to 12 minutes . . . but that makes no sense - is it possible that anyone would have done the division backwards??"

"There's been quite a bit of interest there, and a concern of ours is that a couple data points in the piece appear to be overshadowing the thesis of the piece, which was a focus on mobility and how a PC-centric or tethered experience challenges virtual worlds for mainstream adoption," Christopher Collins told VirtualWorldsNews.com.

"One number seems to be a lightning rod for a lot of attention," he continued. "I followed up with Linden Lab and they've sent along some more information with us. Based on that conversation, we've pulled the article for the time-being so we can revisit that particular piece."

Collins expects to republish the study on virtual worlds and mobility next week. However, he's not entirely satisfied with the available numbers.

"Even based on Linden's own analysis, there are usages that range from an hour a month to 40 hours a month," Collins explained. "We want to use the best available number that we can, and we don't want that to distract from the overall theme of the pass."

Collins focus, though, is on mobility. He views the discussion on the average monthly usage times as significant, but a distraction from the main thesis of the study.

"There are potentially two issues here," he said. "One, did we do a disservice? That was not the intention. Beyond that, in doing so, did we distract people from the point of the article? We're not really in the business of monitoring the blogosphere because we oftentimes have to make unpopular comments about things. But what I did observe is that there has been less debate about what we hope the point was and more focus on a supporting point."

At the time of our conversation, he hadn't read Linden's response. Since the Yankee Group had been in communication with Linden Lab beforehand, he believes the post may have been the result of miscommunication.

"We had said to them what we had said to you," concluded Collins. "I can understand that this can be a lightning rod. In our defense, I feel like there's been a lot of, how would I say it, disputable numbers in Second Life's favor that have come out, but that's just my opinion."

Comments

[Pavig Lok]

I'm amazed they're trying to backpedal out of that one. It's unsurprising that "there has been less debate about what we hope the point was and more focus on a supporting point." When the supporting points of their argument are inaccurate by an order of magnitude, it's fairly safe to assume their main thesis is similarly broken. It will take debate to figure out the validity of their assertions as they clearly can't be supported by the evidence they provide.

Nobody with even the slightest idea about the metrics would have allowed that figure to go out unchecked. They would have saved at least some face had they claimed it was a typo. Trying to justify the figure they arrived at simply proves that the yankee group can't be trusted to provide reasonable business intelligence in this issue.

Folk I work with in the virtual worlds community are frustrated but philosophical about these kinds of inaccuracies coming from the media, but we all understand that news also fulfils an entertainment function. The media can get away with the occasional gaffe in the search for a good story. This however is basically propaganda from a company who has trademarked the terms "Anywhere Consumer" and "global connectivity experts", masquerading as business intelligence. Unlike other consultancies in the area hoverer, the message they're selling and the nature of their IP holdings reveal a brazen conflict of interest.

[Markus Breuer (Pham Neutra)]

Mr. Collins is - kind of - correct in that he has provided the virtual worlds community with a number that's so hilariously wrong and easy to attack that this might distract from the core thesis. This core thesis seems to be that a successful product in the "social software" market has to have a mobile component so that people can stay connected to their buddies from everywhere and in any situation.

The main problem - as far as it relates to Second Life - with this theory is, that Collins uses the monthly usage numbers as proof for it. ("See, Facebook has all these deeply involved users spending 186 minutes there and SL is a loser. People only spend 12 minutes a month inworld - because it has no mobile component, of course!") And because these numbers are an essential part of his line of reasoning it seems a bit laissez faire to me, to say "Oh this little error does not matter, the theory behind it is the thing that counts." (Additionally, Facebooks offerings for staying connected to your network without a PC are not exactly ... "great", to phrase it politely.)

The underlying problem is a different one, though. Mr. Collins is comparing apples with onions. MySpace and Facebook (and I am a HUGE fan of Facebook) are good for asynchronous, casual interaction with a large network of people. Second Life and other virtual worlds are good for synchronous, intense interaction with a limited number of people.

The two kinds of platforms are intended for different purposes and it really does not make that much sense to compare them with the same key performance indicators.

That - and not the absurd 12 min number - is maybe the biggest flaw in this "note" by a senior professional of this renowned analyst group.

Btw: It might be possible, to combine the different purposes of social network sites and virtual worlds. Kaneva certainly tries to do so. Google's rumored new product might try the same. But that is a different topic all together.

[Gwyneth Llewelyn]

Verily, if there was a reason for Linden Lab to ever sue anyone (they seem far too nice for that), this would have been it.

In the first case, Yankee Group, as a reputable company, should never have published a report with wrong maths and a well-argued point on dissing a product based on the wrong maths. It's quite naive to dismiss the whole case as Markus/Pham pointed out saying "oh, the number is not important, the rest of the report focuses on something else entirely".

Even to retrace their steps is not enough. Yankee Group is supposed to present its customers analysis based on numbers; and doing metrics and statistics to caution their customers about bad investments, or promote good ones. This is their core business.

More interestingly is that the Yankee Group is now using Nielsen//netRating's new metric of "minutes spent online" — a brave new metric which will dramatically change the way we measure "success" of a service. As I have previously written ( http://gwynethllewelyn.net/2007/08/26/please-get-your-facts-straight/ ), if you start looking at published statistics, you'll soon figure out that things like Second Life, using the minutes-per-month metrics, will outperform popular venues with far more users like YouTube, even if you take into account long-unused registered accounts. LL's own statistics only reinforce that image since they provide updated statistics.

The Yankee Group is deliberately misleading their readers (and I agree — deleting the post is not enough, Google cache will preserve it forever, or at least long enough for all the world's media to misquote the Yankee Group) by introducing wrong data and drawing conclusions from it. I thought it was a misprint or a different calculation that they made (in order to compare it better to other social sites). By deleting their own report and providing a half-baked answer they're affirming their own incompetence twice: first, that they never checked the values (and had no intention to do so); secondly, that they fail to explain why their whole report still makes sense after admitting they can't do maths.

Gross incompetence — or malice?

There definitely seems to be a world-wide conspiracy forming around the uncanny success of Second Life. The "too good to be true" is seriously nagging at too many sources. Linden Lab is a start-up — granted, with eight years of existence — that is not "affiliated" (yet) with any megacorp. This is seriously bothering many people from all sides. It seems quite clear that nobody wants to believe that Philip is, indeed, pulling this off by himself — with little funding, almost zero advertising, and no technological or strategic partnerships. He's the loner (with a nice number of pro-eminent figures on his Advisory Board) that did what everybody never believed to be possible — and make it profitable.

The "establishment" cannot accept it. Not any longer. Second Life continues to get press coverage everywhere — even if recently it's mostly bad press. It continues to make journalists, opinion makers, bloggers — and now marketing groups — to think about what this means and shrug it off on their reports to the public. Second Life's media attention is so huge now, that the only way to get noticed is to blatantly give bad press to it — the last refuge of a company, person, or organisation that has no way to push their message across, but to place the words "Second Life" and "is bad" somewhere in their articles.

"We're not really in the business of monitoring the blogosphere because we oftentimes have to make unpopular comments about things." Interesting, though, they made a public reply to the blogoshpere. Obviously nobody believes a marketing group that claims "not to monitor the blogoshpere" but is in the business of providing metrics and analysis reports on what goes on in the net! They're not exaclty reporting on the rise of coconut prices in Bolivia, aren't they?

"But what I did observe is that there has been less debate about what we hope the point was and more focus on a supporting point." Indeed. I have heard the argument "we need more applications on mobile phones" from lots of sources. In fact, a good friend of mine of Vodafone does precisely the same argument: "Second Life? It's not interesting at all; we need more mobile phone apps, that's where the future is". (Fortunately, Vodafone is not so crazy as my friend claims, since they've launched http://secondlife.vodafone.com ).

But is it really? Or is that just wishful thinking on the part of the mobile operators, who are finally reaching the dead end of over a decade of successfully squeezing money out of people? Namely, are mobile applications slowly hitting the long tail, and are the revenues from those suddenly not so interesting any more? I still am confident that the iPhone (and its clones that will surely follow) will somehow bring the notion that "mobile apps" doesn't necessarily mean crappy interfaces and thumb cramps and 5-point-fonts on a tiny screen, but that remains to be seen.

Just because I take 2 minutes to type a SMS to send it to Twitter, and 10 seconds to type the same message out of Second Life (which also integrates with Twitter nicely and easily), it doesn't mean I spend more time using my mobile phone for "social applications". It means that I have a crappy interface for typing messages. There go your statistics down the drain.

I would finally also add to Markus/Pham's comment that I desperately tried to estimate how many social sites exist around Second Life and utterly failed. I have this nagging suspicion that the SLogosphere (the total number of external sites around Second Life, social, blogging, media, e-commerce, search engines, etc.) is vastly huge compared to any other virtual world, and starting to come pretty close to the same level of, say, Britney Spears fansites, Apple/Microsoft unofficial support sites, and the like. Anedoctal evidence shows that no other virtual world has, say, their own TV and radio networks. Other circumstantial evidence points just to Google PageRank — add "second life" to your search, and watch the uncannily high number of results it shows on sites specific to Second Life that deal with the very same subject you're searching about. This is for me a revolution, and its implications are not clear yet.

But then again, I have no strong scientific support to further my claims — just a nagging feeling. Of course, I'm not Yankee Group and have no access to accredited mathematicians to search the raw data and provide thoughtful analysis on those and extrapolate it to formulate a conclusion.

The suspicion that Second Life has a far wider impact than anyone ever imagined (even from people that never used it), and thus is making the giants tremble as they are clueless on how to deal with it (since it's totally unforeseen) is quite strong in my mind.

[Markus Breuer (Pham Neutra)]

Hi Gwyn, I would not look for a conspiracy out there. I never look for conspiracies as long as stupidity and incompetence is sufficient to explain what is happening.

What they tried to do was something many well established consulting companies do: the tried to reinforce some marketing focused mantra ("Anywhere Consumer") with numeric evidence. They blotched that and now they have to defend their position (this is the yankee group; they can't simply say "oopsie, please forgive us. some assistent made an error while trying to read an excel sheet ...")

Actually the theory behind the Anywhere Consumer isn't bad. Second Life WOULD profit from more interoperability with web based and mobile systems. I would LOVE an application for my mobile, with which I could take part in an IM or chat discussion with the IM/email lag. (No, I am not satisfied with the Vodafone solution.) And I would love to be able to do at least some "stuff" in SL with a light weight client from my mobile phone, too. That is certainly not impossible. Just not implemented yet. And it WOULD make Second Life more attractive to more people, probably.

But thats not the point in this discussion. What they don't understand is, that virtual worlds and web based social networks fullfil different roles/ are different communications tools. It simply makes no sense to compare a "profile site" with a virtual world. As it makes no sense to compare a telephone call with a lunch meeting. One isn't better or worse than the other. They serve different purposes.

Most of the virtual world bashers out there probably don't think that they are "bashing" at all. They see something that they don't understand, that they don't like personally and come to the conclusion that it has to fail - because "nobody needs it". I guess they really think that they are doing the public a service. :)

This is not actually a new development. 25 years ago the same people (or companies) predicted that the idea of the personal computer had to fail. 12 years later they advised their clients that the world wide web would never endanger newspapers or printed catalogues ...

[Randal Oulton]

>> Posted by: Pavig Lok | October 05, 2007 at 04:32 PMThis core thesis seems to be that a successful product in the "social software" has to have a mobile component

I agree, that was his main point. And my thought was, I just upgraded to a beautiful 24 inch monitor. Why the *hell* would I want to spend hours on the Internet on a 1 1/2 inch square screen that I have to squint at to see?

You know what's overhyped? It's this *mobile* stuff. For Pete's sakes, most North Americans don't even know what a text message is.