Tuesday, November 19, 2013

Chapter 1.3: Star Wars and World Building (Quantifying Star Wars)

Hi everyone - I've been feeling guilty for neglecting the blog here for a few months.  I've been spending more time with the Star Wars stats book than I'd anticipated, and any time I have that I would have devoted to the blog I've been devoting to that.  I've been adding new content to the book, so I figured that one good way to get some more content here on the blog would be to share one of those new chapters with you.

This chapter also has a potentially interactive component.  I was trying to come up with ways to measure the success of world building in the first film of each trilogy, and eventually decided to produce a kind of survey measure.  That measure is contained in the chapter, but I only examined it against two different worlds: (A New Hope) Tatooine and (The Phantom Menace) Naboo.  If you're interested in trying out the measure yourself on other worlds simply copy paste the raw questions and fill them out for a different world.  If you do, feel free to toss up those examples in the comments!

Overall, feel free to use it on any other sci-fi/fantasy worlds, or suggest questions that I might have missed.

Hope you enjoy it!

Chapter 1.3: Star Wars and World Building

The first movie in each of the Star Wars trilogies has a different opportunity to establish and build the world in which that movie (and those that would follow after) take place.  World building is a pretty standard concept, and it’s one of the pillars of establishing a good story when you’re dealing with fictional works.  We don’t know anything about the Star Wars universe (it’s a long time ago and far away), so we have to be told about it.
A narrator could just come out and explain a lot of things about the world (or universe) in question.  If it’s not done right, this can be boring.  The narrator usually has to be woven into the story early or shoehorned in late.  About the only story I can think of that not only lends itself to narration but in fact begs for it is the Hitchiker’s Guide to the Galaxy series.  It works there, but in most cases it usually won’t. 
The problem is that telling the audience things is boring (unless those things are themselves entertaining, like Hitchiker’s Guide).  ‘Tatooine is a desert world, and many inhabitants work as moisture farmers to supply themselves and nearby towns and cities with sources of water.  Notable cities include Mos Eisley, Mos Espa, and Anchorhead.  The primary export of Tatooine is crippling depression.’
The way around this is to simply show your world through the natural dialogue and actions of your characters.  Showing what they do on a normal day, what their jobs are, what they eat, what they wear, etc., all go a long way to covertly building a believable world around them.  A well-constructed world should be one that is easy to portray and easy to pick up on.  It should feel natural, and things shouldn’t have to be explained that often.
I’ll point again at Lord of the Rings, as JRR Tolkien is arguably the best world builder, well, ever.  I had to think a bit about that statement, but it’s kind of hard to come up with anyone else to even round out a top three.  If you don’t believe me, read the Silmarillion.  Then, after that, read the rest of his collected works.  Then read the unpublished works.  Then we’ll talk. 
The point is that you don’t need to have characters telling other characters that, say, Dwarves exist, because they live in this world already and they know that they do.  The reader or audience should know that Dwarves exist because there are Dwarves all over the place.  More than that, you should understand what Dwarves are like by observing them carrying out normal actions.
The more you know about the small aspects of characters’ lives the more you understand the things they’re going through.  This gives a great opportunity to make characters more relatable, as you can show them with normal lives and normal days just like any regular person. 

There’s another way to think about this, to help put things more into perspective.  Imagine that you were someone from the world in question (in this case one of the worlds in Star Wars), and you showed up at a cocktail hour on some other world.  You get into conversation with someone and they start to make small talk.  The questions that they’re likely to ask of you are going to be based on the things that they assume that you should know if you’re even vaguely familiar with your world. 

These are also the things that an active audience would probably ask if they had a chance.  They don’t get that chance (at least during primary source), but the answers are the things that should be put out there to answer those questions anyway.  An audience member that’s been given a good picture of the world in question should be able to stand in for a member of that world in the small talk of an interplanetary cocktail hour.  If they don’t know enough to maintain a minute or two of small talk, then they probably don’t know that much about the world.

So, you just got to an interplanetary cocktail hour but no one you recognize is there yet.  Instead of standing around looking awkward, you decide to just start up some small talk with a stranger.  You learn they’re from another planet that you know nothing about (and they know nothing of yours).  You can both make some big picture assumptions about each other (you’re both standing in the same room, breathing the same air, for instance), but most things wouldn’t be so obvious.  I came up with some questions I’d probably ask to keep things from spiraling into awkward silence.

Questions for your next interplanetary cocktail hour:

1)      What are some common things that people eat?
2)      What kind of living space does the average person live in?
3)      What are some common things that people do for a living?
4)      Are there many large cities?  What are the differences between city and non-city life?
5)      What are some of the main products or exports?
6)      What doesn’t the world have, or what does it import?
7)      What do people do for fun?
8)      What do people dream of or aspire to?
9)      Are there different cultures in close proximity?  How does this impact daily life?
10)   What language do people speak?  Do different cultures/races/species have different languages, and if so how do they communicate across this gap?
11)   What is the climate like?  What are some problems/benefits created by this climate?
12)   Why do people live there?  How long have people lived there?
13)   What sort of power hierarchy or government exists?
14)   How do those in power get and/or keep power?
15)   What does the military look like?  Is there mandatory service?
16)   Are people generally happy with status quo?  Is there any unrest?  Open conflict?
17)   What would it take to disturb the status quo?  Why?
18)   What’s the level of technology?  What technology shapes everyday life?  What technology is unique to this world?
19)   Is there any technology so advanced that it’s basically magic?  Is there just straight up magic?

      This chapter is actually a bit different from others, as what I've (accidentally) done is created a sort of survey that you can give yourself.  You could pretend that you’re standing in for an inhabitant of any given world and see if you can answer these questions accurately with only your knowledge of source material.  This means you don’t get to pull things from secondary source, only from the things you’re looking to examine (in this case the Star Wars movies). 

Give it a try.  Write these questions down (or copy paste them into a different document).  The simple idea is such: the better you can answer these questions in a satisfactory way, the better those questions were answered by your experience with primary source.  The better these questions were answered, the better job of world building was done in that primary source. 

I really do think this would be a good exercise for you as the reader to do on your own, but I also figure it’s worth giving it a go on my part, at least for a few worlds.  If you’re going to try it, copy these into a separate document right now and don’t let my answers spoil yours, but if you just want to be spoon fed what I’m selling then read on. 

Let’s start with Tatooine from A New Hope. 

1)      What are some common things that people eat?
-          Things like salad that can be grown at the home in fairly harsh conditions.
2)      What kind of living space does the average person live in?
-          In the cities people live in small stone buildings that more or less all look the same, outside of the cities people tend to live in isolated locations or even underground. 
3)      What are some common things that people do for a living?
-          Due to the desert climate, water is a scarcity.  People can operate moisture farms to capture back some of this valuable water.  In the cities, people have a wide range of possibilities, though many of them relate to scum and villainy.
4)      Are there many large cities?  What are the differences between city and non-city life?
-          The closest cities are Mos Eisley and Anchorhead, but they’re a trip from out in the desert.  Life outside the cities is much more independent but also much more isolated. 
5)      What are some of the main products or exports?
-          There’s not much here that’s unique and not present on other worlds - if there’s a bright center of the universe this is the planet that it’s farthest from.  The most valuable resource is probably human capital, and in terms of people leaving that’s probably the largest export.
6)      What doesn’t the world have, or what does it import?
-          Pretty much everything.  Moisture farms provide the basic needs of life, but beyond that things are pretty lacking. 
7)      What do people do for fun?
-          Go into town with friends, drive around or race through canyons, ‘bullseye’ things. 
8)      What do people dream of or aspire to?
-          To someday leave and find a better life elsewhere.
9)      Are there different cultures in close proximity?  How does this impact daily life?
-          There are both Jawas and sand people (Tusken Raiders) that make their homes out in the desert leading nomadic lives.  The sand people are somewhat hostile, but the Jawas are friendly.  They’re also scavengers of technology and good in the repair of it, and make a living selling tech that they’ve found and repaired.  In daily life one has to be cautious of venturing too far out into sand people territory, but can also rely on the Jawas for technological needs when they travel close.  They seem to almost be traveling salesmen. 
-          In the cities, especially the spaceports, there is a much greater blending of cultures.  Races from all over the galaxy might find themselves in the same location, so it’s important to be alert to that.
10)   What language do people speak?  Do different cultures/races/species have different languages, and if so how do they communicate across this gap?
-          There are many different languages, though people are usually fluent in at least those they encounter on a frequent basis.  Droids are employed in many cases to translate across these different languages.  Some races seem unable to produce, vocally, the sounds required for certain languages, but can still learn the language and understand it in listening.  In these cases translators or partnerships with those who can speak different languages is all the more important.
11)   What is the climate like?  What are some problems/benefits created by this climate?
-          The climate is a desert, and it is pretty harsh.  It doesn’t have any benefits, and we’ve already talked about a bunch of the problems.  Temperature swings are also problematic, as going out at night is often more dangerous than going out during the day.
12)   Why do people live there?  How long have people lived there?
-          People live here because people will live anywhere.  The solitary life outside the cities is more likely to attract those who enjoy independence or are looking to go unnoticed.  People have been here a while; no one talks about living anywhere else in the past, and the cities look old.
13)   What sort of power hierarchy or government exists?
-          Well, there’s the Empire, right?  This far out they’re somewhat hands off, and as long as you’re not causing trouble you’re likely to be ignored.  The emperor runs things from a far off planet, and with the recent dissolution of the senate local governors are tasked with keeping order.
14)   How do those in power get and/or keep power?
-          Power seems to be won through shows of force, and command of the military establishment.  Power is kept through fear of this same force.  Deadly force is authorized in almost all cases, so the best strategy seems to lay low and off their radar. 
15)   What does the military look like?  Is there mandatory service?
-          Military service involves putting on a suit of white armor and serving as the keepers of order - however those in power define order.  Service doesn’t appear to be mandatory, but must offer reasonable incentives.  As you work up the ranks you get out of the suit of armor and serve in more administrative roles.
16)   Are people generally happy with status quo?  Is there any unrest?  Open conflict?
-          It is a period of civil war.  Most people don’t really seem to care for the Empire, but the risk in acting out against them is too great and simply drives them to do nothing.  There is a small rebellion which has won small victories against the Empire, but these skirmishes tend to be small due to the small size of the rebellion. 
17)   What would it take to disturb the status quo?  Why?
-          This would probably vary from person to person, but for most people it would probably take a lot.  If it was easy to disturb the normal order, there would be more people in the rebellion.
18)   What’s the level of technology?  What technology shapes everyday life?  What technology is unique to this world?
-          Technology has produced artificial consciousness in the form of droids, and ships that can travel between planets are available but not cheap.  The technology from a daily standpoint is that which keeps water flowing, and it might also be fairly unique to this world due to the unique climate.  
19)   Is there any technology so advanced that it’s basically magic?  Is there just straight up magic?
-          Have you seen a lightsaber?  They’re not really a new technology, but I have no idea how they work.  Magic to me.  Oh, some people can also use their understanding of the world around them to heighten their reflexes and feel things before they happen or at a great distance. 

Pretty decent, it would seem.  We only see Tatooine for the first 50 minutes or so, and that’s also interspersed with scenes of the Death Star and Imperial Fleet.  In 50 minutes you can get a pretty great feel for what it would be like to live on Tatooine.  About the only question that I might have struggled with a bit would be the first one, as while you see people eat in A New Hope it’s pretty easy to forget what they’re eating.  I think it was salad, and that would also make a lot of sense in the context. 

The picture painted is pretty clear.  It kind of sucks to live on Tatooine.  It’s pretty easy to relate to Luke wishing he was somewhere else.

How about the primary planet of The Phantom Menace?  Oh, you’re not sure which planet I mean?  Well, not a good sign. 

Since the movie starts on, relates to the struggle of, and finishes with a battle on Naboo, the movie really does seem to be about Naboo.  We could certainly do the same exercise for Tatooine from an Episode I perspective, but I will leave that to the reader.

1)      What are some common things that people eat?
-          Uh, hmmm.  Fruit?
2)      What kind of living space does the average person live in?
-          Average person, eh?  The average person on Naboo.  And an example of an average person on Naboo would be…  They, they live in palaces.  Everyone.
3)      What are some common things that people do for a living?
-          Government.  Planetary government.  General administration.  Security.  That pretty much covers it.  Oh, pilots. 
4)      Are there many large cities?  What are the differences between city and non-city life?
-          Well, there’s one city of humans, and one city of Gungans.  The Gungan one is underwater on the opposite side of the planet from the human one.  I don’t think anyone lives outside of those two cities, though.
5)      What are some of the main products or exports?
-          Well, the Gungans produce these blue glowing balls that destroy machines with electricity, but they also live underwater and are fairly xenophobic, so I don’t think they export them.  The humans, uh, I’m not sure they engage in production or trade. 
6)      What doesn’t the world have, or what does it import?
-          It’s a pretty nice world, so there probably isn’t any shortage of the essentials of life.  They have some pretty nice architecture, but they also presumably mine what they need for that.  There are swamps and forests and plains and lakes and rivers and oceans.  The planet is teeming with life.  Maybe they import scum and villainy?
7)      What do people do for fun?
-          Yeah, fun.  Fun.  *awkward silence*
8)      What do people dream of or aspire to?
-          People.  And their dreams?  Things are pretty good, so who would really want more?
9)      Are there different cultures in close proximity?  How does this impact daily life?
-          Well, there are the humans and the Gungans, but they’re hardly in close proximity.  They also keep to themselves almost entirely, so no, there really aren’t any cultures in close proximity.
10)   What language do people speak?  Do different cultures/races/species have different languages, and if so how do they communicate across this gap?
-          Everyone speaks the same language on Naboo.  If you want to live on Naboo you learn the language or you go back to where you came from.
11)   What is the climate like?  What are some problems/benefits created by this climate?
-          It’s pretty nice.  No complaints.  Problems with the climate?  Well, sometimes it just seems a little too perfect.  Does that count?
12)   Why do people live there?  How long have people lived there?
-          Why would people not live here?  Am I right?  This place is great.  I bet people have been here since the minute they discovered it.  I guess the Gungans came around the same time?  All the buildings seem kind of new, but maybe they just have a rapidly refreshing architectural movement.
13)   What sort of power hierarchy or government exists?
-          Well, we have a queen.  She’s…14.  She has pretty much final word on the human side of things.  We also have a senator in the galactic senate, but he doesn’t visit very often.  I guess he’s a citizen of our world?  Hard to say if he was born here, who really checks on that sort of thing?  He certainly has a lot of holes in his past, though, to be fair.  But hey, what politician doesn’t?  The Gungans also have a series of Bosses who seem to be in control, but they don’t seem to have representation in the galactic senate.  There’s also a congress of the republic, but I don’t think we have representation beyond our one senator.  The supreme chancellor seems to be in control or at least supervision of the senate, but there is only one of him, and so many worlds.
14)   How do those in power get and/or keep power?
-          You mean the queen?  Well, in what I know of monarchy, I guess her parents were probably king and queen and then they died.  No one really asks that, you know?  Oh, you mean the senators?  Uh, well.  I guess maybe he was elected?  Or appointed by the queen?  To a term of…life?  Oh, you mean the chancellor?  Seems like the senators would probably vote him in.  And boy, keeping power in that job is near impossible – you can be voted out on a whim at any moment.  The others, I guess they just keep power because no one tries to take it from them.
15)   What does the military look like?  Is there mandatory service?
-          We are a peaceful people, so we don’t have a military.  We do have some blasters, but that’s just for self-defense.  We also have a few squadrons of space fighters with a design unique to our planet, and trained pilots to fly them, but it’s mostly just for show. 
16)   Are people generally happy with status quo?  Is there any unrest?  Open conflict?
-          People seem pretty happy with things as they are, though the Trade Federation seems to have some problem.  The only problems stem from them, but people don’t really seem too concerned.  There’s no open conflict yet, but we also don’t have a military at all, so yeah.  Oh yeah, the Gungans do, though.  They have a fully formed military and weapons and things like that.  They’re vaguely hostile to the humans, but living on the same planet as them we don’t really see them as any sort of threat that we’d put together any sort of military against.
17)   What would it take to disturb the status quo?  Why?
-          It would have to be pretty extreme, like rounding people up and putting them in camps for some reason.  You know, denying them access to their normal lives on this awesome planet.  Kind of hard to see a reason why anyone would do that, though.
18)   What’s the level of technology?  What technology shapes everyday life?  What technology is unique to this world?
-          Well, like I said we have blasters and spaceships.  We also have some droids, though they don’t really do much except work on our spaceships.  The Trade Federation has a whole army of droids, but they’re kind of stupid.  Also they need a ship in orbit to give them commands and stuff, or they just turn off.  We have a room that has a whole bunch of these big beams of energy from floor to ceiling, and the ceiling and floor are really far apart so it looks pretty cool.  Would you like to hear more about that?  Oh, well I guess that’s about all I actually know about that, sorry. 
19)   Is there any technology so advanced that it’s basically magic?  Is there just straight up magic?
-          We are a civilized people.  We do not believe in such charlatan’s tricks.  There were some Jedi here a while ago, but they didn't really do anything noticeable.

So, I didn’t think it was going to be quite so much of a contrast, but there you go.  There are some questions for Naboo that I just have no idea how to answer.  Some of them, the answers are so awkward that if you really put yourself in the shoes of someone in that situation you can just feel the panic.  Anyway, like I said, try this yourself.  Do Episode I Tatooine, or Hoth, or Coruscant, or Camino, or Cloud City, etc.

Wednesday, August 14, 2013

Kickstarter finish and first-year anniversary

Hi everyone - two great things to talk about today, though to be fair neither of them are particularly statistical.

First off, today at midnight the Quantifying Star Wars kickstarter project that I've been running for the past month finished up.  It was successfully funded and then some, and now I have some work to do in putting together the book.  I'm looking forward to it thanks to all of you, whether you backed the project or shared word of it with others or simply read and enjoyed the posts.

Since I'm going to be going back to those original posts to put things together in the upcoming weeks I might also recommend commenting on those posts if you have anything you think I should add or change, etc.  I'll certainly go back and read them before I get started on each chapter.  Here are the links so you don't even have to go back and find them yourselves:

Part 1: http://theskepticalstatistician.blogspot.com/2012/11/quantifying-star-wars-part-one-episodes.html
Part 2: http://theskepticalstatistician.blogspot.com/2012/11/quantifying-star-wars-part-21-episodes.html
Part 3: http://theskepticalstatistician.blogspot.com/2012/11/quantifying-star-wars-part-25-empires.html
Part 4: http://theskepticalstatistician.blogspot.com/2012/11/quantifying-star-wars-part-3-episodes.html

Secondly, it was almost one year ago today (it's a year ago on Thursday, actually) that I started this blog.  I had a vague idea of what I wanted to do but really had no idea if I'd still be doing it a year later.  I've had a lot of fun with it and come up some interesting findings that I wasn't expecting.

For instance, who would have thought that I was actually right that Reese's Pieces are screwing with our heads (http://theskepticalstatistician.blogspot.com/2012/09/the-reeses-rainbow-validating-childs.html) or that no one really seems to care if the NHL doesn't play huge chunks of their season (http://theskepticalstatistician.blogspot.com/2012/10/nhl-lockouts-do-they-matter.html) - there was a lockout this year?  This year?

Who would have thought that Halo 4 is totally rigged in a really specific and unimportant way (http://theskepticalstatistician.blogspot.com/2013/01/this-one-is-about-halo-4-but-also-about.html) or that it really sucks to be the first The Price is Right contestant to spin The Wheel (http://theskepticalstatistician.blogspot.com/2012/10/games-of-price-is-right-wheel-part-i.html , http://theskepticalstatistician.blogspot.com/2013/01/games-of-price-is-right-wheel-part-ii.html , http://theskepticalstatistician.blogspot.com/2013/04/games-of-price-is-right-wheel-part-iii.html), or anything but the fourth contestant to bid in Contestants' Row (http://theskepticalstatistician.blogspot.com/2013/06/contestants-row-position-is-everything.html).

Who would have thought that I could accidentally come up with a language made from Tetris pieces (http://theskepticalstatistician.blogspot.com/2013/07/tetris-pieces-exponential-growth-and.html)?

For every good piece I've written, though, there have certainly been other pieces that were just not as good, or downright reaching.  The fact that I updated with a post every Wednesday this year (even when I was pretty sick with what turned out to be the flu: http://theskepticalstatistician.blogspot.com/2012/12/on-fevered-temperature-measurements.html) meant that some weeks I was just writing on the best thing I could come up with that week.  Some of those posts could have used more polish, and some of those posts probably shouldn't have even made the cut.

I've been thinking about it for a while, and for a bit wondered if I should just call this a year long project and shut it down.  The interest and kindness I've received from you - the readers - has convinced me that I should keep with this, at least in some capacity.  Some of you will be getting the Quantifying Star Wars book in a few months, but I'd also like to keep tackling new problems and questions as they come to mind.

Particularly, there are a few half started posts I have that I've kept putting off because I could use more than a week rolling them around in the back of my head.  The short time frame I've been keeping has been great for quick easy posts, but has limited me from tackling some of the things that I think could actually be quite interesting.

That said, I want to move away from the 'updating every Wednesday' and instead commit myself to updating at least the first Wednesday of every month.  There will be some months where I come up with some quick stuff I can put together and you'll get some extra posts.  The posts you do get, though, should be better, and more substantial.

I know how I'd feel on the readers side of this.  I've followed plenty of webcomics and podcasts and all sorts of other periodic things that slowly went to reduced schedules, and there's a feeling of disappointment that comes along with it.

Don't think about it as losing a few good posts a month - think of it as losing all the bad posts each month to make the good posts better.  If I'm not writing good posts, call me out on it.  And - good or bad - comment on the posts a bit more, everyone.  =)

Wednesday, August 7, 2013

Wordfeud and you (and me)

I know that many of you play a lot of games on your phones and tablets and facebooks and GameBoys (I swear I know people who still use this as a blanket term for handheld electronics).  I don't play many - I tend to prefer a console or PC experience for my gaming - but do play a few. 

I tend to play DrawSomething at the pace of about a turn a month, and Wordfeud a little more frequently (the mobile app, not the facebook app, though I guess they work cross-platform). 

Wordfeud, for those of you who don't play it, is basically Scrabble.  By basically, I mean almost exactly.  The placement of double and triple word and letter squares on the board are a little different (and you can play a board where they're laid down randomly), but that's about it.  Makes for some subtle but interesting differences.

The cool thing is that a few months ago Wordfeud started recording stats within the app against all the other people you play against.  I generally have a few games going on at a time, so I figured at some point I'd take a look at the stats that I've accumulated. 

Now, there's not a whole lot I can tell from these stats on their own (we'll talk about the problems), but I thought it would be interesting to discuss exactly I'd need to be able to know more.

Basic stats are given about each of your opponents, in terms of win-tie-loss.  I'm not going to call out my friends individually, so we'll just list them anonymously.


Now, like I said, this doesn't tell us that much.  I could be playing against preschoolers, and I could be playing against Nobel Laureates (presumably winning their Nobel prizes in Scrabble).

We can get a range of the people I'm playing against by ranking them a bit in terms of difficulty (or at least the difficulty I seem to be having with them).  We'll do this with simple win percentage (FYI, we'll treat that single tie as not a win or a loss, but simply as not a game. 

That gives us another table.

Win Percent

Which we can change into a rudimentary chart.

You can see that there are a few problems with this chart.  The x-axis is both arbitrarily scaled and ranked.  It's done with the best information I have available at the moment.  Ideally, I'd have some stats on the people I'm playing against in terms of their overall win percent.  This would allow me to rank them in terms of all the games they've played, not just all the games that they've played against me. 

It would also allow for them to be scaled to that win percentage, rather than simply all spaced the same distance apart.  There are two pairs of individuals who I have the same record with.  They're certainly not identical in skill, so more information would help to differentiate their actual ability. 

What we can tell from the graph is that I'm not playing a bunch of people who are all better or all worse than me, but a good mix of people across a good range.  That, or the results of games are just random noise...

Overall, more data is the best way to tell.  Interested in seeing how you measure up?  Play me a game (or 20) - my username is 'paul_28'.  Don't miss the underscore, unless you want to just play against some random guy (which I might actually be to some of you).  I will mine your data.

Wednesday, July 31, 2013

Some Data Gathering Resources

Hi everyone - today I wanted to put together a fairly quick post about some of the resources I've found in that past year that I've found interesting (and occasionally useful) in putting together some of the posts on this blog.  I've also found a lot of resources that I haven't fully utilized (yet), but figured it might be useful to share.  Anyway, here you go:

Reddit Insight - "We downloaded the Reddit"


Who doesn't love Reddit when you're looking for something to kill a few minutes/hours/days?  If you're bored of Reddit, though, you can use this site to kill time while looking at data generated by and about Reddit.  Meta time killing, if you will.  There's some cool tools, and who doesn't like word clouds?

Aww subreddit word cloud

Wikipedia Statistics (Overall) - ""WP:ST" redirects here."


There is a lot of information on Wikipedia, and there is also a lot of information about that information on this page.  I don't know where to begin - this post could just be about this page.  How about the top 25 Wikipedia pages from last week?

Wikipedia Special Pages - "This page contains a list of special pages."


So you think that the last page had a lot of information about Wikipedia, and there's probably not much more that's really interesting enough to talk about?  Well, welcome to the sub-basement of Wikipedia, where people get together to generate lists of all sorts of things, like Long Pages, Orphaned Pages, and perhaps my favorite list on the internet, that of Uncategorized Categories.

Wikipedia Random Page - "Do you feel lucky, punk?"


This doesn't generate any statistics on its own, but I think it's an interesting page nonetheless.  You can certainly run some calculations based on the numbers from other Wikipedia pages, like what your odds are of finding the page you're looking for on Wikipedia by simply clicking that link (it's about 1 in 30 million).

So if you're feeling like learning something, give some Wiki Roulette a try.  

Who knows, someday it might be important that you know something about David Alton, Baron Alton of Liverpool.

Wikipedia Pageview Stats - "How about you tell me in graph form?"


I swear that a few years ago Wikipedia had some tools built into their own site to look at stats from specific pages, but recently this page has been all that I could find.  It's great if you're looking for patterns in data, like if people are more likely to look at articles about particular days of the week on those days of the week.


Google Trends - "Two trends enter, one trend leaves"


Google Trends seems to have two main things going on.  The first is the stuff on the main page, which is letting you know what's trending on Google.  Personally, that's really quite boring.  The fun part of Google Trends is pitting two (or more?) topics against each other to see how search volume has compared over some space of time.  For instance:


Produces a great graph of search volumes of "red" vs "blue":

Yes, I know (and you should too - that's what legends are for!) that red is the blue line and blue is the red line.  It's the order they are put in that matters, and I find the prospect of them switched to be amusing for some reason.  So, I'm keeping it.

That said, looks like red started winning sometime around 2008.  Some of those letters on the graph might help you pull out why that is, as it links time frames to news stories and the like.  It also shows searches that contained these words, etc.  It's a fun tool, especially for those of us who always wondered why people thought it was so hard to compare apples to oranges.

Google Correlate - "Correlating your Googles"


This one is a bit newer to me, but has some cool applications.  It lets you see what words are searched for together, or rather which search terms are correlated to any given search term.

For instance, a search for "turkey" reveals that people are very often searching for "turkey stuffing".

You can also export these results, and do some other stuff with it, I guess?  Like I said, this one is relatively new to me, so I'm in the process of thinking of ways to try to use it.

Professional Football (NFL) Stats (1940 to present) - "All the games, and then some"


If you've been reading the blog for a while you recognize this site, as it's the one that I used to look at historic scores in the first week of the season.  It has all the outcomes from every NFL game ever played, and some stats from games even before that.  There's...a lot of information.

Professional Baseball (MLB) Stats (1916 to present) - "Like football, but baseball"


While I've never used it, you can also find stats on all professional baseball games played in the last century or so.  Again, it's a lot of information.

Professional Hockey (NHL) Stats (1987? to present) - "We taped over all the early games"


Finally, there's also a professional hockey version of the last two pages, though for some reason it only extends back to 1987.  Maybe they're working on it?  Hard to say.

Twitter Stats - "A constant, never ending stream of information"

I actually had a decent amount of trouble finding any official stats on the Twitter page, as it looks like almost all of the stats generating tools are third-party.  Maybe that's not the case, and I'm looking in the wrong place.  If I was going to list one third-party Twitter tool I'd rather list a ton, so maybe that will just wait until a future post.

Wednesday, July 24, 2013

More on the importance of exponential growth OR that part in Wayne's World

Yes, I'm talking about this part of Wayne's World:

It seems that over the past few posts I've touched on exponential growth from a few different directions.  One of those ways was relating to the proliferation of unique Tetris pieces you can make with a set number of 1x1 Tetris blocks, and the other two were touching on the Wayne's World social network method from above.

Those two posts were the one about my Kickstarter project and the one about saving the post office by creating a national culture of everyone removing and returning business reply mail envelopes from junk mail.

Let's get the obligatory Kickstarter plug out of the way.  The reason that Kickstarter works (when it works) is due to the nature of social networks.  If I just wanted to collect money from the people I knew, I'd simple ask every person I knew if I could borrow a dollar the next time I see them.

It's an interesting social experiment, and perhaps one I'll try sometime, but it's not the point.  The point isn't for people I know to give me money (though thanks if you have!), but for them to tell the people they know.
You see, I might be able to say that I 'know' a few hundred people.  People who if I saw them sitting at a bar in an airport while traveling I would sit down and strike up a conversation with (my favorite test of if you actually 'know' a person).  I don't know that I'm too much of an outlier on that - keep in mind I've said 'know' and not 'friends', which is a whole different story.

If each of those people gave me a dollar (from the above example), I'd have a few hundred dollars.  But if those people instead just told their friends about me, and I got a dollar each from them, well, that's a lot more dollars.

How many more dollars?

Well, let's just make this easy.  Let's say I have easy communication access (can I put that in any colder terms?) with 100 people.  Let's also say that they also have 100 people with whom they share the same access, but those people are 100 different people (I guess I'm in there, too, so maybe they need to have 101 people).


If I did somehow find myself in a situation where I knew 100 people who each knew 100 non-overlapping people, that second set of known people is exponentially larger than the first.  Why?  Because there are exponents involved.

Joking aside, exponential growth occurs when the growth in a mathematical function is a product of the current value of the function.  In ideal case, when y = n^x, and where n is some number.  (Yes, I also know that the exponential function - not just exponential growth involves the use of e^x, but that's outside this discussion.)

What I'm getting at is that the number of people in this secondary network is 100*100, or 100 squared (100^2).  100^2 is 10,000.

That's not the function, that's simply one step along the way.  There is a function is how the number of people in the primary network (the people I know) are related to the number of people in the secondary network (the people the people I know know).  That function is a simple square: y = x^2.  This still isn't where exponential growth comes into play, but it's worth discussing first.  A square function (in fact, this square function) looks like this:

If the number of people that people know is 100, then we get the 10,000 above.  If each person knows 10,000 people, then the secondary network is out at 100 million.  If each person knows 3 people, then the secondary network is only 9 people.

If you happen to have friends who also like telling people things (and also happen to miraculously have a completely unique set of friends aside from you), we would move out to a cubic function: y = x^3.  Now, the number of people that can be reached if everyone has 100 people to talk to is 100*100*100 = 1,000,000

That's right, one MILLION people.

By increasing from a secondary to a tertiary network, we're incrementing the value of the power in the function.  It is though this that exponential growth occurs.  The Wayne's World growth is every person telling two friends, so the function y = 2^x shows how many people you are contacting at that stage of the process (e.g. x = 3 is three steps steps removed from the initial person).  That looks like this:

Don't be fooled by the scale into thinking that those numbers below 75 on the x-axis are zero.  They're just really small compared to the end number, but they're still really big.  For instance, 2^25 is still 33 and a half million.  The fact that 33,500,000 looks like zero might give you some perspective on just how big those numbers toward the right end of the graph actually are.

The graph of what we were talking about above, with every person telling a hundred friends is somewhat similar, except all the numbers have a lot more zeros on them.  In fact, since we're working with a nice power of 10 system all the numbers can simply be expressed very easily in scientific notation.  So much so that a table is perhaps more illustrative than a graph.

So, if I told 100 friends about something, and then each of them told 100 (different) friends, and so on, and so on, we'd run out of humans on the planet sometime between the 4th and 5th step.  The trick would really be finding those 100 unique people each time.

You might also note that this is how pyramid schemes work, and why they are always (eventually) unsustainable.  To keep the scheme going you need to keep finding unique people to enter into it.  The longer it goes on the more and more unique people you have to find.

For instance, here's the table for the Wayne's World 2^x method:

So, even if you're just having each person in the system tell two other people, you still run out of people on the planet in about 33 steps through that system.  Of course, an actual pyramid scheme is a bit more complex than this, but this would illustrate one running at peak efficiency.

Let's step back from pyramid schemes for a moment.

Think of it this way.  If two of you each told two friends about the post office plan from last week, and those two people told two people, etc, we'd have the whole US told (again at peak efficiency) at just under 30 or so steps.

Back to pyramid schemes though (I'm kidding), we don't need the whole of the country on the kickstarter - this process at 10 steps still has over a thousand page views.

So, you know, do both those things.

Tetris pieces are growing in perhaps a much more interesting way, that I'm only going to touch on briefly (until I decide to do a post that looks at those 6 and 7 block cases).  I talked about it during that post, but every time you add a block to the system you can place that block on a number of spots on preexisting pieces.  Early in that process you a) have fewer pieces and b) those pieces are smaller.  The growth that occurs at that stage is slow, then.

As you start to get more pieces, and those pieces get bigger, there are both more spots on any given piece to put a new block as well as more pieces on which to do so, which drives this accelerating growth.  Some of these aren't unique, but it's possible that the proportion of non-unique pieces produced at each step has a predictable function as well.

Something to look at later.

Or perhaps something to dream about...