Melodifestivalen: Swedish singing competitions and those silly Brits

I'm suspecting that only a small fraction of my readers happen to be up on the most recent developments in Swedish-based musical competition programs as preliminary selection to the Eurovision song contest.

Well, that's a thing.  And it's actually quite a bit more interesting than any American musical competition show that I've ever seen.  So there's that.

Anyway, after initial selection, eight musical acts perform each week for four weeks.  Based on Swedish call-in vote, two of the competitors from each week go to the final.  In addition, two from each week go to a second chance round.  During this second chance round eight competitors are first cut down to four, who then face off in pairs until the two remaining go to the final.

The final thus consists of 10 competitors, and voting is a mix of international juries and Swedish call-in vote. 

It turns out some really interesting acts, like this:

and this:

and this:

and this:

Overall, though, it also puts a whole bunch of acts through multiple rounds of voting from multiple sources.  This produces some pretty interesting data, and gives us some interesting statistical options.

I hate to keep saying it, but once again Wikipedia provides (collects!) some pretty great data on how everything went down.  The results of all four rounds, the andra chansen round, and the final can be found here:

http://en.wikipedia.org/wiki/Melodifestivalen_2013

There are six big picture instances where the Swedish people were able to call in and express their vote.  These six times are the four main rounds, the second chance round, and part of the final.  I mentioned that part of the final was points from international juries.  In fact, there are 11 international juries: Cyprus, Spain, Italy, Iceland, Malta, the Ukraine, Israel, France, the UK, Croatia, and Germany.

Because only a certain portion of contestants move on to the final (and because there is no distinction before the final between first and second, or between third and forth) there's actually not much variance in scoring from those first few rounds.  The most useful information comes from the final - 11 international juries and the Swedish people rate each of the 10 final contestants.

What can we learn from this?  Well, what I started thinking about was finding which countries scored the contestants in similar ways.  After a little more thinking I got to wondering if we could treat each country as a item by which each contestant is measured.  In such a case we could examine how well each country was measuring the same thing through a reliability analysis.

Now, I recognize that this is not without pitfalls - the fact that this is rank data means first that cases aren't independent.  It also means that reliability based on continuous measurements might not be the most applicable, but I'm going to look the other way for now.

A straight-up reliability analysis of the 11 country votes and the final Swedish phone vote gives us some numbers that are almost respectable - including a Cronbach's alpha of .695.  If this is a means of gathering consensus, though, which countries are simply coming out of left field?

We can take a look at the inter-item and item-total correlations, and what we find is that the UK seems to be the country acting the most strange (this is also backed up by individual item-total(ish) Spearman's rho rank order correlation for the UK).  The inter-item total correlation for the UK is actually negative (-.11), implying that they are measuring something reasonably different than the rest of the countries.

So what happens if we pretend that the UK forgot to show up last weekend?

First off, our Cronbach's alpha jumps up to .734.

Here are the rankings and point totals as they played out in reality:

Competitor	Final         Points	Rank
Robin Stjernberg	166	1
Yohio	133	2
Ulrik Munther	126	3
Anton Ewald	108	4
Louise Hoffsten	85	5
Sean Banan	78	6
Ralf Gyllenhammar	73	7
David Lindgren	69	8
State of Drama	68	9
Ravaillacz	40	10

And here are the rankings and point totals as they would have played out without the UK:

Competitor	Final     Points	Rank
Robin Stjernberg	166	1
Yohio	131	2
Ulrik Munther	114	3
Anton Ewald	104	4
Louise Hoffsten	85	5
Sean Banan	68	7
Ralf Gyllenhammar	65	8
David Lindgren	69	6
State of Drama	62	9
Ravaillacz	39	10

You can see that not much has changed, as the only change seems to be David Lindgren jumping up two places above Sean Banan (boooooooo).

Cyprus also had a negative item-total correlation (albeit smaller), which is still has after the removal of UK.  Removing Cyprus in addition to the UK bumps our Cronbach's alpha up to .775.  It also continues to move Sean Banan down the ranks:

Competitor	Final         Points	Rank
Robin Stjernberg	165	1
Yohio	121	2
Ulrik Munther	108	3
Anton Ewald	96	4
Louise Hoffsten	83	5
Sean Banan	56	9
Ralf Gyllenhammar	65	6
David Lindgren	65	6
State of Drama	62	8
Ravaillacz	39	10

We could keep at it, but given the pretty sizable gap between first and second place it seems like the removal of particular countries isn't going to swing things much in any substantial way (the rest of the countries are also considerably more consistent).  It also pains me to begin to question if Sean Banan doing even as well as he did was simply due to noise in the data.

However painful, let's take a look.

We've been looking so far at the consistency of any given country, but we can also take a look at how stable the individual competitors were in terms of rank.  How to do this?  Well, let's see if means and standard deviations can help to paint a picture.

The idea here would be that if countries can't agree on how to rank a contestant then that contestant should have a higher standard deviation (error) around their mean rank.  Hand-waving again around some of the ceiling and floor suppression effects on SD from this kind of scale, here you go:

Competitor	Mean rank	SD rank
Robin Stjernberg	2.916666667	2.274696
Yohio	5.583333333	2.314316
Ulrik Munther	3.5	2.153222
Anton Ewald	4.916666667	2.466441
Louise Hoffsten	5.666666667	2.348436
Sean Banan	5.833333333	2.552479
Ralf Gyllenhammar	6.166666667	2.480225
David Lindgren	5.083333333	2.84312
State of Drama	5.25	2.261335
Ravaillacz	7.333333333	0.887625

Interestingly, the only competitor that really stands out is Ravaillacz (almost universally ranked toward the bottom of every country's list).  David Lindgren is also a bit high, though Sean Banan doesn't seem to be standing out as I expected.  Most ranks stay pretty consistent.

[You also may notice that Yohio drops quite a bit if we just look at mean ranks.  It is because he did really well with the Swedish phone vote, which is weighted higher than any of the other individual countries.]

Overall, it seems that Robin Stjernberg was pretty safe in his win, though perhaps in the future the British shouldn't get to vote on anything relating to music.  At least if that music is crazy Swedish music.

Sing us out, Mr. Stjernberg!