EU DataViz 2019 – EU elections: the case for harmonized election data

On 12.11.2019 I participated in first EU DataViz conference held in Luxembourg City. I had a presentation entitled “EU elections: the case for a harmonized treatment of European election data” in the first set of thematic sessions

Karim Douieb - EUDataviz2019 Photo.jpg
Photo by Karim Douïeb

The presentation was filmed and can be found HERE (at 2h:51m:48s)

The slides are found at this link.

I took the liberty of publishing the presentation in the form of an article below. Enjoy!


page-0

I’m here to share with you what I’ve learned creating the most detailed maps of the European Elections ever.

Part I – Each election has its map

Today, when elections happen, news sites use maps to easily illustrate the result and patterns in the vote.

page-3

So (for example) when elections happen in the United States we see a lot of maps published in various news sites such as the New York Times, the Washington Post, including non-American ones, like the Guardian. They vary in detail, aesthetic, subject matter – presidential elections, congressional, presidential primaries – , and approach the vote from various angles.

page-4

We also see detailed maps when other big democracies vote, like India for example (on the left), the world’s single largest elections, with over 1 billion voters. You see beautiful detailed maps of Brazil as well (here on the right) made by Gazeta do Povo for the last presidential elections they had.

page-5

But when the European Union holds elections – second largest elections in the world, with more than 400 million eligible voters – we often get maps like these. These 1-county-1-color maps simplify the vote to the extreme. You do get to see the whole union, but with virtually no detail.

page-6

If not we see this alternative: complex maps that focus on the reader’s member state. So El Diario does a detailed cartography of the Spanish vote, without showing other countries. And Le Monde shows only how the French voted, and not the rest of the Union.

There is something missing. A map that merges the best of both: a pan-EU look but on a detailed level.

page-8

These are the only two maps of these kind.

The map on the left was created by me, as a personal project, for the 2014 elections. It took 6 months of on-and-off work. The one on the right was a collaborative work with Julius Tröger and Zeit Online, in June 2019 right after this year’s European elections. It is the first time an online newspaper published such a map.

Here it is on full screen.

page-9

(Apologies for cutting off Finland and Sweden). Almost 80.000 administrative units, mostly municipalities. A historic first.

Part II – Why are there so few maps like this?

So why did it take until now for someone to make these maps? Well the problem is that election data is organized by national authorities. There is no harmonised data to an EU format, done either by national or EU authorities. What this means is that the national news sites will process only familiar data (i.e. data formats they are already familiar with, from national elections).

And I encountered this a lot while working on my maps.

page-11

For example you would think member states publish data on the website of Election Authorities. Some do, some don’t. Sometimes it’s published on a open data portal, and sometimes it’s on a private site.

page-12

You would think that the data is made available for download. Yet sometimes it isn’t. And to get all the data I had to write small programs to scrape those websites page by page.

page-13

Which becomes harder the more complicated the site is. Simple HTML sites are easy, but some of these websites have complicated JavasScript interfaces that makes them pretty to look at, but also makes getting the data a headache.

page-14

Most of the times if you manage to download a file, it’s a simple Excel file of CSV. But in the case of the Netherlands it was a more complicated XML type file (which most people might not be very familiar with).

The data also changes based on the type of elections.

page-15

Most of Europe votes on list systems –each with its own twist: for example Luxembourg is somewhat special with 6 votes per person -, but some states or regions (like Malta, Ireland and Northern Ireland) use Single Transferable Vote, where candidates are ranked. This means no detailed data on the municipality level, since data is first centralised and then counted.

Voting abroad is another layer of complexity.

page-16

Sometimes votes abroad get allocated to where the voter is from, if they vote by mail, and sometimes it’s counted in a category apart. While in some countries embassy votes get added to the votes in the national capital, skewing the data there.

page-17

And even if the data is available to download, and even if the file is a clean Excel file, it’s not sure the data is geo-referenced with the relevant municipality code. Municipality codes (called Local Administrative Unit code by Eurostat) helps with attaching the data to the map’s shapefile.

If only the municipality name is available, one can run into a lot of problems. Especially so if there are spelling variations (like accents) or language variations when a municipality is in a bilingual area and it has the national name in one dataset, and the regional name in another.

Thus it is always better that these datasets have the relevant municipality codes.

So while doing this work, mapping these elections, I thought a lot about how difficult and time-consuming it is to visualize the European vote (and it shouldn’t be more difficult than the US or Indian elections.). So here is what I think could be done about it, because something should be.

Part III – Three ways forward

1) First possibility is a top-down approach: Hard policy of mandating that a version of national data follows a pre-determined format. Anyone who is familiar with the format can interpret it.

Advantage is that every country does things the same way, but since we are talking about 28 states that need to agree on the format, it might take a long time.

2) Second is what I call “The Bridge”: Nations publish their own versions of the data like before, and an EU authority (maybe Eurostat?) publishes a harmonized version, thus bridging the space between the member states and third parties.

Advantage is that harmonizing happens centrally but Eurostat must keep up with each state’s changes and shenanigans.

3) The third one would be Eurostat publishing just a “nation by nation guide” for third parties where it details how to access and harmonize data.

This is obviously the most cost-effective solution, and it requires a minimal intervention from the European Union. But on the other hand, it is others who will be harmonizing the data, so it could very well lead to inconsistencies.

But one solution needs to be implemented as there is need for this data. National and European institutions have already reached out to us.

page-22

By voting together, we are a single political space, although we do not see ourselves as such, partly because we are not represented that way in the media.

This is where dataviz comes in.

If we have harmonized EU data readily available, we will use this data, which means more visualisations in the media, and we will see ourselves represented more as a European Comunity.

12 November 2019 – Luxembourg

Some tweets from the event

 

30 Day Map Challenge (2019)

Day 1 – Points – A dot density map of the 2019 European vote

1 Dot density

Day 2 – Lines – An experimental radial barchart? map of the 2019 European vote

Radial Spiky

Day 4 – Hexagons – Published the second hexagon map from:

What if: the EU had presidential elections like the USA?

Day 15 – Names – Men vs. Women in the street names of Oradea

M-F Out.png

Day 17 – Zones – The many faces of EU-related maritime Exclusive Economic Zones

EEZ-30Days.png

Inspired by a tweet from @hireuter on EEZs.

Day 19 – Urban – The many architectural styles in the city center of Oradea

Stil Cladiri 30Day

Day 22 – Urban – Built Environment of Oradea (1609- 2017) – .gif based on old maps

Oradea_Evol_Loop

Day 23 – Statistics – Geographic centers of the European Vote

I calculated the weighted mean point of:

  • All valid votes
  • Each political group in the
  • All votes that aren’t represented in Parliament

Lat Long Center Vote

To be updated…

Etymology maps

This article is a repository of all the etymology maps I’ve ever done. Most of these maps were made between 2014-2019 and have originally been posted to Reddit, often to the /r/etymologymaps subreddit.

A few word of warning:

  1. The base map used to make these maps used to have errors. For example, the Russian-speaking areas of Latvia are wrongly to the north-east instead of south-east and way too big.
  2. The main dividing line for these maps is linguistic (read: “mother tongue”), not administrative or political.
  3. Minority languages are often given proeminence. It was important to show as many languages as I had data for.
  4. I know German nouns are capitalized. I took a design decision to not capitalize them because I didn’t like having some nouns capitalized, and some in minuscules.

Some maps already featured on this blog:

So without further ado:

I. Countries and peoples

Hungary
🇭🇺 Hungary
Poland
🇵🇱 Poland
Greece
🇬🇷 Greece

Below: 🇦🇹 Austria & 🇷🇺 Russia

1486px-Germany_Name_European_Languages.svg
🇩🇪 Germany (see legend below)

Screenshot_20190814-192847__01

The map above had originally been uploaded to Wikimedia Commons for the Wikipedia article “Names of Germany

Morocco
🇲🇦 Morocco
Jew
🕎 Jews

II. Food and drinks

Beer
🍺 Beer
Tea
🍃 Tea
Sausage
Sausage 🌭
Cucumber
🥒 Cucumber
Pineapple
🍍 Pineapple
Apple
🍊 Orange
Peach
🍑 Peach

III. Other words

Rose
🌹 Rose
Bear
🐻 Bear
Paper
📃 Paper
Architect
🏛️ Architect
Church
💒 Church

IV. One final map on currency etymologies, in a somewhat different style:

MoneyName

 

Bubble Grid Map – 2019 European votes

How do you show patterns across 28 countries without your mind immediately jumping to group the patterns into national areas? While working on mapping the 2014 and 2019 European Elections, I realized that even though I wanted to show a single European voting area, voting patterns are still very national, and thus patterns sometimes change abruptly at the national border.

The maps below, inspired by the work of Maarten Lambrechts, on the “Herds of Europe“, are an experiment. They to try to go against our tendency to wrap our gaze around familiar administrative border.

The maps show the number of votes for each political group in the European Parliament as well as the votes for parties that didn’t make the electoral threshold (i.e. parties outside the European Parliament) . So bigger the bubble, the higher the number of votes that group received in that zone.

The only anomaly of the map is the fact that Ireland’s data is already aggregated on electoral constituency level, which are bigger than the grid I chose, thus the bubbles will always tend to be rather large and far apart (all the other nations have much more detailed, municipality-level data).

Unlike Maarten, who worked in R, my maps were made in QGIS (+a bit of Inkscape).

For the original R code, see also Jonas Schöley’s page.

EPP
SD
RE
G-EFA
ECR
GUE-NGL
ID
NI
NoN

 

European Elections 2014 – Results by Group

This is a long overdue blogpost with maps of the 2014 European Parliament Elections, which transcribes the info I had posted a few months ago on this Twitter thread.

For the 2019 Elections see this article in the German publications Die Zeit.

Gagnant-G-v1

Center-right EPP group did best in Hungarian-minority areas ofRomania, Slovakia and German-speaking South Tyrol in Italy.

I does generally well in Hungary, Germany (especially Bavaria), Slovenia and Croatia, Latvia, Poland (especially Silesia) and northern Iberia.

Weak in France and Italy.

EPP-G-v1

Center-left Socialists and Democrats group had a good score in Romania (especially the south and east of the country), in Sweden, Slovakia as well as Bulgaria & southern parts of the Iberian peninsula.

Industrial areas of the UK, western Germany, Wallonia (in Belgium) and Central Italy also high, as well as the Overseas voting areas.

It’s the only group present in all Member States.

SD-G-v1

The Conservative and Reformist group have a much smaller footprint among the Member states, but high numbers in the UK and Poland, as well as a good showing in Flanders 🇧🇪, Denmark and the Dutch Bible Belt.

Polish minority vote in Lithuania is also a member of the conservatives.

ECR-G-v1

There are some notable areas with a Liberal presence, especially Estonia, Finland and the Benelux countries, and a pattern of regionalist (Spain, Croatia) and minority parties (Turks and Pomaks in Bulgaria, Swedish-speakers in Finland) in this group. But big states missing from ALDE.

ALDE-G-v1

The far left United European Left-Nordic Green Left is strong in rural Finland and Sweden, southern Portugal, the Basque country and the Greek-speaking member states. Good presence overseas as well. It is absent in the East due to toxic legacy, with very notable exception of East Germany and Czechia. This group also fares well oversees.

GUE-NGL-G-v1

Within Greens-EFA group, the Regionalists are strong in Catalonia, Wales and Scotland. On the other hand, the Greens are strongest in lush French Guyana, some Polynesian islands, the cities of north-west Europe as well as regions with a history of environmental activism (like the Hanoverian Wendland). The group is notably absent in the southern and eastern parts of the union.

G_EFA-G-v1

The backbone of the populist EFDD group is UKIP (strongest in England and Wales) and the 5 Star Movement in Italy. Further backing in Sweden and the Baltic-speaking states. Only present in 6 countries but highly concentrated in most of those.

EFDD-G-v1

The Non-Inscrits were made up of mostly far-right parties (Front National, Lega Nord, DUP, Golden Dawn, Jobbik) who went on to form a group in 2015. The Greek Communist Party, oddly enough, was also part of this non-group.

NI-G-v1

And finally a reminder that turnout varied a lot by Member State in 2014.

Turnout-v1.png

Maps made in QGIS + some Python work + post-production in Inkscape.

European Elections 2014 – results by Europarty

First thing’s first, what is a European Political Party (or a Europarty)? According to Wiki:

A European political party (formally, a political party at European level; informally a Europarty) is a type of political party organisation operating transnationally in Europe and in the institutions of the European Union. They are regulated and funded by the European Union and are usually made up of national parties, not individuals. Europarties […] express themselves within the European Parliament by their affiliated political groups and their MEPs.

Now they are important because they also field lead candidates (called “spitzencandidates”), who, according to a decision by the European Parliament and the precedent set in 2014, are the only ones who the EP will accept as a candidate for President of the European Commission.

This series of maps are born from the annoying habit of the mass media of treating the European Elections as national affairs, and only focusing either on patterns within nations, or on overall national results when looking at the entire European Union.

Following the variety of examples of “county level results” we see when the US holds elections, I decided to to create a similar map of Europe (and all its overseas territories where elections are held).

This being the EU, where “varietate” comes before “concordia”, there was an immense variety of problems when trying to have a uniform vision. Each country holds elections its own way (the whole restriction is that it has to be ‘proportional’), stores the data in its own way and is divided into administrative divisions differently. Not to mention the immense variety of national and regional parties, hence the need to group them by European-level affiliation.

So the baseline was trying to map the results by Europarties on the municipal level, which depending on the country are either LAU1 or LAU2 level divisions (see here), but in some countries that wasn’t possible. Ireland, Northern Ireland, Malta and Slovenia could only be mapped by electoral constituencies (which in the case of Malta is the whole country).

So without further ado, here are the maps, by European Political Parties:

EPP-v1-1

PES-v1-1.png

ALDE-v1-2

EDP-v1-2

EGP-v1-1EFA-v1-1

PEL-v1-1

ACR-v1-1ECPM-v1-1

NI-v1-2

Later Edit – Two more maps: the winning party and share won by the coalition backing the current European CommissionGagnant-v1
Coalitions-v1
Maps made in QGIS, with lots of python and pandas in the data-wrangling phase, including xml parsing (bleah!). A good dose of post-processing in Inkscape.

On a personal note I’d love those meddling bureaucrats in Brussels to lay out a minimum standard of collecting, storing and making available election data on a local level, as I would love to see a bit more detail in Malta and the island of Ireland.

Also, Cyprus, answer your emails!

PS. Other articles will follow,  cause I have lots of data 😀

European Elections: which colors for which party?

CompositeSources

Inspired by Lisa Charlotte Rost‘s article titled Election reporting: Which color for which party?, I decided to take a look at color assignments for ed to European Parliament Groups as they are featured in the press. While European elections get less coverage than the US elections or national-level elections in Europe, they increasing in importance from one election cycle to the other.

Before actually talking about how these groups are represented, it is good to understand how European elections work:

Every 5 years, the member states of the European Union hold elections to send a number of representatives to the European Parliament. Since these MEPs belong to different national political parties, they band together in groups that are similar ideologically, as doing so helps them to better coordinate politically, as well as it being useful in getting subsidies and places in various committees of the EP.

The Colors

So we’re going to look at the groups as they existed right around the 2014 elections, since we can get a wide range of presentations from multiple European media sources at roughly the same time. In keeping with Lisa’s article, we can see below the color assignments of these Groups both by the media source (on the right side) and on a color wheel, clustered by group (on the left):

Composite Out
Click on the image for a larger version

What we see first and foremost is that there is surprising coherence in the color choices, even though there the publications span multiple languages, and even includes an example from a site in far away Japan.

This coherence, I suspect,  might be the result of two factors:

  1. A convergence in time, as some earlier examples seem to suggest a lot less coherence in color choice (like the 1999 example from the Economist at the end).
  2. A sort of normative power of official charts put out by the European Parliament website, for example the first two lines in the chart (a lot of online publications just chose to use those)

National ‘flavours’

One interesting aspect is that sometimes color choices seem to be influenced by national conventions with regards to the parties that make up those groups. For example:

• SD is pink in some French sites, because PS, the French socialist party within the group, is often shown as pink on the national level
• EPP is black in one German example (CDU, the biggest EPP member from

Germany is traditionally shown as black, as is the Austrian ÖVP)
• EFDD is purple in 2 British examples, even though the group logo is turquoise. That’s because UKIP, the main party of the group is associated in the UK with the color purple

NationalColors

The fixed and the still evolving

When it comes to the groups themselves, we can see that some groups have clearly occupied certain colors. The liberal ALDE is always yellow (a color traditionally associated with liberalism) and the Greens-EFA group is, well, green.

The left is also pretty consistent in its use of red, as well as the distribution of light – for center-left SD – versus dark red – for the far-left GUE/NGL, thus marking ideological intensity.

The same cannot be said of the right side of the spectrum, where we sometimes have 3 groups using hues of blue, albeit the 3rd group, the EFDD, is shown pretty consistently in turquoise, and if not, some other, non-blue hue, like purple, orange (?) or brown. Thus we shall focus on the other two groups: the center-right EPP and the conservative ECR.

Here, the meaning of the intensity of the color is not yet settled, so we have two competing situations:

• EPP light blue / ECR dark : thus intensity showing degree of ideological intensity, similar to what we saw on the left side of the spectrum. Or…
• EPP dark blue / ECR light : the more frequent case, where intensity correlates with importance of the group, or to put it differently, with its electoral weight, the EPP being easily the larger of the two.

And last, but not least, the “Non-Inscrit” group (NI), which is just a fancy way of saying ‘the group of those outside groups’ are in shades of grey, a color often used in charts to mark the ‘Others’ category, although the occasional use of brown could be explained by the presence of many far-right parties in this group (hinting at “brown-shirts” I presume).

This article is part of a study for a later cartographic project.

Colorwheels done in python/matplotlib, the rest in Inkscape.

The EU is quirky, but so are most European states

EU-Political Systems

The chart above is born out of the feedback that an earlier chart received on-line. I had made a Functional Chart of the EU, in which I tried to “bring home” the political setup of the European Union, by showing where each institution gets its mandate from, and what each institution would be called if instead of being a supra-national entity it was a nation-state. To this end I emphasized that, for example, the Council of the EU (i.e. ‘of Ministers’) is actually the “Upper House” of the EU Legislature.

What sometimes happened was that the complexity of the system was highlighted as proof that the EU is “difficult to understand” for the average Joe, contrary to the straightforward way national political systems function. Another angle of criticism was using some features of national systems as a sine qua non of a democratic system. To put it differently, the argument went: “My nation does X, and is democratic. If the EU does not have X, it is therefore undemocratic”, where X is a feature that is not actually present in all European democracies.

These two lines of criticism seemed to suggest that people don’t always realize how complex national political systems are and how diverse European nations are when it comes how they have evolved as political systems. I myself have learned that Romania’s way of confirming a government by a vote of both houses of Parliament in joint session is not, in fact, that common in our Union.

But there are wonderful details that I found throughout the Single Market states:

  • there is no explicit vote of confidence for the Danish PM
  • the Swedish PM is nominated by the Speaker of the Parliament, not by the King
  • Switzerland has a Collective Presidency
  • Cyprus has no PM
  • Sometimes ministers are nominated by the Head of State after consultations with the PM
  • The bigger European states tend to be complicated
  • Upper Houses are often very… original beats

So while the EU does have its quirks, there is plenty of diversity on the national levels as well.

On a side note, as I made the Functional Chart of the EU, I quickly became fixated with a particular type of chart one often sees on Wikipedia articles, mapping the way the electorate and the 3 powers of the state – executive, legislative and judicial – interact with each other.

There little information on-line on these types of charts. On a summary look, It seems to me that they are, in essence, a type of flowcharts, but I would welcome any further reading on the history of these types of charts.

And I also welcome any feedback in case of any mistake I might have slipped in.

Source: mainly Wikipedia. Tool: Inkscape