April 24, 2016 Hatem

#ODBarometer, how to turn 66% of Government data into OpenData ?

First alarming result from the third edition of the Open Data Barometer is that only 10% of government data is available as Open Data, while 76.3% of these data is available online ! So, what’s missing to turn at least 60% of government data into Open Data ? Let’s have a look on the non-OpenData :

The graphics above show 66.1% of analyzed datasets are available but not open. You can imagine how the world will look like if at least these datasets were open ?

Yes, almost like that ! But most of these data today are somehow locked, whether it require registration, or locked in unusable closed formats, etc…

Analyzing the “closed data” :

We’ll try to have a look deep inside all these datasets that are available online but not available as Open Data, and first let’s see where these datasets are located worldwide :

You may notice that this is happening all over the world and the non open datasets goes from 15 datasets in Japan which is ranked 13th in the ODB, to 2 datasets in UK which is ranked number one in this latest edition of the barometer. Below a table of the top 22 countries (Notice the column counter is for isOpen=0):

ODB-isOpen-count

Even the number one is not having all its datasets published as Open Data, so to be realistic let’s lower the target of 76.2% to 62.9% which remain a great target to reach if we can publish all (13 of 15) the available datasets as Open Data.

So if we want to summarize here, we are talking about :

  • 1381 datasets worldwide
  • 328 datasets not available
  • 913 datasets available but not OpenData.

Of course by “datasets” we mean all datasets related to one of the data categories of the ODB.

The Machine readable datasets

We are talking about datasets that are available online, but not open data. So, the first question, are these datasets available in machine readable format ? the map below summarize the answer to this question

The table below summarize the top 22 countries having datasets in machine readable formats but not open data :

ODB-machinereadable-sum

You may notice that this list is a little different from the first one, and many datasets that are available online are available in a machine readable format. This remain not open data, but we are not so far. But how many online datasets are machine readable format ?

Only 45% of datasets available online are available in machine readable format, and this is the first criteria that we should consider to turn government data into Open Data.

We can also analyze if these datasets are available for free or not. We can see here that only 41.1% of the datasets available online are free. Which mean that most of the data is not available for free !

In the same way we can analyze the Open Data licenses availability for these datasets, and here you will be surprised that only 9.1% of the datasets published online include an open license which is too low !! The entire world seems not giving to legal issues related to the data published any importance, or limiting the usage of these data which does not make them really free for use and reuse.

Conclusion

In this article I focused only on few criteria that, if respected, could lead to a higher amount of published open data worldwide up to 76% instead of the actual 10%.

Overall, we may find the same conclusion and recommendation in the ODB report, in addition to that the world should focus on making data machine readable, with open license, and free of charges to help the adoption of Open Data worldwide. The only remaining issue in the sustainability of the publication process which makes the data usually not updated, not consistent, and with very low quality.

Finally, we are still so far away from linked data, and probably if we focus more on linked data the process could be easier and the adoption will be faster, most important is that we’ll be promoting five stars grade Open Data, rather than promoting low level open data.

, , ,

(HBY) Consultancy