Data.gov one of many open government platforms with usability problems, says paper

The majority of open government data platforms worldwide lack standardized APIs and publish data in proprietary and non machine-readable formats, according to a paper (.pdf) presented at the World Wide Web conference, held in Lyon, France April 16-20. The United States' Data.gov platform is no exception.

In surveying 50 open data platforms, computer science faculty members at the Technische Universität Dresden, in Dresden, Germany sought to determine the usability of public data.

Data.gov and Data.gov.uk fall into the category of "Link Collecting" platforms.

"This class is not only the largest, but also generally the least useful one…[a] large percentages of these links do not resolve to a working file download," write Katrin Braunschweig, Julian Eberius, Maik Thiele and Wolfgang Lehner.

Fifty-five percent of the surveyed platforms fall into this category of sites that host only metadata and store URLs as the only way of accessing data.  

"These platforms have a lower level of standardization and integration between datasets," which makes data less usable, says the paper. Only 20 percent of the link collection platforms have standardized file formats, but 42 percent of platforms overall had standardized file formats, it adds.

In contrast, download catalogs, which made up 29 percent of surveyed platforms host the files instead of links to the files and have "much higher reusability," write authors. Eighty percent of download catalogs had standardized file formats, finds the paper.

Data.gov and Data.gov.uk are singled out in the paper for dead links and myriad file formats. Usability and readability are challenged by the fact that only 79 percent of the metadata on the sites can be used to download actual files, according to the paper.

"Some links were dead, while others led to HTML pages or web applications where the data might be found," write authors.

Data.gov and Data.gov.uk do have the largest data repositories in the survey, however. According to the paper, Data.gov had 4900 data sets and Data.gov.uk had 7400 at the time of review. Most other platforms had 100 to 1000 data sets, write authors, but Estonia was a noteable exception. "Estonia provides a statistics database that has over 3000 indicators in a well structured format," says the paper.

Report authors say they would like to repeat the survey report annually to see how platform trends evolve.

For more:
- go to the survey page
- download the paper (.pdf)

Related Articles:
Transparency groups call for THOMAS bulk downloads
Term 'open government' more murky than transparent, says paper
Europe unveils open data strategy, hopes to see economic benefits