White House releases open data policy
Agencies and departments are now required to collect and create information in a way that supports downstream processing and sharing, through the use of machine-readable and open formats, data standards, and common core and extensible metadata.
The instruction, part of the White House's much-anticipated open data policy, came May 9 in the form of a 12-page memoranda (.pdf) signed by Office of Management and Budget officials including Federal Chief Information Officer Steven VanRoekel and Chief Technology Officer Todd Park.
Under the new policy, agencies must build and modernize information systems in a way that "maximizes interoperability and information accessibility, maintains internal and external data asset inventories, enhances information safeguards, and clarifies information management responsibilities."
The memo defines terminology used in the policy and lays out implementation guidance. Among the open data requirements, agencies must use common core metadata to describe information.
"Metadata should also include information about origin, linked data, geographic location, time series continuations, data quality, and other relevant indices that reveal relationships between datasets and allow the public to determine the fitness of the data source," says the memo.
The memo instructs agency CIOs to incorporate minimum data requirements into the acquisition and technical designs of information systems. These requirements include scalable, flexible architecture that facilitates data extraction.
The policy requires agencies to review and revise data policies within 6 months to create and maintain an enterprise data inventory, a public data listing, a process to engage customers and prioritize data release, and roles and responsibilities for data release.
The memo instructs agencies to incorporate the new open data requirements into their core agency processes by developing and maintaining an information resource management plan. This plan should align with their agency strategic plan and will be reviewed as part of their annual PortfolioStat process, says the memo.
An executive order published the same day by the White House says within 30 days VanRoekel and Park will publish and regularly update a corresponding online repository of tools and best practices to help agencies comply with the policy.
In what appears to be the immediate completion of that milestone, the White House announced a complementary initiative May 9 it calls Project Open Data, which provides free, open source tools on Github. Best practices outlined in Project Open Data cover common core metadata use, for example.
The EO says agencies are required to adhere to the policy's deadlines and specifications. As part of policy implementation each agency's senior official for privacy will identify information that should not be released, says the executive order.
"As agencies consider whether or not information may be disclosed, they must also account for the 'mosaic effect' of data aggregation," says the memo. The mosaic effect occurs when information alone is not identifiable but when coupled with other available information poses a privacy or security risk.
"Agencies should note that the mosaic effect demands a risk-based analysis, often utilizing statistical methods whose parameters can change over time, depending on the nature of the information, the availability of other information, and the technology in place that could facilitate the process of identification," adds the memo.
Within 90 days, several OMB offices and federal councils--such as the Office of Information and Regulatory Affairs and the CIO Council--will issue tools to support the integration of policy requirements into acquisition and grant-making processes, says the EO.
This will include "developing sample requirements language, grant and contract language, and workforce tools for agency acquisition, grant, and information management and technology professionals," says the order.
The White House plans to oversee implementation of the policy with a "cross-agency priority goal", or CAP goal, to be established by the federal chief performance officer and the president's management council.
Within 90 days the CPO will meet with agencies to set incremental performance goals, including metrics and milestones that will allow agencies to monitor advancement toward the CAP goal, says the order. And in 180 days agencies will report their progress on the CAP goal to the CPO, and report progress quarterly after that.
The White House mentions no plans for additional funds to support the many agency requirements outlined in the memo and executive order.
"Policy implementation may require upfront investments depending on the maturity of existing information life cycle management processes at individual agencies," notes the policy memo.
"However, effective implementation should result in downstream cost savings for the enterprise through increased interoperability and accessibility of the agency's information resources," adds the memo, which goes on to ask agencies to consider up-front investment in the "context of future benefits."
The new open data policy and executive order "will make troves of previously inaccessible or unmanageable data easily available to entrepreneurs, researchers, and others," said the White House in a May 9 statement.
Other, complementary initiatives will take shape in the coming months, says the White House press release. Data.gov will launch improved visualization, mapping and metadata tools, as well as more application programming interface access, says the statement. The White House also plans to continue to information sessions, presentations, and "code-a-thons" focused on using open data, it says.
- download the policy, M-13-13 "Open Data Policy-Managing Information as an Asset" (.pdf)
- read the executive order, "Making Open and Machine Readable the New Default for Government Information"
- read a White House blog post from Park and VanRoekel
- watch an embedded video address from Park and VanRoekel
- read the White House press release