Panel on official statistics database revamp proposes proof-of-concept before planning NIDS
The idea of NIDS was mooted as a possible integrating framework that would enable users of official statistics to have a single view of data available in the official statistical system.
The Committee on Analytics, one of the five expert groups set up by the National Statistical Commission (NSC) to examine ways to improve the methodology and database issues related to GDP calculation, has stated that the depth and breadth of India's official statistics does not favour the establishment of a one single central repository of database. It wants NSC to make creation of a National Integrated Data System (NIDS) a long term goal of the Indian Official Statistical System, while working on a proof-of-concept to begin with.
In its report, the committee, headed by Prof. N. L. Sarda, Emeritus Fellow, IIT Bombay said that given the federal structure of Indian official statistics system, such a single central repository is not administratively feasible.
It recommended that the diversity in datasets in regard to its various attributes like storage system, data granularity, data frequency, metadata etc. can be brought under an integrating framework only by imposing standards in the data governance architecture that each statistical agency is required to adopt and implement. "These standards relate primarily to establishing a common metadata framework, prescribing a standard for technological infrastructure, a standard protocol for data flow between statistical agencies and a common data dissemination framework". In the absence of requisite as-is position data in the above areas, the committee proposed a proof-of concept project to gather requisite data to set the standards.
The idea of NIDS was mooted as a possible integrating framework that would enable users of official statistics, to have a single view of data available in the official statistical system, irrespective of the fact that underlying databases are distributed and managed by different central and state government ministries/departments. The committee felt that an apex coordination committee should be set up at the centre with representatives from the states as well as MOSPI and other ministries of GOI to set out standards for development of NIDS. It also proposed a similar coordination committee at the state level to ensure that the standards and frameworks designed at the apex level get implemented in the respective states.
"The committee is proposing an incremental approach towards building NIDS. To begin with the committee is proposing that a proof-of-concept project be taken up. The apex committee should take up developing the standards that will define the integration features of the proposed NIDS in parallel to directing and monitoring of the proof-of-concept project. These are - common metadata framework, standard protocol for data flow between various state and central producers and consumers of data and a common data dissemination framework," it said.
NSC, which has put out the committee's report for public comments till September 30, said that it does not necessarily agree with the views, data and other contents of the report.