When we at Chennai Smart City (CSCL) were tasked with collecting, collating, cleaning, streamlining and uploading 50 datasets onto the open data portal it seemed a herculean task. A herculean task not because of the number of datasets, but the mere logistics of coordinating between different departments, quenching their doubts on the need for this exercise and finally following up to ensure they provide correct data in the prescribed form was a tall challenge. However, right from the word go we adopted a pro-active approach carefully combining both top-down and bottom-up approaches to ensure an efficient end-to-end data life cycle process.
First, we ensured the top management of CSCL and the Greater Chennai Corporation (GCC) was on board. For this we appraised in detail the Corporation Commissioner, who is also the Chairman of CSCL and Deputy Commissioner (Works), who is also the Managing Director of CSCL and the CEO of CSCL about the exercise, its benefits and the approach we intend to undertake to fulfill the task. Once they were on board, we sent out letters to all departments from the Commissioner of GCC outlining the data collection exercise, the datasets we required and the format in which they have to submit.
Simultaneously, we undertook a micro-level exercise and appointed a two member unit from our team to personally contact all the departments, strike a conversation with them regarding the exercise, visit them if necessary and ensured the essence of this exercise and the benefits the department shall reap through this exercise was conveyed. We also identified a data coordinator in each department, who was passionate and efficient in delivery. In majority of the cases, we managed to map the same official who was tasked with Ease of Living Index data collection the previous year, so that the official is aware of data and its associated nuances.
These two, combined with persistent follow up and regular tracking ensured CSCL uploaded more than 80% of the tasked data sets in the prescribed format within one month of the task handed over to us. During this task, our team encountered few challenges such as non-availability of city wide, zone wise, ward wise data of indicators. Air quality data was one such case, which was measured location wise in Chennai. Robust statistical methods compliant with the requirements of the Open Data Portal were undertaken to aggregate the location wise data, to arrive at a city wide figure. Similarly, the city profile mandated us to categorize population data by age groups which did not match the standard Census categorization. In this case, our team collaborated with the Census team to arrive at the required categorization.
It was robust planning and effective follow up alone which enabled us to efficiently achieve this number. We are sure these datasets would be put to good use by our top management in the years to come as CCSL and GCC have already moved onto a data driven governance framework in the last few years. In fact, with the Integrated Command Control Centre, IUDX and the Data Observatory in the pipeline, these secondary datasets would add enormous value and aid decision making.