A Comparative Study of Two People-Centric Smart Cities Evaluation Frameworks

Written by Kiana Etemadi and Alireza Fereidunian

Smart cities performance evaluation and measurement can widely regarded as a standardization process. Various government agencies and organizations have worked on evaluating different parts of smart cities’ instructures and services through different frameworks. This article focuses on two important people-centric smart cities assessment frameworks, namely Smart City Measurement Framework, proposed by OECD, and IMD-SUTD Smart City Index (SCI). The two frameworks are discussed and compared in terms of six criteria, concluding that they are essentially similar. The following strategies are recommended to evaluate the smart cities' performance: measuring the performance of smart cities on a regular basis, reflecting all dimensions of smart cities objectives, capturing stakeholders’ engagement, and comparing cities on different scales.

Introduction

Essentially, standards set out a common understanding of the best methods to perform tasks or achieve goals [1], thus standardization is the process of creating protocols to guide the creation of a good or service based on the consensus of the relevant parties in the industry. Standards facilitate and enable systems performance evaluation, let the open systems interoperately work, and assure quality, reliability, safety, and market accessibility [2].

Centrally planned decisions and budget deficiencies and failures are of the main concerns regarding the smart cities development. Active participation of people in smart cities development might tacle this concerns, as it makes financial and decision-making burden be distributed within people [3,4].

Assessing smart city performance consistutes an important part of the standardization process, which is essential to ensure operational effectiveness. It also helps ground policy intervention in solid evidence by guiding decision-makers at national and local levels, in setting realistic targets, tracking progress, and adjusting policy interventions for greater efficiency and effectiveness [5,6].

Assessment Frameworks

A wide range of institutions, organizations, and also cities have worked on measuring smart cities through a variety of frameworks [5]. In this research two well-known, people-centric smart cities assessment frameworks are investigated, namely:

• IMD-SCO-SUTD Smart City Index (SCI) by the Institute for Management Development (IMD), in collaboration with Smart City observatory and Singapore University of Technology and Design (SUTD), [6]
• Smart City Measurement Framework, proposed by Organisation for Economic Co-operation and Development (OECD). [7]

IMD-SCO-SUTD SCI

In 2017, three institutions, IMD, SCO and SUTD, joined forces to introduce a smart city index, offering a balanced focus on economic and technological aspects of smart cities on one hand, and “humane dimensions” of smart cities on the other hand. IMD-SCO-SUTD has released the 2021 Smart City Index. Hundreds of citizens from 118 cities were surveyed and asked questions on the technological provisions of their city across five key areas: health and safety, mobility, activities, opportunities, and governance [6].

OECD Smart City Measurement Framework

The proposed OECD Smart City Measurement Framework is intended to answer what to measure and how, and for whom the framework is intended. Rather that focusing on digitalization, it concerns the level of stakeholders’ engagement, and their contribution to the well-being of citizens, and building more inclusive, resilient, and sustainable cities. The OECD Smart City Measurement Framework aims to assess the following pillars:

1. Leverage digitalization
2. Engage stakeholders
3. Improve people’s well-being and build more inclusive, sustainable, and resilient societies [7].

Discussions and Comparisons

The discussion section is dedicated to the main purpose of the article, which is to compare two frameworks introduced in this research. For this purpose, first, in Table 1, the general characteristics of the frameworks considered in this article are placed.

Table 1: A comparison of the general features of IMD-SCO-SUTD and OECD frameworks

The most important features of the frameworks are compared as follows in Tables 2 to 7. Since each of the evaluation methods has its own methodology, and also, the criteria and metrics of each are different, for easier comparison, the criteria introduced in the six dimensions of smart cities that are accepted and recognized in smart cities literature are included as: smart people, smart economy, smart governance, smart environment, smart mobility, and smart living. [3-5]

Table 2: Comparison of smart people dimension's criteria

Table 3: Comparison of smart governance dimension's criteria

Table 4: Comparison of smart economy dimension's criteria

Table 5: Comparison of smart environment dimension's criteria

Table 6: Comparison of smart mobility dimension's criteria

Table 7: Comparison of smart living dimension's criteria

As it can be seen in the Tables 2 to 7, most of the criteria defined for each of the OEC and IMD frameworks overlap and are similar. However, a couple of criteria are only considered in one of the frameworks.

Conclusions

Measuring the performance of smart cities is essential to ensure their effectiveness, enhance accountability, and help citizens monitor how governments deliver on their commitments.

As reviewed in two sample performance evaluation frameworks in this articl, the first step is to define a set of criteria, followed by specific indicators or indices for each criterion. Indicators are expected to be objective, simple, measurable, available, and comparable.

A smart city performance evaluation framework may be instrumental measuring the performance of smart cities per se, reflecting all dimensions of smart cities’ objectives, capturing stakeholders’ engagement, and comparing cities nationally or globally.

This research has been supported by Iranian National Elites Foundation (INEF).

References

1. ISO “Benefits of Standards”, International Standard Organization (ISO), : https://www.iso.org/benefits-of-standards.html, Retrieved at June 8, 2022
2. NIST, “Standards & Measurements”, National Institute of Standards and Technology (NIST), https://www.nist.gov/services-resources/standards-and-measurements, Retrieved at June 8, 2022
3. Etemadi, K., “Centering People in Smart Cities” in Smart Cities: People-Centric Situation-Aware Approaches",  Smart Cities Week, Smart Cities Ambassadors Program, Avialable at: https://www.youtube.com/watch?v=1yKDLTKY5H8
4. Kojury-Naftchali, M., Fereidunian, A. “Smart Energy-Aware Cities: Customer Characterization by Energy Data Analytics to Improve Demand Response Performance”, In: Shafie-khah, M., Amini, M.H. (eds) Flexible Resources for Smart Cities. Springer, Cham, 2021.
5. Nozarian M, Fereidunian A. Smart City as an Smart Energy Hub: A Bibliographic, Analytic and Structural Review. IEIJQP Journal, Vol. 9, No. 4, pp: 62-82, 2020.
6. IMD-SCO-SUTD, “Smart City Index 2021: A tool for action, an instrument for better lives for all citizens”, Institute for Management Development (IMD), in collaboration with Smart City observatory (SCO) and Singapore University of Technology and Design (SUTD), 2021.
7. OECD, “Measuring Smart Cities’ Performance: Do smart cities benefit everyone?”, In 2nd  OECD Roundtable on Smart Cities and Inclusive Growth, Organisation for Economic Co-operation and Development, December 3, 2020. IEEE Wireless Communications, vol. 24, no. 6, pp. 14-21, Dec. 2017, doi: 10.1109/MWC.2017.1600414.

This article was edited by Bernard Fong

To view all articles in this issue, please go to June 2022 eNewsletter. For a downloadable copy, please visit the IEEE Smart Cities Resource Center.

Kiana Etemadi is a Ph.D. student in urban community and regional planning at Tarbiat Modares University (TMU), Tehran, Iran. She received her M.Sc. degree in regional planning from Shahid Beheshti University (SBU). Her research interests cover urban growth patterns, smart cities and smart citizens, regional and local development, inclusion, and sustainability. She is also a university lecturer and urban planning specialist in Tehran municipality. She has published more than 10 articles associated with urban and regional planning. 

Alireza Fereidunian (S’02, M’09, SM’16) is an Assistant Professor at the K. N. Toosi University of Technology, Tehran, Iran. His is interested in research in smart environments (city, eGrid, and home), high reliability distribution systems, energy analytics, and data-driven maintenance of smart city infrastructures. Moreover, he enthusiastically enjoys learning by research in complexity theory, systems reliability, energy economics, and human-automation interactions areas. Alireza has been serving as an independent consultant to utility companies and municipalities for more than ten years. He is an IEEE Smart Cities Ambassador 2022.