A Comparative Study of Two People-Centric Smart Cities Evaluation Frameworks

Written by Kiana Etemadi and Alireza Fereidunian

Smart cities performance evaluation and measurement can widely regarded as a standardization process. Various government agencies and organizations have worked on evaluating different parts of smart cities’ instructures and services through different frameworks. This article focuses on two important people-centric smart cities assessment frameworks, namely Smart City Measurement Framework, proposed by OECD, and IMD-SUTD Smart City Index (SCI). The two frameworks are discussed and compared in terms of six criteria, concluding that they are essentially similar. The following strategies are recommended to evaluate the smart cities' performance: measuring the performance of smart cities on a regular basis, reflecting all dimensions of smart cities objectives, capturing stakeholders’ engagement, and comparing cities on different scales.


Essentially, standards set out a common understanding of the best methods to perform tasks or achieve goals [1], thus standardization is the process of creating protocols to guide the creation of a good or service based on the consensus of the relevant parties in the industry. Standards facilitate and enable systems performance evaluation, let the open systems interoperately work, and assure quality, reliability, safety, and market accessibility [2].

Centrally planned decisions and budget deficiencies and failures are of the main concerns regarding the smart cities development. Active participation of people in smart cities development might tacle this concerns, as it makes financial and decision-making burden be distributed within people [3,4].

Assessing smart city performance consistutes an important part of the standardization process, which is essential to ensure operational effectiveness. It also helps ground policy intervention in solid evidence by guiding decision-makers at national and local levels, in setting realistic targets, tracking progress, and adjusting policy interventions for greater efficiency and effectiveness [5,6].


Assessment Frameworks

A wide range of institutions, organizations, and also cities have worked on measuring smart cities through a variety of frameworks [5]. In this research two well-known, people-centric smart cities assessment frameworks are investigated, namely:

  • IMD-SCO-SUTD Smart City Index (SCI) by the Institute for Management Development (IMD), in collaboration with Smart City observatory and Singapore University of Technology and Design (SUTD), [6]
  • Smart City Measurement Framework, proposed by Organisation for Economic Co-operation and Development (OECD). [7]



In 2017, three institutions, IMD, SCO and SUTD, joined forces to introduce a smart city index, offering a balanced focus on economic and technological aspects of smart cities on one hand, and “humane dimensions” of smart cities on the other hand. IMD-SCO-SUTD has released the 2021 Smart City Index. Hundreds of citizens from 118 cities were surveyed and asked questions on the technological provisions of their city across five key areas: health and safety, mobility, activities, opportunities, and governance [6].


OECD Smart City Measurement Framework

The proposed OECD Smart City Measurement Framework is intended to answer what to measure and how, and for whom the framework is intended. Rather that focusing on digitalization, it concerns the level of stakeholders’ engagement, and their contribution to the well-being of citizens, and building more inclusive, resilient, and sustainable cities. The OECD Smart City Measurement Framework aims to assess the following pillars:

  1. Leverage digitalization
  2. Engage stakeholders
  3. Improve people’s well-being and build more inclusive, sustainable, and resilient societies [7].


Discussions and Comparisons

The discussion section is dedicated to the main purpose of the article, which is to compare two frameworks introduced in this research. For this purpose, first, in Table 1, the general characteristics of the frameworks considered in this article are placed.


Table 1: A comparison of the general features of IMD-SCO-SUTD and OECD frameworks

Title Scale Main Purposes Methodology
  • Leading to an annual global ranking of smart cities
  • Assessing residents' perceptions of their city's application of technology
  • Offering a balanced focus on economic and technological aspects of smart cities and and “humane dimensions” of smart cities
The SCI is evaluated based on defined questions (using the perceptions of the last three years of the survey) related to two factors. Each factor is evaluated over five key areas. Cities are grouping based upon its HDI values.
OECD National and Local
  • Focus on people and consider urban residents as designers of smart cities
  • Encompass not only digitalisation in cities, but also how digital innovation can improve well-being outcomes, inclusion, sustainability, and resilience
  • Enable benchmarking of cities across countries
  • Monitoring over time
The OECD framework revolves around three pillars. Each pillar has some dimensions, for each of which sets of indicators considered to be measured by governments.

The most important features of the frameworks are compared as follows in Tables 2 to 7. Since each of the evaluation methods has its own methodology, and also, the criteria and metrics of each are different, for easier comparison, the criteria introduced in the six dimensions of smart cities that are accepted and recognized in smart cities literature are included as: smart people, smart economy, smart governance, smart environment, smart mobility, and smart living. [3-5]


Table 2: Comparison of smart people dimension's criteria

sc june 2022 table2 edited


Table 3: Comparison of smart governance dimension's criteria

sc june 2022 table3 edited


Table 4: Comparison of smart economy dimension's criteria

sc june 2022 table4 edited


Table 5: Comparison of smart environment dimension's criteria

sc june 2022 table5 edited


Table 6: Comparison of smart mobility dimension's criteria

sc june 2022 table6 edited


Table 7: Comparison of smart living dimension's criteria

sc june 2022 table7 edited


As it can be seen in the Tables 2 to 7, most of the criteria defined for each of the OEC and IMD frameworks overlap and are similar. However, a couple of criteria are only considered in one of the frameworks.



Measuring the performance of smart cities is essential to ensure their effectiveness, enhance accountability, and help citizens monitor how governments deliver on their commitments.

As reviewed in two sample performance evaluation frameworks in this articl, the first step is to define a set of criteria, followed by specific indicators or indices for each criterion. Indicators are expected to be objective, simple, measurable, available, and comparable.

A smart city performance evaluation framework may be instrumental measuring the performance of smart cities per se, reflecting all dimensions of smart cities’ objectives, capturing stakeholders’ engagement, and comparing cities nationally or globally.

This research has been supported by Iranian National Elites Foundation (INEF).



  1. ISO “Benefits of Standards”, International Standard Organization (ISO), : https://www.iso.org/benefits-of-standards.html, Retrieved at June 8, 2022
  2. NIST, “Standards & Measurements”, National Institute of Standards and Technology (NIST), https://www.nist.gov/services-resources/standards-and-measurements, Retrieved at June 8, 2022
  3. Etemadi, K., “Centering People in Smart Cities” in Smart Cities: People-Centric Situation-Aware Approaches",  Smart Cities Week, Smart Cities Ambassadors Program, Avialable at: https://www.youtube.com/watch?v=1yKDLTKY5H8
  4. Kojury-Naftchali, M., Fereidunian, A. “Smart Energy-Aware Cities: Customer Characterization by Energy Data Analytics to Improve Demand Response Performance”, In: Shafie-khah, M., Amini, M.H. (eds) Flexible Resources for Smart Cities. Springer, Cham, 2021.
  5. Nozarian M, Fereidunian A. Smart City as an Smart Energy Hub: A Bibliographic, Analytic and Structural Review. IEIJQP Journal, Vol. 9, No. 4, pp: 62-82, 2020.
  6. IMD-SCO-SUTD, “Smart City Index 2021: A tool for action, an instrument for better lives for all citizens”, Institute for Management Development (IMD), in collaboration with Smart City observatory (SCO) and Singapore University of Technology and Design (SUTD), 2021.
  7. OECD, “Measuring Smart Cities’ Performance: Do smart cities benefit everyone?”, In 2nd  OECD Roundtable on Smart Cities and Inclusive Growth, Organisation for Economic Co-operation and Development, December 3, 2020. IEEE Wireless Communications, vol. 24, no. 6, pp. 14-21, Dec. 2017, doi: 10.1109/MWC.2017.1600414.



This article was edited by Bernard Fong

To view all articles in this issue, please go to June 2022 eNewsletter. For a downloadable copy, please visit the IEEE Smart Cities Resource Center.

K Etemadi crop
Kiana Etemadi is a Ph.D. student in urban community and regional planning at Tarbiat Modares University (TMU), Tehran, Iran. She received her M.Sc. degree in regional planning from Shahid Beheshti University (SBU). Her research interests cover urban growth patterns, smart cities and smart citizens, regional and local development, inclusion, and sustainability. She is also a university lecturer and urban planning specialist in Tehran municipality. She has published more than 10 articles associated with urban and regional planning. 
Alireza Fereidunian
Alireza Fereidunian (S’02, M’09, SM’16) is an Assistant Professor at the K. N. Toosi University of Technology, Tehran, Iran. His is interested in research in smart environments (city, eGrid, and home), high reliability distribution systems, energy analytics, and data-driven maintenance of smart city infrastructures. Moreover, he enthusiastically enjoys learning by research in complexity theory, systems reliability, energy economics, and human-automation interactions areas. Alireza has been serving as an independent consultant to utility companies and municipalities for more than ten years. He is an IEEE Smart Cities Ambassador 2022. 

IEEE Smart Cities Publications Journals and Magazines Special Issues

This web page displays the effort of IEEE Smart Cities Publications Committee in proposing and guest editing special issues for IEEE Journals and Magazines which is of interests to IEEE Smart Cities Community. Please click here to view.

Past Issues

To view archived articles, and issues, which deliver rich insight into the forces shaping the future of the smart cities. Older eNewsletter can be found here. To download full issues, visit the publications section of the IEEE Smart Cities Resource Center.