Real-Time Road Quality Indicator (RQI) for Smart Driving

Written by Dr. Hrishikesh Venkataraman

Road transport is one of the major modes of transport across the world. It covers every corner of almost each country/continent, connecting the most remote places. It also contributes significantly to the economic development of countries. In the era of smart vehicles and Intelligent Transportation System (ITS), roads are no longer merely a physical entity or solid ground. They have to be ‘empowered’ with information, communication, intelligence and sensing capability that were never possible decades ago. Having said that, if one leaves out the highways/freeways, the conditions of the remaining roads are not always best suitable to cruise-in. In fact, if one looks at emerging economies such as Asia, South America and Africa, which form the bulk of next generation growth in the smart transportation, the roads are not being well-maintained, primarily owing to the high cost of operation. Most of the times, the existing roads are not at all conducive to operate a smooth and efficient road transport system. In fact, there are several challenges that still needs to be identified and addressed in order to make roads and driving conditions better in multi-vehicle-type driving, diverse road conditions.

---

Motivation for the Road Quality Indicator (RQI)

According to the Global Report on Road Safety 2019 by World Health Organization (WHO), more than 1.3 million people die on the roads each year globally. Particularly, the number of fatal accidents tend to increase exponentially [1]. It is well-known that on-road traffic is remarkably influenced by environment, vehicle parameters and spatio-temporal factors. Hence, as the technology is developed for an intelligent road network, it is imperative to have a mechanism that would provide a score for the road, indicating how good or bad the road is, for driving the vehicle. It is essential to design a quantification mechanism – Road Quality Indicator (RQI) – in order to facilitate assessment of the attributes of road in real-time basis, to compute a qualitative opinion in terms of a numerical score, that would immensely assist the drivers. Significantly, it is essential that the RQI has a subjective and vehicle-type based quality indication component [2]. This is because, the RQI could be different for different types of vehicles such as two-wheelers, three-wheelers, cars, trucks, etc. For instance, in the context of residential and small streets prevalent in many of the streets in towns and cities of countries in Asia, Africa and South America; a particular road may be perfectly suitable for two-wheelers (bikes, scooters, etc.) while not being suitable for others such as three-wheelers and four-wheelers.

 

Factors in RQI

Fig. 1 shows a typical scenario of a road network in urban/sub-urban cities with multiple possibilities to reach from one end to another, with different vehicle types (Scooter, Bike, 3-wheelers, Car, Bus, Truck, etc.). As shown in Fig. 1, the main road has two lanes: one for each direction, with no provision for overtaking. The side lanes are small, bi-directional lanes. In fact, some of the lanes have potholes. It is essential that a smart transportation network investigate different aspects of road quality (lane markings, road curvature, etc.), driving characteristics, etc. Further, it needs to investigate challenges in real-time visual traffic perception which includes: illumination effects, specular and diffuse reflections, occlusion, etc. This requires majorly two types of investigation: multi-sensor based analysis of present road conditions and mechanism for quantifying the RQI.

Fig. 1 - Typical Lane and Road with Different Vehicle Types (Scooter, Bike, Car, Bus, Truck)

 

The RQI would be based broadly on the following factors:

1. Lane characteristics: Lane width, shoulder-width, etc.
2. Lanes: Presence of lane for overtake
3. Potholes: Number, Dimensions, Cracks, etc.
4. Markings: No. of lanes, Lane marking, Zebra crossing, etc.
5. Roads: Road Terrain, Reflector, Curvature, Seasonal effects, etc.
6. Lightings: Street light, Day/Night, etc.
7. Terrain: Flat road, Slippery road, Rain road, Snow road, Hilly terrain

 

The RQI would typically consist of ‘static’, ‘near-static’ and ‘dynamic’ factors depending on how frequently their value changes. The static or ‘near-static’ factors such as lane markings, lighting conditions, shoulder width, central median, etc. are relatively easy to quantify and could be given due weightage. On the other hand, the ‘dynamic’ factors such as weather, traffic, road obstacles, road conditions, traffic, etc. are much more difficult to quantify and require accurate identification for which a multitude of sensors such as accelerometer, cameras, RADAR, LIDAR, etc. are required.

 

Quantification of Factors for RQI

Some of the recent mechanisms that could be used to identify the different lane or road characteristics are based on various vision-based and image-based techniques such as LaneNet, YOLO, etc. or state-of-the-art techniques such as CNN-LSTM, RNN, GRU etc. [3, 4]. This could be done (with or without incorporation of edge detection techniques) to identify the lane boundaries by checking the continuity in the edges [5]. An interest aspect is the detection of potholes and the characterization of the same in real-time [6]. Potholes could be detected using camera-based sensors. Camera-based sensors could provide detailed information, but would require extensive processing due to tasks such as semantic segmentation, which is required to mask the exact spatial information of the image. This is in-turn required to get accurate pixel wise area, length, and width [7]. There are existing segmentation datasets such as COCO, VOC, etc. which could be fine-tuned to segment the potholes. Notably, in a night-time or dawn/dusk time, where the lighting may not be sufficient, vision-based techniques could fail to provide the required details. When the image is very dark, image enhancement would not provide sufficient brightness for detecting the objects. Hence, image translation needs to be done for such scenarios. Though night-to-day GAN (General Adversarial Network) could be used to translate night images to day images, it may still not give accurate information. Hence, this information needs to be added-up with data from other sensors such as RADAR, accelerometer, etc. Once sufficient data points are obtained, technologies like machine learning, sensor data analytics and computer vision could be applied for investigating the road quality. However, it should be noted that each technique comes with its own challenges. Hence, an integration of the multi-sensory mechanisms would be an important aspect to investigate to bring quantification of RQI into reality.

Importantly, in order to provide a rating for the overall road quality, a multi-criteria logic control block, using multi-attribute decision making approaches could be used. Some of the standard techniques include Simple Multi Attribute Rating Technique (SMART), Analytic Hierarchy Process (AHP) and hybrid models that need to be still further investigated [8]. It is to be noted that for each stage, a multi-sensory approach would be adopted to get the individual characteristics; and ranking for a score of 1-10, which would be then integrated to get an overall RQI. However, the multi-sensory approach comes with its own challenges of time and space synchronization. This is because, each sensor type would provide details of a particular region at different time intervals. For instance, an accelerometer sensor would give information of the presence of pothole only when it passes through the pothole. On the other hand, a vision-based sensor would provide the information of pothole, couple of meters before encountering the pothole. Also, the granularity level in the information between different sensor types would be different. Further, there are several other computational aspects that need to be considered in the RQI score calculation. Additionally, the detection of lane markings, visibility of lanes, the condition of potholes would change with time. Also, each sensor would not capture the data accurately. Hence the RQI would be a time-varying factor. Finally, it should be noted that the values of the factors captured by each vehicle for the RQI score would vary. This would be not only based on the type of vehicle, but also over time instants, i.e., when the same type of vehicle passes over different time instants, even within a time span of few minutes. Therefore, it is imperative to develop a centralized cloud-based mechanism where the calculated scores would be updated/ improved over a period of time. This cloud-based score could be then used by the drivers/vehicles to choose the road to take for their movement.

 

Summary

With the rapidly evolving technology, the safety and the drivability of the roads can be significantly improved by providing a multi-sensor-based, real-time, cloud-enabled road quality indicator (RQI). Having said that, there are several technical challenges in terms of characterizing the lane, road, etc. and also challenges in terms of computing capabilities, that still need to be addressed. It is anticipated that over the next 3 to 5 years, OEMs and solution providers would come up with several types of framework and scoring mechanism for the roads. Once the technology is well developed, the next step would be to develop mechanisms for standardization and global penetration. It would be an interesting phenomenon to see how this intelligent mechanism of quantification of the road quality would span out over the coming times.

 

 

 

References

1. WHO - https://www.who.int/news-room/fact-sheets/detail/road-traffic-injuries - last accessed on 7th August 2021.
2. R. Deshmukh, “A Road Surface Evenness and Roughness Measurement System”, Patent, WO2017208264A1, filed on June 20166, Priority No. IN201621018351, published in 2017.
3. Z. Wang, R. Weiqiang and Q. Qiu. "LaneNet: Real-Time Lane Detection Networks for Autonomous Driving", arXiv preprint arXiv:1807.01726, 2018.
4. Che-Tsung Lin, et al, "GAN-based Day-to-Night Image Style Transfer for Night-Time Vehicle Detection", IEEE Transactions on Intelligent Transportation Systems, Vol. 22, No. 2, pp. 951-963, 2020.
5. W. Zhang, et. al, “Full-Speed Lane Sensing with a Surrounding View System”, GM Global Technology Operations LLC, Patent, US20130293717A1, 2013.
6. K. Maheshwari, Ch. Mounika, K. Harshitha and H. Venkataraman, “PotSense: Pothole Detection on Indian Roads using Smartphone Sensors”, ACM Autonomous and Intelligent Mobile Systems (AIMS), IEEE/ACM COMSNETS, Bangalore, India, 8-11 Jan. 2020.
7. A. Dhiman and R. Klette, “Pothole Detection using Computer Vision and Learning”, IEEE Transactions on Intelligent Transportation System, Vol. 21, No. 8, August 2020.
8. A.K. Darmawan, et al. "A Hybrid Approach with Balanced Score Card and Fuzzy Topsis Method for Identifying Critical Factors Affecting Smart Regency Development", 5th IEEE International Conference on Informatics and Computing (ICIC), Bari, Italy, 2 – 5 October 2020.

 

 

 

This article was edited by Sara Paiva

For a downloadable copy of the October 2021 eNewsletter which includes this article, please visit the IEEE Smart Cities Resource Center.