A 5-Step Framework for Measuring the Quality of Experience (QoE) of Internet of Things (IoT) Services

Muhammad Suryanegara, Dimas Agung Prasetyo, Fery Andriyanto, Nur Hayati

Research output: Contribution to journalArticle

Abstract

The ultimate benefit of the Internet of Things (IoT) is to improve quality of life by supporting the automation and interconnection of various services; this interconnection includes public facilities in platforms such as smart cities. IoT implementation is believed to improve services and yield additional benefits. However, assessment of how the IoT brings such an experience to users is still a challenge that is not yet standardized. Given that little research has discussed this issue, this paper aims to formulate a framework for measuring the quality of experience (QoE) regarding IoT services on the basis of the Absolute Category Rating with Hidden Reference (ACR-HR) scale, which measures the perception of experience by having the IoT users provide scores for their experiences before and after IoT implementation. The proposed framework consists of 5 steps: setting up the focus of the IoT services and the QoE parameters being assessed, defining users of IoT services, conducting a mean opinion score (MOS) survey of the users, calculating the DMOS (differential MOS) on the basis of the ACR-HR quantitative scale, and finally providing the strategic implications to those who implement the service. To test the proposed framework, we have conducted a proof of concept by measuring the QoE for the IoT services in the Jakarta Smart City platform. Six institutional users operate the IoT for their public services, ranging from fire and rescue brigades to public bus transportation. The results imply that the IoT-based services provide a good experience to institutional users when compared to the state of services without IoT. A strategic implication is that some features of Jakarta Smart City must be evaluated and improved to increase the level of satisfaction of users. To conclude, our framework is sufficiently reliable in measuring the QoE for IoT-based services and can be recommended for use for any other IoT-type service.

Original languageEnglish
Article number8922616
Pages (from-to)175779-175792
Number of pages14
JournalIEEE Access
Volume7
DOIs
Publication statusPublished - 1 Jan 2019

Fingerprint

Internet of things
Bus transportation
Fires
Automation
Smart city

Keywords

  • ACR-HR
  • Indonesia
  • IoT
  • Jakarta
  • quality of experience
  • smart city

Cite this

Suryanegara, Muhammad ; Prasetyo, Dimas Agung ; Andriyanto, Fery ; Hayati, Nur. / A 5-Step Framework for Measuring the Quality of Experience (QoE) of Internet of Things (IoT) Services. In: IEEE Access. 2019 ; Vol. 7. pp. 175779-175792.
@article{7690e7eb59c749949fb02a434644c16e,
title = "A 5-Step Framework for Measuring the Quality of Experience (QoE) of Internet of Things (IoT) Services",
abstract = "The ultimate benefit of the Internet of Things (IoT) is to improve quality of life by supporting the automation and interconnection of various services; this interconnection includes public facilities in platforms such as smart cities. IoT implementation is believed to improve services and yield additional benefits. However, assessment of how the IoT brings such an experience to users is still a challenge that is not yet standardized. Given that little research has discussed this issue, this paper aims to formulate a framework for measuring the quality of experience (QoE) regarding IoT services on the basis of the Absolute Category Rating with Hidden Reference (ACR-HR) scale, which measures the perception of experience by having the IoT users provide scores for their experiences before and after IoT implementation. The proposed framework consists of 5 steps: setting up the focus of the IoT services and the QoE parameters being assessed, defining users of IoT services, conducting a mean opinion score (MOS) survey of the users, calculating the DMOS (differential MOS) on the basis of the ACR-HR quantitative scale, and finally providing the strategic implications to those who implement the service. To test the proposed framework, we have conducted a proof of concept by measuring the QoE for the IoT services in the Jakarta Smart City platform. Six institutional users operate the IoT for their public services, ranging from fire and rescue brigades to public bus transportation. The results imply that the IoT-based services provide a good experience to institutional users when compared to the state of services without IoT. A strategic implication is that some features of Jakarta Smart City must be evaluated and improved to increase the level of satisfaction of users. To conclude, our framework is sufficiently reliable in measuring the QoE for IoT-based services and can be recommended for use for any other IoT-type service.",
keywords = "ACR-HR, Indonesia, IoT, Jakarta, quality of experience, smart city",
author = "Muhammad Suryanegara and Prasetyo, {Dimas Agung} and Fery Andriyanto and Nur Hayati",
year = "2019",
month = "1",
day = "1",
doi = "10.1109/ACCESS.2019.2957341",
language = "English",
volume = "7",
pages = "175779--175792",
journal = "IEEE Access",
issn = "2169-3536",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

A 5-Step Framework for Measuring the Quality of Experience (QoE) of Internet of Things (IoT) Services. / Suryanegara, Muhammad; Prasetyo, Dimas Agung; Andriyanto, Fery; Hayati, Nur.

In: IEEE Access, Vol. 7, 8922616, 01.01.2019, p. 175779-175792.

Research output: Contribution to journalArticle

TY - JOUR

T1 - A 5-Step Framework for Measuring the Quality of Experience (QoE) of Internet of Things (IoT) Services

AU - Suryanegara, Muhammad

AU - Prasetyo, Dimas Agung

AU - Andriyanto, Fery

AU - Hayati, Nur

PY - 2019/1/1

Y1 - 2019/1/1

N2 - The ultimate benefit of the Internet of Things (IoT) is to improve quality of life by supporting the automation and interconnection of various services; this interconnection includes public facilities in platforms such as smart cities. IoT implementation is believed to improve services and yield additional benefits. However, assessment of how the IoT brings such an experience to users is still a challenge that is not yet standardized. Given that little research has discussed this issue, this paper aims to formulate a framework for measuring the quality of experience (QoE) regarding IoT services on the basis of the Absolute Category Rating with Hidden Reference (ACR-HR) scale, which measures the perception of experience by having the IoT users provide scores for their experiences before and after IoT implementation. The proposed framework consists of 5 steps: setting up the focus of the IoT services and the QoE parameters being assessed, defining users of IoT services, conducting a mean opinion score (MOS) survey of the users, calculating the DMOS (differential MOS) on the basis of the ACR-HR quantitative scale, and finally providing the strategic implications to those who implement the service. To test the proposed framework, we have conducted a proof of concept by measuring the QoE for the IoT services in the Jakarta Smart City platform. Six institutional users operate the IoT for their public services, ranging from fire and rescue brigades to public bus transportation. The results imply that the IoT-based services provide a good experience to institutional users when compared to the state of services without IoT. A strategic implication is that some features of Jakarta Smart City must be evaluated and improved to increase the level of satisfaction of users. To conclude, our framework is sufficiently reliable in measuring the QoE for IoT-based services and can be recommended for use for any other IoT-type service.

AB - The ultimate benefit of the Internet of Things (IoT) is to improve quality of life by supporting the automation and interconnection of various services; this interconnection includes public facilities in platforms such as smart cities. IoT implementation is believed to improve services and yield additional benefits. However, assessment of how the IoT brings such an experience to users is still a challenge that is not yet standardized. Given that little research has discussed this issue, this paper aims to formulate a framework for measuring the quality of experience (QoE) regarding IoT services on the basis of the Absolute Category Rating with Hidden Reference (ACR-HR) scale, which measures the perception of experience by having the IoT users provide scores for their experiences before and after IoT implementation. The proposed framework consists of 5 steps: setting up the focus of the IoT services and the QoE parameters being assessed, defining users of IoT services, conducting a mean opinion score (MOS) survey of the users, calculating the DMOS (differential MOS) on the basis of the ACR-HR quantitative scale, and finally providing the strategic implications to those who implement the service. To test the proposed framework, we have conducted a proof of concept by measuring the QoE for the IoT services in the Jakarta Smart City platform. Six institutional users operate the IoT for their public services, ranging from fire and rescue brigades to public bus transportation. The results imply that the IoT-based services provide a good experience to institutional users when compared to the state of services without IoT. A strategic implication is that some features of Jakarta Smart City must be evaluated and improved to increase the level of satisfaction of users. To conclude, our framework is sufficiently reliable in measuring the QoE for IoT-based services and can be recommended for use for any other IoT-type service.

KW - ACR-HR

KW - Indonesia

KW - IoT

KW - Jakarta

KW - quality of experience

KW - smart city

UR - http://www.scopus.com/inward/record.url?scp=85076958171&partnerID=8YFLogxK

U2 - 10.1109/ACCESS.2019.2957341

DO - 10.1109/ACCESS.2019.2957341

M3 - Article

AN - SCOPUS:85076958171

VL - 7

SP - 175779

EP - 175792

JO - IEEE Access

JF - IEEE Access

SN - 2169-3536

M1 - 8922616

ER -