Performance Testing of Mobile Chatbot Applications

In CAMAD 2018, a paper (PDF) that I co-authored introduces a new metric related to the Service Stability of mobile Chatbot Applications. The service stability metric is directly dependent to the following observed issues:

  • Image Loss,
  • Message Repetition,
  • Message Reorder and
  • Auxiliary Keyboard Loss.

Each of these observed issues has a different bias in the calculation formula of service stability. In the following equation the N represents the total number of steps in each scenario and m represents the number of successfully executed steps. The variable Ii is equal to 1 for each step that is observed the corresponding issue and equal to 0 elsewhere.

Each test score is normalized to the maximum score that can be achieved for each chatbot, so the best value for the service stability is 1 and the worst is 0. This metric provides an insight of the QoE that the user experiences under different reception conditions, quantifying the impact of the various impairments to the final service provision.

For the experimental needs of the paper, the proposed metric was validated based on experimental data retrieved by the 5G-TRIANGLE experimental testbed. Therefore, three different types of chatbots over Viber platform used for the execution of the experiment, each one having a different degree of complexity and requirements. The following Table summarizes the features of each chatbot.

Chatbot No. Features
Static Messages Database Communication API-based Communication
Chatbot #1
Chatbot #2
Chatbot #3

For the execution of the experiment it was mandatory to use the commercial application of Viber, via which the user would have access to the chatbot service. For the emulation of cellular network, the TRIANGLE testbed is using the UXM Wireless Test Platform device by Keysight. This device is capable of modifying a number of parameters of the wireless physical layer and thus emulate various network conditions. The parameters that modified in this experiment are:

  • the number of Downlink and Uplink Physical Resource Blocks (PRBS),
  • the number of Downlink and Uplink Subframes,
  • the Multipath Fading Propagation Conditions (EPA, EVA, ETU),
  • the antenna output power,
  • the mode, the type (AWGN) and the power of the environmental noise,
  • the max Doppler shift

Each mobile device was directly connected to the Keysight Source Management Unit instead of the battery and supplied with 5V DV voltage. This set-up was offering a flexible configuration to meet the power sourcing and analysis requirements.

Screenshot_1

For the orchestration of the experimental process as well as the configuration of the UXM and SMU devices the TRIANGLE testbed used the Keysight KS8400A Test Automation Platform (TAP). This software was enabling a powerful, flexible and extensible test sequence and test plan creation.

The mobile devices were controlled via the Quomation WebDriver, a test automation framework for use with native, hybrid and mobile web apps. Furthermore, the Quamotion Frontend provided a device monitoring and controlling interface. The mobile devices used in this experiment were the Samsung Galaxy S7 and Samsung Galaxy S4.

 

Advertisements

Special Issue for IEEE MultiMedia Magazine

I am co-guest editing this special issue. You are invited to submit your research work. Please do not hesitate to contact me if there are questions.

Harilaos

Special Issue for IEEE MultiMedia

Advancing Multimedia Distribution with SDN/NFV-based Systems

Submission deadline: September 1, 2016
Publication: July-September, 2017

 

Recently, software-enabled appliances that support network virtualization have seen wide adoption by data centers and service providers. The main advantage of Software Defined Networking (SDN) is that the control plane is separated from the data plane, thereby allowing for network visibility, dynamic resource provisioning, and flexible service deployments. Complementary to SDN, Network Function Virtualization (NFV) focuses on virtualization of software-based network node functions, such as firewalls and intrusion detection. The increased agility and flexibility in the SDN/NFV-based network infrastructure opens up endless possibilities for multimedia content delivery and distribution. For example, virtual content delivery nodes can be dynamically deployed at the edge of the network in an on-demand fashion, potentially reducing energy costs and improving user satisfaction. This special issue aims to showcase and disseminate original research works that explore how multimedia content distribution can be supported and enhanced over SDN/NFV-based networks. Topics of interest for this special issue include, but are not limited to:

  • SDN/NFV-related architectures for next generation multimedia networks and services
  • Dynamic resource provisioning and management for SDN/NFV-based multimedia systems
  • QoE assessment for multimedia delivery over SDN/NFV-based multimedia systems
  • QoE management with dynamic policy deployments over SDN/NFV-enabled networks
  • Algorithms for live analysis and monitoring of audiovisual streaming quality in that correlates with network events in SDN or NFV-based multimedia systems
  • Real-time video adaptation algorithms over SDN/NFV-enabled networks
  • HTTP-based adaptive video streaming over SDN/NFV-enabled networks
  • Interactions between SDN/NFV architectures with content delivery networks (CDNs)
  • Applicability of SDN/NFV in mobile networks for multimedia provision
  • Integration of SDN/NFV-enabled mobile and fixed networks for multimedia services
  • Composition and deployment of Service Function Chains (SFCs) of video and/or VoIP services

Submission Guidelines

See https://www.computer.org/web/peer-review/magazines for general author guidelines. Submissions should not exceed 6,500 words, including all text, the abstract, keywords, bibliography, biographies, and table text. Each table and figure counts for 200 words.

Manuscripts should be submitted electronically (https://mc.manuscriptcentral.com/mm-cs), selecting this special issue option.

 

Guest editors

Xiaoqing Zhu, Cisco Systems Inc., USA

Harilaos Koumaras, NCSR Demokritos, Greece

Mea Wang, University of Calgary, Canada

David Hausheer, Technische Universitaet Darmstadt, Germany

 

P2MNet 2010 6th IEEE International Workshop on Performance and Management of Wireless and Mobile Networks

CFP – P2MNet 2010
Fifth IEEE International Workshop on
Performance and Management of
Wireless and Mobile Networks
http://paradise.site.uottawa.ca/p2mnet2010/about.html

held in conjunction with

The 35th IEEE Conference on Local Computer Networks (LCN’10) October 11-14, 2010, Denver, Colorado, USA

Submission deadline: May 5th, 2010
In Recent years, wireless and mobile communication systems have become increasingly popular as an inexpensive and promising means for ubiquitous communications. However, without creating a new set of issues and trade-offs. The performance and resource management of wireless and mobile communication systems are becoming a very crucial phase for future generation of wireless and mobile networks.

This workshop focus upon the design, performance and resource management of wireless and mobile networks.

Topics of interest include (but are not limited to):

– Performance and management of mesh and Bluetooth networks
– Performance and resource management of wireless ad hoc and sensor networks.
– Performance and management of wireless multimedia systems. – Resource allocation.
– Power and energy management.
– Wireless network design and management.
– Measurement, analysis, modeling, and generation.
– Traffic measurement (wireless TCP, etc..).
– Billing and wireless web services.
(3G/4G wireless billing systems, GPRS billing, etc..).
– Web-Based wireless management.
– Wireless security management.
– Wireless Network planning.
– Wireless Intelligent Networks.
– Performance and Management of Broadband Wireless Management. – Management and monitoring of wireless and mobile systems.
– User mobility modeling
– Topology construction and maintenance
– Interoperability, and performance of heterogeneous wireless networks of different standards
– PerManet Paper submission related dates:

IMPORTANT DATES:

Full Paper submission deadline: May 5th, 2010.
Notification of acceptance: June 30th, 2010.
Camera-ready paper due: July 28th, 2010.
Author registration by: July 28th, 2010.

Submission:

Authors are required to submit their papers through LCN 2010 EDAS. The following EDAS web site (http://edas.info/N8626) will take you directly to P2MNet Web-site submission.

The authors of accepted papers must guarantee that their paper will be presented at the conference. At least one author of each accepted paper must be registered for the conference, in order for that paper to appear in the proceedings and to be scheduled for presentation.

All papers will appear in the IEEE LCN Conference proceedings.

For more information contact the Program Co-Chairs

Prof. Lynda Mokdad or Dr. S. Samarah