Performance Testing of Mobile Chatbot Applications

In CAMAD 2018, a paper (PDF) that I co-authored introduces a new metric related to the Service Stability of mobile Chatbot Applications. The service stability metric is directly dependent to the following observed issues:

  • Image Loss,
  • Message Repetition,
  • Message Reorder and
  • Auxiliary Keyboard Loss.

Each of these observed issues has a different bias in the calculation formula of service stability. In the following equation the N represents the total number of steps in each scenario and m represents the number of successfully executed steps. The variable Ii is equal to 1 for each step that is observed the corresponding issue and equal to 0 elsewhere.

Each test score is normalized to the maximum score that can be achieved for each chatbot, so the best value for the service stability is 1 and the worst is 0. This metric provides an insight of the QoE that the user experiences under different reception conditions, quantifying the impact of the various impairments to the final service provision.

For the experimental needs of the paper, the proposed metric was validated based on experimental data retrieved by the 5G-TRIANGLE experimental testbed. Therefore, three different types of chatbots over Viber platform used for the execution of the experiment, each one having a different degree of complexity and requirements. The following Table summarizes the features of each chatbot.

Chatbot No. Features
Static Messages Database Communication API-based Communication
Chatbot #1
Chatbot #2
Chatbot #3

For the execution of the experiment it was mandatory to use the commercial application of Viber, via which the user would have access to the chatbot service. For the emulation of cellular network, the TRIANGLE testbed is using the UXM Wireless Test Platform device by Keysight. This device is capable of modifying a number of parameters of the wireless physical layer and thus emulate various network conditions. The parameters that modified in this experiment are:

  • the number of Downlink and Uplink Physical Resource Blocks (PRBS),
  • the number of Downlink and Uplink Subframes,
  • the Multipath Fading Propagation Conditions (EPA, EVA, ETU),
  • the antenna output power,
  • the mode, the type (AWGN) and the power of the environmental noise,
  • the max Doppler shift

Each mobile device was directly connected to the Keysight Source Management Unit instead of the battery and supplied with 5V DV voltage. This set-up was offering a flexible configuration to meet the power sourcing and analysis requirements.

Screenshot_1

For the orchestration of the experimental process as well as the configuration of the UXM and SMU devices the TRIANGLE testbed used the Keysight KS8400A Test Automation Platform (TAP). This software was enabling a powerful, flexible and extensible test sequence and test plan creation.

The mobile devices were controlled via the Quomation WebDriver, a test automation framework for use with native, hybrid and mobile web apps. Furthermore, the Quamotion Frontend provided a device monitoring and controlling interface. The mobile devices used in this experiment were the Samsung Galaxy S7 and Samsung Galaxy S4.

 

Advertisements

Participating in the annual review of H2020 SMESEC project

On 5 July 2018 I had the chance to participate in the annual review of H2020 SMESEC project at FORTH in Heraklion, Crete as independent expert of EU. SMESEC aims at providing a unified security framework for Small Medium Enterprises (SME). Since SME’s are one of the most important drivers for innovation, but they often tend not properly to plan their cybersecurity defence, either by underestimating the risks and consequences of cyber attacks or by not being capable of keeping pace with the progress in this ever-evolving field, the main goal of SMESEC is to identify what are the needs from the SME perspective and translate them into requirements for a unified framework, which will eventually consist of the SMESEC partners’ contributed products.

The SEMSEC products can cover a wide range of security market segments, and it is expected that the unification will bring even higher added value to the products and the project framework itself.

It was really an interesting review meeting, which showcased the added value of SMESEC framework. I would like to thank the consortium for the hospitality and the great organization of the meeting.

 

Presenting the ICT-17 5GENESIS project at the EuCNC 2018 in Ljubljana

In EuCNC 2018 in Ljubljana, I had the opportunity to present the ICT-17 5GENESIS project in a special session of the 5G-PPP for the phase 3 projects and as an introduction to the forthcoming ICT-19-2019 call “Advanced 5G validation trials across multiple vertical industries”. 5GENESIS will provide large scale prototype 5G testbeds in five European cities (Athens, Limassol, Malaga, Berlin and Surrey). These testbeds will be used by representatives of relevant business sectors (e.g. media and entertainment, e-health, automotive etc.) to validate the technological advances that 5G brings to these markets.

In this context, the main goal of 5GENESIS will be to validate 5G KPIs for various 5G use cases, in both controlled set-ups and large-scale events. This will be achieved by bringing together results from a considerable number of EU projects as well as the partners’ internal R&D activities in order to realise an integrated End-to-end 5G Facility.

The 5GENESIS Facility, as a whole, will:

  • Implement and verify all evolutions of the 5G standard, via an iterative integration and testing procedure;
  • Engage a wide diversity of technologies and chain innovations that span over all domains, achieving full-stack coverage of the 5G landscape;
  • Unify heterogeneous physical and virtual network elements under a common coordination and openness framework exposed to experimenters from the vertical industries and enabling end-to-end slicing and experiment automation; and
  • Support further experimentation projects, in particular those focused on vertical markets.

The five platforms of the 5GENESIS Facility, and their main features/orientation, are:

  • The Athens Platform. An edge-computing-enabled shared radio infrastructure (gNBs and small cells), with different ranges and overlapping coverage that are supported by an SDN/NFV enabled core, to showcase secure content delivery and low latency applications in large public-events.
  • The Málaga Platform. Automated orchestration and management of different network slices over multiple domains, on top of the 5G NR and fully virtualised core network to showcase mission critical services in the lab and in outdoor deployments.
  • The Limassol Platform. Radio interfaces of different characteristics and capabilities, combining terrestrial and satellite communications, integrated to showcase service continuity and ubiquitous access in underserved areas.
  • The Surrey Platform. Multiple radio access technologies that can support massive Machine Type Communications (mMTC), including 5G NR and NB-IoT, combined under a flexible Radio Resource Management (RRM) and spectrum sharing platform to showcase massive IoT services.
  • The Berlin platform: Ultra dense areas covered by various network deployments, ranging from indoor nodes to nomadic outdoor clusters, coordinated via advanced backhauling technologies to showcase immersive service provisioning.

Satellite Communications in the 5G Era

Our book chapter entitled “NFV-based Scenarios for Satellite -Terrestrial Integration” by H. Koumaras, G. Gardikis, Ch. Sakkas, G. Xilouris, V. Koumaras, M. A. Kourtis was published by IET at the book “Satellite Communications in the 5G Era” (Editors Shree Krishna Sharma, Symeon Chatzinotas and Pantelis-Daniel Arapoglou) (link to book site).

sat-com-120[1]

The book explores promising scenarios for 5G Satellite Communications (SatCom), novel paradigms for satellite-terrestrial integration and emerging technologies for the next generation of satellite systems and hybrid/integrated satellite-terrestrial systems, and focuses on recent research efforts towards 5G and beyond.

Our contribution focuses on describing NFV-based scenarios for satellite-terrestrial integration.