HOW TO PREVENT TECHNICAL ISSUES IN LARGE MULTIPARTY MEDICAL VIDEOCONFERENCING

Shunta Tomimatsu MS1, Kuriko Kudo PhD1, Taiki Moriyama MD, PhD1, Tomohiko Moriyama MD, PhD1,2, Tomoaki Taguchi MD, PhD3, Shuji Shimizu MD, PhD1

1 Telemedicine Development Center of Asia, Kyushu University Hospital, Fukuoka, Japan.
2 Department of Medicine and Clinical Science, Graduate School of Medical Sciences, Kyushu University, Fukuoka, Japan.
3 Department of Pediatric Surgery, Graduate School of Medical Sciences, Kyushu University, Fukuoka, Japan.


Abstract
Introduction: Videoconferencing (VC) is useful for physicians who need to learn about many cases without moving from one institution to another. However, this advantage can be hampered by technical issues. This study aims to analyse the factors relating technical support that cause technical issues in regular multiparty medical VC to provide high-quality VC to meet participants’ demands. Methods: The study includes large multiparty VC between the Kyushu University Hospital Department of Paediatric Surgery and different institutions within Japan that were held from September 2014 to January 2017. Technical tests, a “previous-week test” and a “last-hour test,” were conducted for checking conditions prior to the VC. The chi-square test was used for factors: participation for previous-week and last-hour test, and attendance by an engineer VCs in each participating institution. A questionnaire survey was distributed among the participants to collect feedback on the quality of VC, ease of preparation and necessity of previous-week testing. Results: Participation in the last-hour test (p=0.002) and the presence of an engineer (p=0.049) significantly decreased overall technical issues. The last-hour tests significantly decreased disconnection (p=0.015) and audio (p=0.019) issues. The engineer’s attendance decreased content-sharing issues (p=0.027). Participants reporting “very good” and “good” audio and visual quality were 92% (109/118) and 96% (105/110). Eighty-three percent of participants (82/99) found the preparation “very easy” or “easy”; while 61% (63/103) found the previous-week test, “unnecessary.” Conclusions: Based on our study, “engineers’ attendance” and “last-hour” technical testing significantly reduced technical problems; these factors help provide high-quality output VC and meet the needs of the participants.

Keywords: videoconferencing; distance learning; education; technology; paediatrics

J Int Soc Telemed eHealth 2018;6:e10

Introduction

In comparison with in-person meeting, videoconferencing (VC) provides benefits for participants in terms of saved time and expenses for transportation especially those in institutions with great geographical barriers.1-3 Participants do not need to travel away from their institutions to learn from the experts. In the medical field, this can be very useful for physicians’ education because they can share their knowledge and opinions, or even study and be reached with this kind of advanced technology that has developed remarkably in recent years4-9 However, preparation for VC is more complicated than that for an in-person meeting because all participating institutions need to coordinate and maintain the videoconference equipment, Internet, and audio and visual devices which may be different between institutions for participating in VC.3,10 Lack of preparation will cause technical issues, and these “issues” affect the host and all the participating institutions. An example of this is when a loud noise from one site interferes with the discussion. Therefore, technical issues should be decreased or identified promptly and managed for the purpose of improving the quality of VC.

Technical issues will be more serious when large multiparty VC is organized. Because each institution has the possibility to individually be the source of the technical issue, the risk of technical issues arising is related to the number of participating institutions.11 Furthermore, it’s difficult for a remote support to grasp all technical situations in participating institutions due to the variety of hardware and Internet. The simplest solution for technical issues in VC could be decreasing the number of participating institutions, but it spoils the benefits of VC. Therefore, the management of technical issues in large multiparty VC should be considered.

The Telemedicine Development Center of Asia (TEMDEC) at Kyushu University Hospital has extensive experience in hosting high-quality VC. As of January 2017, TEMDEC has done 703 VCs between 57 countries in the medical field.12,13 In this study, regular monthly VCs (connecting 21 Japanese institutions) were analysed to investigate the factors that contribute to technical issues, so as to prevent them.

Methods

The subject of this investigation was a regular monthly VC in the field of paediatric cancer that was held 30 times from September 2014 to January 2017. These VCs were organized for the purpose of sharing the experiences of 21 institutions that cared for paediatric cancer patients in the Kyushu-Okinawa region. Kyushu University Hospital, one of the Paediatric Cancer Based Hospitals in Japan, was identified by the secretariat to lead other institutions. The VC was one of the activities of this secretariat. There was a demand for VC among these institutions for two reasons: first, paediatric cancer cases are rare, and only comprise 0.2 % of all cancers, and not all institutions in provincial cities had enough cases; second, there are geographic barriers between institutions - Okinawa Prefectural Nanbu Medical Centre and Children's Medical Centre, located in Okinawa Island (a separate island from Kyushu), more than 850 kilometres away from Kyushu University Hospital, was the farthest institution.14 This regular VC started with 19 institutions at the beginning. Two institutions joined it at the 11th and the 13th event, respectively. All these institutions were advised to participate in all events.

Vidyo, which was developed based on H.264 compression and scalable video coding by Vidyo™ (Hackensack, NJ) was used for this regular VC.4,15,16 Each participating institution installed the VidyoDesktop videoconference software for the Vidyo system on their own personal computers (PCs). Individual accounts, which were allowed to access the videoconference server (VidyoPortal) located in Kyushu University Hospital, were issued to each participant. After login to VidyoPortal, participants could enter the virtual conference room for the VC. The Internet lines were commercial network or national research and education network (NREN) installed in each institution. The cameras used were commercially marketed webcams or laptop built-in cameras. The Vidyo recommended speakerphone, Chat 50 by ClearOne Communications Inc. (Salt Lake City, UT), was used by all participants as the audio device. All institutions were required to connect from a conference room and turn off microphones while they were not speaking to prevent any background noise. One or few physicians attended at each institution. Each institution was given the opportunity to moderate and present in the VC. Presentation materials were displayed in the VC via the “Content Sharing” function of Vidyo. In addition, all institutions were required to arrange for their engineer to provide support in setting up the needed equipment and connections during tests and VC. The engineers in this study were not necessarily specialists in videoconference or information technology; their main responsibility was to take care of the technical aspects of VC to ensure high quality and smooth flowing VC for the physicians.

TEMDEC supported the testing at each institution, for both the “previous-week test” and the “last-hour test” as detailed in Table 1.

Table 1. Profile of the two different tests.

Support Item

Detail

Previous-week test

  • Two-hours’ test was held 1 week before each VC.
  • Purpose was to check that everything worked well and learn how to control the videoconference system.
  • Moderating institution for next conference was required to participate.
  • Participation by others was voluntary, but institutions that previously reported difficulty were invited.
  • Participants were required to connect from the same environment as the actual event.

Last-hour test

  • All institutions were requested to start connection 1 hour before the VC.
  • Each engineer were required to check condition by communicating TEMDEC.

Technical issues such as: “participants could not see or hear the VC at their institution,” “participants missed an opportunity to speak,” or “an institution disturbed the proceeding of VC,” were defined and listed in Table 2.4-6,10,11,17 “Participation / absence” in VC, previous-week test and last-hour test in each event and technical issues during the VC were tallied. In addition to the classified technical issues, overall issue, which is when an institution had one or more types of technical issue during a VC, was defined.

Table 2. Technical issues.

Issue

Detail

Disconnection

Sudden disconnection from virtual conference room.

Visual

Failure to transmit the camera image.
No camera image from other institutions on the screen.

Audio

Sound feedback from other institutions.
Too loud / low mic volume compared with other institutions.
Noise over voice audio.
Failure to transmit sound.
No sound from all other institutions.

Login

Unavailable to login to VidyoPortal with non-server problem.
Unavailable to find the virtual conference room.

Content-Sharing

Not knowledgeable of how to share presentation material during the VC.
Interference with other institutions’ Content Sharing.
No presentation image from other institutions on the screen.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

The relationships between the previous-week test and technical issue and the last-hour test and technical issue were analysed. To analyse the relationship between the previous-week test and technical issue, institutions that participated in in each VC event were separated into two groups: “participation in the test” and “absence from the test.” The total number of “technical issues” and “non-issues” in the two groups were compared.

In the analysis of the relationship between last-hour test and technical issue, institutions that connected before the VC starting time in each event or were late for the conference starting time because of any technical issue despite having announced their participation in the test, were counted as “participating in the test.” The total number of “technical issues” and "non-issues” in the two groups were compared. The total numbers of “technical issues” and “non-issues” were compared with attendance by an engineer. Although all sites were required to arrange with an engineer to attend every test and VC, because of organizational structure and situations some physicians took care of the technical matters by themselves. Given this background, attendance and non-attendance by an engineer were used as factors influencing on technical issues.

Data were analysed using JMP Pro 11.0.0 by SAS Institute (Cary, NC) with the chi-square test. Significance was accepted at the p < 0.05 level. When an institution had a login issue and could not participate in the VC they were counted as having participated in the VC only for login and overall issue of previous-week test and engineer attendance analysis. This is because they could participate if they did not have any technical issues. However, these cases weren’t counted for last-hour test analysis because there was no data on whether the login issue occurred during or after last-hour test. The reason these cases were excluded from other issues was that there were no possibility for other issues to occur.

A questionnaire survey was held at the first six events. Physicians and engineers answered questions and the results were analysed. Details are shown in Table 3, and all questions were optional. There was a free description in the option “other” for the last question.

Table 3. Questionnaire survey.

Questions

 

How was audio quality?

very good

good

poor

very poor

How was image quality?

very good

good

poor

very poor

How was the preparations?

very easy

easy

hard

very hard

Is previous-week test necessary for future VCs?

necessary

unnecessary

other

Results

Participating institutions
The number of participating institutions and participation ratio for the previous-week test and last-hour test among institutions connecting for each VC is shown in Figure 1.

Figure 1
Figure 1. Number of participating institution for each event and participating ratio of the previous-week test and last-hour test.

A total of 30 VC were included in the study, and each event is labelled accordingly in Figure 1. At the same time, the number of participating institutions and participation ratio of previous-week test and last-hour test were also plotted in the graph. The average number of participating institutions was 16.7, wherein the maximum and minimum numbers were 19 and 13, respectively. 

Although the “previous-week test” participation ratio shows 95% at the first event, which was held in July 2014, this dropped to as low as 44% at the second event, and remained lower than 40% after the third event, except for the 18th VC, in which all institutions were strongly urged to participate in the previous-week test because of changing specifications for VidyoPortal. Therefore, the ratio for the 18th VC was 71%. Some institutions participated the previous-week test spontaneously, especially when they made changes to their network, devices, or engineer.

The participation rate in the last-hour test was more than 80% of institutions except for the 7th and 22nd VC. All institutions that did not participate in the last-hour test also did not participate in the previous-week test in the same month.

Table 4 shows the total number of “participation / absence” in the previous-week test and last-hour test, and institutions which had or did not have technical issues during the VC. The ratio of overall (p = 0.002), disconnection (p = 0.015) and audio issues (p = 0.019) in the last-hour test “participation” group were significantly lower than in the “absence” group.

Table 4. Total number of each test and technical issues.

Type of issue

Previous-week test

Last-hour test

Participation

Absence

p =

Participation

Absence

p =

Overall

issue

18

(15%)

60

(16%)

0.939

65

(14%)

10

(34%)

0.002

non-issue

100

(85%)

326

(84%)

407

(86%)

19

(66%)

Disconnection

issue

4

(3%)

19

(5%)

0.476

19

(4%)

4

(14%)

0.015

non-issue

114

(97%)

364

(95%)

453

(96%)

25

(86%)

Visual

issue

1

(1%)

11

(3%)

0.209

10

(2%)

2

(7%)

0.102

non-issue

117

(99%)

372

(97%)

462

(98%)

27

(93%)

Audio

issue

7

(6%)

17

(4%)

0.507

20

(4%)

4

(14%)

0.019

non-issue

111

(94%)

366

(96%)

452

(96%)

25

(86%)

Login

issue

1

(1%)

13

(3%)

0.145

9

(2%)

2

(7%)

0.075

non-issue

117

(99%)

373

(97%)

463

(98%)

27

(93%)

Content Sharing

issue

6

(5%)

8

(2%)

0.084

14

(3%)

0

(0%)

0.347

Engineer attendance
Table 5 shows the total number of institutions that had or did not have technical issues during VC, and engineer “attendance / non-attendance” in the VC. The ratio of overall issues (p = 0.049) and content-sharing issues (p = 0.027) in the engineer “attendance” group were significantly lower than in the “non-attendance” group.

Table 5. Total number of engineers in attendance and technical issues. (*=p<0.05)

Type of

Issue.

Engineer

Attendance

Non-attendance

p =

Overall

issue

47

(13%)

31

(20%)

0.049*

non-issue

304

(87%)

122

(80%)

Disconnection

issue

12

(3%)

11

(7%)

0.062

non-issue

337

(97%)

141

(93%)

Visual

issue

8

(2%)

4

(3%)

0.819

non-issue

341

(98%)

148

(97%)

Audio

issue

17

(5%)

7

(5%)

0.898

non-issue

332

(95%)

145

(95%)

Login

issue

10

(3%)

4

(3%)

0.883

non-issue

341

(97%)

149

(97%)

Content Sharing

issue

6

(2%)

8

(5%)

0.027

non-issue

343

(98%)

144

(95%)

Questionnaire
Results of questionnaires are shown in Figures 2 and 3.

Fig2
Figure 2.
Questionnaire responses on (a) audio quality and (b) image quality.

Figure 3
Figure 3.
Questionnaire responses on (c) difficulty of the preparation for event and (d) necessity of the previous-week test.

There were 127 respondents throughout the first six events. The total number of “very good” and “good” were more than 90% for audio quality (92%, 109/118) and visual quality (96%, 105/110). Furthermore, no one answered “very poor” to either question. Respondents answered that 83% (82/99) of answers were “very easy” or “easy” for preparation. Regarding the previous-week test, 61% (63/103) of participants answered that they were “unnecessary.” People who answered “other” left comments such as “It’s enough to participate in just the last-hour test” or “I need to participate when the environment on my site has changed.”

Discussion

This study evaluated three elements that can help to avoid various technical issues during VC.

Overall issues with the last-hour test in the “participation” group was significantly lower. In the analysis of groups with individual technical issues, disconnection and audio issues were significantly lower when institutions participated in the last-hour test. Because most attendees answered that they found the preparation to be easy work, it was suggested that checking each institution’s condition just before the VC leads to quality improvement by decreasing overall technical issues with minimal problems. Disconnection issues seems to be decreased by the observation time of last-hour test. Running PC tasks by other applications or automatic operating system update, which does not always appear immediately, sometimes interferes with VidyoDesktop operation. If there is more time just before the actual VC, it is possible not only to find but also manage this kind of issue before the event starts. With regard to audio issues, it seems that active checking by each remote institution helped to decrease it. All attendees were always able to recognize whether their reception and image transmission during VC went well or not. However, there were no opportunities to check their audio transmission if attendees are only passive. If institutions failed to participate in the last-hour test, the next chance to check their audio transmission was during the actual VC. Although content-sharing requires attendance for active checking as well, there was no significant difference between participation and absence groups of last-hour test. However, more information and analysis could better clarify these issues in future studies if the categories of transmission and reception were recorded for each classified issue as well as the incidence of content-sharing issues between attending and non-attending engineers were gathered.

Attendance by an engineer during a VC significantly decreased overall issues (p = 0.049). Separating the roles in a teleconference can help facilitate the smooth flow of VC and reduce interruptions. Simultaneously, the physician/speaker is able to focus more on their presentation, rather than worrying about troubleshooting, which in the end will benefit all VC participants. This may also have contributed to the significant decrease in content-sharing issues (p = 0.027) because disruptions are lessened owing to the control by the skilled engineer during VC. Expertise and engineers’ resources are very useful, particularly when handling the technical aspects of a videoconference system, which may be too complicated for a physician or presenter.18 Dedicating an established team of network engineers or staff who are adept with the equipment or system of an institution is a more practical approach than training multiple staff for every department.

For the previous-week test, no significant differences were obtained from the study. Furthermore, the participation ratio was lower compared with that of the last-hour test; while in the questionnaire study, the majority answered that the previous-week test was unnecessary for future VC. It is presumed that preparation for the previous-week testing might be too tedious for these centres, considering the availability of the requirements: venue, equipment, and manpower. It was also noted that some institutions participated in the previous-week test spontaneously whenever there were system or equipment changes at their institutions. The provision of additional opportunities for the institutions to freely participate and check their settings prior to the actual conference makes the previous-week test worth holding.6

There are some limitations in this study. Although it is suggested that an engineer be present for technical support during VC, it is still the institution’s prerogative to establish such systems, bearing in mind that the organisational structures and circumstances are unique in each centre. The applicability of the approach depends on each centre; hence, it is still a debatable issue. In this study, the sample was a regular monthly VC with a fixed group of Japanese institutions. It is possible that the results would differ if the conditions of sample were changed. Are there any additional factors to consider when a VC is held between foreign countries, such as network conditions, time differences, or language barriers? 4,19-21 Do participants remember how to set up and control the videoconference system if they use it only once a year? Is it possible to participate in tests or arrange for an engineer every time if the VC is held on short notice? These and other possible technical issues can be resolved by the best design and solutions in future studies and investigations.  

The effect of technical issues for interactive VC cannot be understated; the VC is interrupted easily by loud noises, even when originating from only one institution. Such issues have a variety of sources: accidents with devices, incorrect settings on the PC, network breakdown, or human factors. Because of multiple variations, each site has the potential to create/produce/cause technical issues. Therefore, there is an increased possibility of more technical issues when there is a larger number of connecting institutions. Interruptions caused by technical issues decrease the benefit of VC. Issues that can be anticipated should be prevented so as to increase the value of VC. Although the responses to the questionnaires about audio and image quality were mostly positive, there is much room such as previous-week test, last-hour test and engineer attendance for improvement with regard to quality of VC.


Corresponding author:
Shunta Tomimatsu
Telemedicine Development Center of Asia
Kyushu University Hospital
3-1-1 Maidashi, Higashi-ku
Fukuoka 812-8582
Japan
E-mail: s-tomi@tem.med.kyushu-u.ac.jp

Conflict of interest. The authors declare no conflicts of interest.

Acknowledgements. The authors are grateful for the cooperation of all the medical and engineering staff who participated in the regular VC. We also thank the secretariat of the Pediatric Cancer-Based Hospitals in Kyushu University Hospital for organizing the VC. This work is supported by JSPS Core-to-Core Program, B. Asia-Africa Science Platforms, and KAKENHI Grant Number 15K08556 and 16H02773. We thank Scott Wysong, ELS, from Edanz Group (www.edanzediting.com/ac) for editing a draft of this manuscript.

References

  1. Rupert DJ, Poehlman JA, Hayes JJ, Ray SE, Moultrie RR. Virtual versus in-person focus groups: Comparison of costs, recruitment, and participant logistics. J Med Internet Res 2017;19:e80.
  2. Karis D, Wildman D, Mané A. Improving remote collaboration with video conferencing and video portals. Hum Comput Interact 2016;31:1-58.
  3. Freeman M. Video conferencing: A solution to the multi-campus large classes problem? Br J Educ Technol 1998;29:197-210.
  4. Brooks M. Feasibility of developing a pediatric telehealth network in Honduras with international consultation support. Rural Remote Health 2017;17(2):3965.
  5. Foronda C. Graduate Nursing students’ experience with synchronous, interactive videoconferencing within online courses. Q Rev Dis Educ 2014;15(2):1-8;69-70.
  6. Banbury A, Parkinson L, Nancarrow S, et al. Multi-site videoconferencing for home-based education of older people with chronic conditions: the telehealth literacy project. J Telemed Telecare 2014;20(7):353-359.
  7. Chipps J, Ramlall S, Mars M. Videoconference-based education for psychiatry registrars at the University of KwaZulu-Natal, South Africa. Afr J Psychiatry 2012;15:248-254.
  8. Cameron M, Ray R, Sabesan S. Physicians' perceptions of clinical supervision and educational support via videoconference: A systematic review. J Telemed Telecare 2014;20:272-281.
  9. Cameron M, Ray R, Sabesan S. Remote supervision of medical training via videoconference in northern Australia: A qualitative study of the perspectives of supervisors and trainees. BMJ Open 2015;5:E006444.
  10. Karaman S. Virtual classroom participants’ views for effective synchronous education process. Turk Online J Dis Educ 2013;14(1):290-301.
  11. Eady MJ. Employing the EPEC hierarchy of conditions (version II) to evaluate the effectiveness of using synchronous technologies with multi-location student cohorts in the tertiary education setting. Int Rev Res Open Dis 2017;18(3).
  12. Shimizu S, Kudo K, Antoku Y, et al. Ten-year experience of remote medical education in Asia. Telemed J E Health 2014;20:1021-1026.
  13. Ho SH, Rerknimitr R, Kudo K, et al. Telemedicine for gastrointestinal endoscopy: The Endoscopic Club E-conference in the Asia Pacific Region. Endosc Int Open 2017;5:E244-E252.
  14. Yang L, Fujimoto J. Childhood cancer mortality in Japan, 1980-2013. BMC Cancer 2015;15:446.
  15. Kudo K, Shimizu S, Chiang TC, et al. Evaluation of videoconferencing systems for remote medical education. Creat Educ 2014;5:1064-1070.
  16. Minh CD, Shimizu S, Antoku Y, et al. Emerging technologies for telemedicine. Korean J Radiol 2012;13:S21-S30.
  17. Marlow J, Carter S, Good N, Chen JW. Beyond talking heads: multimedia artifact creation, use, and sharing in distributed meetings. Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing 2016;1703-1715.
  18. Harnett B. Telemedicine systems and telecommunications. J Telemed Telecare 2006;12(1):4-15.
  19. Moes NCCM, Velasquez A, Leung WS, Papadopoulou AE. Development of design competences for international team-based cyber physical systems. Proceedings of the 10th International Symposium on Tools and Methods of Competitive Engineering 2014;461-474.
  20. Oertig M, Buergi T. The challenges of managing cross-cultural virtual project teams. Team Perform Manage 2006;12:23-30.
  21. Caniglia G. Transnational collaboration for sustainability in higher education: Lessons from a systematic review. J Clean Prod 2017;168:764-779.

Tomimatsu S, et al, J Int Soc Telemed eHealth 2018;6:e10