PHYSICIAN USER PERSPECTIVES IN THE PRACTICE OF TELEMEDICINE IN THE PHILIPPINES

Paul M Pasco MD1,2

1 National Telehealth Center, National Institutes of Health, University of the Philippines Manila
2 Department of Neurosciences, College of Medicine, University of the Philippines Manila


Abstract
Telemedicine can bridge gaps in healthcare by providing expert opinion to health care providers in remote areas, but has remained underutilised locally. Objectives: To determine how the National Telehealth Center, Manila, Philippines, can manage user physician expectations of telemedicine in order to maximise its benefits, by determining which factors influence physicians to adopt and use telemedicine. Methods: A questionnaire was adapted from the Unified Theory of Acceptance and Use of Technology model of technology acceptance and sent to rural physicians. Focus group discussions were also held among key informants to determine their perceptions and attitudes towards telemedicine. Results: Social influence was the most important factor that influenced intention to use telemedicine, and with other factors studied (performance expectancy, effort expectancy, attitude, and facilitating conditions) explained 54.8% of the variance in the decision to use telemedicine. Major findings that emerged from the group discussions were the inadequacy of the present telecommunication infrastructure, the need to have specialists always available to answer referrals, while retaining e-mail and SMS for elective referrals. Conclusion: Practical steps such as transmission of constant reminders, improvement of the existing support system, and recruitment of thought leaders can increase the use and adoption of telemedicine among its target user physicians.

Keywords: telemedicine; telehealth; technology acceptance; UTAUT

J Int Soc Telemed eHealth 2016;4:e26

Introduction

Telemedicine is the use of information and communication technologies for the management of patients from a distance, and its origins are thought to arise from NASA’s manned space flight programme, when there was a need to relay biomedical information and communication in space, as well as from various initiatives to transmit radiographs and broadcasts of medical procedures during the 1950s and 1960s.

In the Philippines, growth in telemedicine began with the establishment by the National Telehealth Center (NTHC) of the National Telemedicine Service Project in 1998, which sought to link specialists at the Philippine General Hospital with rural physicians. In the roughly five years that telemedicine services have been available, about three hundred government physicians, mostly in rural areas, have completed a two-day certificate training course.1 When these rural physicians register referrals via e-mail or SMS, these are forwarded to specialists at the Philippine General Hospital, and to regional hospitals such as Baguio General Hospital and East Visayas Regional Medical Center. The specialists then respond to the referrals, usually within a few hours. The referring physicians then act accordingly on the information given in the responses. Based on informal surveys taken after the training, most physicians reported willingness to use the service. However uptake has been quite slow, with only 1-2 referrals received per week, and only 20-30 physicians actively referring cases, out of about 500 that have been trained.1

In a developing country like the Philippines, and given local telemedicine’s roots of being founded and supported by a state-run university, the focus in the local setting has been three fold: how to use telemedicine as a tool for increasing access to quality healthcare (especially for those living in geographically isolated and disadvantaged areas), maintain reasonable costs, and provide acceptable quality of care. Although NTHC has the most telemedicine experience and capabilities of any local institution, other agencies of the Department of Health (DoH), as well as private health care facilities, have also begun to offer telemedicine services. Thus NTHC can use its experience as the pioneer of telemedicine in the local setting to draw lessons and help formulate policies for scaling up this potentially game-changing application in the local healthcare delivery system.

As with any other technological innovation, it is not only technical but also organisational and social aspects that must be considered if telemedicine is to be integrated into routine clinical care. Thus, aside from the effectiveness of telemedicine in terms of patient outcomes and user satisfaction, it has become increasingly clear that user acceptance of information technology (IT) is also equally important. Dunnebeil2 surveyed German physicians and found that the perceived importance of standardization and of current IT utilization were drivers for accepting electronic health services. In the early 2000's Venkatesh used existing models of IT acceptance to formulate a Unified Theory of Acceptance and Use of Technology (UTAUT).3 Eight available models that explained the likelihood of successful introduction of new technologies and understood drivers such as acceptance were integrated based on similarities into a single UTAUT framework. Four constructs were postulated to have a significant role in determining user acceptance: performance expectancy (degree to which one believes using a system will help improve job performance); effort expectancy (degree of ease with using the system); social influence (degree to which one perceives that other important people want one to use the system); and facilitating conditions (degree to which one believes an infrastructure exists to support use of the system).

The UTAUT model has subsequently been validated in several settings. Kohnke at al.4 tried to determine predictors for understanding how patients and clinicians in home care settings in the US state of Michigan accepted healthcare telemedicine equipment They found that performance expectation and social influence significantly predicted behavioural intent, and also that as effort expectancy rises, the level of anxiety increases. Cilliers and Flowerday5 surveyed health workers in South African clinics where telemedicine was used. However, only descriptive analysis was done, so the author could not draw any conclusions regarding factors that affect telemedicine use. They did mention that barriers to effective implementation of a health information system were the lack of knowledge and awareness regarding the telemedicine system, indicating more must be done with regard to the promotion and education of telemedicine. Also in South Africa, Nwabueze et al.6 extended the UTAUT model by incorporating cultural constructs. By surveying health workers, they found that cultural factors affected behavioural intentions only for prospective users, but not for actual users, with the exception of social influence, which was strongly affected by individuality-collectivity. Finally, Kijsanayotin et al.7 used the UTAUT model to study user IT acceptance in a developing country’s healthcare system, and found that community health centre personnel in Thailand had a high level of IT acceptance and use. As predicted by the model, intention to use health IT depended on performance expectancy, effort expectancy, social influence, and voluntariness. Health IT use was influenced by past IT experience, facilitating conditions, and intention to use IT.

Underutilisation of telemedicine services in the Philippines continues, with only one to two referrals received per week, mostly through SMS and occasionally through e-mail. There also remain many gaps in knowledge about user satisfaction with and expectations of telemedicine, and how users will integrate this new technology into their daily routine. Thus it is important to know how to manage expectations of rural physicians who will be using the service, and what can be done to promote the use of telemedicine in the country. At present there is no data on user acceptance and expectations of telemedicine in the local setting of the Philippines. The UTAUT model’s comprehensiveness, together with other studies cited in this review, provide a strong basis for investigating user acceptance of telemedicine, and will add a further developing country perspective to the literature on the subject. Results from this study may also usefully extend the applicability of UTAUT among professionals who work largely independently of any organisation.

The research objectives were to assess the perceptions and attitudes of rural physicians with regard to the use of telemedicine, and to determine if performance expectancy, effort expectancy, social influence, and facilitating conditions are factors which influence clinician expectations of telemedicine.

Methods

The study was initiated after obtaining approval from the IRB of the University of the Philippines Manila. All participants of a telemedicine training programme were requested to answer a questionnaire. The questionnaire was adapted from that used by Venkatesh.3 Only minor changes were made to fit the local context (that is, rural physicians who would be using telemedicine). The questionnaire was unchanged from the original model to validate whether the UTAUT model would be applicable in the study setting. Seventeen items that reflect the significant domains or factors that determine user acceptance of new technology were included, i.e. performance expectancy, effort expectancy, social influence, attitude, and facilitating conditions. The last three items in the questionnaire reflect the respondents’ behavioural intention to use telemedicine.

More in-depth and extended interviews were conducted with key respondents, such as officers and active members of their local medical associations, and previously active users of the telemedicine service. Nine respondents were identified, all members of the “Doctors to the Barrios” training programme of the DoH. By special agreement with the DoH, NTHC undertakes to train these physicians in the use of its telemedicine services. The training consists of a series of lectures on the principles of telehealth, as well as its ethical and legal bases, over one morning; in the afternoon there are practical demonstrations on the use of the SMS and e-mail services wherein the participants actually learn to use the service.

Results and Discussion

The training of rural health physicians in the use of telemedicine was carried out in May 2014 at the Development Academy of the Philippines in Tagaytay City. During these two days a total of 55 physicians attended, and 37 of them consented to answer the questionnaire while 13 consented to participate in the focus group discussions.

Of the 37 questionnaire respondents, males comprised 56%, with the average age being 28.3 years. This may be because the training specifically targeted members of the Doctors to the Barrios (DTTB) programme, which deploys recent medical school graduates to geographically isolated and economically depressed areas. Due to time constraints and the limited opportunities to gather many respondents in one place for common training sessions, only those physicians who attended the telemedicine training sessions were recruited as respondents. The sample of respondents was all young, largely inexperienced physicians. Therefore a true picture of what factors influence adoption of new technologies such as telemedicine by physicians in general may not have been obtained.

Table 1 shows the mean Likert scores obtained from all 37 respondents (from “strongly agree” = 5, to “strongly disagree” = 1). All constructs showed an acceptable level of 0.70 or above. The overall Cronbach’s alpha score for the entire questionnaire was acceptable at 0.826. Thus the questionnaire can be deemed to have good reliability, with each of the constructs being internally consistent.

Table 1: Results of the questionnaire.
Table1

The distribution of responses showed a normal or Gaussian distribution and a single peak around which were clustered the other choices with fewer respondents. This was as expected because this is a previously validated questionnaire with only minor adjustments being made to reflect the local situation.

Since there were three separate items for behavioural intention, it was first deemed necessary to determine which was the best measure of the construct of behavioural intention. To do this, Pearson correlation was performed, with results showing a persistently high correlation of the first item (“I intend to use telemedicine in the next six months”) with all the other factors of performance expectancy, effort expectancy, attitude, social influence, and facilitating conditions. The correlation was higher than that for the other two items and for the three items as a whole. Thus for the subsequent regression analysis, only the first item for behavioural intention was used.

Evaluation of the UTAUT model was done by regression analysis. The outcome or dependent variable, which is the behavioural intention to use telemedicine, was regressed on the five constructs, or independent variables (i.e. performance expectancy, effort expectancy, attitude, social influence, and facilitating conditions).

By Pearson correlation coefficient analysis, the construct of social influence had the highest correlation coefficient with behavioural intention to use telemedicine, with a score of 0.608. On the other hand, after performing univariate logistic regression, inclusion of all the five constructs would produce the regression equation as follows:

Y = 0.005 + 0.111PE – 0.305EE + 0.194AT + 0.092FC + 0.194SI

where y is the behavioural intention. The regression equation was significant, with a p-value of <0.001 and R2 = 0.58.

This is not as high as that reported in the original study by Venkatesh,3 who found that all the factors in his model could explain 70% of the variability in the intention to use telemedicine, as well as some follow-up studies such as that by Kijsanayotin. Possible reasons for this relatively low figure are the small sample size of 37 respondents, and perhaps the existence of some redundant items in the questionnaire. The scoring system could also be improved by adding more items on the scale, instead of the usual five items.

To summarise, the results from the questionnaire show that it has internal consistency and validity. Social influence was the single most important factor that influenced intention to use telemedicine, which was in contrast to other studies such as that by Kijsayanotin7 who showed that performance expectancy was by far the most important influence. All the five constructs taken together can explain 54.8% of the variability in intention to use telemedicine.

Results from the Focus Group Discussion

The questions and responses were as follows:

“How do you refer difficult cases? To whom do you refer? What is your preferred mode of communication? What factors do you think affect how you refer?”

Most respondents refer difficult cases to their peers, or classmates in medical school. This is even if their peers are only starting their specialty training and thus may not be very much more advanced compared to them. A few would consult specialist residents and consultants whom they have befriended during their training. Others would follow the established network, i.e. doctors in a certain village would refer to the specialists residing in the same village or district. These specialists would in any case be the ones who would eventually manage the patients should they be transferred to the referral hospital.

The preferred mode of communication when referring difficult cases was by conversing using a mobile phone, because text messages took too long to compose and wait for a reply. Internet connectivity was also a problem in many areas. But where Internet connection was not a problem, respondents used other forms of social media and communication, such as Facebook or Viber. Being heavily dependent on mobile phone networks and the Internet, NTHC should in turn contribute to building communities or networks where rural physicians can easily get in direct contact with the experts whose advice they may need at any time. NTHC’s expertise in putting physicians and experts in touch with one another through the use of mobile phones or the Internet could play a significant role, as could its ability to organise activities and events where these groups of people could network and become better acquainted.

“What do you think are the advantages/disadvantages with using telemedicine? Do you have concerns regarding confidentiality, or other ethical or legal issues?”

Some focused on the increased efficiency and reduced data loss that the respondents perceived would be the result if they computerised their data collection and other office procedures. However this seems to be a misconception of what telemedicine is, mistaking it for medical informatics, or the computerisation of medical and other health information. The rest of the answers focused on the ease of access to expert medical opinion through the network of specialists that the NTHC would provide.

The question on the ethics and legality of telemedicine was included because of NTHC’s advocacy to increase awareness of the social issues surrounding the use of this relatively new technology. There is no direct doctor-patient relationship between the specialist physician who answers a telemedicine referral and the patient on whose behalf the rural physician makes the referral; the referring physician still has the primary responsibility for patient care, with the specialist only providing their expert medical opinion. True enough, most respondents are conscious of the ethical issues surrounding telemedicine, saying they only disclose patient initials and age when referring via telemedicine.

“Do you plan to use NTHC telemedicine services in the next month? Why or why not?”

Faced with such a direct question, those respondents who had not used the service before all voiced their enthusiasm. But for those who had previously used the telemedicine service, it was instead an opportunity to voice their disappointing experiences, and to verbalise why they would probably not use the service again.

On the other hand, non-emergency image-intensive referrals such as in dermatology and radiology would be suitable for e-mail. In many advanced countries where the use of telemedicine is well established, consults for image-intensive specialties are well developed,8 and include cardiology, neurology, and others.

For those who did not plan to use telemedicine in the near future, some reasons offered were: forgot to save the e-mail address or mobile phone number of the service, or lost the kit and other materials necessary for using the service. Others will not use the service due to the need for speedy resolution, or out of concern that the advice they get from urban specialists may not be practical or useful in the rural context.

“What would convince you to try to use NTHC telemedicine services? Do you foresee any problems with your use of telemedicine?”

The most common suggestion was to improve the basic communication infrastructure, such as mobile phone signals in remote areas. Another oft-repeated suggestion was that NTHC should provide a hotline, especially for emergency referrals. This was unexpected; the possibility of offering “live” services had often been mooted, but was rejected due to the perceived prohibitive costs of maintaining “on demand” service. Given the frequency with which this issue arose, it requires further discussion by NTHC management.

“Would you recommend using telemedicine to other rural physicians like yourself? Why or why not?”

Most respondents seemed to answer cautiously, saying that they would be happy to recommend use of telemedicine if the identified problems were settled first, and their suggestions to improve the general telecommunication infrastructure and to have “live” responders on a hotline were to be considered.

“If you have used telemedicine before, how and why did you use it? How did you integrate it into your daily work routines? What factors do you think could affect your use of telemedicine?”

Feedback was obtained from previous users of telemedicine. This question was deemed especially crucial as there had been no evaluation of the telemedicine programme since it was started 7 years ago. Among those who had used the service before, the experience left much to be desired. One respondent had tried using the service twice before, but both times there was no reply or nor even an acknowledgement that the referral was received. Thus they would prefer a live person on the other end to respond to the call and answer the referral. This respondent, when pressed, admitted a willingness to try to use the service again if such issues were resolved. However in the meanwhile, the respondents would have already built up and strengthened alternative referral systems, such as those already existing in their communities of practice.

As mentioned, some respondents did not or could not continue using the service as they had lost the required e-mail address or phone number for the service. This indicated a need for frequent reminders from the NTHC on how to access the service. The service should also be simple to use, though this was probably not a problem as these young doctors are familiar with mobile phone messaging and the Internet.

Finally, respondents frequently voiced concern that the advice they would get from specialists (more familiar with an urban, tertiary care setting) may not be applicable to what is readily available or practical in a rural setting with limited resources. However, although seemingly a valid concern, the specialists that NTHC recruits are familiar with low resource settings – which include government hospitals and the university hospital where the specialists have their main area of practice. Despite this, management options may indeed be limited in the rural areas, which is why the respondents are often advised to transfer problematic patients to the nearest hospital.

In summary, group discussions revealed the general inadequacy of the current telemedicine service of NTHC. Mobile phone and Internet signals remain a problem in many rural areas, and are probably the single most important reason behind the underutilisation of the service. If communications were not a problem then a “call centre” system with specialists available around the clock to answer referrals should probably be developed. e-Mails can be retained for image-intensive referrals such as for dermatology or radiology, while SMS through mobile phones can still be used for non-emergency consults. Some recent studies9 have also demonstrated the range of barriers and facilitators that impact on the decision to adopt and engage with telemedicine services, such as the role of referrers and need for continued reassurance.

Conclusion

This study assessed the present state of telemedicine services offered by the National Telehealth Center (NTHC) of the University of the Philippines. The objective was to understand how to manage physician user expectations of telemedicine, in order to maximise its use in the local setting. Social influence was the most important factor that predisposed intention to use telemedicine and, with other factors studied (performance expectancy, effort expectancy, attitude, and facilitating conditions), explained 54.8% of the variance in the decision to use telemedicine. Major findings that emerged from the group discussions were the inadequacy of the present telecommunication infrastructure, the need to have specialists always available to answer referrals, while retaining e-mail and SMS for elective referrals. Practical steps such as transmission of constant reminders, improvement of the existing support system, and recruitment of influential endorsers can increase the use and adoption of telemedicine among its target users. Recommendations are presented in Appendix A.


Corresponding author:
Paul Matthew Pasco, MD
National Telehealth Center
University of the Philippines Manila

PGH Compound, Taft Ave
Manila 1000
Philippines

pasco.paul@gmail.com
(63)9275326831

Conflict of interest: The authors declare no conflicts of interest.

References

  1. National Telehealth Center. (2013). Annual Center Report.
  2. Dunnebeil S, Sunyaev A, Blohm I, et al. Determinants of physicians’ technology acceptance for e-health in ambulatory care. Int J Med Inform 2012;81(11):746-760.
  3. Venkatesh V. User acceptance of information technology: toward a unified view. MIS Quarterly 2003;27(3):425-478.
  4. Kohnke A, Cole ML, Bush R. (2009). Incorporating UTAUT predictors for understanding home care patients’ and clinicians’ acceptance of healthcare telemedicine equipment. J Technol Manage Innov 2009;9(2):29-41.
  5. Cilliers L, Flowerday SV. Health information systems to improve health care: A telemedicine case study’. S Afr J Inform Manage 2013;15(1):1-5.
  6. Nwabueze S. Meso PN, Mbarika VW, et al. The effects of culture of adoption of telemedicine in medically underserved communities. Proceedings of the 42nd Hawaii International Conference on System Sciences 2009;1-10.
  7. Kijsanayotin B, Pannarunothai S, Speedie SM. Factors influencing health information technology adoption in Thailand’s community health centers: adopting the UTAUT model. Int J Med Inform 2009;78(6):404-416.
  8. Whitten P, Love B. Patient and provider satisfaction with the use of telemedicine: overview and rationale for cautious enthusiasm. J Postgrad Med 2005;51(4):294-300.
  9. Cook EJ, Randhawa G, Sharp C, et al. Exploring the factors that influence the decision to adopt and engage with an integrated assistive telehealth and telecare service in Cambridgeshire, UK: a nested qualitative study of patient 'users' and 'non-users'. BMC Health Serv Res 2016; 16:137. doi: 10.1186/s12913-016-1379-5

Appendix A.

Recommendations and Limitations.

Other study results can be applied to the practical situation of optimising the use of telemedicine by rural physicians in the local setting. Here, the behavioural intention to use telemedicine is a function of: the perception that telemedicine is useful in their daily work (performance expectancy); ease of use (effort expectancy); that other important opinion makers believe that they should use telemedicine (social influence); that using telemedicine is a good idea and makes work interesting (attitude); and finally that there is a functional support system to assist them and help solve any problems they may encounter (facilitating conditions). The predictive power of these five factors explains more than half of the variability in intention to use telemedicine, which is substantial given that there are other possible influencing factors, such as voluntariness, affect, and compatibility, but which were not evaluated as these were not part of the UTAUT model.

From this, NTHC should recruit for its cause thought leaders from academia, as well as from the local governments and hospitals. These individuals could help reinforce the need to use the telemedicine service, by, for example, sending personalised reminders or being otherwise visible during NTHC activities, or through advocacy. NTHC can also apply other steps making use of the influencing factors mentioned in the UTAUT model. For example, staff members can send frequent tips or reminders on using the service. These reminders should be practical and actionable (for example, “This is to remind you that the e-mail address of NTHC that you can use for referring patients is….”), so that potential users can keep in mind the availability of the service, and use it when the need next arises. These reminders should also emphasise that the service is easy to use, and that it can help them resolve vexing and specific diagnostic or therapeutic management problems right away. The contact numbers of these experts should be readily available to these rural physicians so that they can get in touch with them directly for faster resolution of problems that they encounter.

NTHC should take steps to project the notion that telemedicine can make their work more interesting, by, for example, providing a new way of making referrals and keeping in touch with new developments in the medical field, as well as with colleagues near and far.

Finally, NTHC should invest in a support system that can quickly respond to any concerns that users may raise regarding availability or ease of use of the service. This could consist of having technical staff available around the clock (available to respond to any technical concerns that they may have about using the service), or who at the very least can acknowledge that their referrals were received. NTHC should also consider offering continuous training and technical support so that the target users can be updated about new applications of telemedicine and new services that are available.

Given that mobile phone use for information and communication is widespread among young rural physicians, this form of technology should be maximised and developed for telemedicine, perhaps even more than Internet, whose coverage is less consistent and reliable. Where feasible, NTHC should consider giving or increasing financial support to the rural physicians it supports, in order to facilitate procuring Internet services. This could be used for purchasing mobile broadband services or subscriptions, or even mobile computers, tablets, or smart phones. If funds for procuring equipment are limited, then this could be offered as an incentive for these physicians in order to increase interest in the telemedicine services of NTHC and promote their use.

NTHC should also make use of the existing communications networks among these rural physicians. NTHC can and should recruit trusted experts and mentors who can serve as specialists who will receive telemedicine referrals. These experts would serve the dual purpose of being social influencers who can convince them to try the telemedicine services, and also be a concrete demonstration of the usefulness of the service by being a readily available source of support and expertise. NTHC could do its part in building up communities of physicians who can serve as support groups and contacts when the need for expert consult arises, or whenever they feel the need for peer support and networking arises. Finally, NTHC staff should always be readily available to respond to any issues or difficulties that may arise with the use of the service.

This research has shown that the telemedicine programme of NTHC requires improvement in order to reach its full potential. Despite its origin in 1998, NTHC seems not to have outgrown the voluntary spirit and informality of its early days. It is staffed mostly by young people in their twenties and in their first job out of college. Even the director has just a few years of experience. The senior administrative staff are the ones who seem to provide the continuity and experience to the younger employees.

NTHC should take advantage of the relative youth of its staff by promoting creativity and minimising hierarchy in its organisational set-up. Creative organisations can constantly identify and promote new ways of using ICT in healthcare. Conversely, sustainability of NTHC and its programmes can be promoted by having continuity in its senior management and effective communication with its workers about the organisation’s mission and vision. Business plans and formal programmes would provide workers with clarity and job stability, while creating champions. Finally NTHC management should work towards financial independence and stability. Grants will end, requiring other sources to replace them, such as charging user fees for NTHC services.7

There are limitations to this study. As noted earlier, the homogeneity of respondents (all young physicians, recently graduated from a publicly funded hospital and college of medicine) challenges the representativeness of the general population of rural physicians, and therefore the spectrum of possible responses and ways in which physicians could integrate and use telemedicine in their daily practice. In addition, only a few rural areas were studied (time limitations; destructive effects of Typhoon Haiyan; logistical difficulties of gathering physicians from all over the country). However a fair selection of responses was seen in the focus group discussions.

A second limitation was the cross-sectional design of this study, which captured only a snapshot of factors operative during a certain specific time period. A longitudinal study would allow examination of changes over time, which might shed light on any causal relationships existing between the variables studied. Further, it might help determine the effectiveness of any interventions that may be undertaken as a result of study findings. Additionally, direct observation of rural physicians as they go about daily routines would add insight about work patterns and how telemedicine may be integrated into practice.

Finally the results of this study will only apply to the use of telemedicine by individual users in the local setting, not the adoption of other aspects of eHealth such as use of electronic medical records or other mobile applications, or the use and adoption of such technologies at the organisational level. Different factors may be operative at the organisational level, and thus different models may need to be used. But studies of the adoption of other health related technologies can be guided by the results of the present study.

Future considerations concern the questionnaire. As the use by respondents of the telemedicine service increases, adding another dependent variable (the actual usage rate of telemedicine) would serve as a more definitive indicator of actual use, rather than merely the intention to use telemedicine. However, if more items per construct, more choices per question, or use of a seven point or ten point Likert scale, were included, then the proportion of variability in the intention to use telemedicine might have increased from this study’s 54.8%. A larger sample size would definitely also have helped increase the coefficient of determination.

Considering the above, the following recommendations for additional research may be offered:

1) Recruit a more diverse sample of rural physicians with a wider range of experience and practice to identify additional issues with regard to adoption of telemedicine;

2) Adopt a longitudinal study design, and include direct observation of representative rural physicians to identify changes over time, causal relationships, and even the effects of interventions;

3) Develop a questionnaire with more choices and constructs and administer to a larger number of respondents to improve coefficient of determination calculations; and

4) Study factors that determine adoption of telemedicine at the organisational level, as well as the adoption of other health technologies. This would provide a more accurate and complete picture of the role of telehealth and telemedicine in everyday clinical practice.

Pasco PM, J Int Soc Telemed eHealth 2016;4:e26