Applying an integrated approach to the design, implementation and evaluation of a telemedicine system.

  • Susan K Hansen University of Technology, Sydney
  • Laurence S Wilson CSIRO Australia
  • Toni Robertson Univerasity of Technology Sydney
Keywords: telehealth, telemedicine, evaluation, evaluation model, participatory design, action research, intensive care


One characteristic of telemedicine evaluations is that they are usually decoupled from other aspects of design and implementation, and therefore not considered until late in the project cycle.  In addition to this, evaluations are often performed by outside bodies without an interest in the outcome, a practice intended to strengthen the independence and objectivity of the findings.  In this paper, we describe an alternative approach in which the design, implementation and evaluation of a telemedicine system were considered to be complementary stages of the project occurring in parallel and where members of the project team were involved in all three.  We demonstrate the advantages of this approach through the ECHONET project - a trial of an experimental system implemented in the context of Intensive Care across two hospitals in Tasmania, Australia.

This paper describes benefits associated with this integrated approach. Innovative aspects of the approach include the number of pre-implementation activities such as stakeholder interviews, design workshops and a baseline study, which enabled the project to adapt to the complex needs of its health context.

The integration of the design, implementation and evaluation activities enabled the project to adapt to changing needs as the project evolved, and ensured that it was evaluated against appropriate criteria.

Author Biographies

Laurence S Wilson, CSIRO Australia
Post Retirement Fellow
Toni Robertson, Univerasity of Technology Sydney
Professor, Interaction Design and Human Practice Lab


Li J, Wilson LS, Qiao R-Y et al. Development of a Broadband Telehealth System for Critical Care: Process and Lessons Learned. Telemedicine and e-Health. 2006;12(5).

Wilson LS, Stevenson DR, Cregan P. Telehealth on Advanced Networks. Telemedicine and e-Health. 2010;16(1).

Li J, Wilson L. Telehealth trends and the challenge for infrastructure. Telemed J E Health. 2013;In Press.

Demiris G, Afrin LB, Speedie S et al. Patient-centered applications: use of information technology to promote disease management and wellness. A white paper by the AMIA knowledge in motion working group. Journal of the American Medical Informatics Association. 2008;15(1).

Latifi R, Weinstein RS, Porter JM et al. Telemedicine and telepresence for trauma and emergency care management. Scand J Surg. 2007;96(4):281-289.

Reynolds H, Latifi R, Poropatich R, Hadeed G. The Tele-ICU: Why, How and What. What the future may look like. Telemedicine for trauma, emergencies, and disaster management. 2011.

Sanderson P. Designing and Evaluating Healthcare ICT Innovation: A Cognitive Engineering View. Proceedings of the 3rd international conference on information technology in healthcare: socio-technical approaches. Amsterdam: IOS Press, 2007: 3-12.

Tanriverdi H, Iacono CS. Diffusion of telemedicine: a knowledge barrier perspective. Telemed J. 1999;5(3):223-244.

Linderoth CJL. Implementation and evaluation of telemedicine-a catch 22? Proceedings of the 35th Annual Hawaii International Conference on Systems Science. IEEE, 2002.

Whitten P, Johannessen LK, Soerensen T, Gammon D, Mackert M. A systematic review of research methodology in telemedicine studies. J Telemed Telecare. 2007;13(5):230-235.

Jackson DE, McClean SI. Trends in telemedicine assessment indicate neglect of key criteria for predicting success. J Health Organ Manag. 2012;26(4-5):508-523.

Ekeland AG, Bowes A, Flottorp S. Methodologies for assessing telemedicine: a systematic review of reviews. Int J Med Inform. 2012;81(1):1-11.

Kidholm K, Ekeland AG, Jensen LK et al. A model for assessment of telemedicine applications: mast. Int J Technol Assess Health Care. 2012;28(1):44-51.

Goran SF. Measuring tele-ICU impact: does it optimize quality outcomes for the critically ill patient? J Nurs Manag. 2012;20(3):414-428.

Robson C. Real world research. 2 ed. Oxford: Blackwell; 2002.

Hansen SK, Robertson T, Wilson LS, Hall R. Using an Action Research Approach to Design a Telemedicine System for Critical Care: A Reflection. Proceedings of OZCHI 2008. 2008: 255-8.

Sutcliffe A, Thew S, De BO et al. User engagement by user-centred design in e-Health. Philos Transact A Math Phys Eng Sci. 2010;368(1926):4209-4224.

Chan AW, Tetzlaff JM, Altman DG, Dickersin K, Moher D. SPIRIT 2013: new guidance for content of clinical trial protocols. Lancet. 2013;381(9861):91-92.

Conde JG, De S, Hall RW, Johansen E, Meglan D, Peng GC. Telehealth innovations in health education and training. Telemed J E Health. 2010;16(1):103-106.

Thomas EJ, Lucke JF, Wueste L, Weavind L, Patel B. Association of telemedicine for remote monitoring of intensive care patients with mortality, complications, and length of stay. JAMA. 2009;302(24):2671-2678.

How to Cite
Hansen, S., Wilson, L., & Robertson, T. (2013). Applying an integrated approach to the design, implementation and evaluation of a telemedicine system. Journal of the International Society for Telemedicine and EHealth, 1(1), Pages 19-29. Retrieved from
Original Research