Implementation of evidence-based clinical nutrition: Usability of the new digital platform clinicalnutrition.science

DOI: https://doi.org/https://doi.org/10.57187/s.3764

Introduction

Around a third of hospitalised patients are malnourished or at high risk of malnutrition when admitted to hospital [1]. In Switzerland, this corresponds to approximately 300,000 inpatients per year [2]. An estimated 20% of outpatients are at increased risk of malnutrition [3, 4]. Disease-related malnutrition is triggered by the underlying disease and can result from both inadequate nutrient intake and the systemic inflammatory response [5]. Malnutrition is strongly associated with an increased risk of adverse clinical outcomes. These include increased morbidity and mortality, functional decline and prolonged hospitalisation [1]. The two large randomised controlled trials “Effect of early nutritional support on Frailty, Functional Outcomes, and Recovery of malnourished medical inpatients Trial (EFFORT)” and “Nutrition effect On Unplanned ReadmIssions and Survival in Hospitalised patients (NOURISH)” showed that individualised nutritional intervention can improve outcomes, including reducing the risk of mortality (number needed to treat [NNT]: 37 and 20, respectively), morbidity and non-elective hospital readmission [6, 7]. Evidence-based nutritional management includes nutritional risk screening, nutritional assessment, nutritional plan and intervention, as well as nutritional monitoring and re-evaluation [5]. The European Society of Clinical Nutrition and Metabolism (ESPEN) and other international and national societies have published numerous guidelines with the objective of facilitating the implementation of evidence-based nutritional management in daily clinical practice [8–10].

Clinical nutrition is frequently overlooked in practice due to a lack of education on the subject and inappropriate prioritisation under time and resource pressures. Timely, straightforward and reliable access to relevant information can assist healthcare professionals in integrating clinical nutrition into multimodal patient care in an optimal and appropriate manner. Such an approach can facilitate more effective performance of tasks, thereby contributing to the long-term assurance of quality and safety in patient care. We have developed clinicalnutrition.science (https://clinicalnutrition.science/en/) to provide evidence-based information for nutritional management. This platform is independent, freely available and it can be accessed remotely at the bedside. The platform consists of six tools: NutriScreen, providing validated nutritional risk screening tools; NutriRisk, estimating the reduction in risk of complications and short- or long-term mortality if a nutritional intervention is initiated; NutriCalc, calculating nutritional goals based on established equations; NutriGo, providing interactive nutritional advice for specific situations based on current guidelines; NutriPro, a comprehensive database of nutritional products available in Switzerland; and NutriBib, a synthesis of the most evidence-based literature in the field of clinical nutrition.

The aim of this project was to assess the usability of the newly developed digital platform, clinicalnutrition.science, and to evaluate its impact on key steps in the nutritional management process carried out by healthcare professionals. The main outcomes of the present study were an enhancement in patient safety, as evidenced by the identification of malnutrition and subsequent recommendation of adequate protein and energy intake, and more accurate prescription of appropriate nutritional products. Additionally, the study also aimed to show an improvement in decision effectiveness, workflow integration and work efficiency.

Materials and methods

The usability study comprised three distinct parts: (1) collection of demographic data from participants, (2) performance of pivotal steps within nutritional management for two patient cases, one with and one without the clinicalnutrition.science digital platform, and (3) the provision of feedback on the usability of the platform.

Questionnaires, case vignettes and respective outcomes

1. Demographic data was collected using the Healthcare Systems Usability Scale (HSUS) demographic questionnaire, including age, sex, clinical experience, position, activity, place of work and use of other digital platforms [11]. We added a question about interest in the field of clinical nutrition and questions about current use of clinicalnutrition.science.

2. Two cases were selected as prototypes of commonly encountered conditions/types of patients with malnutrition: one case of a patient with liver cirrhosis (hepatology) and one case of a patient with a malignant tumour (oncology). These cases were chosen because of their high prevalence in clinical practice and the availability of corresponding European Society of Clinical Nutrition and Metabolism guidelines [9, 10, 12]. The two cases were developed on the basis of two real adult cases from our clinic and were presented in a similar format with comparable anonymised information. A series of questions were posed to the participating healthcare professionals regarding the simulated individualised nutritional management care of the two patients and the rationale behind their decision:

• Presence of risk of malnutrition
• Recommendation for energy and protein intake
• Estimation of basal metabolic rate and energy deficit
• The selection of the optimal route of nutrition, i.e. oral, enteral, parenteral or a combination thereof
• A list of the main objectives of nutritional therapy
• A description of the indicated nutrition therapy measures
• Decision of whether nutritional products are indicated
• The identification of an appropriate nutritional product, i.e. foods for special medical purposes or parenteral nutritional products
• Determination of an appropriate clinical nutrition guideline.

3. The validated HSUS questionnaire was used to assess the usability of clinicalnutrition.science in a clinical context. The HSUS questionnaire is based on four different categories: patient safety and decision effectiveness, workflow integration, work effectiveness and user control. It contains a total of 22 previously published items, which are rated on a 7-point Likert scale. The authors of the HSUS questionnaire interpreted the overall usability score as follows:

• 20% to <50% : “critical need to address the system’s usability issues”
• 50% to <70% : “a need to address the system’s usability concerns, some of which may be major”
• 70% to <90% : “a good usability score with the potential to improve”
• 90% to 100% : “an excellent and easy to use system” [11].

The final questionnaires were pre-tested by a dietitian and two physicians with a medical, nutritional or educational background. All questionnaires were completed using Google Forms [13]. Full questionnaires are provided in the appendix. All outcomes were considered as main outcomes due to their equal relevance to improving nutritional management.

Study design and population

Healthcare professionals were recruited by email and personal invitation from the department of general internal medicine of two teaching hospitals: the University Hospital of Bern (cohort A) and the Cantonal Hospital of Aarau (cohort B). In order to be eligible for the study, participants had to belong to one of the future user groups of healthcare professionals, including physicians, dietitians, nurses, pharmacists and scientists, and be willing to participate in the study. There were no limitations placed on level of clinical experience, previous use of clinicalnutrition.science or the time taken to complete the questionnaires.

The study was conducted at the two aforementioned centres on 28 September 2023 and 16 November 2023, respectively. Participants completed the questionnaire form anonymously in a supervised room in order to prevent the exchange of results. Participants were first asked to complete a background questionnaire. Secondly, the patient case questionnaires were completed in an adapted cross-over design.

In the initial round, cohort A responded to the oncology case and cohort B answered the hepatology case using only the resources normally employed in clinical practice (e.g. internet, hospital internal sheets), excluding the platform clinicalnutrition.science. In the second round, cohort A responded to the hepatology case, while cohort B answered the oncology case using clinicalnutrition.science (figure 1). The participants were unaware of the patient’s underlying conditions, yet they were required to interpret them from the patient case. Thirdly, participants completed the validated HSUS questionnaire. The local ethics committee decided that no ethical approval was required for this usability study (BASEC Req-2023-01186).

Figure 1Adapted cross-over design of the usability study with cohort A and B. In round 1, cohort A responded to the oncology case and cohort B to the hepatology case using standard resources. In round 2, the cohorts switched cases and responded using clinicalnutrition.science.

Statistical analysis

We performed all analyses in R version 4.3.0 (R Core Team, Austria) [14]. Visualisations were created using the ggplot package [15] and statistical analysis was performed using nortest [16], lme4 [17] and car packages [18]. To allow for a self-paired comparison between groups, only participants who completed both case vignettes were included in the analysis. Where ranges were recommended (e.g. 1.2–1.5 g protein), the mean of the ranges was calculated for both the ESPEN guideline recommendations and the participants’ responses and used for analysis. Results are presented as median and interquartile range (IQR) with quartile 1 (Q1) and quartile 3 (Q3) or as absolute numbers, unless stated differently. No protocol was published prior to analysis.

An unpaired Wilcoxon rank-sum test was used to test whether the medians of the two cohorts were significantly different to account for unequal sample sizes and non-normal distribution of the data. For categorical outcomes, we used Fisher’s exact test (appropriate for small sample sizes) to test whether there was a statistically significant association between the two cohorts [19]. Statistical significance was defined as a p <0.05. We performed a descriptive analysis to assess the difference in variance of the results obtained with and without the use of the digital platform, as highly divergent results and outliers should be avoided in clinical practice. These results should be avoided in clinical practice due to the adverse effects associated with inadequate protein and energy intake [9, 10].

For the subgroup analysis by profession, the deviation of the participants’ recommended protein and energy intakes from the protein and energy intakes recommended in the guidelines was calculated (i.e. recommendation of participants – recommendation of guidelines: Oncology: Protein: 1.2–1.5 g per kg body weight per day = 85–107 g per day [mean 96 g]; Energy: 25–30 kcal per kg body weight per day = 1775–2130 kcal per day [mean 1952.5 kcal]; Hepatology: Protein: 1.5 g per kg body weight per day = 105 g per day; Energy: 30–35 kcal per kg body weight per day = 2100–2450 kcal per day [mean 2275 kcal]) [9, 10]. This allowed for the combination of both cases and thus increased power. All items of the HSUS questionnaire were weighted equally to calculate the overall usability score. The overall score and the score of each item or category were calculated as: (sum of achieved points / sum of possible points) × 100 [%]. The responses “Not applicable” and “Statement not clear” were excluded from the score calculation.

Results

In total, 106 healthcare professionals from the University Hospital of Bern and 85 healthcare professionals from the Cantonal Hospital of Aarau were contacted. Ultimately, 56 healthcare professionals from the University Hospital of Bern (cohort A, n = 38) and from the Cantonal Hospital of Aarau (cohort B, n = 18) completed their questionnaires, which were analysed. The majority of included healthcare professionals were physicians (68%), followed by dietitians (21%), clinical nutrition scientists (7%) and nurses (4%). Overall, 73% of the participants had accumulated over one year of clinical experience, while 86% expressed a moderate or high level of interest in clinical nutrition (table 1). Three participants were unable to complete the second case questionnaire due to clinical commitments and were therefore excluded from all subsequent analyses.

Case study

Recommended protein and energy intake

Table 2 summarises the results of the cases. The use of clinicalnutrition.science enabled 55 participants (98%) to identify patient’s malnutrition, whereas without its use, 54 participants (96%) were able to do so. The median recommended protein intake was significantly different for the oncology case (p = 0.03; 96.5 and 80.0 g/d), but was comparable for the hepatology case with and without use of the platform (p = 0.76; both 94.5 g/d; figure 2). The median recommended energy intake was similar with and without use of the platform for the oncology case (p = 0.07, 1991.5 and 2139.0 kcal/d) and the hepatology case (p = 0.63; 2000.0 and 2100.0 kcal/d; figure 2).

Figure 2Recommended protein intake (g per day) and recommended energy intake (kcal per day) for the hepatology and oncology patient without (green) and with (blue) use of the clinicalnutrition.science digital platform. The green and blue boxes represent the lower and upper quartiles. The point within the box represents the median. The vertical lines are the whiskers extending to the maximum and minimum values, excluding the outliers (black points outside the box). The orange rectangle indicates the intake recommended by the corresponding European Society of Clinical Nutrition and Metabolism guideline [9, 10].

Estimated basal metabolic rate and energy deficit and time to complete case

Estimated basal metabolic rate was significantly reduced for the oncology case (p = 0.03; 1420.8 and 1755.5 kcal/d) and was comparable for the hepatology case when using the platform (p = 0.08; 1400.0 and 1500.0 kcal/g; figure S1). The estimated energy deficit was similar with and without use of the platform for the oncology and the hepatology case (p = 0.32 [oncology case] and 0.29 kcal/d [hepatology case]; figure S1). Time to complete the cases was similar with and without use of the platform (p = 0.15 [oncology case] and p = 0.52 [hepatology case], figure S2).

Variance of outcomes with and without clinicalnutrition.science

The variance was reduced by using the digital platform for recommended protein intake (oncology: IQR 0.5 [with platform] and 35.2 g/d [without platform]; hepatology: IQR 10.0 [with platform] and 30.0 g/d [without platform]; figure 2), for the recommended energy intake (oncology: IQR 170.3 [with platform] and 460.3 kcal/d [without platform]; hepatology: IQR 456.0 [with platform] and 525.0 kcal/d [without platform]; figure 2), for the estimated basal metabolic rate (oncology: IQR 99.3 [with platform] and 497.0 g/d [without platform]; hepatology: IQR 97.0 [with platform] and 350.0 g/d [without platform]; figure 2) and for the estimated energy deficit (oncology: IQR 324.0 [with platform] and 657.0 g/d [without platform]; hepatology: IQR 521.0 [with platform] and 537.5 g/d [without platform]; figure 2).

Subgroup analysis

The difference between the median recommended protein intake and the corresponding ESPEN guideline was similar with and without use of clinicalnutrition.science for physicians (p = 0.49; −10.5 [with platform], −16.0 g/d [without platform]) and significantly increased for dietitians (p = 0.02; 0.0 [with platform]; −9.25 g/d [without platform]; figure 3). The median difference between the recommended energy intake and the corresponding ESPEN guideline recommendation increased significantly with use of the platform for physicians (p = 0.02; −181.0 [with platform], 97.5 kcal/d [without platform]) and was comparable for dietitians (p = 0.64; 28.3 [with platform]; 23.8 kcal/d [without platform]; figure 3). The variance in the recommended protein intake was lower with the platform compared to without the platform for physicians (IQR 14.0 and 36.2 g/d), dietitians (IQR 1.3 and 20.3 g/d) and the others (IQR 30.5 and 32.6 g/d; figure 3). The variance in the recommended energy intake was lower when using the platform compared to not using the platform for physicians (IQR 503.5 and 585.3 kcal/d), dietitians (IQR 157.3 and 350.0 kcal/d) and the others (IQR 386.8 and 950.0 kcal/d; figure 3).

Figure 3Deviation of the recommended protein intake (g per day) and recommended energy intake (kcal per day) from the corresponding European Society of Clinical Nutrition and Metabolism guideline [9, 10] for the hepatology and oncology patient without (green) and with (blue) use of the clinicalnutrition.science digital platform, by profession. The green and blue boxes represent the lower and upper quartiles. The point within the box represents the median.

Usability questionnaire

The HSUS questionnaire was completed by 55 participants. Overall, 46 participants (83%) had never used the platform before. A total of 9 participants (16%) had used at least one of the clinicalnutrition.science tools (NutriScreen, NutriCalc, NutriGo, NutriPro, NutriBib) prior to the usability study. Seven (13%) had used them for less than 3 months and 2 (4%) for between 3 months and 1 year. Previous users used the platform between 1 and 3 hours per week.

The overall usability score of the HSUS questionnaire for clinicalnutrition.science was 71.8%. By category, scores were 71.8% for Patient safety and decision effectiveness, 74.1% for Workflow integration, 71.4% for Work effectiveness and 69.8% for User control (figure 4). The four highest scoring items on the HSUS questionnaire were:

• “I believe the recommendations are reliable” (score 79.6%, Work effectiveness category question 5)
• “I can easily remember how to use clinicalnutrition.science” (score 78.8%, Workflow integration category question 4)
• “Clinicalnutrition.science supports my decision-making rather than dictating it” (score 75.3%, User control category question 1)
• “I am able to provide better quality of care for patients by using clinicalnutrition.science” (score 75.1%, Patient safety and decision effectiveness category question 3).

The four lowest scoring items were:

• “Clinicalnutrition.science helps me prioritise my daily workload” (score 59.2%, Work effectiveness category question 1)
• “Clinicalnutrition.science makes it easier to collaborate with colleagues” (score 65.1%, Patient safety and decision effectiveness question 2)
• “Clinicalnutrition.science highlights potential data entry errors” (score 66.2%, User control category question 3)
• “Recommendations do not unnecessarily interrupt my workflow” (score 66.4%, User control category question 4).

Figure 4Usability of the clinicalnutrition.science digital platform based on the validated Healthcare Systems Usability Scale (HSUS) questionnaire. For each of the four categories (Patient safety and decision effectiveness, Workflow integration, Work effectiveness, User control), 4–7 items were rated on a 5-point Likert scale (1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree; see the appendix for detailed questions).

Discussion

Key findings of the study

The results of this study indicate that the overall usability of the newly developed clinicalnutrition.science platform is satisfactory, with a positive score of 71.8%. Use of the platform led to a notable enhancement in the precision of the recommended protein intake and the estimation of the basal metabolic rate for the oncology patient. Furthermore, the variance of the recommended protein and energy intake as well as the estimated basal metabolic rate and energy deficit was reduced when using the clinicalnutrition.science platform. It is of significant importance to note that these improvements were achieved without an increase in the time required to complete the key steps in the nutritional management process. The use of the platform did not result in a significant impact on the detection rate of malnutrition, the provision of appropriate product recommendations or the identification of suitable guidelines for the oncology and hepatology patient cases.

Recommendation on protein and energy intake

The results of our study indicated that use of the clinicalnutrition.science platform was associated with a reduction in the variability of recommended protein and energy intakes, and a decrease in the discrepancy between the ESPEN guidelines and recommended intake in this study [9, 10]. It is of paramount importance to prevent overfeeding and underfeeding to ensure the safety of patients. This can, for instance, help to mitigate cancer cachexia and prevent glycogen store depletion in cirrhosis patients [9, 10]. The EFFORT trial demonstrated that meeting protein and energy targets significantly reduced adverse outcomes and mortality in malnourished patients [6]. It is notable that the reduction in variance and adherence to guideline recommendations were more pronounced in the oncology patient case compared to the hepatology case. The reported acute kidney injury in addition to liver cirrhosis in the hepatology case may have influenced the formulation of nutritional therapy, as it complicated the process. In patients with kidney injury, guidelines recommend that, for example, the protein intake should be reduced [20–22].

Subgroup analysis by profession

A subgroup analysis revealed that dietitians and physicians recommended similar median protein and energy intakes, both with and without use of clinicalnutrition.science. Although dietitians were already more accurate than physicians in recommending protein and energy intakes using their standard resources, both were able to reduce deviations from the guideline recommendations when using the platform. It is of paramount importance to address deviations from the guideline recommendations to minimise the occurrence of treatment errors and adverse clinical outcomes. The results of this study indicate that the digital platform is advantageous for both dietitians and physicians.

Added value of the platform clinicalnutrition.science and strength of study

In comparison to other clinical nutrition resources, clinicalnutrition.science guides the majority of the nutritional management process, from nutritional risk screening to the prescription of nutritional products. It is an independent, content-proven and applicable resource that can be used directly at the bedside. While the ESPEN guidelines are of great importance for evidence-based clinical nutrition, their complex workflow integration constrains their impact and implementation in daily practice. The recently launched ESPEN interactive guideline app only covers a limited number of steps in the nutritional management process [23]. Previous studies have demonstrated that a simple online training tool or multifaceted nutritional education, when employed alone, is insufficient to improve nutritional management [24, 25].

The clinicalnutrition.science platform received an overall usability score of 71.8% based on the HSUS questionnaire, which states that a score between 70% and 90% reflects good usability, with room for improvement [11]. Poor usability of health information systems must be prevented as they are associated with reduced efficiency, workflow disruption, increased risk of medical treatment errors and increased incidence of adverse events [11]. It is of significant importance to note that participants found our digital platform to be straightforward to use, intuitively structured and efficacious in enhancing their nutritional management skills. The questionnaire item with the lowest score (59%) was “prioritisation of daily workload”, which was not the aim of the platform. All other items scored above 65%. While the HSUS score may underestimate usability by equally weighting all items, it provides a comprehensive assessment. The validation phase involved healthcare professionals with diverse backgrounds, skills and experiences, which is essential for a robust and reliable validation process [26].

Limitations of the current study

The current study has several limitations. The sample size was limited to 56 participants, with a preponderance of physicians and dietitians with at least a moderate interest in clinical nutrition. This may limit the generalisability of the results. Furthermore, the evaluation based on two patient cases that merely mimicked the key steps of the nutritional management process may have introduced a degree of bias in the results. For example, the malnutrition detection rate of over 90% in our study contrasts strongly with malnutrition detection rates in clinical practice, which usually range from 20% to 60% [27]. The participants were not randomly assigned to the study cohort; rather, they were allocated based on their working hospital due to time and resource constraints. In addition, the participants were new to the clinicalnutrition.science platform, which may have increased the time needed for the nutritional plan formulation.  Consequently, it is of paramount importance to subject the clinicalnutrition.science platform to rigorous testing in a genuine clinical setting, including comprehensive training and a diverse range of participants.

Current state and outlook

We are currently engaged in a collaborative endeavour with the Swiss Society of Clinical Nutrition and Metabolism (SSNC) with the objective of raising awareness about the free resource clinicalnutrition.science throughout Switzerland. We will actively seek feedback from frequent users and distribute regular questionnaires such as the one presented here, to ascertain the effectiveness of the platform and to identify areas for improvement and enhancement. Furthermore, the content will be updated on a regular basis to guarantee the reliability and currency of the information provided.

Conclusion

The new and independent digital platform clinicalnutrition.science was found to be intuitive and exhibited a high positive degree of usability. The platform supported the recommendation of accurate protein and energy intake and optimised the nutritional management process without increasing the time required compared to standard resources. In collaboration with the SSNC, we will raise awareness about this digital platform across Switzerland and will ensure that it is continuously updated and improved.

Data availability statement

Questionnaires are present in the appendix. The corresponding author will provide access to raw data and full analysis code in this manuscript upon request. Case vignettes are not published for reasons of patient protection, but were available to the academic editor and reviewers during the manuscript revision process.

Acknowledgments

We would like to express our gratitude to all the clinical nutrition experts who have supported us in the development of the digital platform. We would like to thank all the healthcare professionals who participated in the validation and took the time to answer the questionnaire.

Author contributions: Conceptualisation and methodology: Valentina V. Huwiler, Pascal Tribolet, Caroline Rimensberger, Christine Roten, Philipp Schuetz, Zeno Stanga – Formal analysis and visualisation: Valentina V. Huwiler – Original draft preparation: Valentina V. Huwiler – Funding acquisition: Stefan Mühlebach, Philipp Schuetz, Zeno Stanga – Critical revision and editing of draft manuscript: Valentina V. Huwiler, Pascal Tribolet, Katja A. Schönenberger, Stefan Mühlebach, Philipp Schuetz, Zeno Stanga – Supervision: Stefan Mühlebach, Philipp Schuetz, Zeno Stanga.

All authors have read and agreed to the final version of the manuscript.

References

1. Schuetz P, Seres D, Lobo DN, Gomes F, Kaegi-Braun N, Stanga Z. Management of disease-related malnutrition for patients being treated in hospital. Lancet. 2021 Nov;398(10314):1927–38. doi: https://doi.org/10.1016/S0140-6736(21)01451-3 

2. Bundesamt für Statistik. Patient/innen, Hospitalisierungen. 2021; Available from: https://www.bfs.admin.ch/bfs/de/home/statistiken/gesundheit/gesundheitswesen/spitaeler/patienten-hospitalisierungen.html 

3. Cawood AL, Elia M, Sharp SK, Stratton RJ. Malnutrition self-screening by using MUST in hospital outpatients: validity, reliability, and ease of use. Am J Clin Nutr. 2012 Nov;96(5):1000–7. doi: https://doi.org/10.3945/ajcn.112.037853 

4. Álvaro Sanz E, Garrido Siles M, Rey Fernández L, Villatoro Roldán R, Rueda Domínguez A, Abilés J. Nutritional risk and malnutrition rates at diagnosis of cancer in patients treated in outpatient settings: early intervention protocol. Nutrition. 2019 Jan;57:148–53. doi: https://doi.org/10.1016/j.nut.2018.05.021 

5. Cederholm T, Barazzoni R, Austin P, Ballmer P, Biolo G, Bischoff SC, et al. ESPEN guidelines on definitions and terminology of clinical nutrition. Clin Nutr. 2017 Feb;36(1):49–64. doi: https://doi.org/10.1016/j.clnu.2016.09.004 

6. Schuetz P, Fehr R, Baechli V, Geiser M, Deiss M, Gomes F, et al. Individualised nutritional support in medical inpatients at nutritional risk: a randomised clinical trial. Lancet. 2019 Jun;393(10188):2312–21. doi: https://doi.org/10.1016/S0140-6736(18)32776-4 

7. Deutz NE, Matheson EM, Matarese LE, Luo M, Baggs GE, Nelson JL, et al.; NOURISH Study Group. Readmission and mortality in malnourished, older, hospitalized adults treated with a specialized oral nutritional supplement: A randomized clinical trial. Clin Nutr. 2016 Feb;35(1):18–26. doi: https://doi.org/10.1016/j.clnu.2015.12.010 

8. Wunderle C, Gomes F, Schuetz P, Stumpf F, Austin P, Ballesteros-Pomar MD, et al. ESPEN guideline on nutritional support for polymorbid medical inpatients. Clin Nutr. 2023 Sep;42(9):1545–68. doi: https://doi.org/10.1016/j.clnu.2023.06.023 

9. Bischoff SC, Bernal W, Dasarathy S, Merli M, Plank LD, Schütz T, et al. ESPEN practical guideline: clinical nutrition in liver disease. Clin Nutr. 2020 Dec;39(12):3533–62. doi: https://doi.org/10.1016/j.clnu.2020.09.001 

10. Muscaritoli M, Arends J, Bachmann P, Baracos V, Barthelemy N, Bertz H, et al. ESPEN practical guideline: clinical Nutrition in cancer. Clin Nutr. 2021 May;40(5):2898–913. doi: https://doi.org/10.1016/j.clnu.2021.02.005 

11. Ghorayeb A, Darbyshire JL, Wronikowska MW, Watkinson PJ. Design and validation of a new Healthcare Systems Usability Scale (HSUS) for clinical decision support systems: a mixed-methods approach. BMJ Open. 2023 Jan;13(1):e065323. doi: https://doi.org/10.1136/bmjopen-2022-065323 

12. Aubert CE, Fankhauser N, Marques-Vidal P, Stirnemann J, Aujesky D, Limacher A, et al. Patterns of multimorbidity in internal medicine patients in Swiss university hospitals: a multicentre cohort study. Swiss Med Wkly. 2019 Jun;149:w20094. doi: https://doi.org/10.4414/smw.2019.20094 

13. Google. Google Forms. 2023 05. [January 2024]; Available from: https://docs.google.com/forms 

14. R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2023. 

15. Wickham H. ggplot2: Elegant graphics for data analysis. 2016, New York: Springer-Verlag. 11-31. 

16. Gross, J. and U. Ligges, nortest: Tests for Normality. R package version, 2015. 1(4). 

17. Bates D, Mächler M, Bolker B, Walker S. Fitting linear mixed-effects models using lme4. J Stat Softw. 2015;67(1):1–48. doi: https://doi.org/10.18637/jss.v067.i01 

18. Fox J, Weisberg S. An R companion to applied regression. 3rd ed. Sage publications; 2018. 

19. Fisher RA. Statistical methods for research workers. Breakthroughs in statistics: Methodology and distribution. Springer; 1970. pp. 66–70. 

20. Sobotka L. Basics in clinical nutrition. 5th ed. Galen; 2020.[S.l.]. 

21. Cano N, Fiaccadori E, Tesinsky P, Toigo G, Druml W, Kuhlmann M, et al.; DGEM (German Society for Nutritional Medicine); ESPEN (European Society for Parenteral and Enteral Nutrition). ESPEN Guidelines on Enteral Nutrition: adult renal failure. Clin Nutr. 2006 Apr;25(2):295–310. doi: https://doi.org/10.1016/j.clnu.2006.01.023 

22. Fiaccadori E, Sabatino A, Barazzoni R, Carrero JJ, Cupisti A, De Waele E, et al. ESPEN guideline on clinical nutrition in hospitalized patients with acute or chronic kidney disease. Clin Nutr. 2021 Apr;40(4):1644–68. doi: https://doi.org/10.1016/j.clnu.2021.01.028 

23. Bischoff SC, Cuerda C, Barazzoni R. Practical guidelines and apps for improvement of guideline implementation. Clin Nutr. 2020 Oct;39(10):2943–4. doi: https://doi.org/10.1016/j.clnu.2020.07.031 

24. Aeberhard C, Birrenbach T, Joray M, Mühlebach S, Perrig M, Stanga Z. Simple training tool is insufficient for appropriate diagnosis and treatment of malnutrition: A pre-post intervention study in a tertiary center. Nutrition. 2016 Mar;32(3):355–61. doi: https://doi.org/10.1016/j.nut.2015.09.012 

25. Reber E, Messmer Ivanova A, Cadisch P, Stirnimann J, Perrig M, Roten C, et al. Does multifaceted nutritional education improve malnutrition management? Nutrition. 2020 Oct;78:110810. doi: https://doi.org/10.1016/j.nut.2020.110810 

26. Wyatt JC, Wyatt SM. When and how to evaluate health information systems? Int J Med Inform. 2003 Mar;69(2-3):251–9. doi: https://doi.org/10.1016/S1386-5056(02)00108-9 

27. Imoberdorf R, Meier R, Krebs P, Hangartner PJ, Hess B, Stäubli M, et al. Prevalence of undernutrition on admission to Swiss hospitals. Clin Nutr. 2010 Feb;29(1):38–41. doi: https://doi.org/10.1016/j.clnu.2009.06.005 

Appendix

The appendix is available in the pdf version of the article at https://doi.org/10.57187/s.3764.