Tests Overview

Each practice is asked to complete the following tests

At least two End-To-End test as a Clinician which will run over 2 days.
At least two Usability Testing - Cohorts. At least once as a Clinician, and once as an Analyst (who should see nothing).
At least two Usability Testing - Dashboards. At least once as a Clinician, and at least once as an Analyst.

Each User should also complete a Skyline Usability Feedback.

We are asking for screenshots below, however if users can video their test (e.g. record a screen share using a Teams, zoom etc) that would be even better.

Guidance

Screenshots/Video Naming

Each screenshot should be named as follows:

a video should be named

<Section Shortname>-DDMMYY-HHMM

If necessary add a sequence number at the end

i.e. Screenshot of cohort with patient highlighted/identified would be named E2E-1-210521-1127.jpeg the screenshots of adding symptoms to calculator would be named E2E-4-1-210521-1134.jpeg and E2E-4-2-210521-1137.jpeg

Screenshot File formats

Screenshots can be in any standard image format; e.g jpeg, jpg, jpe, png, bmp, tif, tiff

End-To-End Test

Section shortname - E2E

Role for test - Clinician

E2E testing must ONLY be done using TEST PATIENTS

	Action	Test	Evidence
1	Open a TEST patient in Clinical System and open record for patient identified above	Does the Cue Card show? Does the cue card show the ‘correct’ cancer (or no cancer)	Screenshot of patient record & cue card
2	Open calculator from Cue card	Does the calculator show the correct patient details and expected symptoms (or no symptoms)	Screenshot of calculator
3	Add symptoms to calculator	Does the score change as expected	Screenshot of calculator with additional symptoms screenshot of calculated risk score tab.
4	Check coding tab	Are the expected codes listed	screenshot of coding tab.
5	Press OK	Are any errors displayed, or other messages	screenshot of messages.
6	Wait until next day
7	Open Skyline and select an ERICA Cohort	Does the test patient now appear in the list or show the new score?	Screenshot of cohort with patient highlighted/identified
8	Repeat steps 1 - 7 for the same patient

Usability Testing - Cohorts

Section shortname - UTC

Role for test - Clinician

	Action	Test	Evidence
1	Open Skyline and select an ERICA Cohort	Identify 2 patients	Screenshot of cohort with patient highlighted/identified
2	Open a ‘real’ patient in Clinical System and open record for 1 patient identified above	Does the Cue Card show? Does the cue card show the ‘correct’ cancer	Screenshot of patient record & cue card
3	Open calculator from Cue card	Does the calculator show the correct patient details and expected symptoms DO NOT Press OK as this will potentially write codes back to the clinical system	Screenshot of calculator
4	Repeat steps 2-3 for the second patient identified
5	Repeat steps 1-4 for each cohort

Usability Testing - Dashboards

Dashboards may not be available in the initial release.

Section shortname - UTD

Role for test - Clinician & Analyst

	Action	Test	Evidence
1	Open Skyline and select a Dashboard	View dashboard	Screenshot of dashboard

Skyline User Test Reporting