Evaluation (ArmCoach)
ArmCoach Evaluation & Iteration
To get a first qualitative feedback about our product, we need to go to the future use context. There we test the acceptance, aesthetics, emotions, function also look at the visual and haptic feedback of the bracelet and the application. The whole testing procedure is filmed so that we can analyze it afterwards.
For assuring the privacy of our test-patients and therapist we agreed to dont publish those videos. If you are interested in seeing those videos you are free to contact us.
____________________________________________________________________________________
User-testing procedure
At first we explain the functions of the bracelet and put it on patients impaired arm. The goal is to reach within one hour 16 movement-points without the alarm ringing. The bracelet needs to be ignored by the user, so that we can test if the reminder works. Due to this testing we will interview the patients about different topics and ask them to do something in daily routine like making coffee, thus creating a distraction.After an elapsed time of 15 min we asked them to start the application on their own personal computer without an explanation. They were requested to tell us what they had thought and had expected during use. Here we could find out if the navigation inside this system works and if it is clearly visible what we want to communicate. We also give certain tasks to fulfill inside this system like: “buy a new hint” and so on.At the end of the testing phase we asked them about their experiences during the testing and also what they liked or disliked about the bracelet and application. We then asked about our sketch of the proposed look of the bracelet which we showed them.
As we get new important insights we need to asses the value of each finding and implement them into the concept according to their significance to it.
____________________________________________________________________________________
User-test Prototypes
Our first completed prototypes contain the basic functions of each object (bracelet and applications).
____________________________________________________________________________________
Bracelet Prototype
The prototype contains the LED-Bar which counts and reacts on movement. Within 45 minutes the client has to fill 15 LEDs (movement-points - mp). When he has not enough mp’s (1.6 mp’s in 4.5 minutes / 3.2 in 9 minutes and so on) the reminder sounds an alarm at fixed times (each 4.5 minutes). When 15 mp’s are reached, a winning-melody sounds.
____________________________________________________________________________________
Screen Prototype Patient
This click-dummy represents the categories “Übersicht” (overview), “Bewegungen” (movements), “Zeitleiste” (timeline) and “Armband” (bracelet) displayed with sample data. In the category “Bewegung” new ADL-Hints can be bought.
____________________________________________________________________________________
Screen Prototype Therapist
As this application would be used in therapy on a shared computer, there is a login screen implemented and also a list, in which therapists have an overview of their clients. The categories are “Bewegungen” (movements), “Zeitleiste” (timeline) and “Armband” (bracelet) and represent the selected patient’s status.
____________________________________________________________________________________
The user-tests
We planned to test the bracelet for one and a half hours. During this time we want to get feedback on the usability of the bracelet and the application(s) in their future use-context. That means for patients at home and for therapists in therapy. In order to analyze this session later, we filmed their actions and reactions to those objects. We tested the product with three patients and one therapist.
____________________________________________________________________________________
User-test findings
Bracelet Acceptance The bracelet was accepted by all users. They did not feel distracted in any way during activities at home although the prototype is in a big case. - Approved by users Aesthetics and emotions All our users liked sports-watch-like sketch. First of all it is impor- tant to them that it does not have the look of a medical device. What they liked was also the smiley which represents the actual state of the LED-bar. Function After some vibrating and sounding reminders went off without having been attended to, most of the users started to talk to the bracelet like to a living object. They said: “Oh no, you again” or “Hey, I am moving right now”. If the reminder had been preset to ring at fixed times, it happened sometimes that it rang during a movement. This was very confusing. After having been tested the bracelet does not need an additional smiley to strengthen an emotional relationship with the patient. The bracelet gives off an alarm when no movement has been de- tected for a while. The repetition period can be set through the thera- pist now. Visual feedback The LED-bar worked as anticipated. The users were motivated to fill them and were curious at times how many movement-points they have right now. As some of them looked at it, they were instantly promoted to move the arm. The instant feedback (lighting up LED during movement) worked very well and is important. - Approved by users Haptic and auditive feedback Interestingly one of the users interpreted the vibration of the alarm as a muscle activator. For future work it is important that the users are beeing advised that this should remind to move. But all of them felt the vibration. It was often remarked that the sound would be annoying in public space. The therapist said that the vibration strength should be adjustable and it should be possible to turn off the sound. - Since the sound does not create additional motivation to move, we eliminate it from the concept. - To intensify the relationship to the patient, we want to adjust the du-ration and strength of the vibration alarm.The user should notice how many movement are required to reach the target for the day and also enforce the feeling that the bracelet cares about the patient. For this more research is needed in the field of Haptic Emotion Transmission. ____________________________________________________________________________________ Patient application Acceptance The whole ADL leveling system was accepted by almost everybody. Some of them were more curious and wanted to see which hints were under the covered ones. The statistic is accepted and important for the users too. All of them want to see the progress and also compare and push themselves to move more than during the previous. The monitoring by the therapist is also very well accepted. Some of them said that this system will strengthen the relationship between them. One patient said that he would use this system also independently from the therapy. - Approved by users Aesthetics and emotions The category structure was well understood. Most users said that the look of the application is too dark and should be light-colored. The fonts should be bigger and have a stronger contrast. - Since most of our main-users are Digital Beginners we adapted the design-structure to well known applications. The fonts will be bigger and the contrast will be set higher. The whole feel and look is brighter. ADL-System When we implemented ADLs at a really early state some of the patients said that they could do this movement already. One user adapted the idea behind a hint to a movement that was not visible. All were motivated to get points to make new hints visible and fill the empty chart. - Approved by users ____________________________________________________________________________________ Therapist Application Bracelet Settings Because patients have differing perceptions, the therapist should be able to adjust the vibration strength and the period of reminder. Therapist found also the possibility missing to adjust the difficulty level (how strong must the arm be moved to count). - Three adjustments are added: - Frequency of the reminder - Vibration intensity - Level of difficulty Login In smaller clinics one log-in would be sufficient because they regularly share patients between each other. - In further development an advanced therapist-context research should be done. Patients It is important that once registered patients are always accessable. - Approved by user Activities All covered Hints should be visible by therapist. An additional category for Very Hard Exercises should be integrated. Individual activities should be able to be implemented by therapist. - In further development an advanced therapist-context research should be done. Unlock Arm-part Activities It became necessary that therapist can unlock different difficulties for several different arm-parts. - Implemented. Aesthetics Some elements do not distinctly convey what to do with them straight away which makes some patients get confused. - Therapists GUI (Graphical User Interface) is adapted to the new clients version.____________________________________________________________________________________