User Testing 1

On Thursday October 12, 2023 at Alberta Children’s Hospital, BCI4Kids Lion’s den.

User(s):

• Beverly Collison

Dev(s):

• Anup
• Eli
• Dan
• Emily

Set-up

Used DSI-Flex connected to “Morpheus” desktop computer via Bluetooth.

What versions were tested

• CLIP main version 46eee01.
• CLIP-Unity was tested from both the “Dev” environment (run out of Unity editor) and from the exe compiled by Elena that morning.
• CLIP-Python was tested from the exe compiled by Hosein on 2023-10-11.
  • It may have also been tested from the .py script version by Eli and Dan while they were debugging.

How it went

• Emily served as the BCI user. We set-up the DSI-Flex and got it connected via Bluetooth. We set-up the headset, ensuring good contact. We launched dsi2lslgui.exe. We entered the correct COM port value and started the LSL stream, checking that we were receiving streams from the headset, Unity and Python – noted major bug 1.
• We ran CLIP-Unity from the exe and then ran CLIP-Unity from the exe.
• We created a profile, navigated to the scene, set-up the instruments. We ran training. We then ran stimulus selection – noted major bug 4. Stimulus selection didn’t work.
• Eli and Dan tried to debug it – however, there were issues and we weren’t able to do stimulus selection.
• Anup had Beverly play with the CLIP-Unity software and got feedback from a User experience perspective.

Bugs noted

1. Major – LSL output stream from Unity only happening on primary shelf scene, but not on menu scenes. The Unity software doesn’t send a LSL marker stream when not in the primary shelf scene (e.g. in the main menu or options menus) – This cause the Python software to bug out and/or crash when it stops receiving Unity markers via LSL, such as from the beginning of the software, or when the User leaves the primary shelf scene.
   1. Proposed fix: Ensure that the LSL marker stream stays open in all scenes of the Unity software
2. In the exe, Anup noted that setting the resolution from the options menu doesn’t retain the resolution in all scenes. In other words, if you change the resolution from the primary shelf scene, this is not retained in the Main Scene (and I think vice-versa).
   1. Proposed fix: Make the resolution setting persist across all scenes (“global”)
3. Anup noted that there were repeated resolutions on the resolution drop-down – e.g. 2560x1440 would appear 3 times.
4. Major – The Python exe didn’t send the stimulus selection to Unity. The backend is receiving from LSL but not sending back to Unity. In other words, after running the stimulus presentation there was no selection made in the Unity software and no audio was played.
   1. Eli and Dan tried to debug this, and may have also tested the Python .py script. The outcome of this is _______.
5. Major – BCI stimulus menu needs to be hooked up so that the options actually change the items. Additionally, the settings order and how they respond need to change.

User Feedback

• The items should be presented in the inventory as grayscale as a default but without an outline.
• The order of items on the BCI stimulus settings menu, and how they should respond, should be:
  • Base size: A slider that increases the presented “base size” of the object. It starts at 100% (original) and goes up to some amount (200%? 300%?). Beverly said that they may want to increase the size of the presented object to improve visibility for the student, but won’t need to decrease it. When the BCI stimulus animation runs, it will grow and shrink relative to the set “base size”.
  • Colour: A single toggle (Yes or No – default No) – Enabling this should make the sprite material go from grayscale to full colour (basically make the “colour” slider either 0% or 100%)
  • Outline: A single Toggle (Yes or No – default No) and a dropdown of a few colour presets that will be provided by Beverly (we can go with Yellow, Red, Green, Blue with default to Yellow for now)
  • Texture: A single Toggle (Yes or No – Default No) for whether or not the sprite will have a Texture overlay – It is unclear to Anup what this texture is supposed to be. I think Beverly, Eli or Emily know better.
  • Animation: A dropdown menu of the type of stimulus presenting animation. Defaults to Zoom.
    • What are the other options? Do they even work? If there’s only “Zoom” then is there a point to this?
• Beverly was surprised that the BCI stimulus was grow/shrink/grow/shrink. She was expecting either a “Bounce zoom” sequence (item grows larger than base size, then shrinks to smaller than base size, then returns to base size) or a “flashing” presentation where item appears and disappears

Dev Feedback

• Anup: Maybe the ability to clone profiles would be useful?
• Anup: Would providing the current profile name somewhere on the screen be useful for the User, to ensure they are using the correct one.
• Anup: The scene needs to redone for the “actual shelf” rather than the stand-in shelf currently in place.

Proposed Roadmap to get the MVP out the door

• Migrate the primary P300 shelf scene to using the actual shelf.
• Update the BCI stimulus settings menu
• Hook up the BCI stimulus menu so that the options make the changes
• Remove settings that aren’t currently functional (different animations or texture) – we could gray them out just to indicate that they are planned but not working. At least that way the User isn’t pressing buttons that don’t do anything.
• Fix the Python backend
• Test all the software on the Surface Pro