Notes From Demo 1

Friday September 8, 2023 @ 9 am on Zoom

Attendees:

• Speech Language Pathology|Speech Language Pathologists
  • Beverly Collisson
  • Georgie Brewin
  • Missing 3rd person
• Dev team
  • UofC
    • Eli Kinney
    • Anup Tuladhar
    • Dan Comaduran
  • UofA
    • Hosein Bahari
    • Elena Arnold
    • Alex Liu

Demo notes:

• Demo run by Eli Kinney
• Current plan is to run this in the Unity editor for now – as part of being a Minimum Viable Product. Eventually have a more “clean” piece of software that can be simply launched as an executable.
• Demonstrated the VSD_P300-3D scene created by Eli Kinney in the past week.
• Do want the final product to be 2D
• Goal for next week: The 2D items can be selected.
• Proposed using generated Voice/Sound output from AudioLDM2

Feedback:

• Georgie Brewin:
  • The ability to modify the visuals will be of value, as many of the students that the product will be used with have aspects of visual impairment, and the vision “therapists” will have some input on what the best visuals are for the children.
  • Expressed interest in having an “outline” indicator in the actual product as happens in the editor (which shows an orange outline on selected objects) – as the “vision therapists” had previously recommended that having “thick apparent outlines” on objects is helpful for kids with visual challenges.
  • Prefers the 2D real image
    • The interface where the 2D real image pops up on selection shows a real image of the object, but the size, angle, lighting is different (e.g. the picture was taken at a different angle for the specific object, like maracas, than for the whole shelf of objects). She thinks this is too distracting.
  • She likes the features on the 3D model
  • Ideally love to have the “real image” but with the features of the 3D model (outline, black and white, etc)
  • To do: Get a picture of an empty shelf
  • In the long run, definitely see the advantage of going with the 3D. It can be realistic, and can manipulated easily.
  • For the purpose of the current research study, using the 2D scene with real images is fine and preferred.
    • While there is concern on the labour required to add new objects, for the current research study they’ll stick to the 5 existing objects.
    • To Do: Get good real images of the 5 objects such that we can clip out only the object without the background
• Beverly Collisson
  • Important to minimize the cognitive load for students using the product – if the visuals are overwhelming or require more cognitive load to process, then they lose cognitive bandwidth for doing the actual learning.
  • In terms of 2D vs. 3D, expressed some concern on the tradeoffs in using 2D. Namely, in terms of the manipulations of the visual object (such as texture, colour) that may enable a better VEP. So there’s a tradeoff between visual ease of use vs. BCI appropriateness.
  • For sound output, prefer a female and youth sounding voice, not too adult not too young, middle frequency. This is the student’s voice, so we want it to be unique. We decided to go with MP4 voice clips provided by the SLP’s to play upon selection.

Next week:

• Expecting from SLP’s
  • MP4 voice clips
  • Empty shelf
  • Pictures of the objects on some sort of white/green screen
• From us:
  • Focus on 2D real image version.
  • Use the actual pictures of the objects.
  • Highlight selected objects with a border, like the orange border we see when objects are selected in the editor
  • Make it work with BCI-Essentials-Python