Communicating Warnings From Bessy Python to Unity

Right now, Bessy-Python logs several warning messages with the Logger class. It would be useful to communicate some of these messages to Bessy-Unity to effectively indicate warnings or problems to users. This spike is to look into how this could be done (for task B4K-355).

Bessy-Python’s Logger class implements the Python logging module and its log levels: DEBUG, INFO, WARNING, ERROR, CRITICAL. An instance of the Logger class is created in the files that require logging (e.g. bci_controller.py, channel_selection.py, the classifiers, etc.) to log specific messages.

Note: The task description refers to Bessy-Python warnings specifically, but this might also be useful for the ERROR and CRITICAL messages as well. So, throughout this spike, “warnings” refers to any message/information that needs to be sent from Bessy-Python to Bessy-Unity and indicated to the user.

TLDR

Possible Approach - Communicate Warnings using LSL

Using an LSL stream is one approach to communicate warnings from Python to Unity. This is probably the simplest way to do this, as it would use much of the existing functionality of LSL communication already implemented between Bessy-Python and Unity.

Sending warnings from Bessy-Python through LSL

The LslMessenger class in Bessy-Python contains methods to send different kinds of information to the LSL Stream. For example, the ping() method sends the “ping” message, and the prediction(...) method sends the value corresponding to the prediction made by Bessy-Python.

LslMessenger Method

To communicate warnings, an additional LslMessenger method could be created, i.e. send_warning() or something similar, to push that information to the LSL Stream. This method would take a string as a parameter, as the message that would be sent to Unity, so it should identify the problem in some way. See below for a rough example of how this method would be used.

class LslMessenger(Messenger):
...
  def send_warning(self, warning_message):
    self.__outlet.push_sample("Warning: " + warning_message)
    
## Example
logger.warning("This is an example warning log")
self._messenger.send_warning("Specific warning message!") ## This is the what will be sent to Unity

The send_warning() method would be called wherever a warning or an error needs to be communicated to Unity. Some warnings logged by Bessy-Python may not be relevant to the user, but the send_warning() method can be added where applicable.

Python Implementation Summary

  1. Create another LslMessenger method specifically for warning messages.
  2. Call this method whenever a warning/error should be passed along to Unity.

Receiving warnings in Bessy-Unity

In Unity, information from Python (such as the predictions) is received through an LSL stream. Communicating Bessy-Python warnings to Bessy-Unity would follow that same process.

Create LSL Response for Warnings

In LSLResponses.cs, information from an LSL stream is parsed based on a Regex corresponding to each type of message (i.e. ping messages, “marker received” messages, and predictions).

A Regex for warning messages would have to be added. In the Parse method, that Regex would be used to detect a warning and create a LSLWarningResponse. This would be very similar to how the Prediction messages are parsed to create the LSLPredictionResponse class.

Create callback method for LSLWarningResponse

First, a method would need to be added to LSLResponseProvider to subscribe the LSLWarningResponse to its callback method, similar to the SubscribePredictions method.

public void SubscribeWarnings(Action<LSLWarningResponse> callback)
=> Subscribe(callback);

Then, call the SubscribeWarnings method in StartReceivingMarkers in BCIControllerBehavior, and define the callback method.

// Define warning callback method
// This follows the same implementation as the prediction response callback
protected virtual void OnWarningReceived(LSLWarningResponse warning)
{
  // Logic to handle warnings
}

Unity Implementation Summary

  1. Parse the LSL Stream for warnings and create a LSLWarningResponse.
  2. Subscribe LSLWarningResponse to a callback method that handles warnings.

Indicate Warnings to the User

After Bessy-Unity receives and processes a warning, there are a couple different ways the warning could be indicated to the user:

Handling Specific Warnings

If we decide that the response on the Unity side should differ depending on the content of the warning itself, the warning message would need to be parsed further to extract the specific warning string piped in from Bessy-Python.

Then, those warnings would each be mapped to a specific method or action, using a dictionary or enumerated types or something similar. With that mapping, specialized callback methods could be used, instead of just a “catch-all” callback method that handles all warnings generically.

See the section below for an example of how the whole warning communication process would look.

Example Warning Communication

This is a rough overview of the warning communication process.

For this example, let’s say the warning “No EEG data available” needs to be sent from Bessy-Python to Unity.

  1. Bessy Python logs the warning in the logger.

The following steps are based on the implementation described in this spike.

  1. Bessy-Python calls the send_warning method from LslMessenger to push the string “Warning: No EEG data available” to the LSL Stream.
  2. Bessy-Unity parses the LSL Stream and finds a warning.
  3. The warning message is extracted as “No EEG data available.” The mapping for the warning strings is used to call the callback method for the “No EEG data available” warning.
  4. The callback method provides the logic to indicate the warning to the user. For example, a textbox appears overlaid in the Unity scene with the words “No EEG data” or something similar.

TLDR

To communicate warnings from Bessy-Python to Bessy-Unity, using the LSL communication system already in place would probably be the “easiest” approach. We would just need to implement some additional functionality in Bessy-Python to send warning messages through LSL, and in Bessy-Unity to receive and respond to warnings.