Error Handling and Outputs

Problem

There’s several instances where BessyPy crashes due to imperfect input. Crashing isn’t acceptable, but … what should happen then?

If we imagine BessyPy as a server that stays running no matter what, then it must gracefully handle errors. Spike on what this looks like and get team on board. Example cases:

• If a trial has been ended before enough markers were collected, then the data is just discarded (rather than crash Bessy crashes if we don’t have enough training data for predictions).
• If there’s no EEG data available, a real server would just sit tight and wait. Perhaps signalling that it’s waiting for data to the outside world (rather than crash Bessy exits if EEG stream not available and Bessy crashes if EEG stream is available but no data has been sent)
• When the EEG data stream contains gaps caused by dropped packets, perhaps that data is discarded, but the training session may continue (rather than crash We have an issue in bci_data.py with dropped packets)

Considerations:

Error handling is in part due to not having a standardized manner for checking and validating our expected outcomes across the code. Structuring so that we can effectively both handle errors, report these errors and then more easily parse information coming from different objects can help us better structure the code base in general.

General Thoughts:

Bessy Python (and eventually Unity) should opt to loudly yell, rather than crash, at errors. This means we should look at what the expected behavior outcome should be, validate that there is the correct data available to achieve that expected behavior (using Pydantic), and report separately errors that occur either in the data available or the code processing the data.

Pydantic:

In short, Pydantic is the most widely used data validation library for Python, with over 70 million downloads a daty. What it does is enable you to define how data should be in a pure pythonic way, then validate it with Pydantic. This also plays nicely with LLMs and other JSON Schema systems, which can make our lives easier down the line.

Here is an example:

Another example with both validation and error outputs

What it does:

Pydantic uses instantiated models to define a schema which inherit from the pydantic.BaseModel with defined fields as annotated attributes. This is similar in ppractice to structs in C, or the requirements of a single endpoint in an API. In the above, you can see the use of BaseModel in this custom class Delivery to make sure it is validated by pydantic.

The true advantage of using Pydantic is that it is a python package providing data parsing and validation based on type hints. This is enforced at runtime, providing user-friendly errors when data is invalid. It has robust “Validation” schemes that can output errors including how they occurred in a JSON representation of the errors, as well as human-readable representations.