Automated Model Card Generation V2

• ssvep_offline_test_tf–temperature0.3–seed42–20240510_163410.json
• ssvep_offline_test–temperature0.3–seed42–20240510_163334.json
• rs_offline_test–temperature0.3–seed42–20240510_163056.json
• p300_SingleChannelERP_offline_test–temperature0.3–seed42–20240510_162036.json
• p300_offline_test–temperature0.3–seed42–20240510_155135.json
• p300_ch_select_test–temperature0.3–seed42–20240510_162014.json
• mi_offline_test–temperature0.3–seed42–20240510_161506.json
• mi_ch_select_test–temperature0.3–seed42–20240510_161438.json

What this is about

Generated “model cards” based on the Proposed JSON schema for BCI model cards – v0.0.0.1 for a number of examples from BessyPython – the offline series of examples and the ch_select_test series of examples. A quick way to view them in a more human-readable way is to copy the file contents into the online JSON Crack Editor. There is also a VSCode extension version of JSON Crack for visualizing it within VSCode.

Summary

The two inputs to the pipeline are:

1. A subset of the stacktrace
2. The experiment file (i.e. the python code run, e.g. ssvep_offline_test.py),

Overall the current pipeline works well. For example, for the ssvep_offline_test_tf.py example it pulled out the array of sampling frequencies. It also seems to work well for the ch_select_test series of examples – it would be good to get Brian Irvine ’s feedback on whether the ch_select_test series of model cards are correct.

It does struggle with categorizing the classification parameters into the 3 sub-categories I defined (“model architecture” related, hyperparameter related, or training parameter related) – but nonetheless still captures the parameters. This is something that can be refined, and would probably be improved if we also included the relevant files from the library used (e.g. the relevant code files from BessyPython). It could also be that my separation of the classification parameters into these is ill defined.