Page 1 of 1

CSV file format errors for markers

Posted: Thu Nov 28, 2019 6:43 pm
by debra peri
Hello,

I'm encountering errors when reading Muse Monitor CSV files that contain "blink" and "jaw_clench" messages. The format changes, the number of columns are not consistent with the rest of the file. For example:

As per the header layout, the columns are:

# TimeStamp,
# Delta_TP9,Delta_AF7,Delta_AF8,Delta_TP10,
# Theta_TP9,Theta_AF7,Theta_AF8,Theta_TP10,
# Alpha_TP9,Alpha_AF7,Alpha_AF8,Alpha_TP10,
# Beta_TP9,Beta_AF7,Beta_AF8,Beta_TP10,
# Gamma_TP9,Gamma_AF7,Gamma_AF8,Gamma_TP10,
# RAW_TP9,RAW_AF7,RAW_AF8,RAW_TP10,AUX_RIGHT,
# Mellow,Concentration,
# Accelerometer_X,Accelerometer_Y,Accelerometer_Z,
# Gyro_X,Gyro_Y,Gyro_Z,
# HeadBandOn,
# HSI_TP9,HSI_AF7,HSI_AF8,HSI_TP10,
# Battery,
# Elements

However, a row that contains a "blink" or a "jaw_clench" message breaks this format:

# 2019-11-24 18:56:23.826,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,/muse/elements/blink

# 2019-11-24 18:56:23.826,
# , , , ,
# , , , ,
# , , , ,
# , , , ,
# , , , ,
# , , , , ,
# , ,
# , , ,
# , , ,
# ,
# , , , /muse/elements/blink

# 2019-11-24 18:27:14.523,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,/muse/elements/jaw_clench
# 2019-11-24 18:27:14.523,
# , , , ,
# , , , ,
# , , , ,
# , , , ,
# , , , ,
# , , , , ,
# , ,
# , , ,
# , , ,
# ,
# , , , /muse/elements/jaw_clench


As opposed to a typical EEG message:
# 2019-11-24 18:10: 37.955,-0.056779124,-0.4869447,0.0513708,-0.11849284,-0.22336058,-0.6603433,-0.22185676,-0.3285773,-0.29817685,-0.4057947,0.10669098,0.11137576,-0.07063401,-0.44569674,0.4892052,0.107916296,-0.26754653,-0.82047045,0.3018993,-0.117138065,843.73627,847.3626,822.381,853.4066,786.52014,-0.190185546875,0.1046142578125,0.9686279296875,0.28411865234375,1.487884521484375,3.521575927734375,1,1.0,1.0,1.0,1.0,75.09

# 2019-11-24 18:10:37.955,
# -0.056779124, -0.4869447, 0.0513708, -0.11849284,
# -0.22336058 -0.6603433, -0.22185676, -0.3285773,
# -0.29817685, -0.4057947, 0.10669098, 0.11137576,
# -0.07063401, -0.44569674, 0.4892052, 0.107916296,
# -0.26754653, -0.82047045, 0.3018993, -0.117138065,
# 854.21246, 855.82416, 853.4066, 859.0476, 853.4066,
# -0.190185546875, 0.1046142578125, 0.9686279296875,
# 0.28411865234375, 1.487884521484375, 3.521575927734375
# ,1
# ,1.0,1.0,1.0,1.0,
# 5.09


Can you confirm this is what Muse Monitor is outputting and if so please update the app?


Thanks,
Debra Peri

Re: CSV file format errors for markers

Posted: Thu Nov 28, 2019 8:42 pm
by James
You're right. That's a bug.
I need to add two extra commas when the Algorithms Mellow and Concentration are enabled.

Re: CSV file format errors for markers

Posted: Fri Nov 29, 2019 9:54 am
by James
Hi Debra,
I checked my code and I am already putting in the correct number of commas.
The only way I am able to re-create the situation you experienced was to have the algorithms enabled, start recording and then disable the the algorithms while recording.

Is this what you did? If not can you please let me know what version of Muse Monitor you are running and what platform, Android or Apple.

Re: CSV file format errors for markers

Posted: Fri Nov 29, 2019 11:40 pm
by debra peri
Hi James,

I'm going to be double checking again today, I'm not entirely sure yet but what you describe may have happened.
I found that odd error in an older CSV file.

Muse Monitor is running on Android. I'll try to get to this tonight, very busy with holiday stuff. Thanks for the reply and advice!


--Debra

Re: CSV file format errors for markers

Posted: Sun Dec 01, 2019 4:50 pm
by debra peri
Hi James,

I think that the few files I encountered this error were somehow corrupted by changing options while recording. It's the only thing that makes sense.

In the end it was a good exercise to go through, it helped me cleanup some code and to also consider ways of identifying the file before running analysis on the data. I don't seen an option on your app (running on Android) to select or disable the algorithms, so I have to assume it must of been something I fiddled with during the recording process.

Anyway, I think I have it sorted out now!

Just about ready to release a toolset to generate charts from the data. Yay for pandas, matplotlib and numpy!!! :-)


Thanks again,
Debra

Re: CSV file format errors for markers

Posted: Sun Dec 01, 2019 5:21 pm
by James
The algorithms are enabled/hidden by tapping on the app name at the bottom of the settings list 10 times.
They are hidden like this because I don't think they're very good! So, I would recommend leaving them off ;-)

Re: CSV file format errors for markers

Posted: Sun Dec 01, 2019 6:12 pm
by debra peri
Oh! I must have stumbled upon the option accidentally! Another reason to get more sleep ....

I do encourage you to continue to develop algorithms in your app, there's so much fascinating ways to look at the data.


Take care,
Debra