Crash if trying to read register with no data

[ablot]

If I have a behaviour box and do a recording where the output valve was never triggered, the OutputSet register is empty and the corresponding file (Behavior_34.bin) is empty.

Trying to access the register (to check if there is any event) returns a cryptic "KeyError"

Code:

import harp as harp_pkg

reader = harp_pkg.create_reader(harp_folder)
reward_messages = reader.OutputSet.read()

Error:

  File "/nemo/lab/znamenskiyp/home/users/blota/code/cottage_analysis/cottage_analysis/io_module/harp.py", line 547, in read_from_binaries
    reward_messages = reader.OutputSet.read()
  File "/nemo/lab/znamenskiyp/home/users/blota/code/harp-python/harp/reader.py", line 91, in parser
    result = f(df)
  File "/nemo/lab/znamenskiyp/home/users/blota/code/harp-python/harp/reader.py", line 111, in parser
    return DataFrame({n: f(df[0]) for n, f in lookup}, index=df.index)
                           ~~^^^
  File "/camp/home/blota/.conda/envs/onix-3dvision/lib/python3.13/site-packages/pandas/core/frame.py", line 4113, in __getitem__
    indexer = self.columns.get_loc(key)
  File "/camp/home/blota/.conda/envs/onix-3dvision/lib/python3.13/site-packages/pandas/core/indexes/range.py", line 415, in get_loc
    raise KeyError(key) from err
KeyError: 0

Suggestion: if the output from np.fromfile is empty, return an empty dataframe

[bruno-f-cruz]

I agree that the behavior can be improved on the reader side. But I am a bit puzzled how you ended up with a 0-byte file. We usually create files "lazily,", i.e. only when a message is propagated to the writer in Bonsai. Can you share your acquisition pipeline?

[ablot]

We use the messagewriter with dump_register to True in the harp behaviour. There might have been some bonsai crash? It did create files and the analog input can be read normally from the binary file. But all the registers that I don't use created files with 0 bytes. I'll see if it happens again, I can't recall anything particular that day.

HarpBehaviour.bonsai.zip

[m0ose01]

I agree that the behavior can be improved on the reader side. But I am a bit puzzled how you ended up with a 0-byte file. We usually create files "lazily,", i.e. only when a message is propagated to the writer in Bonsai. Can you share your acquisition pipeline?

I have been troubleshooting issues with empty files as well. Some data doesn't save when I launch Bonsai via the CLI and terminate the program with Ctrl-C. This also creates a bunch of empty register dump files. My guess is that the writer opens/creates the files when it receives the message but doesn't flush the buffer until the sequence properly terminates, or enough data is acquired for that register. I assume a crash could also cause this.