Saving to Local starts giving error by hsemwal · Pull Request #198 · miniscope/mio

hsemwal · 2026-07-23T00:20:02Z

Hi @sneakers-the-rat - I am getting the following terminal text repeated when I attempt to record video with MIO for long (>10 minutes)

--- Logging error ---
Traceback (most recent call last):
File "\Programs\Python\Python313\Lib\logging\handlers.py", line 80, in emit
self.doRollover()
~~~~~~~~~~~~~~~^^
File "\Lib\logging\handlers.py", line 185, in doRollover
self.rotate(self.baseFilename, dfn)
~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
File \Lib\logging\handlers.py", line 121, in rotate
os.rename(source, dest)
~~~~~~~~~^^^^^^^^^^^^^^
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: '\logs\mio.log' -> '\mio\mio\logs\mio.log.1'
Call stack:
File "", line 198, in _run_module_as_main
File "", line 88, in run_code
File "C:\Users\hsemwal\Documents\Github\mio.venv\Scripts\mio.exe_main.py", line 6, in
sys.exit(cli())
File "C:\Users\hsemwal\Documents\Github\mio.venv\Lib\site-packages\click\core.py", line 1157, in call
return self.main(*args, **kwargs)
File "C:\Users\hsemwal\Documents\Github\mio.venv\Lib\site-packages\click\core.py", line 1078, in main
rv = self.invoke(ctx)
File "C:\Users\hsemwal\Documents\Github\mio.venv\Lib\site-packages\click\core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "C:\Users\hsemwal\Documents\Github\mio.venv\Lib\site-packages\click\core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "C:\Users\hsemwal\Documents\Github\mio.venv\Lib\site-packages\click\core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "C:\Users\hsemwal\Documents\Github\mio.venv\Lib\site-packages\click\core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "C:\Users\hsemwal\Documents\Github\mio\mio\cli\stream.py", line 106, in capture
daq_inst.capture(
File "C:\Users\hsemwal\Documents\Github\mio\mio\stream_daq.py", line 616, in capture
self._handle_frame(
File "C:\Users\hsemwal\Documents\Github\mio\mio\devices\gs\daq.py", line 110, in _handle_frame
self.logger.debug("Saving header metadata")
Message: 'Saving header metadata'
Arguments: ()
WARNING [mio.streamDaq.buffer] Frame 17010; Buffer 136083 (#2 in frame) stream_daq.py:182
Expected buffer data length: 13120, got data with shape 13120.
Padding to expected length
DEBUG [mio.GSStreamDaq] Saving header metadata daq.py:110
DEBUG [mio.streamDaq.buffer] ----buffer #2 stored stream_daq.py:389
--- Logging error ---
Traceback (most recent call last):
File "C:\Users\hsemwal\AppData\Local\Programs\Python\Python313\Lib\logging\handlers.py", line 80, in emit
self.doRollover()
~~~~~~~~~~~~~~~^^
File "C:\Users\hsemwal\AppData\Local\Programs\Python\Python313\Lib\logging\handlers.py", line 185, in doRollover
self.rotate(self.baseFilename, dfn)
~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\hsemwal\AppData\Local\Programs\Python\Python313\Lib\logging\handlers.py", line 121, in rotate
os.rename(source, dest)
~~~~~~~~~^^^^^^^^^^^^^^
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\Users\hsemwal\AppData\Local\mio\mio\logs\mio.log' -> 'C:\Users\hsemwal\AppData\Local\mio\mio\logs\mio.log.1'
Call stack:
File "", line 198, in _run_module_as_main
File "", line 88, in run_code
File "C:\Users\hsemwal\Documents\Github\mio.venv\Scripts\mio.exe_main.py", line 6, in
sys.exit(cli())
File "C:\Users\hsemwal\Documents\Github\mio.venv\Lib\site-packages\click\core.py", line 1157, in call
return self.main(*args, **kwargs)
File "C:\Users\hsemwal\Documents\Github\mio.venv\Lib\site-packages\click\core.py", line 1078, in main
rv = self.invoke(ctx)
File "C:\Users\hsemwal\Documents\Github\mio.venv\Lib\site-packages\click\core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "C:\Users\hsemwal\Documents\Github\mio.venv\Lib\site-packages\click\core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "C:\Users\hsemwal\Documents\Github\mio.venv\Lib\site-packages\click\core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "C:\Users\hsemwal\Documents\Github\mio.venv\Lib\site-packages\click\core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "C:\Users\hsemwal\Documents\Github\mio\mio\cli\stream.py", line 106, in capture
daq_inst.capture(
File "C:\Users\hsemwal\Documents\Github\mio\mio\stream_daq.py", line 616, in capture
self._handle_frame(
File "C:\Users\hsemwal\Documents\Github\mio\mio\devices\gs\daq.py", line 110, in _handle_frame
self.logger.debug("Saving header metadata")
Message: 'Saving header metadata'
Arguments: ()
DEBUG [mio.GSStreamDaq] Saving header metadata daq.py:110
--- Logging error ---
Traceback (most recent call last):
File "C:\Users\hsemwal\AppData\Local\Programs\Python\Python313\Lib\logging\handlers.py", line 80, in emit
self.doRollover()
~~~~~~~~~~~~~~~^^
File "C:\Users\hsemwal\AppData\Local\Programs\Python\Python313\Lib\logging\handlers.py", line 185, in doRollover
self.rotate(self.baseFilename, dfn)
~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\hsemwal\AppData\Local\Programs\Python\Python313\Lib\logging\handlers.py", line 121, in rotate
os.rename(source, dest)
~~~~~~~~~^^^^^^^^^^^^^^
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\Users\hsemwal\AppData\Local\mio\mio\logs\mio.log' -> 'C:\Users\hsemwal\AppData\Local\mio\mio\logs\mio.log.1'
Call stack:
File "", line 198, in _run_module_as_main
File "", line 88, in run_code
File "C:\Users\hsemwal\Documents\Github\mio.venv\Scripts\mio.exe_main.py", line 6, in
sys.exit(cli())
File "C:\Users\hsemwal\Documents\Github\mio.venv\Lib\site-packages\click\core.py", line 1157, in call
return self.main(*args, **kwargs)
File "C:\Users\hsemwal\Documents\Github\mio.venv\Lib\site-packages\click\core.py", line 1078, in main
rv = self.invoke(ctx)
File "C:\Users\hsemwal\Documents\Github\mio.venv\Lib\site-packages\click\core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "C:\Users\hsemwal\Documents\Github\mio.venv\Lib\site-packages\click\core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "C:\Users\hsemwal\Documents\Github\mio.venv\Lib\site-packages\click\core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "C:\Users\hsemwal\Documents\Github\mio.venv\Lib\site-packages\click\core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "C:\Users\hsemwal\Documents\Github\mio\mio\cli\stream.py", line 106, in capture
daq_inst.capture(
File "C:\Users\hsemwal\Documents\Github\mio\mio\stream_daq.py", line 616, in capture
self._handle_frame(
File "C:\Users\hsemwal\Documents\Github\mio\mio\devices\gs\daq.py", line 110, in _handle_frame
self.logger.debug("Saving header metadata")
Message: 'Saving header metadata'
Arguments: ()
WARNING [mio.streamDaq.buffer] Frame 17010; Buffer 136084 (#3 in frame) stream_daq.py:182
Expected buffer data length: 13120, got data with shape 13120.
Padding to expected length
DEBUG [mio.GSStreamDaq] Saving header metadata daq.py:110
DEBUG [mio.streamDaq.buffer] ----buffer #3 stored stream_daq.py:389
--- Logging error ---
Traceback (most recent call last):
File "C:\Users\hsemwal\AppData\Local\Programs\Python\Python313\Lib\logging\handlers.py", line 80, in emit
self.doRollover()
~~~~~~~~~~~~~~~^^
File "C:\Users\hsemwal\AppData\Local\Programs\Python\Python313\Lib\logging\handlers.py", line 185, in doRollover
self.rotate(self.baseFilename, dfn)
~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\hsemwal\AppData\Local\Programs\Python\Python313\Lib\logging\handlers.py", line 121, in rotate
os.rename(source, dest)
~~~~~~~~~^^^^^^^^^^^^^^
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'Cmio\logs\mio.log' -> 'C~\mio\logs\mio.log.1'

[26-07-22T17:01:35] DEBUG [mio.streamDaq.iter_buffers] Got end of recording exception, breaking

📚 Documentation preview 📚: https://miniscope-io--198.org.readthedocs.build/en/198/

- trying to change the config file to match the firmware - generalize the config file and stream_daq.py

- Investigating where buffer per frame value is getting set at 3, likely self._buffer_npix is getting its values - adjusting the StreamBit header file and the config file according to

- implemented on VS code - modified pixel value depth to 8 bits instead of 10 - csv not outputting correct frame (frame number increasing on oscilloscope, but not on the csv counter through MIO)

- changing yml file

- added MSUS config file

and changed baud rate to 5 MHz for FPGA

…number of buffers and dummy words to better match the firmware)

- changing the settings on the config and stream.py files, still not getting any images - only 3 data buffers, possibly error is coming from here

…h/buffer block length) NOT WORKING

- added MSUS config file

and changed baud rate to 5 MHz for FPGA

…number of buffers and dummy words to better match the firmware)

- changing the settings on the config and stream.py files, still not getting any images - only 3 data buffers, possibly error is coming from here

…h/buffer block length) NOT WORKING

Copied last function in types.py and copied to my code (maybe accidentally changed)

Changed yml MSUS file bitstream pointer to 5 MHz

Changed blocksize back to 4000, which should include the data+header

For some reason MSUS config had an incorrectly tabbed if statement, hopefully fixed in this commit

Output data that the FPGA receives (so far, just \x00\x8f\xff)

right now the data is not properly being read by MIO and the output text file gives some 0x00....0x8f....0xff which is not what is being sent out from the dev board

msus - test frame processing values with bright/dark sample

Converting datastream to 8bit value. Will need to modularize this a bit better, currently inside of buffer_to_array

Increased time for debug and comments

Just modified some comments and added a test as a proof of concept

This reverts commit 17a69c2.

We now have video recording capabilities on MIO

minor update to mio before starting branch to fix 12 MHz version

quick check before branching

Implemented ROI row by row flexibility

Implementing ROI based MSUS/MIO interface (changed to 8 MHz)

Branch for 12 to 8 bit conversion that is already processed on board the MSUS

Corrected yaml file for new data block sizes

Created an MSUS ROI Yaml file for testing

testing for ROI from MSUS

changes to MSUS ROI for the smaller region of interest MSUS ROI (8 MHz output, 120x120x8 pixel)

Modifeid some of the settings for ROI on GSM (12 to 8 bit processing) full frame

sneakers-the-rat · 2026-07-23T00:38:02Z

this would be an issue, not a pull request, but let me see

sneakers-the-rat · 2026-07-23T00:39:10Z

note the error message:

PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\Users\hsemwal\AppData\Local\mio\mio\logs\mio.log' -> 'C:\Users\hsemwal\AppData\Local\mio\mio\logs\mio.log.1'

is either of those files open in another program?

hsemwal · 2026-07-23T15:53:21Z

Noted! No there is no other process that should be using those log files, I also noticed a ".log.1" file being created, maybe the log file is getting too large so it creates a new log.# file and that is where the errors start to show?

sneakers-the-rat · 2026-07-24T01:09:14Z

right, yeah, that's what's happening, it is trying to roll over the log files, but it can't because something is holding a reference to it. this could be just a windows thing, but we probably need to split out the loggers by process anyway. i'm going to close this and convert to an issue

sneakers-the-rat · 2026-07-24T01:11:42Z

an example of raising an issue - #199

hsemwal and others added 30 commits December 3, 2024 14:04

- config file specific for the MSUS

0d10be6

MSUS Io

4ceec64

- trying to change the config file to match the firmware - generalize the config file and stream_daq.py

MIO GS Setup

bcc1739

- Investigating where buffer per frame value is getting set at 3, likely self._buffer_npix is getting its values - adjusting the StreamBit header file and the config file according to

MIO GS

28d8a9f

- implemented on VS code - modified pixel value depth to 8 bits instead of 10 - csv not outputting correct frame (frame number increasing on oscilloscope, but not on the csv counter through MIO)

MIO

9d3fa94

- changing yml file

MIO MSUS

9149d73

- added MSUS config file

Created MSUS config file

2cde49b

Creating stream buffer header for MSUS file/firmware

905feb8

Changed the stream buffer header to MSUS firmware values

6995e1b

and changed baud rate to 5 MHz for FPGA

MSUS MIO: Changing some config files to see effect on image (changed …

3bf16cc

…number of buffers and dummy words to better match the firmware)

MIO MSUS

fc4c589

- changing the settings on the config and stream.py files, still not getting any images - only 3 data buffers, possibly error is coming from here

Adapting code for custom MS project (changed bitstream and pixel dept…

c218efb

…h/buffer block length) NOT WORKING

Changed the baud rate configuration, to enable 12 MHz output

0c12053

Changed the baud rate configuration, to enable 12 MHz output

9ee4a11

Merge remote-tracking branch 'origin/MSUS_config' into MSUS_config

826b686

MIO MSUS

b16aa25

- added MSUS config file

Created MSUS config file

773b0af

Creating stream buffer header for MSUS file/firmware

6691f83

Changed the stream buffer header to MSUS firmware values

9b9d91c

and changed baud rate to 5 MHz for FPGA

MSUS MIO: Changing some config files to see effect on image (changed …

205cf4c

…number of buffers and dummy words to better match the firmware)

MIO MSUS

13d4b3a

- changing the settings on the config and stream.py files, still not getting any images - only 3 data buffers, possibly error is coming from here

Adapting code for custom MS project (changed bitstream and pixel dept…

aafc76d

…h/buffer block length) NOT WORKING

Changed the baud rate configuration, to enable 12 MHz output

bc5f823

Merge remote-tracking branch 'origin/MSUS_config' into MSUS_config

e20698b

MIO MSUS types

64fa022

Copied last function in types.py and copied to my code (maybe accidentally changed)

Changed bitstream rate to 5Mhz

3c854a5

Changed yml MSUS file bitstream pointer to 5 MHz

Changed Blocksize (Buffersize)

497f0ab

Changed blocksize back to 4000, which should include the data+header

Fixed broken tabbed return in types.py

a898133

For some reason MSUS config had an incorrectly tabbed if statement, hopefully fixed in this commit

Added Debug Mode Output

1102cef

Output data that the FPGA receives (so far, just \x00\x8f\xff)

Debugging MSUS and streaming in data

ac7ae55

right now the data is not properly being read by MIO and the output text file gives some 0x00....0x8f....0xff which is not what is being sent out from the dev board

sneakers-the-rat and others added 22 commits August 1, 2025 20:55

lint

e29432f

test for bright and dark frames

f794465

finish bright dark test

201c229

Merge pull request #129 from Aharoni-Lab/MSUS-test-bright-dark

c954037

msus - test frame processing values with bright/dark sample

Convert to 8 bit value

5bde1f7

Converting datastream to 8bit value. Will need to modularize this a bit better, currently inside of buffer_to_array

Mino Changes

cf11cfe

Increased time for debug and comments

Minor Changes and additions

2c0c638

Increased time for debug and comments

Small edits to comments

32c9fcd

Just modified some comments and added a test as a proof of concept

Merge remote-tracking branch 'origin/newMSUS' into MSUS

17a69c2

Revert "Merge remote-tracking branch 'origin/newMSUS' into MSUS"

52b7c2c

This reverts commit 17a69c2.

Hotfix: Video recording on MIO with MSUS

82cb959

We now have video recording capabilities on MIO

Merge remote-tracking branch 'origin/newMSUS' into MSUS

15fb8fc

minor changes

10667ab

minor update to mio before starting branch to fix 12 MHz version

Just checking to see if everything still works

e861876

quick check before branching

ROI Row by Row flexibility

354fd7b

Implemented ROI row by row flexibility

First ROI commit

0046601

Implementing ROI based MSUS/MIO interface (changed to 8 MHz)

12 to 8 bit conversion Files

b0effd4

Branch for 12 to 8 bit conversion that is already processed on board the MSUS

Correction for new block size

cbc1c6d

Corrected yaml file for new data block sizes

MSUS ROI Yaml file

6b1e4d4

Created an MSUS ROI Yaml file for testing

testing for ROI

d43f894

testing for ROI from MSUS

MSUS ROI changes

099d1d4

changes to MSUS ROI for the smaller region of interest MSUS ROI (8 MHz output, 120x120x8 pixel)

Modification for GSM

9210eaf

Modifeid some of the settings for ROI on GSM (12 to 8 bit processing) full frame

hsemwal requested a review from sneakers-the-rat July 23, 2026 00:20

sneakers-the-rat mentioned this pull request Jul 24, 2026

File access error when rotating logs #199

Open

sneakers-the-rat closed this Jul 24, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Saving to Local starts giving error - #198

Saving to Local starts giving error#198
hsemwal wants to merge 183 commits into
mainfrom
MSUS_onboard_12_to8bit

hsemwal commented Jul 23, 2026 •

edited

Loading

Uh oh!

sneakers-the-rat commented Jul 23, 2026

Uh oh!

sneakers-the-rat commented Jul 23, 2026

Uh oh!

hsemwal commented Jul 23, 2026

Uh oh!

sneakers-the-rat commented Jul 24, 2026

Uh oh!

sneakers-the-rat commented Jul 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

hsemwal commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sneakers-the-rat commented Jul 23, 2026

Uh oh!

sneakers-the-rat commented Jul 23, 2026

Uh oh!

hsemwal commented Jul 23, 2026

Uh oh!

sneakers-the-rat commented Jul 24, 2026

Uh oh!

sneakers-the-rat commented Jul 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

hsemwal commented Jul 23, 2026 •

edited

Loading