Skip to content

mef3io 2.1.0

Released 2026-10-31

A minor release that makes writing many channels much faster. Nothing that worked in 2.0 changes.

What's new

  • Write many channels in one call. Writer.write_int32_many(channels, data, ufact, start_uutc, fs) (integer counts) and Writer.write_many(...) (float data, NaN gaps, precision decided per channel) write the same stretch of time for all channels at once. The files are the same as a loop of write_int32 / write calls would produce (a fresh segment differs only in its random UUIDs). The channels are encoded and written in parallel on the Writer's n_threads, and the disk is waited for once per call instead of once per channel. Also in C++ (SessionWriter::write_int32_many, write_float_many), the C ABI (mef3io_writer_write_int32_many) and MATLAB (Writer.writeInt32Many, integer counts only). Speed: on synthetic data (64 channels of 10 minutes at 2048 Hz, appended in 4 rounds, 12 threads, one SSD, one 12-core machine) a loop of write_int32 took 5.2 s and write_int32_many 0.76 s. Your gain depends on your disk and channel count; it is largest with many channels and a disk where waiting for a sync is slow. Numbers and the setup are in the benchmarks.
  • Writer(flush="close") (opt-in). No wait for the disk during writes; close() or flush() does one combined wait. If the power is cut while writing, the whole session can become unreadable, and recover_session cannot be relied on to fix it. Use it only for output you rebuild after any failure. The default, flush="each", is unchanged. It is refused together with wait_for_disk=True and on a session that already existed when the Writer opened, so earlier recordings are never put at risk. Batching with write_int32_many already removes most of the waiting; in the same synthetic test flush="close" saved only about 10 % more on top of it. See When is my data on disk?.

Failures in a multi-channel write

Every refusal (a duplicate channel, a different rate or conversion factor than what is stored, data starting before the stored end) is decided before any file is touched. A failure before the metadata files are published rolls every channel back. A failure while they are being published raises PartialWriteError, whose written_channels and incomplete_channels say what was written. If a roll-back itself fails, RollbackIncompleteError names the channels whose files may be torn, never a silent "nothing written": run validate and recover on them.

Changed

  • Writing a fresh segment copies the encoded blocks once less.

Upgrading from 2.0

Nothing is required: no signature, default or file format changed, and the new calls and option are additive. To get the speed-up, replace a loop of write_int32 / write over channels with one write_int32_many / write_many call, and write long runs in chunks of minutes (each call holds its channels' data and encoded blocks in memory).