Skip to content

Python guide

import mef3io

Two API layers ship in the package:

  • mef3io.Reader / mef3io.Writer — the native API (recommended).
  • mef3io.MefReader / mef3io.MefWriter — a drop-in for legacy mef_tools.io code.

Times everywhere are uUTC: microseconds since the Unix epoch, as int.

Errors from the core raise a subclass of mef3io.MefError carrying the C++ message: PasswordError (missing or wrong password), CrcError (a checksum does not match), FormatError (not valid MEF 3.0), IoError (file system, naming) and WriteConflictError (a write that does not fit the session, such as an append that overlaps). MefError subclasses RuntimeError, so older except RuntimeError code keeps working. mef3io.Cancelled (see Cancelling and progress) is deliberately not a MefError.

Writing

with mef3io.Writer("session.mefd", overwrite=True, units="uV") as w:
    w.write("ch1", data, start_uutc, fs=256.0)

Writer(path, overwrite=False, password1="", password2="", units=None, block_length=None, n_threads=0, metadata=None, wait_for_disk=False, flush="each")

  • overwrite=True deletes an existing session first. With overwrite=False, an existing session is reopened for appending — state is recovered from disk, so this works across program runs.
  • Encryption: pass both passwords (MEF has no "level-1 only" files; see the encryption model). It encrypts the subject fields, technical metadata and annotation text, not the samples or their timing, which decode without a password; see What encryption protects. Passwords are keyed exactly as meflib keys them, so a session opens with the same password in pymef and mef_tools. A password longer than 15 characters (or 15 UTF-8 bytes), or one with a character above U+00FF, raises a PasswordInteropWarning; mef3io.password_interop_issue(password) returns the same reason (or None) without writing anything, so an application can warn first. A session that mef3io 1.2.0 or earlier encrypted with a non-ASCII password still opens, with a LegacyPasswordWarning (see password derivation).
  • Appending to an encrypted session needs the level-2 password. That includes sessions at offset 0. Section 3, which holds the recording-time offset every new time is stored against, is level-2 encrypted, and without it the offset is unknown. A Writer opened with only the level-1 password is refused (PasswordError) for anything that stores times into that session, rather than guessing the offset.
  • Writer.unreadable_metadata lists segment metadata the writer could not use ("path: reason"):
  • .tmet files it could not read while choosing the offset. At offset 0 such a file is skipped, so one damaged segment does not block unrelated writes. Where the offset is non-zero, or nothing else is readable, the write is refused instead.
  • Segment directories without metadata (what a crashed write can leave behind). These are ignored, never adopted. Segments are numbered by the highest real segment number, so such a directory is neither appended to nor collides with a new segment. An entry is dropped once a new segment fills that directory.

Every entry raises an UnreadableSegmentWarning, when the Writer opens or during the write that finds it (mef3io:unreadableMetadata in MATLAB). - A refused write leaves the session exactly as it was. Every check runs before any directory or file is created: the record type, the time, and the offset (known and agreed). A write that fails later anyway removes the directories it created. - Cost of choosing the offset. A session-level annotation or a new channel needs the offset that every segment of the session agrees on. The first one in a Writer reads and CRC-checks each segment's .tmet record (16 KB; the offset in section 3 has no CRC of its own, so it is trusted only after the body CRC), e.g. 1.4–1.8 s and 157 MB at 64 channels × 150 segments. A channel-level annotation reads only that channel's segments. Each answer is kept for the Writer's lifetime, so every later one costs nothing. Keep one Writer open across an acquisition, as for appends. - metadata= sets subject and acquisition fields (see Session metadata). - When is my data on disk? write() returns once the data is handed to the operating system. w.flush() returns only when everything written so far is on disk (one combined wait for all channels); close() and leaving a with block do the same. wait_for_disk=True makes every write() return only when its own data is on disk. After a power cut, run recover_session. See When is my data on disk?. (2.0 removed durability=; passing it raises TypeError.) - flush="close" (opt-in, default "each") removes every wait for the disk during writes; close() / flush() does one combined wait. A power cut while writing can leave the whole session unreadable, and recover_session cannot be relied on to fix it, so use it only for output you rebuild after any failure. When close() returns, everything is on disk. Only for a session this Writer creates (ValueError for one that already existed); ValueError with wait_for_disk=True too. See flush="close". - recover_session and repair_session always keep a backup of what they change. backup=False (and the CLI's --no-backup) is still accepted but ignored since 2.0.1, with one BackupAlwaysKeptWarning. - block_length overrides the RED block size (default: fs samples for fs ≥ 5000, else 10*fs, at least 1). A block can hold at most 858 993 396 samples (mef3io._mef3io.MAX_BLOCK_SAMPLES: its byte size is a 32-bit field and the worst case is 5 bytes per sample); a longer one, or a rate whose default block would be longer, raises ValueError before anything is created. n_threads (also a settable property) is how many threads the Writer may use: 0 = automatic, 1 = no extra threads; see Threads.

Arguments are checked before anything is written. Every write (write, write_int32, and appends) raises ValueError and leaves the session as it was when fs is not a finite number greater than 0, when the conversion factor is 0, NaN or infinite (every sample would read back as 0 or NaN; 0 is also meflib's "no entry" value for the field, meflib.h:439), or when the samples would end beyond the representable time range (a rate like 1e-300). A negative conversion factor is allowed: it is a polarity inversion and reads back correctly in mef3io, pymef and mef_tools (pymef handles it explicitly, pymef3_file.c:972-978). The section-2 native minimum and maximum and block_envelope()'s min/max are swapped accordingly.

write(channel, data, start_uutc, fs, precision=-1, new_segment=False)

Float path. data is float64; NaN runs are not stored — they become discontinuity gaps that read back as NaN. Values are quantized to int32 counts as round(data * 10**precision) with the conversion factor 10**-precision kept in metadata; precision=-1 infers it from the data (and on appends reuses the segment's stored precision so appends can never conflict). precision="step" stores the data on its own quantization step instead (below).

precision="step": the smallest files for data from an amplifier

Data that came from an analog-to-digital converter is counts * step: an amplifier with 0.195 µV per count produces only multiples of 0.195. The default path cannot see that and stores powers of ten: at precision 3 every sample is counts * 195, so the differences between neighbouring samples are 195 times larger than they need to be, and RED, which stores those differences, needs far more bytes for them. With precision="step" the writer finds the step and stores the original counts with the step as the conversion factor:

s = w.write("ch1", data, start_uutc, fs=512.0, precision="step")
s["quantization"], s["units_conversion_factor"]   # ('step', 0.195)

Measured on one hour at 512 Hz of 16-bit counts × 0.195 µV: the default path writes 6.03 MB and "step" 1.52 MB, the same as write_int32 of the original counts with factor 0.195. The default path stays the default; "step" is opt-in.

How it works, and what it guarantees:

  • The step is estimated from a fixed-size sample (the first 65,536 values and 16 windows of 2,048 spread through the rest; NaN ignored), so finding it costs the same on any length: about 10 ms on 50 million samples. The estimate is snapped to the step's own float64 value, so 0.195, 0.1, 1e-3/3 or a binary step like 10e6 / 2**24 is stored as that exact number.
  • Every value is checked while it is converted (the conversion pass every write makes anyway, not a second read). A value stored this way reads back as count * step, which is exactly the value written when the data was computed as counts * step in float64; in every other case it is within float64 rounding (at most 64 × 2⁻⁵² relative). There is no third case: data that is not a whole multiple of one step is not stored on a step.
  • Data that is not on a step falls back to the default path for that write, with a StepFallbackWarning: filtered or resampled data, noise, or a step whose counts would not fit in int32. So does data that shows too little of a step to trust it: fewer than 16 distinct magnitudes in the sample, such as a DC level, a calibration square wave or a handful of levels (the level itself is "a step", and the channel would keep it). The file is then what a default write produces. A filter whose coefficients are short decimals or powers of two can keep data on a finer step, and so can adding an offset that is not a multiple of the step (`counts * 0.195
  • 0.05` is on a 0.005 step); that step is then found and the data is stored exactly — losslessly, but not always smaller than the default.
  • A channel keeps one conversion factor for new data (older segments may differ; the reader scales each segment by its own). So once a channel has data (an append, or new_segment=True), "step" stores new data on the channel's factor: exactly when the data is a whole multiple of it, and otherwise not at all — the write raises WriteConflictError and writes nothing, rather than round the data. Write such data to a new channel, or quantize it yourself and use write_int32 at the channel's factor.

The hazard for long acquisitions: the first chunk decides. If the first chunk written with "step" shows a coarser step than the data really has — only even counts (step 0.39 instead of 0.195), or an amplifier gain changed later to a finer one — every later, finer chunk is refused, for the rest of the channel's life. Nothing is lost or rounded, but the acquisition stops writing that channel. What to do: when you know the amplifier's step, write write_int32(counts, ufact) with it from the first chunk (no estimate at all); otherwise make the first chunk long enough to show the real step, and check the first summary's units_conversion_factor against what you expect. A channel already locked to a coarse factor can be continued only in a new channel (or by your own quantization at that factor with write_int32). - Keep passing precision="step" when appending to a channel written that way (or with write_int32 at a factor that is not a power of ten). The default precision=-1 can only reuse a power-of-ten factor and refuses the append with a WriteConflictError saying so; the same holds for MefWriter.write_data. With new_segment=True the default path still infers a new precision, which gives the channel a second factor (#150). - No count ever wraps. A sample whose count would leave int32 at the factor, or +inf/-inf, is stored as the nearest storable value, ±(2**31 - 1) counts (meflib's own positive/negative "infinity" values), and the write succeeds. Every float write, step or default, does this; the summary's clamped (and clamped_infinite) counts them and one ClampedSamplesWarning per write says how many. NaN is never clamped: it stays a gap. The inferred precision is chosen so both ends of the data fit, so ordinary data never needs a clamp; a typical cause is a saturation artifact in an append to a channel whose factor is fixed.

The returned summary says what happened: quantization is "step" (stored exactly on the factor) or "decimal" (rounded to 10**-precision), step_fallback is None or the reason the step was not used, and clamped / clamped_infinite count the samples stored as the nearest storable value. MefWriter.write_data(..., precision="step") does the same; Python and C++ only (Precision::detect_step() in SessionWriter::write_float), not the C ABI or MATLAB.

The first write to a channel creates segment 0; later writes append in-segment (extending the existing files, like the legacy writer); new_segment=True forces a fresh segment. Appends must match the segment's fs and conversion factor and start at/after its end time — violations raise WriteConflictError. An append extends the channel's last segment folder under its real name, so a session from a writer that does not zero-pad segment numbers (ch1-5.segd) can be appended to; new segments are always named <channel>-NNNNNN.

Returns a summary dict: samples_written, blocks, gaps_skipped, segment, units_conversion_factor, quantization, step_fallback (see above; write_int32 reports quantization="int32").

write_int32(channel, data, ufact, start_uutc, fs, valid=None, new_segment=False)

The primitive path: int32 counts stored verbatim (bit-exact round trip) with ufact (e.g. an amplifier's volts-per-bit) in metadata. Optional valid mask (same length, anything numeric — nonzero = valid): marks gap samples. Because counts are never altered, data must be integer-typed (any width): floating-point input raises TypeError (use write() for float data), and values outside the int32 range raise ValueError instead of silently wrapping. A valid mask of the wrong length raises ValueError.

write_int32_many(channels, data, ufact, start_uutc, fs, new_segment=False) and write_many(channels, data, start_uutc, fs, precision=-1, new_segment=False)

The same run of samples for many channels in one call: data is (n_channels, n_samples) (int32 counts with one ufact or one per channel, or float with NaN gaps and the precision decided per channel). The files are the same as a loop of write_int32 / write calls would produce (a fresh segment differs only in its random UUIDs), but channels are written in parallel on the Writer's n_threads and the disk is waited for once per call instead of once per channel. Returns one summary dict per channel. Refusals are decided before any file is touched; a failure before publishing rolls every channel back, one while publishing raises PartialWriteError (written_channels, incomplete_channels), and a roll-back that itself fails raises RollbackIncompleteError. Details: Many channels in one call.

write_annotations(annotations, channel=None)

Records: an iterable of dicts (time required; type defaults to "Note"; optional text, duration) or a pandas DataFrame with those columns. channel=None writes session-level records. Always appends to the records already stored at that level (the files are created the first time) and never replaces or deletes one. A later call may carry an earlier time: it is appended too (the file keeps the order of writing), and records() sorts the records within each level by time (stable: equal times keep their write order); the levels stay session, channel, segment. inspect() lists a file's records in file order. Legacy readers (mef_tools, pymef) return file order, so they show an out-of-order append unsorted. In encrypted sessions record bodies are level-2 encrypted. To start from nothing, write to a new session (or level). If a power cut interrupted an append, the next write is refused until recover_session has repaired the file (see long recordings).

Reading

with mef3io.Reader("session.mefd") as r:          # password="..." if encrypted
    r.channels                                    # list[str]
    info = r.info("ch1")
    x  = r.read("ch1", t0, t1)                    # float64, NaN gaps
    X  = r.read(["ch1", "ch2", "ch3"], t0, t1)    # 2-D: one row per channel, one call
    x4 = r.read("ch1", t0, t1, dtype="float32")   # float32 output
    xi = r.read_raw("ch1", t0, t1)                # int32 + validity mask
    r.segments("ch1")                             # per-segment map
    r.toc("ch1")                                  # block-level index
    r.block_envelope("ch1", t0, t1)               # per-block min/max, index only
    r.records()                                   # session-level annotations
    r.records("ch1")                              # ch1's channel- + segment-level ones

Reader(path, password="", backend="cpp", n_threads=0, cache=None, warn_declarations=True, strict=True)

  • A level-2 password unlocks everything; a level-1 password reads the signal and technical metadata while section-3 subject metadata stays locked (info()["section3_available"] == False, subject fields None) and records are unavailable. Without a password the Reader refuses an encrypted session; that is mef3io's policy, not protection, because the samples themselves are not encrypted (details).
  • n_threads is how many threads the Reader may use for everything it does (opening, file access, decoding, decrypting): 0 = automatic, 1 = no extra threads. It is also a settable property, used from the next call on; see Threads.
  • strict=True (the default) raises when any segment cannot be read. strict=False skips it instead: its time span reads back as NaN, it is listed in Reader.problems, and an UnreadableSegmentWarning is raised. Check problems before trusting such a read, because a skipped segment looks exactly like a recording gap.
  • warn_declarations=True raises one SessionDeclarationWarning when the session leaves buffer-size declarations unset (files from mef3io 1.1.2 and earlier, and from the legacy stack). Reading with mef3io is unaffected; see Validating and repairing.
  • A BlockCopyWarning means a block's index entry and its own header disagree on its time or length. The header is used, as meflib and pymef do; the detail is in read_raw(...)["block_copy_mismatches"].

Warnings carry the path as data

Every warning mef3io emits is a mef3io.MefWarning (a UserWarning). Its message never contains the session's location (its directory or archive path) or its name: they appear as <session>, also inside an archive and in the session's own file names. Only those are scrubbed. Channel names and the names of files inside the session (<session>/ch1.timd/ch1-000001.segd/...) can appear, and they can be identifying too. The session is on the warning object instead, with the structured items the message summarises:

with warnings.catch_warnings(record=True) as caught:
    warnings.simplefilter("always")
    r = mef3io.Reader(path, strict=False)
for w in caught:
    if issubclass(w.category, mef3io.MefWarning):
        log.warning("%s", w.message)   # no session path or name (channel names may appear)
        w.message.path                 # the session, as it was passed in
        w.message.details              # e.g. the Reader.problems entries (may hold paths)

warnings.filterwarnings("ignore", category=mef3io.MefWarning) silences all of them. Python's "default" (and "module") filter action shows each warning once per session, not once per process.

read(channel, t0=None, t1=None, *, dtype="float64", cancel=None, progress=None)

Float64 samples on the uniform fs grid over [t0, t1) (whole channel by default): N = round((t1 - t0) * fs / 1e6) samples, discontinuity gaps filled with NaN, each value scaled by the conversion factor of the segment it comes from (segments of one channel may store different ones). A segment whose factor is not set (0, MEF's "no entry": what meflib's and pymef's templates hold until a writer sets it) reads as its raw stored counts (factor 1), with one mef3io.UnscaledSamplesWarning per Reader and channel; before 2.0 it read back as zeros. A factor that is NaN or infinite is invalid and refused with mef3io.FormatError (read_raw still returns the counts). Windowed reads fetch only the needed bytes — cheap on huge sessions. cancel= / progress=: see Cancelling and progress. The thread count is the Reader's n_threads; there is no per-call override (removed in 2.0).

Several channels in one call. Pass a list (or tuple) of channel names and read returns a 2-D array, one row per name in the order given (a name may repeat):

X = r.read(["Fp1", "Fp2", "F3", "F4"], t0, t1)   # shape (4, N)
  • Every row is exactly what read() of that channel alone returns over the same window, value for value and NaN for NaN: the same grid, gaps, per-segment scaling, warnings and errors. The tests compare the bytes, at every thread count, on gappy, jittered, overlapping-block and mixed-factor channels.
  • The channels must share one sampling rate (one grid for every row). Mixed rates raise ValueError; read each rate in its own call. Unknown names (IndexError), an empty list (ValueError) and mixed rates are refused before anything is read.
  • Without t0/t1 the window runs from the earliest start to the latest end of the listed channels.
  • It is much faster than a loop: every channel's block index is read at once on the Reader's threads, then the data a group of channels at a time (as many channels as the Reader's file-access threads), with one decode pool over every block of the group (benchmarks). cancel= stops the whole call; progress= is one sequence with one constant total (the compressed bytes of every block in the window, all channels).
  • Errors are the ones a loop of single reads raises: the first failing channel in list order, whatever went wrong (its index, its data, its conversion factor or a damaged block). Nothing is started for channels after a channel known to fail, so a damaged first channel fails the call about as fast as reading it alone. With strict=False, segments skipped during the read are added to problems as each channel meets them, so entries of different channels can come in any order.
  • Memory: the result (8 bytes per sample, 4 with float32, for every channel), plus the compressed blocks of one group of channels at a time (each group's are freed once it is decoded): about 2 bytes per sample for typical iEEG, times the window, times the group size (n_threads channels; automatic = the number of cores, at least 8). A loop of single reads stacked with np.stack needs the result twice.
  • read_raw stays one channel per call.

dtype="float32" (any spelling numpy.dtype accepts) returns float32, also for one channel: each value is the float64 value rounded to the nearest float32, exactly read(...).astype(numpy.float32), without the float64 array in between. float32 keeps about 7 significant digits (relative error at most 2^-24); integer counts up to 2^24 = 16,777,216 in magnitude stay exact with factor 1. Gaps stay NaN. "float64" is the default; anything else is a ValueError.

read_raw(channel, t0=None, t1=None, *, cancel=None, progress=None)

Dict with the stored samples (int32), a valid mask (0 in gaps), start_uutc, sampling_frequency, units_conversion_factor, units_conversion_factors. Each segment stores its own conversion factor, so the counts of a window spanning segments with different factors mean different things: units_conversion_factors lists where each applies, as dicts with start_index, end_index (half-open sample indices) and units_conversion_factor — one per run of valid samples sharing a factor, sorted, from its first valid sample to just past its last:

raw = r.read_raw("ch1", t0, t1)
x = np.full(len(raw["samples"]), np.nan)
for s in raw["units_conversion_factors"]:
    x[s["start_index"]:s["end_index"]] = raw["samples"][s["start_index"]:s["end_index"]] * s["units_conversion_factor"]
x[raw["valid"] == 0] = np.nan      # == r.read("ch1", t0, t1)

There is one span for an ordinary window and none when nothing in it is valid. Factors are as stored: a span with factor 0 is a segment whose factor is not set (read() returns those counts unscaled). units_conversion_factor is that one span's factor, None when the window holds samples of segments with different factors, and info()'s value when nothing is valid.

info(channel)

Dict: sampling_frequency, units_conversion_factor (None when the channel's segments store different factors; segments() gives each segment's), units_description, start_time, end_time, number_of_samples (stored samples — gaps are not counted, so a gridded read usually returns more), recording_time_offset, n_segments, section3_available, and the section-3 subject fields (subject_name_1/2, subject_id, recording_location; None without level-2 access).

segments(channel) and toc(channel)

segments maps what data is where — one dict per segment with start_time/end_time, start_sample, number_of_samples, number_of_blocks, units_conversion_factor (the segment's own; 0.0 is MEF's "no entry"), path, and provenance (which mef3io versions created and last modified the segment and which operations it has been through; None for files from another writer or from mef3io before 1.2; see the provenance region) — from metadata only (nothing decoded), so it is cheap even for huge, gap-riddled sessions. toc is the finer per-RED-block view (start time, sample counts, extrema, discontinuity flags).

block_envelope(channel, t0=None, t1=None)

channel may also be a list of names: the result is then a dict keyed by channel name (in the order given, a repeated name once), each value the dict one name returns, {"ch1": {"start_uutc": ..., "min": ..., ...}, ...}. The channels' indexes are read concurrently on the Reader's threads; it is still index only, and any mix of sampling rates is fine (there is no shared grid).

The minimum and maximum of every data block overlapping [t0, t1), read from the block index alone: no signal is decoded and segments outside the window are skipped. It is the fast way to draw a whole day, or a whole recording, as an envelope.

env = r.block_envelope("ch1")                     # the whole channel
env["start_uutc"], env["end_uutc"]                # int64 arrays, one row per block
env["min"], env["max"]                            # float64, in the channel's units
  • The resolution is the block length: mef3io writes 10-second blocks by default (1-second blocks at 5 kHz and above); other writers choose their own. A day of one channel in 10-second blocks is 8,640 rows.
  • Values are scaled like read(): each block by its own segment's conversion factor, swapped for a negative one (a factor not set reads as the counts, with the same warning), with the same refusal of a NaN or infinite factor for the blocks read() of the same window would take samples from (read's grid rule; toc() returns the counts). A block of such a segment that the read would not touch gets NaN. A block whose writer left its extremes unset gives NaN.
  • Gaps are absent rows: compare end_uutc with the next start_uutc.
  • toc() returns the same blocks for the whole channel as one dict each, unscaled; block_envelope is the one to use for drawing.
  • The compat MefReader has it too (MefReader.block_envelope(channel, t_stamp1, t_stamp2)), so code built on the legacy API needs no second reader.

records(channel=None, segment=None, level=None)

MEF keeps record (annotation) files at three levels — session, channel and segment — and every returned dict says which: level ("session"/"channel"/"segment"), channel (None at session level) and segment (None unless segment level).

r.records()                       # session-level records
r.records("ch1")                  # ch1's channel-level AND segment-level records
r.records("ch1", segment=0)       # only segment 0's
r.records("ch1", level="channel") # only the channel file
r.records(level="all")            # every record in the session (see below)

Each dict has type, time (absolute uUTC), the three location keys, version_major/version_minor, then the body fields of its type, always the same keys for a type and None where the body does not carry one: text (Note, SyLg); duration + text (EDFA); earliest_onset, latest_offset, duration, number_of_channels, onset_code, marker_name_1/2, annotation and channels (a list of {"name", "onset", "offset"}) for Seiz; and the Epoc, Curs, CSti, ESti and LNTP fields listed in the format reference. Body times (earliest_onset, latest_offset, a Seiz channel's onset/offset, Epoc timestamp / end_timestamp, Curs trace_timestamp) are absolute uUTC like time; the format reference explains how they are resolved. If a channel's (or the session's) segments disagree on the recording-time offset and its record file stores offset-dependent times, that one file raises FormatError rather than misplacing its records (a lenient Reader skips it, lists it in problems and warns; the segment-level records are unaffected). A record file that cannot be read is handled the same way. Reading every body type is supported; writing is limited to Note, SyLg and EDFA.

Written times are stored as −t in a session at offset 0 (every session mef3io creates). In a foreign session "anonymized" with meflib's deprecated recording-time offset they are stored as rto − t, the session's own form, so mef3io never writes the date there in plain. A segment or channel mef3io adds to such a session inherits its offset. The offset does not de-identify the session, though: the date is in section 3 (see De-identification).

A write is refused, rather than misplacing times or breaking the session's own rto − t form, when: - the time is at or before the offset (WriteConflictError); - the level's segments disagree on the offset (WriteConflictError); - the offset is locked because only the level-1 password was given (PasswordError); - the offset cannot be verified because segment metadata in scope is unreadable, and the readable segments use a non-zero offset or none are readable at all (WriteConflictError).

A channel that has no data yet takes the session's offset for its annotations. Its records are listed: records("chNew") and records(level="all") include it. It is not added to Reader.channels, which lists channels with samples.

The mef_tools-compatible MefReader.get_annotations(channel) keeps the legacy meaning — that channel's channel-level records only — while Reader.records(channel) also includes its segment-level records.

Warm-start cache (opt-in)

mef3io.Reader("session.mefd", cache="auto")           # per-user OS cache dir
mef3io.Reader("session.mefd", cache="/data/s.cache")  # persistent snapshot

Snapshots channel metadata so warm opens serve channels/info without reading the session tree, deferring the backend until a data read.

Whether a snapshot is still valid is decided only when a cached open happens; writing to a session never touches it. The check is small and costs the same whatever the number of segments. Per channel it compares:

  • the segment count;
  • the start and end times;
  • the last segment's file sizes;
  • the last segment's provenance region: which mef3io operation last modified it, and how many times;
  • the random file IDs of the first and last segment's metadata files. This catches a session rewritten with overwrite=True from the same samples but with new units, conversion factor or subject.

A snapshot is used only by the exact mef3io build that wrote it (the version plus the compiled extension). On any mismatch it is ignored, the session is opened normally, and a fresh snapshot is written.

No metadata snapshot is saved for a session with no more segments than channels, for example 128 channels with one segment each. There a snapshot cannot make an open faster: measured, a cached open matched a plain one, and building the snapshot made the first open about twice as slow. A tiny marker is saved instead. It makes later cache= opens go straight to a plain open, at the cost of one small file read. The marker is dropped once the session grows past one segment per channel, and a real snapshot follows.

Encrypted sessions are never cached. They are always opened normally, and any snapshot file found for one is deleted, whatever version wrote it.

Blind spot: an in-place change to a segment that is not the channel's last is not detected if it leaves the channel's start and end, and everything checked in its last segment, unchanged. That covers:

  • edits by other tools;
  • mef3io's own repair_session or recover_session on an older segment, in the cases where it could not stamp the channel's last segment. Normally such a repair also stamps the last segment segment-repaired, which the check does see. The stamp is skipped when the last segment's provenance region belongs to another application, its CRC fails, or a writer's .mef3io-tmp file sits next to it. The repair report then says that cached opens will not see the change.

If a session may have been edited that way, open it without cache=: a plain open always reads everything.

Inspecting a session without reading its samples

mef3io.inspect(path, password="") returns a SessionSummary: everything a tool needs to know about a session, read from the metadata and the record files only. It never reads a .tdat. Data files appear only as file sizes, so inspecting a terabyte session costs about the same as opening a Reader.

s = mef3io.inspect("session.mefd")            # or "session.mefd.tar", read in place

s.layout, s.total_bytes                       # "directory" / "tar", bytes
s.start_time, s.end_time                      # first / last sample, all channels (uUTC)
s.first_time, s.last_time                     # samples AND annotations
for c in s.channels:
    c.name, c.sampling_frequency, c.number_of_samples
    c.channel_description                     # with include_values=True, else None
    c.segments                                # SegmentSummary: times, samples, sizes, provenance
    c.gaps                                    # [(start, end), ...] uUTC spans with no samples
s.records                                     # RecordSummary at every level, with times
s.encryption                                  # what is encrypted, what the password opened
s.time_offset                                 # meflib's recording-time offset, in plain words
s.identity_field("subject_id").present        # True / False / None (locked)
s.legacy_password_mapping                     # opened only through mef3io <= 1.2.0's password mapping
[w.code for w in s.warnings]
s.to_dict()                                   # plain data, ready for json.dumps

From the command line:

python -m mef3io inspect session.mefd           # text
python -m mef3io inspect session.mefd --json    # the same as to_dict()

What it reports:

Part Contents
Session name, path, layout, total size, first/last sample and record times, the recording-time offset and how it was read, GMT offset and DST fields
channels sampling rate, units, the channel description (with include_values=True), start/end, stored samples, the segment map (segments, the same fields as Reader.segments() plus file sizes, declared discontinuities and the offset state), gaps, record count
records every record at the session, channel and segment level: type, time, duration, and first_time / last_time over everything the record mentions (header time, body times such as Seiz onsets, time + duration)
identity subject name 1/2, ID and location (section 3), the session, channel and reference descriptions (section 2), session_name and anonymized_name (every universal header), and record text: where each lives, in how many places it is set, its longest length, how many distinct values. Values only with include_values=True.
warnings each with a code, a plain-words message, and blocks_anonymize; s.anonymize_blockers lists the ones anonymize_session refuses on
problems with strict=False, what could not be read (same shape as Reader.problems)
incomplete_channels channel folders without one complete segment. A segment folder missing its .tmet, .tidx or .tdat is skipped silently by a Reader; inspect lists it as readable=False, counts its files, reports what it may hold as unknown, and warns (incomplete-segments)

Options:

  • include_values=True returns identity values: subject fields, descriptions, record text and the full records (RecordSummary.fields, as Reader.records() returns them), each channel's channel_description, and the recording-time offset. Off by default. Names and paths are not de-identified either way: the session's folder name (MEF names it after the session, so it usually is the session_name), channel names and file paths, also inside warning and problem messages, are always returned. A summary of an identifiable session is itself identifiable.
  • read_index=True also reads every segment's block index (about 2% of the data), which finds gaps inside a segment. Without it, gaps lists only the gaps between segments and gaps_complete is False. An appended recording is usually one segment per channel, so ask for the index when the gaps matter. Gaps come from the index's copy of each block's time and sample count; the authoritative copy, the RED block header, is in the .tdat and is never read. gaps_certain is False when a listed gap might hold data from a segment that could not be read or placed, or when a segment whose times assume offset 0 sits next to one whose non-zero offset was read.
  • strict=False reports an unreadable segment or record file in problems and a warning, marks the segment readable=False, and describes the rest. Strict, the default, raises as a Reader does.
  • cancel= and progress= work as for the other long operations (see Cancelling and progress).

Encrypted sessions are described, not refused. Without a password, inspect still lists channels, segments, sizes and the plain universal-header fields. What the password does not open is reported as locked: encryption["access"] is "none", "level-1" or "full", the locked sections and record bodies are counted, and values that live there are None. A password that is given and wrong raises PasswordError.

A non-ASCII password that opens the session only through the password mapping of mef3io 1.2.0 and earlier (#84) is accepted, as a Reader accepts it. inspect reports it as data: legacy_password_mapping is True and a legacy-password warning in warnings counts the files. Unlike a Reader, it does not emit LegacyPasswordWarning, as it emits no Python warnings at all.

Times are never guessed silently. Every time is absolute UTC. A segment's times are resolved with its own offset.

  • Section 3 locked (the level-1 password, or none on an encrypted session): the offset cannot be read, so times are resolved assuming offset 0, as a Reader does, and flagged offset_assumed_zero=True on the session, the channel, the segment and each record concerned (only where a stored time was negated, that is relative to the offset; a time stored as an absolute value needs no assumption and is not flagged), with an offset-assumed-zero warning. mef3io and pymef always write offset 0, so these times are right for almost every file. Under meflib's deprecated date masking they are the masked times, which is what meflib meant a level-1 user to see. They are never called verified, and the warning blocks anonymization (which needs the level-2 password anyway).
  • Segment unreadable (lenient mode): its times are None, with a times-unresolved warning.

A channel- or session-level record file is read with the offset all the segments in its scope share. When they disagree, or one is unreadable, its offset-dependent times are None (RecordSummary.times_known is False, and the times in fields are None too). A locked segment takes part with its assumed 0, exactly as in a Reader, so against a read non-zero offset it is a dispute (time_offset["status"] is "disputed"), never a silent choice.

The recording-time offset is reported plainly. meflib's recording-time offset is deprecated as anonymization: the real date stays in section 3 (see De-identification and the recording-time offset). time_offset["status"] is "none", "applied", "disputed", "locked" or "unknown" (no segment readable). It describes the readable segments; segments_unknown counts the ones that could not be read, whose offset is unknown; when an offset is set, date_recoverable is True and recoverable_by says by whom ("anyone" if section 3 is plain, "level-2 password" if encrypted), with a legacy-time-offset warning. The offset's value is itself identifying (it is the local midnight of the recording day), so it is returned only with include_values=True. ambiguous is True when a time is stored non-negative under a non-zero offset, where it cannot be told apart from an absolute time; that blocks anonymization. It is checked on the metadata universal headers, record header times and, with read_index=True, the block index (header and entries). The .tdat universal header is never checked, because inspect never opens a .tdat.

Python and C++ only

inspect is available in Python and C++ (mef3io/inspect.hpp, inspect_session), not in MATLAB or the C ABI.

Several recordings read as one

Many recorders start a new session every day, so one patient's stay is a folder of .mefd sessions. mef3io.MultiReader reads them as ONE recording, without writing anything:

days = sorted(Path("/data/patient7").glob("*.mefd"))   # any order: sorted by time
with mef3io.MultiReader(days) as r:
    x = r.read("Fp1", t0, t0 + 3_600_000_000)          # an hour across midnight

It is a drop-in for Reader: the same methods and properties, with the same arguments, return types, dict keys and dtypes, so code written for a Reader works unchanged. Only the first argument differs (a list of sessions, folders or .mefd.tar archives, mixed as you like), and n_threads is the total for the whole MultiReader, split between the recordings it opens or reads at once and never multiplied by them (Threads).

What a read returns is exactly what Reader returns for the same window of the same recordings merged on disk with merge_sessions: the same length, the same values, NaN in the same places. Each recording is read on the window's own sampling grid, so nothing is rounded twice, also at rates whose period is not a whole number of microseconds.

MultiReader(paths, password="", backend="cpp", n_threads=0, cache=None, warn_declarations=True, strict=True)

  • Refused on open (ValueError): a channel with different sampling rates in different recordings; two recordings holding samples of the same channel at the same time. The overlap rule is merge_sessions': compared on the recorded blocks, and up to half a sample is recorder rounding. Recordings that take turns are fine. A single path (not in a list) is a TypeError, an empty list or the same session twice a ValueError.
  • Values are combined over every recording, never taken from one. channels is every recording's, sorted; a channel a recording lacks is a gap over its time. info() gives the earliest start, the latest end and the summed sample and segment counts; any other field is the value the recordings agree on, or None where they differ. The patient fields and the session description are combined over every recording (also those without the channel), so info() and metadata agree; metadata is a mef3io.Metadata built the same way. A difference raises one MultiReaderDifferenceWarning on open naming the fields (not units_conversion_factor: reads scale each segment by its own factor), never their values; each recording's own values are in r.metadata_by_source (and r.paths lists the recordings in time order). Different patient names or IDs can mean the recordings are not the same patient's.
  • Session descriptions. mef3io and the legacy stack store the session's name there by default, so day files named by date would always differ. A description equal to its own recording's name therefore counts as unset: real descriptions that agree give that value, only defaults give None (a single recording keeps its own), and no warning is raised for them. merge_sessions keeps the first source's description, so for this one field metadata is not what a merged session gives.
  • segments() numbers every recording's segments from 0 in time order, as a merged session does; path is the real folder, so it says which recording. toc(), block_envelope() and records() combine every recording's in time order, and records(channel, segment=k) uses the same numbering. problems and declaration_issues list every recording's.
  • Each recording's own warnings (declarations, unreadable segments, ...) are emitted once, on open, with w.message.path naming that recording.
  • cache= takes "auto" or one cache path per recording.
  • read(list_of_channels, dtype=...) and block_envelope(list) follow the Reader's rules and return what a Reader of the merged session returns. A MultiReader reads the listed channels one after the other, each with its whole budget, so it does not get the Reader's multi-channel speed-up. Its progress for a list has one constant total too, counted in units of channels (2^20 per listed channel) rather than bytes.

Where it cannot be the same as a merged session:

  • Recordings that take turns keep their own segments: merge_sessions splits a segment around the other recording's data, a MultiReader does not, so segments() and n_segments can be smaller.
  • Memory: each recording a read touches is read from the window's start (that keeps the grid exact), so a recording late in a long window also holds int32 counts and a validity byte for the part of the window before it: about 5 bytes per sample for each recording the window touches, plus 5 for the combined counts, beside the float64 result.

Ordering and naming many sessions

mef3io.time_range(path) gives a session's first and last sample time reading only the first and last segment's metadata of each channel, never an index or data file. That makes it fast enough to sort hundreds of recordings, even on a network drive:

paths = sorted(Path("incoming").glob("*.mefd"))
ranges = {p: mef3io.time_range(p) for p in paths}
by_time = sorted(paths, key=lambda p: ranges[p].start_uutc)

To read them as one recording, see Several recordings read as one.

offset_known is False when the times cannot be shown as absolute (an encrypted session opened without the password that unlocks its time offset, or an ambiguous legacy offset).

mef3io.name_drift(path) lists where a session's name is stored: its folder, its session-level annotation file and every file's header. It also says whether they agree. A folder renamed with the file manager leaves them disagreeing, and other MEF software then silently misses the session's annotations. mef3io.rename_archive(tar, new_name) renames a .mefd.tar file without ever replacing another. The archive's inside keeps the old name.

Renaming a session

Do not rename a .mefd folder with the file manager. A session's name is stored in four places, and other MEF software reads all of them: 1. the folder, <name>.mefd; 2. the session's annotation files, <name>.rdat / <name>.ridx; 3. the name in every file's header; 4. each channel's session description, where it is just the name.

mef3io.rename_session changes all four together:

rep = mef3io.rename_session("incoming/3f2a9c.mefd", "pt1")
print(rep.new_path)          # incoming/pt1.mefd
rep.undo()                   # back to 3f2a9c, exactly

What it guarantees: - Only the name changes. No sample, block index or time is touched, and a description that merely mentions the name is kept. Each changed file is stamped rename in its provenance region. - A crash cannot leave it half-done. - Before the first change, it writes the old headers and a journal of the planned steps to the backup folder, then a marker into the session. - If the operation is interrupted, the next rename_session or fix_name_drift call on the session (with the same backup_dir) rolls it back first. - While the marker is there, mef3io refuses to write to the session, copies refuse it as a source, and the Reader warns with SessionOperationWarning. - Nothing is ever replaced. Every rename uses the operating system's no-replace rename. A file system without one refuses rather than risk it. - Refused up front, with nothing changed: - a name that is not a portable file name or is over 250 bytes (never truncated); - a target that exists, or a session being written (a .partial copy beside it, a writer's temporary file inside it); - a damaged header (validate and repair first); - an encrypted session without the password (the descriptions are encrypted); - a name already out of step (fix that first, below).

Do not rename a session that is being recorded into. A rename refuses a session with a writer's temporary file in it, and an open Writer refuses to write once the folder has moved. Nothing stops a recording that appends while the rename runs, though, so stop the acquisition (or rename between recordings).

A rollback puts the old names back. If nothing else wrote to a file since, its old bytes are restored exactly; if something did (a program that does not know the marker appended data), only the name and the description are restored, so that write is kept.

backup_dir chooses where the journal and old headers live while the operation runs. They are deleted when it completes. The default is <session>.repair-backup beside the session. The old name is identifying, so an application should pass a folder of its own.

Undo is a logical inverse, never a byte restore: rep.undo() renames back and reverts only the descriptions the rename changed. Data appended since is kept. rep.files gives each changed file's size and modification time right after the operation, so an application can tell whether the session was written to since.

A folder renamed by another tool leaves name_drift(path).consistent False. mef3io.fix_name_drift(path) sets the stored names and the annotation files to the folder's name, with the same safety. It never renames the folder. It refuses when the old name is ambiguous (the files store different names, or there are several session-level annotation files).

A marker whose journal is lost (the backup folder was deleted, or another machine started the operation): every write refuses, and the message says why. mef3io.clear_operation_marker(path) removes the marker, but only when name_drift shows the name consistent. A half-changed session keeps its marker.

A .mefd.tar archive is renamed with rename_archive. To change the name inside it, extract, rename and archive again.

De-identified copies

mef3io.anonymize_session(src, dst, shift_us=None) writes a copy of a session that can be shared: the subject's identity, the session name, descriptions and (by default) annotation text removed, and the samples copied byte for byte. Without shift_us the copy keeps every date and time exactly, so it still shows when the recording was made; with shift_us every time is moved by it. The original is never changed. mef3io.anonymize.check runs the same checks first without writing anything, and every copy is verified against its source before it gets its final name.

start = mef3io.inspect("patient.mefd").start_time
shift = mef3io.anonymize.reference_shift(start, "2000-01-01")   # copy starts on that date
# (leave the shift out to keep the real dates and times)

rep = mef3io.anonymize.check("patient.mefd", "S001.mefd", shift, exclude_channels=["ECG"])
if not rep.ok:
    print(rep.codes)            # stable refusal codes, e.g. ('name-contains-identity',)
else:
    report = mef3io.anonymize_session("patient.mefd", "S001.mefd", shift,
                                      exclude_channels=["ECG"])
    print(report.verification.summary())

Everything else (choosing the shift, what changes, encryption of the copy, the refusal codes, how the copy is checked, what it does not do) is in De-identification; a runnable example is examples/13_anonymize.py. From a terminal: python -m mef3io anonymize SRC DST --reference-date 2000-01-01. Python and C++ only, not MATLAB.

Modified copies

mef3io.transform_session(src, dst, ...) writes a copy with only the changes you ask for: - a choice of channels; - new passwords, or none; - the session name; - edited details: subject names and ID, recording location, the session, channel and reference descriptions.

Every time and every sample is copied as stored, and the original is never changed. It is the same copy engine as anonymize_session, with the same check / verify pair and refusal codes.

rep = mef3io.transform_session(
    "patient.mefd", "patient_eeg.mefd",
    exclude_channels=["ECG"],
    output_password=("level-1 pw", "level-2 pw"),     # or None: no passwords
    metadata={"subject_id": "S-017", "channel_descriptions": {"Fp1": "frontal pole"}},
)
print(rep.summary())

time_range=(start_us, end_us) keeps only that window (a cut: edge blocks trimmed exactly, segments renumbered); it works the same way with anonymize_session. mef3io.merge_sessions([a, b], dst) merges recordings into one (see Merging recordings). The details are in Modified copies. From a terminal: python -m mef3io transform SRC DST --exclude-channel ECG --set subject_id=S-017. Python and C++ only, not MATLAB.

Tar archives (single-file sessions)

A session directory can be packed into a single uncompressed tar file — one file to share or archive, immune to piecemeal corruption of the tree:

tar = mef3io.archive_session("session.mefd")   # -> "session.mefd.tar"
r = mef3io.Reader(tar)                          # read in place, no extraction

Reader (and the compat MefReader) accept the archive path anywhere a .mefd directory is accepted; because the archive is uncompressed, windowed reads still fetch only the byte ranges they need. Everything else carries over — encryption, records, the warm-start cache. The source directory is left untouched, output is deterministic (same session ⇒ identical bytes), and any tar tool restores the directory (tar -xf session.mefd.tar). Foreign archives work too (GNU/bsdtar/tarfile, with or without directory entries); compressed input (.tar.gz etc.) is rejected with a clear error.

A mef3io addition, not part of MEF 3.0

meflib, pymef and mef_tools cannot open a .mefd.tar directly. It is a plain uncompressed tar file, so it can always be unpacked: extract_session, tar -xf session.mefd.tar (built into Windows 10 and later, macOS and Linux) or the free 7-Zip on Windows. The unpacked .mefd folder works with the legacy tools.

Tar sessions are read-only: Writer refuses .tar paths. To modify one, unpack it first — extract_session is the exact inverse (archive → extract → archive reproduces identical bytes), and the restored directory is a normal writable session again:

session = mef3io.extract_session(tar)          # -> "session.mefd" back
with mef3io.Writer(session) as w:              # appends work again
    w.write("ch1", more, t, fs=256.0)

Pass tar_path= / dest_dir= to choose targets and overwrite=True to replace an existing one.

Session naming is enforced at every entry point (reading, writing, archiving, extracting, in all language bindings): directories must end .mefd, archives .mefd.tar — anything else is refused with a clear error. This is a safety contract: a stray directory or file can never be misread as a session, and a session can never be created or overwritten under an unmarked name.

Cancelling and progress

Reads and the long operations take two optional keyword arguments:

cancel= progress=
Reader.read, Reader.read_raw yes compressed bytes decoded
archive_session, extract_session yes bytes copied
inspect yes (per segment and record file) bytes of .tmet (+ .tidx with read_index=True) read
Validator.validate, validate_session yes bytes of .tmet + .tidx + .tdat covered
repair_session yes (validation pass only) as validate
recover_session yes (scan only) bytes of .tidx + .tdat covered
anonymize_session yes bytes, over the copy and its verification
anonymize.check yes —
anonymize.verify yes bytes, as for anonymize_session
transform_session yes bytes, over the copy and its verification
transform.check yes —
transform.verify yes bytes, as for transform_session
token = mef3io.CancelToken()

def show(done, total):          # both in bytes
    bar.set(done / total if total else 1.0)

# on a worker thread:
try:
    x = reader.read("ch1", t0, t1, cancel=token, progress=show)
except mef3io.Cancelled:
    pass                        # superseded by a newer request

# from any other thread (a UI's Stop button, a newer pan/zoom):
token.cancel()

Several reads at once. Reads on one Reader can run from several threads at once (see Parallelism). When a newer request supersedes a read, cancel the old one with its own token; the cancelled call returns promptly and the other reads are unaffected.

Cancelling. token.cancel() is safe from any thread. The operation notices between units of work — before each segment is fetched and each block is decoded on a read, between 1 MB chunks when archiving or extracting, per segment (and per block examined) when validating or recovering — and raises mef3io.Cancelled. Tokens are one-shot: once cancelled they stay cancelled, so make a new one per request; one token can be passed to several calls to stop them together (token.raise_if_cancelled() lets your own loop stop at the same points). Cancelled subclasses Exception directly, not MefError: stopping on request is not a failure.

Nothing is left behind. A cancelled call returns nothing:

  • a read returns no partial array;
  • archive_session builds the archive in a temporary file and extract_session in a temporary sibling directory, renamed into place only when complete — a cancel leaves neither, and with overwrite=True the existing target is untouched;
  • repair_session and recover_session decide everything during a read-only pass and write only after it. A cancel during that pass (which is where the time goes) leaves the session byte-identical and creates no backup directory. The short write phase that follows — a .tmet replaced by rename and 1024-byte header patches — is not interruptible: a cancel that arrives then is ignored and the call returns its report, so a session is never left half-repaired;
  • anonymize_session and transform_session build the copy in <dst>.partial/ and rename it to dst only after it has been verified; a cancel removes the partial copy.

Progress. progress(done, total) is called first with (0, total), then at a bounded rate (about every 1 % of total, otherwise at most every 250 ms), with done never decreasing, and finally with (total, total) exactly once, when the cancellable work completes (only the short commit step after it — a rename, the header patches of a repair — can still fail, with an I/O error, never with Cancelled). For repair_session and recover_session that final call comes at the end of the read-only pass, before anything is written, so a callback that raises there still leaves the session untouched. Calls never overlap. The work runs with the GIL released; the callback takes the GIL only for its own call and may run on a decode worker thread rather than the calling one, so keep it short and hand UI updates to your UI thread. If it raises, the operation stops as if cancelled and your exception propagates. The callback must not call back into the same operation.

On a read, progress counts decoding. Each segment's bytes are fetched in one read first, so a whole-channel read of a huge segment on slow storage shows (0, total) until that fetch is done, and a cancel during it takes effect at the next segment or block. Windowed (viewer) reads fetch only their window. With total == 0 (an empty window) the single (0, 0) call is also the final one.

extract_session(..., overwrite=True) briefly needs room for both trees. A hard kill mid-extract can leave a <dest>.mef3io-extract-<hex> staging directory, and a replaced tree that could not be deleted is left as <dest>.mef3io-replaced-<hex>; both are safe to delete.

When neither keyword is given, nothing changes: the hooks cost one pointer test per unit of work (see benchmarks/cancel_overhead_benchmark.py).

MATLAB does not offer cancelling or progress (a MEX call blocks MATLAB's only thread); the C ABI does (mef3io_cancel_token_* and the *_ex functions, see the C++ and C ABI guide).

Session metadata (subject, acquisition)

Set subject and acquisition metadata with the Metadata object — grouped into a Subject block (MEF section 3) and an Acquisition block (section 2). Every field is defaulted, so fill in only what you have:

from mef3io import Metadata, Subject, Acquisition

md = Metadata(
    subject=Subject(id="MRN-123", name_1="Jane", recording_location="EMU-4"),
    acquisition=Acquisition(session_description="pre-surgical", line_frequency=60.0),
)
with mef3io.Writer("session.mefd", overwrite=True, metadata=md) as w:
    w.write("ch1", data, start_uutc, fs=256.0)
# or later, before writing:  w.set_metadata(md)   (also accepts a flat dict)

Read it back via Reader.metadata:

r = mef3io.Reader("session.mefd", password="l2pw")
r.metadata.subject.id                     # "MRN-123"
r.metadata.acquisition.session_description
r.metadata.to_dict()                      # nested plain dict

Subject fields (names, id, location) live in the level-2 encrypted section, so they read back empty with only a level-1 password; the acquisition descriptions and filter settings are level-1. Metadata is session-wide (written to every channel); channel_description defaults to the channel name. See the API reference for every field.

Legacy drop-in (mef_tools compatibility)

# from mef_tools.io import MefReader, MefWriter
from mef3io import MefReader, MefWriter

Same call shapes, defaults, and return shapes as mef_tools.io: write_data (NaN-gap splitting, precision inference, integer arrays take the int32 primitive path, in-segment appends), write_annotations, get_data / get_raw_data / get_property / get_channel_info / get_annotations, plus the legacy writer knobs (mef_block_len, get_mefblock_len, max_nans_written, record_offset). Differences from the legacy stack are measured and documented in the comparison. On a channel whose segments have different conversion factors the compat MefReader scales each segment by its own factor (the legacy mef_tools returns zeros there; see Merging). get_data with a list of channels reads them with the multi-channel read (one call per sampling rate and window; unset stamps keep meaning each channel's own start and end), so the arrays are the same as before, only faster: each one its own array, and on an error the exception of the first failing channel in the order asked for.

Metadata works both the legacy and the modern way on MefWriter: mutate the section3_dict / section2_ts_dict (e.g. w.section3_dict['subject_ID'] = 'Smith') as in mef_tools, or pass a mef3io.Metadata via metadata= / set_metadata.

Parallelism

Every Reader, MultiReader and Writer follows one rule: its n_threads covers everything it does, 0 = automatic, 1 = no extra threads, and it can be changed later. The full rule, what "automatic" means and how a MultiReader shares its budget are in Threads. Output is byte-identical regardless of thread count, and the bindings release the GIL, so external dataloader parallelism is safe.

Opening a session releases the GIL too: Reader(...) (and the writer's reopen of an existing session) scans the tree and parses every segment's metadata in C++, on the Reader's threads, while other Python threads keep running. So a GUI can open a large session on a worker thread without freezing its event loop, and several sessions can be opened at once on different threads. toc, segments, records, block_envelope, the multi-channel read and Writer.write_annotations release the GIL as well.

Sharing one Reader: reads, toc, segments, block_envelope and records on one Reader may run at once, also on a Reader opened with cache= (its backend is built once, by whichever call comes first), and n_threads may be changed meanwhile. close() waits for a backend still being built and then stays closed; a later call raises. A skipped record file is warned about once per Reader. A Writer is not meant to be shared between threads. The Python-level parts of an open (the warm-start cache, building the open-time warnings) are short and run with the GIL as usual.