Skip to content

Python API reference

Auto-generated from the source docstrings. For a task-oriented walkthrough see the Python guide.

The native API:

mef3io.Reader

Reader(path: str, password: str = '', backend: str = 'cpp', n_threads: int = 0, cache=None, warn_declarations: bool = True, strict: bool = True, *, _defer_warnings: bool = False)

Read-only interface to a MEF 3.0 session.

Times throughout are uUTC (microseconds since the Unix epoch). Windowed reads fetch only the bytes they need, so they stay cheap on huge sessions.

Parameters:

Name Type Description Default
path str

Path to the .mefd session directory, or to an uncompressed tar archive of one (name.mefd.tar, from :func:mef3io.archive_session) — tar sessions are read in place, without extraction. The suffix is enforced: anything else is refused, so a stray directory or file can never be misread as a session.

required
password str

Password for encrypted sessions. A level-2 password unlocks everything; a level-1 password reads the signal and technical metadata but leaves subject metadata locked. Empty for unencrypted sessions. MEF encryption does not cover the samples themselves: refusing an encrypted session without a password is mef3io's policy, not protection (see the encryption model in the docs).

''
backend ('cpp', 'pure')

Which implementation to use. Defaults to the C++ backend; the pure backend is not yet implemented.

"cpp"
n_threads int

How many threads this Reader may use for everything it does — opening, file access, decoding, decrypting. 0 (default) = automatic, 1 = no extra threads. Results never depend on it. Changeable later through :attr:n_threads. See the threads guide.

0
cache str or None

Opt-in warm-start cache for channel metadata. None (default) disables it; "auto" uses the per-user OS cache directory; a path makes it persistent. Warm opens serve :attr:channels / :meth:info without touching the session tree.

None
warn_declarations bool

Emit a :class:SessionDeclarationWarning when the session leaves section-2 size declarations unset. The check is free — the metadata is already parsed at open — and never affects the data read here; it flags a file that can mislead other MEF readers. Set False, or filter the warning category, to silence it.

True
Notes

The open itself (scanning the tree, parsing every segment's metadata) runs in C++ with the GIL released, as reads do, so opening a large session on a worker thread does not freeze other Python threads (a GUI's event loop, for instance).

Examples:

>>> with mef3io.Reader("session.mefd") as r:
...     x = r.read(r.channels[0], t0, t1)   # float64, NaN in gaps

problems property

problems: list

What a strict=False Reader skipped instead of raising.

A list of {"channel", "segment_number", "segment", "reason", "kind"} dicts, empty when everything loaded. Always empty in strict mode, where the same failure raises instead. kind is:

  • "segment" — a segment that could not be read; its time span reads back as NaN. Found when the session is opened.
  • "records" — a record (.rdat) file that could not be placed (nothing of it is read), or, when the file itself opened, ONE damaged record in it (CRC failure, unusable header, truncated tail; reason names the record and its byte offset). Every other record of that file is still returned, resynchronised through the .ridx offsets where one exists. Found by :meth:records (so the list can grow after the open); the samples are unaffected. segment is the file's path, and segment_number is -1 for a session- or channel-level file (channel is "" at session level). Nothing is repaired or written.

Segments that fail during a read (a damaged index or data extent) are added when the read meets them. A multi-channel :meth:read reads its channels concurrently, so the entries it adds for different channels can come in any order (one channel's stay in segment order).

Check it before trusting a read from a lenient Reader: a skipped segment's samples come back as NaN, exactly like a genuine gap, so the array alone cannot tell you data is missing.

declaration_issues property

declaration_issues: list

Section-2 size declarations this session leaves unset.

The structured form of the :class:SessionDeclarationWarning raised on open: a list of {"channel", "segment", "field"} dicts, empty when the session declares everything. Free — computed at open from metadata that was already parsed. It sees only what is missing; use :class:mef3io.Validator to find what is merely wrong.

n_threads property writable

n_threads: int

How many threads this Reader may use (0 = automatic).

Settable at any time: the new count applies from the next call (calls already running keep theirs). The open used the value given to the constructor. One count covers everything the Reader does — opening, file access, decoding, decrypting; see the threads guide. Results never depend on it.

channels property

channels: list[str]

Channel names in the session, sorted.

Returns:

Type Description
list of str

metadata property

metadata

Session subject/acquisition metadata as a :class:mef3io.Metadata.

Built from the first channel (metadata is session-wide). Subject fields are empty unless the reader was opened with a level-2 password.

Returns:

Type Description
Metadata

close

close() -> None

Release the backend and its file handles.

After this the reader is CLOSED: a further read raises rather than quietly reopening the session. A caller who closed to release handles — before archiving the directory, or to stay under an fd limit — got them back without notice, which is the opposite of what they asked for. Closing twice is harmless.

info

info(channel: str) -> dict

Channel metadata.

Parameters:

Name Type Description Default
channel str

Channel name.

required

Returns:

Type Description
dict

Keys: sampling_frequency (Hz), units_conversion_factor (the factor every segment stores, None when the channel's segments store different ones: :meth:segments lists each segment's, and the reads scale each sample by its own), units_description, start_time / end_time (uUTC), number_of_samples (stored samples — NaN gaps are not counted, so a gridded :meth:read usually returns more), recording_time_offset, n_segments, section3_available, and the section-3 subject fields (subject_name_1 / subject_name_2 / subject_id / recording_location; None without level-2 access).

read

read(channel, t0: Optional[int] = None, t1: Optional[int] = None, *, dtype='float64', cancel: Optional['CancelToken'] = None, progress: Optional[Callable[[int, int], object]] = None) -> np.ndarray

Read samples in units on the uniform sampling grid.

Parameters:

Name Type Description Default
channel str or list of str

A channel name, or a list (or tuple) of names for a MULTI-CHANNEL read: one call returns a 2-D array with one row per name, in the order given (duplicates allowed). Every row equals :meth:read of that channel alone over the same window, value for value and NaN for NaN. The channels must share ONE sampling rate (they share one grid): mixed rates raise ValueError; read each rate in its own call. All channels' file reads run concurrently on this Reader's threads and one decode pool works through every block of every channel, which is much faster than a loop of single reads (see the benchmarks page).

required
t0 int or float

Half-open time window [t0, t1) in uUTC. Defaults span the whole channel (for a list: from the earliest start to the latest end of the listed channels). The number of samples returned is round((t1 - t0) * fs / 1e6).

None
t1 int or float

Half-open time window [t0, t1) in uUTC. Defaults span the whole channel (for a list: from the earliest start to the latest end of the listed channels). The number of samples returned is round((t1 - t0) * fs / 1e6).

None
dtype ('float64', 'float32')

The output type (any spelling numpy.dtype accepts). float32 is the float64 value rounded to the nearest float32, exactly read(...).astype(numpy.float32), in half the memory: about 7 significant digits (relative error at most 2-24), so counts up to 224 in magnitude stay exact with factor 1. Gaps stay NaN.

"float64"
cancel CancelToken

Stop the read from another thread: it is checked before each segment is fetched and before each block is decoded, and a cancelled read raises :class:Cancelled and returns nothing. For a list it stops the whole call.

None
progress callable

progress(done, total) — compressed bytes of the blocks being decoded (for a list: of every channel, one sequence for the whole call). Called at a bounded rate, possibly from a decode worker thread (with the GIL held for the call only), never concurrently. If it raises, the read stops and the exception propagates. Reads on one Reader may run from several threads at once (also with cache=), and cancelling one does not affect the others (see the Python guide, "Parallelism").

None

Returns:

Type Description
ndarray

For a name: a 1-D array on the sampling grid; for a list: a 2-D array of shape (len(channel), n). Discontinuity gaps are filled with NaN; each value is scaled by the units-conversion factor of the segment it comes from (segments of one channel may store different ones). A segment whose factor is not set (0, MEF's "no entry") gives its raw stored counts (factor 1), with one :class:UnscaledSamplesWarning per Reader and channel.

Raises:

Type Description
ValueError

An empty list, or channels of different sampling rates.

IndexError

An unknown channel (for a list: checked before anything is read).

FormatError

A sample in the window comes from a segment whose factor is invalid: NaN or infinite. Nothing is decoded; :meth:read_raw still returns the counts.

See Also

read_raw : the unscaled int32 form with an explicit validity mask (one channel per call).

read_raw

read_raw(channel: str, t0: Optional[int] = None, t1: Optional[int] = None, *, cancel: Optional['CancelToken'] = None, progress: Optional[Callable[[int, int], object]] = None) -> dict

Read the stored int32 counts with an explicit validity mask.

Parameters:

Name Type Description Default
channel str

Channel name.

required
t0 int or float

Half-open [t0, t1) window in uUTC; defaults span the channel.

None
t1 int or float

Half-open [t0, t1) window in uUTC; defaults span the channel.

None
cancel optional

As in :meth:read.

None
progress optional

As in :meth:read.

None

Returns:

Type Description
dict

Keys: samples (int32 ndarray, on the grid), valid (uint8 ndarray; 0 marks gap samples with no data), start_uutc, sampling_frequency, units_conversion_factor, units_conversion_factors. Physical units are samples * factor where valid, with the factor of the sample's own segment: units_conversion_factors lists the spans (dicts with start_index, end_index — half-open sample indices — and units_conversion_factor), one per run of valid samples sharing a factor, sorted, each from its first valid sample to just past its last; one span for an ordinary window, none when nothing in it is valid. Factors are as stored: 0 means "not set" (:meth:read then gives the counts unscaled). units_conversion_factor is that span's factor, None when the window holds samples of segments with different factors, and :meth:info's value when nothing is valid. block_copy_mismatches (list of dict) lists any block whose .tidx entry disagreed with its RED block header, each with segment, block_index, index_start_uutc, header_start_uutc, index_number_of_samples and header_number_of_samples. Empty for a well-formed session; non-empty raises :class:BlockCopyWarning as well, and means the FILE is inconsistent — the header copy was used, which is what meflib and pymef do, so the read still matches theirs.

segments

segments(channel: str) -> list[dict]

Per-segment map of a channel — what data is where.

Read from metadata only (nothing is decoded), so it is cheap even for huge, gap-riddled sessions. Use it to locate data across large recording gaps, then :meth:toc for the block-level view within a segment.

Parameters:

Name Type Description Default
channel str

Channel name.

required

Returns:

Type Description
list of dict

One dict per segment (sorted by segment number) with keys segment, start_time / end_time (uUTC), start_sample (channel-wide index of the first sample), number_of_samples, number_of_blocks, units_conversion_factor (the segment's own, as stored: 0.0 is MEF's "no entry"), the on-disk path, and provenance: None for a file from another writer or from mef3io before 1.2, otherwise a dict with created_by / last_modified_by (version strings, "" if unknown), last_operation, operations (every operation ever applied: "create", "append", "header-repair", "recovery", ...), modification_count and layout_version. Versions and operations only — no times are recorded.

block_envelope

block_envelope(channel, t0: Optional[int] = None, t1: Optional[int] = None) -> dict

Minimum and maximum of every data block, from the block index alone.

A fast envelope for viewers: only each segment's block index is read (about 2% of the data), no signal is decoded, and segments outside the window are skipped. A whole day of a channel recorded in 10-second blocks is 8,640 rows.

Parameters:

Name Type Description Default
channel str or list of str

Channel name, or a list (or tuple) of names: then the result is a dict keyed by channel name (in the order given, duplicates once), each value the dict one name returns. The indices of all listed channels are read concurrently on this Reader's threads; still index only. Any mix of sampling rates is fine here.

required
t0 int

Window in uUTC, [t0, t1); a block is returned when it overlaps the window. None (default) means the start or end of the channel.

None
t1 int

Window in uUTC, [t0, t1); a block is returned when it overlaps the window. None (default) means the start or end of the channel.

None

Returns:

Type Description
dict of numpy.ndarray

One row per block, in time order: start_uutc (int64, the first sample), end_uutc (int64, just past the last sample: start + samples / fs), min and max (float64, in the channel's units, scaled like :meth:read: each block by its own segment's conversion factor). min/max are NaN where the writer left a block's extremes unset. Gaps in the recording are simply absent rows: compare end_uutc with the next start_uutc.

Notes

The resolution is the block length: mef3io writes 10-second blocks by default (1-second blocks at 5 kHz and above), Writer(block_length=) sets it, and other writers choose their own. Block times come from the index; :meth:read places samples by each block's own header, which agrees on every consistent file (see :meth:read_raw for blocks whose two copies disagree).

Raises:

Type Description
FormatError

A block that :meth:read of the same window would take samples from (decided by read's own window and grid rule) comes from a segment whose conversion factor is NaN or infinite, as :meth:read refuses it; :meth:toc returns the counts. A block of such a segment that the read would not touch (one overlapping the window by less than half a sample) gets NaN min/max.

toc

toc(channel: str) -> list[dict]

Block-level table of contents, for seeking and viewers.

Parameters:

Name Type Description Default
channel str

Channel name.

required

Returns:

Type Description
list of dict

One dict per RED block with start_uutc, start_sample, number_of_samples, maximum_sample_value, minimum_sample_value, and discontinuity (True when the block does not continue seamlessly from the previous one).

records

records(channel: Optional[str] = None, segment: Optional[int] = None, level: Optional[str] = None) -> list[dict]

Read records (annotations) at the session, channel or segment level.

MEF 3.0 keeps record files at three levels: the session (<session>.mefd), each channel (<ch>.timd) and each segment (<ch>-NNNNNN.segd). Every returned record says where it came from.

Parameters:

Name Type Description Default
channel str or None

None (default) for the session scope; a channel name for that channel's records.

None
segment int or None

A segment number (requires channel): only that segment's records.

None
level (None, 'session', 'channel', 'segment', 'all')

Which levels to return. None (default) keeps each scope's natural meaning: session-level records when channel is None; the channel's channel-level AND segment-level records when it is given; the segment's when segment is given. "all" without a channel returns every record in the session, including channels that have records but no data yet (not in :attr:channels, which lists channels with samples). A level outside the scope ("session" with a channel) raises ValueError.

None

Returns:

Type Description
list of dict

One dict per record, grouped session -> channel -> its segments. Every dict has type (e.g. "Note"), time (absolute uUTC), level ("session"/"channel"/"segment"), channel (None at session level), segment (None unless segment level), version_major and version_minor, then the body fields of its type, always the same keys for a type (None where the body does not carry one) — e.g. text for Note/SyLg, duration + text for EDFA, earliest_onset ... channels for Seiz. Body times are absolute uUTC like time. The full per-type list is in the MEF format reference.

Raises:

Type Description
FormatError

A record file cannot be read, or the segments a channel or the session spans disagree on the recording-time offset AND its file stores offset-dependent (negated) times, so they cannot be placed. Only when strict: a lenient Reader skips that file, lists it in :attr:problems and emits :class:UnreadableSegmentWarning; the other files (e.g. the segment-level ones) are still returned.

ValueError

A segment without a channel, or a level outside the scope.

IndexError

Unknown channel or segment number.

mef3io.MultiReader

MultiReader(paths, password: str = '', backend: str = 'cpp', n_threads: int = 0, cache=None, warn_declarations: bool = True, strict: bool = True)

Several MEF 3.0 recordings read as one, without merging them on disk.

A drop-in for :class:mef3io.Reader: the same methods and properties, with the same arguments, return types, dict keys and dtypes. The only differences are the first argument (a sequence of sessions) and that n_threads is the total for the whole MultiReader, shared between the recordings it opens or reads at once (never multiplied by them).

What a read returns is exactly what :class:Reader returns for the same window of the same recordings merged with :func:mef3io.merge_sessions: the same length, the same values and NaN in the same places. Samples are placed on the same sampling grid by the same rule (each recording is read on the window's own grid, so nothing is re-rounded), and where two recordings touch within half a sample, the block that starts later wins, as it does in a merged session.

Parameters:

Name Type Description Default
paths sequence of str or os.PathLike

The sessions, each anything :class:Reader opens (a .mefd folder or a .mefd.tar archive), in any order: they are ordered by their first sample. A single path (not in a sequence) is a TypeError; an empty sequence or the same session twice is a ValueError.

required
password str

The password of every recording.

''
backend optional

Passed to every :class:Reader.

'cpp'
warn_declarations optional

Passed to every :class:Reader.

'cpp'
strict optional

Passed to every :class:Reader.

'cpp'
n_threads int

How many threads this MultiReader may use IN TOTAL, for everything it does (0, the default, = automatic; 1 = no extra threads). It is split, never multiplied: the open shares it between the recordings it opens at once, and a read that touches several recordings reads them at once, each with its share; a read inside one recording gets all of it. Changeable later through :attr:n_threads (re-split from the next call). Results do not depend on it. See the threads guide.

0
cache (None, 'auto' or sequence)

None (default) or "auto" as for :class:Reader, applied to every recording; or one cache path per recording, in the order of paths. One path for all is a ValueError (the recordings would overwrite each other's snapshot).

None

Raises:

Type Description
ValueError

A channel has different sampling rates in different recordings, or two recordings hold samples of the same channel at the same time (compared on the recorded blocks; overlaps up to half a sample are recorder rounding and allowed, the rule of :func:mef3io.merge_sessions). Recordings that take turns (A, then B, then A again) are fine.

Notes

How values are combined:

  • :attr:channels: every channel of any recording, sorted. A channel a recording does not have reads as a gap over that recording's time.
  • :meth:info: start_time the earliest, end_time the latest, number_of_samples and n_segments the sums, section3_available only if every recording has it. The patient fields (subject_*, recording_location, gmt_offset) and session_description are combined over EVERY recording, also those without the channel, so :meth:info and :attr:metadata always agree. Every other field is the value the recordings holding the channel agree on. A field they disagree on is None (a :class:MultiReaderDifferenceWarning names such fields once, on open; not units_conversion_factor, which every read handles per segment). A session description equal to its own recording's name (what mef3io and the legacy stack write by default) counts as unset: day files named by date do not "disagree" on it, and when every recording has only the default it is None (one recording keeps its own). :func:merge_sessions keeps the FIRST source's description, so for this one field metadata is not what a merged session gives.
  • :attr:metadata: a :class:mef3io.Metadata with those same combined patient fields and session description, the rest from :meth:info of the first channel (as :class:Reader does). Each recording's own values: :attr:metadata_by_source.
  • :meth:segments: every recording's segments in time order, numbered from 0 like the segments of a merged session; path is the segment's real folder. start_sample counts the samples stored before it. Unlike a merged session, a segment is never split around another recording's data, so recordings that take turns can give fewer segments than :func:merge_sessions would.
  • :meth:toc and :meth:block_envelope: every block in time order; start_sample as above.
  • :meth:records: session- and channel-level records merged in time order, then each segment's own; segment uses the numbering of :meth:segments (records(channel, segment=k) too).
  • :attr:problems, :attr:declaration_issues: every recording's, in time order; a declaration issue's segment uses the numbering of :meth:segments, a problem keeps the number of the recording's own folder (an unreadable segment is in no segment list) and its path.
  • Warnings: each recording's own (declarations, unreadable segments, ...) are emitted once, on open, with w.message.path naming that recording, plus one :class:MultiReaderDifferenceWarning.

  • Recordings (or segments) that store different units-conversion factors for one channel: as in a :class:Reader, :meth:read scales each sample by its own segment's factor, :meth:read_raw lists where each factor applies (units_conversion_factors), and info reports None.

Where it cannot equal a merged session:

  • A :meth:read reads each recording it touches from the window's start, so the window's own grid applies with no re-rounding: a recording late in a long window allocates int32 counts and a validity byte for the part of the window before it as well: about 5 bytes per sample per recording touched, plus 5 for the combined counts, beside the float64 result.
  • progress(done, total): total can grow while the recordings start reading (each part knows its size only when it starts); done never goes back.

Calls on one MultiReader may overlap across threads, as on a :class:Reader (with the same limits; see the Python guide).

Examples:

>>> days = sorted(Path("/data/patient7").glob("*.mefd"))
>>> with mef3io.MultiReader(days) as r:
...     x = r.read("Fp1", t0, t0 + 3_600_000_000)   # across midnight

paths property

paths: tuple

The recordings, in time order (as given, as str). Not on :class:Reader.

metadata_by_source property

metadata_by_source: dict

Each recording's own :attr:Reader.metadata, keyed by its path as given, in time order. Not on :class:Reader.

problems property

problems: list

Every recording's :attr:Reader.problems, in time order.

Same dicts as :attr:Reader.problems; segment holds the path of the file, so it says which recording. segment_number is the number of that recording's own segment folder (an unreadable segment is not in :meth:segments).

declaration_issues property

declaration_issues: list

Every recording's :attr:Reader.declaration_issues, in time order.

segment is the number in :meth:segments; the warning emitted on open (one per recording) carries each recording's own numbers.

channels property

channels: list[str]

Channel names of every recording, sorted.

Returns:

Type Description
list of str

metadata property

metadata

Session metadata as a :class:mef3io.Metadata, combined over every recording.

The subject fields and the session description: the value every recording holds, or None where they differ (a description equal to its own recording's name is the default and does not count; see the class notes). The other acquisition fields come from :meth:info of the first channel, as on :class:Reader (so None where the recordings holding it differ). Subject fields are empty unless every recording was opened with level-2 access.

n_threads property writable

n_threads: int

How many threads this MultiReader may use in total (0 = automatic).

Settable at any time: re-split between the recordings from the next call (calls already running keep theirs). See the threads guide.

close

close() -> None

Close every recording. As :meth:Reader.close: a further read raises.

info

info(channel: str) -> dict

Channel metadata, combined over the recordings that hold the channel.

Same keys as :meth:Reader.info. start_time is the earliest, end_time the latest, number_of_samples and n_segments the sums, section3_available true only if every recording has it. The patient fields and session_description are combined over every recording (as :attr:metadata); any other field is the value the recordings holding the channel agree on. A field they disagree on is None. See the class notes for the session-description rule.

read

read(channel, t0: Optional[int] = None, t1: Optional[int] = None, *, dtype='float64', cancel: Optional['CancelToken'] = None, progress: Optional[Callable[[int, int], object]] = None) -> np.ndarray

Read samples in units on the uniform sampling grid, across recordings.

As :meth:Reader.read, and equal to it on the same recordings merged with :func:mef3io.merge_sessions. Time a recording does not cover (or a channel it does not have) is NaN. Each recording's samples are scaled by its own units-conversion factor. The read shares :attr:n_threads between the recordings it touches; cancel stops every recording's read; progress reports the sum over them.

channel may be a list of names, with dtype= and the rules of :meth:Reader.read (one rate, a 2-D array, the same errors and the same defaults). Here it reads the channels one after the other, each with the whole budget, so it returns what a Reader of the merged session returns but without the Reader's concurrent multi-channel speed-up. progress is one sequence with one constant total for the whole call, counted in units of channels (total = 2**20 per listed channel, each channel's share advancing with its own decoded bytes), not in bytes as for one channel.

read_raw

read_raw(channel: str, t0: Optional[int] = None, t1: Optional[int] = None, *, cancel: Optional['CancelToken'] = None, progress: Optional[Callable[[int, int], object]] = None) -> dict

Read the stored int32 counts with a validity mask, across recordings.

As :meth:Reader.read_raw (same keys and dtypes); the counts are placed as :meth:read places them. block_copy_mismatches holds every recording's, in time order (each segment is a path, so it says which recording).

units_conversion_factors says which factor applies where, over the combined window, exactly as a :class:Reader of the merged session gives it.

segments

segments(channel: str) -> list[dict]

Every recording's segments of a channel, in time order.

As :meth:Reader.segments (same keys). segment numbers them from 0 across the recordings, as a merged session does; path is the segment's own folder (so it says which recording); start_sample counts the samples stored before it.

block_envelope

block_envelope(channel, t0: Optional[int] = None, t1: Optional[int] = None) -> dict

Minimum and maximum of every data block, across recordings.

As :meth:Reader.block_envelope (same keys and dtypes): one row per block, every recording's in time order. A list of names gives a dict keyed by channel, as a Reader does.

toc

toc(channel: str) -> list[dict]

Block-level table of contents, across recordings.

As :meth:Reader.toc (same keys): every block in time order; start_sample counts the samples stored before it.

records

records(channel: Optional[str] = None, segment: Optional[int] = None, level: Optional[str] = None) -> list[dict]

Records (annotations) of every recording, as :meth:Reader.records.

Same arguments and dicts. Session- and channel-level records are merged in time order; segment-level ones follow their segment, whose segment is the number in :meth:segments (also for the segment argument). The grouping is Reader's: session, then each channel's own, then its segments'.

mef3io.MultiReaderDifferenceWarning

MultiReaderDifferenceWarning(message: Any = '', *, path: Optional[str] = None, details: Iterable = ())

Bases: MefWarning

The recordings a :class:MultiReader combines disagree on a value.

Raised once when the MultiReader is opened. A value the recordings agree on reads as that value; a value they disagree on (a subject field, the units, a filter setting, a session description, ...) reads as None in :meth:MultiReader.info and :attr:MultiReader.metadata, never as one recording's value. Different patient names or IDs can mean the recordings are not the same patient's: check before analysing them together.

The message names the differing FIELDS only, never their values or a path. w.message.details is the tuple of field names; w.message.path is the first recording (in time order). Each recording's own values are in :attr:MultiReader.metadata_by_source, or open it with :class:Reader.

mef3io.Writer

Writer(path: str, overwrite: bool = False, password1: str = '', password2: str = '', units: Optional[str] = None, block_length: Optional[int] = None, n_threads: int = 0, metadata=None, wait_for_disk: bool = False, flush: str = 'each')

Write a MEF 3.0 session.

Times throughout are uUTC (microseconds since the Unix epoch). The first write to a channel creates segment 0; later writes append in-segment (extending the existing files) unless new_segment=True.

Parameters:

Name Type Description Default
path str

Path to the .mefd session directory to create or extend (the suffix is enforced). Tar archives are read-only sessions: a .tar path raises — write a directory and pack it with :func:mef3io.archive_session.

required
overwrite bool

True deletes any existing session first. False (default) reopens an existing session for appending — state is recovered from disk, so appends work across program runs.

False
password1 str

Level-1 and level-2 passwords. MEF has no "level-1 only" files, so to encrypt a session pass both; leave both empty for no encryption.

''
password2 str

Level-1 and level-2 passwords. MEF has no "level-1 only" files, so to encrypt a session pass both; leave both empty for no encryption.

''
units str or None

Physical units label stored in metadata (e.g. "uV").

None
block_length int or None

RED block size in samples. None (default) derives it from fs (fs samples for fs >= 5000, else 10*fs, at least 1). At most mef3io._mef3io.MAX_BLOCK_SAMPLES (858 993 396: a block's byte size is a 32-bit field); more raises ValueError before anything is created, as does a rate whose derived block exceeds it.

None
flush ('each', 'close')

"each" (default): the waits described in the data-on-disk guide. "close": no waits during writes, one combined wait at :meth:close/:meth:flush. Only for a session this Writer creates (overwrite=True or a new path; an existing session raises ValueError) and only for output you rebuild after any failure: a power cut while writing can leave the whole session unreadable and recover_session cannot be relied on. Not combinable with wait_for_disk.

"each"
n_threads int

How many threads this Writer may use (RED encoding): 0 (default) = automatic, 1 = no extra threads. Output is byte-identical regardless of thread count. Changeable later through :attr:n_threads. See the threads guide.

0
metadata Metadata or dict

Session-wide subject/acquisition metadata written to every channel. Also settable later via :meth:set_metadata (before writing).

None
wait_for_disk bool

False (default): :meth:write returns once its data is handed to the operating system; :meth:flush and :meth:close wait for the disk. True: every :meth:write returns only when its own data is on disk (one combined wait per call). See When is my data on disk? <https://bnelair.github.io/mef3io/long_recordings/#when-is-my-data-on-disk>_.

False
Notes

When is my data on disk? By default :meth:write returns once its data is handed to the operating system. Only each segment's .tmet is flushed before it is published (a short .tmet would make the whole session fail to open). :meth:flush and :meth:close (also leaving a with block) return only when everything written so far is on disk, in one combined wait. With wait_for_disk=True every write returns only when its own data is on disk. A power cut before that can leave the last append of a segment torn; :func:mef3io.recover_session with apply=True restores a valid session holding every sample whose bytes reached the disk. Earlier appends are never at risk: nothing rewrites them. A write raises if a flush it waits for fails. The durability= argument of 1.x was removed in 2.0 (passing it raises TypeError).

Examples:

>>> with mef3io.Writer("session.mefd", overwrite=True, units="uV") as w:
...     w.write("ch1", data, start_uutc, fs=256.0)   # NaN marks gaps

n_threads property writable

n_threads: int

How many threads this Writer may use (0 = automatic).

Settable at any time; applies from the next write. See the threads guide.

unreadable_metadata property

unreadable_metadata: list[str]

Segment metadata this writer could not use, as "path: reason".

.tmet files that could not be read while choosing the recording-time offset, and empty segment directories ignored on open (what a crashed write can leave behind). In a session at offset 0 an unreadable file is skipped so it does not block unrelated writes (with an :class:UnreadableSegmentWarning); where the offset is non-zero, or nothing else is readable, the write is refused instead.

set_metadata

set_metadata(metadata) -> None

Set session-wide subject/acquisition metadata (a :class:mef3io.Metadata, or a flat dict of fields). Call before writing; applies to every channel.

flush

flush() -> None

Return only when everything this Writer has written so far is on disk: every file it created or extended since the last flush, in ONE combined wait (not one per channel). Raises OSError/RuntimeError when the system reports the flush failed, and keeps raising it on every later call (a retry would otherwise claim data that is gone). :meth:close does the same. See When is my data on disk? <https://bnelair.github.io/mef3io/long_recordings/#when-is-my-data-on-disk>_.

close

close() -> None

Flush (see :meth:flush), then release the writer. Called automatically on context-manager exit. Returns only when everything written is on disk.

write

write(channel: str, data: ndarray, start_uutc: int, fs: float, precision: 'int | str' = -1, new_segment: bool = False) -> dict

Write float data; NaN runs become discontinuity gaps.

Values are quantized to int32 counts as round(data * 10**precision) with the conversion factor 10**-precision kept in metadata. NaN samples are not stored — they read back as NaN.

precision="step" (opt-in) stores the data as whole counts of its own quantization step instead — e.g. an amplifier's 0.195 uV per count — which gives much smaller files for data that came from an ADC. The step is estimated from a bounded sample of the data (fast on any length) and every value is checked while it is converted: a value stored this way reads back equal to what was written (exactly, when the data was counts * step in float64; otherwise within float64 rounding, 64 eps relative). Data that is not on one step (filtered, resampled) falls back to the default path with a :class:StepFallbackWarning, as does data showing too few distinct values to establish a step (a DC level, a square wave). A channel keeps the conversion factor of its first segment: on a channel that already has data, "step" stores data on that factor exactly, and REFUSES data off it (WriteConflictError, nothing written) rather than round it. Appending with the default precision to a channel whose factor is not a power of ten (a step, or write_int32) is refused too: keep passing "step".

Parameters:

Name Type Description Default
channel str

Channel name (created on first write).

required
data array_like

Any numeric input; coerced to float64 (lists, int arrays, float32 all accepted).

required
start_uutc int or float

Timestamp of the first sample, uUTC.

required
fs float

Sampling frequency in Hz.

required
precision int or 'step'

Decimal precision for quantization. -1 (default) infers it from the data — or, when appending, reuses the segment's stored precision so the append cannot conflict. "step" detects the data's own quantization step (see above).

-1
new_segment bool

Force a fresh segment instead of appending in-segment.

False

Returns:

Type Description
dict

samples_written, blocks, gaps_skipped, segment, units_conversion_factor (the factor the stored counts carry), quantization ("decimal": rounded to 10**-precision; "step": stored exactly on the factor) and step_fallback (precision="step" only: None when the step was used, else why not), clamped and clamped_infinite (samples stored as the nearest storable value; see :class:ClampedSamplesWarning).

Raises:

Type Description
ValueError

If fs is not a finite number > 0, precision makes the conversion factor 10**-precision 0, or is a string other than "step" — checked before anything is written. (A value whose count would leave int32, or +-inf, is not an error: it is stored as the nearest storable value, counted in the summary's clamped and reported by :class:ClampedSamplesWarning.)

WriteConflictError

On an append conflict (fs / conversion-factor mismatch, data starting before the segment's end, a default-precision append to a factor that is no power of ten, or "step" data off the channel's factor). Nothing is written.

write_int32

write_int32(channel: str, data: ndarray, ufact: float, start_uutc: int, fs: float, valid: Optional[ndarray] = None, new_segment: bool = False) -> dict

Write integer counts verbatim with a conversion factor (bit-exact).

The primitive path: counts are stored exactly as given, with ufact (e.g. an amplifier's volts-per-bit) in metadata. Physical units on read are counts * ufact.

Parameters:

Name Type Description Default
channel str

Channel name (created on first write).

required
data array_like of int

Integer counts (any integer width). Stored bit-exact.

required
ufact float

Conversion factor from counts to physical units.

required
start_uutc int or float

Timestamp of the first sample, uUTC.

required
fs float

Sampling frequency in Hz.

required
valid array_like or None

Same length as data; any numeric/bool mask where nonzero marks a real sample and zero a discontinuity gap. None = all valid.

None
new_segment bool

Force a fresh segment instead of appending in-segment.

False

Returns:

Type Description
dict

samples_written, blocks, gaps_skipped, segment.

Raises:

Type Description
TypeError

If data is floating-point (use :meth:write for float data — counts are never silently rounded here).

ValueError

If any value is outside the int32 range (it would wrap), if fs or ufact is not a finite number > 0 (a ufact of 0 or NaN would make every sample read back as 0 or NaN), or if valid does not have one entry per sample — all checked before anything is written.

RuntimeError

On an append conflict.

write_int32_many

write_int32_many(channels, data: ndarray, ufact, start_uutc: int, fs: float, new_segment: bool = False) -> list

Write the same run of integer counts for many channels in one call.

Byte-identical to calling :meth:write_int32 for each channel in the given order (fresh segments differ only in their random UUIDs), but the channels are encoded and written in parallel on this Writer's n_threads (one budget for channels and blocks together), and the .tmet barriers of all channels are ONE combined wait instead of one per channel. See When is my data on disk? <https://bnelair.github.io/mef3io/long_recordings/#when-is-my-data-on-disk>_.

Every refusal, including an append that conflicts with what is on disk (rate, factor, start before the segment's end), is decided before any byte is written. A failure while writing or flushing rolls every channel of the call back and raises the underlying error (nothing was written); if a rollback itself does not complete, :class:RollbackIncompleteError names the channels (incomplete_channels) whose files may be torn. A failure while publishing raises :class:PartialWriteError, whose written_channels lists the channels that are on disk (the rest were rolled back; incomplete_channels as above).

Memory: besides the data, each channel being written holds a copy of its samples and its encoded blocks (freed when it is written), and only about n_threads channels run at once; write_many also holds the converted counts and mask of every channel (about 5/8 of the input), so write long recordings in chunks (minutes per call).

Parameters:

Name Type Description Default
channels sequence of str

Distinct channel names.

required
data array_like of int, shape (n_channels, n_samples)

Counts, stored bit-exact (no gap mask: split the run, or use :meth:write_many with NaN).

required
ufact float or sequence of float

One conversion factor, or one per channel.

required
start_uutc int or float

Timestamp of the first sample, uUTC.

required
fs float

Sampling frequency in Hz.

required
new_segment bool

Force a fresh segment for every channel instead of appending.

False

Returns:

Type Description
list of dict

One summary per channel, in order (as :meth:write_int32).

Raises:

Type Description
ValueError

Duplicate channels, a shape other than (len(channels), n), ufact neither a scalar nor one per channel, or the checks of :meth:write_int32.

TypeError

If data is floating-point.

PartialWriteError

Some channels were published before a failure; see above.

write_many

write_many(channels, data: ndarray, start_uutc: int, fs: float, precision: 'int | str' = -1, new_segment: bool = False) -> list

Write the same run of float data for many channels in one call.

The float counterpart of :meth:write_int32_many, with the semantics of :meth:write per channel: NaN runs become discontinuity gaps, and the precision (-1 = inferred) is decided per channel. Parallel across channels, one combined barrier, all-or-nothing rollback; see :meth:write_int32_many.

Parameters:

Name Type Description Default
channels sequence of str

Distinct channel names.

required
data (array_like, shape(n_channels, n_samples))

Coerced to float64.

required
start_uutc int

As :meth:write.

required
fs int

As :meth:write.

required
precision int

As :meth:write.

required
new_segment int

As :meth:write.

required

Returns:

Type Description
list of dict

One summary per channel, in order (as :meth:write). The warnings of :meth:write (step fallback, clamped samples) are emitted per channel.

write_annotations

write_annotations(annotations, channel: Optional[str] = None) -> None

Write records (annotations).

Always appends to the records already stored at the given level (the files are created the first time); it never replaces or deletes a stored record. The new records are added in place, so the cost does not grow with the number already stored. A record earlier than the stored ones is appended as well; :meth:mef3io.Reader.records returns them sorted by time. In encrypted sessions record bodies are level-2 encrypted. A file left torn by a power cut is refused until :func:mef3io.recover_session has repaired it.

Parameters:

Name Type Description Default
annotations iterable of dict or pandas.DataFrame

Each record needs time (uUTC); type defaults to "Note"; text and duration are optional. A DataFrame with those columns is accepted.

required
channel str or None

Channel name for channel-level records, or None (default) for session-level records.

None

mef3io.archive_session

archive_session(session_path, tar_path: Optional[str] = None, overwrite: bool = False, *, cancel: Optional[CancelToken] = None, progress: Optional[Callable[[int, int], object]] = None) -> str

Pack a session directory into a single uncompressed tar archive.

The archive (conventionally name.mefd.tar) is a plain ustar file: :class:Reader opens it directly — no extraction — and any tar tool (tar -xf) reproduces the original directory. Because it is uncompressed, windowed reads still fetch only the byte ranges they need. The source directory is left untouched; output is deterministic, so archiving the same session twice yields identical bytes. Tar sessions are read-only: :class:Writer refuses .tar paths.

Parameters:

Name Type Description Default
session_path str or path - like

The .mefd session directory to pack (the suffix is enforced).

required
tar_path str

Target archive path; must end .mefd.tar. Default derives <session_path>.tar (name.mefd becomes name.mefd.tar).

None
overwrite bool

Replace an existing target archive. Default False (an existing target raises).

False
cancel CancelToken

Stop the archive from another thread; checked between 1 MB chunks. The archive is built in a temporary file and renamed into place only when complete, so a cancelled call raises :class:Cancelled and leaves no archive and no temporary file (an existing target is untouched).

None
progress callable

progress(done, total) in bytes of session files copied. Called at a bounded rate, never concurrently; (total, total) exactly once, when the copy completes. If it raises, the archive is abandoned like a cancel and the exception propagates.

None

Returns:

Type Description
str

Path of the created archive.

Examples:

>>> tar = mef3io.archive_session("session.mefd")
>>> with mef3io.Reader(tar) as r:
...     x = r.read(r.channels[0])

mef3io.extract_session

extract_session(tar_path, dest_dir: Optional[str] = None, overwrite: bool = False, *, cancel: Optional[CancelToken] = None, progress: Optional[Callable[[int, int], object]] = None) -> str

Unpack a session archive back into a .mefd directory.

The inverse of :func:archive_session — after extraction the directory is a normal writable session again. The session root inside the archive is stripped, so dest_dir becomes the session directory itself; archives from foreign tar tools work too. The session is unpacked into a temporary sibling directory and moved into place only when complete, so a failed or cancelled extraction never leaves a half-written directory behind — and with overwrite=True the existing directory survives until the new one is whole.

Parameters:

Name Type Description Default
tar_path str or path - like

The session archive to unpack; must end .mefd.tar.

required
dest_dir str

Target directory; must end .mefd. Default strips the .tar suffix (name.mefd.tar becomes name.mefd next to the archive).

None
overwrite bool

Replace an existing target directory. Default False (an existing target raises).

False
cancel CancelToken

Stop the extraction from another thread; checked between 1 MB chunks. Raises :class:Cancelled and leaves nothing behind.

None
progress callable

progress(done, total) in bytes of member data written; as in :func:archive_session.

None

Returns:

Type Description
str

Path of the extracted session directory.

Examples:

>>> session = mef3io.extract_session("session.mefd.tar")
>>> with mef3io.Writer(session) as w:      # writable again
...     w.write("ch1", more_data, t, fs=256.0)

mef3io.password_interop_issue

password_interop_issue(password: str) -> 'str | None'

Why a password will not behave as expected in other MEF readers, or None.

The same rule :class:Writer (and :func:mef3io.anonymize_session) use for :class:PasswordInteropWarning, so an application can warn before it writes. A password has an issue when it has:

  • more than 15 characters: meflib and pymef refuse it, so only mef3io opens the session;
  • more than 15 bytes in UTF-8: pymef (and so mef_tools) copies it into a 16-byte buffer;
  • a character above U+00FF: MEF keeps only the low byte of each character's code point, so a different password with the same low bytes also opens the session.

mef3io itself writes and reads such a password correctly.

Parameters:

Name Type Description Default
password str
required

Returns:

Type Description
str or None

The reason in plain words (it never repeats the password), or None when the password interoperates.

Examples:

>>> mef3io.password_interop_issue("secret")
>>> mef3io.password_interop_issue("a" * 20)
'it has 20 characters; meflib and pymef refuse passwords longer than 15 characters, ...'

Inspecting a session

See the Python guide.

mef3io.inspect

inspect(path, password: str = '', *, include_values: bool = False, read_index: bool = False, strict: bool = True, cancel: Optional[CancelToken] = None, progress: Optional[Callable[[int, int], object]] = None) -> SessionSummary

Describe a session without reading any sample data.

Reads each segment's metadata (.tmet, the same bytes a :class:mef3io.Reader reads when it opens), the record files, and — only with read_index=True — the block indices. It never reads a .tdat: those are measured by size only, so the cost does not grow with the amount of data recorded.

Available in Python and C++ (mef3io::inspect_session) only; there is no MATLAB or C ABI equivalent.

Parameters:

Name Type Description Default
path str or PathLike

A .mefd directory or a .mefd.tar archive (read in place).

required
password str

Opens encrypted sections. Without it an encrypted session is still described: what stays locked is reported (encryption, None values, a "locked" warning) rather than guessed. A password that is given and wrong raises :class:mef3io.PasswordError. The level-2 password opens everything; the level-1 password leaves section 3 (subject fields, the recording-time offset) and level-2 record bodies locked. A non-ASCII password that matches only through the mapping of mef3io 1.2.0 and earlier is accepted, as the Reader accepts it, and flagged in legacy_password_mapping rather than warned about.

''
include_values bool

Return identity VALUES: subject name/ID/location, descriptions, session_name/anonymized_name, record text and the full records, and the recording-time offset. Off by default, when identity fields report presence and length only (and ChannelSummary.channel_description is None). Names and paths are NOT de-identified either way: the session's folder name (usually the same as session_name), channel names and file paths are always returned, so a summary of an identifiable session is identifiable.

False
read_index bool

Also read every segment's block index (about 2% of the data) to find gaps inside segments. Off by default.

False
strict bool

As everywhere in mef3io, a segment or record file that cannot be read raises. False reports it in problems and an "unreadable-segments" / "unreadable-records" warning, marks the segment readable=False, and describes the rest — the mode for an explorer that must show a damaged session.

True
cancel CancelToken

Checked before each segment and each record file. Once set, the call raises :class:mef3io.Cancelled and returns nothing.

None
progress callable

progress(done, total) in bytes of the files the segment pass reads (every .tmet, plus every .tidx with read_index=True; record files are not counted), called at a bounded rate and never concurrently. Sizing the total costs one stat per file, paid only when a callback is given.

None

Returns:

Type Description
SessionSummary

Raises:

Type Description
IoError

path is not a .mefd directory or .mefd.tar archive.

PasswordError

A password was given and is wrong.

MefError

(strict only) A segment or record file cannot be read.

Cancelled

cancel was set.

Examples:

>>> s = mef3io.inspect("session.mefd")
>>> s.first_time, s.last_time            # samples AND annotations
>>> s.identity_field("subject_id").present
>>> [w.code for w in s.anonymize_blockers]

mef3io.SessionSummary dataclass

SessionSummary(name: str, path: str, layout: str, total_bytes: int, archive_bytes: Optional[int], index_read: bool, offset_assumed_zero: bool, legacy_password_mapping: bool, encryption: dict, time_offset: dict, gmt_offset: Optional[int], dst_start_time: Optional[int], dst_end_time: Optional[int], start_time: Optional[int], end_time: Optional[int], records_start_time: Optional[int], records_end_time: Optional[int], first_time: Optional[int], last_time: Optional[int], channels: list, record_only_channels: list, video_channels: list, incomplete_channels: list, records: list, identity: list, warnings: list, problems: list, declaration_issues: list)

Everything about a session that does not need its samples.

Returned by :func:mef3io.inspect. All times are absolute uUTC (UTC), or None where they cannot be known without guessing.

Attributes:

Name Type Description
name, path, layout

The session's name, the path given, and "directory" or "tar".

total_bytes, archive_bytes

Sizes of every MEF file (.tdat measured, never read); the archive file's size for a tar (else None).

encryption dict

encrypted; access — what the password opened for the worst segment: "full", "level-1" or "none"; the highest stored section2_level / section3_level (0 = plain); and how many segments' section2_locked / section3_locked stayed closed and how many record_bodies_locked.

time_offset dict

meflib's recording-time offset (RTO). status: "none", "applied", "disputed" (segments disagree), "locked" (section 3 encrypted and not opened, so unknown) or "unknown" (no segment could be read). The status describes the READABLE segments; segments_unknown counts those that could not be read, whose offset is unknown (and which make dependent record times None). date_recoverable is True when an offset is set — the real date is then stored in section 3, and recoverable_by says by whom ("anyone" when section 3 is plain, "level-2 password" when encrypted). ambiguous: a time is stored non-negative under a non-zero offset. value holds the offset only with include_values=True. interpretation says in plain words how the times were read. See docs/anonymization.md.

gmt_offset, dst_start_time, dst_end_time

Section 3 time-zone fields (None when locked or unset).

start_time, end_time

First / last sample over all channels.

records_start_time, records_end_time

Earliest / latest time any record mentions.

first_time, last_time

Samples and records together: the span a time shift must cover.

channels list of ChannelSummary
record_only_channels list of str

Channels with records but no data yet.

video_channels list of str

.vidd channels. mef3io does not read video; listed only.

incomplete_channels list of str

Channel folders without one complete segment (a segment needs its .tmet, .tidx and .tdat). A :class:mef3io.Reader does not list them. Incomplete segment folders of other channels appear in their channel's segments as readable=False; both raise an "incomplete-segments" warning.

records list of RecordSummary

Session, then channel and segment records.

identity list of IdentityField

Subject name/ID/location (section 3), descriptions (section 2), session_name / anonymized_name (every universal header) and record free text — presence and length, values only on request.

warnings list of SummaryWarning
problems list of dict

What a strict=False inspection skipped, shaped like :attr:mef3io.Reader.problems.

declaration_issues list of dict

Section-2 size declarations left unset (as :attr:mef3io.Reader.declaration_issues).

index_read bool

Whether the block indices were read (read_index=True).

legacy_password_mapping bool

The password opened some metadata or record file only through the password-to-key mapping of mef3io 1.2.0 and earlier (issue #84): the session was most likely encrypted by one of those versions with a non-ASCII password. The data is read correctly, but meflib and pymef cannot open those files with this password. A "legacy-password" warning (in warnings) says how many. :func:inspect reports this as data and does NOT emit :class:mef3io.LegacyPasswordWarning, as it reports every other condition (a :class:mef3io.Reader opened on the same session does emit it).

offset_assumed_zero bool

Some time here assumes recording-time offset 0 because a section 3 is locked (level-1 password, or none). Only a time stored negated (relative to the offset) needs the assumption; absolute stored times do not set it. The same flag is on every channel, segment and record it concerns. mef3io and pymef always write offset 0, so such times are right for almost every file; under meflib's deprecated date masking they are the masked times, which is what meflib meant a level-1 user to see. They are never called verified, and an "offset-assumed-zero" warning blocks anonymization (which needs the level-2 password anyway).

anonymize_blockers property

anonymize_blockers: list

The warnings anonymize_session refuses on.

channel

channel(name: str) -> ChannelSummary

The :class:ChannelSummary called name (KeyError if none).

identity_field

identity_field(name: str) -> IdentityField

The :class:IdentityField called name (KeyError if none).

to_dict

to_dict() -> dict

Plain nested dicts/lists, ready for json.dumps. Gap spans become 2-element lists.

mef3io.ChannelSummary dataclass

ChannelSummary(name: str, sampling_frequency: Optional[float], units_conversion_factor: Optional[float], units_description: Optional[str], channel_description: Optional[str], start_time: Optional[int], end_time: Optional[int], number_of_samples: Optional[int], gaps: list, gaps_complete: bool, gaps_certain: bool, offset_assumed_zero: bool, bytes: int, records: int, segments: list)

One channel: signal metadata, its segment map and its gaps.

gaps are (start, end) uUTC spans with no samples. Without read_index=True they are only the gaps BETWEEN segments and gaps_complete is False: a gap inside a segment is found only in the block index (whose copy of each block's time and count is used; the authoritative RED block header lives in the .tdat, which is never read). gaps_certain is False when a listed gap may not be one, because a segment that could not be read or placed may hold data there, or because a segment whose times assume offset 0 sits next to one whose non-zero offset was read. number_of_samples counts stored samples (gaps excluded). bytes is every file of the channel; records counts its channel- and segment-level records.

sampling_frequency, units_conversion_factor and units_description come from the first segment whose metadata section 2 is readable, and are None when every section 2 is locked (encrypted, no level-1 password). Segments CAN differ (each has its own section 2); only the first readable one is reported here, except that units_conversion_factor is None when the readable segments store different factors (each segment's is in segments). channel_description is section 2's free-text description from the same segment; every distinct value across segments and channels is in the identity table (identity_field("channel_description").values, with include_values=True). It can identify the subject, so, like every identity value, it is returned only with include_values=True: otherwise it is None, and so it is when section 2 is locked. Its presence is always reported in the identity table (identity_field("channel_description")).

mef3io.SegmentSummary dataclass

SegmentSummary(segment: int, path: str, readable: bool, problem: Optional[str], section2_available: bool, section3_available: bool, start_time: Optional[int], end_time: Optional[int], start_sample: Optional[int], number_of_samples: Optional[int], number_of_blocks: Optional[int], number_of_discontinuities: Optional[int], units_conversion_factor: Optional[float], time_offset: str, recording_time_offset: Optional[int], offset_assumed_zero: bool, tmet_bytes: int, tidx_bytes: int, tdat_bytes: int, record_bytes: int, has_records: bool, provenance: Optional[dict], unset_declarations: list)

One segment, from its metadata only.

Times are absolute uUTC (UTC). With section 3 locked (level-1 password, or none) a time stored relative to the offset is resolved ASSUMING offset 0, as :class:mef3io.Reader does, and offset_assumed_zero is True: not verified (absolute stored times need no assumption). They are None only for a segment that could not be read. time_offset is "none", "applied" (meflib's offset is set), "locked" (section 3 encrypted and not opened) or "unknown" (the segment is unreadable); recording_time_offset holds its value only with include_values=True (the value is itself identifying: it is the local midnight of the recording day). number_of_discontinuities is the DECLARED count (legacy writers leave 0 there even with gaps). units_conversion_factor is the segment's own, as stored (0.0 is MEF's "no entry"; None when section 2 is locked). File sizes are in bytes; the .tdat is measured, never read. provenance is the dict described in :meth:mef3io.Reader.segments, or None.

mef3io.RecordSummary dataclass

RecordSummary(type: str, level: str, channel: Optional[str], segment: Optional[int], time: Optional[int], times_known: bool, offset_assumed_zero: bool, duration: Optional[int], first_time: Optional[int], last_time: Optional[int], text_bytes: int, body_locked: bool, fields: Optional[dict])

One record (annotation), at any level.

time is the header time (absolute uUTC). first_time / last_time span every time the record mentions — the header, the body times (Seiz onsets/offsets, Epoc timestamp/end, Curs trace timestamp) and time + duration — which is what a time range must cover. They are None when the file's times depend on an offset that is disputed or unknown (an unreadable segment in scope); times_known is then False, and the times inside fields are None too — never a placeholder. offset_assumed_zero is True when they were placed with an assumed offset 0 because a section 3 in their scope is locked (as the Reader does): not verified. text_bytes is the length of its free text; the text itself is in fields only with include_values=True (the full record, as :meth:mef3io.Reader.records returns it). body_locked means the body is encrypted and the password does not open it.

mef3io.IdentityField dataclass

IdentityField(name: str, location: str, occurrences: int, locked: int, max_length: int, distinct: int, values: list, present: Optional[bool])

Whether one identifying field holds anything, and where.

occurrences is the number of places (files, or records for record_text) where it is non-empty, locked the number of places it could not be read (encrypted, or a file that failed in lenient mode), max_length the longest value in bytes and distinct the number of different values. values lists them only with include_values=True; otherwise it is empty. present is True if it is set anywhere, False if it is set nowhere, and None if it may be set somewhere that could not be read.

mef3io.SummaryWarning dataclass

SummaryWarning(code: str, message: str, blocks_anonymize: bool)

Something a caller should know before using the session.

blocks_anonymize marks the conditions anonymize_session refuses: video channels, an ambiguous or disputed time convention, unresolved times, locked (encrypted) parts, unreadable files.

Ordering and naming sessions

mef3io.time_range

time_range(path, password: str = '', *, cancel: Optional[CancelToken] = None) -> TimeRange

The first and last sample time of a session, reading as little as possible.

Only the first and last complete segment's .tmet of each channel is read (about 16 KB each), never a block index or data file. Only those segments' files are checked for existence, the others are not touched, so it suits ordering many sessions by time. Segments are assumed numbered in time order, as every known writer numbers them: a middle segment earlier than the first and last is not seen. Works on .mefd folders and .mefd.tar archives. An encrypted session can be ordered without its password (offset_known is then False); a wrong password raises :class:PasswordError.

mef3io.naming.TimeRange dataclass

TimeRange(start_uutc: Optional[int], end_uutc: Optional[int], offset_known: bool, channels: int, files_read: int)

A session's first and last sample time.

Attributes:

Name Type Description
start_uutc, end_uutc int or None

uUTC; None when no channel holds samples. The end is the end of the last block, as the files' headers state it.

offset_known bool

False when the times cannot be shown as absolute: the session is encrypted and the password does not open the part holding the recording-time offset, or it uses that offset with times stored in the ambiguous form. Order such sessions with care.

channels, files_read int

Channels with data looked at, and metadata files read (two per channel at most).

mef3io.name_drift

name_drift(path, *, cancel: Optional[CancelToken] = None) -> NameDrift

Where a session's name is stored and whether those places agree. Writes nothing.

Reads the 1 KB header of every file (and lists the folders).

mef3io.naming.NameDrift dataclass

NameDrift(folder: str, record_files: Tuple[str, ...], stored_names: Dict[str, int], files_read: int, consistent: bool)

Where a session's name is stored, and whether the places agree.

Attributes:

Name Type Description
folder str

The folder's (or archive's) name without .mefd / .mefd.tar.

record_files tuple of str

The names of the session-level annotation files (<name>.rdat); consistent also requires their .ridx to carry the same name.

stored_names dict of str to int

Each distinct name stored in the files' headers, with how many files hold it.

files_read int
consistent bool

All the places name the session the same. A folder renamed by another tool leaves this False, and other MEF software then silently misses the session's annotations.

mef3io.rename_archive

rename_archive(path, new_name: str) -> ArchiveRename

Rename a .mefd.tar archive to <new_name>.mefd.tar in the same folder.

The rename never replaces an existing file: where the file system cannot guarantee that, it is refused rather than done unsafely. Also refused: a name over 246 bytes of UTF-8 (never truncated); a name that is not a portable file name (a folder, a control character, :*?"<>|, a Windows device name such as CON, a trailing dot or space); an archive being written (.tmp or .partial beside it). A symlink is renamed itself, not the file it points to. Undo it with rename_archive(new_path, old_name).

mef3io.naming.ArchiveRename dataclass

ArchiveRename(old_path: str, new_path: str, inside_keeps_old_name: bool)

Result of :func:rename_archive. inside_keeps_old_name is always True: the archive's internal folder, member names and stored names still carry the old name (extract, rename and archive again to change them).

mef3io.rename_session

rename_session(path, new_name: str, *, password: str = '', backup_dir=None, descriptions: Optional[Sequence[str]] = None, _crash_at: int = -1, _fail_at: int = -1) -> RenameReport

Rename a .mefd session in every place its name lives.

A session's name is stored in four places, and other MEF software reads all of them: renaming only the folder makes it silently miss the session's annotations. This changes, together:

  1. the folder, <old>.mefd -> <new_name>.mefd;
  2. the session's annotation files, <old>.rdat / .ridx;
  3. the name stored in every file's header;
  4. session_description in each channel's metadata, only where it is exactly the old name (a description that merely mentions it is kept).

No sample, block index or time changes, and every renamed file is stamped rename in its provenance region.

A crash cannot leave it half-done. The old headers and a journal of the planned steps go to backup_dir first, then a marker into the session; mef3io refuses to write to a marked session. If the operation is interrupted, the next rename_session / fix_name_drift call on the session (with the same backup_dir) rolls it back first. The journal and the backups are deleted when the operation completes.

Parameters:

Name Type Description Default
path str or PathLike

The .mefd folder.

required
new_name str

Without .mefd. At most 250 bytes of UTF-8 and a portable file name (no /\:*?"<>|, control characters, Windows device names, or trailing dot or space). Never truncated.

required
password str

Needed when the session is encrypted: the descriptions are in the level-1 encrypted part.

''
backup_dir str or PathLike

Where the journal and old headers are kept while it runs. Default: <session>.repair-backup beside the session. The old name is identifying, so an application passes a folder of its own.

None
descriptions list of str

Only change the descriptions in these .tmet files (session relative paths). Used by :meth:RenameReport.undo.

None

Raises:

Type Description
ValueError

A name that is not allowed, the same name, or a .mefd.tar (use :func:rename_archive).

IoError

The target exists, a header is damaged (validate and repair first), or the file system cannot rename without the risk of replacing a file.

WriteConflictError

The session is being written, carries the marker of an operation whose journal is not in backup_dir, or its name is already out of step (run :func:fix_name_drift first).

PasswordError

The session is encrypted and password does not open its metadata.

Notes

Undo with report.undo(), which renames back and reverts only the descriptions this call changed. On a macOS HFS+ volume (which stores names decomposed) a long accented name can be refused by the file system at the last step; the operation is then rolled back.

mef3io.fix_name_drift

fix_name_drift(path, target: Optional[str] = None, *, password: str = '', backup_dir=None, descriptions: Optional[Sequence[str]] = None, _crash_at: int = -1, _fail_at: int = -1) -> RenameReport

Bring a session's stored names in line with its folder.

For a session whose folder another tool renamed (:func:name_drift shows it). Sets the name in every file's header, the session's annotation files and the descriptions equal to the old name to target (default: the folder's name). The folder itself is never renamed. Same safety as :func:rename_session (journal, marker, rollback, no-replace renames, provenance rename).

Refused when the old name is ambiguous: the files store different names, or there are several session-level annotation files. Undo with report.undo().

mef3io.naming.RenameReport dataclass

RenameReport(old_path: str, new_path: str, old_name: str, new_name: str, folder_renamed: bool, files_changed: Tuple[str, ...], descriptions_changed: Tuple[str, ...], records_renamed: Tuple[Tuple[str, str], ...], files: Dict[str, Tuple[int, int]], rolled_back: str, journals_discarded: int, operation: str, notes: Tuple[str, ...] = ())

Result of :func:rename_session or :func:fix_name_drift.

Attributes:

Name Type Description
old_path, new_path str

The session folder before and after (equal for fix_name_drift).

old_name, new_name str

The stored session name before and after.

folder_renamed bool
files_changed tuple of str

Session-relative paths of the files whose header changed.

descriptions_changed tuple of str

Session-relative paths of the .tmet files whose session_description changed (it was equal to the old name). :meth:undo reverts exactly these.

records_renamed tuple of (str, str)

Session-level annotation files renamed, as (old, new) file names.

files dict of str to (int, int)

Each changed file's size and modification time (ns) right after the operation, so a caller can tell whether the session was written to since (an undo is still exact after an append).

rolled_back str

The id of an earlier, interrupted operation that was rolled back first, or "".

journals_discarded int

Journals of operations that never started or had finished, removed.

notes tuple of str

Things to know, e.g. a marker that could not be removed after the operation completed.

operation str

"rename" or "fix-name-drift".

changed property

changed: bool

Whether anything was changed (a session already named so is left alone).

undo

undo(*, password: str = '', backup_dir=None) -> 'RenameReport'

Reverse this operation, logically: rename back to the old name and revert exactly the descriptions this operation changed.

Never a byte restore, so samples, times and annotations written since (an append, say) are kept. It is a full operation with every check of the original (marker, journal, no-replace renames, the password of an encrypted session). For fix_name_drift the folder is left as it is.

mef3io.operation_marker

operation_marker(path) -> Optional[Dict[str, str]]

The marker of an unfinished in-place operation, or None.

A dict with tool (e.g. "mef3io 1.5.0"), operation ("rename", "fix-name-drift") and id. It names no session, path or person.

mef3io.clear_operation_marker

clear_operation_marker(path) -> None

Remove the marker of an operation whose journal is lost.

Only for a session whose rename was interrupted and whose backup folder is gone (another machine, a deleted folder). Refused unless :func:name_drift shows the name consistent, so a session the operation left half-changed keeps its marker. When the journal exists, run the operation again instead: it rolls back.

mef3io.SessionOperationWarning

SessionOperationWarning(message: Any = '', *, path: Optional[str] = None, details: Iterable = ())

Bases: MefWarning

The session is in the middle of an in-place operation (a rename, #72).

The operation was interrupted, or is still running in another process: its marker is in the session folder. Until it is finished or rolled back the session's name can be half-changed, so other MEF software may miss its annotations, and mef3io refuses to write to it. Reads are not affected. Run the same operation again (with the same backup folder) to roll it back; details holds the marker (tool, operation, id).

De-identified copies

See De-identification for the workflow. Python and C++ only (no MATLAB, no C ABI).

mef3io.anonymize_session

anonymize_session(src, dst, shift_us: Optional[int] = None, *, pseudonym: Optional[str] = None, clear: Iterable[str] = _CLEAR_CHOICES, text: Union[str, TextCallback] = 'drop', rename: bool = True, password: str = '', output_password: Optional[Sequence[str]] = None, channels: Optional[Sequence[str]] = None, exclude_channels: Optional[Sequence[str]] = None, time_range: Optional[Tuple[int, int]] = None, exact: bool = True, records: str = 'keep', progress: Optional[Callable[[int, int], object]] = None, cancel: Optional[CancelToken] = None) -> AnonymizeReport

Write a de-identified copy of a session: identity removed, times kept or shifted.

The source is never modified. The copy is built next to dst in <dst>.partial/, checked with :func:verify, and only then moved to dst. A failure, a refusal or a cancel leaves nothing behind.

The copy pass reads the source once and writes the copy once, streaming data blocks; :func:verify then reads the source's data and the copy again. Data blocks are never decoded or re-encoded: only their start time (and, for encrypted blocks, the 16 encrypted statistics bytes) changes, and the regions other programs may fill are cleared, so the samples are bit-identical.

Times. By default (shift_us=None) every time is KEPT exactly as the source stores it, and so is its recording-time offset: the copy reads back the same times (and dates) as the source. The copy still shows when the recording was made. Pass shift_us (see :func:reference_shift) to hide the dates: every time then moves by shift_us, each stored in the sign convention it came in: universal-header start/end in every file, block-index entries, data-block headers, record headers, record-index entries and the times inside Seiz, Epoc and Curs records; the copy's recording_time_offset is 0. In both modes GMT_offset = 0 and DST start/end = "no entry" (they can say where the recording was made, and no time mef3io returns depends on them). Identity: subject names, ID and recording location are cleared (clear=); the session name is replaced in every header, in the session-level record files' names and in the descriptions; anonymized_name is set to pseudonym; every UUID is regenerated; other applications' private header regions are cleared; every file is written fresh. Each file is stamped with provenance operation 6, anonymize.

What is refused (nothing is written): a destination that exists or lies inside the source; a session inspect marks as blocking (video channels, unreadable or incomplete segments, parts the password does not open); damaged data (a failing block or record CRC is never re-sealed). When shifting, also: a disputed or ambiguous recording-time offset, a stored time at or before a non-zero recording-time offset, an Epoc/Curs id_number holding a time of the recording, and a shift that is zero, smaller in magnitude than the session's time span plus one day, or that would take any time to 0 or below.

Parameters:

Name Type Description Default
src str or path - like

The session: a .mefd directory or a .mefd.tar archive (read in place).

required
dst str or path - like

The copy, a new .mefd directory. Its stem becomes the session name.

required
shift_us int

None (default): keep every time; the copy shows the real dates. An int: microseconds added to every time. :func:reference_shift computes one from a chosen new date. Must satisfy abs(shift_us) >= span + 1 day so that no copied time can fall inside the original range (0 is refused: leave it out to keep the times).

None
pseudonym str

Written to anonymized_name in every file. Default: the copy's session name. The pseudonym and the copy's name must not contain the patient's names (or any of their words), ID or location, or the original session name (3+ characters anywhere, shorter values as a whole word, ignoring case and Latin accents): ValueError before anything is written.

None
clear iterable of {"subject", "location", "descriptions"}

What to clear. Default: all three. Leaving "descriptions" out keeps the section-2 descriptions with the old session name replaced by the new one and every removed name redacted.

_CLEAR_CHOICES
text ('drop', 'keep')

Record free text (Note/SyLg/EDFA text, Seiz markers and annotation, Epoc type and text, Curs name, CSti and ESti strings). "drop" (default) clears it; the records themselves, their types and (shifted) times are kept. "keep" keeps it, with only the names mef3io knows removed (the old session name, cleared subject and location values, excluded channel names). Names shorter than 4 characters and numbers are replaced only as whole words / whole numbers, and kept channel names are never changed; a short name left inside a longer word makes :func:verify refuse the copy. It does not find dates, ages, other people's names or anything else written in free text. A callable text(value, info) returns the new text (None clears); the same redaction is applied after it. Text that is not valid UTF-8 is passed with errors="surrogateescape" and encoded back the same way, so every byte survives a callable that returns it unchanged. Python only. info is a dict with record_type, field, level, channel, segment and time (the time in the copy, uUTC; None when it cannot be known: without a shift, a time stored against a recording-time offset the file's segments disagree on). Names in a seizure's channel list that are not a kept channel's exact name are cleared with "drop" and redacted otherwise.

"drop"
rename bool

True (default) gives the copy the stem of dst as its session name everywhere. False keeps the source's session name, and the stem of dst must equal it.

True
password str

Opens an encrypted source. It must grant level-2 (full) access: section 3 holds the recording-time offset and the subject fields.

''
output_password (str, str)

(level1, level2) to encrypt the copy, with the same pairing rule as :class:Writer; None (default) writes it unencrypted. The source's passwords are never carried over. Data blocks the source had encrypted are re-encrypted with these passwords at their level, or decrypted when there are none; blocks the source had plain stay plain (and readable by pymef and meflib).

None
channels sequence of str

Copy only these channels. Mutually exclusive with exclude_channels.

None
exclude_channels sequence of str

Leave these channels out. Nothing of an excluded channel stays in the copy: no folder, no records, no entry in a Seiz record, and its name is redacted from kept text.

None
time_range (int, int)

Cut: keep only the samples whose time lies in [start, end), uUTC as the SOURCE holds it (before any shift); annotations are kept when their time lies in it (a Seiz or Epoc record also when its span overlaps it). See :func:mef3io.transform_session.

None
exact bool

With time_range: True (default) trims the edge blocks to the window exactly; False keeps them whole (nothing is decoded).

True
records str

"keep" (default) copies the annotations. "drop" leaves ALL of them out (session, channel and segment level): the copy has no .rdat/.ridx file, like a session that never had annotations; a channel that holds only annotations is left out too. The source's annotations are not copied or checked; the report's records_dropped counts them. Anything else is refused (records-invalid).

'keep'
progress callable

progress(done, total) in bytes (the copy and its verification).

None
cancel CancelToken

Stop the copy from another thread; :class:Cancelled is raised and nothing is left behind.

None

Returns:

Type Description
AnonymizeReport

Raises:

Type Description
ValueError

A bad argument: a given shift (zero, below the floor, into times <= 0), an unknown channel, both channels and exclude_channels, a selection that leaves no channel, an unknown clear= entry, an unpaired output password, rename=False with a differently named dst, a copy name or pseudonym holding the patient's identity.

WriteConflictError

The session cannot be copied without risking a misplaced or leaked date, or the copy failed verification.

PasswordError

The source is encrypted and the password does not grant level 2.

IoError

dst or <dst>.partial exists, or dst is inside src.

(FormatError, CrcError)

Damaged source data (never re-sealed into the copy).

Every refusal above carries ``.code``, a stable string from
:data:`REFUSAL_CODES` (e.g. ``"dst-exists"``, ``"shift-below-floor"``,
``"name-contains-identity"``), and ``.details``, a dict with
``channel``, ``field`` and ``inspect_code`` (each ``None`` when it does
not apply). Match on ``code``, never on the message. :func:`check` returns
the same refusals without writing. Errors that are not refusals (a
cancel, an exception raised by a callback, an internal error) carry no
code. The refusals :func:`check` can report are all raised before
anything is written; those marked ``"copy"`` in :data:`REFUSAL_CODES`
come from the copy pass, which removes the partial copy.

Examples:

Keep the times (the copy shows the real dates):

>>> rep = mef3io.anonymize_session("patient.mefd", "S001.mefd")
>>> rep.shift_us is None
True

Hide the dates by moving the recording to another day:

>>> shift = mef3io.anonymize.reference_shift(start_us, "2000-01-03", tz="Europe/Prague")
>>> rep = mef3io.anonymize_session("patient.mefd", "S002.mefd", shift)
>>> rep.verification.ok
True

mef3io.anonymize.check

check(src, dst, shift_us: Optional[int] = None, *, pseudonym: Optional[str] = None, clear: Iterable[str] = _CLEAR_CHOICES, text: Union[str, TextCallback] = 'drop', rename: bool = True, password: str = '', output_password: Optional[Sequence[str]] = None, channels: Optional[Sequence[str]] = None, exclude_channels: Optional[Sequence[str]] = None, time_range: Optional[Tuple[int, int]] = None, exact: bool = True, records: str = 'keep', cancel: Optional[CancelToken] = None) -> CheckReport

Every check :func:anonymize_session makes before it writes, without writing.

Takes the same arguments as :func:anonymize_session (no progress) and runs the same checks, through the same code: one pre-flight function in the C++ core serves both, so the two cannot disagree. Instead of raising the first refusal it returns all of them, each with its stable code (:data:REFUSAL_CODES).

It writes nothing (no dst, no <dst>.partial) and reads only metadata: what :func:mef3io.inspect reads, plus each kept segment's 1 KB metadata header and the kept channels' record files. It never reads the data (.tdat) files. The GIL is released while it runs.

shift_us decides WHICH copy is checked, exactly as for :func:anonymize_session: None (default) the copy that keeps every time, an int the shifted copy. The checks that concern only a shift (marked below) are made only with one.

What it checks, in order: the options (shift zero, both channels and exclude_channels, output passwords); the destination (exists, a leftover <dst>.partial, not .mefd, inside the source); the source (opens, password grants level 2, nothing :func:mef3io.inspect marks as blocking, including the recording-time-offset refusals); the channel selection; the shift* (floor, times above 0); the copy's name and pseudonym (identity, length, rename=False); segment numbering; record files (readable, index matches, an Epoc/Curs id_number holding a time inside the recording). ( only with a shift.)

What it cannot decide (codes marked "copy" in :data:REFUSAL_CODES): a data block failing its CRC or running past its file (block-crc, data-damaged), and the final verification of the copy (verify-failed), which needs the copy. With a shift, every time in the metadata headers and the record files is put through the copy's own shifting rule up front (time-before-rto, shift-not-positive, time-overflow); only a time in a block index or data file header, or a block's own time, is met by the copy pass alone, with the same code.

No refusal message, here or raised, holds one of the source's times.

Available in Python and C++ (mef3io::check_anonymize); not in MATLAB.

Parameters:

Name Type Description Default
src

As for :func:anonymize_session.

required
dst

As for :func:anonymize_session.

required
shift_us int

The shift of the copy to check; None (default) checks the copy that keeps every time (as :func:anonymize_session without a shift makes it), to which no shift check applies.

None
pseudonym Optional[str]

As for :func:anonymize_session. A text callable is never called.

None
clear Optional[str]

As for :func:anonymize_session. A text callable is never called.

None
text Optional[str]

As for :func:anonymize_session. A text callable is never called.

None
rename Optional[str]

As for :func:anonymize_session. A text callable is never called.

None
password Optional[str]

As for :func:anonymize_session. A text callable is never called.

None
output_password Optional[str]

As for :func:anonymize_session. A text callable is never called.

None
channels Optional[str]

As for :func:anonymize_session. A text callable is never called.

None
exclude_channels Optional[str]

As for :func:anonymize_session. A text callable is never called.

None
time_range Optional[str]

As for :func:anonymize_session. A text callable is never called.

None
exact Optional[str]

As for :func:anonymize_session. A text callable is never called.

None
cancel CancelToken

Stop the check from another thread; :class:Cancelled is raised.

None

Returns:

Type Description
CheckReport

Raises:

Type Description
TypeError

A channel list that is not made of strings (code argument-type).

Cancelled

cancel was set.

Examples:

>>> rep = mef3io.anonymize.check("patient.mefd", "S001.mefd", shift, pseudonym="S001")
>>> rep.ok, rep.codes
(False, ('pseudonym-contains-identity',))

mef3io.anonymize.verify

verify(src, dst, shift_us: Optional[int] = None, *, password: str = '', output_password: Union[None, str, Sequence[str]] = None, clear: Iterable[str] = _CLEAR_CHOICES, time_range: Optional[Tuple[int, int]] = None, exact: bool = True, records: str = 'keep', progress: Optional[Callable[[int, int], object]] = None, cancel: Optional[CancelToken] = None) -> VerifyReport

Check a de-identified copy against its source. Reads both; writes nothing.

:func:anonymize_session runs this on every copy before publishing it; call it yourself to re-check a copy later or one made elsewhere.

It checks that:

  • the validator reports nothing on the copy the source did not already have;
  • the copy holds a subset of the source's channels, segment for segment;
  • every data block's compressed payload and its 256-byte statistics table (compared decrypted) are byte-identical to the source's, so the samples are too, with nothing decoded;
  • with shift_us=None (a copy that keeps the times) every time — universal-header start/end, block-index entries, data-block headers, record headers, record index, the times inside records — equals the source's as stored (so also as read), and so does each segment's recording-time offset; with a shift, every time differs from the source's by exactly shift_us, in the same sign convention, and the offset is 0. In both, GMT offset 0 and DST "no entry";
  • the records match in count, order and type (with records="drop": the copy holds none);
  • a field-wise leak scan finds nothing: with a shift, every time field (universal headers, section 3, block index, data-block headers, record headers, record index, Seiz/Epoc/Curs body times) is checked against the source's recording range, plain or negated, widened by a day; every text field (session and anonymized names, descriptions, subject fields, record text) is checked for the names the copy must not carry; no source UUID is reused. Sizes, counts and sample values are never read as times, and the compressed data is compared rather than scanned, so the scan has no false positives from random bytes.

Parameters:

Name Type Description Default
src str or path - like

The source session and the copy.

required
dst str or path - like

The source session and the copy.

required
shift_us int

The shift the copy was made with; None (default) for a copy that keeps every time.

None
password str

Opens an encrypted source (level 2).

''
output_password str or (str, str)

Opens an encrypted copy: its level-2 password, or the pair.

None
clear iterable of str

What the copy was asked to clear (as for :func:anonymize_session).

_CLEAR_CHOICES
time_range optional

The cut the copy was made with (as for :func:anonymize_session).

None
exact optional

The cut the copy was made with (as for :func:anonymize_session).

None
progress optional

As for :func:anonymize_session.

None
cancel optional

As for :func:anonymize_session.

None

Returns:

Type Description
VerifyReport

mef3io.anonymize.reference_shift

reference_shift(reference_start_us: int, new_date, *, tz: Optional[str] = None, keep_weekday: bool = False) -> int

The shift that moves a reference recording to new_date.

D is a whole number of days, measured on the clock being kept:

  • tz given (keep the LOCAL time of day): D = new_date - (the reference's local start date in tz) and the shift is D + off_ref, where off_ref is the reference start's UTC offset in tz at that instant. The copy's UTC clock then reads the original local time of day.
  • tz omitted (keep the UTC time of day): D = new_date - (the reference's UTC start date) and the shift is D.

keep_weekday rounds D to the nearest multiple of 7 days, so the copy falls on the same weekday (off_ref is never rounded).

Parameters:

Name Type Description Default
reference_start_us int

The reference recording's start, uUTC.

required
new_date date or str

The date the reference should start on, e.g. "2000-01-03".

required
tz str

An IANA time zone (e.g. "America/Chicago").

None
keep_weekday bool
False

Returns:

Type Description
int

The shift, in microseconds.

mef3io.anonymize.redact_text

redact_text(src, dst, texts: Sequence[str], *, pseudonym: Optional[str] = None, clear: Iterable[str] = _CLEAR_CHOICES, rename: bool = True, password: str = '', channels: Optional[Sequence[str]] = None, exclude_channels: Optional[Sequence[str]] = None, cancel: Optional[CancelToken] = None) -> List[str]

The text :func:anonymize_session would keep, for each of texts.

Runs the exact removal the copy applies to text it keeps (record text with text="keep", and the descriptions when "descriptions" is not in clear), for the same source, destination name and options. An application can then compare a copy's text with what mef3io itself would have left, without copying mef3io's rules. It writes nothing, reads only metadata (what :func:check reads), and never logs or stores its inputs.

It is not a safety check on the result: :func:check refuses text the removal cannot take out (code text-unremovable), and :func:verify checks every copy.

Parameters:

Name Type Description Default
src

As for :func:anonymize_session; dst gives the copy's name and need not exist (an existing one is fine: nothing is written).

required
dst

As for :func:anonymize_session; dst gives the copy's name and need not exist (an existing one is fine: nothing is written).

required
texts sequence of str

The texts to redact. Text that is not valid UTF-8 can be passed with errors="surrogateescape"; it comes back the same way.

required
pseudonym Optional[str]

As for :func:anonymize_session.

None
clear Optional[str]

As for :func:anonymize_session.

None
rename Optional[str]

As for :func:anonymize_session.

None
password Optional[str]

As for :func:anonymize_session.

None
channels Optional[str]

As for :func:anonymize_session.

None
exclude_channels Optional[str]

As for :func:anonymize_session.

None

Returns:

Type Description
list of str

One redacted text per input, in order.

Raises:

Type Description
What the copy itself would refuse, raised as :func:`anonymize_session`
raises it (its ``.code``): a source that cannot be read or is locked
(``source-missing``, ``password-required``, ``password-level1-only``),
an unknown channel, a destination, pseudonym or kept channel name holding
the patient's identity (``channel-name-identity``), and so on. Only refusals that do not decide the redaction are
ignored: the destination already existing (``dst-exists``,
``partial-exists``) and text the removal cannot take out
(``text-unremovable``, which is what :func:`check` reports).

mef3io.AnonymizeReport dataclass

AnonymizeReport(destination: str, session_name: str, pseudonym: str, shift_us: Optional[int], source_recording_time_offset: int, channels_kept: Tuple[str, ...], channels_excluded: Tuple[str, ...], channels_without_samples: Tuple[str, ...], segments: int, blocks: int, blocks_reencrypted: int, blocks_decrypted: int, records: int, record_text_fields_changed: int, record_bodies_cleared: int, seizure_channel_entries_dropped: int, blocks_trimmed: int, records_outside_window: int, bytes_written: int, output_encrypted: bool, not_copied: Tuple[str, ...], notes: Tuple[str, ...], verification: VerifyReport, source_start_us: Optional[int] = None, source_end_us: Optional[int] = None, source_channel_times: Dict[str, Tuple[Optional[int], Optional[int]]] = dict(), records_dropped: int = 0)

Result of :func:anonymize_session.

Attributes:

Name Type Description
destination str

The copy.

session_name, pseudonym str

The copy's session name (its folder's stem) and the value written to every anonymized_name.

shift_us int or None

The shift applied to every time, in microseconds; None when the copy keeps every time as it was.

source_recording_time_offset int

The source's recording-time offset (meflib's deprecated date masking). A shifted copy has 0; a copy that keeps the times keeps it (each segment its own). 0 when the source's segments disagree on it: there is no single one (notes says so).

channels_kept, channels_excluded tuple of str
channels_without_samples tuple of str

A cut: channels left out because they have no sample in the window (also in channels_excluded).

segments, blocks, records int

What was copied.

blocks_reencrypted, blocks_decrypted int

Data blocks the source had encrypted, written encrypted with the new passwords, or decrypted because the copy has none.

record_text_fields_changed int

Record text fields the text= policy (and the name redaction) changed.

record_bodies_cleared int

Record bodies of an unknown type or version, not carried over because they may hold text or dates that cannot be found.

seizure_channel_entries_dropped int

Per-channel entries of Seiz records that named an excluded channel.

blocks_trimmed, records_outside_window int

A cut (time_range): edge blocks re-encoded to the window, and annotations left out.

records_dropped int

records="drop": annotations of the channels and segments the copy keeps (and the session's own) that were left out; records is then 0. Those of a channel the copy leaves out are not counted.

bytes_written int
output_encrypted bool
not_copied tuple of str

Files in the source that are not part of the MEF layout and were left out (source-relative paths).

notes tuple of str
verification VerifyReport

The check the copy passed before it was published.

source_start_us, source_end_us int or None

The SOURCE's first and last sample time, uUTC (None without samples).

source_channel_times dict of str to (int or None, int or None)

The same for each kept channel (None for a channel with records only).

Notes

source_start_us, source_end_us and source_channel_times are the re-identification key: with shift_us they give the real dates back (a copy that keeps the times shows them anyway, but they are handled the same way). They are returned to the caller, in memory, and nowhere else: mef3io never writes them into the copy (the copy's leak scan refuses it if any appears), never logs them and never puts them in a warning or a message. They are left out of repr(), :meth:summary, :meth:to_dict (unless include_original_times=True) and pickling: a pickled report arrives without them unless it was marked with :meth:with_original_times_in_pickle (for example to pass it to a helper process you control; never to a cache). copy.copy and copy.deepcopy keep them. dataclasses.asdict(report) INCLUDES them: prefer :meth:to_dict. Store them as you would the original recording.

with_original_times_in_pickle

with_original_times_in_pickle() -> 'AnonymizeReport'

A copy of this report whose pickle KEEPS the original times.

By default a pickled report drops them (the re-identification key). Use this only to hand the report to a process you control, never for a cache or a file.

to_dict

to_dict(*, include_original_times: bool = False) -> dict

Plain dict (JSON-serialisable).

The source's original times (the re-identification key) are left out unless include_original_times=True, so that logging or sharing the dict cannot leak them by accident.

summary

summary() -> str

A few lines for a terminal.

mef3io.anonymize.CheckReport dataclass

CheckReport(ok: bool, refusals: Tuple[Refusal, ...], notes: Tuple[str, ...], session_name: str, pseudonym: str, channels_kept: Tuple[str, ...], channels_excluded: Tuple[str, ...], channels_without_samples: Tuple[str, ...] = ())

Result of a pre-flight check (anonymize.check, transform.check).

Attributes:

Name Type Description
ok bool

True when the copy would start. It can still be refused by what only the copy pass meets (codes marked "copy" in REFUSAL_CODES).

refusals tuple of Refusal

Every refusal found, in the order the copy meets them (it raises the first). Checks that depend on one that failed are skipped: nothing about the source is checked when it cannot be opened.

notes tuple of str

What would be noted up front: a source that uses meflib's recording-time offset (anonymize), an output password other MEF readers will not handle as expected.

session_name str

The copy's session name ("" when dst is unusable).

pseudonym str

Anonymize only: the copy's pseudonym ("" for a transform).

channels_kept, channels_excluded tuple of str

The channel selection (empty when the source could not be read).

channels_without_samples tuple of str

A cut: kept channels with no sample in the window, which the copy leaves out (also in channels_excluded).

codes property

codes: Tuple[str, ...]

The refusals' codes, in order.

summary

summary() -> str

A few lines for a terminal.

mef3io.anonymize.Refusal dataclass

Refusal(code: str, message: str, error: str, channel: Optional[str] = None, field: Optional[str] = None, inspect_code: Optional[str] = None)

One reason a copy would be refused.

Attributes:

Name Type Description
code str

Stable identifier (REFUSAL_CODES of :mod:mef3io.anonymize or :mod:mef3io.transform). Match on this, never on message.

message str

Plain words, as the raised exception carries them. It may name the source's or the destination's path.

error str

The exception type the copy raises for it: "ValueError", "TypeError", "IoError", "WriteConflictError", "PasswordError", "FormatError" or "CrcError".

channel str or None

The channel concerned, if any.

field str or None

The argument or field concerned ("dst", "pseudonym", "channels", "subject_id", ...), if any.

inspect_code str or None

For a condition :func:mef3io.inspect marks as blocking, that warning's code (e.g. "video-channels").

mef3io.anonymize.REFUSAL_CODES module-attribute

REFUSAL_CODES: Dict[str, Tuple[str, str]] = {**COPY_REFUSAL_CODES, 'clear-unknown': ('ValueError', 'check'), 'text-invalid': ('ValueError', 'check'), 'shift-zero': ('ValueError', 'check'), 'text-callback-missing': ('ValueError', 'defensive'), 'rto-disputed': ('WriteConflictError', 'check'), 'rto-unreadable': ('WriteConflictError', 'check'), 'time-before-rto': ('WriteConflictError', 'check'), 'shift-below-floor': ('ValueError', 'check'), 'shift-not-positive': ('ValueError', 'check'), 'name-contains-identity': ('ValueError', 'check'), 'pseudonym-contains-identity': ('ValueError', 'check'), 'id-number-time': ('WriteConflictError', 'check'), 'time-overflow': ('FormatError', 'defensive'), 'text-unremovable': ('WriteConflictError', 'check'), 'channel-name-identity': ('WriteConflictError', 'check')}

mef3io.VerifyReport dataclass

VerifyReport(ok: bool, problems: Tuple[VerifyIssue, ...], leaks: Tuple[VerifyIssue, ...], notes: Tuple[VerifyIssue, ...], inherited_findings: Tuple[str, ...], channels: Tuple[str, ...], excluded_channels: Tuple[str, ...], segments: int, blocks: int, records: int, times_checked: int, text_fields_checked: int)

Result of a copy's verification (anonymize.verify, transform.verify).

Attributes:

Name Type Description
ok bool

True when nothing is wrong: no problem and no leak.

problems tuple of VerifyIssue

Everything wrong with the copy, leaks included.

leaks tuple of VerifyIssue

Anonymize only: the problems found by the leak scan (a time field holding a value in the source's recording range, plain or negated; a name that should be gone; a source UUID; an uncleared field).

notes tuple of VerifyIssue

Anonymize only: overlaps reported but not refused, worth showing the user (an identity value inside a kept channel's name as part of a longer token or shorter than 3 characters, inside the copy's own name, or a number of fewer than 4 digits inside a longer number in kept text).

inherited_findings tuple of str

Validator findings the source already had; a copy carries them over unchanged and they do not count against ok.

channels, excluded_channels tuple of str

Channels in the copy, and the source's channels it leaves out.

segments, blocks, records, times_checked, text_fields_checked int

How much was compared.

summary

summary() -> str

A few lines for a terminal.

mef3io.anonymize.VerifyIssue dataclass

VerifyIssue(location: str, field: str, message: str)

One problem a verification found.

message names what is wrong in plain words. It never repeats an identifying string (a name it found is described, not quoted), so a report can be logged or shared.

Modified copies

See Modified copies for what changes and what is copied as stored. time_range= (a cut) is described there too. The report, check and refusal types are shared with the de-identified copies above. Python and C++ only (no MATLAB, no C ABI).

mef3io.transform_session

transform_session(src, dst, *, channels: Optional[Sequence[str]] = None, exclude_channels: Optional[Sequence[str]] = None, output_password: Optional[Sequence[str]] = None, metadata: Optional[Mapping] = None, time_range: Optional[Tuple[int, int]] = None, exact: bool = True, records: str = 'keep', rename: bool = True, password: str = '', progress: Optional[Callable[[int, int], object]] = None, cancel: Optional[CancelToken] = None) -> TransformReport

Write a modified copy of a session: a choice of channels, passwords, name and details.

The source is never modified. The copy is built next to dst in <dst>.partial/, checked with :func:verify, and only then moved to dst. A failure, a refusal or a cancel leaves nothing behind.

What stays as stored. Every time: universal headers, block index, data blocks, records, the recording-time offset and the time-zone fields, so no date is moved or revealed. Every sample: data blocks are copied without being decoded, only the 16 encrypted statistics bytes of an encrypted block are re-keyed. The regions other programs fill in MEF headers. Record bodies (except the Seiz entries of left-out channels).

What changes. Only what you ask for: the channels (channels or exclude_channels), the passwords (output_password; the source's are never carried over), the session name (the stem of dst, in every header and in the session's own annotation files) and the metadata fields you set. Every file gets a new file UUID, and its provenance records the transform (operation 8) while keeping the link to the source file it came from.

Parameters:

Name Type Description Default
src str or path - like

The session: a .mefd directory or a .mefd.tar archive (read in place).

required
dst str or path - like

The copy, a new .mefd directory. Its stem becomes the session name.

required
channels sequence of str

Copy only these channels. Mutually exclusive with exclude_channels.

None
exclude_channels sequence of str

Leave these channels out: no folder, no records, no entry in a Seiz record.

None
output_password (str, str)

(level1, level2) to encrypt the copy, with the same pairing rule as :class:Writer; None (default) writes it unencrypted. Data blocks the source had encrypted are re-encrypted with these passwords at their level, or decrypted when there are none.

None
metadata mapping

Descriptive fields to set in the copy (:data:METADATA_FIELDS): subject_name_1, subject_name_2, subject_id, recording_location, session_description (each a str; "" clears it, None or absent keeps it), and channel_descriptions / reference_descriptions (dicts from a channel of the copy to its str). Each value must fit its field (ValueError otherwise; it is never cut). Fields not given are copied as stored.

None
time_range (int, int)

Cut (#103): keep only the samples whose time lies in [start, end), uUTC. Segments outside it are not read; blocks outside it are left out; the segments it keeps are renumbered from 0 and their declarations describe what they hold. Annotations are kept when their time lies in the window (a Seiz or Epoc record also when its span overlaps it). The copy is verified against the source over the window: blocks kept whole are byte-identical, trimmed ones hold exactly the source's samples.

None
exact bool

With time_range: True (default) decodes, trims and re-encodes the edge block of each segment at each end, so the copy holds exactly the window (the cost does not depend on the window's length); False keeps whole edge blocks and decodes nothing.

True
records str

"keep" (default) copies the annotations. "drop" leaves ALL of them out (session, channel and segment level): the copy has no .rdat/.ridx file, like a session that never had annotations; a channel that holds only annotations is left out too. The source's annotations are not copied or checked; the report's records_dropped counts them. Anything else is refused (records-invalid).

'keep'
rename bool

True (default) gives the copy the stem of dst as its session name. False keeps the source's session name, and the stem of dst must equal it.

True
password str

Opens an encrypted source. It must grant level-2 (full) access.

''
progress callable

progress(done, total) in bytes (the copy and its verification).

None
cancel CancelToken

Stop the copy from another thread; :class:Cancelled is raised and nothing is left behind.

None

Returns:

Type Description
TransformReport

Raises:

Type Description
ValueError

A bad argument: an unknown channel, both channels and exclude_channels, a selection that leaves no channel, an unpaired output password, rename=False with a differently named dst, a metadata field that is unknown, too long, or names a channel not in the copy.

TypeError

A channel list or a metadata value of the wrong type.

WriteConflictError

Parts of the session cannot be copied whole (video channels, unreadable or incomplete segments or annotation files), or the copy failed verification.

PasswordError

The source is encrypted and the password does not grant level 2.

IoError

dst or <dst>.partial exists, or dst is inside src.

(FormatError, CrcError)

Damaged source data (never re-sealed into the copy).

Every refusal carries ``.code`` (:data:`REFUSAL_CODES`) and ``.details``
(``channel``, ``field``, ``inspect_code``), as for
:func:`mef3io.anonymize_session`. :func:`check` returns the same refusals
without writing.

Examples:

>>> rep = mef3io.transform_session(
...     "patient.mefd", "patient_eeg.mefd", exclude_channels=["ECG"],
...     metadata={"channel_descriptions": {"Fp1": "frontal pole, left"}})
>>> rep.verification.ok
True

mef3io.transform.check

check(src, dst, *, channels: Optional[Sequence[str]] = None, exclude_channels: Optional[Sequence[str]] = None, output_password: Optional[Sequence[str]] = None, metadata: Optional[Mapping] = None, time_range: Optional[Tuple[int, int]] = None, exact: bool = True, records: str = 'keep', rename: bool = True, password: str = '', cancel: Optional[CancelToken] = None) -> CheckReport

Every check :func:transform_session makes before it writes, without writing.

Takes the same arguments (no progress) and runs the same checks, through the same pre-flight code in the C++ core, so the two cannot disagree. Instead of raising the first refusal it returns all of them, each with its stable code (:data:REFUSAL_CODES). It writes nothing and reads only metadata (never a .tdat). What it cannot decide: a data block failing its CRC (block-crc, data-damaged) and the final verification (verify-failed).

Returns:

Type Description
CheckReport

pseudonym is always "" (a transform has none).

mef3io.transform.verify

verify(src, dst, *, channels: Optional[Sequence[str]] = None, exclude_channels: Optional[Sequence[str]] = None, output_password: Optional[Sequence[str]] = None, metadata: Optional[Mapping] = None, time_range: Optional[Tuple[int, int]] = None, exact: bool = True, records: str = 'keep', password: str = '', progress: Optional[Callable[[int, int], object]] = None, cancel: Optional[CancelToken] = None) -> VerifyReport

Check a transformed copy against its source. Reads both; writes nothing.

:func:transform_session runs this on every copy before publishing it. Pass the metadata (and output_password) the copy was made with. The channels are read from the copy; channels and exclude_channels are accepted for symmetry and not needed.

It checks that the validator reports nothing new; the copy holds a subset of the source's channels, segment for segment; every data block's payload and statistics (compared decrypted) are byte-identical; every time equals the source's; every other byte of every header, metadata section and record body equals the source's, except the names (when renamed), UUIDs, password fields, encryption flags, provenance and CRCs, and the metadata fields, which hold exactly the requested edits.

Returns:

Type Description
VerifyReport

mef3io.TransformReport dataclass

TransformReport(destination: str, session_name: str, channels_kept: Tuple[str, ...], channels_excluded: Tuple[str, ...], channels_without_samples: Tuple[str, ...], segments: int, blocks: int, blocks_reencrypted: int, blocks_decrypted: int, records: int, seizure_channel_entries_dropped: int, blocks_trimmed: int, records_outside_window: int, metadata_fields_changed: int, bytes_written: int, output_encrypted: bool, not_copied: Tuple[str, ...], notes: Tuple[str, ...], verification: VerifyReport, records_dropped: int = 0)

Result of :func:transform_session.

Attributes:

Name Type Description
destination str

The copy.

session_name str

The copy's session name (its folder's stem).

channels_kept, channels_excluded tuple of str
channels_without_samples tuple of str

A cut: channels left out because they have no sample in the window (also in channels_excluded).

segments, blocks, records int

What was copied.

blocks_reencrypted, blocks_decrypted int

Data blocks the source had encrypted, written encrypted with the new passwords, or decrypted because the copy has none.

seizure_channel_entries_dropped int

Per-channel entries of Seiz records that named a left-out channel.

blocks_trimmed, records_outside_window int

A cut (time_range): edge blocks re-encoded to the window, and annotations left out.

records_dropped int

records="drop": annotations of the channels and segments the copy keeps (and the session's own) that were left out; records is then 0. Those of a channel the copy leaves out are not counted.

metadata_fields_changed int

Metadata fields whose stored value changed (one per field per segment).

bytes_written int
output_encrypted bool
not_copied tuple of str

Files in the source that are not part of the MEF layout and were left out (source-relative paths).

notes tuple of str
verification VerifyReport

The check the copy passed before it was published.

to_dict

to_dict() -> dict

Plain dict (JSON-serialisable).

summary

summary() -> str

A few lines for a terminal.

mef3io.transform.REFUSAL_CODES module-attribute

REFUSAL_CODES: Dict[str, Tuple[str, str]] = {**COPY_REFUSAL_CODES, 'metadata-unknown': ('ValueError', 'check'), 'metadata-too-long': ('ValueError', 'check'), 'metadata-invalid': ('ValueError', 'check'), 'metadata-channel-unknown': ('ValueError', 'check')}

mef3io.transform.METADATA_FIELDS module-attribute

METADATA_FIELDS: Dict[str, int] = {'subject_name_1': 128, 'subject_name_2': 128, 'subject_id': 128, 'recording_location': 512, 'session_description': 2048, 'channel_descriptions': 2048, 'reference_descriptions': 2048}

Merged sessions

See Merging recordings. Python and C++ only (no MATLAB, no C ABI).

mef3io.merge_sessions

merge_sessions(sources: Sequence, dst, *, passwords: Optional[Sequence[str]] = None, channels: Optional[Sequence[str]] = None, exclude_channels: Optional[Sequence[str]] = None, output_password: Optional[Sequence[str]] = None, metadata: Optional[Mapping] = None, records: str = 'keep', time_ranges: Optional[Sequence[Optional[Tuple[int, int]]]] = None, exact: bool = True, anonymize: Union[None, bool, Mapping] = None, n_threads: int = 0, progress: Optional[Callable[[int, int], object]] = None, cancel: Optional[CancelToken] = None) -> MergeReport

Merge two or more recordings into ONE new session.

One call can do the whole pipeline: leave channels out, trim each recording to its own time range, make the result anonymous, leave the annotations out, and write ONE new session. The copy pass reads each source byte once and writes it once (several channels at a time, see n_threads); :func:verify then reads the sources and the result again before anything is published.

The sources are never modified. The result is built next to dst in <dst>.partial/, checked with :func:verify against every source, and only then moved to dst. A failure, a refusal or a cancel leaves nothing behind.

What is refused before anything is written (:func:check lists every one):

  • merge-overlap: two sources hold samples of the same channel at the same moment. This is decided on the RECORDED data (from the block indexes), so recordings that take turns merge: A records and stops, B records and stops, A continues. A source's own slightly overlapping blocks (recorder jitter) never count.
  • merge-rate-differs: a channel has different sampling rates.
  • merge-rto-differs: the sources use different recording-time offsets.
  • merge-identity-differs: the sources' subject fields (names, ID, location) differ, which may mean different patients. Set the fields for the result with metadata to merge anyway. With anonymize the fields it clears (clear=, default all) may differ: the result carries none of them.

How the result is laid out. Each source segment becomes a segment of the result. A segment that has another source's data in one of its gaps is split there (at a block boundary, nothing decoded), so that the result's segments never overlap in time. They are ordered by time and numbered from 0 in each channel. Channels held by only some sources are kept. Annotations at the session and channel level are merged in time order; a segment's own annotations stay with it.

Parameters:

Name Type Description Default
sources sequence of str or path-like

Two or more sessions (.mefd directories or .mefd.tar archives).

required
dst str or path - like

The result, a new .mefd directory; its stem is the session name.

required
passwords sequence of str

The password opening each source, in order (level 2 for encrypted ones). Omit for unencrypted sources.

None
channels sequence of str

Which channels the result holds, over all sources.

None
exclude_channels sequence of str

Which channels the result holds, over all sources.

None
output_password (str, str)

Encrypt the result with these two passwords; None = unencrypted.

None
records str

"keep" (default) copies the annotations. "drop" leaves ALL of them out (session, channel and segment level): the copy has no .rdat/.ridx file, like a session that never had annotations; a channel that holds only annotations is left out too. The source's annotations are not copied or checked; the report's records_dropped counts them. Anything else is refused (records-invalid).

'keep'
metadata mapping

Descriptive fields for the result, as for :func:mef3io.transform_session. Not combined with anonymize (merge-anonymize-metadata).

None
time_ranges sequence

One entry per source, in order: (start_us, end_us) keeps only the samples of that source in [start, end) (uUTC as the SOURCE stores them), None keeps the whole source. The cut rules of :func:mef3io.transform_session's time_range apply to each source: edge blocks are decoded, trimmed and re-encoded, annotations outside the range are left out. Overlap between sources is judged on the TRIMMED data, so two recordings that overlap only outside their ranges merge. A source with no sample in its range is refused (time-range-empty); a channel no source has a sample of in its range is left out (channels_without_samples). The number of entries must equal the number of sources (merge-time-ranges-count).

None
exact bool

With time_ranges: True (default) trims the edge blocks to the range exactly; False keeps whole blocks (nothing is decoded).

True
anonymize True or mapping

Make the result anonymous in the same pass, with :func:mef3io.anonymize_session's options: shift_us, pseudonym, clear and text (rename only as True: a merge is named after dst). True = the defaults (times kept, identity cleared, record text dropped). ONE shift applies to every source, judged against the span of ALL of them (the whole recordings, trimmed or not). The identity of EVERY source (subject names, ID, location, session names, earlier pseudonyms) is removed and scanned for in the result, and the result is verified exactly as :func:mef3io.anonymize.verify verifies a copy. The report's pseudonym and shift_us say what was applied. A text callable is called serially, in a fixed order (the copy pass then uses one thread).

None
n_threads int

Threads for the whole operation (:doc:threads </threads>): opening the sources, copying channels, verifying. 0 (default) = automatic (at least 8, as for any file access), 1 = no extra threads. The result never depends on it: every data, index and metadata byte after the 1024-byte universal headers is identical (the headers hold fresh random UUIDs).

0
progress optional

As for :func:mef3io.transform_session. progress is called at a bounded rate, never concurrently and never backwards, from whichever worker thread crosses a step (with the GIL held).

None
cancel optional

As for :func:mef3io.transform_session. progress is called at a bounded rate, never concurrently and never backwards, from whichever worker thread crosses a step (with the GIL held).

None

Returns:

Type Description
MergeReport

mef3io.merge.check

check(sources: Sequence, dst, *, passwords: Optional[Sequence[str]] = None, channels: Optional[Sequence[str]] = None, exclude_channels: Optional[Sequence[str]] = None, output_password: Optional[Sequence[str]] = None, metadata: Optional[Mapping] = None, records: str = 'keep', time_ranges: Optional[Sequence[Optional[Tuple[int, int]]]] = None, exact: bool = True, anonymize: Union[None, bool, Mapping] = None, n_threads: int = 0, cancel: Optional[CancelToken] = None) -> CheckReport

Every check :func:merge_sessions makes before it writes, without writing.

Reads only metadata and the block indexes of segments that overlap in time across sources (with time_ranges, of the blocks around each range); never a .tdat. Takes the arguments of :func:merge_sessions.

mef3io.merge.verify

verify(sources: Sequence, dst, *, passwords: Optional[Sequence[str]] = None, channels: Optional[Sequence[str]] = None, exclude_channels: Optional[Sequence[str]] = None, output_password: Optional[Sequence[str]] = None, metadata: Optional[Mapping] = None, records: str = 'keep', time_ranges: Optional[Sequence[Optional[Tuple[int, int]]]] = None, exact: bool = True, anonymize: Union[None, bool, Mapping] = None, n_threads: int = 0, progress: Optional[Callable[[int, int], object]] = None, cancel: Optional[CancelToken] = None) -> VerifyReport

Check a merged session against its sources. Reads all; writes nothing.

Pass the arguments the result was made with. Every segment of the result is checked against the piece of the source segment it came from (blocks byte-identical, metadata as the source's plus the edits); annotations against the sources' merged in time order; segments must not overlap; and the validator must report nothing on the result. With anonymize every time must be exactly the shift away and no text field or path may hold the identity of ANY source. time_ranges and anonymize are the ones the result was made with.

mef3io.MergeReport dataclass

MergeReport(destination: str, session_name: str, sources: Tuple[str, ...], channels_kept: Tuple[str, ...], channels_excluded: Tuple[str, ...], segments: int, segments_split: int, blocks: int, blocks_reencrypted: int, blocks_decrypted: int, records: int, seizure_channel_entries_dropped: int, metadata_fields_changed: int, bytes_written: int, output_encrypted: bool, not_copied: Tuple[str, ...], notes: Tuple[str, ...], verification: VerifyReport, channels_without_samples: Tuple[str, ...] = (), blocks_trimmed: int = 0, records_outside_window: int = 0, pseudonym: str = '', shift_us: Optional[int] = None, source_recording_time_offset: int = 0, record_text_fields_changed: int = 0, record_bodies_cleared: int = 0, source_count: int = 0, source_start_us: Optional[int] = None, source_end_us: Optional[int] = None, source_channel_times: Dict[str, Tuple[Optional[int], Optional[int]]] = dict(), records_dropped: int = 0, units_mixed: Dict[str, Tuple[str, ...]] = dict())

Result of :func:merge_sessions.

Attributes:

Name Type Description
destination, session_name str
sources tuple of str

As given. ANONYMIZED result: memory only, like the re-identification key below (the paths name the patient): empty in :meth:to_dict (unless include_original_times=True) and in pickles, and notes there name source N instead of the path. source_count always holds the number of recordings.

channels_kept, channels_excluded tuple of str

Over all sources.

channels_without_samples tuple of str

time_ranges: channels left out because no source has a sample of them inside its range (also in channels_excluded).

segments int

Segments of the result.

segments_split int

Source segments split in two or more pieces because another source's data lies in one of their gaps.

blocks, records, blocks_reencrypted, blocks_decrypted int
blocks_trimmed, records_outside_window int

time_ranges: edge blocks re-encoded to a range, and annotations left out for lying outside their source's range.

records_dropped int

records="drop": annotations of the channels and segments the copy keeps (and the session's own) that were left out; records is then 0. Those of a channel the copy leaves out are not counted.

seizure_channel_entries_dropped, metadata_fields_changed int
bytes_written int
output_encrypted bool
not_copied, notes tuple of str
verification VerifyReport

The check the result passed before it was published.

pseudonym str

anonymize=: the value written to every anonymized_name (empty otherwise).

shift_us int or None

anonymize=: the one shift applied to every time of every source; None when the times were kept (or without anonymize).

source_recording_time_offset int

anonymize=: the sources' shared recording-time offset (0 without).

record_text_fields_changed, record_bodies_cleared int

anonymize=: record text fields the text policy and the name redaction changed; record bodies of an unknown type or version not carried over.

source_start_us, source_end_us int or None
source_channel_times dict of str to (int or None, int or None)

anonymize=: THE RE-IDENTIFICATION KEY over all sources (earliest first sample, latest end, and the same per kept channel). Handled exactly as in :class:mef3io.anonymize.AnonymizeReport: in memory only, out of repr(), :meth:summary, :meth:to_dict (unless include_original_times=True) and pickles.

with_original_times_in_pickle

with_original_times_in_pickle() -> 'MergeReport'

A copy of this report whose pickle KEEPS the original times (see :meth:mef3io.anonymize.AnonymizeReport.with_original_times_in_pickle).

to_dict

to_dict(*, include_original_times: bool = False) -> dict

Plain dict (JSON-serialisable). The sources' original times (the re-identification key) are left out unless include_original_times.

summary

summary() -> str

A few lines for a terminal.

mef3io.merge.REFUSAL_CODES module-attribute

REFUSAL_CODES: Dict[str, Tuple[str, str]] = {**_TRANSFORM_CODES, **_ANONYMIZE_CODES, 'merge-too-few': ('ValueError', 'check'), 'merge-passwords-count': ('ValueError', 'check'), 'merge-time-range': ('ValueError', 'check'), 'merge-time-ranges-count': ('ValueError', 'check'), 'merge-anonymize-metadata': ('ValueError', 'check'), 'merge-threads-invalid': ('ValueError', 'check'), 'merge-rename': ('ValueError', 'check'), 'merge-rto-differs': ('WriteConflictError', 'check'), 'merge-rate-differs': ('WriteConflictError', 'check'), 'merge-identity-differs': ('WriteConflictError', 'check'), 'merge-overlap': ('WriteConflictError', 'check')}

Cancelling and progress

See the Python guide.

mef3io.CancelToken

CancelToken()

A flag that stops a running read or long operation.

Pass it as cancel= to :meth:Reader.read, :meth:Reader.read_raw, :func:archive_session, :func:extract_session, :func:inspect, :meth:Validator.validate, :func:validate_session, :func:repair_session or :func:recover_session, and call :meth:cancel from any thread. The operation notices between units of work (a decoded block, a 1 MB copy chunk, a segment) and raises :class:Cancelled.

A token is one-shot: once cancelled it stays cancelled, so create a new one per request. One token may be shared by several calls to stop them together (for example every channel of one viewer request).

Cancellation and progress are available in Python and C++ only. MATLAB has neither: a MEX call blocks MATLAB's only interpreter thread, so nothing could call :meth:cancel while it runs.

Examples:

>>> token = mef3io.CancelToken()
>>> threading.Timer(0.5, token.cancel).start()
>>> try:
...     x = reader.read("ch1", cancel=token)
... except mef3io.Cancelled:
...     pass  # a newer request superseded this one

cancelled property

cancelled: bool

Whether :meth:cancel has been called.

cancel

cancel() -> None

Request cancellation. Safe from any thread; idempotent.

raise_if_cancelled

raise_if_cancelled() -> None

Raise :class:Cancelled if the token is set.

For a caller's own loops between mef3io calls — for example reading many channels for one request — so they stop at the same points the library does.

mef3io.Cancelled

Bases: Exception

An operation was stopped through its :class:CancelToken.

Validation and repair

See the validation guide for what each check means.

mef3io.Validator

Validator(path: str, password: str = '', channels: Sequence[str] | None = None, segments: Sequence[int] | None = None, exact_difference_bytes: bool = True)

Check one MEF 3.0 session. Read-only: it cannot write a byte.

There is no repair method here by design — see :func:repair_session.

Parameters:

Name Type Description Default
path str

A .mefd directory or a .mefd.tar archive.

required
password str

Needed to read section 2 of an encrypted session.

''
channels sequence of str

Restrict to these channels. Default: every channel.

None
segments sequence of int

Restrict to these segment numbers. Default: every segment.

None
exact_difference_bytes bool

Read every RED block header to learn the real maximum_difference_bytes. It is the one declaration not derivable from the block index, so the exact answer costs one small read per block. Set False on very large sessions: the check then only reports a clearly-unset value, bounded by meflib's worst case, without touching .tdat at all.

True

available_checks staticmethod

available_checks() -> list[Check]

Every check in the registry, in the order they run.

describe_check staticmethod

describe_check(check_id: str) -> Check | None

One check by id, or None if unknown.

validate

validate(check_ids: Iterable[str] | None = None, *, cancel: CancelToken | None = None, progress: Callable[[int, int], object] | None = None) -> Report

Run every check (or only check_ids) and return the full report.

Never modifies the session.

Parameters:

Name Type Description Default
check_ids iterable of str

Run only these checks (the integrity checks always run).

None
cancel CancelToken

Stop the pass from another thread. Checked per segment and, with exact_difference_bytes, per block; raises :class:Cancelled and returns no report.

None
progress callable

progress(done, total) in bytes of .tmet + .tidx + .tdat covered. Called at a bounded rate, never concurrently; (total, total) exactly once, when the pass completes. If it raises, the pass stops and the exception propagates.

None

check

check(check_id: str) -> Report

Run one named check. Never modifies the session.

mef3io.Report dataclass

Report(findings: tuple[Finding, ...] = (), skipped: tuple[SkippedSegment, ...] = (), segments_checked: int = 0, segments_repaired: int = 0, checks_run: tuple[str, ...] = (), checks_repaired: tuple[str, ...] = (), path: str = '', channels: tuple[str, ...] = (), segments: tuple[int, ...] = (), measured_difference_bytes: bool = True)

The result of a validate or repair pass.

is_narrowed property

is_narrowed: bool

True when this run cannot speak for the whole session.

ok property

ok: bool

True when nothing worse than a warning is left outstanding.

Findings this pass repaired no longer count against it, so a repair that resolved everything reports ok. A skipped segment always does count — an unchecked segment is not a clean one.

errors property

errors: tuple[Finding, ...]

Every error-severity finding, repaired or not.

unresolved property

unresolved: tuple[Finding, ...]

Findings that are still outstanding — nothing repaired this pass.

repairable_check_ids property

repairable_check_ids: list[str]

Distinct check ids that reported something a repair could fix.

Pass these to :func:repair_session to opt in deliberately.

scope_caveat

scope_caveat() -> str

One sentence naming what this run did not cover, or "" if it covered everything.

by_check

by_check() -> dict[str, list[Finding]]

Findings grouped by check id, in registry order.

summary

summary(max_examples: int = 3) -> str

A full human-readable report: counts, then each check that fired.

mef3io.Finding dataclass

Finding(check_id: str, severity: str, channel: str, segment: int, path: str, field: str, stored: str, expected: str, message: str, repairable: bool, repaired: bool)

One problem, found by one check, in one segment.

Findings about a record (annotation) file at channel or session level carry segment == -1 (and channel == "" for the session).

mef3io.Check dataclass

Check(id: str, title: str, description: str, severity: str, repairable: bool)

One check in the registry.

mef3io.available_checks

available_checks() -> list[Check]

Every check, in the order they run. Ids are stable API.

mef3io.describe_check

describe_check(check_id: str) -> Check | None

One check by id, or None if unknown.

mef3io.SkippedSegment dataclass

SkippedSegment(channel: str, segment: int, path: str, reason: str)

A segment that could not be checked, and why. Never silently dropped.

mef3io.validate_session

validate_session(path: str, *, cancel: CancelToken | None = None, progress: Callable[[int, int], object] | None = None, **kwargs) -> Report

Check a session and return the report. Reads only.

Accepts the same keyword arguments as :class:Validator, plus cancel and progress as in :meth:Validator.validate.

mef3io.repair_session

repair_session(path: str, check_ids: Sequence[str], backup: bool = True, *, cancel: CancelToken | None = None, progress: Callable[[int, int], object] | None = None, **kwargs) -> Report

Rewrite the declarations named in check_ids — and only those.

This is the only function in mef3io that modifies an existing session, and it is deliberately not a method of :class:Validator: inspecting a session and rewriting one should not be reachable through the same object.

Every check still runs, so the returned report describes the whole session; Finding.repaired marks what was actually written — a repair that declines to change anything is not counted.

Only declarations are rewritten: metadata section 2 and the universal headers of .tmet/.tidx/.tdat. Sample data and the block index are never touched. A segment whose CRCs do not verify is reported and left alone, and a .mefd.tar archive is refused outright (extract it first).

Parameters:

Name Type Description Default
path str

A .mefd directory. Archives cannot be rewritten in place.

required
check_ids sequence of str

Which repairs to apply. Required and non-empty — repairs are never implicit. :attr:Report.repairable_check_ids lists the candidates.

required
backup bool

Ignored since 2.0.1: a backup is always kept, in <session>.repair-backup/ (outside the session tree, so no reader mistakes a backup for data), and an existing backup is never overwritten. backup=False is still accepted and emits one :class:BackupAlwaysKeptWarning.

True
cancel CancelToken

Stop the repair from another thread. Repairs are decided during the validation pass and written only after it: a cancel during the pass raises :class:Cancelled with nothing written — no file and no backup directory. The write phase that follows (a .tmet replaced by rename and at most two 1024-byte header patches per segment) is not interruptible; a cancel arriving then is ignored and the report is returned.

None
progress callable

progress(done, total) in bytes, as in :meth:Validator.validate.

None
**kwargs

The same keyword arguments as :class:Validator (password, channels, segments, exact_difference_bytes).

{}

Raises:

Type Description
ValueError

If check_ids is empty, or names an unknown or non-repairable check.

mef3io.recover_session

recover_session(path: str, apply: bool = False, backup: bool = True, password: str = '', *, cancel: CancelToken | None = None, progress: Callable[[int, int], object] | None = None) -> RecoveryReport

Make each segment's block index and data agree after an interrupted write.

This is not :func:repair_session, which only ever rewrites declarations and is safe to run on anything. Recovery may truncate the index, rebuild index entries from the data file, and drop a trailing fragment — so it is a dry run unless you ask for it, and it backs up what it changes first.

An interrupted append leaves one of two shapes, and they are handled differently because one has lost data and the other has not:

  • index ahead of data — entries reference bytes that never landed. Those samples do not exist, so the entries are dropped.
  • data ahead of index — blocks reached .tdat but the index was not extended. Those samples do exist, so they are recovered: a RED block header carries the sample count, byte count, start time and discontinuity flag, which is everything an index entry needs. Only blocks whose CRC verifies are indexed; a torn tail is not a block.

Both are reachable after a power cut or OS crash during an append: the writer flushes only each segment's .tmet before renaming it, not the .tdat or .tidx it extends in place, and this function is what makes that trade safe. Only the last append of a segment can be affected.

Parameters:

Name Type Description Default
path str

.mefd session directory. Tar archives are refused.

required
apply bool

False reports what it would do and writes nothing.

False
backup bool

Ignored since 2.0.1: what changes is always saved to <session>.recover-backup/ first (only the .tidx, the .tdat's 1024-byte header and any dropped fragment — never the whole data file, which may be tens of gigabytes). A dataless segment is moved there, never deleted. backup=False is still accepted and emits one :class:BackupAlwaysKeptWarning.

True
password str

Only needed for encrypted sessions.

''
cancel CancelToken

Stop recovery from another thread. Every segment is scanned before anything is written: a cancel during the scan raises :class:Cancelled with nothing written — no file and no backup. The write phase that follows is not interruptible (whole-file renames and 1024-byte header patches); a cancel arriving then is ignored and the report is returned.

None
progress callable

progress(done, total) in bytes of .tidx + .tdat covered. Called at a bounded rate, never concurrently; (total, total) exactly once, when the scan completes (before anything is written).

None

Returns:

Type Description
RecoveryReport
Notes

With apply=True the declarations of every segment it changed (and of a channel's last segment whose .tmet is newer or older than its index, which a power cut can leave) are re-derived from the index through :func:repair_session — so recovery alone leaves a valid session after an unclean shutdown. Only the segments it reports are touched (the dry run lists their check ids in RecoveredSegment.declarations); a healthy segment's findings are left to :func:repair_session. A later segment whose .tmet was lost is rebuilt from the previous segment's.

mef3io.RecoveryReport dataclass

RecoveryReport(segments: tuple[RecoveredSegment, ...] = (), record_files: tuple[RecoveredRecordFile, ...] = (), skipped: tuple[str, ...] = (), segments_examined: int = 0, applied: bool = False, backup_root: str = '', notes: tuple[str, ...] = ())

What recover_session found, and did or would do.

mef3io.RecoveredSegment dataclass

RecoveredSegment(channel: str = '', segment: int = 0, path: str = '', blocks_before: int = 0, blocks_after: int = 0, blocks_recovered: int = 0, blocks_dropped: int = 0, tdat_bytes_dropped: int = 0, action: str = '', declarations: tuple[str, ...] = ())

One segment that an interrupted write left inconsistent.

mef3io.RecoveredRecordFile dataclass

RecoveredRecordFile(path: str = '', records_before: int = 0, records_after: int = 0, bytes_dropped: int = 0, records_damaged: int = 0, index_rebuilt: bool = False, action: str = '')

One annotation file (.rdat + .ridx) an interrupted write left inconsistent.

Recovery keeps every whole record, drops only a torn fragment after the last whole one (backed up) and rebuilds the index and headers from the records. A record that fails its CRC (records_damaged) is damage, not an interrupted write: it is reported and kept, never removed.

Errors

mef3io raises mef3io.MefError and its subclasses FormatError (damaged or malformed files), CrcError (a checksum does not match), PasswordError (a missing or wrong password), IoError (files, paths, session naming) and WriteConflictError (a write that would break the session). All of them subclass RuntimeError. A refusal of anonymize_session also carries a stable code and details (Refusal codes); bad arguments raise ValueError or TypeError. mef3io.Cancelled is deliberately not a MefError. mef3io.have_cpp_backend() says whether the compiled core is available.

Warnings

Every warning carries its session in w.message.path and never in its text; see Warnings carry the path as data.

mef3io.MefWarning

MefWarning(message: Any = '', *, path: Optional[str] = None, details: Iterable = ())

Bases: UserWarning

Base class of every warning mef3io emits.

The message never contains the session's location (its directory or archive path) or its name: they appear as <session>, also inside an archive (<session>::<session>/...) and in the session's own file names (<session>.rdat). Nothing else is scrubbed: channel names and the names of files inside the session can appear, and those can be identifying too. A caller with a stricter rule should log only the class name, or nothing.

What the message summarises is on the warning object, which Python passes as w.message (warnings.catch_warnings(record=True), pytest.warns) or raises under -W error:

Attributes:

Name Type Description
path str or None

The session (or, for a writer, the session being written) the warning is about, exactly as it was passed in. None when there is none.

details tuple

The structured items behind the message: the problem dicts of an unreadable segment, the block-copy mismatches, the unset declarations, and so on (each subclass says which). These MAY contain paths.

Python's ``"default"`` and ``"module"`` filter actions show a given
warning once per session (per calling file), not once per process the
path is no longer in the text, so each session keeps its own registry.
Filter every mef3io warning at once with :

warnings.filterwarnings("ignore", category=mef3io.MefWarning)

Examples:

>>> with warnings.catch_warnings(record=True) as caught:
...     warnings.simplefilter("always")
...     r = mef3io.Reader(path)
>>> for w in caught:
...     if issubclass(w.category, mef3io.MefWarning):
...         log.warning("%s", w.message)        # no session path or name
...         where = w.message.path              # the path, as data

mef3io.SessionDeclarationWarning

SessionDeclarationWarning(message: Any = '', *, path: Optional[str] = None, details: Iterable = ())

Bases: MefWarning

A session leaves size declarations unset that other readers rely on.

Raised on open, once per process per session (Python's default warning filter suppresses a repeat of the same message from the same line). It never affects mef3io's own reads — mef3io sizes every buffer from the block headers themselves — but meflib-based readers (most established MEF tooling) allocate from metadata section 2 before decoding, and cannot tell an unset field from a real measurement.

Silence it like any warning::

warnings.filterwarnings("ignore", category=mef3io.SessionDeclarationWarning)

or per call with Reader(path, warn_declarations=False). To see the detail, or to fix it, use :class:mef3io.Validator.

Like every :class:MefWarning, the message carries no path: w.message.path is the session and w.message.details the unset declarations (dicts with channel, segment and field, as :attr:Reader.declaration_issues).

mef3io.UnreadableSegmentWarning

UnreadableSegmentWarning(message: Any = '', *, path: Optional[str] = None, details: Iterable = ())

Bases: MefWarning

Part of this session could not be read and is being skipped.

Only ever raised by a Reader opened strict=False. The skipped segments' time spans read back as NaN — indistinguishable, in the returned array, from a genuine recording gap. That is the hazard lenient mode trades for availability, so it is never silent: see :attr:Reader.problems for the segment paths and the reason each one failed.

Strict is the default. An unreported CRC failure is how corrupt scaling reaches an analysis unnoticed; lenient mode exists to salvage the intact 99% of an archive, not to relax the checks.

Also raised by :class:Writer (and the compat MefWriter) for segment metadata it had to skip; details are then the strings of Writer.unreadable_metadata. From a Reader, details are the :attr:Reader.problems dicts it is about. w.message.path is the session; the message itself carries no path (see :class:MefWarning).

mef3io.BlockCopyWarning

BlockCopyWarning(message: Any = '', *, path: Optional[str] = None, details: Iterable = ())

Bases: MefWarning

A block's two stored copies of its start time or sample count disagree.

Every RED block records its start time and sample count TWICE — once in the .tidx entry and once in the block's own header inside the .tdat. Nothing in the format keeps them in step. mef3io places data by the HEADER, which is the copy the per-block CRC protects and the copy meflib and pymef read, so a file whose copies have drifted apart still reads the same as it does under the legacy stack.

This warning means the file itself is internally inconsistent — it does not mean the read is wrong. It fires only on such a file: on anything mef3io or the legacy stack wrote, the copies are identical and nothing is emitted.

The structured detail is in the block_copy_mismatches key returned by :meth:Reader.read_raw. Silence it like any warning::

warnings.filterwarnings("ignore", category=mef3io.BlockCopyWarning)

w.message.details holds the same mismatch dicts and w.message.path the session; the message carries no path (see :class:MefWarning).

mef3io.UnscaledSamplesWarning

UnscaledSamplesWarning(message: Any = '', *, path: Optional[str] = None, details: Iterable = ())

Bases: MefWarning

Values read from a segment whose conversion factor is not set.

A segment stores units_conversion_factor 0 — MEF's "no entry", which meflib's and pymef's section-2 templates hold until a writer sets it — so there is no factor to convert its counts into units. :meth:Reader.read, :meth:Reader.block_envelope, :class:MultiReader and the compat MefReader.get_data then return those samples as the RAW STORED COUNTS (factor 1); before 2.0 they read back as zeros. Emitted once per Reader and channel, the first time such a segment is read. :meth:Reader.segments shows which segments store 0, and :meth:Reader.read_raw reports the factor as stored (0) in units_conversion_factors. w.message.details holds the channel name.

mef3io.LegacyPasswordWarning

LegacyPasswordWarning(message: Any = '', *, path: Optional[str] = None, details: Iterable = ())

Bases: MefWarning

The password opened this session only through mef3io's OLD byte mapping.

MEF derives a 16-byte key from the password one byte per character. meflib (and so pymef and mef_tools) keeps the low byte of each character's Unicode code point; mef3io 1.2.0 and earlier kept the last byte of its UTF-8 encoding instead. The two agree on ASCII and disagree on everything else ("é" is 0xE9 in meflib, 0xA9 in old mef3io) — issue #84.

mef3io now writes with meflib's mapping and reads with it first. A session an older mef3io encrypted with a non-ASCII password still opens, through a fallback to the old mapping, and this warning says so (on open for the metadata, from :meth:Reader.records for record files): the data is intact, but meflib and pymef cannot open that session with that password. To make it readable everywhere, make a re-keyed copy: :func:anonymize_session with output_password=(level1, level2) re-encrypts every file with meflib's mapping. That copy is NOT a pure re-key: it is de-identified and every time is shifted (by at least the recording's span + 1 day). A pure re-key is transform_session (see the transform guide).

The fallback cannot tell who wrote a file: it also accepts a SECOND password, the one whose old-mapping bytes equal the real key (for a session keyed with "b©", also "bé"), whoever wrote it. That is inherent to any compatibility fallback and is why it is announced.

Appending to such a session keeps the existing segments' keys (rewriting them would mean re-encrypting them); new segments use meflib's mapping.

w.message.details lists what matched that way: channel names (on open), record file paths (from :meth:Reader.records) or segment paths (from :class:Writer). w.message.path is the session; the message carries no path (see :class:MefWarning). :func:mef3io.inspect does not emit it: it reports the same condition as SessionSummary.legacy_password_mapping.

mef3io.PasswordInteropWarning

PasswordInteropWarning(message: Any = '', *, path: Optional[str] = None, details: Iterable = ())

Bases: MefWarning

A password that other MEF readers handle differently than you may expect.

Raised by :class:Writer for a password longer than 15 characters (meflib and pymef refuse it, so only mef3io will open the session) or one that contains a character above U+00FF (MEF keeps only the low byte of each code point, as meflib does, so a different password can unlock the same session). The session is still written; the warning only tells you about its reach. Short Latin-1 passwords (ASCII, or e.g. "é", "ü") raise nothing.

w.message.path is the session being written; details is ("level-1",) or ("level-2",), which password it is about. The password itself never appears in the warning.

mef3io.MixedFactorsWarning

MixedFactorsWarning(message: Any = '', *, path: Optional[str] = None, details: Iterable = ())

Bases: MefWarning

A copy holds a channel whose segments use different conversion factors.

Raised by :func:mef3io.merge_sessions and :func:mef3io.transform_session when the result has such a channel (sources recorded at different gains, merged or kept as they are). Nothing is wrong with the file: mef3io and pymef scale each segment by its own factor and read the right values. Older tools that scale by the channel-level factor, such as the legacy mef_tools MefReader.get_data, read such a channel as all zeros (meflib stores 0 at channel level when segments differ). Use mef3io or pymef to read it, or write it again with one factor per channel.

w.message.path is the result session; details lists the channel names.

mef3io.MixedUnitsWarning

MixedUnitsWarning(message: Any = '', *, path: Optional[str] = None, details: Iterable = ())

Bases: MefWarning

A merged channel holds segments with different units strings.

Raised by :func:mef3io.merge_sessions when the data the result keeps carries more than one units string for a channel ("mV" next to "uV"). Units are free text, so the merge goes ahead: every segment keeps its own string, samples are never rescaled, and readers label the whole channel with the FIRST kept segment's units. Check that the recordings really use the units you expect. "uV", "µV" (micro sign) and "μV" (Greek mu) count as the same unit and never warn. The merge report's units_mixed and notes give the same information; the validator reports it as units.mixed.

w.message.path is the result session; details lists the channel names.

mef3io.StepFallbackWarning

StepFallbackWarning(message: Any = '', *, path: Optional[str] = None, details: Iterable = ())

Bases: MefWarning

Writer.write(..., precision="step") on a NEW channel could not use a step.

Raised once per such write. The data was still written, with the default (inferred power-of-ten) precision — what a default write stores: the data was not a whole multiple of one step (filtered or resampled data never is), showed too few distinct values to establish one (a DC level, a square wave: fewer than 16 distinct magnitudes in the sample), or its counts would not fit int32. The summary's step_fallback says which.

(On a channel that already has data, "step" never falls back: data off the channel's factor is refused with WriteConflictError.)

w.message.details is [channel, reason]; w.message.path the session.

mef3io.ClampedSamplesWarning

ClampedSamplesWarning(message: Any = '', *, path: Optional[str] = None, details: Iterable = ())

Bases: MefWarning

A float write stored some samples as the nearest storable value.

MEF stores int32 counts; a sample whose count at the conversion factor would leave that range, or that is +inf/-inf, is stored as the largest count, +-(2**31 - 1) — meflib's own "positive/negative infinity" values (RED_POSITIVE_INFINITY/RED_NEGATIVE_INFINITY, meflib.h, which its RED_round clamps to as well) — so it reads back as +-(2**31 - 1) * factor. The write succeeds; nothing wraps. Typical cause: a saturation artifact in an append to a channel whose factor is fine. NaN is never clamped: it stays a gap.

Raised once per such write. w.message.details is ({"channel": ..., "clamped": n, "infinite": k},) (k of the n were infinite); the text carries counts only, never sample values or times. The write summary has the same numbers (clamped, clamped_infinite).

mef3io.WriterFlushWarning

WriterFlushWarning(message: Any = '', *, path: Optional[str] = None, details: Iterable = ())

Bases: MefWarning

A :class:Writer that was never closed could not flush when it was garbage-collected.

Only :meth:Writer.close, :meth:Writer.flush and with can raise; a destructor cannot, so this warning is all it can say. Data written since the last successful flush may not be on disk: run recover on the session. w.message.path is the session; details is [reason].

mef3io.BackupAlwaysKeptWarning

BackupAlwaysKeptWarning(message: Any = '', *, path: Optional[str] = None, details: Iterable = ())

Bases: MefWarning

backup=False (or --no-backup) was passed and ignored.

recover_session and repair_session always keep a backup of what they change; the argument is accepted only so existing callers keep working.

Metadata objects

mef3io.Metadata dataclass

Metadata(subject: Subject = Subject(), acquisition: Acquisition = Acquisition())

Session-wide metadata: a :class:Subject and an :class:Acquisition block. Written to every channel; on read, reflects the first channel.

to_dict

to_dict() -> dict

Nested plain-dict view: {"subject": {...}, "acquisition": {...}}.

from_info classmethod

from_info(info: dict) -> 'Metadata'

Build a Metadata from a :meth:Reader.info dict. Subject fields are empty when the reader lacks level-2 access (section3_available).

mef3io.Subject dataclass

Subject(name_1: str = '', name_2: str = '', id: str = '', recording_location: str = '', gmt_offset: int = 0)

Subject / recording-context metadata (MEF section 3, level-2 encrypted).

mef3io.Acquisition dataclass

Acquisition(session_description: str = '', channel_description: str = '', reference_description: str = '', acquisition_channel_number: int = 1, low_frequency_filter: float = _UNSET_HZ, high_frequency_filter: float = _UNSET_HZ, notch_filter: float = _UNSET_HZ, line_frequency: float = _UNSET_HZ)

Descriptive / acquisition metadata (MEF section 2, level-1 encrypted).

Filter settings default to -1.0 meaning "not recorded".

Legacy mef_tools compatibility

Drop-in replacements for mef_tools.io — from mef3io import MefReader, MefWriter. Same call shapes and defaults as the legacy classes; see the legacy comparison for the measured differences.

mef3io.compat.MefReader

MefReader(session_path: str, password2: Optional[str] = None, warn_declarations: bool = True)

mef_tools.io.MefReader-compatible reader.

session_path may also be an uncompressed tar archive of a session (name.mefd.tar, see :func:mef3io.archive_session).

block_envelope

block_envelope(channel: str, t_stamp1=None, t_stamp2=None) -> dict

Per-block minimum and maximum from the block index alone.

Not part of the legacy mef_tools API: the same as :meth:mef3io.Reader.block_envelope, here so code that holds a MefReader (such as a server) needs no second reader. Returns NumPy arrays start_uutc, end_uutc, min, max, one row per data block overlapping [t_stamp1, t_stamp2), scaled like :meth:get_data.

get_annotations

get_annotations(channel: Optional[str] = None)

Records as a list of dicts (legacy mef_tools signature).

channel=None returns session-level records; a channel name returns that channel's CHANNEL-level records only, as the legacy reader did. Segment-level records are not included here (the native :meth:mef3io.Reader.records does include them for a channel, or use Reader.records(channel, level="segment")). Every dict carries level / channel / segment; see :meth:mef3io.Reader.records for the fields. Returns None (with a printed warning) on failure, as the legacy API did.

mef3io.compat.MefWriter

MefWriter(session_path, overwrite=False, password1=None, password2=None, verbose=False, metadata=None, wait_for_disk=False)

mef_tools.io.MefWriter-compatible writer.

Like the legacy writer, appends extend the channel's last segment in place (in-segment append); pass new_segment=True to start a fresh segment.

Metadata works two ways: mutate the legacy section3_dict / section2_ts_dict (e.g. w.section3_dict['subject_ID'] = 'Smith') as with mef_tools, or use the modern object via set_metadata / the metadata= argument (mef3io.Metadata).

mef_block_len property writable

mef_block_len

RED block length in samples (None = derive from fs, like legacy).

max_nans_written property writable

max_nans_written: int

Kept for legacy compatibility. mef3io always splits data on NaN runs (never stores NaN as values), i.e. it behaves like the legacy writer's recommended setting of 0; other values are accepted and ignored.

record_offset property writable

record_offset: int

Kept for legacy compatibility; non-zero values are accepted and ignored -- mef3io never creates a recording-time offset.

Times are UTC, always. A new session is written at offset 0 with times stored as -t, as before. A session that already carries a non-zero offset (e.g. one the legacy stack wrote with record_offset) keeps it: segments and channels mef3io adds reuse it, and every time mef3io writes there -- data and annotations -- is stored as rto - t, so the real date stays hidden and the existing annotations stay readable. A time at or before the offset cannot be stored that way and is refused, as is any write while the session's segments disagree on the offset or it is locked (level-1 password only).

set_metadata

set_metadata(metadata) -> None

Set session-wide subject/acquisition metadata (a :class:mef3io.Metadata or a flat dict). Applied on the next write.

get_mefblock_len

get_mefblock_len(fs: float) -> int

Block length that will be used for data at fs (legacy formula).

flush

flush()

Return only when everything written so far is on disk (one combined wait), as :meth:mef3io.Writer.flush.