mef3io¶
MEF 3.0 read and write — one C++ core, identical behavior in Python, MATLAB, and C++.
MEF 3.0 (Multiscale Electrophysiology Format) is a compressed, encryptable file format for long electrophysiology recordings: mainly EEG, intracranial EEG (iEEG) and ECG, but it suits any regularly sampled signal, such as EMG, other biosignals or sensor data. mef3io implements it once, in C++17/20, and wraps that core thinly for Python, MATLAB and C++, so float scaling, NaN-gap handling, precision inference, appends, and encryption behave exactly the same everywhere, and every binding carries the same version number.
Long recordings
These files run for days to months and reach tens of gigabytes per channel. Long recordings covers the append workload, what an append flushes, and what is guaranteed at that scale; Validation & repair covers recovering a session after an interrupted write.
Why mef3io¶
- Fast — RED blocks are encoded and decoded in parallel. Full reads run ~9× faster than the legacy pymef/mef_tools stack and whole-session writes ~2× faster, at identical file sizes. That was measured on a 6-core 2013 Xeon; an Apple-silicon Mac measured ~7–8× on both. Each core decodes at the same speed as meflib: the gain comes from using all of them. MATLAB and Python perform identically. See the benchmarks and the legacy comparison.
- Correct — cross-validated in both directions against pymef/mef_tools (values, NaN gaps, times, encryption, fractional sampling rates, records), with committed golden fixtures, Python and C++ test suites, and a MATLAB round-trip suite.
- Drop-in —
from mef3io import MefReader, MefWriterruns existing mef_tools code with only the import changed. - Encryption that works — the two-level password model is implemented as specified: a level-1 password reads the signal and technical metadata while subject identity stays locked. (The legacy reader returns zero samples and garbage metadata at level 1 — details.) Encryption hides identity and annotation text, not the samples themselves — see what it protects.
- De-identified copies for sharing —
mef3io.anonymize_sessionwrites a copy with the subject's identity, the session name, descriptions and annotation text removed (and, with a shift, every date moved by the same amount; by default the real dates are kept), chosen channels left out, and the samples unchanged; the copy can be unencrypted or get new passwords.mef3io.anonymize.checkreports what would stop a copy before anything is written, and every copy is verified against its source before it is saved. Python and C++ — see De-identification. - Modified copies —
mef3io.transform_sessionwrites a copy with only the changes you ask for: chosen channels, new passwords or none, a new name, edited subject details and descriptions, optionally only a time window. Every time and sample is copied as stored, and the copy is verified against its source. Python and C++ — see Modified copies. - Renaming without breaking anything —
mef3io.rename_sessionchanges a session's name in the folder, the session-level annotation files and every file's header as one operation that a crash cannot leave half-done and that can be undone;mef3io.name_driftsays whether they agree. Python and C++ — see Ordering and naming many sessions. - Merging recordings —
mef3io.merge_sessionsmerges two or more recordings into one session: parts of one stay, or recordings that take turns, checked for overlapping data before anything is written; one call can also trim each recording to its own time range and anonymize the result; different units in one channel are merged with a warning. Python and C++ — see Merging recordings. - Deterministic — output is byte-identical regardless of thread count.
- Responsive, with thread-safe reads — opening, reading and writing
release Python's GIL, so a GUI stays responsive while a session opens or
decodes on a worker thread, and reads on one
Readercan run from several threads at once (aWriteris not for sharing). Onen_threadscount per object covers opening, file access and decoding; see Threads. - Made for tools —
mef3io.inspect()describes a session without reading its samples; reads and long operations can report progress and be cancelled from another thread; refusals of a de-identified copy come with stable codes (Python and C++). - Encrypted data blocks written by some third-party tools are read too; see compatibility.
MEF 3.0 and the original libraries¶
mef3io grew out of an earlier stack of MEF libraries and stays compatible with it: files written by either open in the other. MEF 3.0 support is maintained here, in mef3io, as open source (Apache-2.0).
- MEF 3.0 specification: the header of meflib,
meflib.h, is titled "Specification for Multiscale Electrophysiology Format (MEF) version 3.0". The format's design is described in Brinkmann et al., Large-scale electrophysiology: acquisition, compression, encryption, and storage of big data, J Neurosci Methods 180(1):185-192, 2009, doi:10.1016/j.jneumeth.2009.03.022. - meflib: the original reference C library for MEF 3.0, from the msel-source project.
- pymef (docs): a Python wrapper over meflib.
- mef_tools (archived; on PyPI as
mef-tools): high-level Python reading and writing built on pymef. mef3io stands in for it:from mef3io import MefReader, MefWriter. - MATLAB (legacy): matmef (a MATLAB wrapper over meflib), mef_reader_fieldtrip (FieldTrip reader for MEF 2.1, MEF 3.0 and MED 1.0) and MEF_import (EEGLAB plugin). mef3io has its own MATLAB interface.
- The full list of earlier tools and their contributors is on Contributors and credits.
- The older MEF 2.1 format has its own library and specification, mef_lib_2_1; mef3io reads and writes MEF 3.0 only.
mef3io is an independent implementation of the specification, tested against meflib, pymef and mef_tools. New to the format? See MEF 3.0 for students.
Quick taste (Python)¶
import mef3io
with mef3io.Writer("session.mefd", overwrite=True, units="uV") as w:
w.write("ch1", data, start_uutc, fs=256.0) # NaN = gap
w.write_annotations([{"time": start_uutc, "text": "start"}], "ch1")
with mef3io.Reader("session.mefd") as r:
x = r.read("ch1", t0, t1) # float64 on the fs grid, gaps = NaN
r.segments("ch1") # what data is where, across any gaps
The same session opens in MATLAB (mef3io.Reader('session.mefd')) and C++
(mef3io::Reader), and in the legacy pymef/mef_tools stack.
Where to go¶
| Install | pip wheels, MATLAB MEX, building from source |
| Python guide | Reader, Writer, inspect, de-identified and modified copies, cancelling, threads, legacy compat, caching |
| MATLAB guide | +mef3io classes and the MEX |
| C++ guide | using the core library and the flat C ABI |
| How-to | short answers: read in Python or MATLAB, convert EDF to MEF 3.0, long recordings with gaps |
| Examples | thirteen runnable scripts |
| Release notes | what changed in each version, and what an upgrade asks of you |
| Format reference | the MEF 3.0 on-disk structure, field by field |
| De-identification | de-identified copies with anonymize_session, and why meflib's recording-time offset (deprecated) is not anonymization |
| Modified copies | copies with chosen channels, passwords, name and details, with transform_session |
| Python guide: naming | rename_session, name_drift, time_range |
| Merging recordings | two or more recordings into one session, with merge_sessions |
| Performance & legacy comparison | measured speed, encryption, and quantization differences |
Scope and limitations¶
- MEF video files are out of scope; a session's video folders are skipped.
- Encryption is all or nothing per session, and protects identity and annotation text, not the signal — see what encryption protects.
- Single-file
.mefd.tarsessions are a mef3io addition, not part of MEF 3.0: legacy tools need them unpacked first (extract_session,tar -xf, or 7-Zip on Windows); see the Python guide. - Data blocks are written unencrypted. That is not the design of MEF 3.0 but an artifact of past implementations (meflib's default), which mef3io keeps for compatibility; aligning with the specification is under discussion (compatibility).
- Every record type MEF 3.0 defines is read;
Note,SyLgandEDFAcan be written with their text. - De-identification, modified copies, cut, merge, renaming,
inspect, validation and repair, and cancelling and progress are in Python and C++, not in MATLAB. - The pure-Python fallback backend is not implemented.
- Sessions written by mef3io ≤ 1.1.2 declare some buffer sizes as
0, which can crash meflib-based readers (mef3io and pymef read them fine); Validating and repairing shows how to fix them.
What changed in each version is in the release notes.
Related projects¶
- mef3io-studio: a desktop app for exploring and anonymizing MEF 3.0 recordings; installers are on its Releases page.
- mef3io-server: a gRPC server that streams MEF 3.0 data to viewers on a local network.
Maintained by Filip Mivalt, Bioelectronics Neurophysiology and Engineering Laboratory (BNEL) at Mayo Clinic.