Hypnogram#

Tools for analyzing hypnograms such as number of cycles, hypnogram time etc.

Plotting (plot_hypnogram(), score_night(..., plot=True)) needs matplotlib, an optional dependency: pip install "brainmaze-eeg[plot]". Everything else in this module works without it.

Input conventions (since 3.0.0):

A hypnogram is a pandas.DataFrame with one row per scored epoch and the columns annotation (sleep-state label), start and end, optionally duration.

  • start/end are either timezone-aware datetimes (datetime.datetime, pandas.Timestamp or a datetime64[ns, tz] column) or numeric POSIX timestamps in seconds. Naive datetimes raise TypeError (elapsed time across a DST change would be wrong; localize them first). Numeric values whose magnitude exceeds 1e11 (the year 5138 in seconds) raise ValueError, because they are almost certainly milliseconds or finer. Numbers are not checked further: relative times in milliseconds (small numbers) without a duration column pass as seconds, so pass seconds.

  • duration, if present, is numeric seconds or a timedelta column (e.g. df.end - df.start) and must equal end - start (to 1 ms), otherwise ValueError is raised. This also catches millisecond timestamps with durations in seconds. Every function that validates the input (all functions that read start/end) uses end - start. get_time_by_key(), get_stage_times(), get_stage_times_dataset() and valid_dataset_index_by_duration() need only annotation and duration and sum the duration column as given (not validated; a timedelta column gives a pandas.Timedelta).

  • Rows may come in any order; functions sort by start (stable) and never modify the caller’s frame.

  • Labels: 'WAKE' (the label written by brainmaze_utils.annotations.load_NSRR) is an alias of 'AWAKE' everywhere in this module, in the data and in tag arguments. Functions that return a hypnogram (the fill_*/correct_* functions and do_median_filtration()) return it as 'AWAKE'.

  • Arousals are events, not states: rows labelled 'Arousal' (the label written by brainmaze_utils) or 'Arrousal' (the legacy spelling of older data) are removed before scoring and never relabelled.

  • After arousals are removed, epochs must not overlap (ValueError otherwise) in the scoring functions (get_fell_asleep_time(), get_awakening_time(), get_rem_latency(), score_night()) and in get_hypnogram_datarate(), is_sleep_complete(), and in get_number_of_awakenings() and get_transition_counts() when the frame has start/end columns.

  • The scoring functions first merge touching epochs with the same label into bouts (one row per continuous period), so a hypnogram tiled into 30-s epochs and the same hypnogram with merged rows give the same results. Their time windows (sleep onset, awakening) count only the part of each bout that lies inside the window.

  • Unscored time is (a) epochs labelled 'UNKNOWN' or any other label that is neither a sleep stage (N1, N2, N3, REM) nor the awake tag, and (b) time gaps between epochs (no rows) longer than max_gap_s (default 1 s; this includes the time of a removed 'Arousal' row that was tiled in as an epoch). The scoring functions (get_fell_asleep_time(), get_awakening_time(), get_rem_latency(), score_night()) compute each result with all unscored time scored as AWAKE, and again as each sleep stage (N1, N2, N3, REM). If all agree, the result does not depend on the unscored time and is returned without a warning. If they differ, the result is ambiguous and is never returned silently: with on_unscored='warn' (default) a UserWarning names the unscored spans and the alternative results, and NaT/NaN is returned for that result only; on_unscored='raise' raises ValueError. New in 3.0.0.

    Limitations: each scoring relabels all unscored time with one state. A result that changes only when different unscored spans get different states, or when the state changes in the middle of a span, is not detected. Gaps up to max_gap_s are ignored (counted neither as awake nor as sleep) and may still change a result; pass max_gap_s=0 to check every gap. Time before the first and after the last epoch is outside the recording, not unscored. Overlaps are stricter than gaps: epochs that overlap by more than 1e-6 s raise ValueError (epochs overlap), so a hypnogram whose start times jitter by a sample while end = start + 30 must be fixed (e.g. end = the next start) before scoring. With unscored time each scoring function runs once per scoring (five times); a recording with tens of thousands of bouts and no qualifying awakening can then take a minute or more in get_awakening_time().

  • Unscored time in practice: expect NaN. Because every unscored span may be any state, real nights with unscored time often get NaN (and a warning) for the REM latencies, n_awakenings and n_complete_sleep_cycles, while the sleep onset, the awakening and the stage times usually stay computed. Typical causes:

    • a leading unscored segment before lights-off (e.g. NSRR Unscored epochs at the start of the recording): scored as REM it would be the first REM, so both REM latencies are ambiguous. Any unscored time before the first REM does the same.

    • on a night without REM, unscored time anywhere (even after the awakening): scored as REM it would create a REM, so the latencies are ambiguous.

    • an unscored epoch or a gap longer than max_gap_s inside the sleep: it may be an awakening (n_awakenings) or a REM bout (n_complete_sleep_cycles).

    Remedies (they bring every field of the demo night back): (1) drop leading and trailing unscored rows, because time outside the recording is not unscored (here: the REM latencies come back); (2) relabel spans whose state is known, e.g. df.loc[df.annotation == 'UNKNOWN', 'annotation'] = 'AWAKE' if they are known to be wake; (3) raise max_gap_s for short gaps known to be harmless (e.g. max_gap_s=120 for epochs dropped by a loader). Only relabel what you know: a relabelled span gives the same numbers as if it had been scored that way.

  • NSRR data: brainmaze_utils.annotations.load_NSRR (brainmaze-utils 3.0.0) raises KeyError on the NSRR stages 'Unscored|9' and 'Movement|6'. Map both to 'UNKNOWN' (unscored) before scoring; do not drop the rows (dropping leaves a gap, which is treated the same way, but only above max_gap_s). A fix in the loader is a brainmaze-utils follow-up.

brainmaze_eeg.hypnogram.AROUSAL_TAGS = ('Arousal', 'Arrousal')#

'Arousal' (brainmaze-utils) and the legacy spelling 'Arrousal'.

Type:

Labels treated as arousal events

brainmaze_eeg.hypnogram.LABEL_ALIASES = {'WAKE': 'AWAKE'}#

'WAKE' (written by brainmaze_utils.annotations.load_NSRR) means 'AWAKE'.

Type:

Alternative labels and the label they stand for

brainmaze_eeg.hypnogram.correct_hypnogram(df, time_threshold=60)#

Apply a series of corrections to hypnogram data.

This function applies multiple correction steps: fills gaps between same states, fills wake-REM gaps, fills non-REM gaps, fills sleep gaps, and corrects REM annotations.

Parameters:
  • df (pd.DataFrame) – Hypnogram dataframe with ‘annotation’, ‘start’, ‘end’, and ‘duration’ columns.

  • time_threshold (int, optional) – Time threshold in seconds for gap filling. Default is 60.

Returns:

Corrected hypnogram dataframe.

Return type:

pd.DataFrame

Notes

Note

Changed in 3.0.0: Rows are processed in start order (input may be unsorted), datetime input works (comparing a timedelta with a number of seconds raised TypeError), the input is validated (module docstring) and duration is returned as end - start in seconds.

brainmaze_eeg.hypnogram.correct_rem(df, time_threshold=300, initial_state='AWAKE')#

Correct REM annotations that occur shortly after prolonged awake periods.

A REM epoch that starts at most 60 s after an AWAKE epoch, when the AWAKE epochs right before it last at least time_threshold in total, is relabelled AWAKE.

Parameters:
  • df (pd.DataFrame) – Hypnogram dataframe with ‘annotation’, ‘start’, ‘end’, and ‘duration’ columns.

  • time_threshold (int, optional) – Minimum sustained awake duration in seconds. Default is 300 (5 minutes).

  • initial_state (str, optional) – Initial assumed state. Default is ‘AWAKE’.

Returns:

Hypnogram with corrected REM annotations.

Return type:

pd.DataFrame

Notes

Note

Changed in 3.0.0: Rows are processed in start order (input may be unsorted), datetime input works (comparing a timedelta with a number of seconds raised TypeError), the input is validated (module docstring) and duration is returned as end - start in seconds.

brainmaze_eeg.hypnogram.do_median_filtration(df)#

Apply median filtering to remove isolated 30-second stage annotations.

An epoch of 30 s whose neighbours (in start order, after earlier replacements) have the same label takes that label. Arousal rows ('Arousal'/'Arrousal') are events, not stages: they are returned unchanged and are skipped when looking for neighbours.

Parameters:

df (pd.DataFrame) – Hypnogram dataframe with ‘annotation’ and ‘start’/’end’ (or ‘duration’) columns.

Returns:

Filtered copy of the hypnogram, 'WAKE' returned as 'AWAKE'. The input is not modified.

Return type:

pd.DataFrame

Notes

Note

Changed in 3.0.0: It assigned through chained indexing (df.iloc[k]['annotation'] = ...), which does not write into the frame, so it returned the input unchanged. It now returns a filtered copy, in start order, in which arousal rows are unchanged and are not neighbours (a legacy 30-s 'Arrousal' epoch between two N2 epochs stays an arousal).

brainmaze_eeg.hypnogram.fill_nonrem_voids(df, time_threshold=300, initial_state='AWAKE')#

Fill gaps between non-REM sleep stages (N2, N3) with ‘N’ annotations.

Parameters:
  • df (pd.DataFrame) – Hypnogram dataframe with ‘annotation’, ‘start’, ‘end’, and ‘duration’ columns.

  • time_threshold (int, optional) – Maximum gap duration in seconds to fill. Default is 300 (5 minutes).

  • initial_state (str, optional) – Initial assumed state. Default is ‘AWAKE’.

Returns:

Hypnogram with filled non-REM gaps.

Return type:

pd.DataFrame

Notes

Note

Changed in 3.0.0: Rows are processed in start order (input may be unsorted), datetime input works (comparing a timedelta with a number of seconds raised TypeError), the input is validated (module docstring) and duration is returned as end - start in seconds. The inserted epoch’s duration was the length of the following epoch instead of the gap.

brainmaze_eeg.hypnogram.fill_same_voids(df, time_threshold=300, initial_state='AWAKE')#

Fill temporal gaps between consecutive same-state annotations.

Parameters:
  • df (pd.DataFrame) – Hypnogram dataframe with ‘annotation’, ‘start’, ‘end’, and ‘duration’ columns.

  • time_threshold (int, optional) – Maximum gap duration in seconds to fill. Default is 300 (5 minutes).

  • initial_state (str, optional) – Initial assumed state. Default is ‘AWAKE’.

Returns:

Hypnogram with filled gaps.

Return type:

pd.DataFrame

Notes

Note

Changed in 3.0.0: Rows are processed in start order (input may be unsorted), datetime input works (comparing a timedelta with a number of seconds raised TypeError), the input is validated (module docstring) and duration is returned as end - start in seconds.

brainmaze_eeg.hypnogram.fill_sleep_voids(df, time_threshold=300, initial_state='AWAKE')#

Fill gaps between any sleep states (non-AWAKE) with ‘SLP’ annotations.

Parameters:
  • df (pd.DataFrame) – Hypnogram dataframe with ‘annotation’, ‘start’, ‘end’, and ‘duration’ columns.

  • time_threshold (int, optional) – Maximum gap duration in seconds to fill. Default is 300 (5 minutes).

  • initial_state (str, optional) – Initial assumed state. Default is ‘AWAKE’.

Returns:

Hypnogram with filled sleep gaps.

Return type:

pd.DataFrame

Notes

Note

Changed in 3.0.0: Rows are processed in start order (input may be unsorted), datetime input works (comparing a timedelta with a number of seconds raised TypeError), the input is validated (module docstring) and duration is returned as end - start in seconds. The inserted epoch’s duration was the length of the following epoch instead of the gap.

brainmaze_eeg.hypnogram.fill_wakerem_voids(df, time_threshold=300, initial_state='AWAKE')#

Fill gaps between AWAKE and REM states with AWAKE annotations.

Parameters:
  • df (pd.DataFrame) – Hypnogram dataframe with ‘annotation’, ‘start’, ‘end’, and ‘duration’ columns.

  • time_threshold (int, optional) – Maximum gap duration in seconds to fill. Default is 300 (5 minutes).

  • initial_state (str, optional) – Initial assumed state. Default is ‘AWAKE’.

Returns:

Hypnogram with filled wake-REM gaps.

Return type:

pd.DataFrame

Notes

Note

Changed in 3.0.0: Rows are processed in start order (input may be unsorted), datetime input works (comparing a timedelta with a number of seconds raised TypeError), the input is validated (module docstring) and duration is returned as end - start in seconds. The inserted epoch’s duration was the length of the following epoch instead of the gap.

brainmaze_eeg.hypnogram.get_awakening_time(df, t_awake_threshold=90, t_sleep_threshold=10, awake_tag='AWAKE', sleep_cycle_tags=['REM', 'N1', 'N2', 'N3'], after=None, on_unscored='warn', max_gap_s=1.0)#

Determine when the subject woke up based on hypnogram data.

Touching epochs with the same label are merged into bouts first (module docstring).

  • Candidates are the awake_tag bouts that follow a sleep_cycle_tags bout, either directly or through 'N1' bouts only (N2 -> N1 -> AWAKE is a candidate even if 'N1' is not in sleep_cycle_tags), and start at or after after if given.

  • Sustained wake: a candidate qualifies if, in the window [candidate start, candidate start + t_awake_threshold + t_sleep_threshold), at least t_awake_threshold minutes are awake_tag and at most t_sleep_threshold minutes are sleep (N1, N2, N3, REM and sleep_cycle_tags). Only the part of each bout inside the window counts.

  • The awakening is the start of the first qualifying candidate.

  • If none qualifies: when the hypnogram ends with awake_tag, the start of the last candidate (the final transition from sleep to wake; ValueError if there is none); otherwise the end of the recording (the subject was still asleep when it ended).

  • Unscored time (module docstring) is checked: the awakening is returned only if it is the same with all unscored time scored as AWAKE and as each sleep stage (on_unscored).

  • after=None searches the whole recording, so a sustained wake after a short nap before the night’s sleep is found first; score_night() passes the sleep onset.

What the time means. The primary rule (first sustained wake of t_awake_threshold minutes) is a proxy for the end of the night, roughly when the subject got up (lights on); sleep after it (e.g. a morning nap) is not part of the night. It is not the AASM final awakening, which is the end of the last sleep epoch before lights on: the two agree when the subject stays awake after the last sleep epoch. The fallback (no sustained wake, the night ends awake: the last sleep-to-wake transition) is the AASM definition.

Example (defaults): AWAKE 30 min, N2 60, REM 20, AWAKE 5, N2 120, N1 5, AWAKE 120 gives the start of the last AWAKE bout (at 240 min): the brief AWAKE at 110 min holds 95 min of sleep in its window.

Parameters:
  • df (pd.DataFrame) – Hypnogram with annotation, start, end (optionally duration).

  • t_awake_threshold (int, optional) – Minimum sustained awake time in minutes to be considered awakened. Default is 90.

  • t_sleep_threshold (int, optional) – Maximum sleep time in minutes allowed in the awakening window. Default is 10.

  • awake_tag (str, optional) – Tag for awake state. Default is ‘AWAKE’ ('WAKE' is the same state).

  • sleep_cycle_tags (list, optional) – List of tags indicating sleep states. Default is [‘REM’, ‘N1’, ‘N2’, ‘N3’].

  • after (datetime or float, optional) – Only consider awakenings starting at or after this time. It must be of the same kind as start (a timezone-aware datetime for datetime input, a number of seconds for numeric input), otherwise TypeError is raised. score_night() passes the sleep onset. New in 3.0.0.

  • on_unscored ({'warn', 'raise'}, optional) – What to do when the awakening depends on how the unscored time is scored (e.g. ... N2, UNKNOWN, AWAKE: the wake may have begun anywhere in the unknown span). 'warn' (default) emits a UserWarning naming the unscored spans and the alternative times and returns NaT/NaN; 'raise' raises ValueError. New in 3.0.0.

  • max_gap_s (float, optional) – Time gaps between epochs longer than this (seconds) are unscored time; shorter gaps are ignored. Default 1.0. New in 3.0.0.

Returns:

Time of the awakening, in the type of the start column; NaT (datetime input) or NaN (numeric input) if it is ambiguous because of unscored epochs and on_unscored='warn'.

Return type:

datetime or float

Raises:
  • TypeError – If after is not of the same kind as start, or the input is invalid (module docstring).

  • ValueError – If the hypnogram is empty or invalid, or it ends awake without a candidate (with every scoring of the unscored time); or the awakening depends on the unscored time and on_unscored='raise'.

Notes

Note

Changed in 3.0.0: t_sleep_threshold (minutes) was compared with the sleep time in seconds, so 10 meant 10 s: one minute of N2 in the window rejected an awakening. The default sleep_cycle_tags now include 'N1', and a path through N1 bouts counts in any case: v1.0.0 missed an awakening reached through N1 (N2 -> N1 -> AWAKE, the usual way to wake up) and then returned an earlier brief awakening, which truncated the sleep period. The window counts only the part of each bout inside it and is t_awake_threshold + t_sleep_threshold long (v1.0.0 counted the t_awake_threshold window’s epochs in full). A first epoch in a sleep stage is no longer a candidate (it returned the start of that sleep epoch), 'Arousal' rows are removed like 'Arrousal', 'WAKE' is read as 'AWAKE', epochs are sorted and merged into bouts, and ValueError replaces an IndexError when no transition exists. Numeric timestamps (seconds) work. A night whose awakening depends on unscored epochs or gaps (e.g. N2, UNKNOWN, AWAKE, or a night ending N2, UNKNOWN) used to return a silently shifted awakening (the transition before the unknown span, the end of the recording, or an error); it now warns and returns NaT/NaN (on_unscored, max_gap_s).

brainmaze_eeg.hypnogram.get_fell_asleep_time(df, t_sleep_check=60, t_awake_threshold=10, awake_tag='AWAKE', sleep_cycle_tags=['REM', 'N1', 'N2', 'N3'], on_unscored='warn', max_gap_s=1.0)#

Determine when the subject fell asleep based on hypnogram data.

Touching epochs with the same label are merged into bouts first (module docstring). Candidates are the first bout, if it is a sleep stage, and every sleep-stage bout that directly follows an awake_tag bout, in time order. The sleep onset is the first candidate for which less than t_awake_threshold minutes of awake_tag lie in the window [candidate start, candidate start + t_sleep_check). Only the part of each bout inside the window counts (a bout reaching past the window end is clipped to it). Unscored time (module docstring) is checked: the onset is returned only if it is the same with all unscored time scored as AWAKE and as each sleep stage.

Example (defaults): AWAKE 30 min, N2 55, AWAKE 15, N2 200: the window after the N2 at 30 min holds 5 min of AWAKE (the AWAKE bout from 85 to 100 min is clipped at 90), so the onset is at 30 min.

Parameters:
  • df (pd.DataFrame) – Hypnogram with annotation, start, end (optionally duration).

  • t_sleep_check (int, optional) – Time window in minutes to check for sustained sleep. Default is 60.

  • t_awake_threshold (int, optional) – The awake time in minutes inside the window must be below this. Default is 10.

  • awake_tag (str, optional) – Tag for awake state. Default is ‘AWAKE’ ('WAKE' is the same state).

  • sleep_cycle_tags (list, optional) – List of tags indicating sleep states. Default is [‘REM’, ‘N1’, ‘N2’, ‘N3’].

  • on_unscored ({'warn', 'raise'}, optional) – What to do when the onset depends on how the unscored time (epochs labelled neither a sleep stage nor awake_tag, gaps longer than max_gap_s) is scored. 'warn' (default) emits a UserWarning naming the unscored spans and the alternative onsets and returns NaT/NaN; 'raise' raises ValueError. New in 3.0.0.

  • max_gap_s (float, optional) – Time gaps between epochs longer than this (seconds) are unscored time; shorter gaps are ignored. Default 1.0. New in 3.0.0.

Returns:

Start of the sleep-onset bout, in the type of the start column; NaT (datetime input) or NaN (numeric input) if the onset is ambiguous because of unscored epochs and on_unscored='warn'.

Return type:

datetime or float

Raises:

ValueError – If no candidate qualifies (with every scoring of the unscored time), the hypnogram is empty, or epochs overlap; or the onset depends on the unscored time and on_unscored='raise'.

Notes

Note

Changed in 3.0.0: If no candidate qualified, v1.0.0 returned the start of the recording, which looks like a valid onset; it now raises ValueError. Rows labelled 'Arousal' are removed like 'Arrousal' (only the latter was, so an 'Arousal' epoch between AWAKE and sleep hid the onset). Epochs are sorted and merged into bouts, so tiled and merged hypnograms agree. The window counts only the part of each bout inside it (v1.0.0 counted every epoch that started inside it in full, so on merged bouts an AWAKE bout starting 5 min before the window end counted with its whole length). awake_tag is compared for equality (it was a substring test); 'WAKE' is read as 'AWAKE'. Numeric timestamps (seconds) work. Unscored epochs and gaps no longer shift the onset silently (on_unscored, max_gap_s).

brainmaze_eeg.hypnogram.get_hypnogram_datarate(df)#

Calculate the data rate of a hypnogram.

Parameters:

df (pd.DataFrame) – Hypnogram with annotation, start and end columns (optionally duration); see the module docstring for the accepted time formats.

Returns:

Scored time (sum of end - start of all non-arousal epochs) divided by the time span from the earliest start to the latest end. 1.0 means no gaps.

Return type:

float

Raises:

ValueError – If the hypnogram is empty, spans zero time, or epochs overlap.

Notes

Note

Changed in 3.0.0: The span used timedelta.seconds, which drops whole days: a fully scored 25 h hypnogram returned 25.0 instead of 1.0. Numeric timestamps (seconds) now work (they raised AttributeError), arousal rows are no longer counted as scored time, the input may be unsorted, and the input is validated (module docstring).

brainmaze_eeg.hypnogram.get_number_of_awakenings(df, awake_tag='AWAKE', n1_tag='N1', sleep_tags=['N2', 'N3', 'REM'])#

Count the number of awakenings after sleep onset.

The epochs are taken in time order. An awakening is counted at the first awake_tag epoch after a sleep_tags epoch; the next awakening needs another sleep_tags epoch first. With the default sleep_tags (N2, N3, REM) n1_tag epochs neither arm nor break the count: N2, N1, AWAKE is one awakening, and so is N2, AWAKE, N1, AWAKE (the N1/AWAKE alternation of drowsy wake is not counted again). This differs from get_fell_asleep_time() and get_awakening_time(), where N1 is sleep (as in AASM). Pass sleep_tags=['N1', 'N2', 'N3', 'REM'] to count every transition from any sleep stage, N1 included, into awake_tag. Other labels (e.g. 'UNKNOWN') between sleep and awake_tag do not break the count; score_night() checks them (on_unscored).

Parameters:
  • df (pd.DataFrame) – Hypnogram with an annotation column. With start/end columns the input is validated and sorted by start; without them the row order is used.

  • awake_tag (str, optional) – Tag for awake state. Default is ‘AWAKE’ ('WAKE' is the same state).

  • n1_tag (str, optional) – Tag for N1 sleep stage. Default is ‘N1’.

  • sleep_tags (list, optional) – List of tags for sleep states. Default is [‘N2’, ‘N3’, ‘REM’].

Returns:

Number of awakenings.

Return type:

int

Raises:
  • TypeError – With start/end columns: on naive datetimes or other invalid time columns (module docstring).

  • ValueError – With start/end columns: if epochs overlap (after removing arousals) or duration does not equal end - start.

Notes

Note

Changed in 3.0.0: With n1_tag in sleep_tags, AWAKE, N1 no longer counts an awakening at the N1 epoch (falling asleep is not an awakening). Sorted by start when available (it used the row order and label-based indexing, which broke on a non-default index), and 'WAKE' is read as 'AWAKE'. Arousal rows (both spellings) are ignored; this does not change the count (an arousal never reset the state machine between sleep and AWAKE). With start/end columns the input is now validated (v1.0.0 ignored the times): naive datetimes raise TypeError; overlapping states (e.g. a frame that also holds channel annotations) and a duration that differs from end - start raise ValueError.

brainmaze_eeg.hypnogram.get_number_of_sleep_stages(df, tags='REM', delay=30)#

Count the bouts (periods) of a sleep stage, ignoring bouts that start too soon after the previous counted one.

  1. Epochs whose annotation is in tags are sorted by start. Epochs that touch or overlap in time are merged into one bout (several tags are treated as one class: touching epochs of any of them form one bout). A 40-min REM period counts as one bout whether it is one row or eighty 30-s epochs.

  2. The first bout is counted. A later bout is counted if its start is at least delay minutes after the end of the last counted bout (bouts that were not counted do not move this reference).

Example (delay 30 min): REM 40 min, N2 20 min, REM 40 min gives 1 (the second REM bout starts 20 min after the first ends); with N2 30 min instead it gives 2.

Parameters:
  • df (pd.DataFrame) – Hypnogram with annotation, start, end (optionally duration); see the module docstring for the accepted time formats. Rows may be unsorted.

  • tags (str or list, optional) – Sleep stage tag(s) to count. Default is ‘REM’.

  • delay (float, optional) – Minimum time in minutes (>= 0) between the end of the last counted bout and the start of the next counted one. Default is 30. Any value is allowed, including >= 1440 (one day).

Returns:

Number of counted bouts (0 if no epoch has one of the tags).

Return type:

int

Notes

Note

Changed in 3.0.0: v1.0.0 always raised under pandas >= 2 (it used DataFrame.append). Its results under pandas < 2 differ from 3.0.0 as follows:

  • It worked per epoch: the delay was measured from the end of the single epoch that started the last counted occurrence, so on a hypnogram tiled into 30-s epochs one long period was counted several times (one 40-min REM period: 2; 90 min: 3; REM 40 / N2 20 / REM 40 min: 4). Now bouts are counted (1, 1 and 1). On input that is already merged into bouts (as score_night passed it) the result is unchanged.

  • With several distinct tags it required an epoch to equal all of them (none did) and then raised AttributeError; the same happened when no epoch matched. Now several tags are counted together and no match gives 0.

  • It compared timedelta.seconds, which drops whole days and wraps negatives: a gap of 1 day + 1 min counted as 1 min, delay >= 1440 wrapped (delay=1440 acted as 0, so every epoch counted), and unsorted or overlapping rows gave arbitrary counts. Now the rows are sorted, overlapping epochs join one bout, and gaps and delay are exact.

  • Numeric timestamps must be in seconds; values that look like milliseconds raise ValueError (they were silently read as seconds). Naive datetimes raise TypeError.

brainmaze_eeg.hypnogram.get_rem_latency(df, rem_tag='REM', awake_tag='AWAKE', on_unscored='warn', max_gap_s=1.0)#

Calculate REM sleep latency (time from falling asleep or last awake to first REM).

Parameters:
  • df (pd.DataFrame) – Hypnogram with annotation, start, end (optionally duration).

  • rem_tag (str, optional) – Tag for REM sleep state. Default is ‘REM’.

  • awake_tag (str, optional) – Tag for awake state. Default is ‘AWAKE’ ('WAKE' is the same state).

  • on_unscored ({'warn', 'raise'}, optional) –

    Each latency is returned only if it is the same with all unscored time (module docstring) scored as AWAKE and as each sleep stage; unscored time before the first REM may hide an earlier REM, an earlier or later sleep onset, or a later wake. If a latency differs, 'warn' (default) warns and gives NaT/NaN for it; 'raise' raises ValueError. New in 3.0.0.

    On real nights this is common: any unscored time before the first REM (even a leading unscored segment before lights-off), or, on a night without REM, unscored time anywhere, makes both latencies NaT/NaN, because that time may have been REM. To get the latencies, drop leading/trailing unscored rows, relabel the spans whose state is known, or raise max_gap_s for harmless gaps (module docstring, “Unscored time in practice”).

  • max_gap_s (float, optional) – Time gaps between epochs longer than this (seconds) are unscored time; shorter gaps are ignored. Default 1.0. New in 3.0.0.

Returns:

'fall_asleep': start of the first REM epoch minus the sleep onset (get_fell_asleep_time() with its defaults). 'last_awake': start of the first REM epoch minus the end of the last awake_tag epoch before it (the sleep onset if there is none). Timedeltas for datetime input, seconds (float) for numeric input. A latency is negative when the first REM precedes the reference (e.g. REM before the sustained sleep onset).

Return type:

dict

Raises:

ValueError – If there is no rem_tag epoch or no sleep onset (with every scoring of the unscored time); or a latency depends on the unscored time and on_unscored='raise'.

Notes

Note

Changed in 3.0.0: Sorted, validated input; arousal rows (both spellings) removed; 'WAKE' read as 'AWAKE'; ValueError instead of IndexError without REM. On a fully scored night the latencies are defined as before, from the sleep onset of get_fell_asleep_time() (whose onset window changed in 3.0.0, see there). With unscored time (module docstring) a latency that depends on it is no longer returned: it is NaT/NaN with a warning (on_unscored='warn'), or ValueError is raised (on_unscored='raise'); see on_unscored for how often this happens.

brainmaze_eeg.hypnogram.get_stage_times(df, keys)#

Get total times for multiple sleep stages.

Parameters:
  • df (pd.DataFrame) – Hypnogram dataframe with ‘annotation’ and ‘duration’ columns.

  • keys (list) – List of sleep stage tags.

Returns:

List of total durations for each stage in seconds.

Return type:

list

brainmaze_eeg.hypnogram.get_stage_times_dataset(hypnograms: list, keys, verbose=True)#

Get stage times for a dataset of hypnograms.

Parameters:
  • hypnograms (list) – List of hypnogram dataframes.

  • keys (list) – List of sleep stage tags.

  • verbose (bool, optional) – If True, show progress bar. Default is True.

Returns:

Dataframe with stage times for each hypnogram.

Return type:

pd.DataFrame

brainmaze_eeg.hypnogram.get_time_by_key(df, key)#

Calculate total time for a specific sleep stage or combination of stages.

Parameters:
  • df (pd.DataFrame) – Hypnogram dataframe with ‘annotation’ and ‘duration’ columns.

  • key (str or list) – Sleep stage tag(s) to sum durations for.

Returns:

Total duration in the unit of the duration column (seconds by convention; a timedelta column gives a pandas.Timedelta). The duration column is summed as given, not checked against start/end. 'WAKE' and 'AWAKE' are the same state, in df and in key.

Return type:

float

brainmaze_eeg.hypnogram.get_transition_counts(hyp, states=['AWAKE', 'N1', 'N2', 'N3', 'REM'])#

Count transitions between sleep stages.

Consecutive rows (in start order) form a transition; a row whose label is not in states breaks the chain (no transition is counted into or out of it). Time gaps between rows are not considered. On a hypnogram tiled into epochs the diagonal holds the epoch-to-epoch persistence; on merged bouts it is (nearly) zero.

Parameters:
  • hyp (pd.DataFrame) – Hypnogram with an annotation column. With start/end columns it is validated and sorted by start; without them the row order is used.

  • states (list, optional) – List of sleep states to include. Default is [‘AWAKE’, ‘N1’, ‘N2’, ‘N3’, ‘REM’].

Returns:

Matrix of transition counts where [i,j] is count of transitions from state i to state j.

Return type:

np.ndarray

Raises:
  • TypeError – With start/end columns: on naive datetimes or other invalid time columns (module docstring).

  • ValueError – With start/end columns: if epochs overlap (after removing arousals, unless they are listed in states) or duration does not equal end - start; or if states lists the same state twice.

Notes

Note

Changed in 3.0.0: Rows are sorted by start (it used label-based indexing on the row order, which broke on a non-default index), and arousal rows ('Arousal'/'Arrousal') are ignored unless listed in states (an arousal between two epochs hid their transition). 'WAKE' is read as 'AWAKE' in hyp and in states (the matrix keeps the order and names of states). With start/end columns the input is now validated (v1.0.0 ignored the times): naive datetimes raise TypeError; overlapping states (e.g. a frame that also holds channel annotations) and a duration that differs from end - start raise ValueError.

brainmaze_eeg.hypnogram.get_transition_matrix(hyp, states=['AWAKE', 'N1', 'N2', 'N3', 'REM'])#

Calculate transition probability matrix between sleep stages.

Parameters:
  • hyp (pd.DataFrame) – Hypnogram dataframe with ‘annotation’ column.

  • states (list, optional) – List of sleep states to include. Default is [‘AWAKE’, ‘N1’, ‘N2’, ‘N3’, ‘REM’].

Returns:

Normalized transition probability matrix.

Return type:

np.ndarray

brainmaze_eeg.hypnogram.get_transition_matrix_dataset(hypnograms, states=['AWAKE', 'N1', 'N2', 'N3', 'REM'])#

Calculate mean and std of transition matrices across a dataset of hypnograms.

Parameters:
  • hypnograms (list) – List of hypnogram dataframes.

  • states (list, optional) – List of sleep states to include. Default is [‘AWAKE’, ‘N1’, ‘N2’, ‘N3’, ‘REM’].

Returns:

Mean transition matrix and standard deviation matrix (rows of states that never occur in a hypnogram are left out of the mean of that row).

Return type:

tuple

brainmaze_eeg.hypnogram.is_sleep_complete(df, awake_tag='AWAKE')#

Check if the sleep session is complete (starts and ends with awake state).

Parameters:
  • df (pd.DataFrame) – Hypnogram with annotation, start, end (optionally duration).

  • awake_tag (str, optional) – Tag for awake state. Default is ‘AWAKE’ ('WAKE' is the same state).

Returns:

True if the earliest and the latest epoch (arousals excluded) are awake_tag.

Return type:

bool

Raises:
  • TypeError – On naive datetimes or other invalid time columns (module docstring).

  • ValueError – If the hypnogram is empty, epochs overlap (after removing arousals), or duration does not equal end - start.

Notes

Note

Changed in 3.0.0: Uses the earliest and latest epoch by start (it used the first and last rows), ignores arousal rows, reads 'WAKE' as 'AWAKE', and raises ValueError on an empty hypnogram. The input is now validated like everywhere else in the module (v1.0.0 ignored the times): naive datetimes raise TypeError; overlapping states (e.g. a frame that also holds channel annotations) and a duration that differs from end - start raise ValueError.

brainmaze_eeg.hypnogram.plot_hypnogram(orig_df, hypnogram_values=None, hypnogram_colors=None, fontsize=12, fig=None, night_start=22)#

Creates a Matplotlib figure of the hypnogram from the annotations. Time must be in a time-zone aware format.

Parameters:
  • orig_df (pd.DataFrame) – Hypnogram with annotation and timezone-aware start/end columns, in any row order (it is sorted by start). Arousal rows ('Arousal', or the legacy 'Arrousal') are drawn as events, not as states. The frame is not modified.

  • hypnogram_values (dict) – dict of a y-axis values for each hypnogram state. The default covers 'AWAKE' ('WAKE' is drawn as 'AWAKE' unless the dict has a 'WAKE' key), 'Arousal', 'SLP' and 'N' (inserted by fill_sleep_voids() and fill_nonrem_voids()), 'REM', 'N1', 'N2', 'N3' and 'UNKNOWN' (CyberPSG’s unscored label, drawn at 0).

  • hypnogram_colors (dict) – dict of color hex codes for each hypnogram state (default: the same labels)

  • fontsize (int) – Fontsize

  • fig (figure) – Already existing figure object.

  • night_start (int) – Hour at which the night begins. Nights [night_start, night_start + 12 h] are shaded for every calendar date in the plotted range, on the wall clock of the timezone of the first start (UTC data is shaded at night_start UTC: convert it to the recording site’s timezone first, e.g. df.start.dt.tz_convert(...)).

Raises:
  • ImportError – If matplotlib is not installed (optional extra brainmaze-eeg[plot]).

  • TypeError – If start/end are not timezone-aware datetimes.

  • ValueError – If the hypnogram has no sleep-state epochs, or a label has no y value or colour.

Notes

Note

Changed in 3.0.0: Works with pandas 2 (it used DataFrame.append, removed in pandas 2, and called the datetime module instead of datetime.datetime, so it always raised). It no longer adds state_id/state_color columns to the input frame, sorts the rows, and handles a one-epoch hypnogram. Night shading is drawn per calendar date over the plotted range (it was anchored to each day group’s first epoch, so a recording starting after midnight, or a state lasting over several days, lost its night shading); a day column is no longer used. Arousals are recognised as 'Arousal' (the brainmaze-utils label, which was drawn as a sleep state) and 'Arrousal' (which raised KeyError with the default maps). The default maps also cover 'N', 'UNKNOWN' and 'WAKE', so the output of correct_hypnogram() and of the brainmaze-utils loaders can be plotted. Other labels missing from the maps raise ValueError naming them.

brainmaze_eeg.hypnogram.print_sleep_score(score)#

Print a formatted summary of sleep metrics.

Parameters:

score (dict) – Sleep score dictionary from score_night function.

Notes

Note

Changed in 3.0.0: Works with the floats that score_night() returns (it called .seconds on them). Durations keep whole days and negative latencies keep their sign. The relative times are fractions of the sleep period (sleep onset to awakening), which is now labelled as such (it was printed as “Total Sleep Time”).

brainmaze_eeg.hypnogram.score_night(df, plot=False, on_unscored='warn', max_gap_s=1.0)#

Compute comprehensive sleep metrics for a night’s hypnogram.

The hypnogram is validated, sorted, arousal rows ('Arousal'/'Arrousal') are removed, 'WAKE' is read as 'AWAKE', and touching epochs with the same label are merged into bouts (module docstring). Then, with the defaults of the individual functions:

  • fell_asleep_time: get_fell_asleep_time().

  • awakening_time: get_awakening_time() with after=fell_asleep_time.

  • The sleep period is made of the bouts starting at or after fell_asleep_time and before awakening_time; the counts and stage times below are taken over it.

  • Unscored time ('UNKNOWN' or any label that is neither a sleep stage nor AWAKE, and gaps longer than max_gap_s; module docstring): the whole score is computed with all unscored time scored as AWAKE, and as each of N1, N2, N3 and REM (the awakening is searched after the onset found with the same scoring). A field that is the same in all of them is returned; a field that differs is ambiguous and is NaT/NaN with on_unscored='warn' (default; one UserWarning names the fields, the unscored spans and the alternatives), or ValueError is raised with on_unscored='raise'. Other fields stay computed. For example an UNKNOWN epoch inside the sleep usually leaves the onset, the awakening and the stage times unchanged but makes n_awakenings ambiguous (it may have been an awakening), and ... N2, UNKNOWN, AWAKE makes awakening_time ambiguous.

  • Expect NaN on real nights with unscored time. Both REM latencies, n_awakenings and n_complete_sleep_cycles will often be NaN: a leading unscored segment before lights-off (or any unscored time before the first REM) could be the first REM; on a night without REM, unscored time even after the awakening could be a REM; an unscored epoch or gap inside the sleep could be an awakening or a REM bout. Remedies: drop leading/trailing unscored rows (time outside the recording is not unscored), relabel spans whose state is known, and raise max_gap_s for short harmless gaps. On the demo night (5 min UNKNOWN at the start, 12.5 min UNKNOWN in the sleep, one 2-min gap) the four fields are NaN; trimming the leading segment brings back the latencies, and relabelling the inner UNKNOWN plus max_gap_s=120 brings back all fields. See the module docstring, “Unscored time in practice”.

  • The stage times count the scored epochs only: unscored time inside the sleep period is in none of them (so they do not depend on how it is scored; compare get_hypnogram_datarate()). sleep_complete reads the first and last epoch as scored.

Parameters:
  • df (pd.DataFrame) – Hypnogram with annotation, start, end (optionally duration): timezone-aware datetimes or numeric POSIX timestamps in seconds.

  • plot (bool, optional) – If True, plot the hypnogram with sleep onset and awakening markers. Default is False. Needs matplotlib (optional extra brainmaze-eeg[plot]) and datetime input.

  • on_unscored ({'warn', 'raise'}, optional) – See above. New in 3.0.0.

  • max_gap_s (float, optional) – Time gaps between epochs longer than this (seconds) are unscored time; shorter gaps are ignored. Default 1.0. New in 3.0.0.

Returns:

  • sleep_complete: the earliest and latest epochs are AWAKE (is_sleep_complete())

  • fell_asleep_time: time of sleep onset (type of start)

  • rem_latency_fell_asleep: first REM start minus sleep onset, in seconds (float; negative if REM came first; NaN if there is no REM)

  • rem_latency_last_awake: first REM start minus the end of the last AWAKE before it, in seconds (float; NaN if there is no REM); see get_rem_latency()

  • awakening_time: time of final awakening (type of start)

  • n_complete_sleep_cycles: REM bouts in the sleep period, get_number_of_sleep_stages() with tags='REM', delay=30

  • n_awakenings: get_number_of_awakenings() (defaults) over the sleep period: awakenings after N2, N3 or REM; N1/AWAKE alternation is not counted again (unlike the onset and the awakening, where N1 is sleep)

  • n1_sleep_time, n2_sleep_time, n3_sleep_time, rem_sleep_time, awake_sleep_time: scored time in each stage within the sleep period, in seconds

An ambiguous field (see above) is NaT (datetime input) or NaN (numeric input) for fell_asleep_time/awakening_time, and NaN (a float) for the others.

Return type:

dict

Raises:

ValueError – If the hypnogram is empty, invalid (module docstring), or no sleep onset or awakening can be determined (with every scoring of the unscored time); or a field is ambiguous and on_unscored='raise'.

Notes

Note

Changed in 3.0.0: v1.0.0 raised with brainmaze-utils 3 (its merge_annotations accepts only numeric timestamps, while the rest of the function needed datetimes). Now datetime and numeric input both work. Numbers that v1.0.0 would have got wrong are fixed: REM latencies used timedelta.seconds (a latency of -23 min became 85020 s; now negative values stay negative), the awakening ignored t_sleep_threshold units (see get_awakening_time()), it missed a final awakening reached through N1 (N2 -> N1 -> AWAKE) and then cut the sleep period at an earlier brief awakening, and it could be found before the sleep onset (now only after it). The onset and awakening windows count only the part of each bout inside them. 'Arousal' rows were not removed, 'WAKE' (NSRR) was not recognised as 'AWAKE', and a night without REM raised IndexError (now the latencies are NaN). Results that depend on unscored epochs or gaps are no longer returned silently (see on_unscored).

brainmaze_eeg.hypnogram.transition_matrix_to_change_matrix(m)#

Convert transition matrix to change matrix (excluding diagonal/same-state transitions).

Parameters:

m (np.ndarray) – Transition probability matrix.

Returns:

Change matrix with diagonal set to zero and rows renormalized.

Return type:

np.ndarray

brainmaze_eeg.hypnogram.valid_dataset_index_by_duration(hypnograms: list, filt_dict: dict)#

Filter hypnograms by minimum duration requirements for specific stages.

Parameters:
  • hypnograms (list) – List of hypnogram dataframes.

  • filt_dict (dict) – Dictionary mapping stage names to minimum total duration (unit of the duration column, seconds by convention). A stage that does not occur has duration 0, so a threshold of 0 accepts a hypnogram without that stage.

Returns:

Indices of hypnograms in which every listed stage lasts at least its threshold.

Return type:

list

Notes

Note

Changed in 3.0.0: v1.0.0 compared the total duration of the whole hypnogram (all stages) with each stage’s threshold, so e.g. {'REM': 3600} accepted any night longer than 1 h that contained any REM. v1.0.0 also rejected a hypnogram in which a listed stage does not occur, whatever its threshold; now an absent stage has duration 0, so it passes a threshold of 0 (and fails any positive one).