Hypnogram#
Tools for analyzing hypnograms such as number of cycles, hypnogram time etc.
Plotting (plot_hypnogram(), score_night(..., plot=True)) needs matplotlib, an
optional dependency: pip install "brainmaze-eeg[plot]". Everything else in this
module works without it.
Input conventions (since 3.0.0):
A hypnogram is a pandas.DataFrame with one row per scored epoch and the columns
annotation (sleep-state label), start and end, optionally duration.
start/endare either timezone-aware datetimes (datetime.datetime,pandas.Timestampor adatetime64[ns, tz]column) or numeric POSIX timestamps in seconds. Naive datetimes raiseTypeError(elapsed time across a DST change would be wrong; localize them first). Numeric values whose magnitude exceeds1e11(the year 5138 in seconds) raiseValueError, because they are almost certainly milliseconds or finer. Numbers are not checked further: relative times in milliseconds (small numbers) without adurationcolumn pass as seconds, so pass seconds.duration, if present, is numeric seconds or a timedelta column (e.g.df.end - df.start) and must equalend - start(to 1 ms), otherwiseValueErroris raised. This also catches millisecond timestamps with durations in seconds. Every function that validates the input (all functions that readstart/end) usesend - start.get_time_by_key(),get_stage_times(),get_stage_times_dataset()andvalid_dataset_index_by_duration()need onlyannotationanddurationand sum thedurationcolumn as given (not validated; a timedelta column gives apandas.Timedelta).Rows may come in any order; functions sort by
start(stable) and never modify the caller’s frame.Labels:
'WAKE'(the label written bybrainmaze_utils.annotations.load_NSRR) is an alias of'AWAKE'everywhere in this module, in the data and in tag arguments. Functions that return a hypnogram (thefill_*/correct_*functions anddo_median_filtration()) return it as'AWAKE'.Arousals are events, not states: rows labelled
'Arousal'(the label written bybrainmaze_utils) or'Arrousal'(the legacy spelling of older data) are removed before scoring and never relabelled.After arousals are removed, epochs must not overlap (
ValueErrorotherwise) in the scoring functions (get_fell_asleep_time(),get_awakening_time(),get_rem_latency(),score_night()) and inget_hypnogram_datarate(),is_sleep_complete(), and inget_number_of_awakenings()andget_transition_counts()when the frame hasstart/endcolumns.The scoring functions first merge touching epochs with the same label into bouts (one row per continuous period), so a hypnogram tiled into 30-s epochs and the same hypnogram with merged rows give the same results. Their time windows (sleep onset, awakening) count only the part of each bout that lies inside the window.
Unscored time is (a) epochs labelled
'UNKNOWN'or any other label that is neither a sleep stage (N1, N2, N3, REM) nor the awake tag, and (b) time gaps between epochs (no rows) longer thanmax_gap_s(default 1 s; this includes the time of a removed'Arousal'row that was tiled in as an epoch). The scoring functions (get_fell_asleep_time(),get_awakening_time(),get_rem_latency(),score_night()) compute each result with all unscored time scored as AWAKE, and again as each sleep stage (N1, N2, N3, REM). If all agree, the result does not depend on the unscored time and is returned without a warning. If they differ, the result is ambiguous and is never returned silently: withon_unscored='warn'(default) aUserWarningnames the unscored spans and the alternative results, andNaT/NaNis returned for that result only;on_unscored='raise'raisesValueError. New in 3.0.0.Limitations: each scoring relabels all unscored time with one state. A result that changes only when different unscored spans get different states, or when the state changes in the middle of a span, is not detected. Gaps up to
max_gap_sare ignored (counted neither as awake nor as sleep) and may still change a result; passmax_gap_s=0to check every gap. Time before the first and after the last epoch is outside the recording, not unscored. Overlaps are stricter than gaps: epochs that overlap by more than 1e-6 s raiseValueError(epochs overlap), so a hypnogram whosestarttimes jitter by a sample whileend = start + 30must be fixed (e.g.end= the nextstart) before scoring. With unscored time each scoring function runs once per scoring (five times); a recording with tens of thousands of bouts and no qualifying awakening can then take a minute or more inget_awakening_time().Unscored time in practice: expect NaN. Because every unscored span may be any state, real nights with unscored time often get
NaN(and a warning) for the REM latencies,n_awakeningsandn_complete_sleep_cycles, while the sleep onset, the awakening and the stage times usually stay computed. Typical causes:a leading unscored segment before lights-off (e.g. NSRR
Unscoredepochs at the start of the recording): scored as REM it would be the first REM, so both REM latencies are ambiguous. Any unscored time before the first REM does the same.on a night without REM, unscored time anywhere (even after the awakening): scored as REM it would create a REM, so the latencies are ambiguous.
an unscored epoch or a gap longer than
max_gap_sinside the sleep: it may be an awakening (n_awakenings) or a REM bout (n_complete_sleep_cycles).
Remedies (they bring every field of the demo night back): (1) drop leading and trailing unscored rows, because time outside the recording is not unscored (here: the REM latencies come back); (2) relabel spans whose state is known, e.g.
df.loc[df.annotation == 'UNKNOWN', 'annotation'] = 'AWAKE'if they are known to be wake; (3) raisemax_gap_sfor short gaps known to be harmless (e.g.max_gap_s=120for epochs dropped by a loader). Only relabel what you know: a relabelled span gives the same numbers as if it had been scored that way.NSRR data:
brainmaze_utils.annotations.load_NSRR(brainmaze-utils 3.0.0) raisesKeyErroron the NSRR stages'Unscored|9'and'Movement|6'. Map both to'UNKNOWN'(unscored) before scoring; do not drop the rows (dropping leaves a gap, which is treated the same way, but only abovemax_gap_s). A fix in the loader is a brainmaze-utils follow-up.
- brainmaze_eeg.hypnogram.AROUSAL_TAGS = ('Arousal', 'Arrousal')#
'Arousal'(brainmaze-utils) and the legacy spelling'Arrousal'.- Type:
Labels treated as arousal events
- brainmaze_eeg.hypnogram.LABEL_ALIASES = {'WAKE': 'AWAKE'}#
'WAKE'(written bybrainmaze_utils.annotations.load_NSRR) means'AWAKE'.- Type:
Alternative labels and the label they stand for
- brainmaze_eeg.hypnogram.correct_hypnogram(df, time_threshold=60)#
Apply a series of corrections to hypnogram data.
This function applies multiple correction steps: fills gaps between same states, fills wake-REM gaps, fills non-REM gaps, fills sleep gaps, and corrects REM annotations.
- Parameters:
df (pd.DataFrame) – Hypnogram dataframe with ‘annotation’, ‘start’, ‘end’, and ‘duration’ columns.
time_threshold (int, optional) – Time threshold in seconds for gap filling. Default is 60.
- Returns:
Corrected hypnogram dataframe.
- Return type:
pd.DataFrame
Notes
Note
Changed in 3.0.0: Rows are processed in
startorder (input may be unsorted), datetime input works (comparing a timedelta with a number of seconds raisedTypeError), the input is validated (module docstring) anddurationis returned asend - startin seconds.
- brainmaze_eeg.hypnogram.correct_rem(df, time_threshold=300, initial_state='AWAKE')#
Correct REM annotations that occur shortly after prolonged awake periods.
A REM epoch that starts at most 60 s after an AWAKE epoch, when the AWAKE epochs right before it last at least
time_thresholdin total, is relabelled AWAKE.- Parameters:
df (pd.DataFrame) – Hypnogram dataframe with ‘annotation’, ‘start’, ‘end’, and ‘duration’ columns.
time_threshold (int, optional) – Minimum sustained awake duration in seconds. Default is 300 (5 minutes).
initial_state (str, optional) – Initial assumed state. Default is ‘AWAKE’.
- Returns:
Hypnogram with corrected REM annotations.
- Return type:
pd.DataFrame
Notes
Note
Changed in 3.0.0: Rows are processed in
startorder (input may be unsorted), datetime input works (comparing a timedelta with a number of seconds raisedTypeError), the input is validated (module docstring) anddurationis returned asend - startin seconds.
- brainmaze_eeg.hypnogram.do_median_filtration(df)#
Apply median filtering to remove isolated 30-second stage annotations.
An epoch of 30 s whose neighbours (in
startorder, after earlier replacements) have the same label takes that label. Arousal rows ('Arousal'/'Arrousal') are events, not stages: they are returned unchanged and are skipped when looking for neighbours.- Parameters:
df (pd.DataFrame) – Hypnogram dataframe with ‘annotation’ and ‘start’/’end’ (or ‘duration’) columns.
- Returns:
Filtered copy of the hypnogram,
'WAKE'returned as'AWAKE'. The input is not modified.- Return type:
pd.DataFrame
Notes
Note
Changed in 3.0.0: It assigned through chained indexing (
df.iloc[k]['annotation'] = ...), which does not write into the frame, so it returned the input unchanged. It now returns a filtered copy, instartorder, in which arousal rows are unchanged and are not neighbours (a legacy 30-s'Arrousal'epoch between two N2 epochs stays an arousal).
- brainmaze_eeg.hypnogram.fill_nonrem_voids(df, time_threshold=300, initial_state='AWAKE')#
Fill gaps between non-REM sleep stages (N2, N3) with ‘N’ annotations.
- Parameters:
df (pd.DataFrame) – Hypnogram dataframe with ‘annotation’, ‘start’, ‘end’, and ‘duration’ columns.
time_threshold (int, optional) – Maximum gap duration in seconds to fill. Default is 300 (5 minutes).
initial_state (str, optional) – Initial assumed state. Default is ‘AWAKE’.
- Returns:
Hypnogram with filled non-REM gaps.
- Return type:
pd.DataFrame
Notes
Note
Changed in 3.0.0: Rows are processed in
startorder (input may be unsorted), datetime input works (comparing a timedelta with a number of seconds raisedTypeError), the input is validated (module docstring) anddurationis returned asend - startin seconds. The inserted epoch’sdurationwas the length of the following epoch instead of the gap.
- brainmaze_eeg.hypnogram.fill_same_voids(df, time_threshold=300, initial_state='AWAKE')#
Fill temporal gaps between consecutive same-state annotations.
- Parameters:
df (pd.DataFrame) – Hypnogram dataframe with ‘annotation’, ‘start’, ‘end’, and ‘duration’ columns.
time_threshold (int, optional) – Maximum gap duration in seconds to fill. Default is 300 (5 minutes).
initial_state (str, optional) – Initial assumed state. Default is ‘AWAKE’.
- Returns:
Hypnogram with filled gaps.
- Return type:
pd.DataFrame
Notes
Note
Changed in 3.0.0: Rows are processed in
startorder (input may be unsorted), datetime input works (comparing a timedelta with a number of seconds raisedTypeError), the input is validated (module docstring) anddurationis returned asend - startin seconds.
- brainmaze_eeg.hypnogram.fill_sleep_voids(df, time_threshold=300, initial_state='AWAKE')#
Fill gaps between any sleep states (non-AWAKE) with ‘SLP’ annotations.
- Parameters:
df (pd.DataFrame) – Hypnogram dataframe with ‘annotation’, ‘start’, ‘end’, and ‘duration’ columns.
time_threshold (int, optional) – Maximum gap duration in seconds to fill. Default is 300 (5 minutes).
initial_state (str, optional) – Initial assumed state. Default is ‘AWAKE’.
- Returns:
Hypnogram with filled sleep gaps.
- Return type:
pd.DataFrame
Notes
Note
Changed in 3.0.0: Rows are processed in
startorder (input may be unsorted), datetime input works (comparing a timedelta with a number of seconds raisedTypeError), the input is validated (module docstring) anddurationis returned asend - startin seconds. The inserted epoch’sdurationwas the length of the following epoch instead of the gap.
- brainmaze_eeg.hypnogram.fill_wakerem_voids(df, time_threshold=300, initial_state='AWAKE')#
Fill gaps between AWAKE and REM states with AWAKE annotations.
- Parameters:
df (pd.DataFrame) – Hypnogram dataframe with ‘annotation’, ‘start’, ‘end’, and ‘duration’ columns.
time_threshold (int, optional) – Maximum gap duration in seconds to fill. Default is 300 (5 minutes).
initial_state (str, optional) – Initial assumed state. Default is ‘AWAKE’.
- Returns:
Hypnogram with filled wake-REM gaps.
- Return type:
pd.DataFrame
Notes
Note
Changed in 3.0.0: Rows are processed in
startorder (input may be unsorted), datetime input works (comparing a timedelta with a number of seconds raisedTypeError), the input is validated (module docstring) anddurationis returned asend - startin seconds. The inserted epoch’sdurationwas the length of the following epoch instead of the gap.
- brainmaze_eeg.hypnogram.get_awakening_time(df, t_awake_threshold=90, t_sleep_threshold=10, awake_tag='AWAKE', sleep_cycle_tags=['REM', 'N1', 'N2', 'N3'], after=None, on_unscored='warn', max_gap_s=1.0)#
Determine when the subject woke up based on hypnogram data.
Touching epochs with the same label are merged into bouts first (module docstring).
Candidates are the
awake_tagbouts that follow asleep_cycle_tagsbout, either directly or through'N1'bouts only (N2 -> N1 -> AWAKE is a candidate even if'N1'is not insleep_cycle_tags), and start at or afterafterif given.Sustained wake: a candidate qualifies if, in the window
[candidate start, candidate start + t_awake_threshold + t_sleep_threshold), at leastt_awake_thresholdminutes areawake_tagand at mostt_sleep_thresholdminutes are sleep (N1, N2, N3, REM andsleep_cycle_tags). Only the part of each bout inside the window counts.The awakening is the start of the first qualifying candidate.
If none qualifies: when the hypnogram ends with
awake_tag, the start of the last candidate (the final transition from sleep to wake;ValueErrorif there is none); otherwise the end of the recording (the subject was still asleep when it ended).Unscored time (module docstring) is checked: the awakening is returned only if it is the same with all unscored time scored as AWAKE and as each sleep stage (
on_unscored).after=Nonesearches the whole recording, so a sustained wake after a short nap before the night’s sleep is found first;score_night()passes the sleep onset.
What the time means. The primary rule (first sustained wake of
t_awake_thresholdminutes) is a proxy for the end of the night, roughly when the subject got up (lights on); sleep after it (e.g. a morning nap) is not part of the night. It is not the AASM final awakening, which is the end of the last sleep epoch before lights on: the two agree when the subject stays awake after the last sleep epoch. The fallback (no sustained wake, the night ends awake: the last sleep-to-wake transition) is the AASM definition.Example (defaults): AWAKE 30 min, N2 60, REM 20, AWAKE 5, N2 120, N1 5, AWAKE 120 gives the start of the last AWAKE bout (at 240 min): the brief AWAKE at 110 min holds 95 min of sleep in its window.
- Parameters:
df (pd.DataFrame) – Hypnogram with
annotation,start,end(optionallyduration).t_awake_threshold (int, optional) – Minimum sustained awake time in minutes to be considered awakened. Default is 90.
t_sleep_threshold (int, optional) – Maximum sleep time in minutes allowed in the awakening window. Default is 10.
awake_tag (str, optional) – Tag for awake state. Default is ‘AWAKE’ (
'WAKE'is the same state).sleep_cycle_tags (list, optional) – List of tags indicating sleep states. Default is [‘REM’, ‘N1’, ‘N2’, ‘N3’].
after (datetime or float, optional) – Only consider awakenings starting at or after this time. It must be of the same kind as
start(a timezone-aware datetime for datetime input, a number of seconds for numeric input), otherwiseTypeErroris raised.score_night()passes the sleep onset. New in 3.0.0.on_unscored ({'warn', 'raise'}, optional) – What to do when the awakening depends on how the unscored time is scored (e.g.
... N2, UNKNOWN, AWAKE: the wake may have begun anywhere in the unknown span).'warn'(default) emits aUserWarningnaming the unscored spans and the alternative times and returnsNaT/NaN;'raise'raisesValueError. New in 3.0.0.max_gap_s (float, optional) – Time gaps between epochs longer than this (seconds) are unscored time; shorter gaps are ignored. Default 1.0. New in 3.0.0.
- Returns:
Time of the awakening, in the type of the
startcolumn;NaT(datetime input) orNaN(numeric input) if it is ambiguous because of unscored epochs andon_unscored='warn'.- Return type:
datetime or float
- Raises:
TypeError – If
afteris not of the same kind asstart, or the input is invalid (module docstring).ValueError – If the hypnogram is empty or invalid, or it ends awake without a candidate (with every scoring of the unscored time); or the awakening depends on the unscored time and
on_unscored='raise'.
Notes
Note
Changed in 3.0.0:
t_sleep_threshold(minutes) was compared with the sleep time in seconds, so 10 meant 10 s: one minute of N2 in the window rejected an awakening. The defaultsleep_cycle_tagsnow include'N1', and a path through N1 bouts counts in any case: v1.0.0 missed an awakening reached through N1 (N2 -> N1 -> AWAKE, the usual way to wake up) and then returned an earlier brief awakening, which truncated the sleep period. The window counts only the part of each bout inside it and ist_awake_threshold + t_sleep_thresholdlong (v1.0.0 counted thet_awake_thresholdwindow’s epochs in full). A first epoch in a sleep stage is no longer a candidate (it returned the start of that sleep epoch),'Arousal'rows are removed like'Arrousal','WAKE'is read as'AWAKE', epochs are sorted and merged into bouts, andValueErrorreplaces anIndexErrorwhen no transition exists. Numeric timestamps (seconds) work. A night whose awakening depends on unscored epochs or gaps (e.g.N2, UNKNOWN, AWAKE, or a night endingN2, UNKNOWN) used to return a silently shifted awakening (the transition before the unknown span, the end of the recording, or an error); it now warns and returnsNaT/NaN(on_unscored,max_gap_s).
- brainmaze_eeg.hypnogram.get_fell_asleep_time(df, t_sleep_check=60, t_awake_threshold=10, awake_tag='AWAKE', sleep_cycle_tags=['REM', 'N1', 'N2', 'N3'], on_unscored='warn', max_gap_s=1.0)#
Determine when the subject fell asleep based on hypnogram data.
Touching epochs with the same label are merged into bouts first (module docstring). Candidates are the first bout, if it is a sleep stage, and every sleep-stage bout that directly follows an
awake_tagbout, in time order. The sleep onset is the first candidate for which less thant_awake_thresholdminutes ofawake_taglie in the window[candidate start, candidate start + t_sleep_check). Only the part of each bout inside the window counts (a bout reaching past the window end is clipped to it). Unscored time (module docstring) is checked: the onset is returned only if it is the same with all unscored time scored as AWAKE and as each sleep stage.Example (defaults): AWAKE 30 min, N2 55, AWAKE 15, N2 200: the window after the N2 at 30 min holds 5 min of AWAKE (the AWAKE bout from 85 to 100 min is clipped at 90), so the onset is at 30 min.
- Parameters:
df (pd.DataFrame) – Hypnogram with
annotation,start,end(optionallyduration).t_sleep_check (int, optional) – Time window in minutes to check for sustained sleep. Default is 60.
t_awake_threshold (int, optional) – The awake time in minutes inside the window must be below this. Default is 10.
awake_tag (str, optional) – Tag for awake state. Default is ‘AWAKE’ (
'WAKE'is the same state).sleep_cycle_tags (list, optional) – List of tags indicating sleep states. Default is [‘REM’, ‘N1’, ‘N2’, ‘N3’].
on_unscored ({'warn', 'raise'}, optional) – What to do when the onset depends on how the unscored time (epochs labelled neither a sleep stage nor
awake_tag, gaps longer thanmax_gap_s) is scored.'warn'(default) emits aUserWarningnaming the unscored spans and the alternative onsets and returnsNaT/NaN;'raise'raisesValueError. New in 3.0.0.max_gap_s (float, optional) – Time gaps between epochs longer than this (seconds) are unscored time; shorter gaps are ignored. Default 1.0. New in 3.0.0.
- Returns:
Start of the sleep-onset bout, in the type of the
startcolumn;NaT(datetime input) orNaN(numeric input) if the onset is ambiguous because of unscored epochs andon_unscored='warn'.- Return type:
datetime or float
- Raises:
ValueError – If no candidate qualifies (with every scoring of the unscored time), the hypnogram is empty, or epochs overlap; or the onset depends on the unscored time and
on_unscored='raise'.
Notes
Note
Changed in 3.0.0: If no candidate qualified, v1.0.0 returned the start of the recording, which looks like a valid onset; it now raises
ValueError. Rows labelled'Arousal'are removed like'Arrousal'(only the latter was, so an'Arousal'epoch between AWAKE and sleep hid the onset). Epochs are sorted and merged into bouts, so tiled and merged hypnograms agree. The window counts only the part of each bout inside it (v1.0.0 counted every epoch that started inside it in full, so on merged bouts an AWAKE bout starting 5 min before the window end counted with its whole length).awake_tagis compared for equality (it was a substring test);'WAKE'is read as'AWAKE'. Numeric timestamps (seconds) work. Unscored epochs and gaps no longer shift the onset silently (on_unscored,max_gap_s).
- brainmaze_eeg.hypnogram.get_hypnogram_datarate(df)#
Calculate the data rate of a hypnogram.
- Parameters:
df (pd.DataFrame) – Hypnogram with
annotation,startandendcolumns (optionallyduration); see the module docstring for the accepted time formats.- Returns:
Scored time (sum of
end - startof all non-arousal epochs) divided by the time span from the earlieststartto the latestend. 1.0 means no gaps.- Return type:
float
- Raises:
ValueError – If the hypnogram is empty, spans zero time, or epochs overlap.
Notes
Note
Changed in 3.0.0: The span used
timedelta.seconds, which drops whole days: a fully scored 25 h hypnogram returned 25.0 instead of 1.0. Numeric timestamps (seconds) now work (they raisedAttributeError), arousal rows are no longer counted as scored time, the input may be unsorted, and the input is validated (module docstring).
- brainmaze_eeg.hypnogram.get_number_of_awakenings(df, awake_tag='AWAKE', n1_tag='N1', sleep_tags=['N2', 'N3', 'REM'])#
Count the number of awakenings after sleep onset.
The epochs are taken in time order. An awakening is counted at the first
awake_tagepoch after asleep_tagsepoch; the next awakening needs anothersleep_tagsepoch first. With the defaultsleep_tags(N2, N3, REM)n1_tagepochs neither arm nor break the count:N2, N1, AWAKEis one awakening, and so isN2, AWAKE, N1, AWAKE(the N1/AWAKE alternation of drowsy wake is not counted again). This differs fromget_fell_asleep_time()andget_awakening_time(), where N1 is sleep (as in AASM). Passsleep_tags=['N1', 'N2', 'N3', 'REM']to count every transition from any sleep stage, N1 included, intoawake_tag. Other labels (e.g.'UNKNOWN') between sleep andawake_tagdo not break the count;score_night()checks them (on_unscored).- Parameters:
df (pd.DataFrame) – Hypnogram with an
annotationcolumn. Withstart/endcolumns the input is validated and sorted bystart; without them the row order is used.awake_tag (str, optional) – Tag for awake state. Default is ‘AWAKE’ (
'WAKE'is the same state).n1_tag (str, optional) – Tag for N1 sleep stage. Default is ‘N1’.
sleep_tags (list, optional) – List of tags for sleep states. Default is [‘N2’, ‘N3’, ‘REM’].
- Returns:
Number of awakenings.
- Return type:
int
- Raises:
TypeError – With
start/endcolumns: on naive datetimes or other invalid time columns (module docstring).ValueError – With
start/endcolumns: if epochs overlap (after removing arousals) ordurationdoes not equalend - start.
Notes
Note
Changed in 3.0.0: With
n1_taginsleep_tags,AWAKE, N1no longer counts an awakening at the N1 epoch (falling asleep is not an awakening). Sorted bystartwhen available (it used the row order and label-based indexing, which broke on a non-default index), and'WAKE'is read as'AWAKE'. Arousal rows (both spellings) are ignored; this does not change the count (an arousal never reset the state machine between sleep and AWAKE). Withstart/endcolumns the input is now validated (v1.0.0 ignored the times): naive datetimes raiseTypeError; overlapping states (e.g. a frame that also holds channel annotations) and adurationthat differs fromend - startraiseValueError.
- brainmaze_eeg.hypnogram.get_number_of_sleep_stages(df, tags='REM', delay=30)#
Count the bouts (periods) of a sleep stage, ignoring bouts that start too soon after the previous counted one.
Epochs whose
annotationis intagsare sorted bystart. Epochs that touch or overlap in time are merged into one bout (severaltagsare treated as one class: touching epochs of any of them form one bout). A 40-min REM period counts as one bout whether it is one row or eighty 30-s epochs.The first bout is counted. A later bout is counted if its start is at least
delayminutes after the end of the last counted bout (bouts that were not counted do not move this reference).
Example (delay 30 min): REM 40 min, N2 20 min, REM 40 min gives 1 (the second REM bout starts 20 min after the first ends); with N2 30 min instead it gives 2.
- Parameters:
df (pd.DataFrame) – Hypnogram with
annotation,start,end(optionallyduration); see the module docstring for the accepted time formats. Rows may be unsorted.tags (str or list, optional) – Sleep stage tag(s) to count. Default is ‘REM’.
delay (float, optional) – Minimum time in minutes (
>= 0) between the end of the last counted bout and the start of the next counted one. Default is 30. Any value is allowed, including>= 1440(one day).
- Returns:
Number of counted bouts (0 if no epoch has one of the tags).
- Return type:
int
Notes
Note
Changed in 3.0.0: v1.0.0 always raised under pandas >= 2 (it used
DataFrame.append). Its results under pandas < 2 differ from 3.0.0 as follows:It worked per epoch: the delay was measured from the end of the single epoch that started the last counted occurrence, so on a hypnogram tiled into 30-s epochs one long period was counted several times (one 40-min REM period: 2; 90 min: 3; REM 40 / N2 20 / REM 40 min: 4). Now bouts are counted (1, 1 and 1). On input that is already merged into bouts (as
score_nightpassed it) the result is unchanged.With several distinct
tagsit required an epoch to equal all of them (none did) and then raisedAttributeError; the same happened when no epoch matched. Now several tags are counted together and no match gives 0.It compared
timedelta.seconds, which drops whole days and wraps negatives: a gap of 1 day + 1 min counted as 1 min,delay >= 1440wrapped (delay=1440acted as 0, so every epoch counted), and unsorted or overlapping rows gave arbitrary counts. Now the rows are sorted, overlapping epochs join one bout, and gaps anddelayare exact.Numeric timestamps must be in seconds; values that look like milliseconds raise
ValueError(they were silently read as seconds). Naive datetimes raiseTypeError.
- brainmaze_eeg.hypnogram.get_rem_latency(df, rem_tag='REM', awake_tag='AWAKE', on_unscored='warn', max_gap_s=1.0)#
Calculate REM sleep latency (time from falling asleep or last awake to first REM).
- Parameters:
df (pd.DataFrame) – Hypnogram with
annotation,start,end(optionallyduration).rem_tag (str, optional) – Tag for REM sleep state. Default is ‘REM’.
awake_tag (str, optional) – Tag for awake state. Default is ‘AWAKE’ (
'WAKE'is the same state).on_unscored ({'warn', 'raise'}, optional) –
Each latency is returned only if it is the same with all unscored time (module docstring) scored as AWAKE and as each sleep stage; unscored time before the first REM may hide an earlier REM, an earlier or later sleep onset, or a later wake. If a latency differs,
'warn'(default) warns and givesNaT/NaNfor it;'raise'raisesValueError. New in 3.0.0.On real nights this is common: any unscored time before the first REM (even a leading unscored segment before lights-off), or, on a night without REM, unscored time anywhere, makes both latencies
NaT/NaN, because that time may have been REM. To get the latencies, drop leading/trailing unscored rows, relabel the spans whose state is known, or raisemax_gap_sfor harmless gaps (module docstring, “Unscored time in practice”).max_gap_s (float, optional) – Time gaps between epochs longer than this (seconds) are unscored time; shorter gaps are ignored. Default 1.0. New in 3.0.0.
- Returns:
'fall_asleep': start of the first REM epoch minus the sleep onset (get_fell_asleep_time()with its defaults).'last_awake': start of the first REM epoch minus the end of the lastawake_tagepoch before it (the sleep onset if there is none). Timedeltas for datetime input, seconds (float) for numeric input. A latency is negative when the first REM precedes the reference (e.g. REM before the sustained sleep onset).- Return type:
dict
- Raises:
ValueError – If there is no
rem_tagepoch or no sleep onset (with every scoring of the unscored time); or a latency depends on the unscored time andon_unscored='raise'.
Notes
Note
Changed in 3.0.0: Sorted, validated input; arousal rows (both spellings) removed;
'WAKE'read as'AWAKE';ValueErrorinstead ofIndexErrorwithout REM. On a fully scored night the latencies are defined as before, from the sleep onset ofget_fell_asleep_time()(whose onset window changed in 3.0.0, see there). With unscored time (module docstring) a latency that depends on it is no longer returned: it isNaT/NaNwith a warning (on_unscored='warn'), orValueErroris raised (on_unscored='raise'); seeon_unscoredfor how often this happens.
- brainmaze_eeg.hypnogram.get_stage_times(df, keys)#
Get total times for multiple sleep stages.
- Parameters:
df (pd.DataFrame) – Hypnogram dataframe with ‘annotation’ and ‘duration’ columns.
keys (list) – List of sleep stage tags.
- Returns:
List of total durations for each stage in seconds.
- Return type:
list
- brainmaze_eeg.hypnogram.get_stage_times_dataset(hypnograms: list, keys, verbose=True)#
Get stage times for a dataset of hypnograms.
- Parameters:
hypnograms (list) – List of hypnogram dataframes.
keys (list) – List of sleep stage tags.
verbose (bool, optional) – If True, show progress bar. Default is True.
- Returns:
Dataframe with stage times for each hypnogram.
- Return type:
pd.DataFrame
- brainmaze_eeg.hypnogram.get_time_by_key(df, key)#
Calculate total time for a specific sleep stage or combination of stages.
- Parameters:
df (pd.DataFrame) – Hypnogram dataframe with ‘annotation’ and ‘duration’ columns.
key (str or list) – Sleep stage tag(s) to sum durations for.
- Returns:
Total duration in the unit of the
durationcolumn (seconds by convention; a timedelta column gives apandas.Timedelta). Thedurationcolumn is summed as given, not checked againststart/end.'WAKE'and'AWAKE'are the same state, indfand inkey.- Return type:
float
- brainmaze_eeg.hypnogram.get_transition_counts(hyp, states=['AWAKE', 'N1', 'N2', 'N3', 'REM'])#
Count transitions between sleep stages.
Consecutive rows (in
startorder) form a transition; a row whose label is not instatesbreaks the chain (no transition is counted into or out of it). Time gaps between rows are not considered. On a hypnogram tiled into epochs the diagonal holds the epoch-to-epoch persistence; on merged bouts it is (nearly) zero.- Parameters:
hyp (pd.DataFrame) – Hypnogram with an
annotationcolumn. Withstart/endcolumns it is validated and sorted bystart; without them the row order is used.states (list, optional) – List of sleep states to include. Default is [‘AWAKE’, ‘N1’, ‘N2’, ‘N3’, ‘REM’].
- Returns:
Matrix of transition counts where [i,j] is count of transitions from state i to state j.
- Return type:
np.ndarray
- Raises:
TypeError – With
start/endcolumns: on naive datetimes or other invalid time columns (module docstring).ValueError – With
start/endcolumns: if epochs overlap (after removing arousals, unless they are listed instates) ordurationdoes not equalend - start; or ifstateslists the same state twice.
Notes
Note
Changed in 3.0.0: Rows are sorted by
start(it used label-based indexing on the row order, which broke on a non-default index), and arousal rows ('Arousal'/'Arrousal') are ignored unless listed instates(an arousal between two epochs hid their transition).'WAKE'is read as'AWAKE'inhypand instates(the matrix keeps the order and names ofstates). Withstart/endcolumns the input is now validated (v1.0.0 ignored the times): naive datetimes raiseTypeError; overlapping states (e.g. a frame that also holds channel annotations) and adurationthat differs fromend - startraiseValueError.
- brainmaze_eeg.hypnogram.get_transition_matrix(hyp, states=['AWAKE', 'N1', 'N2', 'N3', 'REM'])#
Calculate transition probability matrix between sleep stages.
- Parameters:
hyp (pd.DataFrame) – Hypnogram dataframe with ‘annotation’ column.
states (list, optional) – List of sleep states to include. Default is [‘AWAKE’, ‘N1’, ‘N2’, ‘N3’, ‘REM’].
- Returns:
Normalized transition probability matrix.
- Return type:
np.ndarray
- brainmaze_eeg.hypnogram.get_transition_matrix_dataset(hypnograms, states=['AWAKE', 'N1', 'N2', 'N3', 'REM'])#
Calculate mean and std of transition matrices across a dataset of hypnograms.
- Parameters:
hypnograms (list) – List of hypnogram dataframes.
states (list, optional) – List of sleep states to include. Default is [‘AWAKE’, ‘N1’, ‘N2’, ‘N3’, ‘REM’].
- Returns:
Mean transition matrix and standard deviation matrix (rows of states that never occur in a hypnogram are left out of the mean of that row).
- Return type:
tuple
- brainmaze_eeg.hypnogram.is_sleep_complete(df, awake_tag='AWAKE')#
Check if the sleep session is complete (starts and ends with awake state).
- Parameters:
df (pd.DataFrame) – Hypnogram with
annotation,start,end(optionallyduration).awake_tag (str, optional) – Tag for awake state. Default is ‘AWAKE’ (
'WAKE'is the same state).
- Returns:
True if the earliest and the latest epoch (arousals excluded) are
awake_tag.- Return type:
bool
- Raises:
TypeError – On naive datetimes or other invalid time columns (module docstring).
ValueError – If the hypnogram is empty, epochs overlap (after removing arousals), or
durationdoes not equalend - start.
Notes
Note
Changed in 3.0.0: Uses the earliest and latest epoch by
start(it used the first and last rows), ignores arousal rows, reads'WAKE'as'AWAKE', and raisesValueErroron an empty hypnogram. The input is now validated like everywhere else in the module (v1.0.0 ignored the times): naive datetimes raiseTypeError; overlapping states (e.g. a frame that also holds channel annotations) and adurationthat differs fromend - startraiseValueError.
- brainmaze_eeg.hypnogram.plot_hypnogram(orig_df, hypnogram_values=None, hypnogram_colors=None, fontsize=12, fig=None, night_start=22)#
Creates a Matplotlib figure of the hypnogram from the annotations. Time must be in a time-zone aware format.
- Parameters:
orig_df (pd.DataFrame) – Hypnogram with
annotationand timezone-awarestart/endcolumns, in any row order (it is sorted bystart). Arousal rows ('Arousal', or the legacy'Arrousal') are drawn as events, not as states. The frame is not modified.hypnogram_values (dict) – dict of a y-axis values for each hypnogram state. The default covers
'AWAKE'('WAKE'is drawn as'AWAKE'unless the dict has a'WAKE'key),'Arousal','SLP'and'N'(inserted byfill_sleep_voids()andfill_nonrem_voids()),'REM','N1','N2','N3'and'UNKNOWN'(CyberPSG’s unscored label, drawn at 0).hypnogram_colors (dict) – dict of color hex codes for each hypnogram state (default: the same labels)
fontsize (int) – Fontsize
fig (figure) – Already existing figure object.
night_start (int) – Hour at which the night begins. Nights
[night_start, night_start + 12 h]are shaded for every calendar date in the plotted range, on the wall clock of the timezone of the firststart(UTC data is shaded atnight_startUTC: convert it to the recording site’s timezone first, e.g.df.start.dt.tz_convert(...)).
- Raises:
ImportError – If matplotlib is not installed (optional extra
brainmaze-eeg[plot]).TypeError – If
start/endare not timezone-aware datetimes.ValueError – If the hypnogram has no sleep-state epochs, or a label has no y value or colour.
Notes
Note
Changed in 3.0.0: Works with pandas 2 (it used
DataFrame.append, removed in pandas 2, and called thedatetimemodule instead ofdatetime.datetime, so it always raised). It no longer addsstate_id/state_colorcolumns to the input frame, sorts the rows, and handles a one-epoch hypnogram. Night shading is drawn per calendar date over the plotted range (it was anchored to eachdaygroup’s first epoch, so a recording starting after midnight, or a state lasting over several days, lost its night shading); adaycolumn is no longer used. Arousals are recognised as'Arousal'(the brainmaze-utils label, which was drawn as a sleep state) and'Arrousal'(which raisedKeyErrorwith the default maps). The default maps also cover'N','UNKNOWN'and'WAKE', so the output ofcorrect_hypnogram()and of the brainmaze-utils loaders can be plotted. Other labels missing from the maps raiseValueErrornaming them.
- brainmaze_eeg.hypnogram.print_sleep_score(score)#
Print a formatted summary of sleep metrics.
- Parameters:
score (dict) – Sleep score dictionary from score_night function.
Notes
Note
Changed in 3.0.0: Works with the floats that
score_night()returns (it called.secondson them). Durations keep whole days and negative latencies keep their sign. The relative times are fractions of the sleep period (sleep onset to awakening), which is now labelled as such (it was printed as “Total Sleep Time”).
- brainmaze_eeg.hypnogram.score_night(df, plot=False, on_unscored='warn', max_gap_s=1.0)#
Compute comprehensive sleep metrics for a night’s hypnogram.
The hypnogram is validated, sorted, arousal rows (
'Arousal'/'Arrousal') are removed,'WAKE'is read as'AWAKE', and touching epochs with the same label are merged into bouts (module docstring). Then, with the defaults of the individual functions:fell_asleep_time:get_fell_asleep_time().awakening_time:get_awakening_time()withafter=fell_asleep_time.The sleep period is made of the bouts starting at or after
fell_asleep_timeand beforeawakening_time; the counts and stage times below are taken over it.Unscored time (
'UNKNOWN'or any label that is neither a sleep stage nor AWAKE, and gaps longer thanmax_gap_s; module docstring): the whole score is computed with all unscored time scored as AWAKE, and as each of N1, N2, N3 and REM (the awakening is searched after the onset found with the same scoring). A field that is the same in all of them is returned; a field that differs is ambiguous and isNaT/NaNwithon_unscored='warn'(default; oneUserWarningnames the fields, the unscored spans and the alternatives), orValueErroris raised withon_unscored='raise'. Other fields stay computed. For example anUNKNOWNepoch inside the sleep usually leaves the onset, the awakening and the stage times unchanged but makesn_awakeningsambiguous (it may have been an awakening), and... N2, UNKNOWN, AWAKEmakesawakening_timeambiguous.Expect NaN on real nights with unscored time. Both REM latencies,
n_awakeningsandn_complete_sleep_cycleswill often beNaN: a leading unscored segment before lights-off (or any unscored time before the first REM) could be the first REM; on a night without REM, unscored time even after the awakening could be a REM; an unscored epoch or gap inside the sleep could be an awakening or a REM bout. Remedies: drop leading/trailing unscored rows (time outside the recording is not unscored), relabel spans whose state is known, and raisemax_gap_sfor short harmless gaps. On the demo night (5 minUNKNOWNat the start, 12.5 minUNKNOWNin the sleep, one 2-min gap) the four fields are NaN; trimming the leading segment brings back the latencies, and relabelling the innerUNKNOWNplusmax_gap_s=120brings back all fields. See the module docstring, “Unscored time in practice”.The stage times count the scored epochs only: unscored time inside the sleep period is in none of them (so they do not depend on how it is scored; compare
get_hypnogram_datarate()).sleep_completereads the first and last epoch as scored.
- Parameters:
df (pd.DataFrame) – Hypnogram with
annotation,start,end(optionallyduration): timezone-aware datetimes or numeric POSIX timestamps in seconds.plot (bool, optional) – If True, plot the hypnogram with sleep onset and awakening markers. Default is False. Needs matplotlib (optional extra
brainmaze-eeg[plot]) and datetime input.on_unscored ({'warn', 'raise'}, optional) – See above. New in 3.0.0.
max_gap_s (float, optional) – Time gaps between epochs longer than this (seconds) are unscored time; shorter gaps are ignored. Default 1.0. New in 3.0.0.
- Returns:
sleep_complete: the earliest and latest epochs are AWAKE (
is_sleep_complete())fell_asleep_time: time of sleep onset (type of
start)rem_latency_fell_asleep: first REM start minus sleep onset, in seconds (float; negative if REM came first; NaN if there is no REM)
rem_latency_last_awake: first REM start minus the end of the last AWAKE before it, in seconds (float; NaN if there is no REM); see
get_rem_latency()awakening_time: time of final awakening (type of
start)n_complete_sleep_cycles: REM bouts in the sleep period,
get_number_of_sleep_stages()withtags='REM', delay=30n_awakenings:
get_number_of_awakenings()(defaults) over the sleep period: awakenings after N2, N3 or REM; N1/AWAKE alternation is not counted again (unlike the onset and the awakening, where N1 is sleep)n1_sleep_time, n2_sleep_time, n3_sleep_time, rem_sleep_time, awake_sleep_time: scored time in each stage within the sleep period, in seconds
An ambiguous field (see above) is
NaT(datetime input) orNaN(numeric input) forfell_asleep_time/awakening_time, andNaN(a float) for the others.- Return type:
dict
- Raises:
ValueError – If the hypnogram is empty, invalid (module docstring), or no sleep onset or awakening can be determined (with every scoring of the unscored time); or a field is ambiguous and
on_unscored='raise'.
Notes
Note
Changed in 3.0.0: v1.0.0 raised with brainmaze-utils 3 (its
merge_annotationsaccepts only numeric timestamps, while the rest of the function needed datetimes). Now datetime and numeric input both work. Numbers that v1.0.0 would have got wrong are fixed: REM latencies usedtimedelta.seconds(a latency of -23 min became 85020 s; now negative values stay negative), the awakening ignoredt_sleep_thresholdunits (seeget_awakening_time()), it missed a final awakening reached through N1 (N2 -> N1 -> AWAKE) and then cut the sleep period at an earlier brief awakening, and it could be found before the sleep onset (now only after it). The onset and awakening windows count only the part of each bout inside them.'Arousal'rows were not removed,'WAKE'(NSRR) was not recognised as'AWAKE', and a night without REM raisedIndexError(now the latencies are NaN). Results that depend on unscored epochs or gaps are no longer returned silently (seeon_unscored).
- brainmaze_eeg.hypnogram.transition_matrix_to_change_matrix(m)#
Convert transition matrix to change matrix (excluding diagonal/same-state transitions).
- Parameters:
m (np.ndarray) – Transition probability matrix.
- Returns:
Change matrix with diagonal set to zero and rows renormalized.
- Return type:
np.ndarray
- brainmaze_eeg.hypnogram.valid_dataset_index_by_duration(hypnograms: list, filt_dict: dict)#
Filter hypnograms by minimum duration requirements for specific stages.
- Parameters:
hypnograms (list) – List of hypnogram dataframes.
filt_dict (dict) – Dictionary mapping stage names to minimum total duration (unit of the
durationcolumn, seconds by convention). A stage that does not occur has duration 0, so a threshold of 0 accepts a hypnogram without that stage.
- Returns:
Indices of hypnograms in which every listed stage lasts at least its threshold.
- Return type:
list
Notes
Note
Changed in 3.0.0: v1.0.0 compared the total duration of the whole hypnogram (all stages) with each stage’s threshold, so e.g.
{'REM': 3600}accepted any night longer than 1 h that contained any REM. v1.0.0 also rejected a hypnogram in which a listed stage does not occur, whatever its threshold; now an absent stage has duration 0, so it passes a threshold of 0 (and fails any positive one).