.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "examples/mouse_eye_movements.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_examples_mouse_eye_movements.py>`
        to download the full example code. or to run this example in your browser via Binder

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_examples_mouse_eye_movements.py:

Pupil tracking
=================

Look at eye movements and pupil diameter.

.. GENERATED FROM PYTHON SOURCE LINES 8-10

Imports
-------

.. GENERATED FROM PYTHON SOURCE LINES 10-19

.. code-block:: Python

    import sleap_io as sio
    import xarray as xr
    from matplotlib import pyplot as plt

    import movement.kinematics as kin
    from movement import sample_data
    from movement.filtering import rolling_filter
    from movement.plots import plot_centroid_trajectory








.. GENERATED FROM PYTHON SOURCE LINES 20-29

Load the data
-------------
We will use two datasets from the sample data module. These datasets involve
recordings of the eyes of mice placed on a rotating platform with different
visual stimuli. The ``uniform`` condition features a uniformly lit surround
stimulus, whereas the ``black`` condition was acquired in the dark. These
datasets were tracked using DeepLabCut (DLC) and include four keypoints:
two on either side of the pupil (``pupil-L`` and ``pupil-R``) and two on
either side of the eye (``eye-L`` and ``eye-R``).

.. GENERATED FROM PYTHON SOURCE LINES 29-41

.. code-block:: Python


    ds_black = sample_data.fetch_dataset(
        "DLC_rotating-mouse_eye-tracking_stim-black.predictions.h5",
        with_video=True,
    )
    ds_uniform = sample_data.fetch_dataset(
        "DLC_rotating-mouse_eye-tracking_stim-uniform.predictions.h5",
        with_video=True,
    )
    # Save data in a dictionary.
    ds_dict = {"black": ds_black, "uniform": ds_uniform}








.. GENERATED FROM PYTHON SOURCE LINES 42-43

Print the content of one of the datasets.

.. GENERATED FROM PYTHON SOURCE LINES 43-44

.. code-block:: Python

    print(ds_dict["black"])




.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    <xarray.Dataset> Size: 728kB
    Dimensions:      (time: 7000, space: 2, keypoints: 4, individuals: 1)
    Coordinates:
      * time         (time) float64 56kB 0.0 0.025 0.05 0.075 ... 174.9 174.9 175.0
      * space        (space) <U1 8B 'x' 'y'
      * keypoints    (keypoints) <U7 112B 'pupil-L' 'pupil-R' 'eye-L' 'eye-R'
      * individuals  (individuals) <U12 48B 'individual_0'
    Data variables:
        position     (time, space, keypoints, individuals) float64 448kB 258.8 .....
        confidence   (time, keypoints, individuals) float64 224kB 1.0 1.0 ... 1.0
    Attributes:
        source_software:  DeepLabCut
        ds_type:          poses
        fps:              40.0
        time_unit:        seconds
        source_file:      /home/runner/.movement/data/poses/DLC_rotating-mouse_ey...
        frame_path:       /home/runner/.movement/data/frames/rotating-mouse_eye-t...
        video_path:       /home/runner/.movement/data/videos/rotating-mouse_eye-t...




.. GENERATED FROM PYTHON SOURCE LINES 45-47

Explore the accompanying videos
-------------------------------

.. GENERATED FROM PYTHON SOURCE LINES 47-56

.. code-block:: Python

    for ds_name, ds in ds_dict.items():
        video = sio.load_video(ds.video_path)
        # To avoid having to reload the video again we add it to ds as attribute
        ds_dict[ds_name] = ds.assign_attrs({"video": video})
        n_frames, height, width, channels = video.shape
        print(f"Dataset: {ds_name}")
        print(f"Number of frames: {n_frames}")
        print(f"Frame size: {width}x{height}")
        print(f"Number of channels: {channels}\n")




.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Dataset: black
    Number of frames: 7000
    Frame size: 640x480
    Number of channels: 1

    Dataset: uniform
    Number of frames: 7000
    Frame size: 640x480
    Number of channels: 1





.. GENERATED FROM PYTHON SOURCE LINES 57-59

Plot first frame with keypoints
-------------------------------

.. GENERATED FROM PYTHON SOURCE LINES 59-72

.. code-block:: Python

    fig, ax = plt.subplots(1, 2, figsize=(7.5, 4))
    for i, (da_name, ds) in enumerate(ds_dict.items()):
        ax[i].imshow(ds.video[0], cmap="gray")  # plot first video frame
        for keypoint in ds.keypoints.values:
            x = ds.position.sel(time=0, space="x", keypoints=keypoint)
            y = ds.position.sel(time=0, space="y", keypoints=keypoint)
            ax[i].scatter(x, y, label=keypoint)  # plot keypoints
        ax[i].legend()
        ax[i].set_title(f"{da_name} (First Frame)")
        ax[i].invert_yaxis()  # because the dataset was collected flipped
    plt.tight_layout()
    plt.show()




.. image-sg:: /examples/images/sphx_glr_mouse_eye_movements_001.png
   :alt: black (First Frame), uniform (First Frame)
   :srcset: /examples/images/sphx_glr_mouse_eye_movements_001.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 73-77

Pupil trajectory
----------------
A quick plot of the trajectory of the centre of the pupil using the
``plot_centroid_trajectory`` function from ``movement.plots``.

.. GENERATED FROM PYTHON SOURCE LINES 77-84

.. code-block:: Python

    time_window = slice(1, 24)  # seconds
    position_black = ds_black.position.sel(time=time_window)  # data array to plot
    fig, ax = plot_centroid_trajectory(
        position_black, keypoints=["pupil-L", "pupil-R"]
    )
    fig.show()




.. image-sg:: /examples/images/sphx_glr_mouse_eye_movements_002.png
   :alt: Trajectory
   :srcset: /examples/images/sphx_glr_mouse_eye_movements_002.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 85-88

Pupil trajectories on top of video frame
----------------------------------------
We can look at pupil trajectories plotted on top of a video frame.

.. GENERATED FROM PYTHON SOURCE LINES 88-103

.. code-block:: Python

    fig, ax = plt.subplots(1, 2, figsize=(11, 3))
    for i, (ds_name, ds) in enumerate(ds_dict.items()):
        ax[i].imshow(ds.video[100], cmap="gray")  # Plot frame 100 as background
        plot_centroid_trajectory(
            ds.position.sel(time=time_window),  # Select time window
            ax=ax[i],
            keypoints=["pupil-L", "pupil-R"],
            alpha=0.5,
            s=3,
        )
        ax[i].invert_yaxis()
        ax[i].set_title(f"Pupil Trajectory ({ds_name})")
    fig.show()





.. image-sg:: /examples/images/sphx_glr_mouse_eye_movements_003.png
   :alt: Pupil Trajectory (black), Pupil Trajectory (uniform)
   :srcset: /examples/images/sphx_glr_mouse_eye_movements_003.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 104-109

Keypoint positions over time
----------------------------
For the rest of this example we are only interested in the position data.
For convenience, We will combine the two position arrays into a single
array with a new dimension called ``lighting``.

.. GENERATED FROM PYTHON SOURCE LINES 109-112

.. code-block:: Python

    positions = xr.concat([ds_black.position, ds_uniform.position], "lighting")
    positions.coords["lighting"] = ["black", "uniform"]








.. GENERATED FROM PYTHON SOURCE LINES 113-114

Define plotting parameters for reuse.

.. GENERATED FROM PYTHON SOURCE LINES 114-123

.. code-block:: Python

    plot_params = {
        "x": "time",
        "hue": "keypoints",
        "col": "space",
        "row": "lighting",
        "aspect": 1.5,
        "size": 2.5,
    }
    sel = {"time": slice(8, 25)}







.. GENERATED FROM PYTHON SOURCE LINES 124-125

Plot the keypoint positions over time.

.. GENERATED FROM PYTHON SOURCE LINES 125-128

.. code-block:: Python

    positions.sel(**sel).squeeze().plot.line(**plot_params)
    plt.subplots_adjust(right=0.85)  # Make space on the right for the legend
    plt.show()



.. image-sg:: /examples/images/sphx_glr_mouse_eye_movements_004.png
   :alt: space = x, space = y
   :srcset: /examples/images/sphx_glr_mouse_eye_movements_004.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 129-136

Normalised keypoint positions over time
---------------------------------------
Normalizing the pupil's position relative to the midpoint of the eye reduces
the impact of head movements or artefacts caused by camera movement. By
subtracting the position of the eye's midpoint, we effectively transform the
data into a moving coordinate system, with the eye's midpoint as the origin.
In the rest of the example, the normalised data will be used.

.. GENERATED FROM PYTHON SOURCE LINES 136-138

.. code-block:: Python

    eye_midpoint = positions.sel(keypoints=["eye-L", "eye-R"]).mean("keypoints")
    positions_norm = positions - eye_midpoint







.. GENERATED FROM PYTHON SOURCE LINES 139-140

We plot the x and y positions again, but now using the normalised data.

.. GENERATED FROM PYTHON SOURCE LINES 140-143

.. code-block:: Python

    positions_norm.sel(**sel).squeeze().plot.line(**plot_params)
    plt.subplots_adjust(right=0.85)
    plt.show()



.. image-sg:: /examples/images/sphx_glr_mouse_eye_movements_005.png
   :alt: space = x, space = y
   :srcset: /examples/images/sphx_glr_mouse_eye_movements_005.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 144-150

Pupil position over time
------------------------
To look at pupil position—and later also velocity—over time, we use the
pupil centroid (in this case the midpoint between keypoints ``pupil-L`` and
``pupil-R``). The keypoint ``pupil-C`` is assigned using
``xarray.DataArray.assign_coords``.

.. GENERATED FROM PYTHON SOURCE LINES 150-156

.. code-block:: Python


    pupil_centroid = (
        positions_norm.sel(keypoints=["pupil-L", "pupil-R"])
        .mean("keypoints")
        .assign_coords({"keypoints": "pupil-C"})
    )







.. GENERATED FROM PYTHON SOURCE LINES 157-159

The pupil centroid keypoint ``pupil-C`` is be added to the ``positions_norm``
using ``xarray.concat``.

.. GENERATED FROM PYTHON SOURCE LINES 159-160

.. code-block:: Python

    positions_norm = xr.concat([positions_norm, pupil_centroid], dim="keypoints")







.. GENERATED FROM PYTHON SOURCE LINES 161-162

Now the position of the pupil centroid ``pupil-C`` can be plotted.

.. GENERATED FROM PYTHON SOURCE LINES 162-166

.. code-block:: Python

    positions_norm.sel(keypoints="pupil-C", **sel).squeeze().plot.line(
        x="time", hue="space", row="lighting", aspect=3.5, size=1.5
    )
    plt.show()



.. image-sg:: /examples/images/sphx_glr_mouse_eye_movements_006.png
   :alt: lighting = black, lighting = uniform
   :srcset: /examples/images/sphx_glr_mouse_eye_movements_006.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 167-173

Pupil velocity over time
------------------------
In these experiments, the mouse is being rotated clock- or anti-clock-wise,
triggering the vestibulo-ocular reflex. This reflex involves the
vestibular system in the inner ear, that detects head motion and adjusts eye
position to maintain stable vision.

.. GENERATED FROM PYTHON SOURCE LINES 175-182

When the head turns beyond the range that the vestibulo-ocular reflex
can compensate for, a quick, ballistic eye movement is triggered to
shift gaze to a new fixation point. These fast eye movements are seen in the
previous plot but become even more obvious when the velocity of the pupil
centroid is plotted. To do this, we use ``compute_velocity`` from the
``movement.kinematics`` module to calculate the velocity of the eye
movements.

.. GENERATED FROM PYTHON SOURCE LINES 182-188

.. code-block:: Python

    pupil_velocity = kin.compute_velocity(positions_norm.sel(keypoints="pupil-C"))
    pupil_velocity.name = "pupil velocity"
    pupil_velocity.sel(**sel).squeeze().plot.line(
        x="time", hue="space", row="lighting", aspect=3.5, size=1.5
    )
    plt.show()



.. image-sg:: /examples/images/sphx_glr_mouse_eye_movements_007.png
   :alt: lighting = black, lighting = uniform
   :srcset: /examples/images/sphx_glr_mouse_eye_movements_007.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 189-191

The positive peaks correspond to rapid eye movements to the right, the
negative peaks correspond to rapid eye movements to the left.

.. GENERATED FROM PYTHON SOURCE LINES 193-198

Pupil diameter
--------------
Here we define the pupil diameter as the distance between the two pupil
keypoints. We use ``compute_pairwise_distances`` from ``movement.kinematics``
to calculate the Euclidean distance between ``pupil-L`` and ``pupil-R``.

.. GENERATED FROM PYTHON SOURCE LINES 198-202

.. code-block:: Python

    pupil_diameter: xr.DataArray = kin.compute_pairwise_distances(
        positions_norm, dim="keypoints", pairs={"pupil-L": "pupil-R"}
    )
    pupil_diameter.name = "pupil diameter"







.. GENERATED FROM PYTHON SOURCE LINES 203-204

Now the pupil diameter can be plotted.

.. GENERATED FROM PYTHON SOURCE LINES 204-206

.. code-block:: Python

    pupil_diameter.plot.line(x="time", hue="lighting")
    plt.show()



.. image-sg:: /examples/images/sphx_glr_mouse_eye_movements_008.png
   :alt: mouse eye movements
   :srcset: /examples/images/sphx_glr_mouse_eye_movements_008.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 207-213

The plot of the pupil diameter looks noisy. The very steep peaks are
unlikely to represent real changes in the pupil size.
In fact, these steep peaks are probably caused by tracking errors
during blinking or squinting.
By looking at the distance between the two eye keypoints we can get an idea
of whether (and when) the animal is blinking or squinting.

.. GENERATED FROM PYTHON SOURCE LINES 213-236

.. code-block:: Python

    distance_between_eye_keypoints: xr.DataArray = kin.compute_pairwise_distances(
        positions_norm, dim="keypoints", pairs={"eye-L": "eye-R"}
    )
    distance_between_eye_keypoints.name = "distance (eye-L - eye-R)"

    # Combine the datasets into one DataArray
    combined = xr.concat(
        [distance_between_eye_keypoints, pupil_diameter], dim="variable"
    )
    combined = combined.assign_coords(
        variable=["distance (eye-L - eye-R)", "pupil diameter"]
    )

    # Plot the distance between the eye keypoints alongside the pupil diameter
    combined.plot.line(
        x="time", row="lighting", hue="variable", figsize=(8, 4), add_legend=False
    )
    labels = combined.coords["variable"].values
    plt.legend(labels, loc="center", bbox_to_anchor=(0.5, 1.4), ncol=2)
    plt.xlabel("time (s)")
    [ax.set_ylabel("distance (pixels)") for ax in plt.gcf().axes]
    plt.show()




.. image-sg:: /examples/images/sphx_glr_mouse_eye_movements_009.png
   :alt: lighting = black, lighting = uniform
   :srcset: /examples/images/sphx_glr_mouse_eye_movements_009.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 237-242

We indeed see that the sharp peaks in pupil diameter correspond to abrupt
changes of distance between the two eye keypoints.
Compared to fast eye movements and blinking, changes in pupil size are slow.
Filters can be applied to reduce noise and make underlying trends
in pupil diameter clearer.

.. GENERATED FROM PYTHON SOURCE LINES 244-250

Smooth pupil diameter
---------------------
A rolling mean (moving average) filter is used here to smooth the data by
averaging a specified number of data points (``window_len``).
We achieve this by calling the :func:`movement.filtering.rolling_filter`
function with the ``statistic="mean"`` option.

.. GENERATED FROM PYTHON SOURCE LINES 250-256

.. code-block:: Python


    window_len = 80
    mean_filter = rolling_filter(
        pupil_diameter, window=window_len, statistic="mean"
    )








.. GENERATED FROM PYTHON SOURCE LINES 257-258

Now the filtered pupil diameter can be plotted.

.. GENERATED FROM PYTHON SOURCE LINES 258-261

.. code-block:: Python

    mean_filter.plot.line(x="time", hue="lighting")
    plt.show()




.. image-sg:: /examples/images/sphx_glr_mouse_eye_movements_010.png
   :alt: mouse eye movements
   :srcset: /examples/images/sphx_glr_mouse_eye_movements_010.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 262-264

Instead of the mean, we could also use the median, which is the default
option for the ``statistic`` argument, and should be more robust to outliers.

.. GENERATED FROM PYTHON SOURCE LINES 264-269

.. code-block:: Python


    mdn_filter = rolling_filter(pupil_diameter, window_len, statistic="median")
    mdn_filter.plot.line(x="time", hue="lighting")
    plt.show()




.. image-sg:: /examples/images/sphx_glr_mouse_eye_movements_011.png
   :alt: mouse eye movements
   :srcset: /examples/images/sphx_glr_mouse_eye_movements_011.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 270-272

.. seealso::
  :ref:`sphx_glr_examples_smooth.py` example.


.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 3.760 seconds)


.. _sphx_glr_download_examples_mouse_eye_movements.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: binder-badge

      .. image:: images/binder_badge_logo.svg
        :target: https://mybinder.org/v2/gh/neuroinformatics-unit/movement/gh-pages?filepath=notebooks/examples/mouse_eye_movements.ipynb
        :alt: Launch binder
        :width: 150 px

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: mouse_eye_movements.ipynb <mouse_eye_movements.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: mouse_eye_movements.py <mouse_eye_movements.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: mouse_eye_movements.zip <mouse_eye_movements.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_