Pixel annotations

docs/user-guide/annotation-workflow.md

@@ -80,6 +80,37 @@ For multi-frame images a **frame slider** appears above the viewer. Drag it or u
 
 For NIfTI volumes, a **plane selector** appears (Axial / Coronal / Sagittal / ⊞ 3-Axis). The 3-axis mode opens a 2×2 grid with all three reformats and linked crosshair navigation — click any plane to jump to that position in the other two.
 
+## Segmentation Tool (Pixel-Level Painting)
+
+The segmentation tool is available for annotation types with vector type *Segmentation*. Select such a type in the left panel to reveal the painting palette.
+
+### Tools
+
+| Button | Tool | Description |
+|---|---|---|
+| Hand | Pan | Navigate without painting (default) |
+| Brush | Brush | Paint filled circles; adjust size with the slider |
+| Eraser | Eraser | Remove painted pixels |
+| Wand | Magic wand | Flood-select pixels with similar intensity |
+| Bucket | Fill | Flood-fill a connected region |
+
+### 3D segmentation on NIfTI volumes
+
+Segmentation works in **all three MPR planes** (Axial, Coronal, Sagittal):
+
+1. Select the plane using the plane selector buttons.
+2. Navigate to the slice you want to annotate using the frame slider.
+3. Paint with the brush or other tools.
+4. Switch to another plane and scroll to the anatomical position where you painted — the segmentation appears there automatically.
+
+Annotations drawn in any plane are stored as axial tiles and derived on the fly for the other planes, so the data is always consistent. The **3-Axis mode (⊞)** is the easiest way to verify cross-plane consistency: all three views update simultaneously as you navigate.
+
+### Keyboard shortcuts
+
+| Key | Action |
+|---|---|
+| `Ctrl+Z` | Undo last stroke |
+
 ## Verification
 
 Open the **Verification** view from the imageset page. The view steps through unverified annotations one by one:

docs/user-guide/image-formats.md

@@ -71,6 +71,20 @@ The **⊞** button opens a 2×2 grid layout with all three planes simultaneously
 
 Clicking in any plane moves the crosshair in the other two planes, allowing linked navigation through the volume. The info bar shows voxel indices and millimetre coordinates for the current crosshair position.
 
+### 3D Segmentation
+
+Segmentation annotations (pixel-level painting) are fully supported for NIfTI volumes across all three planes.
+
+**How it works:**
+
+- All segmentation data is stored as **axial tiles**. Coronal and sagittal views are derived on the fly from the stored axial data — no duplicate storage, and cross-plane consistency is automatic.
+- A segmentation drawn in the axial plane is immediately visible in the coronal and sagittal planes at the corresponding position, and vice versa.
+- The annotation must be a *Segmentation* vector type (type 8). Select it in the annotation type panel, then use the brush, eraser, magic wand, or fill tools to paint directly on the slice.
+
+**Navigation tip:** after drawing in one plane, switch to another plane and scroll the frame slider to the anatomical position where you painted. The 3-Axis mode (⊞) is the most convenient way to confirm cross-plane consistency because all three planes update together.
+
+**Anisotropic volumes:** EXACT correctly handles non-isotropic voxel spacings (e.g. thick-slice CT/MRI where the through-plane resolution is coarser than the in-plane resolution). Tile coordinates are scaled by the actual voxel dimensions, so the displayed segmentation aligns with the underlying anatomy in every plane.
+
 ### Coordinate system
 
 NIfTI volumes are reoriented to **RAS+** (Right–Anterior–Superior) at load time using nibabel's `as_closest_canonical`. This means:

exact/exact/annotations/migrations/0050_segmentation_tile.py

@@ -0,0 +1,30 @@
+# Generated by Django 4.2.18 on 2026-05-10 09:02
+
+from django.db import migrations, models
+import django.db.models.deletion
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('annotations', '0049_annotationtype_multi_frame'),
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name='SegmentationTile',
+            fields=[
+                ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('level', models.IntegerField()),
+                ('tile_x', models.IntegerField()),
+                ('tile_y', models.IntegerField()),
+                ('frame', models.IntegerField(default=0)),
+                ('data', models.BinaryField()),
+                ('annotation', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='segmentation_tiles', to='annotations.annotation')),
+            ],
+            options={
+                'indexes': [models.Index(fields=['annotation', 'level', 'frame'], name='annotations_annotat_9dba0a_idx')],
+                'unique_together': {('annotation', 'level', 'tile_x', 'tile_y', 'frame')},
+            },
+        ),
+    ]

exact/exact/annotations/models.py

@@ -453,6 +453,7 @@ class VECTOR_TYPE():
         POLYGON = 5
         FIXED_SIZE_BOUNDING_BOX = 6
         GLOBAL = 7 #Annotations without a shape that are valid for the whole image
+        SEGMENTATION = 8  # Pixel-level mask stored as compressed tiles
 
     name = models.CharField(max_length=50)
     active = models.BooleanField(default=True)
@@ -494,6 +495,8 @@ def get_vector_type_name(vector_type):
             return 'Multi Line'
         if vector_type is AnnotationType.VECTOR_TYPE.FIXED_SIZE_BOUNDING_BOX:
             return 'Fixed Size Bounding Box'
+        if vector_type is AnnotationType.VECTOR_TYPE.SEGMENTATION:
+            return 'Segmentation'
 
     def validate_vector(self, vector: Union[dict, None]) -> bool:
         """
@@ -525,6 +528,8 @@ def validate_vector(self, vector: Union[dict, None]) -> bool:
             return self._validate_bounding_box(vector)
         if self.vector_type == AnnotationType.VECTOR_TYPE.GLOBAL:
             return self._validate_bounding_box(vector)
+        if self.vector_type == AnnotationType.VECTOR_TYPE.SEGMENTATION:
+            return True  # tile metadata; pixel data lives in SegmentationTile
 
         # No valid vector type given.
         return False
@@ -692,4 +697,31 @@ class MediaFileType(IntEnum):
         related_query_name="uploaded_media_file")
 
     def __str__(self):
-        return self.name + ": " + str(self.file)
+        return self.name + ": " + str(self.file)
+
+
+class SegmentationTile(models.Model):
+    """Stores a single compressed PNG tile of a pixel-level segmentation mask.
+
+    Tiles are addressed by (annotation, level, tile_x, tile_y, frame) and
+    stored as 8-bit palette-mode PNG where each pixel value is a class label
+    (0 = background/transparent). Missing tiles are implicitly empty, giving
+    sparse storage that scales to gigapixel WSIs.
+    """
+
+    class Meta:
+        unique_together = ('annotation', 'level', 'tile_x', 'tile_y', 'frame')
+        indexes = [
+            models.Index(fields=['annotation', 'level', 'frame']),
+        ]
+
+    annotation = models.ForeignKey(
+        Annotation, on_delete=models.CASCADE, related_name='segmentation_tiles')
+    level    = models.IntegerField()   # 0 = full resolution, higher = more zoomed-out
+    tile_x   = models.IntegerField()   # column index in the tile grid at this level
+    tile_y   = models.IntegerField()   # row index
+    frame    = models.IntegerField(default=0)  # z-slice / time index for volumetric images
+    data     = models.BinaryField()    # PNG-encoded palette image (class label per pixel)
+
+    def __str__(self):
+        return f'SegmentationTile ann={self.annotation_id} lv={self.level} ({self.tile_x},{self.tile_y}) frame={self.frame}'

exact/exact/annotations/static/annotations/js/exact-image-viewer.js

@@ -27,8 +27,11 @@ class EXACTViewer {
         this.frame = 1;
 
         this.viewer = this.createViewer(options);
-        this.exact_registration_sync = undefined; 
-        this.browser_sync = undefined; 
+        window.exactOSDViewer = this.viewer;  // expose for segmentationTool
+        window.exactImageId   = this.imageId; // expose for segmentationTool (updated on image switch)
+        window.dispatchEvent(new CustomEvent('exactViewerReady', { detail: this.viewer }));
+        this.exact_registration_sync = undefined;
+        this.browser_sync = undefined;
 
         this.exact_image_sync = new EXACTImageSync(this.imageId, this.gHeaders, this.viewer);
         this.initViewerEventHandler(this.viewer, imageInformation);
@@ -46,6 +49,8 @@ class EXACTViewer {
         this.heatmapToggle = false;
 
         this.currentPlane = 0;
+        window.exactCurrentPlane = 0;
+        this._planeFrameMemory = {};   // plane index → last OSD page (0-indexed)
         this.mprPlanes = null;
         this.mprActive = false;
         this.mprViewers = {};   // planeIdx → { osd, canvas, nFrames, dims }
@@ -747,6 +752,10 @@ class EXACTViewer {
 
     switchPlane(plane) {
         if (!this.mprPlanes || plane === this.currentPlane) return;
+
+        // Remember where we are in the current plane before switching.
+        this._planeFrameMemory[this.currentPlane] = this.viewer.currentPage();
+
         this.currentPlane = plane;
         this.updatePlaneButtons();
 
@@ -757,28 +766,44 @@ class EXACTViewer {
         const nFrames = planeInfo.nFrames;
         const zDim = plane + 1;
 
+        // Restore last-seen frame for this plane (0-indexed), clamped to valid range.
+        const restorePage = Math.min(
+            this._planeFrameMemory[plane] ?? 0,
+            nFrames - 1
+        );
+
         const tileSources = [];
         for (let f = 0; f < nFrames; f++) {
             tileSources.push(`${this.server_url}/images/image/${this.imageId}/${zDim}/${f + 1}/tile/`);
         }
 
-        // Rebuild the frame slider for the new plane's frame count so the
-        // range and value are correct before viewer.open() fires its page event.
+        // Rebuild the frame slider with the restored frame position.
         if (this.frameSlider !== undefined) {
             this.frameSlider.destroy();
             this.frameSlider = undefined;
         }
         if (nFrames > 1) {
             this.frameSlider = new Slider("#frameSlider", {
                 ticks_snap_bounds: 1,
-                value: 1,
+                value: restorePage + 1,   // slider is 1-indexed
                 min: 0,
                 tooltip: 'always',
                 max: nFrames - 1
             });
             this.frameSlider.on('change', this.onFrameSliderChanged.bind(this));
         }
 
+        // Expose current plane globally and notify segmentation tool before
+        // opening the new tile sources, so layers can be torn down cleanly.
+        window.exactCurrentPlane = plane;
+        window.exactCurrentPlaneNFrames = nFrames;
+        window.dispatchEvent(new CustomEvent('exactPlaneChanged', { detail: { plane } }));
+
+        // Navigate to the restored frame once OSD has opened the new tile sources.
+        if (restorePage > 0) {
+            this.viewer.addOnceHandler('open', () => this.viewer.goToPage(restorePage));
+        }
+
         this.viewer.open(tileSources);
     }