Performance regression for writing multiscale images

[LucaMarconato]

With the new to_multiscale() implementation introduced in this PR, the performance to write multiscale images dropped, as shown in this benchmark (reproducible with this code):

  ┌─────────┬─────────┬──────────┐
  │         │  0.7.2  │ 0.7.3a0  │
  ├─────────┼─────────┼──────────┤
  │ time    │ 41.7s   │ 1.13 min │
  ├─────────┼─────────┼──────────┤
  │ peakmem │ 15.9 GB │ 21.7 GB  │
  └─────────┴─────────┴──────────┘

[LucaMarconato]

We are lucky! Below is shown the impact of creating multiscale labels and multiscale images. The problem seems to occur only for multiscale images, not multiscale labels.

The PR #1068 was introduced to reduce the memory impact of generating multiscale labels, in doing this we changed the default for computing multiscale images as well, but the new method is slower. We can therefore simply keep using the old method for images, and use the new one for labels.

Xenium benchmark writing only a single multiscale labels element

(ome) macbook@MacBook-Pro-64 spatialdata-sandbox % asv run --config xenium_2.0.0_io/asv.conf.json HEAD^!
Couldn't load asv.plugins._mamba_helpers because
No module named 'libmambapy'
· Creating environments
· Discovering benchmarks
·· Uninstalling from virtualenv-py3.13-spatialdata0.7.2-spatialdata-io0.6.0
· Running 4 total benchmarks (1 commits * 2 environments * 2 benchmarks)
[ 0.00%] · For spatialdata-sandbox commit 7dbe3340 <main>:
[ 0.00%] ·· Benchmarking virtualenv-py3.13-spatialdata0.7.2-spatialdata-io0.6.0
[25.00%] ··· Running (write_benchmark.XeniumWriteSuite.time_write--).
[37.50%] ··· write_benchmark.XeniumWriteSuite.peakmem_write                                                                                 11G
[50.00%] ··· write_benchmark.XeniumWriteSuite.time_write                                                                             3.97±0.02s
[50.00%] ·· Building for virtualenv-py3.13-spatialdata0.7.3a0-spatialdata-io0.6.0
[50.00%] ·· Benchmarking virtualenv-py3.13-spatialdata0.7.3a0-spatialdata-io0.6.0
[75.00%] ··· Running (write_benchmark.XeniumWriteSuite.time_write--).
[87.50%] ··· write_benchmark.XeniumWriteSuite.peakmem_write                                                                               7.72G
[100.00%] ··· write_benchmark.XeniumWriteSuite.time_write                                                                             4.16±0.03s

Xenium benchmark writing only a single multi-scale image element (morphology focus)

(ome) macbook@MacBook-Pro-64 spatialdata-sandbox % asv run --config xenium_2.0.0_io/asv.conf.json HEAD^!
Couldn't load asv.plugins._mamba_helpers because
No module named 'libmambapy'
· Creating environments
· Discovering benchmarks
· Running 4 total benchmarks (1 commits * 2 environments * 2 benchmarks)
[ 0.00%] · For spatialdata-sandbox commit 7dbe3340 <main>:
[ 0.00%] ·· Benchmarking virtualenv-py3.13-spatialdata0.7.2-spatialdata-io0.6.0
[25.00%] ··· Running (write_benchmark.XeniumWriteSuite.time_write--).
[37.50%] ··· write_benchmark.XeniumWriteSuite.peakmem_write                                                                               12.9G
[50.00%] ··· write_benchmark.XeniumWriteSuite.time_write                                                                             23.5±0.07s
[50.00%] ·· Benchmarking virtualenv-py3.13-spatialdata0.7.3a0-spatialdata-io0.6.0
[75.00%] ··· Running (write_benchmark.XeniumWriteSuite.time_write--).
[87.50%] ··· write_benchmark.XeniumWriteSuite.peakmem_write                                                                               23.9G
[100.00%] ··· write_benchmark.XeniumWriteSuite.time_write      

[LucaMarconato]

asv confirms that the regression is addressed (reproducible with giovp/spatialdata-sandbox@f086c59, requires a local editable install).

Below I run the comparison considering only the large morphology focus image. The rest of the time difference between a full run and this small run is almost surely due to the Xenium aligned images.

  ┌─────────────────────┬────────────┬───────────────┐                                                                                          
  │     Environment     │ time_write │ peakmem_write │      
  ├─────────────────────┼────────────┼───────────────┤                                                                                          
  │ spatialdata 0.7.2   │ 23.9±0.3s  │ 13.4G         │      
  ├─────────────────────┼────────────┼───────────────┤
  │ spatialdata 0.7.3a0 │ 37.9±0.9s  │ 21.9G         │
  ├─────────────────────┼────────────┼───────────────┤
  │ local (existing)    │ 23.8±0.06s │ 13.3G         │
  └─────────────────────┴────────────┴───────────────┘