Human in the loop

[remi-braun]

Hello,

I must say I am a bit surprised by the way this library is now evolving.
xarray-spatial was stalled for a couple of years and now it seems development has become crazy, with a multitude of new features appearing almost very day.

You kindly added the Claude repository, so you 100% assume the extensive use generative AI, thanks for being so transparent. However, with this coding pace I must ask: is there still a human in the loop in xarray-spatial or is this lib has become a testing repo for Claude's ability in geospatial?
Moreover, other questions are emerging:

• What is the point of recoding everything here, moreover on your own? What is exactly the scope of this library?
• Don't you create an illusion of completeness?
• What about fragmentation?
• What about the bus number of this library ? Aren't you scared it is now less than 1 as you delegate most of your coding to Claude?
• Do you have any safeguard to avoid these issues?

Best Regards,

Rémi

[brendancol]

@remi-braun thanks for the comments, questions, and frankly for noticing the changes to xarray-spatial. Happy to answer them one by one. Also, thanks for your thoughtful comments on the hillshade discrepancies from a while back on issue #748 .

I don't mean to distract... but let's start with another question first...why did it take so long for me / others to fix issue #748 you were involved in? The issue was filed in 2022 by @thuydotm, 2023 I was triaging issues but something must have distracted...2024 you posted an excellent comment...2026 fixes were made, partly using help from Claude and we merged a fix. This is NOT an ideal timeline for problem to solution to release!!

The reason to why such a timeline, like you pointed out, is that xarray-spatial stalled for a while. It stalled because I personally was unable to support the project due a few things:

• In 2021, xarray-spatial received some funding from a larger company and an aggressive scope that I was optimistic my company's team could complete. We got a lot of good stuff done, but delivered subpar distributed reprojection using dask.arrays. Small, inmemory stuff was easy...but basically irrelevant...and we had accuracy trouble on the larger-than-memory outputs. We lost that funding and I was eventually forced to downsize the team. I had designed the company to be profitable without investors but I found myself with jagged revenue and trouble making payroll. Those failures all resulted from my decisions, including the staff hiring, hours estimates, etc.
• In late 2022, 6 weeks before issue Discrepancy between xarray-spatial hillshade and GDAL hillshade #748 was filed, my first child was born. My pattern of open source contribution at night and on weekends was on longer "temporally viable".
• I was generally depressed about the state of xarray-spatial, and the failed engagement from 2021. Opening the project would give me anxiety. I was still allocating what I could to support the work, but issues simply weren't getting completed or released. Any time I had was allocated to making sure payroll could be met for remaining employees by doing paid work for clients...mostly closed source.

This gives you a bit of context about my perspective on why the project stalled. Let me offer a personal apology that your contribution to the project wasn't better integrated and integrated sooner.

You kindly added the Claude repository, so you 100% assume the extensive use generative AI, thanks for being so transparent.

You're welcome, but I added the Claude files so I myself could share them on the different machines for testing, not transparency. It's also nice that other contributors can use them and I was also excited about hearing the military / US government banned Claude...

is there still a human in the loop in xarray-spatial or is this lib has become a testing repo for Claude's ability in geospatial?

Yes, there is a human in the loop, and it's me. Nope, xarray-spatial is not a "testing repo"...although any software endeavor is going to "test" the ability of its contributors, human or agent, to accomplish goals in its given domain.

What is the point of recoding everything here, moreover on your own?

Is it fair if I rephrased the question as: "Why reimplement things that already have implementations folks use from Python and where do you find the motivation to so?"

I wanted a geospatial stack in Python that doesn't involve GDAL because I have personally found GDAL difficult to install and extend. The world owes GDAL a huge "thank you", and I'm not saying that "write once, and wrap for all other languages" is a bad idea...but I wanted a library for geospatial python that didn't delegate heavy lifting to C/C++ extensions. I wanted a toolbox of geospatial "ufuncs" that didn't create new data structures (e.g. class MyNewRasterContainer(): pass) and I wanted it written using technologies I was excited about around 2018 or 2019 (xarray, numba, dask and later cupy). A few years before, I'd been working on datashader and always wanted more GIS-focused aggregation functions, and it was clear that a separate library would be better that bulking out datashader with too much geo.

Let's be clear, there are redundancies that can't simply be explained by, "I don't like GDAL". xrspatial.hydro overlaps with pysheds...a great library with numba integration, but lacks focus on xarray, cupy or dask. Mapclassify is awesome, but focuses more on vector data than raster...and then there are many libraries that do similar things, but I don't like their spellings / interfaces. Zonal tools would be a good example of that.

There may be many other popular tools I'm unfamiliar with that make features of xarray-spatial redundant, but its not my goal to duplicate for duplication's sake.

To address your comment, "...moreover on your own"...yes its hard. I don't have institutional backing and I contribute a lot of unpaid personal time, but I do love it again now and want to increase momentum.

What is exactly the scope of this library?

There is not "exact" scope, but I do have a some guiding rules:

• No GDAL or GDAL wrappers allowed (e.g. no rasterio, rioxarray, geocube, etc.). Again, great projects...no beef, but this library passes on "goodle".
• Numba for heavy lifting where needed.
• Dask for horizonal scaling
• Generally, no new data structures for folks to learn...mostly just xarray.DataArrays and xarray.Datasets as inputs and outputs...
• The focus is geospatial raster processing, not geospatial vector processing, so we support rasterize and polygonize, but not trying to replace cuSpatial, geopandas, etc.

Don't you create an illusion of completeness?

I don't fully understand this question. Do you feel the library is asserts a claim of "geospatial completeness" or just "geospatial raster completeness"? My hope is that neither because we haven't even made it to 1.0.0 release yet. There is no stable API yet for xarray-spatial. I hope to complete a 1.0.0 release when all existing tools are fully validated for accuracy, performance, security and backend array parity (numpy, dask, cupy, dask+cupy).

What about fragmentation?

Do you mean within the geospatial ecosystem, python, or xarray more specifically?

What about the bus number of this library? Aren't you scared it is now less than 1 as you delegate most of your coding to Claude?

I'm not familiar with "bus number". Happy to learn though if you can provide more details. Claude doesn't scare me, but the issue #748 timeline does scare me. Core developers of popular libraries deciding to have more well rounded lives scare me too.

Do you have any safeguard to avoid these issues?

Not sure about "these issues", but the last few releases of xarray-spatial have been focused on security updates and accuracy updates. Each day I usually am able to run a few modules through security, performance, or accuracy sweeps. I've been finding many CRITICAL, and HIGH category bugs and fixing as fast as possible. Most of the recent frenetic pace has been fixing performance and security bugs...but there are many new features in 2026, and my hope is that "going fast fixes going fast".