Why are Polars dataframes converted to Pandas?

[bmaclin58]

I see a lot of indicators are converted to Pandas before any calculations are done on them. For example in z_score_predictive_zones.py
is_polars = isinstance(df, PlDataFrame)
if is_polars:
pdf = df.to_pandas()
else:
pdf = df

is there a reason for this?

When trying to apply the calculations at scale for multiple tickers or for higher interval charts (1m, 3m etc) the speed and efficiency of Polars is going to beat Pandas. If you need it for output so you have cross compatibility with Pandas frameworks, you can covert the dataframe at the end, or pass the series back as the specific dataframe library needed. Is there a reason I am missing that the calculations are done in Pandas?

[MDUYN]

Hey @bmaclin58, honest answer is: there's no technical reason the calculations need Pandas, it's just legacy debt that hasn't been cleaned up yet and I did not yet have the time to pick it up. The newer indicators already prove the numpy-core approach works for both frameworks.

To go even in more details; This issue was addressed in more recent indicators that where added; The '.to_pandas()'
conversion in the Polars dataframe calculations is a known shortcut, not a fundamental design decision. Ideally, I want to replace this.

Most of our complex indicators (swing detection, order blocks, etc.) use sequential/stateful algorithms that can't be vectorized in either Pandas or Polars, the actual computation runs in Python loops over numpy arrays. Some of our newer indicators (e.g., Nadaraya-Watson Envelope, Zero-Lag EMA Envelope, Pulse Mean Accelerator) already follow the correct pattern: extract numpy arrays directly from either DataFrame type, compute in numpy, and write results back without any Pandas conversion.

The fix is to refactor the remaining indicators to follow this same "numpy core" pattern, both pandas DataFrame and polars.DataFrame support .to_numpy() / .values for column extraction, so the computation layer doesn't need Pandas at all. The Pandas overhead is the conversion itself (O(n) copy), not the math.

However, at this stage I am unsure if the '.to_pandas()' conversion also improves the performances if used on polars dataframes.

For simple indicators that use Pandas convenience methods like .ewm() and .rolling(), we'd need to reimplement those as numpy functions, which we've already done in several files (see _sma, _ema, _rma , pulse_mean_accelerator).

We're happy to accept PRs that refactor individual indicators to this pattern. The template to follow is nadaraya_watson_envelope.py or zero_lag_ema_envelope.py.