[SPARK-58106][DOCS][INFRA] Remove stale references to black

[gaogaotiantian]

What changes were proposed in this pull request?

Remove some stale references to black.

Why are the changes needed?

We have been moved to ruff for a while. branch-4.x and master are using ruff exclusively for formatting, not black. We need to update our docs and some dependency files.

Notice that the --black command in lint-python is also removed. That was already deprecated for a while and was kind of hidden from users. Leaving it available does not gain us anything because it will conflict with ruff. Using black will mess up the format for ruff - it's better to just remove it.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

CI - this should not have any impact on real code.

Was this patch authored or co-authored using generative AI tooling?

No.

[dongjoon-hyun]

Nice, @gaogaotiantian . BTW, please remove this line too in order to be free of black legacy.

spark/dev/tox.ini

Line 19 in 475a771

E203, # Skip as black formatter adds a whitespace around ':'.

[dongjoon-hyun]

+1, LGTM (with one comment).

[gaogaotiantian]

Nice, @gaogaotiantian . BTW, please remove this line too in order to be free of black legacy.

spark/dev/tox.ini

Line 19 in 475a771

E203, # Skip as black formatter adds a whitespace around ':'.

Hi @dongjoon-hyun , thanks for noticing this! tox.ini file is also stale. It was used by flake8 which was replaced by ruff a few months ago too. We do not use flake8 anymore for pyspark. I would propose a PR soon to remove that stale file directly (and the references to flake8 too). I think it should be fine that we leave the comment in the file for this PR as the file would be removed soon?

edit: the PR is proposed - #57258