Decide the fate of undocumented script behavior of some modules

[arhadthedev]

There are three dozens of standard modules that can be called via python -m and their documentation doesn't mention it. They can be grouped into five categories:

• kind of smoke tests:

  • codecs: performs stdin:latin1 → utf-8 → latin1 → stdout passthrough edit: it just wraps stdin and stdout then just exits the script (gh-93096: Remove python -m codecs #94233)
  • curses.has_key: "Compare the output of this implementation and the ncurses has_key, on platforms where has_key is already available"
  • pprint: measures performance (gh-92546: Move pprint benchmark into pyperformance #94613 → Add a benchmark based on python -m pprint pyperformance#222)
  • random: evaluates output statistics of supported generators

• full-fledged crossplatform utils for admin-like users and small automation:

  • asyncio: like python but allows to use await in top-level script code
  • cProfile, profile: runs a script under the profiler
  • encodings.rot_13: a stream converter
  • filecmp: a crossplatform file comparison utility
  • fileinput: prints specified files one by another annotating lines with their source
  • http.server: makes a directory available as a site; useful to quickly test a static site with relative links
  • mimetypes: useful for batch processing of files (maybe) (gh-93096: Make mimetypes CLI tool public #93097)
  • modulefinder: the objdump but for Python source files
  • netrc: prints content of .netrc for a current user
  • pdb
  • platform: returns a single line like Windows-10-10.0.19044-SP0; can be useful in automation
  • quopri: a stream converter
  • tabnanny
  • wsgiref.simple_server - the same as http.server but for APIs; pases a single request and exits

• both:

  • base64:
    • a stream converter
    • base64 -t encodes/decodes Aladdin:open sesame and tests if the result is the same as the original (gh-93096: Remove python -m base64 -t #94230)

• demos with no real world application:

  • curses.textpad: shows an input area; when a user closes it, prints the text back
  • ftplib: a simple one-pass FTP downloader (uses ~/.netrc for login)
  • getopt: just passes arguments to getopt() The module is no longer maintained after gh-106535: Soft deprecate the getopt module #105735
  • imaplib: sending emails to a dead end has no sence but can be used to check if a email client works or got broken
  • shlex: parses stdin using shlex() and prints the list into stdout
  • smtplib: a simple e-mail client
  • xmlrpc.server: serves a datetime service

• complex matter; better leave untouched:

  • idlelib.*
  • tkinter.*
  • turtledemo.*
  • pstats

Eggs and to-be-removed modules aren't listed.

We need to decide what to do with all these undocumented categories.

I propose the following:

• move smoke tests into test module with deduplication
• for full-fledged utils, add Command-Line Usage into the docs like in https://docs.python.org/3/library/ast.html#command-line-usage or https://docs.python.org/3/library/trace.html#command-line-usage
• move demos into the docs of the corresponding module

Linked PRs

• gh-93096: Remove -t and -v flags from pickletools cli #131039
• gh-93096: Remove -t and -v flags from pickle cli #131068
• gh-93096: Load doctests in test_pickle #131069
• [3.13] gh-93096: Load doctests in test_pickle (GH-131069) #131080
• [3.12] gh-93096: Load doctests in test_pickle (GH-131069) #131081
• gh-93096: Update and document pickle CLI #131097
• gh-93096: Remove CLI interface for difflib #131099
• gh-93096: Remove run block in heapq #131130
• gh-93096: Load doctests in test_itertools #131133
• [3.13] gh-93096: Load doctests in test_itertools (GH-131133) #131136
• [3.12] gh-93096: Load doctests in test_itertools (GH-131133) #131137
• gh-93096: Move random benchmark into pyperformance #131144
• gh-93096: Update and document pickletools CLI #131273
• gh-93096: fix test_mimetypes.test_guess_type_conflicting_with_mimetypes #131408
• gh-93096: fix test_mimetypes.test_invocation_error tests on iOS ARM64 #132266

[ericvsmith]

pprint also has a rudimentary performance test. See #92546 for a bug, where it wasn't being tested. If we really care about this performance metric, it should be moved to a proper benchmark.

[arhadthedev]

@ericvsmith Thank you, I've added it to the list.