Allow definition of custom locales

[sonovice]

For a current project I would like to define a custom locale (let's call it an 'in-house-dialect').
It would be nice to be able to add it, preferably by inheriting the properties from an already available language.

Rough sketch:

pybabel init -i messages.pot -d translations --locale="dialect" --base="en"

[akx]

Yep --

This has been discussed on the Gitter channel and sort-of tracked in #283/#39/#31/#30 (and likely other places).

I'll take this opportunity to paste in my thoughts from Gitter (from July):

So I got this idea while showering: What if we don't do patch overlays, but instead support "locale loaders" like Jinja does for templates? There'd be a global shared default_loader object that'd construct locale objects that you can substitute with your own to support patching and even loading locales from zips and whatnot.
Also, I suppose we should change the primary API for acquiring locales to get_locale or something instead of the Locale constructor or Locale.parse.

def get_locale(identifier, loader=None):
   return (loader or default_loader).get_locale(identifier)

The loader would be free to do caching as it likes.

So with that, your scenario would be implemented by way of a custom loader that can construct a my_slang locale.

[sonovice]

Oops, I've edited my comment a few seconds before you posted yours. It reflects a slightly different approach now. Either way would be fine, but simply initializing a custom language might be easier to handle for the general user.

[kolypto]

In the meanwhile, this snippet can be used to hack locale data:

def hack_babel_core_to_support_custom_locales(custom_locales: dict):
    """ Hack Babel core to make it support custom locale names

    Args:
        custom_locales: Mapping from { custom name => ordinary name }
    """
    from babel.core import get_global

    # In order for Babel to know "en_CUSTOM", we have to hack its database and put our custom 
    # locale names there.
    # This database is pickle-loaded from a .dat file and cached, so we only have to do it once.
    db = get_global('likely_subtags')
    for custom_name in custom_locales:
        db[custom_name] = custom_name

    # Also, monkey-patch the exists() and load() functions that load locale data from 'babel/locale-data'
    import babel.localedata

    # Originals
    o_exists, o_load = babel.localedata.exists, babel.localedata.load

    # Make sure we do not patch twice
    if o_exists.__module__ != __name__:
        # Definitions
        def exists(name):
            # Convert custom names to normalized names
            name = custom_locales.get(name, name)
            return o_exists(name)

        def load(name, merge_inherited=True):
            # Convert custom names to normalized names
            name = custom_locales.get(name, name)
            return o_load(name, merge_inherited)

        # Patch
        babel.localedata.exists = exists
        babel.localedata.load = load

        # See that they actually exist
        for normalized_name in custom_locales.values():
            assert o_exists(normalized_name)

Usage:

hack_babel_core_to_support_custom_locales({
    'en_CUSTOM': 'en',  # custom => original
})

[thihara]

I had to modify @kolypto's patch a little bit to support parsing the locale when running the compile command.

This is the modified version.

def hack_babel_core_to_support_custom_locales(custom_locales: dict):
    """ Hack Babel core to make it support custom locale names

    Based on : https://github.com/python-babel/babel/issues/454

    Patch mechanism provided by @kolypto

    Args:
        custom_locales: Mapping from { custom name => ordinary name }
    """
    from babel.core import get_global

    # In order for Babel to know "en_CUSTOM", we have to hack its database and put our custom
    # locale names there.
    # This database is pickle-loaded from a .dat file and cached, so we only have to do it once.
    db = get_global('likely_subtags')
    for custom_name in custom_locales:
        db[custom_name] = custom_name

    # Also, monkey-patch the exists() and load() functions that load locale data from 'babel/locale-data'
    import babel.localedata

    # Originals
    o_exists, o_load, o_parse_locale = babel.localedata.exists, babel.localedata.load, babel.core.parse_locale

    # Make sure we do not patch twice
    if o_exists.__module__ != __name__:
        # Definitions
        def exists(name):
            # Convert custom names to normalized names
            name = custom_locales.get(name, name)
            return o_exists(name)

        def load(name, merge_inherited=True):
            # Convert custom names to normalized names
            name = custom_locales.get(name, name)
            return o_load(name, merge_inherited)
            # Make sure we do not patch twice

        # Patch
        babel.localedata.exists = exists
        babel.localedata.load = load
    if o_parse_locale.__module__ != __name__:

        # Definitions
        def parse_locale(name, sep='_'):
            # Convert custom names to normalized names
            name = custom_locales.get(name, name)
            return o_parse_locale(name, sep)

        # Patch
        babel.core.parse_locale = parse_locale

        # See that they actually exist
        for normalized_name in custom_locales.values():
            assert o_exists(normalized_name)

I couldn't figure out a way to patch this code when using the compile command from the CLI.

So I invoked compiling functionality from a python script.

def compile_messages():
    hack_babel_core_to_support_custom_locales(CUSTOM_LOCALES)

    # Equivalent of pybabel compile -d locale -f
    compile_command = compile_catalog()
    compile_command.initialize_options()
    compile_command.directory = 'locale'
    compile_command.use_fuzzy = True
    compile_command.finalize_options()
    compile_command.run()

[kolypto]

@lt3Dev sure, by all means, please do!
Let's say the code is under the MIT license or beerware license :D