[pull] master from meshtastic:master

.github/copilot-instructions.md

@@ -0,0 +1,206 @@
+# Copilot Instructions for Meshtastic Python
+
+## Project Overview
+
+This is the Meshtastic Python library and CLI - a Python API for interacting with Meshtastic mesh radio devices. It supports communication via Serial, TCP, and BLE interfaces.
+
+## Technology Stack
+
+- **Language**: Python 3.9 - 3.14
+- **Package Manager**: Poetry
+- **Testing**: pytest with hypothesis for property-based testing
+- **Linting**: pylint
+- **Type Checking**: mypy (working toward strict mode)
+- **Documentation**: pdoc3
+- **License**: GPL-3.0
+
+## Project Structure
+
+```
+meshtastic/           # Main library package
+├── __init__.py       # Core interface classes and pub/sub topics
+├── __main__.py       # CLI entry point
+├── mesh_interface.py # Base interface class for all connection types
+├── serial_interface.py
+├── tcp_interface.py
+├── ble_interface.py
+├── node.py           # Node representation and configuration
+├── protobuf/         # Generated Protocol Buffer files (*_pb2.py, *_pb2.pyi)
+├── tests/            # Unit and integration tests
+├── powermon/         # Power monitoring tools
+└── analysis/         # Data analysis tools
+examples/             # Usage examples
+protobufs/            # Protocol Buffer source definitions
+```
+
+## Coding Standards
+
+### Style Guidelines
+
+- Follow PEP 8 style conventions
+- Use type hints for function parameters and return values
+- Document public functions and classes with docstrings
+- Prefer explicit imports over wildcard imports
+- Use `logging` module instead of print statements for debug output
+
+### Type Annotations
+
+- Add type hints to all new code
+- Use `Optional[T]` for nullable types
+- Use `Dict`, `List`, `Tuple` from `typing` module for Python 3.9 compatibility
+- Protobuf types are in `meshtastic.protobuf.*_pb2` modules
+
+### Naming Conventions
+
+- Classes: `PascalCase` (e.g., `MeshInterface`, `SerialInterface`)
+- Functions/methods: `camelCase` for public API (e.g., `sendText`, `sendData`)
+- Internal functions: `snake_case` with leading underscore (e.g., `_send_packet`)
+- Constants: `UPPER_SNAKE_CASE` (e.g., `BROADCAST_ADDR`, `LOCAL_ADDR`)
+
+### Error Handling
+
+- Use custom exception classes when appropriate (e.g., `MeshInterface.MeshInterfaceError`)
+- Provide meaningful error messages
+- Use `our_exit()` from `meshtastic.util` for CLI exits with error codes
+
+## Testing
+
+### Test Organization
+
+Tests are in `meshtastic/tests/` and use pytest markers:
+
+- `@pytest.mark.unit` - Fast unit tests (default)
+- `@pytest.mark.unitslow` - Slower unit tests
+- `@pytest.mark.int` - Integration tests
+- `@pytest.mark.smoke1` - Single device smoke tests
+- `@pytest.mark.smoke2` - Two device smoke tests
+- `@pytest.mark.smokevirt` - Virtual device smoke tests
+- `@pytest.mark.examples` - Example validation tests
+
+### Running Tests
+
+```bash
+# Run unit tests only (default)
+make test
+# or
+pytest -m unit
+
+# Run all tests
+pytest
+
+# Run with coverage
+make cov
+```
+
+### Writing Tests
+
+- Use `pytest` fixtures from `conftest.py`
+- Use `hypothesis` for property-based testing where appropriate
+- Mock external dependencies (serial ports, network connections)
+- Test file naming: `test_<module_name>.py`
+
+## Pub/Sub Events
+
+The library uses pypubsub for event handling. Key topics:
+
+- `meshtastic.connection.established` - Connection successful
+- `meshtastic.connection.lost` - Connection lost
+- `meshtastic.receive.text(packet)` - Text message received
+- `meshtastic.receive.position(packet)` - Position update received
+- `meshtastic.receive.data.portnum(packet)` - Data packet by port number
+- `meshtastic.node.updated(node)` - Node database changed
+- `meshtastic.log.line(line)` - Raw log line from device
+
+## Protocol Buffers
+
+- Protobuf definitions are in `protobufs/meshtastic/`
+- Generated Python files are in `meshtastic/protobuf/`
+- Never edit `*_pb2.py` or `*_pb2.pyi` files directly
+- Regenerate with: `make protobufs` or `./bin/regen-protobufs.sh`
+
+## Common Patterns
+
+### Creating an Interface
+
+```python
+import meshtastic.serial_interface
+
+# Auto-detect device
+iface = meshtastic.serial_interface.SerialInterface()
+
+# Specific device
+iface = meshtastic.serial_interface.SerialInterface(devPath="/dev/ttyUSB0")
+
+# Always close when done
+iface.close()
+
+# Or use context manager
+with meshtastic.serial_interface.SerialInterface() as iface:
+    iface.sendText("Hello mesh")
+```
+
+### Sending Messages
+
+```python
+# Text message (broadcast)
+iface.sendText("Hello")
+
+# Text message to specific node
+iface.sendText("Hello", destinationId="!abcd1234")
+
+# Binary data
+iface.sendData(data, portNum=portnums_pb2.PRIVATE_APP)
+```
+
+### Subscribing to Events
+
+```python
+from pubsub import pub
+
+def on_receive(packet, interface):
+    print(f"Received: {packet}")
+
+pub.subscribe(on_receive, "meshtastic.receive")
+```
+
+## Development Workflow
+
+1. Install dependencies: `poetry install --all-extras --with dev`
+2. Make changes
+3. Run linting: `poetry run pylint meshtastic examples/`
+4. Run type checking: `poetry run mypy meshtastic/`
+5. Run tests: `poetry run pytest -m unit`
+6. Update documentation if needed
+
+## CLI Development
+
+The CLI is in `meshtastic/__main__.py`. When adding new CLI commands:
+
+- Use argparse for argument parsing
+- Support `--dest` for specifying target node
+- Provide `--help` documentation
+- Handle errors gracefully with meaningful messages
+
+## Dependencies
+
+### Required
+- `pyserial` - Serial port communication
+- `protobuf` - Protocol Buffers
+- `pypubsub` - Pub/sub messaging
+- `bleak` - BLE communication
+- `tabulate` - Table formatting
+- `pyyaml` - YAML config support
+- `requests` - HTTP requests
+
+### Optional (extras)
+- `cli` extra: `pyqrcode`, `print-color`, `dotmap`, `argcomplete`
+- `tunnel` extra: `pytap2`
+- `analysis` extra: `dash`, `pandas`
+
+## Important Notes
+
+- Always test with actual Meshtastic hardware when possible
+- Be mindful of radio regulations in your region
+- The nodedb (`interface.nodes`) is read-only
+- Packet IDs are random 32-bit integers
+- Default timeout is 300 seconds for operations

.github/workflows/update_protobufs.yml

@@ -1,6 +1,9 @@
 name: "Update protobufs"
 on: workflow_dispatch
 
+permissions:
+  contents: write
+
 jobs:
   update-protobufs:
     runs-on: ubuntu-latest
@@ -9,23 +12,34 @@ jobs:
       - name: Checkout code
         uses: actions/checkout@v4
         with:
+          fetch-depth: 0
           submodules: true
 
-      - name: Update Submodule
+      - name: Setup Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Install Poetry
         run: |
-          git pull --recurse-submodules
+          python -m pip install --upgrade pip
+          python -m pip install poetry
+
+      - name: Update protobuf submodule
+        run: |
+          git submodule sync --recursive
+          git submodule update --init --recursive
           git submodule update --remote --recursive
 
       - name: Download nanopb
         run: |
-          wget https://jpa.kapsi.fi/nanopb/download/nanopb-0.4.8-linux-x86.tar.gz
+          curl -L -o nanopb-0.4.8-linux-x86.tar.gz https://jpa.kapsi.fi/nanopb/download/nanopb-0.4.8-linux-x86.tar.gz
           tar xvzf nanopb-0.4.8-linux-x86.tar.gz
           mv nanopb-0.4.8-linux-x86 nanopb-0.4.8
 
-      - name: Install poetry (needed by regen-protobufs.sh)
+      - name: Install Python dependencies
         run: |
-          python -m pip install --upgrade pip
-          pip3 install poetry
+          poetry install --with dev
 
       - name: Re-generate protocol buffers
         run: |
@@ -38,4 +52,9 @@ jobs:
           git remote set-url origin https://x-access-token:${{ secrets.GITHUB_TOKEN }}@github.com/${{ github.repository }}
           git add protobufs
           git add meshtastic/protobuf
-          git commit -m "Update protobuf submodule" && git push || echo "No changes to commit"
+          if [[ -n "$(git status --porcelain)" ]]; then
+            git commit -m "Update protobufs"
+            git push
+          else
+            echo "No changes to commit"
+          fi

meshtastic/__main__.py

@@ -37,6 +37,7 @@
 except ImportError as e:
     have_test = False
 
+import meshtastic.ota
 import meshtastic.util
 import meshtastic.serial_interface
 import meshtastic.tcp_interface
@@ -60,7 +61,7 @@
     have_powermon = False
     powermon_exception = e
     meter = None
-from meshtastic.protobuf import channel_pb2, config_pb2, portnums_pb2, mesh_pb2
+from meshtastic.protobuf import admin_pb2, channel_pb2, config_pb2, portnums_pb2, mesh_pb2
 from meshtastic.version import get_active_version
 
 logger = logging.getLogger(__name__)
@@ -158,11 +159,11 @@ def _printSetting(config_type, uni_name, pref_value, repeated):
         config_values = getattr(config, config_type.name)
         if not wholeField:
             pref_value = getattr(config_values, pref.name)
-            repeated = pref.label == pref.LABEL_REPEATED
+            repeated = _is_repeated_field(pref)
             _printSetting(config_type, uni_name, pref_value, repeated)
         else:
             for field in config_values.ListFields():
-                repeated = field[0].label == field[0].LABEL_REPEATED
+                repeated = _is_repeated_field(field[0])
                 _printSetting(config_type, field[0].name, field[1], repeated)
     else:
         # Always show whole field for remote node
@@ -253,7 +254,7 @@ def setPref(config, comp_name, raw_val) -> bool:
             return False
 
     # repeating fields need to be handled with append, not setattr
-    if pref.label != pref.LABEL_REPEATED:
+    if not _is_repeated_field(pref):
         try:
             if config_type.message_type is not None:
                 config_values = getattr(config_part, config_type.name)
@@ -452,6 +453,41 @@ def onConnected(interface):
             waitForAckNak = True
             interface.getNode(args.dest, False, **getNode_kwargs).rebootOTA()
 
+        if args.ota_update:
+            closeNow = True
+            waitForAckNak = True
+
+            if not isinstance(interface, meshtastic.tcp_interface.TCPInterface):
+                meshtastic.util.our_exit(
+                    "Error: OTA update currently requires a TCP connection to the node (use --host)."
+                )
+
+            ota = meshtastic.ota.ESP32WiFiOTA(args.ota_update, interface.hostname)
+
+            print(f"Triggering OTA update on {interface.hostname}...")
+            interface.getNode(args.dest, False, **getNode_kwargs).startOTA(
+                ota_mode=admin_pb2.OTAMode.OTA_WIFI,
+                ota_file_hash=ota.hash_bytes()
+            )
+
+            print("Waiting for device to reboot into OTA mode...")
+            time.sleep(5)
+
+            retries = 5
+            while retries > 0:
+                try:
+                    ota.update()
+                    break
+
+                except Exception as e:
+                    retries -= 1
+                    if retries == 0:
+                        meshtastic.util.our_exit(f"\nOTA update failed: {e}")
+
+                    time.sleep(2)
+
+            print("\nOTA update completed successfully!")
+
         if args.enter_dfu:
             closeNow = True
             waitForAckNak = True
@@ -1131,6 +1167,14 @@ def subscribe() -> None:
 
     # pub.subscribe(onNode, "meshtastic.node")
 
+def _is_repeated_field(field_desc) -> bool:
+    """Return True if the protobuf field is repeated. 
+       Protobuf 6.31.0 and later use an is_repeated property, while older versions compare against the label field.
+       """
+    if hasattr(field_desc, "is_repeated"):
+        return bool(field_desc.is_repeated)
+    return field_desc.label == field_desc.LABEL_REPEATED
+
 def set_missing_flags_false(config_dict: dict, true_defaults: set[tuple[str, str]]) -> None:
     """Ensure that missing default=True keys are present in the config_dict and set to False."""
     for path in true_defaults:
@@ -1904,10 +1948,18 @@ def addRemoteAdminArgs(parser: argparse.ArgumentParser) -> argparse.ArgumentPars
 
     group.add_argument(
         "--reboot-ota",
-        help="Tell the destination node to reboot into factory firmware (ESP32)",
+        help="Tell the destination node to reboot into factory firmware (ESP32, firmware version <2.7.18)",
         action="store_true",
     )
 
+    group.add_argument(
+        "--ota-update",
+        help="Perform an OTA update on the local node (ESP32, firmware version >=2.7.18, WiFi/TCP only for now). "
+        "Specify the path to the firmware file.",
+        metavar="FIRMWARE_FILE",
+        action="store",
+    )
+
     group.add_argument(
         "--enter-dfu",
         help="Tell the destination node to enter DFU mode (NRF52)",