Add YaRN RoPE configuration support and enhance evaluation parameters

[JamesDeng42]

• Implemented _apply_yarn_rope_config to preserve HF model-specific fields.
• Updated harness_runner to include new flags: apply_chat_template, fewshot_as_multiturn, and gen_kwargs.
• Enhanced server_manager to handle token-ID prompts and logprobs.
• Added unit tests for new features and configurations.

Tests

Added corresponding unit test.

Checklist

Before submitting this PR, please make sure (put X in square brackets):

• I have performed a self-review of my code. For an optional AI review, add the gemini-review label.
• I have necessary comments in my code, particularly in hard-to-understand areas.
• I have run end-to-end tests tests and provided workload links above if applicable.
• I have made or will make corresponding changes to the doc if needed, including adding new documentation pages to the relevant Table of Contents (toctree directive) as explained in our documentation.

[github-actions]

🤖 Hi @JamesDeng42, I've received your request, and I'm working on it now! You can track my progress in the logs for more details.

[codecov]

Codecov Report

❌ Patch coverage is 0% with 12 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...rc/maxtext/checkpoint_conversion/to_huggingface.py	0.00%	12 Missing ⚠️

📢 Thoughts on this report? Let us know!

[github-actions]

🤖 I'm sorry @JamesDeng42, but I was unable to process your request. Please see the logs for more details.