docs: add multimodal request guide for CN and EN#1249
Conversation
Add documentation explaining how users can send multimodal requests to the LightLLM server, covering OpenAI-compatible and Legacy API formats with curl, Python requests, and OpenAI SDK examples.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a comprehensive multimodal request guide in both Chinese and English, detailing how to use OpenAI-compatible and Legacy API formats for image and audio inputs. The documentation covers various integration methods including curl, Python requests, and the OpenAI SDK. Feedback focuses on improving the accuracy of the streaming response JSON examples by adding missing fields such as 'created', 'model', and 'role' to match the server implementation. Additionally, it is suggested to clarify how to handle multiple images within the Legacy format prompt using multiple tags.
| { | ||
| "id": "chatcmpl-xxx", | ||
| "object": "chat.completion.chunk", | ||
| "choices": [ | ||
| { | ||
| "index": 0, | ||
| "delta": { | ||
| "content": "这张" | ||
| }, | ||
| "finish_reason": null | ||
| } | ||
| ] | ||
| } |
There was a problem hiding this comment.
流式响应示例中缺少 created 和 model 字段，且 delta 对象在 LightLLM 的当前实现中通常会包含 role 字段。为了与 lightllm/server/api_openai.py 中的实现保持一致，建议完善此 JSON 示例。
| { | |
| "id": "chatcmpl-xxx", | |
| "object": "chat.completion.chunk", | |
| "choices": [ | |
| { | |
| "index": 0, | |
| "delta": { | |
| "content": "这张" | |
| }, | |
| "finish_reason": null | |
| } | |
| ] | |
| } | |
| { | |
| "id": "chatcmpl-xxx", | |
| "object": "chat.completion.chunk", | |
| "created": 1234567890, | |
| "model": "qwen", | |
| "choices": [ | |
| { | |
| "index": 0, | |
| "delta": { | |
| "role": "assistant", | |
| "content": "这张" | |
| }, | |
| "finish_reason": null | |
| } | |
| ] | |
| } |
| { | ||
| "id": "chatcmpl-xxx", | ||
| "object": "chat.completion.chunk", | ||
| "choices": [ | ||
| { | ||
| "index": 0, | ||
| "delta": { | ||
| "content": "The image" | ||
| }, | ||
| "finish_reason": null | ||
| } | ||
| ] | ||
| } |
There was a problem hiding this comment.
The streaming response example is missing the created and model fields, and the delta object in LightLLM's current implementation typically includes the role field. To be consistent with the implementation in lightllm/server/api_openai.py, it is recommended to update this JSON example.
| { | |
| "id": "chatcmpl-xxx", | |
| "object": "chat.completion.chunk", | |
| "choices": [ | |
| { | |
| "index": 0, | |
| "delta": { | |
| "content": "The image" | |
| }, | |
| "finish_reason": null | |
| } | |
| ] | |
| } | |
| { | |
| "id": "chatcmpl-xxx", | |
| "object": "chat.completion.chunk", | |
| "created": 1234567890, | |
| "model": "qwen", | |
| "choices": [ | |
| { | |
| "index": 0, | |
| "delta": { | |
| "role": "assistant", | |
| "content": "The image" | |
| }, | |
| "finish_reason": null | |
| } | |
| ] | |
| } |
| response = run( | ||
| uris=["https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3.5/demo/CI_Demo/mathv-1327.jpg"], | ||
| query="<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n" | ||
| "<|im_start|>user\n<img></img>\n请描述这张图片的内容。<|im_end|>\n" |
There was a problem hiding this comment.
在 Legacy 格式的多图片请求中，用户需要知道如何在 prompt 中放置多个占位符。建议在此处或“多图片请求”部分补充说明：对于多个输入资源，应在 prompt 中按顺序使用多个 <img></img> 标签。
1 participant
Add documentation explaining how users can send multimodal requests to the LightLLM server, covering OpenAI-compatible and Legacy API formats with curl, Python requests, and OpenAI SDK examples.