Skip to content

fix: malformed citation urls#8792

Merged
AdityaHegde merged 13 commits intomainfrom
fix/malformed-citation-urls
Feb 24, 2026
Merged

fix: malformed citation urls#8792
AdityaHegde merged 13 commits intomainfrom
fix/malformed-citation-urls

Conversation

@AdityaHegde
Copy link
Collaborator

@AdityaHegde AdityaHegde commented Feb 9, 2026

Sometimes citation urls have additional brackets in the end. The guess is, the response generation gets messed up when the url itself has brackets and it gets appended to the url.

We are changing how citation urls work. Instead of query being referenced by LLM in response, we instead create a url that links to the conversation and the call id. This is then resolved to a query in UI by calling GetAIToolCall.

Closes APP-493

Checklist:

  • Covered by tests
  • Ran it and it works as intended
  • Reviewed the diff before requesting a review
  • Checked for unhandled edge cases
  • Linked the issues it closes
  • Checked if the docs need to be updated. If so, create a separate Linear DOCS issue
  • Intend to cherry-pick into the release branch
  • I'm proud of this work!

@AdityaHegde AdityaHegde marked this pull request as ready for review February 17, 2026 11:45
Copy link
Contributor

@ericpgreen2 ericpgreen2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, this looks good! Here are some findings, which are all individually pretty small. I'll also tag @begelundmuller for the proper backend review.

Issues

1. Missing tool name and message type validation in GetAIToolCall (runtime/server/chat.go:539-545)

The handler finds any message by ID but doesn't verify it's a MessageTypeCall for query_metrics_view. If a caller passes the ID of a different tool or a result message, the handler would try to decode it as a metrics query, producing confusing errors.

// Suggest adding after the FilterByID lookup:
if callMsg.Tool != ai.QueryMetricsViewName {
    return nil, status.Errorf(codes.InvalidArgument, "call %q is not a query_metrics_view call", req.CallId)
}
if callMsg.Type != ai.MessageTypeCall {
    return nil, status.Errorf(codes.InvalidArgument, "message %q is not a tool call", req.CallId)
}

2. Missing gRPC status codes on internal errors (runtime/server/chat.go:547-555)

Errors from UnmarshalMessageContent, WeakDecode, and ToStruct propagate as raw Go errors. gRPC surfaces these as codes.Unknown. Wrap with status.Errorf(codes.Internal, ...) for consistency.

3. Version downgrade in openapi.yaml

v0.81.2v0.80.2 in both OpenAPI files. Appears to be a merge artifact — should be fixed before merging.

4. callId extraction from URL can fail for trailing-slash URLs (enhance-citation-links.ts:18-19, 119)

The regex /-\/ai\/.*?\/call\/.*?\/?$/ allows trailing slashes, but url.pathname.split("/").pop() returns "" for trailing-slash URLs. These two can disagree silently.

Options: either use a capture regex (e.g., /\/-\/ai\/[^/]+\/call\/([^/]+)\/?$/) so the match and extraction are a single operation, or replace both with explicit URL segment parsing. Either way, the current approach of regex-test + separate split().pop() doesn't handle the trailing slash the regex permits.

Suggestions

5. enhance-citation-links.ts is hard to follow as-is

This file currently mixes three concerns in one blob: the Svelte action (DOM click handling, goto, embed awareness), the reactive store plumbing (subscribing to resources + conversation queries), and the URL mapping logic (parsing citation URLs, looking up messages, constructing explore URLs).

Extracting the URL mapping logic — specifically the mapper closure and the citation URL parsing helpers — into a separate file (e.g., citation-url-mapper.ts) would make the code easier to follow and, importantly, would make the mapping logic unit-testable. Right now it's buried inside a derived store inside a closure, so you can't test it without standing up the full reactive context. Good test cases would include: new citation URL → correct callId extraction (with trailing slash), legacy URL with malformed JSON → bracket-stripping fallback, unknown URL → passthrough, missing message → passthrough.

6. Add GetAIToolCall endpoint tests

The test in metrics_view_query_test.go was updated to verify URL format, but there's no test for the GetAIToolCall gRPC handler itself. This is a new API endpoint with access control — it should have at least a happy-path test and error-path tests (conversation not found, call not found, wrong tool type).

7. Add an eval for citation URL format

Using the existing eval framework (newEval() in runtime/ai/ai_test.go), add an eval case like TestAnalystCitationFormat that:

  1. Calls router_agent with a question that triggers query_metrics_view
  2. Asserts the tool result's open_url matches the expected short link format
  3. Completes the conversation and inspects the assistant's markdown for well-formed citation links (no missing parentheses, no truncation)

This would codify expectations for AI citation behavior and catch regressions if the prompt or URL format changes. The .completions.yaml output serves as a reference artifact.

Comment on lines +300 to +305
// GetAIToolCall Resolves an AI tool call to its query arguments. Returns the query_metrics_view arguments for the given tool call, enabling the frontend to build a dashboard URL.
rpc GetAIToolCall(GetAIToolCallRequest) returns (GetAIToolCallResponse) {
option (google.api.http) = {
get: "/v1/instances/{instance_id}/ai/conversations/{conversation_id}/call/{call_id}"
};
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since they're called "messages" in other APIs, shouldn't this be called GetAIMessage (and message_id and so on)

Comment on lines +1241 to +1245
// Response message for RuntimeService.GetAIToolCall
message GetAIToolCallResponse {
// MetricsResolverQuery as JSON
google.protobuf.Struct query = 1;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this return a Message type to keep it generic? And then check tool == "query_metrics_view" and so forth on the frontend?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ya I think that makes sense. We could use this for other use cases in the future as well without the need to add a new API

require.Equal(t, "United States", res.Response)
}

var citationUrlExtractRegex = regexp.MustCompile(`\(\[[^]]+]\(([^)]+)\)\)`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As this is used only once and its a test case, I suggest inlining it as there is no performance benefit to making it a global

Copy link
Collaborator Author

@AdityaHegde AdityaHegde Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed it by using require.Contains

Comment on lines +170 to +173
citationUrlMatches := citationUrlExtractRegex.FindStringSubmatch(agentRes.Response)
require.Len(t, citationUrlMatches, 2)
expectedCitationUrl := fmt.Sprintf(`https://ui.rilldata.com/-/dashboards/bids_metrics/-/ai/%s/call/%s`, s.ID(), calls[2].ID)
require.Equal(t, expectedCitationUrl, citationUrlMatches[1])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this just be require.Contains(t, agentRes.Response, expectedCitationURL)?

}

openURL.Path, err = url.JoinPath(openURL.Path, "-", "open-query")
openURL.Path, err = url.JoinPath(openURL.Path, "-", "ai", sessionID, "call", callID)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe call the route "message" instead of "call"? Not sure.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ya if we change the API to return a generic message then ya, it makes sense. Probably also makes sense to call it /-/ai/<session_id>/message/<message_id>/-/open. This will allow us to add something at /-/ai/<session_id>/message/<message_id> in the future and not have to have obtuse backwards compatibility handling. What say @ericpgreen2 ?

serveSSEUntilClose(w, events)
}

func (s *Server) GetAIToolCall(ctx context.Context, req *runtimev1.GetAIToolCallRequest) (*runtimev1.GetAIToolCallResponse, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a test for this in chat_test.go?

@AdityaHegde AdityaHegde force-pushed the fix/malformed-citation-urls branch from ecf8fc5 to e465713 Compare February 23, 2026 10:30
Copy link
Contributor

@begelundmuller begelundmuller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Backend looks good!

Copy link
Contributor

@ericpgreen2 ericpgreen2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Frontend looks good too 👍

@AdityaHegde AdityaHegde merged commit 945031c into main Feb 24, 2026
16 of 17 checks passed
@AdityaHegde AdityaHegde deleted the fix/malformed-citation-urls branch February 24, 2026 12:56
AdityaHegde added a commit that referenced this pull request Mar 2, 2026
* fix: malformed citation urls

* Try to sanitise the query param

* Convert citation urls to link with conversation and query IDs

* Revert to using call_id

* Add UI redirect

* Fix naviation in citation links

* PR comments

* More PR comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants