Add visualization for Mermaid with Subgraph nesting to the Pocketflow framework #126
Description
A visualization for the graph would help the users to know what would happen. And it would help the designers to detect the weakness in their deigns, So I built this, a new method for the class Flow to visualize the whole graph. More explanation and details and below.
If you just want to use this feature. you should download the mermaid.py , put it into the sourcecode portfolio and add a new method in class Flow like below. See my repository.
class Flow(BaseNode):
def __init__(self,start=None): super().__init__(); self.start_node=start
def start(self,start): self.start_node=start; return start
def get_next_node(self,curr,action):
nxt=curr.successors.get(action or "default")
if not nxt and curr.successors: warnings.warn(f"Flow ends: '{action}' not found in {list(curr.successors)}")
return nxt
def _orch(self,shared,params=None):
curr,p,last_action =copy.copy(self.start_node),(params or {**self.params}),None
while curr: curr.set_params(p); last_action=curr._run(shared); curr=copy.copy(self.get_next_node(curr,last_action))
return last_action
def _run(self,shared): p=self.prep(shared); o=self._orch(shared); return self.post(shared,p,o)
def post(self,shared,prep_res,exec_res): return exec_res
def visualize(self,namespace: dict,*,direction: str="LR",show_default: bool=False,highlight_starts: bool=True,max_nodes: int=1000)->str:
from .mermaid import visualize as _visualize
return _visualize(self,namespace,direction=direction,show_default=show_default,highlight_starts=highlight_starts,max_nodes=max_nodes)And then I will talk about how this feature work. This is still an experimental idea, and I would appreciate more feedback. Whether you are unsatisfied with the generated results or have thoughts on the design philosophy, please leave a comment on this issue.
Just use the new method visualize to generate the mermaid code ,and then you could copy it into .md file to see the result. The method visualize use the extra function to do its work.
We want to use the variable name in the mermaid, but because of Python objects themselves do not know which variable name references them; therefore, the user must explicitly pass in the namespace:namespace=locals() . Do not assign multiple variable names to the same instance. This will trigger a warning and fall back to using the class name as the label for the Mermaid diagram.
visualize Skips node logic execution and exports the Mermaid diagram based purely on the connected graph structure (successors + start_node). But the pocketflow class Flow is dynamic. We define a flow’s subgraph as the nodes and edges reachable from start_node (searching only via successors). However, this inevitably leads to situations where the same object is covered by multiple Flow bodies (especially in cases of complex nesting or cross-layer connections). We adopt a ‘first win’ strategy: an object is placed only in the first Flow container it is encountered in.
Then about the flow, regarding the start, any connection pointing to a Flow is redirected to that Flow’s actual start node. Conversely, the successors of the Flow , which represent action-based transitions after the Flow ends, will ideally originate from the Flow’s internal terminal node (a node with empty successors). If the terminal node cannot be identified (e.g., in cases of a pure loop), we default to treating the flow’s initial node as its decision node and proceed to the next step based on that decision.”
In the end, Here are design philosophy for pocketflow.
- Regarding flow termination, you should route it to a Terminal node, which has no successors. This ensures no errors occur in the existing framework, and our drawing tool will naturally treat it as an endpoint.
- For autonomous decision scenarios involving internal loops, we still recommend having the decision node determine whether to enter a Terminal and end the flow. In cases involving loops, we fallback to starting from the actual starting point. We assume the flow’s initial node is its decision node, so the next step proceeds based on its decision.
- Every flow is an independent entity, and ‘start’ is its unique entry point. Arbitrarily interrupting the flow to enter directly into its internals makes it difficult for automation tools to interpret.
Mermaid Examples, the red is pointing the flow entry node.
flowchart LR
start((Start)) --> n2
subgraph subflow_n0[pipeline]
subgraph subflow_n1[stage1]
n2["stage1_start"]
n4["s1_clean"]
n7["s1_reject"]
n8["s1_validate"]
n12["s1_end"]
n2 --> n4
n4 -->|invalid| n7
n4 -->|valid| n8
n7 --> n12
n8 --> n12
end
subgraph subflow_n3[stage2]
n5["s2_route"]
n9["s2a1"]
n10["s2b1"]
n13["s2a2"]
n14["s2b2"]
n16["s2_merge"]
n17["s2b3"]
n19["s2_end"]
n5 -->|typeA| n9
n5 -->|typeB| n10
n9 --> n13
n10 --> n14
n13 --> n16
n14 --> n17
n16 --> n19
n17 --> n16
end
subgraph subflow_n6[stage3]
n11["s3_format"]
n15["s3_save"]
n18["s3_end"]
n11 --> n15
n15 --> n18
end
n12 -->|done| n5
n19 -->|done| n11
end
classDef pf_true_start stroke:#d33,stroke-width:3px,fill:#fff5f5;
class n2 pf_true_start
class n5 pf_true_start
class n11 pf_true_start
flowchart LR
start((Start)) --> n1
subgraph subflow_n0[outer_flow]
n1["outer_main"]
n4["outer_final"]
subgraph subflow_n2[mid_flow]
n3["mid_entry1"]
n7["mid_other"]
subgraph subflow_n5[core_flow]
n6["core_a"]
n8["core_b"]
n9["core_c"]
n6 --> n8
n8 --> n9
end
n3 --> n6
n9 --> n7
end
n1 --> n3
n7 --> n4
end
classDef pf_true_start stroke:#d33,stroke-width:3px,fill:#fff5f5;
class n1 pf_true_start
class n3 pf_true_start
class n6 pf_true_start
flowchart LR
start((Start)) --> n1
subgraph subflow_n0[outer]
n1["main_router"]
n7["join_node"]
n12["final_node"]
subgraph subflow_n2[flowA]
n6["a1"]
n11["a2"]
n16["a3"]
n6 --> n11
n11 --> n16
end
subgraph subflow_n3[flowB]
n8["b1"]
n13["b2"]
n17["b3"]
n18["b4"]
n8 --> n13
n13 -->|left| n17
n13 -->|right| n18
end
subgraph subflow_n4[flowC]
n9["c1"]
n14["c2"]
n19["c3"]
n9 --> n14
n14 --> n19
n19 --> n14
end
subgraph subflow_n5[flowD]
n10["d1"]
n15["d2"]
subgraph subflow_n20[d_sub_flow]
n21["d_sub1"]
n22["d_sub2"]
n23["d_sub3"]
n21 --> n22
n22 --> n23
end
n10 --> n15
n15 --> n21
end
n1 -->|A| n6
n1 -->|B| n8
n1 -->|C| n9
n1 -->|D| n10
n7 --> n12
n9 -->|exit| n7
n16 -->|done| n7
n17 -->|done| n7
n18 -->|done| n7
n23 -->|done| n7
end
classDef pf_true_start stroke:#d33,stroke-width:3px,fill:#fff5f5;
class n1 pf_true_start
class n6 pf_true_start
class n8 pf_true_start
class n9 pf_true_start
class n10 pf_true_start
class n21 pf_true_start
flowchart LR
start((Start)) --> n2
subgraph subflow_n0[outer]
n3["after"]
subgraph subflow_n1[inner]
n2["a1"]
n4["a2"]
n2 --> n4
n4 --> n2
end
n2 -->|done| n3
end
classDef pf_true_start stroke:#d33,stroke-width:3px,fill:#fff5f5;
class n2 pf_true_start
flowchart LR
start((Start)) --> n1
subgraph subflow_n0[main_flow]
n1["top_decision"]
n2["a_entry"]
n3["b1"]
n4["c_entry"]
n5["a_decision"]
n6["b2"]
n8["a_action1"]
n9["a_action2"]
n10["b3"]
n12["merge_point"]
n14["post_process"]
n16["final_decision"]
n17["success"]
n18["warning"]
subgraph subflow_n7[c_sub_flow]
n11["c_sub1"]
n13["c_sub2"]
n15["c_sub3"]
n11 --> n13
n13 --> n15
end
n1 -->|A| n2
n1 -->|B| n3
n1 -->|C| n4
n2 --> n5
n3 --> n6
n4 --> n11
n5 -->|A1| n8
n5 -->|A2| n9
n6 --> n10
n8 --> n12
n9 --> n12
n10 --> n12
n12 --> n14
n14 --> n16
n15 --> n12
n16 -->|success| n17
n16 -->|warning| n18
end
classDef pf_true_start stroke:#d33,stroke-width:3px,fill:#fff5f5;
class n1 pf_true_start
class n11 pf_true_start