8. Reusable flows
Extract the filter, aggregation, and shortfall calculation into a child flow. The parent still reads, cleans, enriches, and routes the report. The complete script is runnable on its own.
incoming = ff.FlowInput("invoices", schema=enriched.schema)
child_minimum = ff.add_flow_parameter(incoming, ff.Parameter("min_quantity", default=8, type="integer"))
child_summary = (
incoming.filter(ff.col("quantity") >= child_minimum)
.group_by("city")
.agg(
ff.col("invoice_id").count().alias("orders"),
ff.col("revenue").sum().alias("revenue"),
ff.col("target").first().alias("target"),
)
)
child_report = child_summary.with_columns((ff.col("target") - ff.col("revenue")).alias("shortfall")).sort("city")
child_report.to_flow_output("city_report")
report_flow = ff.register_flow(child_report, name="Tutorial city summary")
This replaces the inline report calculation from chapters 4–5. FlowInput("invoices", schema=...) declares a named input with columns but no rows. to_flow_output("city_report") names the result the parent can retrieve. register_flow() writes the child YAML and registers it in the catalog immediately.
Feed the child
minimum = ff.add_flow_parameter(enriched, ff.Parameter("min_quantity", default=8, type="integer"))
call = ff.RunFlow(report_flow, invoices=enriched, params={"min_quantity": minimum})
report = call["city_report"]
invoices=enriched binds a frame to the child's input slot. params= passes the parent's threshold to the child's parameter. The names serve different purposes: one carries rows, the other carries a value.
The Gate from chapter 6 now reads report. A RunFlow output is deferred: building it does not run the child; collecting it runs its lineage. The complete script leaves out the chapter 7 writers; chapter 10 puts the whole pipeline together.
Compare thresholds with iterate
scenarios = ff.from_dict({"minimum": [1, 8, 10]})
comparison = ff.RunFlow(
report_flow,
invoices=enriched,
params={"min_quantity": ff.col("minimum")},
param_frame=scenarios,
iterate=True,
)
per_threshold = (
comparison["city_report"]
.group_by("param_min_quantity")
.agg(ff.col("city").count().alias("cities"), ff.col("orders").sum().alias("invoices"))
.sort("param_min_quantity")
)
param_frame supplies one threshold per row. ff.col("minimum") binds that column to the child's min_quantity. With iterate=True, the child runs three times and returns the combined outputs with param_min_quantity and run_index metadata. The script then counts cities and sums invoices per threshold.
Iteration does not partition the invoices
Every run receives the same enriched frame. Only the bound parameter changes. The child filter selects the qualifying invoices; iterate=True itself does not split them into groups.
Check it
child_report.flow_graph.print_tree()
print()
routed.flow_graph.print_tree()
print(report.collect())
print(per_threshold.collect())
The script prints:
● Flow Input (20)
│
● Filter data (21)
│
● Group by (22)
│
● Formula (24)
│
● Sort data (25)
│
● Flow Output (26)
● Read data (11) Supermarket invoices
│
● Drop duplicates (12)
│
● Filter data (13)
│
● Formula (14)
├─┐
● │ Filter data (15)
│ ● Filter data (16)
├─┘
● Union data (17)
│ ● Manual input (18)
├─┘
● Join (19)
│
● Run Flow (27)
│
● Gate (28)
├─┐
● │ Formula (30) [then]
│ ● Formula (32) [else]
├─┘
● Union data (33)
shape: (5, 5)
┌───────────┬────────┬──────────┬─────────┬───────────┐
│ city ┆ orders ┆ revenue ┆ target ┆ shortfall │
│ --- ┆ --- ┆ --- ┆ --- ┆ --- │
│ str ┆ u32 ┆ f64 ┆ f64 ┆ f64 │
╞═══════════╪════════╪══════════╪═════════╪═══════════╡
│ Bago ┆ 59 ┆ 28592.7 ┆ 30000.0 ┆ 1407.3 │
│ Mandalay ┆ 73 ┆ 36937.24 ┆ 35000.0 ┆ -1937.24 │
│ Naypyitaw ┆ 51 ┆ 23733.16 ┆ 25000.0 ┆ 1266.84 │
│ Taunggyi ┆ 63 ┆ 32710.95 ┆ 30000.0 ┆ -2710.95 │
│ Yangon ┆ 68 ┆ 34096.0 ┆ 35000.0 ┆ 904.0 │
└───────────┴────────┴──────────┴─────────┴───────────┘
shape: (3, 3)
┌────────────────────┬────────┬──────────┐
│ param_min_quantity ┆ cities ┆ invoices │
│ --- ┆ --- ┆ --- │
│ i64 ┆ u32 ┆ u32 │
╞════════════════════╪════════╪══════════╡
│ 1 ┆ 5 ┆ 1000 │
│ 8 ┆ 5 ┆ 314 │
│ 10 ┆ 5 ┆ 107 │
└────────────────────┴────────┴──────────┘
The first graph is the child: Flow Input and Flow Output around the filter and aggregation. In the parent, a single Run Flow node takes their place between Join and Gate. The child's report matches chapter 4, and the threshold comparison counts 1,000, 314, and 107 qualifying invoices across the same five cities.
Complete script
Show the complete runnable script
"""Extract the city report into a reusable child flow and compare thresholds."""
import flowfile as ff
# Chapters 1-3: read, clean, and enrich the invoices.
SALES = "https://raw.githubusercontent.com/edwardvaneechoud/flowfile/main/data/templates/supermarket_sales.csv"
sales = ff.read_csv(SALES, description="Supermarket invoices")
valid = sales.unique().filter(flowfile_formula="[quantity] > 0 and [unit_price] > 0")
orders = valid.with_columns(flowfile_formulas=["[unit_price] * [quantity]"], output_column_names=["revenue"])
members = orders.filter(ff.col("customer_type") == "Member")
regulars = orders.filter(ff.col("customer_type") == "Normal")
combined = ff.concat([members, regulars], how="diagonal_relaxed")
targets = ff.from_dict(
{
"city": ["Bago", "Mandalay", "Naypyitaw", "Taunggyi", "Yangon"],
"target": [30000.0, 35000.0, 25000.0, 30000.0, 35000.0],
}
)
enriched = combined.join(targets, on="city", how="left")
# Chapter 8: the report calculation from chapters 4-5 becomes a child flow.
incoming = ff.FlowInput("invoices", schema=enriched.schema)
child_minimum = ff.add_flow_parameter(incoming, ff.Parameter("min_quantity", default=8, type="integer"))
child_summary = (
incoming.filter(ff.col("quantity") >= child_minimum)
.group_by("city")
.agg(
ff.col("invoice_id").count().alias("orders"),
ff.col("revenue").sum().alias("revenue"),
ff.col("target").first().alias("target"),
)
)
child_report = child_summary.with_columns((ff.col("target") - ff.col("revenue")).alias("shortfall")).sort("city")
child_report.to_flow_output("city_report")
report_flow = ff.register_flow(child_report, name="Tutorial city summary")
minimum = ff.add_flow_parameter(enriched, ff.Parameter("min_quantity", default=8, type="integer"))
call = ff.RunFlow(report_flow, invoices=enriched, params={"min_quantity": minimum})
report = call["city_report"]
# Chapter 6: the Gate now reads the child flow's report.
gate = ff.Gate(report, ff.col("shortfall") > 0)
review = gate.then.with_columns(ff.lit("review").alias("route"))
ready = gate.otherwise.with_columns(ff.lit("ready").alias("route"))
routed = ff.concat([review, ready], how="diagonal_relaxed")
scenarios = ff.from_dict({"minimum": [1, 8, 10]})
comparison = ff.RunFlow(
report_flow,
invoices=enriched,
params={"min_quantity": ff.col("minimum")},
param_frame=scenarios,
iterate=True,
)
per_threshold = (
comparison["city_report"]
.group_by("param_min_quantity")
.agg(ff.col("city").count().alias("cities"), ff.col("orders").sum().alias("invoices"))
.sort("param_min_quantity")
)
child_report.flow_graph.print_tree()
print()
routed.flow_graph.print_tree()
print(report.collect())
print(per_threshold.collect())
Recap
The report has a reusable interface: invoices in, one threshold, city report out. Next, add a comparable shortfall percentage using SQL and inspect equivalent custom-logic options.
Next: Custom logic