5. Flow parameters
Replace the fixed basket threshold with a typed flow parameter. Keep the aggregation from chapter 4. The complete script contains the complete pipeline.
minimum = ff.add_flow_parameter(enriched, ff.Parameter("min_quantity", default=8, type="integer"))
bulk = enriched.filter(ff.col("quantity") >= minimum)
add_flow_parameter() declares the parameter on the graph and returns the Parameter you pass in. It accepts either a FlowFrame or a FlowGraph: passing enriched.flow_graph instead of enriched declares the same parameter on the same graph.
A Python variable exists in this script. A Parameter declares a named value stored with the flow, so a later run can override it. default=8 is its initial value; type="integer" checks values supplied through the parameter helpers.
Declare the parameter on enriched before building the filter. The filter retains the parameter reference, rather than storing only the number eight.
Change a run
print(report.collect())
ff.set_flow_parameter(report, minimum, 10)
report.flow_graph.run_graph()
print(report.flow_graph.get_node(report.node_id).get_resulting_data().collect())
set_flow_parameter() updates the graph's stored value. For this ordinary lazy frame, collect() still evaluates the plan built earlier. Run the graph explicitly, then read the report node's result to observe the new value.
A parameter is a value
Use it in a comparison, not as a column name. Declaring min_quantity does not create a data column. minimum.default remains eight even after changing the graph's copy with set_flow_parameter().
Check it
The script prints the report at the default of eight, then at ten:
shape: (5, 5)
┌───────────┬────────┬──────────┬─────────┬───────────┐
│ city ┆ orders ┆ revenue ┆ target ┆ shortfall │
│ --- ┆ --- ┆ --- ┆ --- ┆ --- │
│ str ┆ u32 ┆ f64 ┆ f64 ┆ f64 │
╞═══════════╪════════╪══════════╪═════════╪═══════════╡
│ Bago ┆ 59 ┆ 28592.7 ┆ 30000.0 ┆ 1407.3 │
│ Mandalay ┆ 73 ┆ 36937.24 ┆ 35000.0 ┆ -1937.24 │
│ Naypyitaw ┆ 51 ┆ 23733.16 ┆ 25000.0 ┆ 1266.84 │
│ Taunggyi ┆ 63 ┆ 32710.95 ┆ 30000.0 ┆ -2710.95 │
│ Yangon ┆ 68 ┆ 34096.0 ┆ 35000.0 ┆ 904.0 │
└───────────┴────────┴──────────┴─────────┴───────────┘
shape: (5, 5)
┌───────────┬────────┬─────────┬─────────┬───────────┐
│ city ┆ orders ┆ revenue ┆ target ┆ shortfall │
│ --- ┆ --- ┆ --- ┆ --- ┆ --- │
│ str ┆ u32 ┆ f64 ┆ f64 ┆ f64 │
╞═══════════╪════════╪═════════╪═════════╪═══════════╡
│ Bago ┆ 14 ┆ 7334.9 ┆ 30000.0 ┆ 22665.1 │
│ Mandalay ┆ 31 ┆ 16608.6 ┆ 35000.0 ┆ 18391.4 │
│ Naypyitaw ┆ 20 ┆ 10422.1 ┆ 25000.0 ┆ 14577.9 │
│ Taunggyi ┆ 18 ┆ 9752.6 ┆ 30000.0 ┆ 20247.4 │
│ Yangon ┆ 24 ┆ 15643.6 ┆ 35000.0 ┆ 19356.4 │
└───────────┴────────┴─────────┴─────────┴───────────┘
The first report matches chapter 4: its orders column sums to 314 invoices. At ten the same five cities keep 107, and every city falls below its target.
Complete script
Show the complete runnable script
"""Make the basket threshold a flow parameter and change it between runs."""
import flowfile as ff
# Chapters 1-3: read, clean, and enrich the invoices.
SALES = "https://raw.githubusercontent.com/edwardvaneechoud/flowfile/main/data/templates/supermarket_sales.csv"
sales = ff.read_csv(SALES, description="Supermarket invoices")
valid = sales.unique().filter(flowfile_formula="[quantity] > 0 and [unit_price] > 0")
orders = valid.with_columns(flowfile_formulas=["[unit_price] * [quantity]"], output_column_names=["revenue"])
members = orders.filter(ff.col("customer_type") == "Member")
regulars = orders.filter(ff.col("customer_type") == "Normal")
combined = ff.concat([members, regulars], how="diagonal_relaxed")
targets = ff.from_dict(
{
"city": ["Bago", "Mandalay", "Naypyitaw", "Taunggyi", "Yangon"],
"target": [30000.0, 35000.0, 25000.0, 30000.0, 35000.0],
}
)
enriched = combined.join(targets, on="city", how="left")
# Chapter 5: the basket threshold becomes a flow parameter.
minimum = ff.add_flow_parameter(enriched, ff.Parameter("min_quantity", default=8, type="integer"))
bulk = enriched.filter(ff.col("quantity") >= minimum)
# Chapter 4: one report row per city.
summary = bulk.group_by("city").agg(
ff.col("invoice_id").count().alias("orders"),
ff.col("revenue").sum().alias("revenue"),
ff.col("target").first().alias("target"),
)
report = summary.with_columns((ff.col("target") - ff.col("revenue")).alias("shortfall")).sort("city")
print(report.collect())
ff.set_flow_parameter(report, minimum, 10)
report.flow_graph.run_graph()
print(report.flow_graph.get_node(report.node_id).get_resulting_data().collect())
Recap
The saved report now has a typed input instead of a fixed threshold. Next, use the resulting shortfalls to choose a review or ready route for the whole report.
Next: Branching with Gate