Connect your data
Your data lives somewhere else — a database, an object store, a Kafka topic, a REST API, or Google Analytics. This section is the map of every source Flowfile can read from and every sink it can write to, and where each one is configured. Start here to find the right connector, then follow the link to its setup page.
Flowfile connects to external systems in two ways. In the visual editor you add a reader or writer node and point it at a saved connection. In the Python API you call the matching ff.* function. Both paths go through the same stored connections, so a connection you save once in the UI is usable from code, and vice versa.
Connector matrix
Each connector below is read-only, write-only, or both. "Where configured" names the tab on the Connections page (or the node that carries inline settings) where you set it up.
| Connector | Reads | Writes | Where configured | Setup |
|---|---|---|---|---|
| PostgreSQL / MySQL / SQLite | yes | yes | Connections → Database | Databases |
| Cloud storage (S3 / ADLS / GCS) | yes | yes | Connections → Cloud Storage | Cloud storage |
| Kafka / Redpanda | yes | no | Connections → Kafka | Kafka |
| REST API | yes | no | REST API Reader node (inline) | REST APIs |
| Google Analytics 4 | yes | no | Connections → Google Analytics | Google Analytics |
| Data catalog (Delta Lake) | yes | yes | Catalog Reader / Writer nodes | Catalog |
| Saved secrets & connections | — | — | Connections page | Connections · Secrets |
Not in Flowfile Lite
Saved connections and the secrets that back them require the full desktop or server build. The browser-only Flowfile Lite edition has no backend, so none of the connectors above are available there — Lite works with files you load directly in the browser.
Databases
Typed connections to PostgreSQL, MySQL, and SQLite. Both directions are supported: the Database Reader node (ff.read_database) runs a query or reads a whole table, and the Database Writer node (ff.write_database) writes a frame back. Credentials are stored encrypted and referenced by name.
See Databases for the connection form and worked reader/writer examples.
Cloud storage
Read and write Amazon S3, Azure Data Lake Storage (ADLS), and Google Cloud Storage (GCS). Eight authentication methods are available (access_key, iam_role, service_principal, managed_identity, sas_token, aws-cli, env_vars, and service_account), so a connection can use stored keys or delegate to the ambient cloud credentials. Read formats are CSV, Parquet, JSON, Delta, and Iceberg; write formats are CSV, Parquet, JSON, and Delta.
See Cloud storage for provider-specific setup.
Kafka
Consume JSON messages from a Kafka or Redpanda topic through the Kafka Source node (ff.read_kafka). Offsets are tracked broker-side by consumer group, so a flow that runs on a schedule reads only what arrived since the last run. Kafka is read-only — there is no Kafka writer node.
See Kafka for the connection form, security options, and a runnable example.
REST APIs and Google Analytics
The REST API Reader node (ff.read_api) fetches JSON from an HTTP endpoint with configurable authentication and pagination. The Google Analytics reader pulls GA4 property reports through a stored OAuth or service-account connection. Both are read-only.
See REST APIs and Google Analytics.
Catalog
The data catalog is Flowfile's own storage layer, backed by Delta Lake. The Catalog Reader node loads a registered table into a flow and the Catalog Writer node persists a result back as a versioned table. Unlike the external connectors above, the catalog is managed inside Flowfile — see the Catalog section for details.