Public Kafka

Zoho Analytics allows you to stream messages from Apache Kafka topics. Zoho Analytics connects to your Kafka broker, reads messages from the topics you list, and adds each one to the stream table as it arrives. One connection can feed several stream tables, each reading its own topics.

1. Before You Begin

Availability: Available in the Premium plan and above. You can connect up to 5 streaming sources in Premium and up to 20 in Enterprise. To connect more, purchase add-ons.
The broker is reachable over the internet. If it accepts connections only from listed IP addresses, add the Zoho Analytics IP addresses.
The broker uses SASL authentication, and you have a username and password for Zoho Analytics.
The topics exist, and the SASL user can read them.
Messages are JSON whose keys match the table column names exactly, including case.

Note: Use flat JSON. A nested object is stored in a single text column.

2. Connecting to Kafka

Follow the below steps to connect:

Click Import Your Data at the top right corner of the Home page.
Select Streaming Data in the Category list, and click the Kafka tile.
If the Import Streaming Data into Zoho Analytics pane appears, click Proceed.
Enter a Workspace Name, if you started from the Home page.
Under Bootstrap Servers, enter the Host and Port of your broker. To add another broker, click +Add Host.
Enter a Consumer Group ID.
Select the Security Protocol: SASL_PLAINTEXT or SASL_SSL.
Enter the SASL Username and SASL Password.
Enter the Topic Names, pressing Enter after each one. You can add up to 5 topics.
Click Next.

Note: Use a Consumer Group ID that only this connection uses. If another application shares it, your table can miss messages.

3. Designing the Table

In Step 2 - Design Table, choose how to define the columns.

Option	Use it when	What happens
Design table manually	You know the keys and data types	You add each column yourself
Provide sample data	You have a sample message	Columns and data types are filled from your JSON
Fetch data automatically from source	Messages are being published to the topics	Click Fetch Table Metadata. Zoho Analytics reads live messages for up to 120 seconds and fills the columns from them

Design Table Manually

Use this option when you already know the structure of your messages.

Select Design table manually and click Next.
In the empty grid, click + Add New Column for each JSON key, and enter its Column Name and Data Type.

Provide Sample Data

Use this option to build the table from a sample message. Zoho Analytics reads the keys and infers the data type of each column.

Select Provide sample data.
Paste a JSON object or a JSON array of objects in the Provide sample JSON box.
Click Next. The columns appear in Step 3 - Table Details, where you can review them.

Note: If the JSON is not valid, the message Invalid JSON appears.

Fetch Data Automatically from Source

Use this option when messages are being published to your topics. Zoho Analytics builds the table from live data.

Select Fetch data automatically from source.
Click Fetch Table Metadata. Zoho Analytics reads messages from the topics for up to 120 seconds. To stop, click Cancel.
When a message arrives, the columns are filled automatically.

[Image: Fetch data automatically from source selected for Public Kafka, showing the Fetch Table Metadata button and the fetching status.]

Note: If no message arrives within 120 seconds, the message No data was received for validation appears. Try again or choose another option.

Reviewing Table Details

In Step 3 - Table Details:

Enter a unique Table Name.
Review each column:
- Column Name: Must match the JSON key exactly.
- Data Type: How the value is stored.
- Is Mandatory?: Messages missing this key are rejected.
- Default Value: Used when the key is missing.
- Is Personal Data?: The column is encrypted.
To add a column, click + Add New Column below the grid.
Click Next to create the table.

Note: Check the inferred number types. A Positive Number column rejects decimal values, so change it to Decimal Number if your values have decimals.

4. Validating and Importing Data

In Step 4 - Validate and Import Data:

Optionally, set the retention period and turn on Enable Historical Data. See Retention and Historical Tables.
Make sure messages are being published to the topics.
Click Validate & Import Streaming Data. Zoho Analytics reads messages for 120 seconds.

When a message matches, the status changes to Consuming Streaming Data and the import starts. If validation fails, read the Failure Reason(s), fix the issue, and click Validate Again.

Note: If you close the wizard before validation succeeds, you can validate later from the Data Sources page.

5. Managing the Data Source

In the workspace, click Data Sources in the left navigation pane and select the Public Kafka source. From here, you can:

View each table's topics, Last Fetched Time, and Streaming Status.
Validate, Pause, or Resume a table using the action next to its status.
Click Pause All or Resume All to pause or resume every table at once.
Click Edit Setup to change the broker, consumer group, or credentials. Leave the password unchanged to keep it.
Click Add Table to create another table on the same connection, with its own topics.
Click the edit icon on a table row to change its topics or retention period.
Click the error icon on a table row to read the last failure reason.

Note: A topic can be read by only one table of the same data source.

6. Retention and Historical Tables

Retention: Rows stay in the stream table for 1 hour by default. To change this, set 30 minutes to 24 hours under Streaming Data Retention Period and click Save. Older rows are removed automatically.
Historical table: Turn on Enable Historical Data in Step 4, or the Historical Table toggle on the table row. Zoho Analytics creates {Stream table name} - Historical_Table, which keeps a copy of every incoming row.
Turning it off: The historical table and its data stay, but new rows are not added. To delete the historical table, turn it off first.

Note: Historical table rows count toward your workspace row limit.

7. Streaming Statuses

Yet to be validated: The table is created but not validated. Click Validate.
Validating: Zoho Analytics is reading messages for 120 seconds.
Consuming: Data is being imported. Click Pause to stop.
Paused: The import is paused. Click Resume to restart.
Validation Failed: No matching message arrived within 120 seconds. Click Validate.
Failed: The table stopped after repeated messages could not be used. Fix the cause and click Resume.

8. Limits

A stream table holds up to 1 million rows or 1 hour of data.
Each table can read up to 5 topics.
A workspace cannot have two Kafka data sources with the same connection details.
Stream tables cannot be copied.
Reports refresh about every 30 seconds. Dashboards must be refreshed manually.
You can connect up to 5 streaming sources in Premium and up to 20 in Enterprise. To connect more, purchase add-ons.

9. Frequently Asked Questions

Why do I see "The Kafka broker is not reachable"?

Zoho Analytics cannot connect to the host. The broker may be on a private network or behind an IP allow-list. Use a host reachable over the internet, or add the Zoho Analytics IP addresses to the allow-list.

Why do I see "Given host is invalid"?

The host or port is empty or not valid. Enter both Host and Port under Bootstrap Servers.

Why do I see "Stream data source with same credentials exists already"?

A Kafka data source with the same details is already in the workspace. Open it on the Data Sources page and click Add Table.

Why did my stream move to Failed?

Many messages in a row could not be used. Their keys did not match, a mandatory key was missing, or values did not fit the column types. Fix the source and click Resume.

Does pausing a stream delete data?

No. Existing rows stay until the retention period removes them.