Google Pub/Sub Pull Subscription

    Zoho Analytics allows you to stream messages from existing pull subscriptions in your Google Cloud project. You sign in with Google, and Zoho Analytics reads messages from the subscriptions you list and adds each one to the stream table as it arrives. One connection can feed several stream tables, each reading its own subscriptions.

    Note: To have Google send messages to a Push URL instead, use Google Pub/Sub Push Subscription.

    1. Before You Begin

    • Availability: Available in the Premium plan and above. You can connect up to 5 streaming sources in Premium and up to 20 in Enterprise. To connect more, purchase add-ons.
    • The Project ID of your Google Cloud project.
    • Existing subscriptions with Delivery type set to Pull.
    • A Google account that can read those subscriptions.
    • Messages published as JSON whose keys match the table column names exactly, including case.

    Note: Use a subscription that only Zoho Analytics reads. If another application reads from it, your table can miss messages.

    2. Connecting to Google Pub/Sub

    Follow the below steps to connect:

    1. Click Import Your Data at the top right corner of the Home page.
    2. Select Streaming Data in the Category list, and click the Google Pub/Sub Pull Subscription tile.
    3. If the Import Streaming Data into Zoho Analytics pane appears, click Proceed.
    4. Enter a Workspace Name, if you started from the Home page.
    5. Click Authenticate Google and sign in. The message Authenticated with Google successfully confirms the sign-in.
    6. Enter the Project ID.
    7. Enter the Subscription Names, pressing Enter after each one. You can add up to 3.
    8. Click Next.

    Note:Next stays disabled until Google sign-in succeeds. To use a different account, click Re-authenticate.

    3. Designing the Table

    In Step 2 - Design Table, choose how to define the columns.

    OptionUse it whenWhat happens
    Design table manuallyYou know the keys and data typesYou add each column yourself
    Provide sample dataYou have a sample messageColumns and data types are filled from your JSON
    Fetch data automatically from sourceMessages are arriving in the subscriptionsClick Fetch Table Metadata. Zoho Analytics reads live messages for up to 120 seconds and fills the columns from them

    Design Table Manually

    Use this option when you already know the structure of your messages.

    1. Select Design table manually and click Next.
    2. In the empty grid, click + Add New Column for each JSON key, and enter its Column Name and Data Type.

    Provide Sample Data

    Use this option to build the table from a sample message. Zoho Analytics reads the keys and infers the data type of each column.

    1. Select Provide sample data.
    2. Paste a JSON object or a JSON array of objects in the Provide sample JSON box.
    3. Click Next. The columns appear in Step 3 - Table Details, where you can review them.

    Note: If the JSON is not valid, the message Invalid JSON appears.

    Fetch Data Automatically from Source

    Use this option when messages are arriving in your subscriptions. Zoho Analytics builds the table from live data.

    1. Select Fetch data automatically from source.
    2. Click Fetch Table Metadata. Zoho Analytics reads messages from the subscriptions for up to 120 seconds. To stop, click Cancel.
    3. When a message arrives, the columns are filled automatically.

    [Image: Fetch data automatically from source selected for Google Pub/Sub Pull Subscription, showing the Fetch Table Metadata button and the fetching status.]

    Note: If no message arrives within 120 seconds, the message No data was received for validation appears. Try again or choose another option.

    Reviewing Table Details

    In Step 3 - Table Details:

    1. Enter a unique Table Name.
    2. Review each column:
      • Column Name: Must match the JSON key exactly.
      • Data Type: How the value is stored.
      • Is Mandatory?: Messages missing this key are rejected.
      • Default Value: Used when the key is missing.
      • Is Personal Data?: The column is encrypted.
    3. To add a column, click + Add New Column below the grid.
    4. Click Next to create the table.

    Note: Check the inferred number types. A Positive Number column rejects decimal values, so change it to Decimal Number if your values have decimals.

    4. Validating and Importing Data

    In Step 4 - Validate and Import Data:

    1. Optionally, set the retention period and turn on Enable Historical Data. See Retention and Historical Tables.
    2. Make sure messages are being published to the topics of the subscriptions.
    3. Click Validate & Import Streaming Data. Zoho Analytics reads messages for 120 seconds.

    When a message matches, the status changes to Consuming Streaming Data and the import starts. If validation fails, read the Failure Reason(s), fix the issue, and click Validate Again.

    Note: If you close the wizard before validation succeeds, you can validate later from the Data Sources page.

    5. Managing the Data Source

    In the workspace, click Data Sources in the left navigation pane and select the Google Pub/Sub Pull Subscription source. From here, you can:

    • View each table's subscriptions, Last Fetched Time, and Streaming Status.
    • Validate, Pause, or Resume a table using the action next to its status.
    • Click Pause All or Resume All to pause or resume every table at once.
    • Click Edit Setup to change the Project ID or sign in again with Re-authenticate.
    • Click Add Table to create another table on the same connection, with its own subscriptions.
    • Click the edit icon on a table row to change its subscriptions or retention period.
    • Click the error icon on a table row to read the last failure reason.

    [Image: The Data Sources page for a Google Pub/Sub Pull Subscription source, showing Edit Setup, Add Table, Pause All, and the table rows.]

    Note: A subscription can be read by only one table of the same data source.

    6. Retention and Historical Tables

    • Retention: Rows stay in the stream table for 1 hour by default. To change this, set 30 minutes to 24 hours under Streaming Data Retention Period and click Save. Older rows are removed automatically.
    • Historical table: Turn on Enable Historical Data in Step 4, or the Historical Table toggle on the table row. Zoho Analytics creates {Stream table name} - Historical_Table, which keeps a copy of every incoming row.
    • Turning it off: The historical table and its data stay, but new rows are not added. To delete the historical table, turn it off first.

    Note: Historical table rows count toward your workspace row limit.

    7. Streaming Statuses

    • Yet to be validated: The table is created but not validated. Click Validate.
    • Validating: Zoho Analytics is reading messages for 120 seconds.
    • Consuming: Data is being imported. Click Pause to stop.
    • Paused: The import is paused. Click Resume to restart.
    • Validation Failed: No matching message arrived within 120 seconds. Click Validate.
    • Failed: The table stopped after repeated messages could not be used. Fix the cause and click Resume.

    8. Limits

    • A stream table holds up to 1 million rows or 1 hour of data.
    • Each table can read up to 3 subscriptions.
    • Only pull subscriptions are supported. Topics and push subscriptions cannot be read.
    • Stream tables cannot be copied.
    • Reports refresh about every 30 seconds. Dashboards must be refreshed manually.
    • You can connect up to 5 streaming sources in Premium and up to 20 in Enterprise. To connect more, purchase add-ons.

    9. Frequently Asked Questions

    Why do I see "The '{name}' name specified is not valid"?

    The subscription does not exist or is not a pull subscription. This often happens when a topic name is entered. Enter the name of an existing pull subscription.

    Why do I see "You do not have sufficient permissions"?

    The signed-in Google account cannot read the subscription. Ask your Google Cloud admin for access, then click Re-authenticate.

    Why do I see "Stream data source with same credentials exists already"?

    This connection is already in the workspace. Open it on the Data Sources page and click Add Table.

    Why did my stream move to Failed?

    Many messages in a row could not be used. Their keys did not match, a mandatory key was missing, or values did not fit the column types. Fix the source and click Resume.

    Does pausing a stream delete data?

    No. Existing rows stay until the retention period removes them.

    PREVIOUS

    UP NEXT