BigQuery

Sync Expectations

The following table describes the sync behavior of Fullstory events to BigQuery.

Destination	Sync Interval	Update Window*	Relevant Timestamps	Setup Guide
BigQuery	Approximately processed_time rounded to the next hour + 1 hour	99% stable within 3 hours	event_time updated_time processed_time	BigQuery Help Doc

* Update Window: These are observed aggregates and may vary based on the specifics of the data capture for your site or app.

Data Schema

Events Table Schema

The following table contains the schema for the Fullstory events in BigQuery. For information on the data types, see Google's documentation.

Field	Type	Description
event_id	STRING	The unique identifier for the event.
event_time	TIMESTAMP	The time in UTC that the event occurred.
processed_time	TIMESTAMP	The time in UTC that the event was processed by Fullstory's servers.
updated_time	TIMESTAMP	The time in UTC that event was updated in the database. Set via CURRENT_TIMESTAMP when merging.
device_id	INTEGER	The device ID as defined in the base event model.
session_id	INTEGER	The session ID as defined in the base event model.
view_id	INTEGER	The view ID as defined in the base event model.
event_type	STRING	The type of this event.
event_properties	STRING	A json string containing the associated event properties.
source_type	STRING	The source type of the event.
source_properties	STRING	A json string containing the associated source properties.

The table will be partitioned by day on the event_time column, so you will almost always want to include a filter on event_time in your queries to avoid scanning the entire table. For more info, see the Google documentation on partitioned tables.

Additionally, you may want to consider enabling Physical Billing to reduce the storage costs for this table.

Element Definitions Table Schema

The following table contains the schema for Fullstory named element definitions.

Property	Type	Description
id	STRING	The named element's ID. Use the ID to join on the element_definition_id.
name	STRING	The name of the named element.
description	STRING	The description of the named element.
state	STRING	The state of the named element (e.g., active, archived).
created_time	TIMESTAMP	Timestamp of when the named element was created.
created_by	STRING	The Fullstory user that created the named element.
modified_time	TIMESTAMP	Timestamp of when the named element was last modified.
modified_by	STRING	The Fullstory user that last modified the named element.
updated_time	TIMESTAMP	Timestamp of when this record was last synced.

Event Definitions Table Schema

The following table contains the schema for Fullstory defined event definitions.

Property	Type	Description
id	STRING	The defined event's ID. Use the ID to join on the event_definition_id.
name	STRING	The name of the defined event.
description	STRING	The description of the defined event.
state	STRING	The state of the defined event (e.g., active, archived).
created_time	TIMESTAMP	Timestamp of when the defined event was created.
created_by	STRING	The Fullstory user that created the defined event.
modified_time	TIMESTAMP	Timestamp of when the defined event was last modified.
modified_by	STRING	The Fullstory user that modified the defined event.
updated_time	TIMESTAMP	Timestamp of when this record was last synced.

Page Definitions Table Schema

The following table contains the schema for Fullstory page definitions.

Property	Type	Description
id	STRING	The page ID. Use the ID to join on the page_definition_id.
fs_link_id	STRING	The ID that can be used to construct a url in the Fullstory app, e.g. https://app.fullstory.com/ui/<org_id>/settings/pages/<link_id>
name	STRING	The name of the page.
description	STRING	The customer-entered description of the page.
is_user_defined	BOOLEAN	True if the page was defined by the user, False if defined by a Fullstory algorithm.
state	STRING	The state of the page (e.g., active, archived).
created_time	TIMESTAMP	Timestamp of when the page was created.
created_by	STRING	The user that created the page.
modified_time	TIMESTAMP	Timestamp of when the page was last modified.
modified_by	STRING	The user that modified the page.
updated_time	TIMESTAMP	Timestamp of when this record was synced.

Querying the Events Table

When querying the JSON fields in BigQuery, you should favor parsing with PARSE_JSON in a subquery to avoid the overhead of running JSON_EXTRACT_VALUE which combines a parse and then access via path.

To demonstrate how to query the events table, the following example shows how to count the number of rage clicks broken down by browser and URL for a single day.

SELECT
  JSON_VALUE(source_properties, '$.user_agent.browser') AS browser,
  JSON_VALUE(source_properties, '$.url.path') AS path,
  COUNT(1) AS rage_clicks
FROM (
  SELECT
    event_id,
    event_time,
    processed_time,
    updated_time,
    device_id,
    session_id,
    view_id,
    event_type,
    PARSE_JSON(event_properties, wide_number_mode=>'round') AS event_properties,
    source_type,
    PARSE_JSON(source_properties, wide_number_mode=>'round') AS source_properties
  FROM
    `project.dataset.fullstory_events_<org_id>` ) parsed
WHERE
  DATE(event_time) = '2024-01-01'
  AND event_type = 'click'
  AND CAST(JSON_VALUE(event_properties, '$.fs_rage_count') AS INTEGER) > 0
GROUP BY
  browser,
  path
ORDER BY
  rage_clicks DESC;

Analyze with DBT

To help manage the complexity of these queries, Fullstory provides a DBT package to help you get started with BigQuery. The package includes models for Fullstory events, sessions, and users.