Blog

Merkle - GA4 x BigQuery - Daily Fresh

18.12.2024

Alt Text Goes Here
Alt Text Goes Here

Earlier this year, we spoke about why BigQuery is the next step in Digital Maturity for GA4 customers. Since then, Google has updated the export options from GA4 to include the Daily Fresh feature, which is available now in Open Beta to all GA4 360 customers.

The holidays this year and H1 2025 are likely to be the most competitive yet, with inflationary pressures and strained budgets impacting purchasing decisions. Many consumers are expected to cut their spending, with 38% expected to cut back on clothes shopping, and 36% on high-value items over the next six months.

Despite expected decreases in spending on certain items, spending this holiday season is still expected to increase by 7% per shopper. Businesses which leverage data to create personalised experiences and react to market changes will be the winners in peak trading. Here, we explain how Daily Fresh will enable your business to do just this and maximise profitability in this key part of the year.

Source: PwC Holiday Outlook 2024 Holiday Outlook 2024: PwC

What benefits does Daily Fresh bring to the table? 

One of the key benefits to Daily Fresh is that it will be the first BigQuery export that supports the new SLA, guaranteeing the delivery of data by a set-time based on your time-zone. Equally, it will not be impacted by processing delays as opposed to what we see and hear currently with the Daily export, where a table could be populated by 10:00 one day, but 14:00 the next. This removes one of the main challenges to making consistent business decisions.

Not only this, but Daily Fresh can help businesses data maturity and holiday readiness, with significantly faster reporting availability than the Daily export, with tables being available within 30 – 60 minutes. This significantly reduces the lead time to gaining important insights.

Unlike the Streaming export, the Fresh Daily import is not restricted in its schema. It allows marketers to view new user and traffic source information, without having to wait 24 hours for the Daily report. Furthermore, where the Streaming export is a ‘best effort service’, the Daily Fresh export operates on an accuracy basis.

Feature / Point

Schema

Availability

Export Limits

Caveats

SPAM

Audience Evaluation

Attribution Results

Export Cost

Server-Side State

Fresh Daily

Has the same schema as the Daily export

Batched updates are sent throughout the day in 30–60 minute increments. Previous days table is typically available by 5AM the following day *Data processing can still occur up to 72 hours after table population

SLA is not available for XL properties

Only available for ‘Normal’ and ‘Large’ 360 properties

As spam filters are deployed throughout the day, some events will be exported before the filter is deployed

Evaluated without offline and synthetic events

SA360 and app attribution may not be present

N/A

Partial data due to continuous processing every 30- 60 minutes.

Daily

Exports all unsampled event data once per day from the previous day

Exports are typically provided midafternoon the following day, but can encounter delays *Data processing can still occur up to 72 hours after table population 

Standard properties: up to 1M events per day, with filtering options to stay under this limit 360 properties: up to 20B events per day

User attribution data may be delayed by up to 24 hours

Applies spam filters after all events are collected for the day 

Evaluated with all events, including offline and synthetic

Contains SA360 and app attribution results 

N/A

All events from the current data.

Streaming

A more limited schema, without new user and new session traffic source data excluded 

Realtime (~5 minutes) population of tables

No volume limits

New user and new session traffic source data is excluded

Spam events may be present due to when filters are applied

N/A

N/A

A cost of $0.05 per GB (~600,000 GA events) will be incurred through this option to stream the data from GA4 to BigQuery

Differences in dimensions and metrics that rely on server-side state like engaged_sessions and last_purchase_date

Considerations

Daily Fresh can help businesses increase their data maturity level and the richness of data available. There are, however, some discrepancies between the different export options which could impact reporting. Marketers must understand these differences to avoid confusion caused by varying report results, particularly when choosing an export method or using different exports across reports.

Many of these discrepancies arise from how and when these exports are processed. Such as differences in event counts between Daily vs Fresh Daily because there are different events that may be dropped due to the timing of enforcement. When compared to streaming, there can be substantial (20%) differences between the intraday and streamed tables because of streaming being a best-effort service.

Additionally, sub-properties and roll up properties are not covered by the Fresh Daily SLA, and we can detect that more spam events might be present in streaming data (Intra Day Tables). 

Which export should you use?

With these taken into consideration leveraging the right export for your use case could increase your businesses access to enrich data, in a timely manner and enable new insights that can be actioned on to improve; analysis, reporting and marketing strategies. So which export is right for you?

Do you need access to data as quickly as possible?

If so, the streaming export will likely be the best option for you, however, please note that this option does come with an additional cost that needs to be taken into consideration of $0.05 per GB. If you can afford to up to an hour for data however, we would recommend you begin transitioning to Daily Fresh data. 

Do you require the most complete dataset for attribution?

Then we would recommend you leverage the Daily export for this, as it will guarantee marketers have the fullest attribution applied. If neither of the above applies, and even if they do, we recommend you begin implementing Daily Fresh now so that you can take advantage of the SLA as soon as it becomes available. As well as providing a ‘best of both’ worlds of accurate and reliable reporting, whilst also maintaining a comprehensive dataset.

You might also like: