The App > Design > Integrations > DataStream page forwards your app's DataStream events to external analytics and data platforms such as Segment and Kafka, and can also store them locally in brainCloud for a set period. Use the Event Filter to control which events are sent.
DataStream events are generated when players do things in your app, for example logging in, registering, posting a leaderboard score, joining a tournament, levelling up or buying an item. Your client can also send its own custom events with the DataStream service APIs (CustomTrackEvent, CustomScreenEvent, CustomPageEvent and SendCrashReport).
The page has four panels: Event Filter, Local DataStream, Kafka and Segment. Click a panel to expand it, turn on Enable, and click Save Changes in that panel. Each panel is saved separately. If you try to leave the page with unsaved changes, you are asked to confirm.
DataStream events are only generated when at least one destination (Local DataStream, Kafka or Segment) is enabled. If none are enabled, the DataStream custom event APIs return an error.
Event Filter
The Event Filter suppresses selected event types so they are not sent anywhere, which lowers the number of events your app generates. It applies to all destinations: Local DataStream, Kafka and Segment.
When enabled, use Suppress The Following Event Sub Types to choose the event types to suppress. They are grouped by category, for example Account (Logged In, Registered), Leaderboard (Recorded Score), Tournament, Progression (Levelled Up, Awarded Achievement), Lobby, Group, Chat and User Item events. The selected types are listed under Currently Suppressed Event Sub Types, where you can remove them.
Was Sent Notification events are suppressed by default.
The Event Filter is only available with Deep Data access. If your plan doesn't include it, the panel shows N/A.
Local DataStream
Normally, DataStream events aren't stored in brainCloud; they are only sent to Segment or Kafka. Enabling Local DataStream stores them in a Deep Data collection in brainCloud as well, so brainCloud can use them for richer analytics.
Keep entries for N day(s): how long to keep stored events, from 1 to 730 days (90 by default). Older events are removed automatically.
Local DataStream requires Deep Data access, and normal Deep Data charges apply.
Kafka
Apache Kafka is a distributed streaming platform. When enabled, brainCloud sends each DataStream event to your Kafka topic, where you can feed it into your own analytics, marketing and data warehousing tools.
Topic (required): the Kafka topic to send events to.
Override System-wide Apache Kafka integration: when shown, tick this to send events to your own Kafka cluster instead of the default one, and fill in the connection settings below.
Brokers (required with override): your broker addresses. If you leave out the port, 9092 is used.
Client ID (optional): the client ID brainCloud uses when connecting.
Security Protocol (required with override): PLAINTEXT, SSL, SASL_PLAINTEXT or SASL_SSL.
Bypass Endpoint Identification (SSL and SASL_SSL only): turns off checking that the broker's certificate matches its host name.
SASL Mechanism (SASL protocols only): PLAIN, SCRAM-SHA-256 or SCRAM-SHA-512, plus the Username and Password to authenticate with.
Each event is sent as a JSON message with the player's profile ID as the message key. The message contains appId, userId, eventType (IDENTIFY, TRACK, SCREEN or PAGE), eventLabel, context and properties.
Events are sent on a best-effort basis without retries, so a broker outage can cause events to be lost.
Segment
Segment is a customer data hub. When enabled, brainCloud sends each DataStream event to your Segment source, and you can route it from there to hundreds of analytics, marketing and data warehousing tools.
Segment Project ID: your Segment source ID.
Segment Write Key: your Segment source's write key. Events are only sent when a Write Key is set.
You can find both in the Segment Dashboard under your source's settings. Events are sent to Segment as identify, track, screen and page calls, using the player's profile ID as the Segment user ID.
Permissions
Write access to this page is required to change these settings. Users with read-only access can view the page, but all controls are disabled.