Datasets and Profiles

Datasets and Profiles provide reusable data collection rule templates that standardize how telemetry is collected across device fleets. Instead of configuring each device's data collection individually, you define a dataset once and assign it to multiple devices.

Definitions

A Dataset is a reusable data collection rule template that defines what data to collect and how to collect it. Each dataset specifies a collection type (Windows Event Logs, DNS Logs, etc.) and the configuration parameters for that type.

A Profile is a grouping layer that composes multiple datasets into a single assignable unit. Profiles allow you to bundle related collection rules and apply them to devices as a set.

Relationship to Devices: Datasets and profiles have a many-to-many relationship with devices. A single dataset can be assigned to multiple devices, and a single device can have multiple datasets assigned to it. This eliminates repetitive per-device configuration and ensures consistent data collection across your fleet.

Processing flow context: Datasets and profiles operate at the device layer of the DataStream processing flow. They govern what data a device collects before it enters preprocessing and pipeline stages.

Provider → Device (dataset rules applied here) → Preprocessing → Pipeline → Postprocessing → Target → Consumer

Dataset Types

Datasets are categorized by the type of data they collect. The available types depend on the device platform.

Windows Event Logs

Collects Windows Event Log entries. Supports two modes:

• Basic: Select from predefined event log channels (Application, Security, System, etc.)
• Custom: Define XPath filter expressions for granular event selection

Windows DNS Logs

Collects DNS query and response logs from Windows DNS servers. Configuration includes DNS query filtering rules and log type selection.

Windows Security Events

Collects Windows Security event data for audit and compliance monitoring.

Windows Firewall Logs

Collects Windows Firewall log entries for network traffic analysis.

Linux System Events

Collects system logs from the Linux syslog daemon. Configurable file path with distribution-specific defaults.

Linux Audit Events

Collects audit logs from the Linux auditd system. Configurable file path for audit log location.

Linux Firewall Events

Collects firewall logs from iptables/nftables. Configurable file path for firewall log location.

Management

Deletion Constraints

A dataset cannot be deleted if it is assigned to a device or included in a profile. Likewise, a profile cannot be deleted if it is assigned to a device. Remove all associations before deleting.

Creating a Dataset

Dataset creation uses a multi-step wizard:

Step 1 — Define Dataset

Enter the dataset name and description.

Step 2 — Configure Dataset

Configure the type-specific collection rules. The configuration interface adapts based on the dataset type:

• Windows Event Logs (Basic or Custom): Select event log channels or enter XPath filter expressions
• Windows DNS Logs: Configure DNS query filters and log type
• Windows Security Events: Configure security event collection parameters
• Windows Firewall Logs: Configure firewall log collection parameters
• Linux System Events: Specify file path for system logs
• Linux Audit Events: Specify file path for audit logs
• Linux Firewall Events: Specify file path for firewall logs

Step 3 — Assign Devices

Select one or more devices to assign this dataset to. The device list supports multi-select with search filtering.

Step 4 — Review

Review the complete dataset configuration summary before creation. Verify assigned devices and collection rules.

Dataset Detail View

After creation, each dataset has a detail page with three tabs:

General Settings Tab

View and edit the dataset name, description, type, and status.

Assigned Devices Tab

View and manage the list of devices assigned to this dataset. Add or remove device assignments.

Dataset Configuration Tab

View and edit the type-specific collection rules for this dataset.

Dataset Operations

• Clone: Create a copy of an existing dataset with all its configuration. The cloned dataset requires a new name and can be modified independently.
• Delete: Remove a dataset. A confirmation modal displays before deletion to prevent accidental removal.

Creating a Profile

Profile creation uses a multi-step wizard:

Step 1 — Define Profile

Enter the profile name and description.

Step 2 — Select Datasets

Select one or more existing datasets to include in this profile. The dataset list supports multi-select with filtering.

Step 3 — Assign Devices

Select one or more devices to assign this profile to. Device assignment is optional and can be configured later.

Step 4 — Review

Review the profile summary including selected datasets and assigned devices before creation.

Profile Detail View

The profile detail page provides access to the profile's general settings, assigned datasets, and assigned devices.

Permissions

Access to datasets and profiles is controlled by the following permission scopes:

Scope	Description
DATASET_READ	View datasets and their configurations
DATASET_CREATE	Create new datasets
DATASET_EDIT	Modify existing datasets and device assignments
DATASET_DELETE	Delete datasets
PROFILE_READ	View profiles and their configurations
PROFILE_CREATE	Create new profiles
PROFILE_EDIT	Modify existing profiles, dataset selection, and device assignments
PROFILE_DELETE	Delete profiles

Device Integration

Datasets connect to devices through the Configure Data Collection workflow. When configuring a device's data collection:

1. A selection drawer displays available datasets and profiles
2. Select one or more datasets or profiles to assign
3. A confirmation modal with a switch control confirms the assignment change
4. The device begins collecting data according to the assigned dataset rules

Exclusive Assignment

A device can be assigned either datasets or a profile, not both. Assigning one type replaces any existing assignment of the other type.

Each device tracks its configuration mode (dataset or profile), determining whether it receives collection rules from individual datasets or from a profile.

Assigned datasets appear in the device's detail view under the Data Configuration tab (see Devices Management) and can be managed from either the device or dataset side of the relationship.