Test scenario definition

In the current release, it is possible to supply a configuration file in JSON format for running your tests.

In the following sections, we will provide a detailed explanation of the structure and all parameters in the JSON configuration file.

Structure

The JSON configuration file consists of:

ByteBlower server address (and optionally a ByteBlower meeting point)
List of port definitions: Where you want to send traffic from/to
List of flow definitions: We currently support UDP frame blasting and stateful TCP (HTTP) flows.
(optional) Reporting parameters
Test-specific parameters

A quick short reference for the structure:

{
    "server": "<bb_server_name_or_ip:str>",
    "meeting_point": "<bb_meeting_point_name_or_ip:str>",
    "ports": [
        {
            "name": "<port_name:str>",
            "interface": "<bb_interface_name:str>",
            "mac": "<mac_address:str>",
            "uuid": "<endpoint_uuid:str>",
            "ipv4": "<ipv4_address:str>|dhcp|true",
            "netmask": "<ipv4_netmask:str>",
            "gateway": "<ipv4_gateway:str>",
            "nat": "<enable_nat_resolution:bool>",
            "ipv6": "dhcp|slaac|true|<ipv6_address_list:list>",
            "port_group": [
                "<port_group:str>"
            ]
        }
    ],
    "flows": [
        {
            "name": "<flow_name:str>",
            "source": {
                "port_group": [
                    "<source_group:str>"
                ]
            },
            "destination": {
                "port_group": [
                    "<destination_group:str>"
                ]
            },
            "add_reverse_direction": "<add_reverse_direction_flow:bool>",
            "type": "<flow_type:str>",
            "ecn": "<ecn_code_point:str|int>",
            "dscp": "<dscp_code_point:str|int>",
            "initial_time_to_wait": "<initial_time_to_wait:float>"
        }
    ],
    "report": {
        "html": "<enable_html_reporting:bool>",
        "json": "<enable_json_reporting:bool>",
        "junit_xml": "<enable_junit_xml_reporting:bool>"
    },
    "maximum_run_time": "<scenario_max_run_time:float>"
}

The current release supports UDP frame blasting and stateful TCP (HTTP) flows. Each type of flow has some additional specific parameters as follows:

UDP flow specific parameters

{
    "frame_size": "<frame_size_without_crc:int>",
    "bitrate": "<flow_bitrate:float>",
    "frame_rate": "<flow_frame_rate:float>",
    "duration": "<flow_duration:float>",
    "number_of_frames": "<number_of_frames:float>",
    "nat_keep_alive": "<activate_nat_keep_alive:bool>",
    "analysis": {
        "latency": "<enable_latency_analysis:bool>",
        "max_loss_percentage": "<max_loss_percentage:float>",
        "max_threshold_latency": "<max_threshold_latency:float>",
        "quantile": "<quantile:float>"
    }
}

TCP (HTTP) flow specific parameters

{
    "tcp_server_port": "<tcp_server_port_number:int>",
    "tcp_client_port": "<tcp_client_port_number:int>",
    "duration": "<request_duration:float>",
    "request_size": "<request_size:float>",
    "maximum_bitrate": "<maximum_bitrate:float>",
    "receive_window_scaling": "<receive_window_scaling:int>",
    "slow_start_threshold": "<slow_start_threshold:int>",
    "enable_l4s": "<enable_l4s:bool>"
}

Port capture configuration

A port capture can be enabled inside the JSON configuration of a port:

{
  "capture": {
    "enabled": "<enable_port_capture:bool>",
    "filter": "<capture_filter:str>",
    "filename": "<filename:str>"
  }
}

When enabled, the framework will create a PCAP file for that port. If no explicit capture filter is configured, the framework applies a default receive-side filter based on the configured port addressing.

JSON schema

The complete structure and documentation of the file is available in Configuration file JSON schema and documented below.

Test configuration parameters

https://api.byteblower.com/test-framework/json/cli-config-schema.json
Root structure of the JSON configuration file for the ByteBlower Test Framework command-line interface.
type	object
properties
server	ByteBlower server address
meeting_point	ByteBlower meeting point address
ports	Port definitions
flow	Flow definition list
maximum_run_time	Maximum run time
report	Report configuration parameters
additionalProperties	True

ByteBlower server address

Host name or IP address of the ByteBlower Server.
type	string
oneOf	format	hostname
	format	ipv4
	format	ipv6

ByteBlower meeting point address

Host name or IP address of the ByteBlower meeting point.
Added in version 1.2.0: Support for ByteBlower Endpoint.
type	string
oneOf	format	hostname
	format	ipv4
	format	ipv6

Port definitions

Flat list of ports.
type	array
items	Configuration parameters for a ByteBlower Port or Endpoint

Flow definition list

Flat list of flow definitions
type	array
items	oneOf	Frame blasting flow
		HTTP flow parameters

Maximum run time

Maximum run time of the scenario in seconds
type	number
minimum	0

Report configuration parameters

type	object
properties
html	Create HTML report
	Enable/Disable reporting in HTML format
	type	boolean
json	Create JSON report
	Enable/Disable reporting in JSON format
	type	boolean
junit_xml	Create JUnit XML report
	Enable/Disable reporting in JUnit XML format
	type	boolean

Port group

Definition of a port group
type	string

Configuration parameters for a ByteBlower Port or Endpoint

Configuration parameters for a ByteBlower Port or Endpoint
type	object
properties
name	Name of this ByteBlower traffic endpoint.
	(a friendly name to identify this endpoint).
	type	string
port_group	List of port groups where this port belongs to.
	Used to identify to which flows this port will be part of (either as source or destination).
	type	array
	items	Port group
interface	Name of the physical ByteBlower interface where this ByteBlower port is created.
	ByteBlower Port only. Mutually exclusive with `"uuid"`.
	type	string
	pattern	^(nontrunk[-][0-9]+\|trunk[-][0-9]+[-][0-9]+)$
uuid	Device identifier of the ByteBlower Endpoint.
	ByteBlower Endpoint only. Mutually exclusive with `"interface"`. Note Using an Endpoint also requires the `"meeting_point"` to be provided and `"ipv4"` or `"ipv6"` set to `true` for this port configuration. Added in version 1.2.0: Support for ByteBlower Endpoint.
	type	string
	anyOf	format	uuid
mac	MAC address for the ByteBlower port
	ByteBlower Port only. Default: Auto-generated MAC address Note When using Byteblower Endpoint, the device’s mac address will be used
	type	string
	examples	00:1A:2B:3C:4D:5E
		00-1A-2B-3C-4D-5E
		001A.2B3C.4D5E
	pattern	^([0-9A-Fa-f]{2}([-:])){5}[0-9A-Fa-f]{2}$\|^([0-9A-Fa-f]{4}\.){2}[0-9A-Fa-f]{4}$
vlans	List of VLAN (stack) configurations.
	Note Not supported by ByteBlower Endpoint.
	type	array
	items	VLAN (stack) configuration
capture	Port capture configuration
	Configure receive-side packet capture on this port.
	Port capture stores the traffic received on this ByteBlower Port in a PCAP file. Note Port capture is a receive-side feature. It does not capture transmitted traffic. Warning Packet capture can be resource intensive, especially with broad filters.
	type	object
	properties
	enabled	Enable packet capture on this port.
		Defaults to `false`.
		type	boolean
	filter	Optional BPF filter string for the capture.
		If omitted, the framework applies a default receive-side filter based on the configured port addressing.
		type	string
	filename	Optional output filename for the generated PCAP file.
		If omitted, the framework generates a default PCAP filename. By default, the capture file is stored relatively to the report output folder. Use an absolute path for the capture filename to override this.
		type	string
ipv4	IPv4 addressing method
	Enable IPv4 on this traffic endpoint.
	Supports the following values: IPv4 address for static IPv4 IP address (ByteBlower Port only). `"dhcp"` for dynamic IPv4 address allocation (ByteBlower Port only). `true`: To use the IPv4 address(es) of the system (ByteBlower Endpoint only). Default: `"dhcp"` (ByteBlower Port) or undefined (ByteBlower Endpoint). Note For ByteBlower Endpoint, you must specify either `"ipv4": true` or `"ipv6": true`. Changed in version 1.2.0: Support for ByteBlower Endpoint.
	oneOf	type	string
		oneOf	IPv4 automatic address configuration
			enum	dhcp
			IPv4 address for the ByteBlower Port
			format	ipv4
		Use IPv4 on the ByteBlower Endpoint
		type	boolean
netmask	Netmask of the IPv4 network.
	Default: Assigned by DHCP or `255.255.255.0` for static IPv4 address configuration. Note Not supported by ByteBlower Endpoint.
	type	string
	format	ipv4
gateway	Gateway of the IPv4/IPv6 network.
	Set the (default) gateway for IPv4 network or preferred router for IPv6 network Default: Assigned by DHCP or none for static IPv4 address configuration. Note Not supported by ByteBlower Endpoint. Changed in version 1.4.0: Support for manual IPv6 addresses.
	type	string
	oneOf	format	ipv4
		format	ipv6
nat	Define whether this port is located behind a NAT gateway.
	Defaults to `false`. Note Enabled by default for ByteBlower Endpoint.
	type	boolean
ipv6	IPv6 addressing method
	Enable IPv6 on this traffic endpoint.
	Supports the following values: `"dhcp"`: to let the port perform DHCPv6 (ByteBlower Port only) `"slaac"`: to let the port perform stateless address auto-configuration (ByteBlower Port only). IPv6 manual address list: List of manual IPv6 addresses. (ByteBlower Port only) Each address definition consists of: `"address"`: string of IPv6 address or IPv6 address with prefix length. `"prefix_length"`: string or integer defining the prefix length. Only used when the `"address"` field does not contain the prefix length. Defaults to `DEFAULT_PREFIX_LENGTH` (`64`). `true`: To use the IPv6 address(es) of the system (ByteBlower Endpoint only). Default: undefined. Note For ByteBlower Endpoint, you must specify either `"ipv4": true` or `"ipv6": true`. Changed in version 1.4.0: Support for manual IPv6 addresses. Changed in version 1.2.0: Support for ByteBlower Endpoint.
	oneOf	IPv6 automatic address configuration
		type	string
		enum	dhcp, slaac
		List of IPv6 addresses
		type	array
		items	type	object
			properties
			address	IPv6 address
				type	string
				format	ipv6
			prefix	IPv6 address prefix length (0-128)
				maximum	128
				minimum	0
				oneOf	type	string
					type	integer
		Use IPv6 on the ByteBlower Endpoint
		type	boolean
firewall	Define whether this port is located behind a gateway that blocks IPv6 inbound traffic.
	Defaults to `false`. Note IPv6 only. When enabled, inbound traffic will be allowed. Enabled by default for ByteBlower Endpoint. Added in version 1.4.0: Support for IPv6 firewall discovery.
	type	boolean

VLAN (stack) configuration

type	object
properties
protocol_id	VLAN protocol ID (TPID).
	Default value is: `"0x8100"` for a single VLAN configuration `"0x88a8"` for the outer VLAN(s) in a VLAN stack configuration `"0x8100"` for the most inner VLAN in a VLAN stack configuration. Note Configuration of the VLAN protocol ID (TPID) requires ByteBlower Server version >= 2.20.0.
	type	integer
	maximum	65535
	minimum	0
id	VLAN ID (12-bit; 0-4095)
	type	integer
	maximum	4095
	minimum	0
priority	3 bits defining the VLAN PCP bits. Default: 0
	type	integer
	maximum	3
	minimum	0
drop_eligible	1-bit flag. Default: 0
	type	integer
	maximum	1
	minimum	0

Flow definition

type	object
properties
name	Name of this Flow, defaults to auto-generated name when set to `none`
	type	string
source	Specifying the source of transmitted data for this flow
	type	object
	properties
	port_group	List of port groups specifying the transmitting ports
		type	array
		items	Port group
	additionalProperties	False
destination	Specifying the destination for receiving data for this flow
	type	object
	properties
	port_group	List of port groups specifying the receiving ports
		type	array
		items	Port group
	additionalProperties	False
type	Type of the flow. Defines which specific flow parameters will be possible.
	type	string
	enum	frame_blasting, http
ecn	IP ECN
	Explicit Congestion Notification code point. Defaults to `DEFAULT_IP_ECN` (`0x00`)
	oneOf	type	integer
		maximum	3
		minimum	0
		type	string
dscp	IP DSCP
	Differentiated Services Code Point. Defaults to `DEFAULT_IP_DSCP` (`0x00`)
	oneOf	type	integer
		maximum	63
		minimum	0
		type	string
add_reverse_direction	Enables adding a flow in the reverse direction of the original flow.
	The reverse flow uses the same configuration as the original. Defaults to `false`
	type	boolean
initial_time_to_wait	Initial time to wait to start the flow in seconds.
	Defaults to 0 (start immediately)
	type	number
	minimum	0

Frame blasting flow

Specific parameters for frame blasting flow. The `type` MUST be defined as `frame_blasting`.
type	object
allOf	Flow definition
	properties
	bitrate	Rate at which the bits are transmitted (in bits per second).
		Excludes the VLAN tag bytes (when applicable), mutual exclusive with `frame_rate`, defaults to `none`
		type	number
		exclusiveMinimum	0
	frame_size	Frame size in Bytes without CRC, defaults to `none`
		type	integer
		minimum	42
	frame_rate	Rate at which the frames are transmitted (in frames per second).
		Mutual exclusive with `bitrate`. Defaults to `DEFAULT_FRAME_RATE` (`100`) when `bitrate` is not provided.
		type	number
		exclusiveMinimum	0
	number_of_frames	Number of frames to transmit.
		Defaults to `DEFAULT_NUMBER_OF_FRAMES` (`none` == defined by scenario maximum run time).
		type	integer
		exclusiveMinimum	0
	duration	Duration of the flow in seconds.
		Defaults to `none` (use `number_of_frames` instead)
		type	number
		exclusiveMinimum	0
	nat_keep_alive	Enable a flow to keep Network Address (and Port) Translation (NAT/NAPT) entries alive.
		The direction will be from the endpoint behind a NAT/NAPT gateway to the port at the public side of the NAT/NAPT gateway. Defaults to `false`. We always enable the NAT/NAPT keep alive when the user asks for it, even when: The reverse flow is enabled and source and destination UDP ports are the same The source is the endpoint behind the NAT/NAPT gateway: An initial time to wait on the flow can still cause the NAPT entries at the gateway might still timeout before the traffic starts. This might cause the NAPT gateway to create a new (maybe different!) NAPT entry, causing our traffic analysis to fail (with 100% loss).
		type	boolean
	napt_keep_alive	Deprecated since version v1.2.0: Use `nat_keep_alive` instead. Will be removed in v1.4.0.
		type	boolean
	analysis	Sets latency related analysis configuration parameters
		oneOf	Frame loss analysis parameters
			Latency & frame loss analysis parameters

HTTP flow parameters

Specific parameters for HTTP flow. The `type` MUST be defined as `http`.
type	object
allOf	Flow definition
	properties
	tcp_server_port	Port number of the TCP server, defaults to `none`
		type	integer
		maximum	65535
		minimum	0
	tcp_client_port	Port number of the TCP client, defaults to `none`
		type	integer
		maximum	65535
		minimum	0
	duration	The time it takes for a TCP request to be completed.
		Mutual exclusive with `request_size`. Defaults to `none`
		type	number
		exclusiveMinimum	0
	request_size	The size of the TCP request packet in bytes.
		Mutual exclusive with `duration`. Defaults to `none`
		type	integer
		exclusiveMinimum	0
	rate_limit	Limit the data traffic rate (in Bytes per second).
		Mutual exclusive with `maximum_bitrate`. Defaults to `none` (== no limit). Deprecated since version 1.2.0: Deprecated `rate_limit` in favor of `maximum_bitrate`. Will be removed in the next release.
		type	number
		exclusiveMinimum	0
	maximum_bitrate	Limit the data traffic rate (in bits per second).
		Mutual exclusive with `rate_limit`. Defaults to `none` (== no limit). Added in version 1.2.0: Added `maximum_bitrate` deprecating `rate_limit`.
		type	number
		exclusiveMinimum	0
	receive_window_scaling	When given, enable receive window scaling with the given scale factor.
		Defaults to `none`. When ByteBlower Endpoints are involved, this setting will not be applied on them, but only on the HTTP Server on the ByteBlower Port. The ByteBlower Endpoint has no control over the TCP parameters of the host operating system’s. It is then up to the Endpoint’s host configuration whether this setting will be applicable or not.
		type	integer
		maximum	12
		minimum	0
	slow_start_threshold	TCP Slow start threshold value.
		Defaults to `none`. When ByteBlower Endpoints are involved, this setting will not be applied on them, but only on the HTTP Server on the ByteBlower Port. The ByteBlower Endpoint has no control over the TCP parameters of the host operating system’s. It is then up to the Endpoint’s host configuration whether this setting will be applicable or not.
		type	integer
		maximum	2147483647
		minimum	0
	enable_l4s	Activate TCP Prague congestion avoidance algorithm, and enables ECN analysis for Congestion Experienced (CE) codepoint count.
		Defaults to `false` (no L4S analysis, default TCP Prague for ByteBlower Server,disabled for ByteBlower Port, host operating system default for ByteBlower Endpoint). Note L4S support requires at least ByteBlower API v2.22.0, Server and Meeting Point v2.22.0, and ByteBlower Endpoint v2.22.0. When using Endpoints, L4S must be supported and enabled in the hosting OS Added in version 1.3.0: Add support for L4S HTTP flow
		type	boolean

Frame loss analysis parameters

Specific frame loss analysis related parameters
type	object
properties
sampling_interval	Sampling interval for over time results in seconds.
	Defaults to `none` (== 1 second). Interval length of each timestamp sampled by the server. Should be a multiple of 0.1 (1 milliseconds).
	type	number
	maximum	10
	minimum	0.1
max_loss_percentage	Maximum allowed frame/byte loss in %, defaults to `DEFAULT_LOSS_PERCENTAGE` (`1.0`)
	type	number
	maximum	100
	minimum	0

Latency & frame loss analysis parameters

Specific latency & frame loss analysis related parameters
type	object
properties
latency	Set to `true` to enable latency analysis (latency statistics over time, distribution, CDF, CCDF, …), defaults to `false`
	type	boolean
sampling_interval	Sampling interval for over time results in seconds.
	Defaults to `none` (== 1 second). Interval length of each timestamp sampled by the server. Should be a multiple of 0.1 (1 milliseconds).
	type	number
	maximum	10
	minimum	0.1
max_loss_percentage	Maximum allowed frame/byte loss in %, defaults to `DEFAULT_LOSS_PERCENTAGE` (`1.0`)
	type	number
	maximum	100
	minimum	0
max_threshold_latency	Maximum latency threshold
	Maximum allowed latency in milliseconds, defaults to `DEFAULT_MAX_LATENCY_THRESHOLD` (`5.0`)
	type	number
quantile	Quantile for which the latency must be less than the given maximum latency, defaults to `DEFAULT_QUANTILE` (`99.9`)
	type	number
	maximum	100
	minimum	0