Test scenario definition

In the current release, it is possible to supply a configuration file in JSON format for running your tests.

In the following sections, we will provide a detailed explanation of the structure and all parameters in the JSON configuration file.

Structure

The JSON configuration file consists of:

ByteBlower server address (and optionally a ByteBlower meeting point)
List of port definitions: Where you want to send traffic from/to
List of flow definitions: We currently support UDP frame blasting and stateful TCP (HTTP) flows.
(optional) Reporting parameters
Test-specific parameters

A quick short reference for the structure:

{
    "server": "<bb_server_name_or_ip:str>",
    "meeting_point": "<bb_meeting_point_name_or_ip:str>",
    "ports": [
        {
            "name": "<port_name:str>",
            "interface": "<bb_interface_name:str>",
            "uuid": "<endpoint_uuid:str>",
            "ipv4": "<ipv4_address:str>|dhcp|true",
            "netmask": "<ipv4_netmask:str>",
            "gateway": "<ipv4_gateway:str>",
            "nat": "<enable_nat_resolution:bool>",
            "ipv6": "dhcp|slaac|true",
            "port_group": [
                "<port_group:str>"
            ]
        }
    ],
    "flows": [
        {
            "name": "<flow_name:str>",
            "source": {
                "port_group": [
                    "<source_group:str>"
                ]
            },
            "destination": {
                "port_group": [
                    "<destination_group:str>"
                ]
            },
            "add_reverse_direction": "<add_reverse_direction_flow:bool>",
            "type": "<flow_type:str>",
            "ecn": "<ecn_code_point:str|int>",
            "dscp": "<dscp_code_point:str|int>",
            "initial_time_to_wait": "<initial_time_to_wait:float>"
        }
    ],
    "report": {
        "html": "<enable_html_reporting:bool>",
        "json": "<enable_json_reporting:bool>",
        "junit_xml": "<enable_junit_xml_reporting:bool>"
    },
    "maximum_run_time": "<scenario_max_run_time:float>"
}

The current release supports UDP frame blasting and stateful TCP (HTTP) flows. Each type of flow has some additional specific parameters as follows:

UDP flow specific parameters

{
    "frame_size": "<frame_size_without_crc:int>",
    "bitrate": "<flow_bitrate:float>",
    "frame_rate": "<flow_frame_rate:float>",
    "duration": "<flow_duration:float>",
    "number_of_frames": "<number_of_frames:float>",
    "nat_keep_alive": "<activate_nat_keep_alive:bool>",
    "analysis": {
        "latency": "<enable_latency_analysis:bool>",
        "max_loss_percentage": "<max_loss_percentage:float>",
        "max_threshold_latency": "<max_threshold_latency:float>",
        "quantile": "<quantile:float>"
    }
}

TCP (HTTP) flow specific parameters

{
    "tcp_server_port": "<tcp_server_port_number:int>",
    "tcp_client_port": "<tcp_client_port_number:int>",
    "duration": "<request_duration:float>",
    "request_size": "<request_size:float>",
    "maximum_bitrate": "<maximum_bitrate:float>",
    "receive_window_scaling": "<receive_window_scaling:int>",
    "slow_start_threshold": "<slow_start_threshold:int>",
    "enable_l4s": "<enable_l4s:bool>"
}

JSON schema

The complete structure and documentation of the file is available in Configuration file JSON schema and documented below.

Test configuration parameters

https://api.byteblower.com/test-framework/json/cli-config-schema.json
Root structure of the JSON configuration file for the ByteBlower Test Framework command-line interface.
type	object
properties
server	ByteBlower server address
meeting_point	ByteBlower meeting point address
ports	Port definitions
flow	Flow definition list
maximum_run_time	Maximum run time
report	Report configuration parameters
additionalProperties	True

ByteBlower server address

Host name or IP address of the ByteBlower Server.
type	string
oneOf	format	hostname
	format	ipv4
	format	ipv6

ByteBlower meeting point address

Host name or IP address of the ByteBlower meeting point.
Added in version 1.2.0: Support for ByteBlower Endpoint.
type	string
oneOf	format	hostname
	format	ipv4
	format	ipv6

Port definitions

Flat list of ports.
type	array
items	Configuration parameters for a ByteBlower Port or Endpoint

Flow definition list

Flat list of flow definitions
type	array
items	oneOf	Frame blasting flow
		HTTP flow parameters

Maximum run time

Maximum run time of the scenario in seconds
type	number
minimum	0

Report configuration parameters

type	object
properties
html	Create HTML report
	Enable/Disable reporting in HTML format
	type	boolean
json	Create JSON report
	Enable/Disable reporting in JSON format
	type	boolean
junit_xml	Create JUnit XML report
	Enable/Disable reporting in JUnit XML format
	type	boolean

Port group

Definition of a port group
type	string

Configuration parameters for a ByteBlower Port or Endpoint

Configuration parameters for a ByteBlower Port or Endpoint
type	object
properties
name	Name of this ByteBlower traffic endpoint.
	(a friendly name to identify this endpoint).
	type	string
port_group	List of port groups where this port belongs to.
	Used to identify to which flows this port will be part of (either as source or destination).
	type	array
	items	Port group
interface	Name of the physical ByteBlower interface where this ByteBlower port is created.
	ByteBlower Port only. Mutually exclusive with `"uuid"`.
	type	string
	pattern	^(nontrunk[-][0-9]+\|trunk[-][0-9]+[-][0-9]+)$
uuid	Device identifier of the ByteBlower Endpoint.
	ByteBlower Endpoint only. Mutually exclusive with `"interface"`. Note Using an Endpoint also requires the `"meeting_point"` to be provided and `"ipv4"` or `"ipv6"` set to `true` for this port configuration. Added in version 1.2.0: Support for ByteBlower Endpoint.
	type	string
	oneOf	format	uuid
		format	None
vlans	List of VLAN (stack) configurations.
	Note Not supported by ByteBlower Endpoint.
	type	array
	items	VLAN (stack) configuration
ipv4	IPv4 addressing method
	Enable IPv4 on this traffic endpoint.
	Supports the following values: IPv4 address for static IPv4 IP address (ByteBlower Port only). `"dhcp"` for dynamic IPv4 address allocation (ByteBlower Port only). `true`: To use the IPv4 address(es) of the system (ByteBlower Endpoint only). Default: `"dhcp"` (ByteBlower Port) or undefined (ByteBlower Endpoint). Note For ByteBlower Endpoint, you must specify either `"ipv4": true` or `"ipv6": true`. Changed in version 1.2.0: Support for ByteBlower Endpoint.
	oneOf	type	string
		oneOf	enum	dhcp
			format	ipv4
		type	boolean
netmask	Netmask of the IPv4 network.
	Default: Assigned by DHCP or `255.255.255.0` for static IPv4 address configuration. Note Will be supported for IPv6 once manual address configuration is supported. Not supported by ByteBlower Endpoint.
	type	string
	format	ipv4
gateway	(Default) Gateway of the IPv4 network.
	Default: Assigned by DHCP or none for static IPv4 address configuration. Note Will be supported for IPv6 once manual address configuration is supported. Not supported by ByteBlower Endpoint.
	type	string
	format	ipv4
nat	Define whether this port is located behind a NAT gateway.
	Defaults to `false`. Note Enabled by default for ByteBlower Endpoint.
	type	boolean
ipv6	IPv6 addressing method
	Enable IPv6 on this traffic endpoint.
	Supports the following values: `"dhcp"`: to let the port perform DHCPv6 (ByteBlower Port only) `"slaac"`: to let the port perform stateless address auto-configuration (ByteBlower Port only). `true`: To use the IPv6 address(es) of the system (ByteBlower Endpoint only). Default: undefined. Note For ByteBlower Endpoint, you must specify either `"ipv4": true` or `"ipv6": true`. Changed in version 1.2.0: Support for ByteBlower Endpoint.
	oneOf	type	string
		oneOf	enum	dhcp, slaac
			format	ipv6
		type	boolean

VLAN (stack) configuration

type	object
properties
protocol_id	VLAN protocol ID (TPID).
	Default value is: `"0x8100"` for a single VLAN configuration `"0x88a8"` for the outer VLAN(s) in a VLAN stack configuration `"0x8100"` for the most inner VLAN in a VLAN stack configuration. Note Configuration of the VLAN protocol ID (TPID) requires ByteBlower Server version >= 2.20.0.
	type	integer
	maximum	65535
	minimum	0
id	VLAN ID (12-bit; 0-4095)
	type	integer
	maximum	4095
	minimum	0
priority	3 bits defining the VLAN PCP bits. Default: 0
	type	integer
	maximum	3
	minimum	0
drop_eligible	1-bit flag. Default: 0
	type	integer
	maximum	1
	minimum	0

Flow definition

type	object
properties
name	Name of this Flow, defaults to auto-generated name when set to `none`
	type	string
source	Specifying the source of transmitted data for this flow
	type	object
	properties
	port_group	List of port groups specifying the transmitting ports
		type	array
		items	Port group
	additionalProperties	False
destination	Specifying the destination for receiving data for this flow
	type	object
	properties
	port_group	List of port groups specifying the receiving ports
		type	array
		items	Port group
	additionalProperties	False
type	Type of the flow. Defines which specific flow parameters will be possible.
	type	string
	enum	frame_blasting, http
ecn	IP ECN
	Explicit Congestion Notification code point. Defaults to `DEFAULT_IP_ECN` (`0x00`)
	oneOf	type	integer
		maximum	3
		minimum	0
		type	string
dscp	IP DSCP
	Differentiated Services Code Point. Defaults to `DEFAULT_IP_DSCP` (`0x00`)
	oneOf	type	integer
		maximum	63
		minimum	0
		type	string
add_reverse_direction	Enables adding a flow in the reverse direction of the original flow.
	The reverse flow uses the same configuration as the original. Defaults to `false`
	type	boolean
initial_time_to_wait	Initial time to wait to start the flow in seconds.
	Defaults to 0 (start immediately)
	type	number
	minimum	0

Frame blasting flow

Specific parameters for frame blasting flow. The `type` MUST be defined as `frame_blasting`.
type	object
allOf	Flow definition
	properties
	bitrate	Rate at which the bits are transmitted (in bits per second).
		Excludes the VLAN tag bytes (when applicable), mutual exclusive with `frame_rate`, defaults to `none`
		type	number
		exclusiveMinimum	0
	frame_size	Frame size in Bytes without CRC, defaults to `none`
		type	integer
		minimum	42
	frame_rate	Rate at which the frames are transmitted (in frames per second).
		Mutual exclusive with `bitrate`. Defaults to `DEFAULT_FRAME_RATE` (`100`) when `bitrate` is not provided.
		type	number
		exclusiveMinimum	0
	number_of_frames	Number of frames to transmit.
		Defaults to `DEFAULT_NUMBER_OF_FRAMES` (`none` == defined by scenario maximum run time).
		type	integer
		exclusiveMinimum	0
	duration	Duration of the flow in seconds.
		Defaults to `none` (use `number_of_frames` instead)
		type	number
		exclusiveMinimum	0
	nat_keep_alive	Enable a flow to keep Network Address (and Port) Translation (NAT/NAPT) entries alive.
		The direction will be from the endpoint behind a NAT/NAPT gateway to the port at the public side of the NAT/NAPT gateway. Defaults to `false`. We always enable the NAT/NAPT keep alive when the user asks for it, even when: The reverse flow is enabled and source and destination UDP ports are the same The source is the endpoint behind the NAT/NAPT gateway: An initial time to wait on the flow can still cause the NAPT entries at the gateway might still timeout before the traffic starts. This might cause the NAPT gateway to create a new (maybe different!) NAPT entry, causing our traffic analysis to fail (with 100% loss).
		type	boolean
	napt_keep_alive	Deprecated since version v1.2.0: Use `nat_keep_alive` instead. Will be removed in v1.4.0.
		type	boolean
	analysis	Sets latency related analysis configuration parameters
		oneOf	Frame loss analysis parameters
			Latency & frame loss analysis parameters

HTTP flow parameters

Specific parameters for HTTP flow. The `type` MUST be defined as `http`.
type	object
allOf	Flow definition
	properties
	tcp_server_port	Port number of the TCP server, defaults to `none`
		type	integer
		maximum	65535
		minimum	0
	tcp_client_port	Port number of the TCP client, defaults to `none`
		type	integer
		maximum	65535
		minimum	0
	duration	The time it takes for a TCP request to be completed.
		Mutual exclusive with `request_size`. Defaults to `none`
		type	number
		exclusiveMinimum	0
	request_size	The size of the TCP request packet in bytes.
		Mutual exclusive with `duration`. Defaults to `none`
		type	integer
		exclusiveMinimum	0
	rate_limit	Limit the data traffic rate (in Bytes per second).
		Mutual exclusive with `maximum_bitrate`. Defaults to `none` (== no limit). Deprecated since version 1.2.0: Deprecated `rate_limit` in favor of `maximum_bitrate`. Will be removed in the next release.
		type	number
		exclusiveMinimum	0
	maximum_bitrate	Limit the data traffic rate (in bits per second).
		Mutual exclusive with `rate_limit`. Defaults to `none` (== no limit). Added in version 1.2.0: Added `maximum_bitrate` deprecating `rate_limit`.
		type	number
		exclusiveMinimum	0
	receive_window_scaling	When given, enable receive window scaling with the given scale factor.
		Defaults to `none`. When ByteBlower Endpoints are involved, this setting will not be applied on them, but only on the HTTP Server on the ByteBlower Port. The ByteBlower Endpoint has no control over the TCP parameters of the host operating system’s. It is then up to the Endpoint’s host configuration whether this setting will be applicable or not.
		type	integer
		maximum	12
		minimum	0
	slow_start_threshold	TCP Slow start threshold value.
		Defaults to `none`. When ByteBlower Endpoints are involved, this setting will not be applied on them, but only on the HTTP Server on the ByteBlower Port. The ByteBlower Endpoint has no control over the TCP parameters of the host operating system’s. It is then up to the Endpoint’s host configuration whether this setting will be applicable or not.
		type	integer
		maximum	2147483647
		minimum	0
	enable_l4s	Activate TCP Prague congestion avoidance algorithm, and enables ECN analysis for Congestion Experienced (CE) codepoint count.
		Defaults to `false` (no L4S analysis, default TCP Prague for ByteBlower Server,disabled for ByteBlower Port, host operating system default for ByteBlower Endpoint). Note L4S support requires at least ByteBlower API v2.22.0, Server and Meeting Point v2.22.0, and ByteBlower Endpoint v2.22.0. When using Endpoints, L4S must be supported and enabled in the hosting OS Added in version 1.3.0: Add support for L4S HTTP flow
		type	boolean

Frame loss analysis parameters

Specific frame loss analysis related parameters
type	object
properties
max_loss_percentage	Maximum allowed frame/byte loss in %, defaults to `DEFAULT_LOSS_PERCENTAGE` (`1.0`)
	type	number
	maximum	100
	minimum	0

Latency & frame loss analysis parameters

Specific latency & frame loss analysis related parameters
type	object
properties
latency	Set to `true` to enable latency analysis (latency statistics over time, distribution, CDF, CCDF, …), defaults to `false`
	type	boolean
max_loss_percentage	Maximum allowed frame/byte loss in %, defaults to `DEFAULT_LOSS_PERCENTAGE` (`1.0`)
	type	number
	maximum	100
	minimum	0
max_threshold_latency	Maximum latency threshold
	Maximum allowed latency in milliseconds, defaults to `DEFAULT_MAX_LATENCY_THRESHOLD` (`5.0`)
	type	number
quantile	Quantile for which the latency must be less than the given maximum latency, defaults to `DEFAULT_QUANTILE` (`99.9`)
	type	number
	maximum	100
	minimum	0