Test scenario definition

In the current release, it is possible to supply a configuration file in JSON format for running your tests.

In the following sections, we will provide a detailed explanation of the structure and all parameters in the JSON configuration file.

Structure

The JSON configuration file consists of:

• ByteBlower server address (and optionally a ByteBlower meeting point)

• List of port definitions: Where you want to send traffic from/to

• List of flow definitions: We currently support UDP frame blasting and stateful TCP (HTTP) flows.

• (optional) Reporting parameters

• Test-specific parameters

A quick short reference for the structure:

{
    "server": "<bb_server_name_or_ip:str>",
    "meeting_point": "<bb_meeting_point_name_or_ip:str>",
    "ports": [
        {
            "name": "<port_name:str>",
            "interface": "<bb_interface_name:str>",
            "uuid": "<endpoint_uuid:str>",
            "ipv4": "<ipv4_address:str>|dhcp|true",
            "netmask": "<ipv4_netmask:str>",
            "gateway": "<ipv4_gateway:str>",
            "nat": "<enable_nat_resolution:bool>",
            "ipv6": "dhcp|slaac|true",
            "port_group": [
                "<port_group:str>"
            ]
        }
    ],
    "flows": [
        {
            "name": "<flow_name:str>",
            "source": {
                "port_group": [
                    "<source_group:str>"
                ]
            },
            "destination": {
                "port_group": [
                    "<destination_group:str>"
                ]
            },
            "add_reverse_direction": "<add_reverse_direction_flow:bool>",
            "type": "<flow_type:str>",
            "ecn": "<ecn_code_point:str|int>",
            "dscp": "<dscp_code_point:str|int>",
            "initial_time_to_wait": "<initial_time_to_wait:float>"
        }
    ],
    "report": {
        "html_report": "<html_report:bool>",
        "json_report": "<json_report:bool>",
        "junit_xml_report": "<junit_xml_report:bool>"
    },
    "maximum_run_time": "<scenario_max_run_time:float>"
}

The current release supports UDP frame blasting and stateful TCP (HTTP) flows. Each type of flow has some additional specific parameters as follows:

UDP flow specific parameters

{
    "frame_size": "<frame_size_without_crc:int>",
    "bitrate": "<flow_bitrate:float>",
    "frame_rate": "<flow_frame_rate:float>",
    "duration": "<flow_duration:float>",
    "number_of_frames": "<number_of_frames:float>",
    "nat_keep_alive": "<activate_nat_keep_alive:bool>",
    "analysis": {
        "latency": "<enable_latency_analysis:bool>",
        "max_loss_percentage": "<max_loss_percentage:float>",
        "max_threshold_latency": "<max_threshold_latency:float>",
        "quantile": "<quantile:float>"
    }
}

TCP (HTTP) flow specific parameters

{
    "tcp_server_port": "<tcp_server_port_number:int>",
    "tcp_client_port": "<tcp_client_port_number:int>",
    "duration": "<request_duration:float>",
    "request_size": "<request_size:float>",
    "maximum_bitrate": "<maximum_bitrate:float>",
    "receive_window_scaling": "<receive_window_scaling:int>",
    "slow_start_threshold": "<slow_start_threshold:int>"
}

JSON schema

The complete structure and documentation of the file is available in Configuration file JSON schema and documented below.

https://api.byteblower.com/test-framework/json/cli-config-schema.json
Test configuration parameters
Root structure of the JSON configuration file for the ByteBlower Test Framework command-line interface.
type	object
properties
server	ByteBlower server address
meeting_point	ByteBlower meeting point address
ports	Port definitions
flow	Flow definitions
maximum_run_time	Maximum run time
report	Report configuration parameters
additionalProperties	True

ByteBlower server address
Host name or IP address of the ByteBlower Server.
type	string
oneOf	format	hostname
	format	ipv4
	format	ipv6

ByteBlower meeting point address
Host name or IP address of the ByteBlower meeting point. New in version 1.2.0: Support for ByteBlower Endpoint.
type	string
oneOf	format	hostname
	format	ipv4
	format	ipv6

Port definitions
Flat list of ports.
type	array
items	Configuration parameters for a ByteBlower Port or Endpoint

Flow definitions
Flat list of flow definitions
type	array
items	oneOf	Frame blasting flow
		HTTP flow parameters

Maximum run time
Maximum run time of the scenario in seconds
type	number
format	float

Report configuration parameters
type	object
properties
html	Create HTML report
	Enable/Disable reporting in HTML format
	type	boolean
json	Create JSON report
	Enable/Disable reporting in JSON format
	type	boolean
junit_xml	Create JUnit XML report
	Enable/Disable reporting in JUnit XML format
	type	boolean

Configuration parameters for a ByteBlower Port or Endpoint
Configuration parameters for a ByteBlower Port or Endpoint
type	object
properties
name	Name of this ByteBlower traffic endpoint (a friendly name to identify this endpoint)
	type	string
port_group	List of port group where this port belongs to. Used to identify to which flows this port will be part of (either as source or destination).
	type	array
	items	Port group
interface	Name of the ByteBlower interface where this ByteBlower port is created (ByteBlower Port only). Mutually exclusive with "uuid".
	type	string
uuid	Device identifier of the Endpoint to use (ByteBlower Endpoint only).
	Mutually exclusive with "interface". Note Using an Endpoint also requires the "meeting_point" to be provided and "ipv4" or "ipv6" set to true for this port configuration. New in version 1.2.0: Support for ByteBlower Endpoint.
	type	string
vlans	List of VLAN (stack) configurations. See details in VLAN (stack) configuration.
	Note Not supported by ByteBlower Endpoint.
	type	array
	items	VLAN (stack) configuration
ipv4	IPv4 addressing method
	Enable IPv4 on this traffic endpoint
	Supports the following values: IPv4 address for static IPv4 IP address (ByteBlower Port only) "dhcp" for dynamic IPv4 address allocation (ByteBlower Port only) true: To use the IPv4 address(es) of the system (ByteBlower Endpoint only). Default: "dhcp" (ByteBlower Port) or undefined (ByteBlower Endpoint). Note For ByteBlower Endpoint, you must specify either "ipv4": true or "ipv6": true. Changed in version 1.2.0: Support for ByteBlower Endpoint.
	oneOf	type	string
		type	bool
netmask	Netmask of the IPv4 network. Default: Assigned by DHCP or 255.255.255.0 for static IPv4 address configuration. Note Will be supported for IPv6 once manual address configuration is supported.
	Note Not supported by ByteBlower Endpoint.
	type	string
gateway	(Default) Gateway of the IPv4 network. Default: Assigned by DHCP or none for static IPv4 address configuration. Note Will be supported for IPv6 once manual address configuration is supported.
	Note Not supported by ByteBlower Endpoint.
	type	string
nat	Define whether this port is located behind a NAT gateway. Defaults to false.
	Note Enabled by default for ByteBlower Endpoint.
	type	bool
ipv6	IPv6 addressing method
	Enable IPv6 on this traffic endpoint
	Supports the following values: "dhcp" to let the port perform DHCPv6 (ByteBlower Port only) "slaac" to let the port perform stateless address auto-configuration (ByteBlower Port only) true: To use the IPv6 address(es) of the system (ByteBlower Endpoint only). Default: undefined. Note For ByteBlower Endpoint, you must specify either "ipv4": true or "ipv6": true. Changed in version 1.2.0: Support for ByteBlower Endpoint.
	oneOf	type	string
		type	bool

VLAN (stack) configuration
type	object
properties
protocol_id	VLAN protocol ID (TPID). Default value is: ‘0x8100’ for a single VLAN configuration ‘0x88a8’ for the outer VLAN(s) in a VLAN stack configuration ‘0x8100’ for the most inner VLAN in a VLAN stack configuration. Note: Configuration of the VLAN protocol ID (TPID) requires ByteBlower Server version >= 2.20.0.
	type	integer
id	VLAN ID (12-bit; 0-4095)
	type	integer
priority	3 bits defining the VLAN PCP bits. Default: 0
	type	integer
drop_eligible	1-bit flag. Default: 0
	type	integer

Port group
Definition of a port group
type	string

Flow definition
type	object
properties
name	Name of this Flow, defaults to auto-generated name when set to none
	type	string
source	Specifying the source of transmitted data for this flow
	type	object
	properties
	port_group	List of port groups specifying the transmitting ports
		type	array
		items	Port group
	additionalProperties	False
destination	Specifying the destination for receiving data for this flow
	type	object
	properties
	port_group	List of port groups specifying the receiving ports
		type	array
		items	Port group
	additionalProperties	False
type	Type of the flow. Defines which specific flow parameters will be possible.
	type	string
ecn	IP ECN
	Explicit Congestion Notification code point, defaults to DEFAULT_IP_ECN (0x00)
	oneOf	type	number
		type	string
dscp	IP DSCP
	Differentiated Services Code Point, defaults to DEFAULT_IP_DSCP (0x00)
	oneOf	type	number
		type	string
add_reverse_direction	Enables adding a flow in the reverse direction of the original flow with the same configuration, defaults to false
	type	string
initial_time_to_wait	Initial time to wait to start the flow in seconds, defaults to 0 (start immediately)
	type	number

Frame blasting flow
Specific parameters for frame blasting flow. The type MUST be defined as frame_blasting.
type	object
allOf	Flow definition
	properties
	bitrate	Rate at which the bits are transmitted (in bits per second). Excludes the VLAN tag bytes (when applicable), mutual exclusive with frame_rate, defaults to none
		type	string
	frame_size	Frame size in Bytes without CRC, defaults to none
		type	number
	frame_rate	Rate at which the frames are transmitted (in frames per second), mutual exclusive with bitrate, defaults to DEFAULT_FRAME_RATE (100) when bitrate is not provided.
		type	number
	number_of_frames	Number of frames to transmit, defaults to DEFAULT_NUMBER_OF_FRAMES (none == defined by scenario maximum run time).
		type	string
	duration	Duration of the flow in seconds, defaults to none (use number_of_frames instead)
		type	string
	nat_keep_alive	Enable a flow to keep Network Address (and Port) Translation (NAT/NAPT) entries alive. The direction will be from the endpoint behind a NAT/NAPT gateway to the port at the public side of the NAT/NAPT gateway, defaults to false. We always enable the NAT/NAPT keep alive when the user asks for it, even when: the reverse flow is enabled and source and destination UDP ports are the same the source is the endpoint behind the NAT/NAPT gateway: An initial time to wait on the flow can still cause the NAPT entries at the gateway might still timeout before the traffic starts. This might cause the NAPT gateway to create a new (and maybe different!) NAPT entry, causing our traffic analysis to fail (with 100% loss).
		type	boolean
	napt_keep_alive	Deprecated since version v1.2.0: Use nat_keep_alive instead. Will be removed in v1.4.0.
		type	boolean
	analysis	Sets latency related analysis configuration parameters
		oneOf	Frame loss analysis parameters
			Latency & frame loss analysis parameters

HTTP flow parameters
Specific parameters for HTTP flow. The type MUST be defined as http.
type	object
allOf	Flow definition
	properties
	tcp_server_port	Port number of the TCP server, defaults to none
		type	number
	tcp_client_port	Port number of the TCP client, defaults to none
		type	number
	duration	The time it takes for a TCP request to be completed , mutual exclusive with request_size, defaults to none
		type	number
	request_size	The size of the TCP request packet in bytes, mutual exclusive with request_size, defaults to none
		type	number
	rate_limit	Limit the data traffic rate (in Bytes per second), defaults to none (== no limit).
		Mutual exclusive with maximum_bitrate Deprecated since version 1.2.0: Deprecated rate_limit in favor of maximum_bitrate. Will be removed in the next release.
		type	number
	maximum_bitrate	Limit the data traffic rate (in bits per second), defaults to none (== no limit).
		Mutual exclusive with rate_limit New in version 1.2.0: Added maximum_bitrate deprecating rate_limit.
		type	number
	receive_window_scaling	When given, enable receive window scaling with the given scale factor , defaults to none
		When ByteBlower Endpoints are involved, this setting will not be applied on them, but only on the HTTP Server on the ByteBlower Port. The ByteBlower Endpoint has no control over the TCP parameters of the host operating system’s. It is then up to the Endpoint’s host configuration whether this setting will be applicable or not.
		type	number
	slow_start_threshold	TCP Slow start threshold value, defaults to none
		When ByteBlower Endpoints are involved, this setting will not be applied on them, but only on the HTTP Server on the ByteBlower Port. The ByteBlower Endpoint has no control over the TCP parameters of the host operating system’s. It is then up to the Endpoint’s host configuration whether this setting will be applicable or not.
		type	number

Frame loss analysis parameters
Specific frame loss analysis related parameters
type	object
properties
max_loss_percentage	Maximum allowed frame/byte loss in %, defaults to DEFAULT_LOSS_PERCENTAGE (1.0)
	type	number

Latency & frame loss analysis parameters
Specific latency & frame loss analysis related parameters
type	object
properties
latency	Set to true to enable latency analysis (latency statistics over time, distribution, CDF, CCDF, ..), defaults to false
	type	bool
max_loss_percentage	Maximum allowed frame/byte loss in %, defaults to DEFAULT_LOSS_PERCENTAGE (1.0)
	type	number
max_threshold_latency	Maximum allowed latency in milliseconds, defaults to DEFAULT_MAX_LATENCY_THRESHOLD (5.0)
	type	number
quantile	Quantile for which the latency must be less than the given maximum latency, defaults to DEFAULT_QUANTILE (99.9)
	type	number