Test scenario definition

In the current release, it is possible to supply a configuration file in JSON format for running your tests.

In the following sections, we will provide a detailed explanation of the structure and all parameters in the JSON configuration file.

Structure

A quick short reference for the structure:

{
    "server": "<bb_server_name_or_ip:str>",
    "meeting_point": "<bb_meeting_point_name_or_ip:str>",
    "source": {
        "name": "<source_name:str>",
        "interface": "<bb_interface_name:str>",
        "uuid":"<endpoint_uuid:str>",
        "ipv4": "<ipv4_address:str>|dhcp|true",
        "netmask": "<ipv4_address:str>",
        "gateway": "<ipv4_address:str>",
        "nat": "<nat_resolution:bool>",
        "ipv6": "dhcp|slaac|true"
    },
    "destination": {
        "name": "<destination_name:str>",
        "interface": "<bb_interface_name:str>",
        "uuid":"<endpoint_uuid:str>",
        "ipv4": "<ipv4_address:str>|dhcp|true",
        "netmask": "<ipv4_address:str>",
        "gateway": "<ipv4_address:str>",
        "nat": "<nat_resolution:bool>",
        "ipv6": "dhcp|slaac|true"
    },
    "duration": "<trial_duration:int>",
    "max_iterations": "<max_iterations:int>",
    "frame_configs": [
        {
            "size": "<frame_length:int>",
            "initial_bitrate": "<initial_frame_rate:int>",
            "tolerated_frame_loss": "<frame_loss:float>",
            "accuracy": "<accuracy:float>",
            "expected_bitrate": "<expected_frame_rate:int>"
        }
    ]
}

JSON schema

The complete structure and documentation of the file is available in Configuration file JSON schema, and documented below.

Server

ByteBlower server address

Host name or IP address of the ByteBlower Server.

type

string

oneOf

format

hostname

format

ipv4

format

ipv6

Meeting point

ByteBlower meeting point address

Host name or IP address of the ByteBlower meeting point.

New in version 1.0.0: Support for ByteBlower Endpoint.

type

string

oneOf

format

hostname

format

ipv4

format

ipv6

Ports

The "source" and "destination" blocks define ByteBlower ports (simulated hosts) at a given ByteBlower interface. Where a ByteBlower interface is the physical connection on either the non-trunking interface (direct connection at a ByteBlower server) or a trunking interface (connection at a physical port on the ByteBlower switch).

Each port supports IPv4 address assignment using static IPv4 configuration or DHCP. IPv6 address configuration is possible through DHCPv6 or Stateless autoconfiguration (SLAAC).

When an (IPv4) port is located behind a NAT gateway, then the test framework will perform additional setup flow steps to resolve the NAT’s public IPv4 address and UDP port (when required).

Since version 1.2.0, the Byteblower Test Framework also supports Byteblower endpoints. They can be configured by:

  1. Providing the meeting point address and the UUID of the endpoint.

  2. Setting the IPv4 or IPv6 parameter to true.

Configuration parameters for a ByteBlower Port or Endpoint

Configuration parameters for a ByteBlower Port or Endpoint

type

object

properties

  • name

Name of this ByteBlower port (typically Source for the tester entity, and Destination for the NUT)

type

string

  • interface

Name of the ByteBlower interface where this ByteBlower port is created (ByteBlower Port only). Mutually exclusive with "uuid".

type

string

  • uuid

Device identifier of the Endpoint to use (ByteBlower Endpoint only).

Mutually exclusive with "interface".

Note

Using an Endpoint also requires the "meeting_point" to be provided and "ipv4" or "ipv6" set to true for this port configuration.

New in version 1.0.0: Support for ByteBlower Endpoint.

type

string

  • vlans

List of VLAN (stack) configurations. See details in VLAN (stack) configuration.

Note

Not supported by ByteBlower Endpoint.

type

array

items

VLAN (stack) configuration

  • ipv4

IPv4 addressing method

Enable IPv4 on this traffic endpoint

Supports the following values:

  • IPv4 address for static IPv4 IP address (ByteBlower Port only)

  • "dhcp" for dynamic IPv4 address allocation (ByteBlower Port only)

  • true: To use the IPv4 address(es) of the system (ByteBlower Endpoint only).

Default: "dhcp" (ByteBlower Port) or undefined (ByteBlower Endpoint).

Note

For ByteBlower Endpoint, you must specify either "ipv4": true or "ipv6": true.

Changed in version 1.0.0: Support for ByteBlower Endpoint.

oneOf

type

string

type

bool

  • netmask

Netmask of the IPv4 network. Default: Assigned by DHCP or 255.255.255.0 for static IPv4 address configuration.

Note

Will be supported for IPv6 once manual address configuration is supported by the ByteBlower Test Framework.

Note

Not supported by ByteBlower Endpoint.

type

string

  • gateway

(Default) Gateway of the IPv4 network. Default: Assigned by DHCP or none for static IPv4 address configuration.

Note

Will be supported for IPv6 once manual address configuration is supported by the ByteBlower Test Framework.

Note

Not supported by ByteBlower Endpoint.

type

string

  • nat

Define whether this port is located behind a NAT gateway. Defaults to false.

Note

Enabled by default for ByteBlower Endpoint.

type

bool

  • ipv6

IPv6 addressing method

Enable IPv6 on this traffic endpoint

Supports the following values:

  • "dhcp" to let the port perform DHCPv6 (ByteBlower Port only)

  • "slaac" to let the port perform stateless address auto-configuration (ByteBlower Port only)

  • true: To use the IPv6 address(es) of the system (ByteBlower Endpoint only).

Default: undefined.

Note

For ByteBlower Endpoint, you must specify either "ipv4": true or "ipv6": true.

Changed in version 1.0.0: Support for ByteBlower Endpoint.

oneOf

type

string

type

bool

VLAN (stack) configuration

type

object

properties

  • protocol_id

VLAN protocol ID (TPID). Default value is:

  • ‘0x8100’ for a single VLAN configuration

  • ‘0x88a8’ for the outer VLAN(s) in a VLAN stack configuration

  • ‘0x8100’ for the most inner VLAN in a VLAN stack configuration.

Note: Configuration of the VLAN protocol ID (TPID) requires ByteBlower Server version >= 2.20.0.

type

integer

  • id

VLAN ID (12-bit; 0-4095).

type

integer

  • priority

3 bits defining the VLAN PCP bits. Default: 0.

type

integer

  • drop_eligible

1-bit flag. Default: 0.

type

integer

Frame configurations

This defines a list of frame configurations for each frame size. These configurations are related to the RFC 2544 throughput test specifications.

As stated in the RFC 2544 basic requirements, at least five frame sizes SHOULD be tested for at least 60 seconds per test trial. Also, frame size with the largest MTU supported by the protocol under test SHOULD be used.

type

object

properties

  • size

Frame length in Bytes without the FCS (Frame Check Sequence, 4 Bytes)

type

integer

  • initial_bitrate

Initial bitrate to be used in the first trial of the RFC 2544 throughput test.

type

float

  • tolerated_frame_loss

The maximum acceptable frame loss during the test.

type

float

  • accuracy

Defines the test accuracy (precision) at which the test should be ended.

type

float

  • expected_bitrate

The expected maximum Ethernet layer 2 speed for frames without the 4 bytes of FCS. Usually, it is around the maximum theoretical speed of the tested network.

type

string

Default frames configuration

Frame configurations can be removed entirely from the configuration file. In case no frame configuration is provided, default frame configurations in the byteblower.test_case.rfc_2544.definitions module will be used.

[
    {
        "size": 60,
        "initial_bitrate": 3e8,
        "tolerated_frame_loss": 1e-3,
        "expected_bitrate": 3.7e8,
        "accuracy": 1e5
    },
    {
        "size": 124,
        "initial_bitrate": 6e8,
        "tolerated_frame_loss": 1e-3,
        "expected_bitrate": 4.5e8,
        "accuracy": 1e5
    },
    {
        "size": 252,
        "initial_bitrate": 8e8,
        "tolerated_frame_loss": 1e-3,
        "expected_bitrate": 5.7e8,
        "accuracy": 1e5
    },
    {
        "size": 508,
        "initial_bitrate": 8e8,
        "tolerated_frame_loss": 1e-3,
        "expected_bitrate": 6.6e8,
        "accuracy": 1e5
    },
    {
        "size": 1020,
        "initial_bitrate": 1e9,
        "tolerated_frame_loss": 1e-3,
        "expected_bitrate": 7.15e8,
        "accuracy": 1e5
    },
    {
        "size": 1276,
        "initial_bitrate": 1e9,
        "tolerated_frame_loss": 1e-3,
        "expected_bitrate": 7.25e8,
        "accuracy": 1e5
    },
    {
        "size": 1514,
        "initial_bitrate": 1e9,
        "tolerated_frame_loss": 1e-3,
        "expected_bitrate": 7.35e8,
        "accuracy": 1e5
    }
]

These values could be used as is. However, you preferably should change these default values to comply to your network and test specifications.

Other parameters

maximum_run_time

Maximum run time of a single test trial in seconds

type

number

format

float

max_iterations

Maximum number of iteration to test per frame size

type

number

format

integer

Configuration file example

  • Using ByteBlower Ports

    {
    "server": "byteblower-server.example.com.",
    "source": {
        "name": "Source",
        "interface": "nontrunk-1",
        "ipv4": "192.168.5.2",
        "netmask": "255.255.255.0",
        "gateway": "192.168.5.254"
    },
    "destination": {
        "name": "Destination",
        "interface": "trunk-1",
        "ipv4": "dhcp",
        "nat": true
    },
    "duration": 60,
    "max_iterations": 25,
    "frame_configs": [
        {
            "size": 60,
            "initial_bitrate": 3e8,
            "tolerated_frame_loss": 1e-4,
            "expected_bitrate": 1.7e8,
            "accuracy": 1e3
        },
        {
            "size": 124,
            "initial_bitrate": 6e6,
            "tolerated_frame_loss": 1e-3,
            "expected_bitrate": 4.5e8,
            "accuracy": 1e4
        },
        {
            "size": 252,
            "initial_bitrate": 8e6,
            "tolerated_frame_loss": 1e-3,
            "expected_bitrate": 5.7e8,
            "accuracy": 1e3
        },
        {
            "size": 508,
            "initial_bitrate": 8e8,
            "tolerated_frame_loss": 1e-3,
            "expected_bitrate": 6.6e8,
            "accuracy": 1e3
        },
        {
            "size": 1020,
            "initial_bitrate": 1e9,
            "tolerated_frame_loss": 1e-3,
            "expected_bitrate": 7.15e8,
            "accuracy": 1e2
        },
        {
            "size": 1276,
            "initial_bitrate": 1e9,
            "tolerated_frame_loss": 1e-3,
            "expected_bitrate": 7.25e8,
            "accuracy": 1e2
        },
        {
            "size": 1514,
            "initial_bitrate": 1e9,
            "tolerated_frame_loss": 1e-3,
            "expected_bitrate": 7.35e8,
            "accuracy": 1e2
        }
    ]
}

  • Using ByteBlower Endpoint

    {
    "server": "byteblower-server.example.com.",
    "meeting_point": "byteblower-meeting-point.example.com.",
    "source": {
        "name": "Source",
        "interface": "nontrunk-1",
        "ipv4": "192.168.5.2",
        "netmask": "255.255.255.0",
        "gateway": "192.168.5.254"
    },
    "destination": {
        "name": "Destination",
        "uuid": "4d9a5fdb-32b4-4523-ace9-5972518de13b",
        "ipv4": true
    },
    "duration": 60,
    "max_iterations": 25,
    "frame_configs": [
        {
            "size": 60,
            "initial_bitrate": 3e6,
            "tolerated_frame_loss": 1e-3,
            "expected_bitrate": 1.7e8,
            "accuracy": 1e4
        },
        {
            "size": 124,
            "initial_bitrate": 6e6,
            "tolerated_frame_loss": 1e-3,
            "expected_bitrate": 4.5e8,
            "accuracy": 1e4
        },
        {
            "size": 252,
            "initial_bitrate": 8e6,
            "tolerated_frame_loss": 1e-3,
            "expected_bitrate": 5.7e8,
            "accuracy": 1e3
        },
        {
            "size": 508,
            "initial_bitrate": 8e8,
            "tolerated_frame_loss": 1e-3,
            "expected_bitrate": 6.6e8,
            "accuracy": 1e3
        },
        {
            "size": 1020,
            "initial_bitrate": 1e9,
            "tolerated_frame_loss": 1e-3,
            "expected_bitrate": 7.15e8,
            "accuracy": 1e2
        },
        {
            "size": 1276,
            "initial_bitrate": 1e9,
            "tolerated_frame_loss": 1e-3,
            "expected_bitrate": 7.25e8,
            "accuracy": 1e2
        },
        {
            "size": 1514,
            "initial_bitrate": 1e9,
            "tolerated_frame_loss": 1e-3,
            "expected_bitrate": 7.35e8,
            "accuracy": 1e2
        }
    ]
}