Knowledge Base :: BIFIT Mitigator

Ways to Integrate MITIGATOR into the Network

Mon, 01 Jan 0001 00:00:00 +0000

Terminology and methods for implementing the MITIGATOR DDoS protection system into the network. External and Internal Networks Logically MITIGATOR is connected to two networks: External network (external) – the network from which the traffic that needs cleaning comes; Internal network (internal) – the network in which the protected resources are located.

BGP Interaction

Mon, 01 Jan 0001 00:00:00 +0000

Info Update coming soon. MITIGATOR supports various BGP communication scenarios. Each cluster instance can function as an independent BGP speaker, announce prefixes and send and receive FlowSpec rules. Memo Information about all aspects of BGP interaction settings is given below. Local BGP Parameters Local BGP parameters should be defined for each cluster instance and each instance must be given permission to establish BGP connections with its BGP neighbors.

MITIGATOR Protection Quick Setup

Mon, 01 Jan 0001 00:00:00 +0000

The materials in this article are intended to facilitate understanding of the protection organizing basic principles using MITIGATOR. Below is an example of setting up the system in the first iteration. First, all packets are processed in “General Protection”, then 5-tuple routing rules distribute traffic to protection policies, and after the policies, the traffic returns to “General Protection” for the remaining countermeasures. Processing of back traffic from the protected resource is not considered in this article.

BGP Signaling

Mon, 01 Jan 0001 00:00:00 +0000

The article describes how MITIGATOR uses BGP for signaling upstream telecom operators or security service providers (MSSP). Signaling Goals When MITIGATOR works as a perimeter protection against DDoS attacks, it can independently mitigate an attack up to total incoming bandwidth. To prevent the inbound channels from being saturated, coarse filtering can be enabled at upstream telecom operators or protection service providers.

Checklist for Initial System Setup

Mon, 01 Jan 0001 00:00:00 +0000

List of steps Set up network integration. Specify a license key and set limits. Set up interaction via BGP. Upload GeoIP databases into the system. Set up delivery channels for system event notifications. Set up notifications via syslog. Enable protection. (Optional) Upload logos and background images for various interface themes. (Optional) Select the interface theme type and graph display style. Details 1. Set up network integration. The configuration is performed independently for each instance of the system and is based on the composition of the network infrastructure and the tasks to be solved. Integration options are set on the “Instance” page.

Checklist for Initial Configuration of the Protection Policy

Mon, 01 Jan 0001 00:00:00 +0000

List of steps Make sure the routing rules for the policy are set. Configure the policy to display only the necessary countermeasures. Set up countermeasures. Set up automatic packet capture. Set up autodetection. (Optional) Check the effect on legitimate traffic via the test mode. Enable protection policy. (Optional) Configure the log analyzer. (Optional) Pin countermeasure graphs. Details 1. Make sure the routing rules for the policy are set. Make sure that the routing rules are set for the policy on the “Policy Setup” tab of the “Protection policy” page.

Core Isolation for Performance Optimization

Mon, 01 Jan 0001 00:00:00 +0000

By default, the CPU cores that work with network ports are also used by other subsystems. This can degrade performance and cause Input Errors pps/bps spikes on Port extX/intX graphs. You can take some of the load off these cores by preventing non-critical subsystems from running on them. To do so: Specify isolation of the packet processor cores in the core options through the isolcpus=... and rcu_nocbs=... parameters. It is also recommended to add mitigations=off to disable core security patches.

Hardware Bypass

Mon, 01 Jan 0001 00:00:00 +0000

MITIGATOR supports hardware bypass of Silicom and LR-Link adapters. Tested Silicom adapters: PE2G4BPI35LA Quad Port Copper 1G Ethernet PCIe Bypass Network Adapter (Intel i350AM4) PE2G6BPI35 Six Port Copper 1G Ethernet PCIe Bypass Network Adapter (Intel i350AM4) PE210G2BPI9 Dual Port Fiber 10G Ethernet PCIe Bypass Network Adapter (Intel 82599ES) PE310G4BPI71 Quad Port Fiber 10G Ethernet PCIe Bypass Network Adapter (Intel XL710BM1) PE340G2BPI71 Dual Port Fiber 40G Ethernet PCIe Bypass Network Adapter (Intel XL710) P4CG2BPI81 Dual Port Fiber 100G Ethernet PCIe Gen 4.0 Bypass Network Adapter (Intel E810) Tested LR-Link adapters:

Graphs

Mon, 01 Jan 0001 00:00:00 +0000

In MITIGATOR for working with graphs are used: ClickHouse, Graphite, Grafana, Carbon, CarbonAPI. All of the above have the ability to be fine-tuned depending on requirements. Articles describing various set up aspects: Grafana Separate Graphite Metrics Retention Single Graphite

Incident Chart Update Period

Mon, 01 Jan 0001 00:00:00 +0000

The default update period for incident graphs is 60 seconds. For the convenience of working with incident charts, you may need to change the period for their update: Create a file incident.yml and put following in it: services: backend: environment: INCIDENT_UPDATE_PERIOD: "30" Where 30 is the update period value in seconds. In the .env file, set the incident.yml variable: COMPOSE_FILE=docker-compose.yml:docker-compose.override.yml:incident.yml Restart MITIGATOR: systemctl restart mitigator Related Content Access to the Grafana Interface Configuration Change Configuring Tiered Protection with MITIGATOR Core Isolation for Performance Optimization Graphite on a Separate Server Packet Processor Settings Pgfailover Documentation Setting the Storage Time for Metrics in Graphite System Setup for Mellanox (NVIDIA) Adapters Using a Single Graphite for Multiple MITIGATOR Clusters

Managing BGP Announcements When the External Router is Inactive

Mon, 01 Jan 0001 00:00:00 +0000

To ensure uninterrupted traffic through the MITIGATOR, it is usually necessary to remove the BGP announcements when connectivity with bordering routers is lost.\ By default, with the L3-router network deployment scheme, announcement via BGP can only occur if the bordering routers of the external and internal networks are allowed on MITIGATOR router_mac. In case of loss of connectivity with any of them, announcements are removed. However, in some cases it is required to continue BGP announcements even if bordering routers fail, for which following should be specified in the docker-compose.yml file:

Packet Processor Settings

Mon, 01 Jan 0001 00:00:00 +0000

The packet processor is configured through the dataplane.conf file. Put the file into the MITIGATOR working directory and set the required parameters. Other parameters will have the default values. Comments are specified with #, // or /* */. Available parameters (with default values): # Control socket bind address. control_address: 0.0.0.0 # Control socket TCP port. # [1, 65535] control_port: 8888 # Debug control socket TCP port. # [1, 65535] debug_port: 8889 # gRPC control socket TCP port. # [1, 65535] grpc_port: 8890 # Sync/web-challenger control socket UDP port. # [1, 65535] sync_udp_port: 8891 # Number of control socket processing threads. # [1, 1000] control_threads: 20 # Control socket request timeout (seconds). # [1, 10000] control_request_timeout: 15 # Control socket request/response content timeout (seconds). # [1, 10000] control_content_timeout: 300 # Enable control socket event log. control_log: false # Control socket event log message length limit. # [0, 100000] control_log_limit: 100 # Number of sync task threads. # [1, 1000] sync_threads: 10 # Enable sync task log. sync_log: false # Enable additional cores for sync/web-challenger control socket. use_extra_service_cores: false # MAC address of emitted challenge packets. challenge_mac: # VLAN ID of emitted challenge packets. # [1, 4095] challenge_vlan_id: # LACP system ID. # Set to local MAC address if not specified. lacp_system_id: # LACP port operational key. # [0, 65535] lacp_oper_key: 1000 # MAC resolver request retransmit delay (seconds). # [1, 10000] mac_retransmit_time: 5 # MAC resolver entry validity period (seconds). # [1, 10000] mac_reachable_time: 30 # MAC resolver stale entry cleanup timeout (seconds). # [1, 10000] mac_stale_time: 60 # Maximum allowed number of IPv4 policies. # [1, 60000] max_policies: 100 # Maximum allowed number of IPv6 policies. # [1, 60000] max_policies6: 100 # Maximum allowed SIMD instruction bitwidth. # [0, 512] max_simd_bitwidth: # Size of network packet memory pool (per NUMA node). # [0, 2^31] packet_mempool_size: # Enable deferred start of packet processing. # Starts ports only after full application initialization. # Prevents network traffic loops. deferred_start: false # Enable special processing mode of dual-port NICs (for `port_direct_mode: true`). # Process each port on a separate set of cores. # Improves performance for 100G or larger ports when both ports are used. dual_port_nic: false # RX checksum offloading policy: # `false` - do not request the offloading, force software checksum verification. # `true` - enable the offloading for all ports if supported. # Enables offloading only for physical ports if not specified. port_checksum_offload: # Processing mode of network port I/O queues: # `false` - process port I/O queues on dedicated cores, # recommended for low speeds. # `true` - process port I/O queues on worker cores, # recommended for 100G or higher speeds. port_direct_mode: # Network port link speed (Mb/s). # Disables speed autonegotiation if enabled. # [100|1000|10000|...] or [100M|1G|10G|...] port_link_speed: # Enable link state propagation of port pairs. port_lsp: false # Network port MTU. # [0, 65535] port_mtu: 1500 # Size of packet ring buffers between network port I/O queue processing cores # and worker cores (for `port_direct_mode: false`). # [0, 2^20] port_ring_size: 8192 # Number of network port RX descriptors. # [0, 65535] port_rx_desc: # Number of network port TX descriptors. # [0, 65535] port_tx_desc: # Maximum number of retry attempts of network port packet TX. # [0, 2^31] port_tx_retries: 128 # Network port I/O queue processing cores. # Range list [0, 255] port_cores: # Number of network port I/O queue processing cores (per NUMA node). # [0, 256] port_cores_nr: # NUMA nodes of network port I/O queue processing cores. # Range list [0, 7] or `all`: # `all` - use all nodes. port_nodes: # Worker cores. # Range list [0, 255] worker_cores: # Number of worker cores (per NUMA node). # [0, 256] worker_cores_nr: # Worker core NUMA nodes. # Range list [0, 7] or `all`: # `all` - use all nodes. worker_nodes: # Network port list. # Defines network ports used by packet processor. # Configures port pairs, port zones and their order. # Auto-configured for all detected ports if not specified. # If specified, all used ports must be listed. # # Format: # [ext|int] [0, 255] : # port_id: # or [0, 255] or (string). # # Example: ext0: 01:00.0 int0: 01:00.1 ext1: 04:00.0 int1: 04:00.1 Related Content Core Isolation for Performance Optimization Access to the Grafana Interface Configuration Change Configuring Tiered Protection with MITIGATOR Graphite on a Separate Server Incident Chart Update Period MITIGATOR Installation Pgfailover Documentation Setting the Storage Time for Metrics in Graphite System Setup for Mellanox (NVIDIA) Adapters

Performance Optimization for AMD Platforms

Mon, 01 Jan 0001 00:00:00 +0000

BIOS Setup for AMD EPYC and Ryzen Threadripper Platforms Mentioned only essential options that require attention. Option names may differ from listed. Refer to the BIOS configuration manual for your platform. Set one NUMA node per CPU. Use of multiple nodes per CPU is not recommended: NUMA Nodes Per Socket: NPS1 Enable Extended APIC. Recommended for server CPUs:

Programmable Filter

Mon, 01 Jan 0001 00:00:00 +0000

“Programmable filter” (BPF) may be used to create custom countermeasures if the existing ones are not sufficient. Writing programs requires basic programming skills, but allows to solve complex tasks quickly: Protection for applications and protocols that are not supported yet. There’s no need to contact developers and wait for the next release if one understand how an application works and how to protect its traffic. More complex filters than the ACL and REX countermeasures allow. For example one can analyze TCP and IP options, any combinations of packet parameters, headers and payloads either simultaneously or in a sequence.

Reputational Lists From the Analytics Service

Mon, 01 Jan 0001 00:00:00 +0000

The MITIGATOR team generates regularly updated reputational lists of IP addresses, autonomous systems and JA3 fingerprints (hereinafter referred to as “feeds”). Feeds can be imported into MITIGATOR as named lists and used in countermeasures and routing rules. To do this you need to specify Mitigator feeds as a source type. To do this, specify Mitigator feeds as the named list source type and select the required feed. Feeds cannot be downloaded or viewed, even through the MITIGATOR Web interface.

System Setup for Mellanox (NVIDIA) Adapters

Mon, 01 Jan 0001 00:00:00 +0000

System preparation Working with Mellanox (NVIDIA) cards requires the latest driver and device firmware NVIDIA MLNX_OFED. Using the provided link select the latest version of MLNX_OFED for the required distribution of the operating system in the table Current Versions of the Download section. Select the closest version of the OS if the required version is missing. Skip MLNX_OFED installation if the required OS is missing in the list. Tested to work on Debian 10+ and Ubuntu 20.04+.

Configuring Tiered Protection with MITIGATOR

Mon, 01 Jan 0001 00:00:00 +0000

The most effective approach to DDoS protection is a tiered defense architecture, in which surgical filtering of traffic entering the protected network is performed by an on-premise device at the network border, while mitigation mechanisms at the upstream telecom operator level prevent the channel from being saturated. Always on protection at the upstream telecom operator side can cause problems and negatively affect legitimate traffic, so it should be activated only when necessary. For this scenario it is needed to configure interaction between the defense layers - cloud protection.

Pgfailover Documentation

Mon, 01 Jan 0001 00:00:00 +0000

pgfailover monitors the state of a PostgreSQL cluster and acts as a TCP proxy for clients, directing them to the current Primary. When the Primary changes, pgfailover terminates client connections, and they must reconnect. Connection parameters are specified via environment variables with the PGFAILOVER_ prefix: export PGFAILOVER_BIND_ADDRESS=":5432" export PGFAILOVER_SERVERS="postgres://repuser@pg0.example.com/database?sslmode=disable&connect_timeout=5 postgres://repuser@pg1.example.com/database?sslmode=disable&connect_timeout=5" ./pgfailover The server role (Primary/Standby) is checked using pg_is_in_recovery(), by default every 5 seconds (PGFAILOVER_INTERVAL), with the number of connection attempts controlled by the PGFAILOVER_ATTEMPTS environment variable (default: 1).