BGP Interaction

Info

Update coming soon.

MITIGATOR supports various BGP communication scenarios. Each cluster instance can function as an independent BGP speaker, announce prefixes and send and receive FlowSpec rules.

Memo

Information about all aspects of BGP interaction settings is given below.

Local BGP Parameters

Local BGP parameters should be defined for each cluster instance and each instance must be given permission to establish BGP connections with its BGP neighbors.

BGP Neighbors

MITIGATOR instances can establish BGP connections with other network devices defined in the system as BGP neighbors. Any instance can connect to one or more neighbors.

When a BGP neighbor is defined in the system, one needs to specify which instance interacts with it, neighbor’s network parameters, nexthop used in BGP announcement and other parameters. Attribute MED defines priority of traffic route from BGP neighbor to MITIGATOR instance. In case if several equivalent routes are available, the route with the lowest MED value is selected.

Any BGP neighbor can be assigned a property «Prefix checklist source». All prefixes received from such BGP neighbor considered a part of checklist and this helps to avoid traffic blackholing.

MITIGATOR instance can also receive prefixes from BGP neighbor, traffic from which should be directed for scrubbing if “Get prefixes under protection” checkbox is activated for the BGP neighbor. More details about prefixes exchange for traffic scrubbing are in the article.

A BGP neighbor can be associated with a group. FlowSpec rules and prefixes for directing traffic for scrubbing are checked against destination prefixes of the group. This behavior allows avoiding impact on traffic that does not belong to the specified group.

A unique announcement policy can be defined for each BGP neighbor.

Announcement Policy

Announcement policy is a set of parameters for the announces addressed to a specific BGP neighbor. Values for the announcement policy are taken from predefined lists set in the cards “Prefix lists”, “FlowSpec lists” and “Community lists”.

Announcement policy can also have a specific nexthop for a particular prefix list.

If a MITIGATOR instance, which establishes BGP connection with a neighbor, works in “L3 Router” mode, then for announcement to work, each list must be assigned a service zone - a pair of VLAN ID. MITIGATOR will announce lists to BGP neighbor only if the external and internal routers in given service zone are responding to ARP requests.

Prefix Lists

If a prefix list is added to the announcement policy of a BGP neighbor, this neighbor will receive current values of prefixes from the list. A prefix list can be marked as validated (option “Check” next to list’s name should be active), in this case BGP neighbor will receive only those prefixes from the list which remain after validation against the checklist.

Prefix lists can be created manually or automatically.

FlowSpec Lists

If a list of FlowSpec rules is added to the announcement policy of a BGP neighbor, then the neighbor will receive currently active rules for traffic filtering or redirection.

The following actions are supported:

redirect;
drop;
pass;
rate-limit.

Lists of FlowSpec rules can be created manually or automatically.

Community Lists

Community lists are created to manage BGP announcements. Values from the list “Community” can be added as an additional property to the announcement policy of a BGP neighbor for prefix lists and to FlowSpec rules. The mechanism supports Well-known Community: internet, planned-shut, accept-own, route-filter-translated-v4, route-filter-v4, route-filter-translated-v6, route-filter-v6, llgr-stale, no-llgr, blackhole, no-export, no-advertise, no-export-subconfed and no-peer.

Community values can be specified in the following formats:

keyword – Well-known Community name;
32-bit number;
a pair of 16 bit numbers in the format ASN:VALUE, where ASN – ASN number and VALUE – Community number in this ASN;
three 32-bit numbers in the format ASN:LOCAL_DATA_PART1:LOCAL_DATA_PART2, where ASN - autonomous system number, LOCAL_DATA_PART1 and LOCAL_DATA_PART2 - values defined by the user.

Populating Lists

Prefix lists and lists of FlowSpec rules can be defined manually or created by the system automatically.

There are predefined system lists prefixed by system and populated when particular list-specific conditions are met.

Types of Lists

There are two types of system lists: standard and special. Special lists include .blackhole., .signaling. and .amplification.. All other lists are standard.

Populating Standard Lists

Populating standard lists is activated either manually by enabling BGP announcement option in the protection policy, or if any of the autodetection thresholds that has .Announce. in its name is exceeded.

Populating Special Lists

Every special list has its own thresholds for activating addition of new entries:

.blackhole. — the list populates only when an autodetection threshold whose name includes .Amplification. is exceeded. The lists are used to announce prefixes to neighbors, traffic from which should be dropped.
.signaling. — the list populates only when an autodetection threshold whose name includes .Signaling. is exceeded. Lists are used to announce prefixes to neighbors, traffic from which should be directed to the upstream provider or anti DDoS service provider.
.amplification. — the list populates only when an autodetection threshold whose name includes .Amplification. is exceeded. The lists are used to announce flowspec rules for filtering specific type of traffic related to a particular type of UDP amplification.

Populating any special lists cannot be activated manually.

Important: do not confuse .Announce. and .Amplification. thresholds even if they use the same metric.

For example:

Threshold Policy.BGP.Announce.Flow.Arms.InputPps.On = 1 populates standard lists with IP addresses, that, according to Collector, generate traffic that can be classified as amplification and the amount of such traffic is above the threshold. Using a standard list populated in this way, will affect all traffic from the prefixes in the list.

Threshold Policy.BGP.Amplification.Flow.Arms.InputPps.On = 1 populates only one of the FlowSpec rule lists system.policy.flowspec.amplification.arms.prefixes or system.policy.flowspec.amplification.arms.ips depending on the protection policy settings. Using these lists means using the rule template drop udp sport 3283 $prefixes, so they will only affect traffic from port 3283.

Sources of Metrics that can Trigger List Populating

Lists are populated according to MITIGATOR metrics or according to metrics from Collector. All autodetection thresholds that contain .Flow. activate list populating using metrics from Collector. All others — using MITIGATOR metrics.

Monitored Metrics

MITIGATOR provides a large number of different metrics that can trigger autodetection thresholds. Including:

by protection policy ingress traffic rate according to packet processor metrics or according to Collector metrics, either total or for a specific protocol;
by drop rate by particular countermeasures according to packet processor counters;
by particular countermeasures’ drop rate according to Collector metrics in case if MITIGATOR sends sFlow data about dropped packets;
by number of unique source IP addresses according to packet processor metrics or according to Collector’s metrics;
by policy’s “drop rate” in Test Mode (i.e. traffic marked as “dropped”).

The set of metrics for autodetection is constantly growing, up-to-date information about thresholds is available in MITIGATOR Help in “Autodetection parameters” section.

Data for List Population

List name specifies what exactly it will be populated with:

.rules — 5-tuple values from protection policy routing rules;
.prefixes — destination prefixes from protection policy routing rules;
.ips — destination IP addresses, for which HPD or Collector detected exceeding of thresholds.

Possible combinations. For example, if the list name contains .rules.hpd.ips, it means that it is populated based on values of four parameters in routing rules: protocol, source prefix, source port and destination port. Destination prefix is defined using IP addresses, for which HPD detected exceeding of a preset threshold.

Lists with .checked in their names are validated against checklist.

Lists with .protected in their names are used to re-announce prefixes received from BGP neighbor. More details.

Detailed Information about Each List

Below is a list of all system lists and their descriptions.

System Lists of Prefixes for Directing Traffic for Scrubbing

system.policy.prefixes is populated with destination prefixes defined in routing rules for the protection policy. It is not populated if system.policy.ips list is being filled.
system.policy.prefixes.checked is equal to the contents of system.policy.prefixes, but it is validated against the checklist.
system.policy.ips is populated with IP addresses received from Collector which exceed the preset threshold for incoming traffic rate. Checkbox “Add to list IP addresses based on collector data” should be activated in the “Settings” tab on “Protection policy” page. The thresholds for incoming traffic rate in Bps or Pps should be set on the same tab.
system.policy.ips.checked contains the same entries as system.policy.ips but it is validated against the checklist.

System Prefix Lists for Announcing Prefixes Traffic to which should be Dropped

system.policy.blackhole.prefixes is populated with destination prefixes defined in routing rules for the protection policy.
system.policy.blackhole.prefixes.checked contains the same prefixes as system.policy.blackhole.prefixes, but it is validated against the checklist.
system.policy.blackhole.ips is populated with IP addresses received from Collector which exceed the preset threshold for directing traffic to blackhole. The thresholds for incoming traffic rate in Bps or Pps should be set in the “Settings” tab of “Protection policy” page.
system.policy.blackhole.ips.checked contains the same entries as system.policy.blackhole.ips, but it is validated against the checklist.
system.policy.blackhole.hpd.ips is populated with IP addresses which exceed the HPD threshold for incoming traffic.
system.policy.blackhole.hpd.ips.checked contains the same entries as system.policy.blackhole.hpd.ips, but it is validated against the checklist.

System Prefix Lists for Signaling to Upstream Providers or Anti DDoS Service Providers

system.policy.signaling.prefixes is populated with destination prefixes defined in routing rules for the protection policy.
system.policy.signaling.prefixes.checked contains the same prefixes as system.policy.signaling.prefixes, but it is validated against the checklist.
system.policy.signaling.ips is populated with IP addresses received from Collector which exceed the preset threshold for incoming traffic. The thresholds for incoming traffic rate in Bps or Pps should be set in the “Settings” tab of “Protection policy” page.
system.policy.signaling.ips.checked contains the same entries as system.policy.signaling.ips, but it is validated against the checklist.
system.policy.signaling.hpd.ips is populated with IP addresses from protection policy which exceed the preset threshold for incoming traffic according to HPD data.
system.policy.signaling.hpd.ips.checked contains the same entries as system.policy.signaling.hpd.ips, but it is validated against the checklist.

System Prefix Lists for Activating Protection

system.protected.prefixes is populated with prefixes received from BGP neighbor, with activated option “Get prefixes under protection”. If a BGP neighbor is associated with a group, the announced prefixes will be added to the list only if they belong to the group.
system.protected.prefixes.checked contains the same prefixes as system.protected.prefixes, but it is validated against the checklist.

System Lists of FlowSpec Rules for Directing Traffic for Scrubbing

In order to use system lists of FlowSpec rules the user has to define rule templates that are later populated with prefixes or IP addresses. If templates are not defined, the rules are not created. The rules are updated when the templates or the FlowSpec routing rules are changed.

system.policy.flowspec.prefixes is populated with destination prefixes defined in routing rules for the protection policy. The list is not populated if system.policy.flowspec.ips list is populated with Collector’s data.
system.policy.flowspec.rules is populated with full set of parameters defined in routing rules for the protection policy. The list is not populated if system.policy.ips.rules list is populated with Collector’s data.
system.policy.flowspec.ips is populated with IP addresses received from Collector which exceed the preset threshold for incoming traffic rate. On “Settings” tab of “Protection policy” page checkbox “Add to list IP addresses based on collector data” should be active and the thresholds for incoming traffic rate in Bps or Pps should be set.
system.policy.flowspec.ips.rules is populated with values of four parameters in policy’s routing rules: protocol, source prefix, source port and destination port. Destination prefix is defined using IP addresses from Collector which exceed the preset threshold for incoming traffic rate. On “Settings” tab of “Protection policy” page checkbox “Add to list IP addresses based on collector data” should be checked and the thresholds for incoming traffic rate in Bps or Pps should be set.

System Lists of FlowSpec Rules for Announcing Prefixes Traffic from which should be Dropped

system.policy.flowspec.blackhole.prefixes is populated with destination prefixes defined in routing rules for the protection policy.
system.policy.flowspec.blackhole.rules is populated with full set of parameters defined in routing rules for the protection policy.
system.policy.flowspec.blackhole.ips is populated with IP addresses received from Collector traffic rate to which exceeds the preset threshold for directing traffic to blackhole. The thresholds for incoming traffic rate in Bps or Pps should be set on “Settings” tab of “Protection policy” page.
system.policy.flowspec.blackhole.rules.ips is populated with values of four parameters in policy’s routing rules: protocol, source prefix, source port and destination port. Destination prefix is defined using IP addresses from Collector which exceed the preset threshold for incoming traffic rate. The thresholds for incoming traffic rate in Bps or Pps should be set on “Settings” tab of “Protection policy” page.
system.policy.flowspec.blackhole.hpd.ips is populated with IP addresses from protection policy with incoming traffic rate that exceeds the preset threshold for directing traffic to blackhole according to HPD data.
system.policy.flowspec.blackhole.rules.hpd.ips is populated with values of four parameters in policy’s routing rules: protocol, source prefix, source port and destination port. Destination prefix is defined using IP addresses received from HPD incoming traffic rates for which exceed the preset blackholing threshold.

System Lists of FlowSpec Rules for Signaling Upstream providers or Anti DDoS Service Providers

system.policy.flowspec.signaling.prefixes is populated with destination prefixes defined in routing rules for the protection policy.
system.policy.flowspec.signaling.rules is populated with the full set of parameters defined in routing rules for the protection policy.
system.policy.flowspec.signaling.ips is populated with IP addresses received from Collector and whose traffic rate exceeds preset signaling threshold. The thresholds for incoming traffic rate in Bps or Pps should be set on “Settings” tab of “Protection policy” page.
system.policy.flowspec.signaling.rules.ips is populated with IP addresses received from Collector and whose traffic rate exceeds preset signaling threshold. The thresholds for incoming traffic rate in Bps or Pps should be set on “Settings” tab of “Protection policy” page.
system.policy.flowspec.signaling.hpd.ips is populated with protection policy IP addresses whose incoming traffic rate exceeds preset signaling threshold according to HPD.
system.policy.flowspec.signaling.rules.hpd.ips is populated with values of four parameters in policy’s routing rules: protocol, source prefix, source port and destination port. IP addresses whose traffic rate exceeds signaling thresholds according to HPD are used to define the destination prefix.

System FlowSpec Rule Lists for Protection Against UDP amplifications

All lists for protection from amplifications work according to the same logic, but each list is created for protection against a particular type of amplification. * in the name of the list and the corresponding autodetection threshold is a placeholder for amplification type. Available types are: Udp, Dns, IpFragment, Chargen, Cldap, Spss, Arms, Coap, WS, L2tp, mDns, Memcached, MsSqlRs, NetBios, Ntp, RipV1, RpcBind, Snmp, Ssdp.

system.policy.flowspec.amplification.*.prefixes is populated with destination prefixes from policy’s routing rules, for which Collector detected exceeding threshold for traffic rate for corresponding type of amplification.
system.policy.flowspec.amplification.*.ips is populated with IP addresses for which Collector detected exceeding threshold for traffic rate for the corresponding type of amplification.

Activating BGP Announcements

BGP announcements for directing traffic to filtration can be activated manually or automatically.

Manual Activation from Protection Policy

BGP announcements can be activated manually using the “BGP announcement” switch on the “Protection policy” page. If announcements are enabled, the icon is displayed near the switch. Clicking on the icon displays the list of policy prefixes selected for announcement.

Automatic Activation

Automatic BGP announcements activation starts when autodetection thresholds are exceeded.

according to MITIGATOR data;
according to Collector data.

Detailed description of the lists and corresponding activation thresholds is given above.

Automatic Removal of BGP Announcements

BGP announcements can be automatically removed for the following reasons:

meeting the autodetection conditions;
due to packet processor shutdown;
due to gobgp shutdown;
in case if announced prefix is excluded from the checklist;
if external or internal routers IP addresses do not respond to ARP requests.

Disabling by Autodetection Mechanism

When traffic level falls below the threshold set by the predicates for disabling BGP announcements, announcements are disabled automatically. BGP session is not reset.

Shutting down Packet Processor

In case is packet processor is stopped on a particular instance of MITIGATOR, all announcements to BGP neighbors from this instance are automatically removed. BGP session is reset.

Gobgp Shutdown

In case is gobgp is stopped on a particular instance of MITIGATOR, all announcements to BGP neighbors from this instance are automatically removed. BGP session is reset.

Announced Prefix is Removed from the Checklist

If a list with the property “Check” is announced to a BGP neighbor, then if checklist’s source stopped reporting the prefix, the prefix is automatically removed from the announcement to the BGP neighbor. BGP session is not reset.

External or Internal Routers IP Addresses do not Respond to ARP Requests

If a zone is specified in the BGP announcement policy for the list and this zone is not resolved on a MITIGATOR instance, the list is not announced anymore. BGP session is not reset. If needed, this check can be disabled.

Usage Scenarios

Redirecting Traffic to Scrubbing according to BGP Announcement

Let’s consider a network topology with several MITIGATOR instances running in “On-demand” mode.

If no mitigation is required, traffic can be directly routed from R1 to R2 bypassing MITIGATOR. BGP neighbor R1 settings contain a list of prefixes system.policy.prefixes. If BGP announcement is activated, the list is populated with values dst_prefix from routing rules for this policy. By making such announcements MITIGATOR instances inform that the protected prefix 192.0.2.1/32 is available through IPm1 and IPm2. Traffic starts to flow to the MITIGATOR for scrubbing. After the attack is over, BGP announcement is removed and traffic is routed directly from R1 to R2.

The use of the scheme with traffic redirection to MITIGATOR for scrubbing according to BGP announcement provides the following benefits:

increase resilience;
horizontally scale the cluster to increase performance;
protect resources by address;
when needed, allows traffic routing to MITIGATOR in “On-demand” mode, or remove traffic from scrubbing in “Always-on” mode;
simplify maintenance and troubleshooting procedures. To minimize the time of route updates in case of MITIGATOR instance failure, one can use BFD protocol.

Checklists

To avoid a blackhole, when MITIGATOR announces prefixes which are not accessible from downstream router, a mechanism for announcing only prefixes which are guaranteed to be accessible is implemented. In order for this logic to work, MITIGATOR instance needs to get information about the state of resources in the protected network, so one more BGP connection with R2 router is required.

The lists with the property “Check” are validated against the checklist received from a BGP neighbor. System lists that contain .checked in their names are validated, the property “Check” for such lists can not be deactivated.

Let us look at the example of how the announced prefixes list changes depending on what is received from BGP neighbor R2:

Instances MITIGATOR1 and MITIGATOR2 have two BGP neighbors each: Neighbor 1 and Neighbor 2.
Neighbor 1 is a BGP neighbor to which MITIGATOR announces a validated system list of prefixes system.policy.prefixes.checked. Prefix 192.0.2.0/24 is present in the list system.policy.prefixes.checked added from the routing rules for the protection policy.
Neighbor 2 is a BGP neighbor marked as a checklist source.

Example 1:
Neighbor 2 announces that prefix 192.0.0.0/16 is available to MITIGATOR instances.
In such case MITIGATOR instances announce prefix 192.0.2.0/24 to Neighbor 1, since the checklist allows it.

Example 2:
Neighbor 2 announces that prefix 192.0.2.1/32 is available to MITIGATOR instances.
In such case MITIGATOR instances do not announce any prefixes to Neighbor 1, since the mask of the announced prefix is less than the mask of the prefix in the checklist.

More Specific Prefixes

One can permit announcing more specific prefixes than specified in the announced list in local settings.

Example 3:
Local settings of BGP connections of MITIGATOR instances MITIGATOR1 and MITIGATOR2 have “More specific” checkbox activated.
Neighbor 2 announces to MITIGATOR instances that prefix 192.0.2.1/32 is available. In such case MITIGATOR instances announce prefix 192.0.2.1/32 to Neighbor 1 since more specific prefixes than in the list system.policy.prefixes.checked are permitted.

Redirecting Traffic for Scrubbing Using FlowSpec

Redirection of traffic by announcing FlowSpec rules is performed according to similar logic. When BGP announcement is activated, system lists system.policy.flowspec.prefixes and system.policy.flowspec.rules are populated with FlowSpec rules. These lists contain a template with an action, RD and a variable. The list system.policy.flowspec.prefixes is populated with dst_prefix values, and the list system.policy.flowspec.rules is populated with full set of parameters from the routing rules.

For example, a routing rule directing traffic to the Policy1:

Let’s add system list system.policy.flowspec.prefixes to the BGP announcement policy for BGP neighbor R1 a and define a template:

redirect 123:321 $prefixes

In this case the list of FlowSpec rules with value # Policy1 (ID 4) redirect 123:321 dst 192.0.2.1/32 is sent to the BGP neighbor R1. Policy1 (ID 4) is the name and ID of the protection policy.

Similarly, the template for the system.policy.flowspec.rules list:

redirect 123:321 $rules

forms a rule # Policy1 (ID 4) redirect 123:321 protocol 6 dst 192.0.2.1/32 sport 8080 dport 8080 with the full set of parameters from the routing rule.

Upon receiving the rules, the upstream router R1 redirects corresponding traffic for scrubbing.

Directing Traffic to Blackhole according to BGP Announcement

When traffic rate exceeds thresholds in the protection policy or according to the Collector data, the corresponding system lists .blackhole. are populated. When such lists are announced, the community is specified in the BGP announcement politics.

Thus, all the traffic addressed to IP address192.0.2.1/32 will be dropped before reaching MITIGATOR. In this case the legitimate traffic will also be dropped, and it leads to the resource not being available during the attack. After the attack is over, the announcement is withdrawn and the traffic reaches the protected server again.

An indicator near “Send BGP announcements” switch on the page “Protection policy” appears when the blackhole lists are populated.

Placing Prefixes under Protection

MITIGATOR can receive prefixes, whose traffic needs to be scrubbed, from a BGP neighbor. This feature allows to implement a signaling scenario between the client and operator MITIGATOR installations without using additional route reflectors.

For example, an ISP has configured MITIGATOR. Client’s MITIGATOR is added as a BGP neighbor with activated “Get prefixes under protection” function. Client’s MITIGATOR detects an attack and announces signaling lists of attacked prefixes. More details.

Service provider’s MITIGATOR receives prefixes from BGP neighbors and adds them to the system list system.protected.prefixes which is then announced to all BGP neighbors that have the list in the announcement policy. Thus, traffic is directed upstream for scrubbing on a higher infrastructure tier.

Sending FlowSpec Rules for Filtering

When FlowSpec rules list with DROP action is formed and sent to BGP neighbor R1, one can additionally filter traffic on R1 which is an upstream router w.r.t. MITIGATOR instance. Also, one can rate-limit the incoming traffic that satisfies the rule on MITIGATOR’s instance, by using rate-limit action.

Using FlowSpec to Blackhole Traffic

FlowSpec can be used to automatically send rules for traffic blackholing.

When auto-detection conditions for blackhole are met, following FlowSpec rules lists are populated: system.policy.flowspec.blackhole.rules, system.policy.flowspec.blackhole.rules.ips and system.policy.flowspec.blackhole.rules.hpd.ips. These lists contain templates with actions drop dst $PREFIX or drop $RULES.

For example, a routing rule directing traffic to the Policy1 is defined:

Let’s add system list system.policy.flowspec.blackhole.rules to the BGP announcement policy for BGP neighbor R1 and set a template:

drop $RULES

In this case the list of FlowSpec rules with value # Policy1 (ID 4) drop protocol 6 dst 192.0.2.1/32 sport 8080 dport 8080 is sent to BGP neighbor R1. Policy1 (ID 4) is the name and ID of the protection policy.

Upon receiving the rules upstream router R1 drops corresponding traffic.

Receiving Filtering Rules

MITIGATOR can also receive FlowSpec rules from other devices. To do this, a permission for receiving FlowSpec in neighbor’s BGP settings must be enabled.

Neighbors group membership and maximal number of rules are specified in the same place. FlowSpec rules received from group neighbors must have dst_prefix that belongs to the protected prefixes of the group. Rules received from the neighbor are added to FACL countermeasure of General protection. Rules membership in a group is validated by destinationprefix and only non-conflicting rules are added to FACL.

Collector

One can announce specific IP addresses instead of prefixes by activating populating of ips system lists with data from Collector. If Collector detects exceeding threshold of traffic addressed to an IP address, only this address is added to the list and sent to BGP neighbors. Thus, one can direct only traffic addressed to IP addresses under attack for scrubbing, while the rest of the prefix traffic will be sent directly from R1 to R2.

Signaling

MITIGATOR can use BGP interaction for signaling upstream operators or service providers.

More details about signaling to the provider’s equipment.

More details about signaling to the MITIGATOR provider.