Traffic model for the interconnection of networks and operators using MPLS-TE

In this paper, the main features of MPLS Traffic Engineering are presented to illustrate how telecommunication service providers use them to create interconnections between each other in order to offer telecom services satisfying QoS commitments. Based on previous traffic models, a new model, which deals with traffic queue balancing for different Classes of Service, and for a provider using another provider ́s network is presented. The model output shows that carrying another operator’s traffic may increase delays in an undesirable manner, forcing the carrier to increase the serving rate of LSRs until Utilization is below 60 %. In order to validate the model, a number of network scenarios are implemented in the Wolfram Mathematica 10.1 Study Version, based on study case configurations of an MPLS network. The total global model is useful for future implementation of test-beds of interconnected providers under an MPLS environment.


Resumen
Se presentan las principales características de Ingeniería de Tráfico para MPLS, con el fin de mostrar cómo los operadores de telecomunicaciones las usan para ofrecer sus servicios cuando requieren interconectarse, cumpliendo con condiciones de Calidad del Servicio. Este trabajo parte de modelos de tráfico previos para proponer uno nuevo, que haga el balanceo de colas para diferentes Clases de Servicios de un proveedor que está usando la red de otro proveedor. Al final se demuestra que el transporte de tráfico de un segundo operador puede incrementar retardos de forma indeseable, forzando al primer proveedor a incrementar la tasa de servicio de los LSR hasta alcanzar una utilización menor al 60 %. Para validar el modelo son implementados algunos escenarios en Wolfram Mathematica 10.1 Study Version, escenarios que están basados en las configuraciones hechas en un caso de estudio. El modelo global es útil para futuras implementaciones en bancos de pruebas para la interconexión de operadores en ambientes MPLS.
1. The network designer must define the mechanism for the interexchange of IP and VPN addresses between domains. Three mechanisms are available, a, b and c, of which the least common is option c, and the most useful for inter-provider networks is option b [2].
2. Computation of the shortest paths between ingress and egress routers. Two types of methods are employed for this purpose: on-line and off-line. Cerav Erbas and Mathar [5] present an example of an off-line traffic model, while Hao and Jin [6] detail an on-line routing algorithm for path computation.
3. Defining the utilization of links in the whole MPLS domain, which commonly should not be more than 81 %, in accordance with classical teletraffic theory [7]. However, [8], [9], [10], and [11] have suggested that under some conditions of controlled jitter and delay, it is possible to achieve up to 99 % of link utilization.
4. Management of bandwidth assigned for links, paths and queues. MPLS-TE may involve the application of two different mechanisms: IGP metric tuning, and MPLS DS-TE (Diffserv-Aware MPLS-TE) [12].
5. Signaling path computation using messages known as RSVP (Resource Reservation Protocol), defined in [13] with extensions. These messages establish and end TE-LSPs, refresh, and alert failures occurring during the period of TE-LSP functioning.
Section 3 focuses on bandwidth and queue management for different Classes of Traffic (items 1, 3 and 4 above) because these features of MPLS are the most closely related to traffic.
Guichard et al. [14] presented a hypothetical example of a global operator providing Internet, voice, data, layer 3 VPN, and other services. This operator, known as Globenet, provides telecommunication services in several regions of the world through the interconnection of its Points of Peering (POP).
Globenet made agreements of interconnection with other regional providers, partly aimed at reaching more customers with less investment in infrastructure, but mostly to make communication possible between clients in separate regions (for instance, from Europe to Africa). In Africa, Globenet previously had only two POP: one in Algeria and the other in South Africa; thus, to connect customers in Europe with those in South Africa, they used a leased line from Lisbon to Johannesburg. However, this was expensive and not particularly efficient, and as a result, Globenet made an agreement with the African Provider ATC, involving using ATC's network to reach its customers in different parts of Africa. Globenet then installed a Virtual POP (VPOP) in Johannesburg, with the equipment located in ATC's buildings. ATC also has POPs in Paris and Frankfurt to facilitate interconnection with Globenet in Europe (Fig. 1).
In [14], for means of illustration, the authors start from the premise that ATC does not use its EF queues, instead disposing of them and their assigned bandwidth for the transportation of Globenet's traffic as a Virtual Leased Line (VLL). One very important aspect of this agreement is that Globenet EF, AF and DF traffic is transported by ATC in its network as EF traffic; Globenet's traffic must therefore be marked in such a way that it is delivered to its VPOP in Johannesburg classified just as it was before entering ATC's network. In this way, customers can enjoy the standards outlined in the Globenet QoS policy.
Considering all of the above information, the following questions must be answered: It is uncommon for a provider to choose not to use its EF queues when disposing of its network for another provider's traffic? What type of queue management must this operator thus perform in order to use its EF queues for transporting, both its own traffic and that of another provider? What must this operator do if it decides to transport both sets of traffic in the AF or DF queues?
Several studies have defined optimization models and algorithms employed for different interconnection purposes. In addition to that discussed earlier in this section, it is also important to mention the work carried out in [15], which presents an interconnection model that uses a PCE-based interlayer technique to compute shortest paths based on GMPLS Traffic Engineering methods. Although it does not focus on path computation between providers or domains, the model considers path computation between layers rather than network nodes, which is especially useful for systems that combine many different layer technologies.
The model presented in the following section is based on work outlined in [16], a flow-based model designed for queue balancing in LSR. This model can adapt to traffic assignment when QoS applies, and was also employed in the model discussed in Section II by taking into account traffic from another provider borrowing the network. In [17], a similar model for queue balancing is proposed, but instead it is applied over wireless mesh networks. Simulation results for this model revealed stable queue sizes at low congestion levels, after applying the proposed optimization mechanisms.
Reference [18] presents a queuing network model developed primarily for inter-area PCE based on shortest path computation, but which can also be adapted to inter-domain shortest path computation. This work is significant because it focuses on traffic generated by request-response messages for path computation. More recent work presented in [19] includes a procedure that allows the multiple discovery of inter-domain sequences and path computation over each sequence. Although its perspective is similar to that of [18,19] focuses on proposing more efficient and short path algorithms, and inter-domain loop avoidance, rather than on a mathematical model as is the case in the present work. Similar results are outlined in [20], in which a tree schema is defined for the computation of the shortest point-to-multipoint (P2MP) paths. Computation starts in a PCE in the destination domain, with each PCE determining the number of paths downstream, and providing information to upstream PCE, until some PCE in the initial domain complete the whole tree. A very similar approach is undertaken in [18], in which Virtual Shortest Path Trees (VSPT) are shared between PCEs in computing end to end TE-LSPs, without sharing topology information between domains. Reference [20] is also not model-focused, but is centered on mechanisms and algorithms aimed at optimal shortest path computation. The same is true for [21], in which different procedures are proposed with the same aim of optimizing inter-domain path computation.
The mathematical model described in Section 3 represents a reference starting point for the elaboration of routing mechanisms and optimization algorithms as in [19][20][21].
Because the models presented in [16] were not validated, Section 3 also includes the results of model validation carried out using computational tools, starting from the configuration specifications defined for Globenet in [14].

A. Flow-based model for dynamic queue balancing
Due to the sharing of bandwidth in the borrowed network by both operators, queues must fulfill certain conditions such that link and node (servers) congestion avoidance may be accomplished.
In an MPLS context, nodes are referenced as Label Switched Routers or LSRs, and are responsible for packet forwarding between ingress and egress routers. For a TE-LSP computed between a head-end LSR and a tail-end LSR, all of the LSRs in the path are responsible for serving the queues.
A few modifications must be made to the work detailed in [16] in order to achieve the conditions outlined above regarding the Globenet study case.
Let fi (i=1,…, M) represent the i-th type of traffic intensity of provider 1 (user of the borrowed network) incoming into a node, and gj (j=1,…, N) represent the j-th type of traffic intensity of provider 2 (owner of the borrowed network) incoming into the same node as fi.
Let rj (j=1,…, N) be the bandwidth assigned to the j-th queue, and r the bandwidth assigned to the network segment in which the node is located.
The following two conditions must be fulfilled: Condition (1) assures that queue's assigned bandwidth is never greater than the total outgoing bandwidth of the node, thereby avoiding overload. Condition (2) assures that the traffic intensity generated by both providers is never greater than the bandwidth assigned to all traffic (i.e., all queues).
Normally, RED and WRED drop packets when (2) is not achieved, as these algorithms are designed for congestion avoidance. The bandwidth assigned r is taken by RED and WRED to be the maximum threshold, above which all packets are dropped.
In the case of Globenet, five Classes of Service (CoS), which are transported in five queues, are defined. Let us suppose that the African operator ATC has four queues. Globenet's traffic must still be transported self-defined in the queues of ATC (in all four queues or a selection), labeled in such a way that traffic arrives well classified at the VPOP. This is the reason why sub index j is used for both the traffic types of provider 2 (g j ), and for the assigned queue bandwidth (r j ).
In addition, according to [16], it is necessary to define a control variable vij for dynamic queue management: and (3) This function must be a matrix of ones and zeros, with the ones placed in the column corresponding to the queue for each Globenet CoS; consequently, the following condition must be fulfilled: Then, the traffic intensity can be defined as follows: Condition (4) relates to the prevention of packet loss, while condition (5) is aimed at preventing the traffic intensity for both providers in any queue from being less than the bandwidth assigned to that queue. The left-hand side of (5) is equivalent to the average rate of arrivals λ in the j-th queue. For the EF, AF4 and AF3 queues, the traffic intensity may be greater than the respective assigned bandwidth, because in these cases part of the total bandwidth will be used (Russian Doll Model for Bandwidth Constraints) to assure that these queues' traffic arrives complete at their destination, due to their importance. For the DF queue, the traffic intensity must be less than or equal to (never bigger) the assigned bandwidth, otherwise dropping mechanisms (RED) will have to be actioned.
Finally, according to [16], the mean value of waiting arrivals (i.e., queue size) must be lower than the incoming buffer (queue maximum size before packet dropping starts).
After some operations on both sides of (6), the formula can be expressed in terms of maximum delay time (applying classic teletraffic theory [7] for an M/M/1 system): In this way, an expression of maximum time delay that represents a fundamental performance metric is found. This expression is a function of queue size, server utilization and traffic intensity arriving from both providers to the node, and it is very similar to one defined in [16], with the difference being that the former considers traffic from another provider.
It is important to remember that RED computes the average queue size periodically from the behavior of the arriving traffic, using the following expression defined in [22]: (8)

III. Model valIdatIon
In order to validate the presented model, the Wolfram Mathematica 10.1 Student Version software tool was used, in particular to check the raised equations, based on network configurations outlined in [14].
In the study case presented in Section 1, Globenet defines five Classes of Service (CoS) that are carried in five queues (EF, AF4, AF3, AF2 and DF) in case of congestion, with the African provider ATC defining four CoS (EF, AF4, AF3 and DF). Although all traffic must be carried in the four African CoS, Globenet data must be labeled such that when this traffic arrives at the VPOP, it can be differentiated in terms of the five queues defined by Globenet. This is why in expressions (1) through (7), index j is used for both r (bandwidth) and g (traffic of another provider).
It is important that Globenet's traffic be carried by the African provider in queues, which ensure the former's QoS. For example, although Globenet's EF traffic cannot be carried in ATC's AF or DF queues, the opposite is valid (i.e., Globenet's AF or DF traffic can be carried in ATC's EF queue). Therefore, in the best case, ATC must carry Globenet's EF traffic in its own EF queues.
To achieve a smaller Utilization factor, the network provider must first increase the serving rate in P routers, such that Utilization remains below 80 %, and the Average Queue Time varies at close to 50 milliseconds.
The provider's network designers must also define the maximum time delay in router P, such that the addition of hop-by-hop delay does not increase point-to-point delay in a way that affects QoS.
In the case of the EF queue, Table 1   Queue length variation during one minute of system operation is shown in Fig. 2:   FIg. 2. Queue Length variation during a period of one minute with Utilization ρ=76.46 %. Figure 2 shows a substantial improvement in Queue Length behavior over time. The maximum length is around 25 Mb, but the average value is much lower at around 5 Mb. This will be reflected in a maximum time delay of 50 milliseconds. Figure 3 displays the Time Delay variation over 10 samples after increasing the Service Rate in P routers. As indicated in Section 2, the proposed model is applied by defining a maximum value for Time Delay, testing several traffic flows, and verifying expressions (5) and (7). When a successful result is not achieved, congestion control mechanisms (Priority Queueing PQ or Class-Based Weighted Fair Queueing for EF and AF traffic), and dropping mechanisms (RED or WRED for DF queues) are activated.
FIg . 3. Improvement in Time Delay after increasing the Service Rate in P routers.

Iv. dIscussIon
Neither [16] nor [18] include performance testing of the presented models. Furthermore, the authors of [16] did not carry out any simulation testing either; instead they aimed to present hypothetical cases, and thus defined limits to the values of Globenet's network attributes.
For instance, Globenet [14]  In such situations, Traffic Engineering techniques other than DS-TE should be employed to solve the associated problems, but using methods such as bandwidth reservation inside TE-LSP, and preemption values (preemption is an attribute of MPLS-TE that gives priority to traffic according to their preemption value [4]).
Several researchers have developed on the above idea. Although [23] and [24] do not evaluate Time Delay behavior, they combine class-based bandwidth constraints with the dynamic assignation of preemption values.
The bandwidth management model presented in [23] differs from RDM (Russian Doll Model) and MAM (Maximum Allocation Model) methods in not only defining a maximum bandwidth value per CoS, but also a minimum bandwidth value for certain CoS, such that other service traffic has no effect on the increase or decrease in the overall traffic intensity. Shan and Yang [23] also proposed an algorithm to manage allocated bandwidth based on preemption values.
Although the work detailed in [24] is similar to that mentioned above, the algorithm employed for bandwidth management differs in that it sizes both IP Transfer Delay (IPTD) and IP Delay Variation (IPDV), and considerably delays that increase when links utilization is above 80 % -in this paper, link utilization is almost 100 %-. The work presented in [24] also extends its algorithm to an inter-domain environment (without simulation results), defining a "Bandwidth Broker (BB)" as an entity that manages internal and external information regarding available resources both in its domain and in adjacent ones, interchanging it with another domain's BBs.
Finally, the authors of [25] proposed an algorithm for queue management (mainly queue length control), based on prediction of changes (i.e., an increase or decrease) in the packet arrival rate. This algorithm can achieve great queue length stability around a desired value, despite the increase in traffic or even in the Round Trip Propagation Time (RTPT), and as such it could be used in combination with the model presented in Section 2 to control the waiting time of packets in routers.

v. conclusIons
In this paper, a traffic model for provider interconnection that models the queue management necessary for one provider to carry another's traffic was presented. Additionally, the mechanisms that must be applied if this process generates a congestion condition were described, based on values of Queue Length and Time Delay, following (7). After defining a maximum value of Time Delay, congestion control mechanisms manipulate the queue length and other attributes to assure compliance with (7). An M/M/1 queueing system was implemented in Wolfram Mathematica 10.1 Study Version, following (5) and (7), in which the server is a P router acting as an LSR.
The initial results show that Time Delay and Queue Length increase to unacceptable values for QoS requirements when Utilization is around 98 %. In order to reduce the Time Delay, the provider must increase the Service Rate of P routers sufficiently to set Utilization below 60 %, and thereby satisfy Globenet's constraints regarding Queue Length and Time Delay.
Future work should involve validating the presented model via simulation or experimentation, as well as extending the model to the other aspects of MPLS-TE listed in Section II.A that are not covered in the present paper.