Wednesday 11 September 2013

517. Interconnection, Peering, and Settlements


Interconnection,
Peering, and
Settlements
Abstract
Over the past century the telephony industry has developed a relatively sophisticated set of mechanisms for undertaking cost distribution across multiple service providers. The domain of operation of these models of interprovider interaction extends from those of two-party local transactions up through multiparty international transactions.
The Internet industry presents a number of interesting counterpoints to this observation. The number of Internet service providers is now in the tens of thousands, operating within a business space that is predominantly deregulated. The great number of service providers and the sparse mesh of interconnection lead to a complex environment of interaction. Any particular Internet transaction commonly extends not only across the originating and terminating providers, but involves two or three transit providers as well. It is not uncommon to observe transit paths that entail over 10 service providers. To support this relatively complex environment of interconnection, the Internet industry makes use of only the most basic financial systems of cost distribution, most commonly based on the bilateral relationships of customer/provider and mutual peering.
Similarly, the Internet industry uses a relatively small set of physical mechanisms for supporting interconnection, concentrating on the model of a co-location environment with a local LAN (local area network) switch.
While such simple engineering and financial models do manage to support a very diverse Internet provider industry and also manage to support a diverse set of applications for a very large user base, some inevitable problems have arisen from this model.
This paper examines the various engineering models that are used to support Internet provider interaction, looking at the evolution of the Internet exchange concept of the research Internet of the 1980s into the various forms of interprovider exchange evident in today's Internet.
Above this engineering layer is placed a level of financial interaction between providers, commonly termed "financial settlement." The paper will examine the various models of settlement commonly used in the communications industry, and then examine their applicability to the Internet environment. The requirements of a financial settlement will be examined, as will the relationship between retail service models and settlement models. The conclusion drawn in the paper is that the zero-dollar peering relationship and the customer/provider relationship are the only models that are stable within the Internet environment, and other models of financial interaction pose excessive risk to one or both interconnecting parties. This polarization of the interconnection environment into just two models is an important feature of today's Internet industry.
Such a conclusion is not without its consequences in terms of supportable services in the Internet. For example, widespread deployment of end-to-end quality of service is highly unlikely in such an environment, given that there is no stable mechanism of cost distribution to support the transit of elevated-quality packets. The conclusion also has a number of business outcomes, not the least of which is the long-term inability of such an Internet environment to support a highly diverse provider environment, and the current trend of aggregation within the Internet provider industry is seen as a natural outcome of the current polarized interprovider peering environment. The paper will briefly examine these outcomes and look at the likely directions of the Internet provider industry as a consequence.
To provide some motivation for this issue of ISP interconnection, it is first appropriate to examine the nature of the environment. The regulatory framework that defined the traditional structure of other communications enterprises such as telephony or postal services was largely absent in the evolution of the Internet service industry. The resultant service industry for the Internet is most accurately characterized as an outcome of business and technology interaction, rather than a planned outcome of some regulatory process. In this section we will examine this interaction between business and technology within the ISP environment.
A natural outcome of the Internet model is that the effective control of the retail service environment rests with a network client of an access service rather than with the access service provider, as such a client of an ISP access service has the discretionary ability to resell the access service to third-party clients. In this environment, reselling and wholesaling were very natural developments, with or without the explicit concurrence of the provider ISP. The provider ISP may see this reselling as an additional channel to market for its own Internet carriage services, and may adopt a positive stance by actively encouraging resellers into the market as a means of overall market stimulus, while tapping into the marketing, sales, and support resources of these reselling entities to continue to drive the volumes of the underlying Internet carriage service portfolio. The low barriers to entry to the wholesale market provide a means of increasing the scope of the operation, as, to lift business cash-flow levels, the business enters into wholesale agreements that effectively resell the carriage components of the operation without the bundling of other services normally associated with the retail operation. This process allows the ISP to gain higher volumes of carriage capacity, that in turn allow the ISP to gain access to lower unit costs of carriage.
Figure 1. ISP roles and relationships.
Given that a retail operation can readily become a wholesale provider to third-party resellers at the effective discretion of the original retail client, is a wholesale transit ISP restricted from undertaking retail operations? Again, there is no such natural restriction from a technical or business perspective. An Internet carriage service is a commodity service that does not allow for a significant level of intrinsic product discrimination. The relative low level of value added by a wholesale service operation implies a low unit rate of financial return for that operation. This low unit rate of financial return, together with an inability to effectively competitively discriminate the wholesale product, induces a wholesale provider into the retail sector as a means of improving the financial performance of the service operation. The overall result is that many ISPs operate both as a client and as a provider. Few, if any, reasonable technical-based characterizations draw a clear and unambiguous distinction between a client and service provider when access services to networks are considered. A campus network may be a client of one or more service providers, while the network is also a service provider to campus users. Indeed most networks in a similar situation take on the dual role of client and provider, and the ability to resell an access service can extend to almost arbitrary depths of the reselling hierarchy. From this technical perspective, very few natural divisions of the market support a stable segmentation into exclusively wholesale and exclusively retail market sectors. The overall structure of roles is indicated in Figure 1.
The resultant business environment is one characterized by a reasonable degree of fluidity, in which no clear delineation of relative roles or markets exists. The ISP market environment is, therefore, one of competitive market forces in which each ISP tends to create a retail market presence. However, no ISP can operate in isolation. Each client has the expectation of universal and comprehensive reachability, such that any client of any other Internet ISP can reach the client, and the client can reach a client of any other ISP. The client of an ISP is not undertaking a service contract that limits connectivity only to other clients of the same ISP. As no provider can claim ubiquity of access, every provider relies on every other provider to complete the user-provided picture of comprehensive connectivity. Because of this dependent relationship, an individual provider's effort to provide substantially superior service quality may have little overall impact on the totality of client-delivered service quality. In a best-effort public Internet, the service quality becomes something that can be impacted negatively by poor local engineering but cannot be uniformly improved beyond the quality provided by the network's peers, and their peers in turn. Internet wholesale carriage services in such an environment are constrained to be a commodity service, in which scant opportunity exists for service-based differentiation. In the absence of service quality as an effective service discriminator, the wholesale activity becomes a price-based service with low levels of added value, or in other words a commodity market.
The implication in terms of ISP positioning is that the retail operation, rather than the wholesale activity, is the major area where the ISP can provide discriminating service quality. Within the retail operation, the ISP can offer a wide variety of services with a set of associated service levels, and base a market positioning on factors other than commodity carriage pricing.
Accordingly, the environment of interconnection between ISPs does not break down into a well-ordered hierarchical model of a set of wholesale carriage providers and associated retail service providers. The environment currently is one with a wide diversity of retail-oriented providers, where each provider may operate both as a retail service operator, and a wholesale carriage provider to other retailers.
One of the significant issues that arises here is whether an objective determination can be made of whether an ISP is a peer to, or a client of, another ISP. This is a critical question, as, if a completely objective determination cannot be readily made, the question then becomes one of who is responsible for making a subjective determination, and on what basis.
This question is an inevitable outcome of the reselling environment, where the reseller starts to make multiple upstream service contracts with a growing number of downstream clients of the reselling service. At this point, the business profile of the original reseller is little distinguished from that of the original provider. The original reseller sees no unique value being offered by the original upstream provider and may conclude that it is in fact adding value to the original upstream provider by offering the upstream provider high volume carriage and close access to the reseller's client base. From the perspective of the original reseller, the roles have changed, and the reseller is now perceived as a peer ISP to the original upstream ISP provider.
This assertion of role reversal is perhaps most significant when the generic interconnection environment is one of a zero sum financial settlement, in which the successful assertion by a client of a change from client to peer status results in the dropping of client service revenue without any net change in the cost base of the provider's operation. The party making the successful assertion of peer interconnection sees the opposite, with an immediate drop in the cost of the ISP operation with no net revenue change.
The traditional public regulatory resolution of such matters has been through an administrative process of "licensed" communications service providers, who become peer entities through a process of administrative fiat. In this model, an ISP would become a licensed service provider through the payment of license fees to a communications regulatory body. The license then allows the service enterprise access to interconnection arrangements with other licensed providers. The determination of peer or client is now quite simple: a client is an entity that operates without such a carrier license, and a peer is one that has been granted such an instrument. However, such regulated environments are quite artificial in their delineation of the entities that operate within a market, and this regulatory process often acts as a strong disincentive to large-scale private investment, thereby placing the burden of underwriting the funding of service industries into the public sector. The regulatory environment is changing worldwide to shift the burden of communications infrastructure investment from the public sector, or from a uniquely positioned small segment of the private sector, to an environment that encourages widespread private investment. The Internet industry is at the leading edge of this trend, and the ISP domain typically operates within a deregulated valued-added communications service provider regulatory environment. Individual licenses are replaced with generic class licenses or similar deregulated structures in which formal applications or payments of license fees to operate in this domain are unnecessary. In such deregulated environments no authoritative external entity makes the decision as to whether the relationship between two ISPs is that of a provider and client or that of peers.
If no public regulatory body wants to make such a determination, is there a comparable industry body that can undertake such a role? The early attempts of the Commercial Internet eXchange (CIX) arrangements in the United States in the early 1990s were based on a description of the infrastructure of each party, in which acknowledgments of peer capability were based on the operation of a national transit infrastructure of a minimum specified capability. This specification of peering within the CIX was subsequently modified so that CIX peer status for an ISP was simply based on payment of the CIX Association membership fee.
This CIX model was not one that intrinsically admitted bilateral peer relationships. The relationship was a multilateral one, in which each ISP executed a single agreement with the CIX Association and then effectively had the ability to peer with all other association member networks. The consequence of this multilateral arrangement is that the peering settlements can be regarded as an instance of zero sum financial settlement peering, using a single threshold pricing structure.
Other industry models use a functional peer specification. For example, if the ISP attaches to a nominated physical exchange structure, then the ISP is in a position to open bilateral negotiations with any other ISP also directly attached to the exchange structure. This model is inherently more flexible, as the bilateral exchange structure enables each represented ISP to make its own determination of whether to agree to a peer relationship or not with any other co-located ISP. This model also enables each bilateral peer arrangement to be executed individually, admitting the possibility of a wider diversity of financial settlement arrangements.
The bottom line is that a true peer relationship is based on the supposition that either party can terminate the interconnection relationship and that the other party does not consider such an action a competitively hostile act. If one party has a high reliance on the interconnection arrangement and the other does not, then the most stable business outcome is that this reliance is expressed in terms of a service contract with the other party, and a provider/client relationship is established. If a balance of mutual requirement exists between both parties, then a stable basis for a peer interconnection relationship also exists. Such a statement has no intrinsic metrics that allow the requirements to be quantified. Peering in such an environment is best expressed as the balance of perceptions, in which each party perceives an acceptable approximation of equal benefit in the interconnection relationship in their own terms.
This conclusion leads to the various tiers of accepted peering that are evident in the Internet today. Local ISPs see a rationale to view local competing ISPs as peers, and they still admit the need to purchase trunk transit services from one or more upstream ISPs under terms of a client contract with the trunk provider ISP. Trunk ISPs see an acceptable rationale in peering with ISPs with a similar role profile in trunk transit but perceive an inequality of relationship with local ISPs. The conclusion drawn here is that the structure of the Internet is one where there is a strong business pressure to create a rich mesh of interconnection at various levels, and the architecture of interconnection structures is an important feature of the overall architecture of the public Internet.
One of the physical properties of electromagnetic propagation is that the power required to transmit an electromagnetic pulse over a distance varies in accordance with this distance. The shorter the distance between the transmitter and the receiver, the lower the transmission power budget required; closer is cheaper.
This statement holds true not only for electrical power budgets but also for data protocol efficiency. Minimizing the delay between the sender and receiver allows the protocol to operate faster and operate more efficiently as well; closer is faster, and closer is more efficient.
These observations imply that distinct and measurable advantages are gained by localizing data traffic, that is by ensuring that the physical path traversed by the packets passed between the sender and the receiver is kept as physically short as possible. These advantages are realizable in terms of service performance, efficiency, and service cost. How then are such considerations of locality factored into the structure of the Internet?
A strictly hierarchical model of Internet structure is one in which a small number of global ISP transit operators is at the "top"; a second tier is of national ISP operators; and a third tier consists of local ISPs. At each tier the ISPs are clients of the tier above, as shown in Figure 2. If this hierarchical model were strictly adhered to, traffic between two local ISPs would be forced to transit a national ISP, and traffic between two national ISPs would transit a global ISP, even if both national ISPs operated within the same country. In the worst case, traffic between two local ISPs would need to transit a national ISP, and then a global ISP from one hierarchy, then a second global ISP, and a second national ISP from an adjacent hierarchy in order to reach the other local ISP. If the two global providers interconnect at a remote location, the transit path of the traffic between these two local ISPs could be very long indeed.
As noted above, such extended paths are inefficient and costly, and such costs are ultimately part of the cost component of the price of Internet access. In an open competitive market, strong pressure always is applied to reduce costs. Within a hierarchical ISP environment, strong pressure is applied for the two national providers, who operate within the same market domain, to modify this strict hierarchy and directly interconnect their networks. Such a local interconnection allows the two networks to service their mutual connectivity requirements without payment of transit costs to their respective global transit ISP providers. At the local level is a similar incentive for the local ISPs to reduce their cost base, and a local interconnection with other local ISPs would allow local traffic to be exchanged without the payment of transit costs to the respective transit providers.
Figure 2. A purely hierarchical structure for the Internet.
Although constructing a general interconnection regime based on point-to-point bilateral connections is possible, this approach does not exhibit good scaling properties. Between N providers, who want to interconnect, the outcome of such a model of single interconnecting circuits is (N2 - N) / 2 circuits and (N2 - N) / 2 routing interconnections, as indicated in Figure 3. Given that interconnections exhibit the greatest leverage within geographical local situations, simplifying this picture within the structure of a local exchange is possible. In this scenario each provider draws a single circuit to the local exchange and then executes interconnections at this exchange location. Between N providers who want to interconnect, the same functionality of complete interconnection can be constructed using only N point-to-point circuits.
Figure 3. Fully meshed peering.
One model of an exchange is to build the exchange itself as a router, as indicated in Figure 4. Each provider's circuit terminates on the exchange router, and each provider's routing system peers with the routing process on the exchange router. This structure also simplifies the routing configuration, so that full interconnection of N providers is effected with N routing peer sessions. This simplification does allow greater levels of scaling in the interconnection architecture.
However, the exchange router model does become an active component of the interconnect peering policy environment. In effect, each provider must execute a multilateral interconnection peering with all of the other connected providers. Selectively interconnecting with a subset of the providers present at such a router-based exchange is not easily achieved. In addition, this type of exchange must execute its own routing policy. When two or more providers are advertising a route to the same destination, the exchange router must execute a policy decision as to which provider's route is loaded in the router's forwarding table, making a policy choice of transit provider on behalf of all other exchange-connected providers.
Because the exchange is now an active policy element in the interconnection environment, the exchange is no longer completely neutral to all participants. This imposition on the providers may be seen as unacceptable, in that some of their ability to devise and execute an external transit policy is usurped by the exchange operator's policies.
Figure 4. An exchange router.
Typically, providers have a higher expectation of flexibility of policy determination from exchange structures than this base level of functionality as provided by an exchange router. Providers want the flexibility to execute interconnections on a bilateral basis at the exchange, and to make policy decisions as to which provider to prefer when the same destination is advertised by multiple providers. They require the exchange to be neutral with respect to such individual routing policy decisions.
The modification to the inter-provider exchange structure is to use a local layer 2 switch (or local area network (LAN)) as the exchange element. In this model a participating provider draws a circuit to the exchange and locates a dedicated router on the exchange LAN. This structure is indicated in Figure 5. Each provider executes a bilateral peering agreement with another provider by initiating a router peering session with the other party's router. When the same network destination is advertised by multiple peers, the provider can execute a policy-based preference as to which peer's route will be loaded in the local forwarding table. Such a structure preserves the cost efficiency of using N circuits to effect interconnection at the N provider exchange, while admitting the important policy flexibility provided by up to (N2 - N)/2 potential routing peer sessions.

Figure 5. An exchange LAN.
Early inter-provider exchanges were based on an Ethernet LAN as the common interconnection element. This physical structure was simple, and not all that robust under the pressures of growth as the LAN become congested. Subsequent refinements to the model have included the use of Ethernet switches as a higher capacity LAN, and the use of Fiber Distributed Data Interface (FDDI) rings, switched FDDI hubs, fast Ethernet hubs, and switched fast Ethernet hubs. Exchanges are very high traffic concentration points, and the desire to manage ever higher traffic volumes has lead to the adoption of gigabit Ethernet switches as the current evolutionary technology step within such exchanges.
The model of the exchange co-location accommodates a model of diversity of access media, in which the provider's co-located router undertakes the media translation between the access link protocol and the common exchange protocol.
The local traffic exchange hub does represent a critical point of failure within the local Internet topology. Accordingly, the exchange should be engineered in the most resilient fashion possible, using standards associated with a premium quality data center. This structure may include multiple power utility connections, uninterruptible power supplies, multiple trunk fiber connections, and excellent site security measures.
The exchange should operate neutrally with respect to every participating ISP, with the interests of all the exchange clients in mind. Therefore, exchange facilities, which are operated by an entity that is not also a local or trunk ISP, enjoy higher levels of trust from the clients of the exchange.
There are also some drawbacks to an exchange, and a commonly cited example is that of imposed transit. If an exchange participant directs a default route to another exchange router, then, in the absence of defensive mechanisms, the target router will carry the imposed transit traffic even when there is no routing peering or business agreement between the two ISPs. Exchange located routers do require careful configuration management to ensure that route peering and associated transit traffic matches the currently executed interconnection agreements.
Distributed exchange models also have been deployed in various locations. This deployment can be as simple as a metropolitan FDDI extension, in which the exchange comes to the provider's location rather than the reverse, as indicated in Figure 6. Other models that use an Asynchronous Transfer Mode (ATM)-based switching fabric, using LAN Emulation (LANE) to mimic the layer 2 exchange switch functionality, also have been deployed. Distributed exchange models attempt to address the significant cost of operating a single co-location environment with a high degree of resilience and security, but do so at a cost of enforcing the use of a uniform access technology between every distributed exchange participant.
Figure 6. A distributed exchange.
However, the major challenge of such distributed models is that of switching speed. Switching requires some element of contention resolution, in which two ingress data elements that are addressed to a common egress path require the switch to detect the resource contention and then resolve it by serializing the egress. Switching, therefore, requires signaling, in which the switching element must inform the ingress element of switch contention. To increase the throughput of the switch, the latency of this signaling must be reduced. The dictates of increased switching speed have the corollary of requiring the switch to exist within the confines of a single location, if exchange performance is a paramount concern.
Besides speed, we must consider the cost shift. In a distributed exchange model, the exchange operator operates the set of access circuits that form the distributed exchange. This process increases costs to providers, while it prevents the provider from using a specific access technology that matches their business requirements of cost and supportable traffic volume. Not surprisingly, to date the most prevalent form of exchange remains the third-party hosted co-location model. This model admits a high degree of diversity in access technologies, while still providing the substrate of an interconnection environment that can operate at high speed and therefore manage high traffic volumes.
The co-location environment is often broadened to include other functions, in addition to a pure routing and traffic exchange role. For a high-volume content provider, the exchange location offers minimal transit distance to a large user population distributed across multiple local service providers, as well as allowing the content provider to exercise a choice in selecting a non-local transit provider.
The exchange operator can also add value to the exchange environment by providing additional functions and services, as well as terminating providers' routers and large-volume content services. The exchange location within the overall network topology is an ideal location for hosting multicast services, as the location is quite optimal in terms of multicast carriage efficiency. Similarly, Usenet trunk feed systems can exploit the local hub created by the exchange. The overall architecture of a co-location environment that permits value-added services, which can productively use the unique environment created at an exchange, is indicated in Figure 7.
Figure 7. Exchange-located service platforms.
The role of the exchange was broadened with the introduction of the network access point (NAP) in the National Science Foundation (NSF)-proposed post-NSFNET architecture of 1995.
The NAP was seen to undertake two roles: the role of an exchange provider between regional ISPs that want to execute bilateral peering arrangements and the role of a transit purchase venue, in which regional ISPs could execute purchase agreements with one or more of a set of trunk carriage ISPs also connected at the NAP. The access point concept was intended to describe access to the trunk transit service. This mixed role of both local exchange and transit operations leads to considerable operational complexity, in terms of the transit providers being able to execute a clear business agreement. What is the bandwidth of the purchased service in terms of requirements for trunk transit, versus the access requirements for exchange traffic? If a local ISP purchases a transit service at one of the NAPs, does that imply that the trunk provider is then obligated to present all the ISP's routes at remote NAPs as a peer? How can a trunk provider distinguish between traffic presented to it on behalf of a remote client versus traffic presented to it by a local service client?
We also should consider the issue that the quality of the purchased transit service is colored by the quality of the service provided by the NAP operator. Although the quality of the transit provider's network may remain constant, and the quality of the local ISP's network and ISP's NAP access circuit may be acceptable, the quality of the transit service may be negatively impacted by the quality of the NAP transit itself.
One common solution is to use the NAP co-location facility to execute transit purchase agreements and then use so-called backdoor connections for the transit service provision role. This usage restricts the NAP exchange network to a theoretically more simple local exchange role. Such a configuration is illustrated in Figure 8.
Figure 8. Peering and transit purchase.
For the ISP industry, a number of attributes are considered highly desirable for an exchange facility. The common model of an Internet exchange includes many, if not all, of the following elements:
  • Operated by a neutral party that is not an ISP (to ensure fairness and neutrality in the operation of the exchange)
  • Constructed in a robust and secure fashion
  • Located in areas of high density of Internet market space
  • Able to scale in size
  • Operate in a fiscally sound and stable business fashion
A continuing concern exists about the performance of exchanges and the consequent issue of quality of services that traverse the exchange. Many of these concerns stem from an exchange business model that may not be adequately robust under pressures of growth from participating ISPs.
The exchange business models typically are based on a flat-fee structure. The most basic model uses a fee structure based on the number of rack units used by the ISP to co-locate equipment at the exchange. When an exchange participant increases the amount of traffic presented over an access interface, under a flat-fee structure, this increased level of traffic is not accompanied by any increase in exchange fees. However, the greater traffic volumes do imply that the exchange itself is faced with a greater traffic load. This greater load places pressure on the exchange operator to deploy further equipment to augment the switching capacity, without any corresponding increase in revenue levels to the operator.
For an exchange operator to base tariffs on the access bandwidths is not altogether feasible, given that such access facilities are leased by the participating ISPs and the access bandwidth may not be known to the exchange operator. Nor is using a traffic-based funding model possible given that an exchange operator should refrain from monitoring individual ISP traffic across the exchange, given the unique position of the exchange operator. Accordingly, the exchange operator has to devise a fiscally prudent tariff structure at the outset that enables the exchange operator to accommodate large-scale traffic growth, while maintaining the highest possible traffic throughput levels.
Alternatively there are business models in which the exchange is structured as a cooperative entity between a number of ISPs. In these models the exchange is a nonprofit common asset of the cooperative body. This model is widely used, but also one that is prone to the economic condition of the Tragedy of the Commons. It is in everyone's interest to maximize exploitation of the exchange, while no single member wants to underwrite the financial responsibility for ensuring that the quality of the exchange itself is maintained.
The conclusion that can be drawn is that the exchange is an important component of Internet infrastructure, and the quality of the exchange is of paramount importance if it is to be of any relevance to ISPs. Using an independent exchange operator whose income is derived from the utility of the exchange is one way of ensuring that the exchange is managed proficiently and that the service quality is maintained for the ISP clients of the exchange.
Enhancing the Internet infrastructure is quantified by the following objectives:
  • Extension of reachability.
  • Enhancement of policy matching by ISPs.
  • Localization of connectivity.
  • Backup arrangements for reliability of operation.
  • Increasing capacity of connectivity.
  • Enhanced operational stability.
  • Creation of a rational structure of the connection environment to allow scalable structuring of the address and routing space in order to accommodate orderly growth.
We have reached a critical point within the evolution of the Internet. The natural reaction of the various network service entities in response to the increasing number of ISPs will be to increase the complexity of the interconnection structure to preserve various direct connectivity requirements. Today, we are in the uncomfortable position of increasingly complex inter-provider connectivity environment which is stressing the capability of available technologies and equipment. The inability to reach stable cost distribution models in a transit arrangement creates an environment in which each ISP attempts to optimize its position by undertaking as many direct 1:1 connections with peer ISPs as it possibly can. Some of these connections are managed via the exchange structure. Many more are implemented as direct links between the two entities. Given the relative crudity of the inter-AS routing policy tools that we use today, this structure must be a source of some considerable concern. The result of a combination of an increasingly complex mesh of inter-AS connections, together with very poor tools to manage the resultant routing space, is an increase in the overall instability of the Internet environment. In terms of meeting critical immediate objectives, however, such dire general predictions do not act as an effective deterrent to these actions.
The result is a situation in which the inter-AS space is the critical component of the Internet. This space can be viewed correctly as the demilitarized zone within the politics of today's ISP-based Internet. In the absence of any coherent policy, or even a commonly accepted set of practices, the lack of administration of this space is a source of paramount concern.
We have examined the business drivers behind the adoption of the exchange model as the common basis of interconnection, and also examined the advantages and pitfalls associated with the operation of such exchanges within the public Internet. In continuing our examination of the technology and business considerations that are significant within the subject of Internet Service Provider (ISP) interconnection, we will now focus on the topic from a predominately business perspective.
Any large multi-provider distributed service sector has to address the issue of cost distribution at some stage in its evolution. Cost distribution is the means by which various providers can participate in the delivery of a service to a customer who purchases a service from a single provider, and each provider can be compensated for their costs in an equitable structure of inter-provider financial settlement.
As an example, when an airline ticket is purchased from one air service provider, various other providers and service enterprises may play a role in the delivery of the service. The customer does not separately pay the service fee of each airport baggage handler, caterer, or other form of service provider. The customer's original fare, paid to the original service provider, is distributed by the service provider to other providers who incurred cost in providing components of the total service. These costs are incurred through sets of service contracts, and are the subject of various forms of inter-provider financial settlements, all of which are invisible to the customer.
The Internet is in a very similar situation. Some 50,000 constituent networks must interconnect in one fashion or another to provide comprehensive end-to-end service to each client. In supporting a data transaction between two clients, the two parties often are not clients of the same network. Indeed, the two client service networks often do not directly interconnect, and one or more additional networks must act in a transit provider role to service the transaction. Within the Internet environment, how do all the service parties to a transaction, who incur cost in supporting the transaction, receive compensation for their cost? What is the cost distribution model of the Internet?
Here, we examine the basis for Internet inter-provider cost distribution models and then look at the business models currently used in the inter-provider Internet environment. This area commonly is termed financial settlement, a term the Internet has borrowed from the telephony industry.
What exactly is being exchanged between two ISPs that want to interconnect? In the sense of the meaning of currency as the circulating medium, the question is: What is the currency of interconnection? The technical answer to the question is routing advertisements. When two parties exchange routing entries, the outcome is that traffic flows in response to the flow of routing advertisements. The route advertisement and traffic flows move in opposite directions, as indicated in Figure 9.
Figure 9. Routing and traffic flows.
Within the routing environment of an ISP there are a number of different classes of routes, with the classification based predominately on the way in which the route has been acquired by the ISP:
  • Client routes are passed into the ISP's routing domain by virtue of a service contract with the client. The routes may be statically configured at the edge of the ISP's network, learned by a Border Gateway Protocol (BGP) session with the client, or part of an ISP pool of addresses that are dynamically assigned to the client as part of the dial-up session.
  • Internal ISP routes fall into a number of additional categories. Some routes correspond to client services operated by the ISP, solely for access to the clients of the ISP, such as Web caches, point of presence (POP) mail servers, and game servers. Some routes correspond to ISP-operated client services that require Internet-wide access, such as Domain Name System (DNS) forwarders and Simple Mail Transfer Protocol (SMTP) relay hosts. Lastly are internal services with no visibility outside the ISP network, such as Simple Network Management Protocol (SNMP) network management platforms.
  • Upstream routes are learned from upstream ISPs as part of a transit service contract the ISP has executed with the upstream provider.
  • Peer routes are learned from exchanges or private interconnections, corresponding to routers exported from the interconnected ISP.
How then should the ISP export routes so that the inbound traffic flow matches the outbound flows implied by this route structure? The route export policy is generally structured along the following lines:
  • Clients. All available routes in the preceding four categories, with the exception of internal ISP service functions, should be passed to clients, either in the form of a default route or as explicit route entries passed via a BGP session.
  • Upstream providers. All client routes and all internal ISP routes corresponding to Internet-wide services should be passed to upstream providers. Some clients may want further restrictions placed on their routes being advertised in such a fashion. The ability for a client to specify such caveats on the routing structure, and the mechanism used by the ISP to allow this to occur, should be clearly indicated in the service contract.
  • Peer ISPs. All client routes and all ISP routes corresponding to Internet-wide service should be passed to peer ISPs. Again the client may want to place a restriction on such an advertisement of their routes as a qualification to the ISP's own route export policy.
This structure is shown in Figure 10.
Figure 10. External routing interaction.
The implicit outcome of this routing policy structure is that the ISP does not act in a transit role to peer ISPs and does not permit peer-to-peer transit or peer-to-upstream transit. Peer ISPs have visibility only to clients of the ISP. From the service visibility perspective, client-only services are not visible to peer ISPs or upstream ISPs, and, therefore, value-added client services are implicitly visible only to clients and only when they access the service through a client channel.
Financial settlements have been a continual topic of discussion within the domain of Internet interconnection. To look at the Internet settlement environment, let's first look at the use of inter-provider financial settlements within the international telephony service industry. Then, we will look at the application of these generic principles to the Internet environment.
Within the traditional telephony model, inter-provider peering takes place within one of three general models:
The first, and highly prevalent, international peering model is that of bilateral settlements. A call minute is the unit of settlement accounting. A call is originated by a local client, and the local client's service provider charges the client for the duration of the entire end-to-end call. The call may pass through, or transit, a number of providers, and then terminate within the network of the remote client's local provider. The cost distribution mechanism of settlements is handled bilaterally. In the most general case of this settlement model the originating provider pays the next hop provider to cover the costs of termination of the call. The next hop provider then either terminates the call within the local network, or undertakes a settlement with the next hop provider to terminate the call. The general telephony trunk model does not admit many multi-party transit arrangements. The majority of telephony settlements are associated with trunk calls that involve only two providers: the originating and terminating providers. Within this technology model, the bilateral settlement becomes easier, as the model simplifies to the case where the terminating provider charges the originating provider a per-call minute cost within an accounting rate that has been bilaterally agreed between the two parties. As both parties can charge each other using the same accounting currency, the ultimate financial settlement is based on the net outcome of the two sets of call minute transactions with the two call minute termination accounting rates applied to these calls. (There is no requirement for the termination rates for the two parties to be set at the same level.) Each provider invoices the originating end user for the entire call duration, and the financial settlements provide the accounting balance intended to ensure equity of cost distribution in supporting the costs of the calls made between the two providers. Where there is equity of call accounting rates between the two providers, the bilateral inter-provider financial settlements are used in accordance with originating call minute imbalance, in which the provider hosting the greater number of originating call minutes pays the other party according to a bilaterally negotiated rate as the mechanism of cost distribution between the two providers.
This accounting settlement issue is one of the drivers behind the increasing interest in Voice-over IP solutions, because typically no accounting rate settlement component exists in such solutions, and the call termination charges are cost-based, without bilateral price setting. In those cases where accounting rates have come to dominate the provider's call costs, Voice-over IP is perceived as an effective lever to bypass the accounting rate structure and introduce a new price point for call termination in the market concerned.
The second model, rarely used in telephony interconnection, is that of Sender Keep All (SKA), in which each service provider invoices their originating client's user for the end-to-end services, but no financial settlement is made across the bilateral interconnection structure. Within the bilateral settlement model, SKA can be regarded as a boundary case of bilateral settlements, where both parties simply deem the outcome of the call accounting process to be absolutely equal, and consequently no financial settlement is payable by either party as an outcome of the interconnection.
The third model is that of transit fees, in which the one party invoices the other party for services provided. For example, this arrangement is commonly used as the basis of the long-distance provider local access provider interconnection arrangements. Again, this can be viewed as a boundary case of a general bilateral settlement model , where in this case the parties agree to apply call accounting in only one direction, rather than bilaterally.
The international telephony settlement model is by no means stable, and currently significant pressure is being placed on the international accounting arrangements to move away from bilaterally negotiated uniform call accounting rates to rates separately negotiated for calls in each direction of a bilateral interconnection. Simultaneously, communications deregulation within many national environments is changing the transit fee model, as local providers extend their network into the long-distance area and commence interconnection arrangements with similar entities. Criticism also has been directed at the bilaterally negotiated settlement rates, because of the observation that in many cases the accounting rates are not cost-based rates but are based on a desire to create a revenue stream from accounting settlements.
A number of critical differences exist between the telephony models of interconnection and the Internet environment, which have confounded all attempts to cleanly map telephony interconnection models into the Internet environment.
Internet Settlement Accounting by the packet. Internet interconnection accounting is a packet-based accounting issue, because there is no "call minute" in the Internet architecture. Therefore, the most visible difference between the two environments is the replacement of the call with the packet as the currency unit of interconnection. Although we can argue that a Transmission Control Protocol (TCP) session has much in common with a call, this concept of an originating TCP call minute is not always readily identified within the packet forwarding fabric, and accordingly it is not readily apparent that this is a workable settlement unit. Unlike a telephony call, no concept of state initiation exists to pass a call request through a network and lock down a network transit path in response to a call response. The network undergoes no state change in response to a TCP session, and therefore, no means is readily available to the operator to identify that a call has been initiated, and by which party. Of course the use of UDP, and various forms of tunneling traffic, also confound any such TCP call minute accounting mechanism.
Packets may be dropped. When a packet is passed across an interconnection from one provider to another, no firm guarantee is given by the second provider that the packet will definitely be delivered to the destination. The second provider, or subsequent providers in the transit path, may drop the packet for quite legitimate reasons, and will remain within the protocol specification in so doing. Indeed, the TCP protocol uses packet drop as a rate-control signal. For the efficient operation of the TCP protocol, some level of packet drop is a useful and anticipated event. However, if a packet is used as the accounting unit in a general cost distribution environment, should the provider who receives and subsequently drops the packet be able to claim an accounting credit within the interconnection? The logical response is that such accounting credits should apply only to successfully delivered packets, but such an accounting structure is highly challenging to implement accurately within the Internet environment.
Routing and traffic flow are not always paired. Packet forwarding is not a verified operation. A provider may choose to forward a packet to a second provider without reference to the particular routes the second provider is advertising to the first party. A packet also may be forwarded to the second provider with a source address that is not being advertised to the second provider. Given that the generic Internet architecture strives for robustness under extreme conditions, attempts to forward a packet to its addressed destination are undertaken irrespective of how the packet may have arrived at this location in the first place, and irrespective of how a packet with reverse header IP addresses will transit the network.
Comprehensive routing information is not uniformly available. Complete information is not available to the Internet regarding the status and reachability of every possible Internet address. Only as a packet is forwarded closer to the addressed destination does more complete information regarding the status of the destination address become apparent to the provider. Accordingly a packet may have incurred some cost of delivery before its ultimate undeliverability becomes evident. An intermediate transit provider can never be completely assured that a packet is deliverable.

No comments:

Post a Comment

647. PRESENTATION SKILLS MBA I - II

PRESENTATION  SKILLS MBA   I - II There are many types of presentations.                    1.       written,        story, manual...