From CORD to the 5G era

Introduction

Techno-economic drivers are radically changing Telecommunications and ICT scenarios. Network Operators will have to face unprecedented challenges posed, for example, by the impressive increase of video traffic, the wider and wider diffusion of smart phones, tablets, intelligent things and the emerging new promising service scenarios (e.g., Tactile Internet, Industry 4.0, Cloud Robotics).
In order to face these challenges, Network operators aim at make their networks efficient, programmable, elastic and agile to meet the challenges of user bandwidth demands, as well as to create new revenue streams. They want to benefit from both the economies of scale and the agility that cloud providers enjoy today.
In this context, SDN and NFV are two enabling technologies of an overall systemic transformation called “Softwarization” which is steering the evolution not only of Networks, but also of Service Platforms (e.g., Cloud and Edge Computing architectures) and even future terminals, machines and smart things.
Network Operators are adopting different strategies (e.g., bi-modal, inertial, etc.) to pursue a digital business transformation. Key technological directions concern the architectural delayering and simplifications and the optimization of the operational processes (e.g., through orchestrators and new paradigms of OSS/BSS capable of managing the complexity and heterogeneity of SDN and NFV infrastructures). If mastering the software is recognized a must for a successful digital business transformation, nevertheless, standardization of open source software solutions and interfaces is still felt to be a hot issue. Importantly, Telco Operators argue that the ongoing digital business transformation implies a radical cultural change and adaptation: workforce needs to be completely retrained in this new environment, and new skills have to be developed. Eventually SDN and NFV are lowering the thresholds for new Players to enter in the Telecommunications and ICT ecosystems, so it is likely that Telcos Operators’ strategies will be more and more influenced by the dynamics of new Competitors (even Start-ups) entering their business segments, the analogous transformation of the Technology Providers, as well as the regulation rules.
This paper describes CORD (Central Office Re-engineered as Data center) and OMEC (Open Mobile Edge Cloud). CORD - a collaborative effort between AT&T and the Open Networking Lab -is proposing a response to the above challenges by re-architecting the Central Office as a datacenter; OMEC is an long term architectural approach paving the way to the 5G era.

 

CORD

The basic approach of CORD centers on unifying the following three related but distinct threads:

  • The first is SDN, which is about separating the network’s control and data planes. This makes the control plane open and programmable, and that can lead to increased innovation. It also allows for simplification of forwarding devices that can be built using merchant silicon, resulting in less expensive white-box switches.
  • The second is NFV, which is about moving the data plane from hardware appliances to virtual machines. This reduces CAPEX costs (through server consolidation and replacing high-margin devices with commodity hardware) and OPEX costs (through software-based orchestration). It also has the potential to improve operator agility and increases the opportunity for innovation.
  • The third is the Cloud, which defines the state-of-the-art in building scalable services-leveraging software-based solutions, micro-service architecture, virtualized commodity platforms, elastic scaling, and service composition, to enable network operators to rapidly innovate.

The goal of CORD is not only to replace today’s purpose-built hardware devices with their more agile software-based counterparts, but also to make the CO an integral part of every Telco’s larger cloud strategy and to enable them to support more attractive and meaningful networking services.

 

Commodity Hardware

The target hardware for CORD consists of a collection of commodity servers and storage, interconnected by a leaf-spine fabric constructed from white-box switches. An illustrative example is shown in figure 1.
Although similar in design (albeit on a smaller scale) to a conventional datacenter, there are two unique aspects to this hardware configuration:

  • The switching fabric-organized as a leaf-spine topology-is optimized for traffic flowing east-to-west, between the access network that connects customers to the CO and the upstream links that connects the CO to the operator’s backbone. There is no north-south traffic in the conventional sense.
  • The racks of GPON OLT MACS commoditize connectivity to the access network. They replace proprietary and closed hardware that connects millions of subscribers to the Internet with an open, software-defined solution. (GPON is the example access technology in the reference implementation of CORD, but the same argument applies to other access technologies as well.) In fact the argument for OMEC below is to unify all these approaches in a common framework.
 

Figura 1 - Target hardware, constructed from commodity servers, storage, and switches

Software Building Blocks

With respect to software, our reference implementation of CORD exploits three open source projects:

  • OpenStack is the cluster management platform. It provides the core IaaS capability, and is responsible for creating and provisioning VMs (Virtual Machines) and VNs (Virtual Networks).
  • ONOS is the network operating system that manages the underlying white-box switching fabric. It also hosts a collection of control applications that implement services on behalf of Telco subscribers. ONOS is also responsible for embedding virtual networks in the underlying fabric, which is in turn accessed via OpenStack’s Neutron API.
  • XOS is a service abstraction layer that unifies infrastructure services (provided by OpenStack), control plane services (provided by ONOS. It provides explicit support for multi-tenant services, making it possible to create, name, operationalize, manage and compose services as first-class operations.
 

Figura 2 - Legacy CO, including three physical devices to be virtualized

Virtualizing Legacy Devices

The first step is to virtualize the existing hardware devices, transforming each device into its software service counterpart and running it on commodity hardware. In the process, functionality is likely disaggregated and re-packaged in new ways. Figure 2 shows the devices that CORD virtualizes. They include OLT (Optical Line Termination), CPE (Customer Premises Equipment), and BNG (Broadband Network Gateways).
Due to space limitations, this overview focuses on GPON technology and virtualizing the OLT to produce vOLT; there are also virtualized counterparts to the CPE (vCPE) and BNG (vBNG).
The first challenge is to create a commodity I/O Blade. AT&T has developed an open specification for a GPON MAC 1RU “pizza box,” as shown in figure 3. This board includes the essential GPON MAC (Media Access Control) chip under control of a remote control program via OpenFlow. This replaces an existing closed and proprietary OLT chassis (not shown) that integrates this GPON MAC chip with GPON protocol management, 802.1ad-compliant VLAN bridge, and Ethernet MAC functions.
The end result is to bring the access network interface under the same SDN-based control paradigm as the white-box based switching fabric. In other words, vOLT (virtual OLT), is implemented as a combination of: (1) Merchant Silicon (i.e., commodity GPON interface boards) and (2) an SDN control function that sets up and manages control plane functions of an OLT (e.g., 802.1X, IGMP Snooping, and OAM).
The vOLT control application running on top of ONOS facilitates attachment and authentication (AAA), establishes and manages VLANs connecting consumer devices (see next section) and the CO switching fabric on a per-subscriber basis, and manages other control plane functions of the OLT.

 

Figura 3 - GPON OLT IO Blade

Service Orchestration

Replacing hardware devices with software running in virtual machines is a necessary first step, but is not by itself sufficient. Just as all the devices in a hardware-based CO must be wired together in a meaningful way, their software counterparts must also be managed as a collective. This process is often called service orchestration, but if network operators are to enjoy the same agility as cloud providers, the abstractions that underlie the orchestration framework must fully embrace (1) the elastic scale-out of the resulting virtualized functionality, and (2) the composition of the resulting disaggregated (unbundled) functionality. A model that simply “chains” VMs together as though it is operating on their hardware-based counterparts will not achieve either goal.
Our approach is to adopt Everything-as-a-Service (XaaS) as a unifying principle. This brings the disparate functionality introduced by virtualizing the hardware devices under a single coherent model. The control functions run as scalable services (these functions run on top of ONOS), the data plane functions run as scalable services (these functions scale across a set of VMs), the commodity infrastructure is itself managed as a service (commonly known by the generic name IaaS), and various other global cloud services running in the CO are also managed as scalable services. No matter the role it plays in the overall CORD architecture, each service is structured in exactly the same way: it supports a logically centralized interface, called a service controller; it is elastically scale across a set of service instances (corresponding to VMs and OpenFlow switches); and it is multi-tenant with an associated tenant abstraction.
While vOLT, vCPE, and vBNG refer to the virtualized counterpart of the three physical devices, they are not complete and “pluggable” elements in CORD until they are packaged as services-each includes a multi-tenant service controller, and each scales independently across a set of service instances. Because, the new architecture no longer requires functionality to be bundled along the same boundaries as before, it is more intuitive to think of the virtualization process as resulting in three generic, multi-tenant services:

  • ACCaaS (Access-as-a-Service): Implemented by a vOLT control application running on ONOS, where each tenant corresponds to a Subscriber VLAN.
  • SUBaaS (Subscriber-as-a-Service): Implemented by a vCPE data plane function scaled across a set of containers, where each tenant corresponds to a Subscriber Bundle.
  • INTaaS (Internet-as-a-Service): Implemented by a vBNG control application running on ONOS, where each tenant corresponds to a Routable Subnet.

If a CDN (Content Distribution Network) - itself a scalable cloud service deployed throughout the operator’s network, including caches in the CO-is added to these three new services, we have an example of the three kinds of services outlined in the Introduction: a cloud service (CDN), a data plane service (Subscriber-as-a-Service), and two control plane services (Access-as-a-Service, Internet-as-a-Service). This results in the legacy CO depicted in figure 2 being re-architected into the datacenter version shown in figure 4.

 

Figura 4 - Four scalable services running in a CO, with a bare-metal switch located on the customer premises

In summary, CORD is a significant milestone in bringing cost effectiveness and agility to Telco Central Offices. The plan is to build a fully functional reference implementation, followed by validating the architecture through lab trials and eventually field trials, and hardening the system along the way based on trial data. Taken as a whole, the goal is to bring CORD close to readiness for commercial deployments in operator networks. Moving towards 5G era networks, many extensions of this work need to be completed, some of those require a Wireless network OS that operate in realtime as opposed to ONOS above. In the OMEC proposal below, SoftRAN OS [nota 1] is used for this purpose, in addition, new reconstruction of functions traditionally residing in core networks must be pushed towards the “edge”, mobility has to be completely redefined as a service that can be attached in a more distributed way to new services etc.

 

Towards the 5G era

5G era is a new vision that is shaping up in the industry and academia and is an E2E vision. It includes an eRAN (evolved RAN), a ubiquitous and responsive mobile-broadband network that will not only carry the (future) Internet traffic but also enable new applications and services, a NG core, and a  management/control plane that extends UE to Core and beyond.  It is not just an air interface where people sometimes equate 5G to. It includes both evolutionary components of current generations of mobile networks (under a unifying umbrella), and revolutionary components that will enable efficiently (energy and spectral), a new resilient framework (i.e., responsive, auto-manageable QoS/QoE, secure, survivable, traffic and disruption tolerant) services to everyone and everything (applications and machines). Many new drivers, “Immersive Experience”, “Tactile Internet” enriched by “Context Information”, and “X as a Service”, will be adopted and devices such as significant large amount of IoTs be introduced beyond the current “client-server” models, in which the network infrastructure is reduced to  simplistic pipes.
This vision requires a complete revamping of the E2E architecture, services and service capabilities of the new infrastructures, and a rethinking of interfaces, management and control frameworks, access and non-access protocols and related procedures, functions, and advanced algorithms (e.g., AAA, auto- maintenance and management of services and any type resources (both physical and virtual).
Several challenges still are in the process of being addressed to meet stringent performance targets as set out by the 5G community (x1000 higher mobile data volume per area, x10 to x100 higher typical user data rate, x10 to x100 higher number of connected devices, x10 longer battery life for low power devices, x5 times reduced End-to-End (E2E) latency), coverage (seamless experience), E2E QoS, etc [nota 2]. Moreover, the infrastructure needs to be largely flexible and scalable thus meeting foreseen and unknown requirements, and resiliency and responsiveness is built into the design. The complexity is a big issue that needs to be measured and evaluated as part of this comprehensive redesign.
Service Providers and Network Operators currently are deploying transformative approaches to provide network functions in appropriate infrastructures (utilizing both centralized and distributed flexible architectural concepts) and thus providing flexible and scalable capabilities according to required use cases and their traffic demands. This flexibility will be achieved using a software defined ecosystem and NFV technologies as well as data path programmability; target architecture has to be cost efficient and resource efficient as well as auto-managed and flexible for new innovations.
A large adoption of cloud computing, software defined networks and network functions virtualization require new thinking in various key areas to be able to fully utilize and monetize the capabilities presented: e.g., distributed system architecture, provide minimum “state” information, elastic and scalable systems in a consistent way, loosely coupling and necessary event handling etc. Auto-management and control of mobile networks using new and innovative paradigms would be crucial.
In particular, many aspects, such as context management (e.g., related to service, network, and device information), a  Control, Orchestration, Management and Policy (COMP) [nota 3] details related to eRAN, spectrum management, E2E resilient service composition, mobility management, low power-long range & various MTC, D2D services, RRM, and modular radio interfaces and a new protocol independent layering require new and innovative work. Similarly, a new set of devices would also have modular capabilities developed around context, interference and RAT management, since end devices would be an integral part of eRAN.
Most of the research and innovation efforts need to be in place in the next few years so that with large field trials and testing can take place for early deployments to take place in 2020.  This can be realized only through a global collaboration and investments in key technologies and related fields. Since the required set of capabilities is very broad, establishing ecosystems (both mobile and wireline) that will allow global participation through open frameworks are needed.

 

Open Mobile Edge Cloud (OMEC) Vision

As mentioned earlier, various research and standardization bodies have identified that 5G systems should achieve system capacity growth by 1000 fold, energy efficiency gains in the order of 10.
Various efforts are ongoing in the RAN and Core areas to address the architectural principles outlined above. In the RAN space, one of the promising architectures towards that goal was identified as CRAN (various flavors) as it provides a transition path to the cloud computing based architecture. CRAN architectures have been in trials in various countries and research labs for the past few years to determine the major benefits, challenges and solutions to those. The major challenges are fronthaul requirements (e.g., delay, jitter, cost, technology) and the ability of centralized BBUs to provide adequate signal processing in performance targets.  These basically determine the spectral and energy efficiencies targeted.
Several variants of CRAN is proposed to address the fronthaul restrictions, one of the promising architectural direction is to decouple user and control planes and progress using the SDN strategy. This also allows a major rethinking of the mobility edge (and subsequently the converged wireline/wireless edge). In this framework, basically a deconstruction of basic functions of RAN and core networks are proposed, as a next step using the deconstructed functions new architectural elements are proposed. One key area of this exercise is the introduction of a new functional node as an intersection point of these functions to be able to create a future proof architecture. This functional node, basically envisioned an “Open” node is called, Open Mobile Edge Cloud node will be deployed to provide seamless coverage and execute various control plane functions as well as some of the “core functions” currently placed in various nodes of EPC. More functionalities related to Compute and Storage will be added to enable truly cloud capabilities in iser proximities. Since many location based applications are on the rise, social, analytics, video etc. fronthaul load will be considerably higher in the future and requirements on local storage, compute and networking processing of “edge” services almost forces a new architectural direction.
ETSI/MEC [nota 4] and “Fog computing [nota 5]” are efforts in this direction; trying to address similar issues and identified that a substantial amount of storage, communication, control, configuration, measurement and management should be placed at the “edge” of a network, in addition to the current cloud paradigms. This idea is based on the premise to certain extensions of cloud computing architectures to the network edge. Figure 5 is a functional decomposition of NG UE, RAN and core functions for an E2E architecture of mobile networks in the 5G era. All these are related approaches but much more needs to be done, and a proposed framework will be provided in summary below.

 

Figura 5 - The functional decomposition of NG UE, RAN, core and E2E functions

The deconstruction of functions is a prelude to reconstruction and optimal placement of functionalities much more than networking, to refactored nodes to address all the considerations outlined above. It is envisioned that NG Mobile edge (subsequently converged edge) will be the center of all 5G era networks with compute and storage functionalities attached, as will be described below briefly. OMEC is an architectural paradigm based on this framework.
There are multiple variants of this key idea that essentially pushes some applications, analytics and computing, content and storage to the edge (including the edge devices); what is being done in the networking space, should converge with similar ideas on Compute, Storage dimensions. To include the UE’s and other CPEs part of this methodology require many new collaboration capabilities to be developed to execute the traditional RAN functions (e.g., RRM) as well as some others such as mobility management, security etc. in this architectural framework as well as content delivery, storage and compute functionalities.
Several edge initiatives are already focusing on different aspects of this framework in the 5G related work; i.e Tactile Internet, fog networks, ETSI/MEC etc. The purpose of this note is to propose a collaboration to bring all these activities under a unifying umbrella.

 

“Edge” definition and how does it fit to 5G era networks

Given the vision outlined above, and the strategic role it would play as an intersection of NG Core and RAN functions, the edge needs to be properly defined. In the IEEE workshop that took place in November 2015 [nota 6] on OMEC (Open Mobile Edge Cloud), it was defined as:
An open cloud platform that uses some end-user clients and located at the “mobile edge” to carry out a substantial amount of storage (rather than stored primarily in cloud data centers) and computation (including edge analytics, rather than relying on cloud data centers) in real time, communication (rather than routed over backbone networks), and control, policy and management (rather than controlled primarily by network gateways such as those in the LTE core).

Note that this definition substantially re-architects the whole network.  Key components are an application delivery framework on a cloud based system with key functionalities refactored from NG RAN and Core.
The broad set of use cases outlined in various research and standards bodies points to new set of applications that are limited by human Physiology and Psychology [nota 7].
What differentiates the 5G era networks is the ability to address varying degrees of requirements (in delay, throughput, types and quantities of devices, etc.) concurrently with a unified framework. This almost dictates a new architectural component that is in close proximity to end users/devices with at most 10km distance to provide the new control and steering applications brought by new use cases.
This requirement are already known very well within the industry and there are these approaches within the 5G community to address those.
All Architectural work should be based on a set of NFR (Nonfunctional Requirements) on Technology, Business, and Quality. These are the high level requirements and constraints that determine the evolution direction.  Then the architecture  becomes the functional implementation of these requirements based on these constraints. The strategic NFR might be:

  • Technology: e.g., use this vs that hypervisor
  • Business: e.g., create ROI on almost everything, create monetization at the edge, future proof everything etc.
  • Quality: e.g., provide Resiliency as a key architectural principle, Look at QoE in addition to QoS, do complexity/robustness analysis on services and infrastructure, etc.

Key questions that needs to be answered for the architecture of OMEC is to determine how to merge various activities at the edge on a common platform based on the NFR constraints.

 

Compute, Storage and Communication needs in a holistic way

Storage: Cloud-based services has originated with Storage and Compute to leverage economies of scale of these common tasks on common (inexpensive) components, and provide high level functions such as resilience through architectural means (SDN/NFV, etc.).  However, to address the use cases covered under initiatives such as fog networks, where there are lots of “under-utilized storage” resources in the edge devices, one should create pooling mechanisms and control and management of those. Depending on use case, “ad-hoc” storage networks consisting of enterprise workstations, home servers, local CDN servers, TV set-top boxes, etc. can be pooled to create user slices. Resilience (security, reliability, performance etc. ) can be provided by the OMEC infrastructure using variety of methods (Reed-Solomon coding to provide additional reliability for example.
Compute: Computing at the edge with user devices included are already playing important roles. Edge analytics is a rapidly growing field to process and make use of local traffic in ms latencies. There is also a broad set of new requirements coming from 5G use cases such as Tactile Internet, realtime video traffic optimization etc. Various industry initiatives are creating environments where Application programs such as Analytics are easily pluggable into the OMEC. Furthermore, additional resources obtained from edge networks (e.g., fog networks) that are time-critical and client-centric tasks should be placed in close proximity to the users. For example, congestion (both data and actual traffic) are easily detectable if the intelligence to process the data is built into the system. Similar to storage, tasks that require additional computation can be moved back and forth on OMEC platforms to increase efficiency in distributed computation and also add more compute power to intensive requirements. There is a project called SETI within UC Berkeley  that allows millions of users to participate massive computation needed. Think of OMEC as a smaller and realtime SETI.
Communication Cellular, Wi-Fi, wireline, ad-hoc networks use a great many devices and edge components as communications infrastructure. 5G era will include a communication infrastructure quite different than what LTE networks currently provide. The cellular networks provide a data plane that extends all the way to the mobile core, it is totally centralized. The control plane is a distributed infrastructure consisting of MMEs, S=gateways, P-gateways, etc. a very distributed system. What we have learned in the study of SDN/NFV frameworks that is being eployed in effect is exact opposite: the 5G era will have a quite centralized control plane and a distributed data plane. The desegregated network functions view outlined in Figure 1 gives us a new opportunity to implement this vision. The necessary additional control, configuration and establishment of reachability through the underlying OMEC mechanisms will help provide a new no-cell approach (or create your cell as needed) for all users (e.g.,Wi-Fi , sensory, and ad-hoc networks) currently relying different approaches. Fog networking architecture will be part of the overall communication infrastructure through APIs for new functionalities including network measurement, control, configuration and service definition. Since the  future edge devices are much more powerful and diverse, the advanced functionalities can be easily implemented in the edge devices with the benefits of being closer to the network edge. However, some of the networking tasks that require global connectivity and visibility of the network should be still carried out in the OMEC. Some additional helper nodes, such as NG access points will be part of the overall vision.
5G era networks would be reconstructed from the deconstructed functions that are shown in figure 1. A possible template to address this with NFRs is shown in figure 3. Modular functions are used in defining and programming of data plane functions as data plane is envisioned a concatenation of connectivity substrates  and processing units depending on the location and purpose. Control plane similarly requires a set of modular functions that can be used to program the underlying data planes.  The modularity is essential to compose (e.g., service chaining ) and create an instance of a network where functional requirements are based on service requirements, user subscriptions, device type etc. This will also prevent vendor lock-ins and allow flexibility for future proofing the network.
In Cloud based systems compute, storage and networking components are key to deploying a system that could address the applications we could deploy. There are key questions of how to mix the C, S and N components to deliver a set of applications. This is similar to determination of “eigenvalues”, that is, scaling factors, and “eigenvectors”, that is, fundamental composition of C, S and N. Initially cloud based systems were deployed to have compute and storage components only, however bringing a broad set of networking applications and combining them with compute and storage is the right approach in creating systems that will benefit most of future applications envisioned. Many questions should be addressed quickly through sets of Proof of Concept studies to answer some of these quickly and through global collaborations. There are a great many organizations, groups, research bodies that are addressing the “cloud” based service architectures: IEEE (Open Mobile Edge Cloud), ETSI MEC, IETF, different protocols to address the networking issues, 3GPP on radio interface and control plane/user plane separation, local traffic issues, ITU-T FG on IMT 2020 on fundamental gaps etc. Now is a good time to bring together some of these ideas and put the weight of all industry to specific key areas to make meaningful progres.

 

NOTE

  1. See for example: http://platformlab.stanford.edu/Presentations/scalable-control-plane.pdf
  2. See METIS documents for a comprehensive set of requirements, architectural options: https://www.metis2020.com/documents/publications/
  3. COMP: many SP’s have example implementations of Control, Orchestration, Management & Policy frameworks that are part of a larger ecosystem that specifies standardized abstractions and interfaces that enable efficient interoperation of the ecosystem components. They are collections of software components which collectively are responsible for the efficient control, operation and management of capabilities and functions.
  4. http://sdn.ieee.org/pre-industrial
  5. See various work on Tactile Internet, e.g., http://eandt.theiet.org/magazine/2015/03/tactile-internet-5g.cfm
  6. http://sdn.ieee.org/pre-industrial
  7. See various work on Tactile Internet, e.g., http://eandt.theiet.org/magazine/2015/03/tactile-internet-5g.cfm
 

comments powered by Disqus