Grounded Approaches to an Abstract World: Data Analytics, Automation and Visualization Solutions for Managing Virtual Networks
A long array of new technologies–softwarization, virtualization, self-organization, automation, APIs, clouds and more–have promised to streamline network operations. The savings seem to be real and the impact on network operations, the associated tools and processes is tremendous. In a perfect world, these new technologies would allow the network to take care of itself-adapt, self-organize, self-optimize and self-heal. But we don't live in a perfect world.
Operators need to evaluate several different tools and approaches, including data analytics, automation, and visualization, to help them reap the benefits of these new technologies like SDN, while also maintaining a certain level of oversight for when issues do occur.
SDN provides a way for controllers to manage traffic flows through a network of switches, which is achieved by providing an abstract model of the network topology to network applications running on the central server. The abstract model allows a programmer to build into the software an end-to-end service that would previously have been delivered by deploying physical infrastructure. Now, instead of a physical design with cables, routers and servers, the programmer builds a virtual network out of abstracted virtual connections, virtual routers and virtual servers. This enables the fast paced creation and update of virtual network models and services over a physical network that changes at a much slower pace.
The abstract model allows a programmer to build into the software an end-to-end service that would previously have been delivered by deploying physical infrastructure
So while abstractions are helpful improving network operations, they also make it hard to understand what physical resources are really used–which link on the network introduces latency? How will you know which flows and end-users might be affected by maintenance? If a user requests additional capacity, how would you know if the network has enough spare capacity to handle the additional as well as the legacy traffic?
The first option could be to implement a data analytics solution. While the volume of data in the network and its traffic is enormous, big data applications are now emerging to help process and correlate this information. It is not only about dealing with a high volume of data, but also processing that data through Artificial Intelligence for meaningful insights on network activities.
But Artificial Intelligence requires entrusting a machine to make decisions. While machines are extremely efficient at resolving problems that are well defined, they can not effectively deal with situations that were not foreseen by the programmer.
Another emerging approach is around automation. This begins by building models of the relationship between the virtual/logical connections and the physical resources supporting them. Because the connection is complex and changing and the routing tables and policies may be dynamic, the paths that flow through the network may alter. Tools are needed to audit the flows on the network, the status of the different databases and the utilization of resources. Some of these tasks can be automated by external audit programs to make sure that databases are consistent, resource utilization is consistent with demand, and performance is at the appropriate level. The most advanced solutions can also offer automated resolution to common problems.
One of the newest approaches is the visualization of the physical and virtual resources and components of the network. This requires providing the technician with a diagram of the virtual network that is being built, the applications, the controllers, the vSwitches and vRouter. From that diagram, the technician can select a function and view the physical resources that are supporting it or vice versa.
Such functionality will enable to quickly troubleshoot a problem in the network. An example would be when facing performance degradation on a customer’s service, the technician can view the abstract network, select the resource that is experiencing difficulty (i.e. view the associated logs and alarms) and drill down to view the physical resources that support it and their associated status.
Visualization will also help the technician perform a root cause analysis. It could be used to display information about the context of a problem, its on-set, its duration, its extent and correlate that to recent events. In this way, a user can quickly narrow-down the source of a performance degradation to network aspects. Human oversight through visualization can be compatible with automation, which enhances it by ensuring that even unforeseen events can be dealt with effectively.
While abstractions, such as SDN, will help manage network complexity and improve performance, it is important to maintain some oversight. Visualisation, with automation, could offer the best all around solution by mapping a high-level abstraction into better-defined physical implementations. But, no matter what solution you use, it is vital to understand that we don't live in a perfect world and your network performance will benefit from a grounded approach to abstract environments.