In principle there is no difference between testing a physical network function or the same function virtualized in a server. If the virtual machine in the server is doing the same job, then you can simply run the same test on it.
Figure 1. Click image to enlarge.
Figure 1 illustrates this with a physical test system used to validate virtual functions in a standard server. In this example PPPoE clients, BGP routers and MPLS tunnels are emulated in the test systems, and connected to peer sessions in the server using control plane protocols, at the same time as user data-plane streams are emulated for originating and terminating traffic. Data plane measurements could include: latency on (tens of thousands of data streams); throughput and forwarding rate; packet delay variation, dropped and corrupted frames. On the control plane, states and state transitions for hundreds of control plane protocols, error notifications, and the scaling of multiple simultaneous protocols are measured.
Figure 2. Click image to enlarge.
In Figure 2, however, we see a service chain that comprises a virtual load balancer, virtual firewall and virtual CE router, being tested using virtual test appliances to emulate realistic user behaviour and originate and terminate stateful HTTP, FTP and video traffic. Metrics being measured can include: sustained data transfer rates; number and speed of connection and transactions per second; round trip time; denial of service handling; packet loss and leakage across service chains and time between VM instantiation and first available packet.
What are the pros and cons of these two approaches – physical or virtual?
Physical testing is more expensive and labour intensive. Considering the importance of cost savings as a driver for NFV, there is a similar need to reduce the cost of testing. Virtual test appliances are an attractive solution, even allowing the creation of an entire test lab in a single server.
There is however the possibility that the virtual test software, by consuming some of the server’s processing resources, will have an effect on the measured performance of the NFV in the server. While this might be compensated for under laboratory test conditions, physical test appliances are still recommended for virtual environments demanding the highest levels of data-plane performance (line-rate) or microsecond-level timing accuracy.
In most other cases, however, virtual test systems are every bit as good as physical appliances. Virtual test appliances are ideal for organizations that need the flexibility of testing network functions on multiple, geographically disparate servers. They are also the right choice for organizations that need to test individual network functions within a service chain, or organizations that are on a limited budget.