5G NR & LTE Latency Analysis in a Public Deployed Network
03 October 2023
Figure 1: Allocated measurement area
The widely-referenced 5G requirements triangle includes the ultra-reliable low latency (URLLC) aspect, generally associated with automotive and industrial applications.
Such applications depend on communication links, which are stable, dependable and always available - otherwise capabilities such as autonomous driving will not be possible. Another good example of where URLLC will be essential is in sending control commands to autonomous mobile robots (AMRs) or automated guided vehicles (AGVs). Not only do these activities require low latency, but that the achieved latency stays within defined thresholds too - in other words the jitter of the latency also needs to be assessed.
Table 1: Frequency bands and channels that were allocated during measurement work
3GPP, the standards specification body behind 5G New Radio (5G NR), provided performance results as part of its self-evaluation reports - including comprehensive data on latency figures. However, the corresponding data shows round-trip delays, i.e. including both the downlinks (DLs) and uplinks (ULs) of the cellular networks. Furthermore, they are based on simulations using a fixed (fairly small) packet size data transmission, generally not akin to the data rates found in commercial deployed networks. The obvious question remains whether real life networks deployed today can perform as promised.
Table 2: List of traffic load patterns used in the measurement project
Globally, 5G NR network deployments are already prevalent, mostly based on the 5G non-standalone (NSA) architecture. In private campus network deployments, there is a strong adoption of 5G stand-alone (SA) architectures. This article provides measurement results for both one-way latency (OWL) and round-trip-time (RTT) delays in a commercially-deployed, public 5G NSA network. Comparisons with latency performance of an equivalent LTE network will also be given. The results were obtained using the commercially-available interactivity test based on the QualiPoc Android solution for RTT from Rohde & Schwarz. With respect to OWL, a prototype solution was used with connected GPS resources at the transmitting and receiving end, again based on QualiPoc Android. This solution would allow the applying of different data rates - from 100kbps through to 15Mbps. As expected, a superior 5G NR performance was seen compared to LTE, although most of the improvements came in UL direction, i.e. for packets sent from the user device through the radio access and core network to the receiving entity in the server. Latency enhancements with higher data rates with generally significantly better performance were realised in the DL than in the UL. Best case 5G NR measurements revealed smaller than 7ms OWL in DL for a 100kbps data rate service. Worst case 5G NR measurements showed around 18ms OWL in UL for a 15Mbps data service.
Figure 2: Test setup for RTT and OWL measurements
Cellular network latency measurement activity
The measurement work relied on use of a Samsung S20 device, including the QualiPoc Android measurement solution with a commercial Deutsche Telekom SIM card allowing access to LTE and 5G services. Multiple locations were analysed in detail. Measurements included ones relating to RTT as well as OWL, enabling delays to be determined on the UL and DL direction separately. The measurement area showing initial 5G NR RTT results is illustrated in Figure 1.
Figure 3: Locations of 2nd measurement phase (map shows mean RTT values on medium traffic load pattern of each location)
Although initial measurements were done in locations where 5G NR network coverage was possible, LTE measurements were then taken for comparison purposes. Table 1 provides an overview of LTE and 5G frequencies identified in the coverage area.
Figure 4: Latency box plots per technology used for comparing RTT with OWL results for low traffic patterns
After the initial phases, multiple measurement runs were undertaken. Having got a basic understanding of coverage and latency performance, subsequent execution of measurements was done at the most reliable location for acquiring both LTE and 5G NR measurements. The QualiPoc Android based solution enables the sending of data streams with the traffic characteristics, as shown in Table 2.
Figure 5: Latency box plots per technology used for comparing RTT with OWL results for medium traffic patterns
The packets sent from the device were received and immediately reflected by a server using the two way active measurement protocol (TWAMP) specified in IETF RFC 5357. The TWAMP server was accessible by a public IP address. The test set-up is illustrated in Figure 2. This also describes the applied prototype solution for OWL measurements. In this case, dedicated software on the server and the device is used. Furthermore, additional GPS sources at the transmitting and receiving end provided a pulse per second (PPS) signal pulse, which was used during both the pre-synchronisation and actual measurement phases. Consequently, the OWL test duration increased compared with the interactivity test. In order to ensure a fair comparison of RTT and OWL results, stationary tests were concentrated on.
Figure 6: Latency box plots per technology used for comparing RTT with OWL results for high traffic patterns
For the results, evaluation box plot representation was employed. Box plots are a great way of illustrating data distributions and not just the central tendency (i.e. median, mean and mode), also showing how the data is spread. Furthermore, box plots helped with identification of the outliers inside the data sets.
Table 3: Summary of the latency performance improvements based on median values that the 5G network provides compared to LTE
Based on statistical theory, outliers of a normal distribution correspond to 0.7% of the whole data. In many cases, outlier values indicate a measurement error, the decision was thus taken to exclude them from the results obtained. For the LTE vs. 5G NR comparison, 2 locations were used. These provided stable LTE and 5G NR results, as shown in Figure 3.
It is worth noting that the expectation was for the sum of the OWL-UL and OWL-DL not to exceed the RTT delay, since the RTT also includes additional delays within the server (e.g. reflections, processing time and suchlike). By merging together all the NR data samples for the 2 locations (L1-NR, L2-NR) and all the LTE data samples (L1-LTE, L2-LTE), a large data set could be created in order to get an averaged overview on the latency performance between 5G NR and LTE. Significant changes depending on the day or the time that measurements were taken were not seen. These overall results are shown in Figures 4, 5 and 6 for the different data rate services respectively.
Upon completing lengthy discussions on the measurement results, several conclusions were derived. These are as follows.
1.In general latency in the UL direction was higher than for the DL direction.
2.Improvements gained from 5G in terms of OWL latency were predominantly in the UL direction.
3.In high traffic load patterns, less overall OWL improvement was seen compared to the RTT improvement.
4.More improvement on the DL direction was witnessed as the traffic load increased.
5.The RTT measurements were consistently higher compared to the sum of OWL-UL and OWL-DL.
Obviously, 5G NR network deployment is an ongoing process, which may result in improvements due to increased coverage and additional capacity installed. The latency performance is of particular interest in private 5G NR deployments, especially if a private 5G network is used in an industrial environment to support use cases like process automation, remote control of automation equipment or operation of AMRs and AGVs. Also, higher flexibility is expected in these deployments, to adapt network settings as requirements dictate. Rohde & Schwarz has deployed its own private 5G network at its Teisnach site, in Northern Bavaria. Testing work will continue at this location.