QU-RPL: Queue Utilization based RPL for Load Balancing in Large Scale Industrial Applications June 24. 2015 Hyung-Sin Kim, Jeongyeup Paek, and Saewoong Bahk School of Electrical and Computer Engineering, Seoul National University School of Computer Information Communication Engineering, Hongik University
SEOUL NATIONAL UNIVERSITY Ubiquitous Network Laboratory
Introduction Low power and lossy network (LLN): Low power/cost/rate Challenges in LLNs: Link dynamics, overhead, and complexity Solution: IETF RPL (Routing Protocol for LLN) [RFC 6550] New challenge: Large scale industrial application (Smart Grid) Ex) Cisco’s CG-Mesh network (5,000 nodes/network) High rate traffic near the root (10mins/packet 0.12sec/packet)
Big Question RPL and high rate traffic… Are they happy together? 2
SEOUL NATIONAL UNIVERSITY Ubiquitous Network Laboratory
RPL operation (default RPL on TinyOS) Key metrics 𝑅𝑅𝐴𝐴𝐴𝐴𝐴𝐴 𝑘𝑘 = 𝐻𝐻𝐻𝐻𝐻𝐻 𝑘𝑘 + 1, propagated via DIO message broadcast 𝐸𝐸𝐸𝐸𝐸𝐸 𝑘𝑘, 𝑝𝑝𝑘𝑘 =
# 𝑜𝑜𝑜𝑜 𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡 𝑡𝑡𝑡𝑡.(𝑘𝑘→𝑝𝑝𝑘𝑘 ) # 𝑜𝑜𝑜𝑜 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 𝑡𝑡𝑡𝑡.(𝑘𝑘→𝑝𝑝𝑘𝑘 )
, measured by child node 𝑘𝑘
Parent selection mechanism
Parent candidate condition: 𝑅𝑅𝐴𝐴𝐴𝐴𝐴𝐴 𝑝𝑝𝑘𝑘 < 𝑅𝑅𝐴𝐴𝐴𝐴𝐴𝐴 𝑘𝑘 and 𝐸𝐸𝐸𝐸𝐸𝐸 𝑘𝑘, 𝑝𝑝𝑘𝑘 < 𝛿𝛿 Routing metric: 𝑅𝑅 𝑝𝑝𝑘𝑘 = 𝑅𝑅𝐴𝐴𝐴𝐴𝐴𝐴 𝑝𝑝𝑘𝑘 + 𝐸𝐸𝐸𝐸𝐸𝐸 𝑘𝑘, 𝑝𝑝𝑘𝑘 Best parent candidate: smallest 𝑅𝑅 𝑝𝑝𝑘𝑘 Parent change condition: significantly smaller 𝑅𝑅 𝑝𝑝𝑘𝑘 than current parent
DIO broadcast period – Trickle Timer Low overhead: Double the period after every DIO transmission Fast route recovery: Reset the period to the minimum value when inconsistency is detected 3
SEOUL NATIONAL UNIVERSITY Ubiquitous Network Laboratory
Problems and challenges – Environment 30-node indoor testbed in office building PHY: IEEE 802.15.4, MAC: default CSMA, Routing: RPL, UDP/IP: BLIP Routing topology: 6 hop with RPL Traffic load: 30~75 ppm/node (1800~4500 ppm/network) [Routing topology snapshot] 14
4
6
11
3
13 12
15
9 5 10 8
7
27
29
2
1
17
16
18
19
20
21 22
23 24
25
7m
0
28 26
LBR RPL node
30
4
SEOUL NATIONAL UNIVERSITY Ubiquitous Network Laboratory
Problems and challenges
Question 1 Can an RPL-based network deliver high rate traffic? - Does packet loss increase gracefully? -
5
SEOUL NATIONAL UNIVERSITY Ubiquitous Network Laboratory
Problems and challenges – Packet delivery Worse than anticipated based on bandwidth PRR degradation mainly due to queue loss Queue loss at only few nodes very severely [Arrival rate vs. PRR]
No!
[Arrival rate vs. Loss rate] 80
100
Left: Link loss 80
60
Loss ratio [%]
PRR [%]
[Answer]
60 40
Outliers
Right: Queue loss
40
20 20 0
0 30
60 45 Arrival rate [ppm/node]
75
30
6
60 45 Arrival rate [ppm/node]
75
SEOUL NATIONAL UNIVERSITY Ubiquitous Network Laboratory
Problems and challenges
Question 2 PRR degradation comes from unbalanced queue loss. Is it inevitable?
7
SEOUL NATIONAL UNIVERSITY Ubiquitous Network Laboratory
Problems and challenges – Queue loss ratio Unbalanced queue loss among nodes with the same hop Most congested node with hop distance 2 Queue loss comes from inefficient parent selection of RPL There is room for improvement!! [Hop distance vs. Queue loss]
Queue loss ratio [%]
50
2 hop!
40 30 20
30 ppm/node 60 ppm/node
[Answer] No!
Large gap!
10 0 1
2
3 4 Hop distance
8
5
6
SEOUL NATIONAL UNIVERSITY Ubiquitous Network Laboratory
Problems and challenges
Question 3 Does ETX reflect the congestion?
𝐸𝐸𝐸𝐸𝐸𝐸 𝑘𝑘, 𝑝𝑝𝑘𝑘
# 𝑜𝑜𝑜𝑜 𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡 𝑡𝑡𝑡𝑡. (𝑘𝑘 → 𝑝𝑝𝑘𝑘 ) = # 𝑜𝑜𝑜𝑜 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 𝑡𝑡𝑡𝑡. (𝑘𝑘 → 𝑝𝑝𝑘𝑘 ) 9
SEOUL NATIONAL UNIVERSITY Ubiquitous Network Laboratory
Problems and challenges - ETX ETX does not reflect congestion Small queue size of a low cost device Queue capacity ≪ Link capacity
[Answer]
[Arrival rate vs. Link layer ETX]
No!
3
Link ETX
2.5 2 1.5 1 0.5 0
30
45 60 Arrival rate [ppm/node]
75
We need to redesign RPL for load balancing! 10
SEOUL NATIONAL UNIVERSITY Ubiquitous Network Laboratory
QU-RPL design – Basic structure Queue utilization ratio (QU) 𝑄𝑄 𝑘𝑘 =
# 𝑜𝑜𝑜𝑜 𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 𝑖𝑖𝑖𝑖 𝑞𝑞𝑞𝑞𝑞𝑞𝑞𝑞𝑞𝑞 𝑜𝑜𝑜𝑜 𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛 𝑘𝑘 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 𝑞𝑞𝑞𝑞𝑞𝑞𝑞𝑞𝑞𝑞 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 𝑜𝑜𝑜𝑜 𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛 𝑘𝑘
Propagation of QU – New RANK 𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑄𝑄𝑄𝑄 𝑘𝑘 = 𝛽𝛽 ℎ 𝑘𝑘 + 1 + 𝛽𝛽 − 1 𝑸𝑸(𝒌𝒌)
Each node can extract both ℎ 𝑘𝑘 and 𝑄𝑄 𝑘𝑘 from 𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑄𝑄𝑄𝑄 𝑘𝑘
Fast propagation of QU – Trickle Timer
Trickle Timer re-initialization when a node experiences consecutive losses
Routing metric 𝑅𝑅𝑄𝑄𝑄𝑄 𝑝𝑝𝑘𝑘 = ℎ 𝑝𝑝𝑘𝑘 + 1 + 𝐸𝐸𝐸𝐸𝐸𝐸 𝑘𝑘, 𝑝𝑝𝑘𝑘 + 𝛼𝛼𝑸𝑸(𝒑𝒑𝒌𝒌 ) Original RANK
Original 𝑅𝑅(𝑝𝑝𝑘𝑘 )
11
SEOUL NATIONAL UNIVERSITY Ubiquitous Network Laboratory
QU-RPL design – Additional challenge
Problem: Herding effect!
Congestion!
Vicious Cycle
12
Congestion!
SEOUL NATIONAL UNIVERSITY Ubiquitous Network Laboratory
QU-RPL design – Probabilistic parent change Congestion indicator µ𝑘𝑘
1) When µ𝑘𝑘 > γ (congested environments)
Parent changes with a probability of 𝑚𝑚𝑚𝑚𝑚𝑚{𝜿𝜿(𝑄𝑄 𝑃𝑃𝑘𝑘 − 𝑄𝑄(𝑃𝑃�𝑘𝑘 )), 0}
2) Otherwise (non-congested environments), Fall back to default RPL
Congestion!
Load balance
13
SEOUL NATIONAL UNIVERSITY Ubiquitous Network Laboratory
Performance evaluation – Routing topology 2
1
[RPL]
3
[QU-RPL]
1
9
5
7
8
3
5
4
6
2
4
6
8
10
19
7
9 18
10 13
14
11
12
15
16
25 11
17 19 20 22 23
21
24
20
26
27
29
12
14
13
15
21
17
16
23
18
25
22
24
26
28
28
Balanced! 8 vs. 11 nodes
30
Congestion! 2 vs. 17 nodes 14
27
29
30
SEOUL NATIONAL UNIVERSITY Ubiquitous Network Laboratory
Performance evaluation – Packet delivery Balanced and reduced queue loss ratio Significant PRR improvement [Arrival rate vs. Queue loss ratio]
[Arrival rate vs. PRR] 100
RPL (worst node) QU-RPL (worst node) RPL (average) QU-RPL (average)
60
40
78% reduction!
20
0
80
PRR [%]
Queue loss ratio [%]
80
60 40 20
30
60 45 Arrival rate [ppm/node]
0
75
15
145% improvement!
Left: RPL Right: QU-RPL 30
60 45 Arrival rate [ppm/node]
75
SEOUL NATIONAL UNIVERSITY Ubiquitous Network Laboratory
Performance evaluation – Route quality Similar hop distance Similar link ETX [Arrival rate vs. Hop distance]
[Arrival rate vs. Link layer ETX] 3
RPL (farthest node) QU-RPL (farthest node) RPL (average) QU-RPL (average)
8
Left: RPL 2.5
Link ETX
Hop distance from LBR
10
6 4
2 1.5 1
2 0
Right: QU-RPL
0.5
30
60 45 Arrival rate [ppm/node]
0
75
16
30
60 45 Arrival rate [ppm/node]
75
SEOUL NATIONAL UNIVERSITY Ubiquitous Network Laboratory
Performance evaluation – Routing overhead More overhead by reinitializing Trickle Timer more frequently But, we get much better PRR despite more DIO transmission
Less parent changes due to more strict condition when network is balanced [Arrival rate vs. # of parent changes] Average parent changes / node / hour
[Arrival rate vs. DIO overhead] Average DIOs / node / hour
60 RPL QU-RPL
50
46.4%
40 20.8% 30
27.9%
1.2%
20 10 0
30
45 60 Arrival rate [ppm/node]
75
17
10 RPL QU-RPL
8 6
-32.7% 4 2 0
-20%
30
+12%
-43.3%
45 60 Arrival rate [ppm/node]
75
SEOUL NATIONAL UNIVERSITY Ubiquitous Network Laboratory
Performance evaluation – Effect of parameters 𝜶𝜶 ∶ Effect of QU on routing metric
𝑅𝑅 𝑝𝑝𝑘𝑘 = 𝐻𝐻𝐻𝐻𝐻𝐻 𝑝𝑝𝑘𝑘 + 1 + 𝐸𝐸𝐸𝐸𝐸𝐸 𝑘𝑘, 𝑝𝑝𝑘𝑘 + 𝜶𝜶𝑄𝑄(𝑝𝑝𝑘𝑘 )
𝜿𝜿 ∶ Aggressiveness of parent changes (avoid herding effect) 𝑃𝑃𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 𝑘𝑘 = max{𝜿𝜿(𝑄𝑄 𝑃𝑃𝑘𝑘 − 𝑄𝑄 𝑃𝑃�𝑘𝑘 ), 0}
[𝜿𝜿 vs. PRR] (𝜶𝜶 =2)
100
100
80
80
PRR [%]
PRR [%]
[𝜶𝜶 vs. PRR] (𝜿𝜿 =0.5)
60 40
40 20
20 0
60
-1
0
1
α
2
0
3
18
0
0.125 0.25
0.5
κ
0.75
1
0
SEOUL NATIONAL UNIVERSITY Ubiquitous Network Laboratory
Conclusions Large scale industrial applications require delivery of heavy traffic over RPL-based LLN We found out that default RPL does not consider load balancing We design QU-RPL, lightweight mechanism based on queue utilization ratio, for load balancing to support heavy traffic Performance evaluation on an indoor testbed shows significant improvement
19
SEOUL NATIONAL UNIVERSITY Ubiquitous Network Laboratory
Thanks! Email:
[email protected] Homepage: netlab.snu.ac.kr/~hskim
20
SEOUL NATIONAL UNIVERSITY Ubiquitous Network Laboratory
Appendix – Congestion indicator µ𝑘𝑘 First candidate: My QU ( 𝑄𝑄 𝑘𝑘 )
Can be small even when my parent suffers from frequent queue losses
Second candidate: My parent’s QU ( 𝑄𝑄 𝑃𝑃𝑘𝑘 ) Can be small after traffic load is balanced
Third candidate: Maximum QU among my parent candidates ( 𝑄𝑄𝑘𝑘,𝑚𝑚𝑚𝑚𝑚𝑚 = max 𝑄𝑄𝑘𝑘,𝑚𝑚𝑚𝑚𝑚𝑚 , 𝑚𝑚𝑚𝑚𝑚𝑚𝑝𝑝𝑘𝑘 ∈𝐏𝐏𝑘𝑘 𝑄𝑄 𝑝𝑝𝑘𝑘 ) Remain high after traffic load is balanced (memory)
𝝁𝝁𝒌𝒌 = 𝑸𝑸𝒌𝒌,𝒎𝒎𝒎𝒎𝒎𝒎 21
SEOUL NATIONAL UNIVERSITY Ubiquitous Network Laboratory