RuntimeFilter 的等待时间调整
更新时间：2025-05-29
RuntimeFilter 的等待时间调整
概述
实际生产场景会遇到因为 RuntimeFilter 等待时间不合理，引起的性能问题的情况。RuntimeFilter 是一种查询优化技术，它通过运行时生成过滤条件，从而避免了对无关数据的扫描。这种优化方式能够大幅减少 I/O 操作和计算量，进而加速查询执行。下面介绍几种常见的案例，帮助在数据倾斜场景下进行调优。
案例：RuntimeFilter 等待时间过短
参考下面 Profile 的信息：
SQL
1OLAP_SCAN_OPERATOR (id=22. nereids_id=1764. table name = test_doris(test_doris)):(ExecTime: 62.870ms)
2 - RuntimeFilters: : RuntimeFilter: (id = 6, type = minmax, need_local_merge: true, is_broadcast: false, build_bf_cardinality: false, RuntimeFilter: (id = 7, type = in_or_bloomfilter, need_local_merge: true, is_broadcast: false, build_bf_cardinality: false,
3 - PushDownPredicates: []
4 - KeyRanges: ScanKeys:ScanKey=[null(-9223372036854775808) : 9223372036854775807]
5 - TabletIds: [1732763414173, 1732763414187, 1732763414201, 1732763414215]
6 - UseSpecificThreadToken: False
7 - AcquireRuntimeFilterTime: 969ns
8 - BlocksProduced: 1.829K (1829)
9 - CloseTime: 0ns
10 - ExecTime: 62.870ms
11 - InitTime: 75.703us
12 - KeyRangesNum: 0
13 - MaxScannerThreadNum: 32
14 - MemoryUsage:
15 - PeakMemoryUsage: 0.00
16 - NumScanners: 32
17 - OpenTime: 19.276ms
18 - ProcessConjunctTime: 30.360us
19 - ProjectionTime: 0ns
20 - RowsProduced: 7.433056M (7433056)
21 - RowsRead: 0
22 - RuntimeFilterInfo:
23 - ScannerWorkerWaitTime: 0ns
24 - TabletNum: 4
25 - TotalReadThroughput: 0
26 - WaitForDependency[OLAP_SCAN_OPERATOR_DEPENDENCY]Time: 0ns
27 - WaitForRuntimeFilter: 1000ms
28 RuntimeFilter: (id = 6, type = minmax):
29 - Info: [IsPushDown = false, RuntimeFilterState = NOT_READY, HasRemoteTarget = true, HasLocalTarget = false, Ignored = false]
30 RuntimeFilter: (id = 7, type = in_or_bloomfilter):
31 - Info: [IsPushDown = false, RuntimeFilterState = NOT_READY, HasRemoteTarget = true, HasLocalTarget = false, Ignored = false]从 Profile 中可以看到：WaitForRuntimeFilter: 1000ms。这里 RuntimeFilter 等待了 1000ms，但是这个 ScanOperator 并没有等到对应的 RuntimeFilter，RuntimeFilterState = NOT_READY。
SQL
1 RuntimeFilter: (id = 6, type = minmax):
2 - Info: [IsPushDown = false, RuntimeFilterState = NOT_READY, HasRemoteTarget = true, HasLocalTarget = false, Ignored = false]
3 RuntimeFilter: (id = 7, type = in_or_bloomfilter):
4 - Info: [IsPushDown = false, RuntimeFilterState = NOT_READY, HasRemoteTarget = true, HasLocalTarget = false, Ignored = false]所以这里对应的 RuntimeFilter 的 id 6 和 7 都没有等到。通过 Profile 定位到生成 RuntimeFilter 的 Join，发现 Join 耗时
SQL
1 HASH_JOIN_OPERATOR (id=26 , nereids_id=37948):
2 - PlanInfo
3 - join op: RIGHT OUTER JOIN(PARTITIONED)[]
4 - equal join conjunct: (id = ID)
5 - runtime filters: RF006[min_max] <- ID(6418/8192/1048576), RF007[in_or_bloom] <- ID(6418/8192/1048576)
6 - cardinality=6,418
7 - vec output tuple id: 27
8 - output tuple id: 27
9 - vIntermediate tuple ids: 25
10 - hash output slot ids: 396 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 447
11 - projections: USER_ID
12 - project output tuple id: 27
13 - BlocksProduced: sum 1, avg 1, max 1, min 1
14 - CloseTime: avg 10.111us, max 10.111us, min 10.111us
15 - ExecTime: avg 364.497us, max 364.497us, min 364.497us
16 - InitTime: avg 26.653us, max 26.653us, min 26.653us
17 - MemoryUsage: sum , avg , max , min
18 - PeakMemoryUsage: sum 0.00 , avg 0.00 , max 0.00 , min 0.00
19 - ProbeKeyArena: sum 0.00 , avg 0.00 , max 0.00 , min 0.00
20 - OpenTime: avg 45.985us, max 45.985us, min 45.985us
21 - ProbeRows: sum 0, avg 0, max 0, min 0
22 - ProjectionTime: avg 211.930us, max 211.930us, min 211.930us
23 - RowsProduced: sum 1, avg 1, max 1, min 1
24 - WaitForDependency[HASH_JOIN_OPERATOR_DEPENDENCY]Time: avg 1sec780ms, max 1sec780ms, min 1sec780ms可以看到这个 Join 耗时大概是1sec780ms，所以 RuntimeFilter 在 1s 内并没有等到。于是调整 RuntimeFilter 的等待时间：
SQL
1set runtime_filter_wait_time_ms = 3000;调整之后，查询耗时从 5s 降低到 2s。
总结
RuntimeFilter 的等待时间需要根据场景定义，PALO 正在进行一些自适应的优化改造。通过 EXPLAIN 和 PROFILE 工具观察查询执行瓶颈，定位对应问题，通过 SQL HINT 修改 RuntimeFilter 等待时间，规避对应问题对性能的影响。