Skip to content

Conversation

@eldenmoon
Copy link
Member

  • Add a block fast path for explode-like table functions to expand contiguous nested ranges without per-row get_value()
  • Fall back to the original row-wise path when the nested range is non-contiguous or unsupported
  • Add/extend unit tests to cover fast path and fallback cases

Test: ./run-be-ut.sh --run --filter='TableFunctionOperatorTest.:UnnestTest.'

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@eldenmoon eldenmoon requested a review from zclllyybb as a code owner January 29, 2026 08:09
@Thearas
Copy link
Contributor

Thearas commented Jan 29, 2026

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@eldenmoon eldenmoon force-pushed the opt-explode-fastpath-593ec84d branch 2 times, most recently from 53d4f4d to a75d128 Compare January 29, 2026 08:29
@eldenmoon eldenmoon changed the base branch from opt-explode-fastpath-593ec84d to master January 29, 2026 08:29
@eldenmoon
Copy link
Member Author

run buildall

@eldenmoon eldenmoon force-pushed the opt-explode-fastpath-593ec84d branch 2 times, most recently from 9a164c3 to ead5f70 Compare January 29, 2026 08:32
@eldenmoon
Copy link
Member Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 31816 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ead5f70a7342b5642f66822e6033d74a3003d076, data reload: false

------ Round 1 ----------------------------------
q1	17648	5484	5078	5078
q2	2143	314	193	193
q3	10136	1308	791	791
q4	10192	809	320	320
q5	7505	2121	1901	1901
q6	205	181	148	148
q7	895	742	613	613
q8	9263	1345	1141	1141
q9	5249	4771	4797	4771
q10	6792	1966	1579	1579
q11	510	317	281	281
q12	330	376	227	227
q13	17798	4063	3198	3198
q14	240	241	222	222
q15	909	822	823	822
q16	686	676	626	626
q17	643	846	431	431
q18	6822	6473	6548	6473
q19	1241	988	604	604
q20	409	345	233	233
q21	2649	2062	1891	1891
q22	354	320	273	273
Total cold run time: 102619 ms
Total hot run time: 31816 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5294	5286	5247	5247
q2	253	339	249	249
q3	2172	2675	2269	2269
q4	1347	1748	1311	1311
q5	4274	4159	4140	4140
q6	220	185	139	139
q7	2176	2156	1886	1886
q8	2624	2436	2393	2393
q9	7508	7440	7883	7440
q10	2856	3041	2610	2610
q11	582	495	466	466
q12	705	754	656	656
q13	3839	4246	3576	3576
q14	352	335	283	283
q15	897	849	833	833
q16	700	714	669	669
q17	1139	1366	1350	1350
q18	8164	8125	8075	8075
q19	910	827	913	827
q20	2131	2137	2016	2016
q21	4694	4509	4364	4364
q22	542	537	528	528
Total cold run time: 53379 ms
Total hot run time: 51327 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 28.06 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ead5f70a7342b5642f66822e6033d74a3003d076, data reload: false

query1	0.05	0.04	0.04
query2	0.11	0.05	0.04
query3	0.26	0.09	0.09
query4	1.61	0.11	0.11
query5	0.26	0.25	0.24
query6	1.17	0.68	0.66
query7	0.03	0.02	0.03
query8	0.05	0.04	0.05
query9	0.57	0.49	0.50
query10	0.54	0.53	0.56
query11	0.14	0.09	0.10
query12	0.13	0.10	0.10
query13	0.64	0.61	0.62
query14	1.06	1.02	1.07
query15	0.88	0.86	0.86
query16	0.39	0.39	0.39
query17	1.15	1.16	1.13
query18	0.22	0.21	0.20
query19	2.07	1.95	2.04
query20	0.02	0.01	0.02
query21	15.41	0.25	0.14
query22	5.02	0.05	0.05
query23	15.96	0.28	0.11
query24	1.33	0.24	0.86
query25	0.13	0.10	0.06
query26	0.14	0.13	0.14
query27	0.09	0.07	0.07
query28	4.96	1.14	0.96
query29	12.56	3.91	3.14
query30	0.28	0.13	0.11
query31	2.82	0.63	0.39
query32	3.26	0.58	0.49
query33	3.23	3.24	3.21
query34	15.98	5.38	4.70
query35	4.77	4.77	4.87
query36	0.65	0.50	0.49
query37	0.11	0.07	0.06
query38	0.07	0.05	0.04
query39	0.05	0.03	0.04
query40	0.19	0.16	0.16
query41	0.09	0.04	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 98.53 s
Total hot run time: 28.06 s

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 74.05% (137/185) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.73% (19270/36548)
Line Coverage 36.13% (179181/495939)
Region Coverage 32.59% (138982/426498)
Branch Coverage 33.51% (60132/179424)

- Add a block fast path for explode-like table functions to expand contiguous nested ranges without per-row get_value()
- Fall back to the original row-wise path when the nested range is non-contiguous or unsupported
- Add/extend unit tests to cover fast path and fallback cases

Test: ./run-be-ut.sh --run --filter='TableFunctionOperatorTest.*:UnnestTest.*'
@eldenmoon eldenmoon force-pushed the opt-explode-fastpath-593ec84d branch from ead5f70 to 933e147 Compare January 29, 2026 10:31
@eldenmoon
Copy link
Member Author

run buildall

1 similar comment
@eldenmoon
Copy link
Member Author

run buildall

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 73.54% (139/189) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.73% (19270/36548)
Line Coverage 36.13% (179187/495943)
Region Coverage 32.60% (139027/426508)
Branch Coverage 33.52% (60137/179429)

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 73.54% (139/189) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.73% (19270/36548)
Line Coverage 36.13% (179193/495943)
Region Coverage 32.59% (138984/426508)
Branch Coverage 33.52% (60143/179429)

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 82.01% (155/189) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 73.13% (26198/35826)
Line Coverage 56.20% (278021/494710)
Region Coverage 54.02% (232745/430826)
Branch Coverage 55.63% (100217/180145)

1 similar comment
@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 82.01% (155/189) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 73.13% (26198/35826)
Line Coverage 56.20% (278021/494710)
Region Coverage 54.02% (232745/430826)
Branch Coverage 55.63% (100217/180145)

@doris-robot
Copy link

TPC-H: Total hot run time: 31928 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 933e14797518d5246185e61e56b310780fcf226b, data reload: false

------ Round 1 ----------------------------------
q1	17609	5161	5030	5030
q2	2003	307	193	193
q3	10232	1290	742	742
q4	10221	881	323	323
q5	7512	2128	1960	1960
q6	203	189	154	154
q7	868	727	613	613
q8	9279	1333	1151	1151
q9	5260	4800	4847	4800
q10	6857	1938	1562	1562
q11	507	295	284	284
q12	342	380	234	234
q13	17782	4075	3238	3238
q14	250	246	226	226
q15	918	830	821	821
q16	667	678	636	636
q17	635	827	461	461
q18	6644	6524	6475	6475
q19	1258	984	595	595
q20	390	352	227	227
q21	2570	1987	1923	1923
q22	356	323	280	280
Total cold run time: 102363 ms
Total hot run time: 31928 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5281	5269	5251	5251
q2	256	364	252	252
q3	2164	2657	2270	2270
q4	1328	1721	1282	1282
q5	4269	4260	4336	4260
q6	213	182	138	138
q7	2122	2173	1876	1876
q8	2679	2534	2379	2379
q9	7426	7629	7492	7492
q10	2791	3099	2666	2666
q11	549	475	450	450
q12	693	803	700	700
q13	3885	4519	3650	3650
q14	295	329	319	319
q15	884	833	823	823
q16	703	748	693	693
q17	1151	1410	1347	1347
q18	8004	7859	7929	7859
q19	892	858	860	858
q20	2173	2060	1877	1877
q21	4478	4247	4067	4067
q22	568	571	514	514
Total cold run time: 52804 ms
Total hot run time: 51023 ms

@doris-robot
Copy link

TPC-H: Total hot run time: 32118 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 933e14797518d5246185e61e56b310780fcf226b, data reload: false

------ Round 1 ----------------------------------
q1	17613	5189	5054	5054
q2	2030	304	191	191
q3	10233	1288	748	748
q4	10201	793	304	304
q5	7547	2174	1895	1895
q6	192	178	146	146
q7	886	728	607	607
q8	9263	1374	1079	1079
q9	5274	4769	4777	4769
q10	6807	1951	1546	1546
q11	508	295	277	277
q12	337	375	221	221
q13	17826	4026	3256	3256
q14	243	243	212	212
q15	899	818	820	818
q16	672	670	612	612
q17	635	823	442	442
q18	6650	6599	7226	6599
q19	1281	1024	667	667
q20	415	359	241	241
q21	2972	2463	2144	2144
q22	364	328	290	290
Total cold run time: 102848 ms
Total hot run time: 32118 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5469	5539	5603	5539
q2	268	340	272	272
q3	2351	2803	2524	2524
q4	1555	1795	1476	1476
q5	4969	4486	4470	4470
q6	236	182	136	136
q7	2012	1927	1803	1803
q8	2588	2400	2406	2400
q9	7713	7625	7441	7441
q10	2802	2939	2420	2420
q11	518	456	445	445
q12	616	724	598	598
q13	3543	4012	3272	3272
q14	280	286	258	258
q15	834	795	803	795
q16	637	680	647	647
q17	1104	1314	1337	1314
q18	7406	7360	7393	7360
q19	805	771	762	762
q20	1958	2058	1884	1884
q21	4519	4192	4052	4052
q22	564	537	535	535
Total cold run time: 52747 ms
Total hot run time: 50403 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 28.25 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 933e14797518d5246185e61e56b310780fcf226b, data reload: false

query1	0.06	0.04	0.04
query2	0.10	0.05	0.05
query3	0.26	0.08	0.08
query4	1.60	0.12	0.11
query5	0.26	0.24	0.25
query6	1.16	0.67	0.66
query7	0.03	0.03	0.02
query8	0.05	0.04	0.04
query9	0.57	0.52	0.50
query10	0.55	0.53	0.54
query11	0.14	0.10	0.09
query12	0.14	0.10	0.10
query13	0.63	0.61	0.62
query14	1.05	1.05	1.05
query15	0.89	0.86	0.87
query16	0.39	0.42	0.39
query17	1.13	1.07	1.12
query18	0.24	0.21	0.21
query19	2.11	2.02	2.05
query20	0.02	0.02	0.01
query21	15.40	0.25	0.14
query22	5.21	0.05	0.05
query23	15.91	0.28	0.10
query24	2.22	0.29	0.57
query25	0.09	0.08	0.06
query26	0.14	0.13	0.13
query27	0.09	0.07	0.07
query28	4.55	1.14	0.96
query29	12.58	3.98	3.14
query30	0.27	0.14	0.11
query31	2.81	0.64	0.41
query32	3.25	0.60	0.49
query33	3.25	3.27	3.28
query34	16.46	5.44	4.74
query35	4.77	4.81	4.75
query36	0.64	0.50	0.49
query37	0.10	0.06	0.06
query38	0.07	0.04	0.03
query39	0.04	0.03	0.03
query40	0.18	0.16	0.15
query41	0.09	0.03	0.03
query42	0.05	0.03	0.03
query43	0.05	0.04	0.04
Total cold run time: 99.6 s
Total hot run time: 28.25 s

@doris-robot
Copy link

ClickBench: Total hot run time: 28.26 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 933e14797518d5246185e61e56b310780fcf226b, data reload: false

query1	0.06	0.05	0.05
query2	0.09	0.04	0.05
query3	0.26	0.09	0.08
query4	1.60	0.11	0.11
query5	0.28	0.25	0.26
query6	1.17	0.68	0.67
query7	0.03	0.03	0.03
query8	0.05	0.04	0.04
query9	0.58	0.50	0.50
query10	0.57	0.54	0.56
query11	0.15	0.09	0.09
query12	0.14	0.11	0.10
query13	0.64	0.62	0.62
query14	1.07	1.05	1.07
query15	0.86	0.85	0.88
query16	0.43	0.39	0.40
query17	1.14	1.14	1.11
query18	0.22	0.20	0.20
query19	2.14	1.97	2.08
query20	0.02	0.01	0.02
query21	15.39	0.27	0.14
query22	5.26	0.05	0.04
query23	15.98	0.29	0.10
query24	2.42	0.38	0.30
query25	0.08	0.05	0.06
query26	0.14	0.14	0.14
query27	0.10	0.05	0.06
query28	3.47	1.15	0.96
query29	12.56	3.83	3.14
query30	0.28	0.13	0.14
query31	2.81	0.64	0.41
query32	3.24	0.59	0.50
query33	3.17	3.21	3.31
query34	16.16	5.35	4.70
query35	4.77	4.80	4.79
query36	0.65	0.50	0.50
query37	0.12	0.07	0.07
query38	0.07	0.04	0.04
query39	0.05	0.03	0.03
query40	0.19	0.16	0.16
query41	0.10	0.03	0.03
query42	0.04	0.04	0.03
query43	0.05	0.05	0.04
Total cold run time: 98.6 s
Total hot run time: 28.26 s

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 82.01% (155/189) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.42% (25588/35826)
Line Coverage 54.05% (267394/494710)
Region Coverage 51.69% (222711/430826)
Branch Coverage 53.09% (95630/180145)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants