Skip to content

Conversation

morrySnow
Copy link
Contributor

pick from master #39352

use double to match string

  • stddev
  • stddev_samp

use largeint to match string

  • group_bit_and
  • group_bit_or
  • group_git_xor

optimize error message

  • multi_distinct_sum

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@morrySnow
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 50931 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1916952aa9a9fab18c481f7d59dd773f108241e8, data reload: false

------ Round 1 ----------------------------------
q1	18344	4489	4397	4397
q2	2110	211	195	195
q3	10154	1946	1981	1946
q4	10142	1267	1352	1267
q5	8512	4040	3958	3958
q6	277	145	145	145
q7	2109	1671	1725	1671
q8	9588	2798	2775	2775
q9	14252	10518	10399	10399
q10	8661	3604	3568	3568
q11	435	280	286	280
q12	501	337	349	337
q13	18363	4051	4067	4051
q14	385	355	353	353
q15	557	528	507	507
q16	680	612	614	612
q17	1152	1001	957	957
q18	7260	6930	7002	6930
q19	1741	1677	1629	1629
q20	586	347	341	341
q21	4397	4186	4178	4178
q22	538	435	456	435
Total cold run time: 120744 ms
Total hot run time: 50931 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4348	4401	4388	4388
q2	362	272	312	272
q3	4225	4182	4164	4164
q4	2810	2779	2778	2778
q5	7306	7293	7281	7281
q6	260	144	138	138
q7	3283	2972	2957	2957
q8	4483	4562	4577	4562
q9	17432	17267	17098	17098
q10	4311	4343	4300	4300
q11	786	742	703	703
q12	1080	884	874	874
q13	6949	3779	3755	3755
q14	477	460	455	455
q15	559	508	517	508
q16	781	711	715	711
q17	3930	3843	3853	3843
q18	8972	8813	8750	8750
q19	1827	1759	1738	1738
q20	2430	2180	2147	2147
q21	8511	8653	8514	8514
q22	1061	977	978	977
Total cold run time: 86183 ms
Total hot run time: 80913 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 211100 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1916952aa9a9fab18c481f7d59dd773f108241e8, data reload: false

query1	985	424	410	410
query2	6766	2351	2340	2340
query3	6968	231	221	221
query4	20825	18409	18391	18391
query5	19940	6775	6790	6775
query6	329	256	284	256
query7	4190	337	349	337
query8	484	452	482	452
query9	3281	2829	2772	2772
query10	504	387	384	384
query11	11575	10987	10960	10960
query12	157	116	117	116
query13	5672	750	747	747
query14	18503	13833	13785	13785
query15	386	263	252	252
query16	6496	340	308	308
query17	1571	1462	942	942
query18	2343	489	480	480
query19	252	184	187	184
query20	116	118	114	114
query21	246	156	146	146
query22	5389	5153	5087	5087
query23	33031	32278	32385	32278
query24	7032	6661	6582	6582
query25	572	493	483	483
query26	564	199	197	197
query27	1812	346	339	339
query28	6015	2473	2416	2416
query29	2998	2804	2841	2804
query30	286	210	207	207
query31	1044	850	849	849
query32	102	94	94	94
query33	489	337	351	337
query34	911	520	538	520
query35	1211	1014	1018	1014
query36	1376	1431	1294	1294
query37	118	89	92	89
query38	3140	2970	2992	2970
query39	1496	1455	1433	1433
query40	261	150	150	150
query41	157	159	153	153
query42	124	121	113	113
query43	752	624	697	624
query44	1209	792	793	792
query45	288	271	274	271
query46	1305	1027	1030	1027
query47	1961	1923	1844	1844
query48	1056	758	752	752
query49	855	588	595	588
query50	945	756	675	675
query51	4849	4732	4845	4732
query52	132	107	109	107
query53	521	383	377	377
query54	2745	2538	2547	2538
query55	102	102	115	102
query56	331	305	290	290
query57	1307	1245	1216	1216
query58	329	311	307	307
query59	3573	3526	3518	3518
query60	298	281	285	281
query61	168	165	164	164
query62	924	502	561	502
query63	544	401	405	401
query64	2753	1688	1661	1661
query65	3657	3610	3628	3610
query66	1258	824	826	824
query67	15560	16037	15142	15142
query68	9363	699	729	699
query69	652	423	418	418
query70	1681	1264	1494	1264
query71	474	361	379	361
query72	6793	3585	3618	3585
query73	784	373	366	366
query74	6475	5982	6096	5982
query75	5511	3870	3743	3743
query76	5926	1240	1256	1240
query77	1130	432	424	424
query78	13047	11780	12320	11780
query79	9987	693	699	693
query80	1315	557	580	557
query81	537	282	276	276
query82	1587	127	126	126
query83	265	218	220	218
query84	291	99	99	99
query85	987	418	426	418
query86	369	337	322	322
query87	3283	3087	3073	3073
query88	5176	2555	2564	2555
query89	532	343	343	343
query90	1981	263	262	262
query91	197	173	167	167
query92	93	87	86	86
query93	6350	594	631	594
query94	752	243	258	243
query95	1189	1114	1106	1106
query96	660	346	347	346
query97	6642	6597	6488	6488
query98	235	218	209	209
query99	3002	931	973	931
Total cold run time: 324405 ms
Total hot run time: 211100 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.92 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1916952aa9a9fab18c481f7d59dd773f108241e8, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.03	0.02
query3	0.24	0.05	0.05
query4	1.80	0.07	0.07
query5	0.54	0.52	0.53
query6	1.28	0.62	0.60
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.53	0.50	0.48
query10	0.54	0.55	0.54
query11	0.13	0.09	0.10
query12	0.13	0.10	0.10
query13	0.62	0.62	0.61
query14	0.79	0.79	0.80
query15	0.78	0.77	0.78
query16	0.37	0.38	0.37
query17	1.02	1.04	1.02
query18	0.26	0.23	0.25
query19	1.98	1.89	1.82
query20	0.02	0.01	0.01
query21	15.46	0.57	0.57
query22	2.30	2.27	1.33
query23	16.50	0.97	1.07
query24	6.94	1.55	0.69
query25	0.39	0.07	0.06
query26	0.76	0.16	0.17
query27	0.05	0.04	0.05
query28	5.91	0.77	0.82
query29	12.70	2.33	2.33
query30	0.82	0.73	0.72
query31	2.82	0.40	0.40
query32	3.36	0.52	0.51
query33	3.12	3.11	3.10
query34	15.26	4.81	4.85
query35	4.93	4.89	4.90
query36	1.09	1.03	1.04
query37	0.08	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.04
query40	0.19	0.16	0.17
query41	0.09	0.04	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 104.11 s
Total hot run time: 30.92 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 1916952aa9a9fab18c481f7d59dd773f108241e8 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       22.0 seconds inserted 10000000 Rows, about 454K ops/s

…9352)

pick from master apache#39352

use double to match string
- stddev
- stddev_samp

use largeint to match string
- group_bit_and
- group_bit_or
- group_git_xor

optimize error message
- multi_distinct_sum
@github-actions github-actions bot added the area/planner Issues or PRs related to the query planner label Aug 16, 2024
@morrySnow
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 50684 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a5de74136bed621381f4e48304411b0d98fbba03, data reload: false

------ Round 1 ----------------------------------
q1	18180	4380	4386	4380
q2	2085	205	193	193
q3	10172	1917	1957	1917
q4	10350	1253	1337	1253
q5	8732	4010	3970	3970
q6	277	145	141	141
q7	2073	1640	1639	1639
q8	9321	2834	2729	2729
q9	11060	10635	10614	10614
q10	8661	3549	3534	3534
q11	460	282	282	282
q12	496	340	337	337
q13	18365	3974	4022	3974
q14	377	362	356	356
q15	542	502	494	494
q16	723	607	603	603
q17	1137	988	992	988
q18	7343	6886	6930	6886
q19	1744	1642	1539	1539
q20	576	354	339	339
q21	4445	4195	4088	4088
q22	537	428	430	428
Total cold run time: 117656 ms
Total hot run time: 50684 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4368	4344	4349	4344
q2	365	269	262	262
q3	4167	4158	4180	4158
q4	2785	2751	2756	2751
q5	7269	7179	7221	7179
q6	256	135	137	135
q7	3306	2903	2893	2893
q8	4415	4547	4510	4510
q9	17404	17219	17147	17147
q10	4303	4286	4316	4286
q11	777	773	709	709
q12	1068	881	872	872
q13	7286	3741	3789	3741
q14	476	450	460	450
q15	554	513	509	509
q16	765	705	712	705
q17	3875	3957	3910	3910
q18	8710	8747	8921	8747
q19	1751	1761	1693	1693
q20	2420	2149	2159	2149
q21	8482	8604	8500	8500
q22	1064	960	1000	960
Total cold run time: 85866 ms
Total hot run time: 80610 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 210841 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a5de74136bed621381f4e48304411b0d98fbba03, data reload: false

query1	985	421	413	413
query2	6783	2102	2208	2102
query3	6950	227	224	224
query4	20151	18431	18270	18270
query5	19942	6728	6735	6728
query6	320	261	269	261
query7	4191	336	357	336
query8	509	460	493	460
query9	3310	2849	2772	2772
query10	522	394	416	394
query11	11484	10889	10984	10889
query12	166	117	110	110
query13	5678	740	741	740
query14	17756	13869	14178	13869
query15	396	257	252	252
query16	6528	332	303	303
query17	1567	1471	951	951
query18	2334	494	481	481
query19	250	186	186	186
query20	116	116	119	116
query21	241	150	153	150
query22	5092	4999	5034	4999
query23	32521	32255	32061	32061
query24	7203	6565	6632	6565
query25	597	498	498	498
query26	655	203	193	193
query27	1951	337	339	337
query28	6089	2449	2379	2379
query29	2930	2791	2809	2791
query30	287	215	211	211
query31	1012	861	851	851
query32	103	98	92	92
query33	490	346	345	345
query34	903	527	537	527
query35	1212	1028	947	947
query36	1537	1205	1268	1205
query37	122	92	91	91
query38	3150	2985	3053	2985
query39	1497	1442	1445	1442
query40	268	150	150	150
query41	158	160	154	154
query42	115	116	109	109
query43	753	591	631	591
query44	1239	791	791	791
query45	296	272	274	272
query46	1270	1009	1050	1009
query47	1947	1845	1839	1839
query48	1043	768	748	748
query49	841	587	587	587
query50	946	672	725	672
query51	4906	4736	4791	4736
query52	129	99	107	99
query53	524	390	383	383
query54	2757	2520	2550	2520
query55	117	103	106	103
query56	309	294	297	294
query57	1296	1243	1126	1126
query58	334	318	315	315
query59	3664	3689	3401	3401
query60	310	284	298	284
query61	163	164	158	158
query62	849	611	574	574
query63	544	401	412	401
query64	2786	1724	1591	1591
query65	3680	3642	3613	3613
query66	1248	826	836	826
query67	15814	16682	15662	15662
query68	8826	713	731	713
query69	623	423	425	423
query70	1637	1356	1447	1356
query71	469	367	372	367
query72	6619	3539	3600	3539
query73	785	374	370	370
query74	6415	5990	5972	5972
query75	5167	3805	3860	3805
query76	5127	1222	1263	1222
query77	901	428	425	425
query78	12861	12148	12894	12148
query79	8578	692	691	691
query80	1340	556	553	553
query81	541	282	281	281
query82	567	127	130	127
query83	256	226	220	220
query84	287	99	99	99
query85	979	420	421	420
query86	383	331	352	331
query87	3332	3081	3063	3063
query88	4936	2552	2550	2550
query89	453	339	358	339
query90	2022	252	262	252
query91	198	164	160	160
query92	94	91	87	87
query93	3483	609	579	579
query94	817	249	243	243
query95	1198	1138	1106	1106
query96	671	352	351	351
query97	6644	6404	6418	6404
query98	235	214	206	206
query99	3022	1056	905	905
Total cold run time: 315302 ms
Total hot run time: 210841 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.06 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a5de74136bed621381f4e48304411b0d98fbba03, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.03	0.02
query3	0.24	0.05	0.06
query4	1.77	0.08	0.11
query5	0.54	0.54	0.54
query6	1.24	0.62	0.62
query7	0.02	0.02	0.02
query8	0.05	0.03	0.03
query9	0.52	0.51	0.49
query10	0.56	0.55	0.54
query11	0.13	0.10	0.10
query12	0.13	0.11	0.10
query13	0.62	0.62	0.62
query14	0.79	0.79	0.81
query15	0.78	0.77	0.76
query16	0.37	0.39	0.37
query17	0.96	1.02	1.02
query18	0.25	0.25	0.26
query19	1.91	1.86	1.81
query20	0.01	0.01	0.02
query21	15.44	0.55	0.55
query22	2.14	1.91	1.58
query23	17.26	1.12	0.85
query24	5.91	0.82	1.86
query25	0.41	0.09	0.05
query26	0.66	0.16	0.16
query27	0.05	0.04	0.04
query28	6.56	0.78	0.74
query29	12.58	2.22	2.30
query30	0.78	0.72	0.72
query31	2.83	0.40	0.38
query32	3.34	0.51	0.51
query33	3.12	3.08	3.09
query34	15.29	4.83	4.83
query35	4.86	4.86	4.87
query36	1.07	1.04	1.05
query37	0.07	0.06	0.06
query38	0.06	0.04	0.04
query39	0.04	0.03	0.03
query40	0.19	0.17	0.17
query41	0.09	0.04	0.03
query42	0.04	0.03	0.03
query43	0.05	0.04	0.03
Total cold run time: 103.82 s
Total hot run time: 31.06 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit a5de74136bed621381f4e48304411b0d98fbba03 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      33 seconds loaded 861443392 Bytes, about 24 MB/s
Insert into select:       22.3 seconds inserted 10000000 Rows, about 448K ops/s

@morrySnow morrySnow merged commit 7fd2f96 into apache:branch-2.0 Aug 16, 2024
22 of 23 checks passed
@morrySnow morrySnow deleted the 2.0_39352 branch August 16, 2024 10:07
mongo360 pushed a commit to mongo360/doris that referenced this pull request Dec 11, 2024
…9352) (apache#39466)

pick from master apache#39352

use double to match string
- stddev
- stddev_samp

use largeint to match string
- group_bit_and
- group_bit_or
- group_git_xor

optimize error message
- multi_distinct_sum
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/nereids area/planner Issues or PRs related to the query planner kind/test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants