Hadoop????SQL?????????·
???????????? ???????[ 2016/12/27 10:58:48 ] ????????SQL Server ?????
?????????????SQL????????????λ?????????????????????SQL??????????????????豸??????????????????????????????????????????????????????????????????????????????
?????????????SQL???????????????????????????豸?????????????Щ?????????????????????????1000Mbps??????????????7200/10000????
?????????????????Щ????????????????????
????
?1 I/O???????????????
??????????????????豸???????????????????
????· ???(??????)???????????????????????
????· ????(??????)??????????????????????
?????????????????????????????????????????????????????CPU→Cache(L1-L2-L3)→???→SSD???→????→????
????????????????????????????????Hadoop??????SQL???????????????????
????CPU????棺??????????????????????????SQL???????????????????JOIN?????????????????;
???????磺???????Shuffle????????SQL???????????????;
????????????????????д????????????????Shuffle???
????????????е?????????????乤??????????????Hadoop?????????SQL???????????????????????????
????1?????????????(??????????)
????2???????м?????(???????紫?????????)
????3?????????????(???????紫????????????)
????4??????????????????CPU????(????CPU????濪??)
??????????????????????????????????????????????С???С?
????????????????????????????????????????????????????????????????????Ч???????潫????????????????·??????????????????
????1. ???????????
?????????????????????MySQL??Oracle???????????????????????????????????????????????????Hadoop???????(Key)??????????SQL on Hadoop?????????????????????????????????????????????
????????????????????????????????????裬SQL on Hadoop???????????????????????????????????Inceptor????????????п?????????????????????????????????????????????MinMaxFilter??BloomFilter???RowFilter????????????????????????????Щ???????????????????????????????????????й?????????????????Χ?????????????????洢????跽????????????????????????????????????????????Щ??????и?Ч????????????????????
????2. ????????????
??????????????????????????SQL???????SELECT??????С?????????ε???????????????????????????????棬??????????????????????????y????????????Ρ????????“SELECT *”?????????????????????????????ε???????
????????1?????????????????????????????????????id??name??????Σ?
????SELECT * FROM product WHERE company_id = 456723
????LIMIT 100;
???????????????????齫???д???
????SELECT id?? name FROM product
????WHERE company_id = 456723
????LIMIT 10;
??????????SELECT???????????ж??Щ????????????????EXISTS???????????????????????????
????????2??????????????????????????????EXISTS?????????????????????????????
????SELECT … FROM table_name_2 WHERE
????… EXISTS (
????SELECT * FROM table_name_1
????WHERE table_name_1.col1 = table_name_2.col1
????);
????????EXISTS???????????ж????????????????????????EXISTS??????踐????????Ρ????????EXISTS????е?“SELECT *”????“SELECT 1”??
????SELECT … FROM table_name_2 WHERE
????… EXISTS (
????SELECT 1 FROM table_name_1
????WHERE table_name_1.col1 = table_name_2.col1
????);
????3. ???????????
???????????????????????????????????????????????????????????????
????Batch DML
???????????????DML????????????????????????????Inceptor?????????????????????????????????????????????????????????????в???1000???????????????????????????Insert???????????????1000??ν??????????????????????????10000?ι?????????й??????????1000????????????????????????1000??????Σ???????????????????????1??Σ????????????????????????????????????
??????
???·???
??????????????????
2023/3/23 14:23:39???д?ò??????????
2023/3/22 16:17:39????????????????????Щ??
2022/6/14 16:14:27??????????????????????????
2021/10/18 15:37:44???????????????
2021/9/17 15:19:29???·???????·
2021/9/14 15:42:25?????????????
2021/5/28 17:25:47??????APP??????????
2021/5/8 17:01:11