Qhov teeb meem nrog cov ntaub ntawv me me hauv Hadoop yog dab tsi?
Qhov teeb meem nrog cov ntaub ntawv me me hauv Hadoop yog dab tsi?

Video: Qhov teeb meem nrog cov ntaub ntawv me me hauv Hadoop yog dab tsi?

Video: Qhov teeb meem nrog cov ntaub ntawv me me hauv Hadoop yog dab tsi?
Video: Data Science with Python! Analyzing File Types from Avro to Stata 2024, Kaum ib hlis
Anonim

1) Cov teeb meem me me hauv HDFS : khaws cia ntau cov ntaub ntawv me me uas yog heev me me tshaj qhov block loj tsis tuaj yeem ua haujlwm zoo los ntawm HDFS . Nyeem dhau cov ntaub ntawv me me koom nrog ntau txoj kev nrhiav thiab ntau ntau ntawm hopping ntawm cov ntaub ntawv node rau cov ntaub ntawv node, uas yog inturn inefficient data processing.

Ib sab ntawm no, cov ntaub ntawv twg cuam tshuam nrog cov teeb meem me me hauv Hadoop?

1) UAS ( Hadoop Archive) Cov ntaub ntawv tau qhia rau daws cov teeb meem me me . HAR tau qhia ib txheej rau saum HDFS , uas muab interface rau ntaub ntawv nkag mus. Siv Hadoop archive command, HAR cov ntaub ntawv yog tsim, uas khiav a MapRub hauj lwm los ntim cov cov ntaub ntawv tau archived rau hauv me me tus lej ntawm HDFS cov ntaub ntawv.

Tsis tas li ntawd, kuv puas tuaj yeem muaj ntau cov ntaub ntawv hauv HDFS siv ntau qhov sib txawv? Default qhov loj ntawm thaiv yog 64 MB. koj ua tau hloov nws nyob ntawm koj qhov yuav tsum tau ua. Los rau koj cov lus nug yog koj tuaj yeem tsim ntau cov ntaub ntawv los ntawm kev sib txawv thaiv qhov ntau thiab tsawg tab sis hauv Real-Time qhov no yuav tsis txaus siab rau kev tsim khoom.

Ntxiv mus, vim li cas HDFS tsis tuav cov ntaub ntawv me me kom zoo?

Teeb meem nrog cov ntaub ntawv me me thiab HDFS Txhua ntaub ntawv , directory thiab thaiv hauv HDFS yog sawv cev raws li ib yam khoom nyob rau hauv lub namenode lub cim xeeb, txhua tus uas nyob 150 bytes, raws li txoj cai ntawm tus ntiv tes xoo. Tsis tas li ntawd, HDFS tsis yog npaj kom muaj txiaj ntsig zoo cov ntaub ntawv me me : nws yog feem ntau tsim los rau streaming kev nkag tau ntawm loj cov ntaub ntawv.

Vim li cas Hadoop qeeb?

qeeb Processing Speed Qhov no disk nrhiav siv sij hawm yog li ua rau tag nrho cov txheej txheem heev qeeb . Yog Hadoop txheej txheem cov ntaub ntawv nyob rau hauv me me ntim, nws yog heev qeeb piv. Nws yog qhov zoo tagnrho rau cov ntaub ntawv loj. Raws li Hadoop muaj batch ua cav ntawm cov tub ntxhais nws ceev rau real-time ua yog tsawg.

Pom zoo: