Qhov sib npaug hauv HDFS yog dab tsi?
Qhov sib npaug hauv HDFS yog dab tsi?

Video: Qhov sib npaug hauv HDFS yog dab tsi?

Video: Qhov sib npaug hauv HDFS yog dab tsi?
Video: Data Science with Python! Analyzing File Types from Avro to Stata 2024, Kaum ib hlis
Anonim

HDFS muab ib tug balancer utility. Qhov kev siv hluav taws xob no txheeb xyuas qhov thaiv qhov chaw thiab sib npaug cov ntaub ntawv hla DataNodes. Nws txuas ntxiv mus txuas ntxiv mus txog thaum pawg ntseeg tau suav tias yog qhov sib npaug, uas txhais tau hais tias kev siv ntawm txhua DataNode tsis sib xws.

Kuj nug, kuv yuav ua li cas rov qab hauv HDFS?

HDFS muab ib balancer ” kev siv hluav taws xob los pab sib npaug cov blocks hla DataNodes hauv pawg.

?Rebalancing HDFS

  1. Hauv Ambari Web, xauj rau Kev Pabcuam> HDFS> Summary.
  2. Nyem Kev Ua Haujlwm, thiab tom qab ntawd nyem Rebalance HDFS.
  3. Nkag mus rau Tus Nqi Them Nqi Them Nqi raws li feem pua ntawm cov peev txheej disk.
  4. Nyem Pib.

Tsis tas li ntawd, qhov ntsuas qhov load hauv Hadoop yog dab tsi? A Thauj khoom - Sib npaug Algorithm rau Hadoop Distributed File System. Cov ntaub ntawv tau muab faib ua cov blocks thiab cov blocks replicated ces muab khaws cia rau ntau DataNodes hauv ib qho kev faib tawm.

Yog li ntawd, dab tsi yog balancer hauv Hadoop?

HDFS muab ib balancer Kev siv hluav taws xob uas txheeb xyuas qhov thaiv qhov chaw thiab sib npaug cov ntaub ntawv hla DataNodes. Nws faib cov ntaub ntawv nyob rau hauv ib qho kev sib xws ntawm tag nrho cov disks ntawm ib datanode. Disk balancer yog txawv ntawm Balancer , uas txheeb xyuas cov ntaub ntawv thaiv qhov chaw thiab sib npaug cov ntaub ntawv hla cov datanodes.

Kuv yuav rho kuv tus Hdfs balancer li cas?

Nyob rau hauv tam sim no cov ntaub ntawv hadoop, nws yog "hadoop balancer [-threshold]" los pib a balancer thiab rau nres tus balancer nias ctrl-c. Tab sis nyob rau qee qhov chaw (YDN thiab cov ntaub ntawv qub qub ntawm hadoop version), hu rau "pib- balancer .sh" pib nws thiab hu rau " nres - balancer .sh" ua nres nws.

Pom zoo: