Cov txheej txheem:

Kuv yuav pib li cas kuv lub spark History server?
Kuv yuav pib li cas kuv lub spark History server?

Video: Kuv yuav pib li cas kuv lub spark History server?

Video: Kuv yuav pib li cas kuv lub spark History server?
Video: Jack GV - Kuv Thov Txim Ft. Vicky Yang (Official Audio) 2024, Tej zaum
Anonim

Txhawm rau pab kom Spark keeb kwm server:

  1. Tsim ib phau ntawv teev npe rau kev tshwm sim hauv DSEFS cov ntaub ntawv system: dse hadoop fs -mkdir /spark $ dse hadoop fs -mkdir /spark/events.
  2. Thaum qhib qhov kev tshwm sim, qhov kev coj cwj pwm tsis zoo yog rau txhua lub cav kom tau txais kev cawmdim, uas ua rau kev cia khoom loj tuaj raws sijhawm.

Ib sab ntawm qhov no, kuv yuav teeb tsa lub txim Keeb Kwm server li cas?

Txhawm rau pab kom Spark keeb kwm server:

  1. Tsim ib phau ntawv teev npe rau kev tshwm sim hauv DSEFS cov ntaub ntawv system: dse hadoop fs -mkdir /spark $ dse hadoop fs -mkdir /spark/events.
  2. Thaum qhib qhov kev tshwm sim, qhov kev coj cwj pwm tsis zoo yog rau txhua lub cav kom tau txais kev cawmdim, uas ua rau kev cia khoom loj hlob dhau sijhawm.

Ib yam li ntawd, kuv yuav ua li cas thiaj nrhiav tau kuv qhov chaw taws teeb keeb kwm server URL? Ej., for the keeb kwm server , lawv feem ntau yuav nkag tau ntawm server - url >:18080/api/v1, thiab rau daim ntawv thov khiav, ntawm https://localhost:4040/api/v1. Hauv API, ib daim ntawv thov raug xa mus los ntawm nws daim ntawv thov ID, [app-id].

Ib yam li ntawd, dab tsi yog spark History server?

Cov Spark keeb kwm server yog ib lub cuab yeej saib xyuas uas qhia cov ntaub ntawv hais txog ua tiav Teeb daim ntawv thov. Cov ntaub ntawv no yog rub los ntawm cov ntaub ntawv uas daim ntawv thov los ntawm lub neej ntawd sau rau hauv phau ntawv teev npe ntawm Hadoop Distributed File System (HDFS).

Sparkui yog dab tsi?

Spark Web UI ua rau koj pom cov npe ntawm cov sijhawm teem sijhawm thiab cov haujlwm, cov ntaub ntawv ib puag ncig, cov ntaub ntawv hais txog cov neeg ua haujlwm khiav thiab ntau dua. Saib qhov 30-feeb demo no kom paub txog: Cheebtsam thiab lub neej voj voog ntawm Spark program. Yuav ua li cas koj daim ntawv thov Spark khiav ntawm ib pawg Hadoop.

Pom zoo: