基于統(tǒng)計機(jī)器翻譯與主題傳播的聚焦爬蟲的研究與實現(xiàn).pdf

1、浙江大學(xué)研究生學(xué)位論文獨創(chuàng)性聲明本人聲明所呈交的學(xué)位論文是本人在導(dǎo)師指導(dǎo)下進(jìn)行的研究工作及取得的研究成果。除了文中特別加以標(biāo)注和致謝的地方外，論文中不包含其他人己經(jīng)發(fā)表或撰寫過的研究成果，也不包含為獲得逝姿盤堂或其他教育機(jī)構(gòu)的學(xué)位或證書而使用過的材料。與我一同工作的同志對本研究所做的任何貢獻(xiàn)均已在論文中作了明確的說明并表示謝意。學(xué)位論文作者簽名：簽字日期：年月日學(xué)位論文版權(quán)使用授權(quán)書本學(xué)位論文作者完全了解逝望盤堂有權(quán)保留并向國家有關(guān)部門

2、或機(jī)構(gòu)送交本論文的復(fù)印件和磁盤，允許論文被查閱和借閱。本人授權(quán)逝姿盤鱟可以將學(xué)位論文的全部或部分內(nèi)容編入有關(guān)數(shù)據(jù)庫進(jìn)行檢索和傳播，可以采用影印、縮印或掃描等復(fù)制手段保存、匯編學(xué)位論文。(保密的學(xué)位論文在解密后適用本授權(quán)書)學(xué)位論文作者簽名：導(dǎo)師簽名：簽字日期：年月日簽字日期：年月日浙江大學(xué)碩士學(xué)位論文AbstractAbstractWiththedevelopmentofthesearchengines，theuseoftheIntem

3、etisbecomingmoreandmoreconvenientThroughasearchengine，wecaneasilygatheralotofrelevantinformationfromtheIntemetTheIntemetcontainsmassivedata，butmodemsearchenginesonlyindexedpartoftheresourcesCrawleronepartofthesearchengin

4、e，playsavitalroleinsearchenginesHowtocrawlmorerelatedpagespeopleinterestedinwithlimitedresources，hasbecomeapopulartopicbothinindustryandacademicresearch。FocusedcrawlerhasemergedinsuchcircumstanceThispaperfocusesontherese

5、archandimplementationofafocusedcrawlerbasedontopicrelevancepredictionofanchortextandtopicpropagationinlinkstructuresAsanchortextisshortinmostcases，existedfocusedcrawlersusinganchortextintroducetheconceptoflinkcontext，whi

6、chcontainsthesurroundingtextofthegivenanchortextHowever，thelinkcontextmayintroducerelevantcontexttoanirrelevantanchortextWeregardthetopicrelevancepredictiononagivenanchortextasanencodingprocessinthenoisychannelcodingtheo

7、rem，andthusproposedapredictionmethodbasedonstatisticalmachinetranslationOnlyusethewebcontentsuchasanchortext，islikelytodiscardthepageswhicharenotrelevanttothetopicbutcontainsalargenumberofrelatedlinksTheanalysisofthewebl

8、inkstructureiSoneapproachtelievetheaboveproblem0nthebasisofsomepreviouswork，weproposedafocusedcrawlingalgorithmbasedontopicpropagationTheworkofthetopicrelevancepredictionofanchortextisalsointegratedintotheframeworkFinall

9、ywerealizedafocusedcrawlerprototypeWemakecomparisonofourproposedalgorithmwithsomeotherfocusedcrawlingalgorithmsExperimentalresultsshowthatourproposedalgorithmachievedsomeimprovementKeywords：FocusedCrawler，AnchorText，Stat