基于數(shù)據(jù)并行的BP神經(jīng)網(wǎng)絡訓練算法.pdf

1、華中科技大學碩士學位論文基于數(shù)據(jù)并行的BP神經(jīng)網(wǎng)絡訓練算法姓名：張弦申請學位級別：碩士專業(yè)：計算機應用技術(shù)指導教師：馬光志20080607華中科技大學碩士學位論文IIAbstractBP(BackPropagation)algithmalsoknownastheerrpropagationalgithmisawidelyusedtrainingmethodintheapplicationofneuralwksfitsfinecapabi

2、lityofnonlinearapproximation.Howeveritisknowntohavesomedefectssuchasconvergingslowlyfallinginafalselocalminimum.AlthoughsomeoptimizationalgithmsuchasRPROPhelptospeedupthelearningprocessftheneuralwkswithtremendoussizeextr

3、emelylargetrainingsetthesealgithmscouldnotsatisfythedemofimplementation.Theabilityofparallelprocessingisinherentinneuralwksoitisfeasibletoreducethelongtrainingtimewiththeparalleltechniques.Therearetwodifferentparallelimp

4、lementationschemesfBPwksthestructureparallelismthedataparallelism.Inthedataparallelismthetrainingdataisdistributedtodifferentcomputingnodeseachnodehasalocalcopyofthecompleteweighmatricesaccumulatesweightchangevaluesftheg

5、iventrainingpatternsthentheweightchangevaluesofeachnodearesummedusedtoupdatetheglobalweightmatrices.Thedataparallelismwithalargegrainsizereducesthecommunicationtime.Therefeitismostlyimplementedinthecluster.Byconnectingth

6、ePCswithaTCPIPEtherlocalareawkwebuiltupaclustersystemwithMPI(MessagePassingInterface).TheparallelBPwkisimplementedinmasterslavemode:Thetrainingdataisdistributedtoeachslavenodethemasternodegatherstheresultprocessedbythesl

7、aveupdatestheneuralwk.Wealsoproposedanoptimizedmethodwhichhelpstochoosebetterweightmatricestospeeduptheconvergence.Intheoptimizationtechniqueeachnodestartedtotrainthewholesamplesetwithdifferentinitialweightsafterseverali

8、terationsthenodewiththeminimumerrisfounditsweightmatricesarebroadcastedtoeachnodetostarttheparalleltraining.WechosethehypertensiondataprovidedbytheTongjiMedicalCollegeofHuazhongUniversityofScienceTechnologyasthetrainingd

9、ataimplementedtheparallelBPwktoevaluatetheproposedalgithm.TheexperimentshowsthattheparallelalgithmtremendouslyreducesthetrainingtimecomparedtothesequentialBPalgithmtheoptimizedmethodincreasesthespeedupparallelefficiency.