ãصС˵ www.27txt.com ӭȥŶ
վԴתԻ˲κθ,֧棬ȨУ
ȨУغ24Сʱɾñ鲻빺鼮лߵ֧֣

Ǽһ
ߣ

part1

ʥҹѩ
·ߵ˷õૡܰ£ĳʯҲɡ
24Ц쵱СݮӦҪСݮһʥװ
22ӣǰѲ飬СݮڸԼӣʵô
˿йص£Ҫ¡ǵѵСݮù졣
ϽķɫԼ˿󣬡ܶԣФ÷ͬ־ǰ;
춣ˣȥСݮһۿǱ̬ʥҹѲ顣24Тҡͷһ󣬸ϽٰŮóա
ţķ۵ײȻѽƤŶһСϣټƲƤСӣǳԸаСݮţλ˾̡
СƱպá´Ŷ
˾κȥʱŸƯûײϡ
Сݮ꣡һ³ɹһ룬У
סڸСÿն185+׳ݿݡżҵǴѩɽˮһڣɻꡣƾСݮ˵ǾͲ˭֪ȽϻؼСݮְġСݮΪX䵭ƣһ仰мֱԸе˹ͷ
нҡһΣ˾޶ණˢʱǩԼ֣СݮֵĿپ֣ʱͿѳֲסˡ
ģԸеֵزĨr(s_t)q
˵ǣԸж֪ޱġվǰҲеʮǧľ룬Ǹ
лǸСݮô֪ģΪÿʮ࣬ͻһװ̣Ǹ°ӡ
ôӢǷҵгɣװϿģǵһˣУһҪ
СݮԼĹľó;ˮĨһЩаЦ˵˾Ӳģ

춣ˣѾȵòͷСݮһ顣ȻнˡŵƶͷϻŰ߰߾Өѩ
ð̡˵գ˵ģΪСݮƶ̷ŵ̨ϡ
ҲзŰСݮɨһ̨ѵ˵ѾȻˣǣҲ񰡣
Ǯպÿ̨ϵ̡ղŻΪۻԱ֪Լʲôء̣Ǯŵ̨ϣСƱҲȣתߡ
ôãˮ𣿲ѾСݮһżг¡
ʹĿˣҲͻ״ûһͷ
һСݮ޹Щӡɤӿ˼ͨϣֻóˡ꡷˨r(s_t)q
ۻԱ֡Ӧеİɣ΢΢üͷ߳ȥ

ǡ
ˣ
Ƿ𣡡
УУ㿴УܵСݮһĹ˿ʹغϵļٺӶһƲ
˿Żˣһ˵ʲôٵһ˵šղۻۻһܡ
СݮʥñȦģĵ绰ιʲôˮ
ôûã
˼ʣ
ӳһ֮ڱһ仰һˣr(s_t)q
ȥ

ҵЬײѩأ֨֨һЩþûָоˡǵֲѩ졣
һСÿ֮䶼ԶСм仹иݿ̳ƽĻ԰ҵķӾڻ԰ϡ
ſڵȴ
ϤӰĳʵϲûôʱ
ǰӰתϤƤϸˡǸϲԭǷʵȻ
Ŀģÿ춼ô𣿡
šӦһ˵
֪Ϊʲô
šصһʾճ̰ʱһӦʡ
ԶĮɹĵģάͷڷˡȻ֪ΪʲôΪʲôô
ҵָԿת͸һСȴûмſš
˵ΪʲôôǼ𣿡ĵһŭơ
̫ˡֻǹ
ֻǹȫô๤ųɶӣΪʲôҪӡء
õģֲ°ղŻһ±еңȴЩȤȱȱˣôʲô˼ζͬ
Ĺ˾ˣĹŲӣ۾ת
Ҳû˵ʲôôҪˣҿĵ۾ԽԽİˡ
ԡĥԼ񣬡ҲҵǮҵ𣿡
ҡҡͷ
ǡһϣϾͰ鲹Ⱥõġ
ҡҡͷ
Ҫô죿
ȷԼ޹صᣬҲǰˡҲˡ
ǣǺ𣿱˽ӵĻҲûոù
ܡ
ֱ˲̵̾˿㲻ӣѾˣллҿԽȥ䡣Ĵ֣΢΢һ´۾ѧʱѩӡ
ƿš

ķȡɳϣȥˡҺȵʲô𣿡
ֻơơҲװŵһֻɳϣԼҲ
ơƾͺãȣ
ûж
㬵գգۣԼȥ̨ơƣŵɳмĲ輸ϣڣ̳žƾζ
ĺһڣЩ촽ţúȡ𣿡
ҲϲѩȾơ
ضġѩ족ıһ͡漴̾Ǹٳͣ
ظ̸
ǳ˫ļǮ
˵˲СᣬҾջƵӵĻĲ˵ˡҲԼ׻ʲôأ
óְӵġǲûʲô캦飬̳ϵ£˭ܱ֤ôɾһϸҵһЩġϸǸӡ
ֻǰɰ¡
ֻҵ뷨¡ơҪԼǰãֹ
ķơȴץҵֲţԼǣ
̫ûŮ˵ҾЩջԼЩץס
ȥԼ֡ıһ͸
𣿡
ĿڵĿӣԼҲôûϡףס˵ĲӰӵŵĳŶȥ

ůصǸоȡ߳ţֻų㡣
ѩ£Ѿʮ׺䵽ĵĻִһճ
νıУ붫һξ룬˵жô䡣ǽĵһѩ
߽С԰ףһʥҹ˵ûжٿ񻶵ζ֮ҲִųСİסְģЩ¶˼ңҪôǵģҪôǵ׵ģʹмģǲҪСģĺܶԵС֡ׯ԰
԰ʱǿԲβݵأڲݵƽѩ档ʦƱ磬˶΢ʵһܶ˶֪Сȱݡ300ȣӡԲݵĵǸѩˣһ塣۾ôϳƺһһСѩһӦǸѩ˰ɡС˶䣬ҲмСëͷģرԼǼӣҪСȮһԡֻСëͷĽӡѩסˣǸªѩ˾Լӵðһ˵һ硣
ʮ䣬ͯرĹˣƽʱ˹ǹһеȤڴѧҵȻֹʱԷǽΪһֹ˿̣ȻŴӳǸѩȥ
ߵǰſѩ˫óĴ۾㽶ĴͣͨƵġ
Цѩˣ϶ǳСëͷ֮֡ҲԽЦһ£漴ɫһҲֵĶһ¡ôˣǻȥԼһ°ɡ
תʱĽӡߡһʮˡСǿװӻ׼ʱϰߣŲһͣͼˣȻûѩ΢΢һ¹⣡

Ҵԡҳͷ졣
ô˻˭ѵĻģ
ƿţһۣӦ״ȴӰûɻһתһţһŮվԼǳŸտʱԼ߽ݵġ
˭
ҽ򡣡ľֺа춯Ůӿз
㵽Ҽңʲô£
Ҳҵļˡ
ʲôҰŭЦȻԼ˵Ȩ⣿
ҵˡ
ˣҲòϸһ¡һϮɫȹɫĳͲ࣬湻ġ
Ǹѧİɣжϣڸʲôʥ
ҡҡͷˣҶȻԼһֱߡ
ҶԼ˵ŮӲûΪ˵ʱ򣬱̫޹ˣ񱻺ĵ¶СŮԹ޺ֻǺܵĶˡ
Ʈſڣ·ʵǹᣬ˵Ʈ֣ҹаʮ߿ɲͣôҲһˣѵǼʶóɵˣ
ھþװʢΪʦңڵĴЩ治״ˡ
űԵķʽק̽ͷ̽Ե˼عͷʧңôûѩ⣿
ѩ⣿ģôᱸѩ⣿㳦˾𣿡
ҲЩ
ַĵģҸѵ䣬Ǿ伫޹ġҶ
ˣҪѩ⡣
ûС
Եȥ򡣡
ڿԼˣȴָʹԼȥꣿŮӣԼݣҲ̫ˣռʱľѾˣҲװ޹ŮʧȥȤеòƣڳȥɣҪ˯ˡ
ˣҶ治Լܻӭ״
ûбҪȥѧʱԼǳʿȣҵʱܵıѴ֮󣬾ͳһѪˣûٸ̸飬ҲΪԼ԰Ů˺һԴˣóʱ飬һŮӣһǶȥקſӳȥֻɫˡϧῴɫ
ǣˡգϾͱ˾Өı
ôܣҲ䰡ʲôϷеûڿﱸһ۾Ǵģ
ղһֱڻ԰
԰Ҹմӻ԰ôû㣿
ҿˡʱһۣͿ˵ĺ󱳡
ʱԼֻһѩ˰
Ŷ˵ҾǸѩˡ
ϸ˿򣬷ЩƳˣƤϸµòʵǲţҳȥ԰￴һ
ǣˡ
ѩˣܿղŲǰĽӡŽӡ߽԰ӡľڲݵ벻ˣǸЦѩҲʧó׵ûټ

part2

ʧԸӣͲѵҵСݮǷϵſڽԵļϣ춣ˣҲû̧ͷ
ҵ˼֧ͬζѩŵ̨ϣۻԱķڼϣòιһ
ҹҹ˼˲ͨСݮζ͵̧ͷûȥңƫȥָ˸ݣϣû̫Ƿϣһİѹ˿͵˵Цתҵǰ
ֻʣһƲȴȻ֪ǿסЦ⣬ûԼǺ
Цˣ찡ѩɣСݮǷݡšЦ۲ղˣˮͦȵġ
ˡСݮ˱ڻֻٿһ¡
Сݮ˻񣬿̨ϵƷѽɿѩ⣬ţѩ⣬ݱ衣ôĨr(s_t)q
죬ҹģصѩ⣬˶ԵԱСͼı죬ʲô飿Сݮߵĵͷڷθ￪ĵԼɣʱɬ̧ۣĥ˺ðĵɨǣԷһЩꡣ
ʮ¡ʱͣס졣
Ҳ̫׸˰ɣءѩͳˡ
һôߡͷһ׷ŮӰɣǰڴССݮΪԼ˻ɣСݮ˿Ѿ׵׻أˣչ̫ĻҲԼܲŶ\(^o^)/
ڱſڵأһ˳ˣȥĽӹҵݹѩ⡣ϸָһҵѵղŽòף

Ҵˮͦк󾢵ġ
Ҿ˵𣬻Ǿ仰ˣ˵Сݮҹģ𰡣ȥϴһϴһ˿꣬ʲôåн

߻طӵʱѩѾԵһʣˡһֱߵ򣬸ѩ˵θڿǲòࡣ
԰֮ǰ
ϰ
Ҳѩͱꣿ
ûа
۱һһͷµЩԼ֪ѩ֪ʶһԵȻֻôһѡȴ˸ղŴָĸоҪǡ
Ҫ룡ԼôôޣҰ˼·ô֪ѩͱģ
ԭ˰
ֱʵԼֲȥˣҲŴԵ֪ÿ궬춼֮䣬ʥʱ˵ԼˡҲĲͬط˲ٸˣҲϰһЩȥĶСһӣΪǰ˶ûѩˣе桢
ˣҪ˯ˡҴһĹǷҲ˯ɡ
ˣ˯
ţ㲻ѩ𣬵վȥͳ𣿡
ͷһܵŰ޹۾һֻŵСޱ͸һĵˡ(>_<)
Ҳ˯ˡ
㵽ͷȥɡˣһָԱߵӣԼ¥ȥˡ
߽ңһţһת˯ɹɢˣ֪ӣԼҡ
㵽ͷȥ˯
ͷࡣ
Լɨ
(o)ŶôɨмԼɨ
ȥ򣬾͵ȥסղűɱ˫۵ڻֱܷ˵ϲŭģҵ棬߳ȥˡ
ҵķµһϡѩ̫ˡԼ״ġ

ۣ뵽Լһϵ˼άȴԽһвʵˡ
¥ĥŻ᲻ֻһΣϾͱ¥µĳסˡĵϷһı˪뵽ʱӡ
滬˪ôûڻŽŲߵͷſڣ̧ȥţûܵĿˡ
ڴϣһڱŵ˫һر˪ԴȪ

¶ȱ˪յúܵͣҲò룬ڱɳҧ˾ȸǽʳθȫ
˻ѵ
ѩǱԭ˳軵ˣ˯ͷί
СʱҲСкһʱ͸СһóԵģʱͰС߹߹ȥģһ𣬾ͺͬĵħѵСֺԼĶСʵѣȻ˵ĲˣһһؼһƨƨߵĲȥԣҶΪ˵֪ͣ϶ѡ
ˣӿͷ߳Ϳһҡ
֪ԼˣҲ뵱ˣұ˰ɡ
ˣۣͿ㣬Ҳ˱ģǴˣڻٻصȥ
Ҳ㡣ⲻ
ǣˡˣҪùġ
вԼҲùˣ߰ɡ
ˡͷ쿪Ŷȥ
ô򷢡ҹһЦݣһ˾Žһ룬ͱҭסˡ
ȥ̨ˮͻĵϡһŹŵأ۵Լ졣
ɺϰɣӦǡղûע¡ĹҰοһԼȥϰࡣ
ûſڡˡҵٹһЦݣ־Ȼе㡣ǲǰѲĳӵʱָо
סĵطжʮӵĳ̡Ҳϲÿ춼Ƴ̳ȥСſڵʮӣûסһЩſճƵļƳ̳ûԼһƮ
òףһ㣬ƺ̾Ҫ͵ƷվļƳ̳ͣ˽ȥƳ̳˾һͷһ죬Ŀ̫Сƶᱻ
ӴáüӳĿĵأͰ󿿣˵һ֡˾֪ȤƫƫѲø׷һӴģ
ɣӴö֪뻻һ뵽ղŵȳʱ״ҾһˡŴ·Աߡ
Ŷˡ
ϻ࣬ʲôȵģΪԼǷɻɣ
˾һţҾͿĵǰϨr(s_t)q
Ͳаˡ
ҧҧһʢŭ
˾Ӿ￴ҵӣһ˵ͷҲø칫Ƶģλǰɣ̽ſڣа
Ҳ˦ˡ
˾ֱģôǺã³ҲǣüǧǮNֳأſô죬û̻ǣһ᲻ϵǮֻжȥأìܡԸǽٷ˰ɣֻһ㣬ԣӦ
ҵֻ൱ɭƫƫƫҧѼӵʱѿ˾ĺëŵøӡ
ιһ糿ҵҲò
ʲôü֤˲ͥˣѾ˵õġðʦֵҲôȥſڶ¡žĺֻӵλϡ
˾СĸͱˣղŻһ뻩ֶǡȥſڶˣ˻ŵģʲô˰ŷ̵ָġ
ιһ˾ûͷ
һתʲôҷ˾ڸԼȦӡ

˾˵תΪиɳr(s_t)q
˺ð죬ű֣Ҳʶ·
˦һͷ޲ð⹷ƨ˾ȥʶ·㲻
ͣĿ˳ţߡ
˾ллذͷҸոĵط۾ֱˣҵֻ
ι˾ҡ´Ĳӣ㡪û˵꣬֨ˣͻһԼȻһųƱ
д·ʶǼǵҪǮʱ䣬Ҳƽˡִ󲽵߿
˾ּ쵼˻˼ģȴ֡ԼŵǮⲻôźãŵ˾ҵֻӵƱҲһȥվûãԼСƳһ̵ķɳ۶ȥ񿪷ɻһ

°ǰϵֲȵXX˶̫ƽ׹ĵ¥ңȵȣȥ
ǸڷϽĹ˾ˣϽȥ㱨ǿųеѻ¶Ѹղȥ˵˾˵ǲǸ𣿣
ǣƫƫԼҶ£µ𣿡

˾ģôҲˣһװ˷ӵĴһġһֻڷһܡлʹ˵
֤ȥȼˣ˵ǲȵů
ʲôһ˲ɶſ·ǰ̨򡣰ǸԱʶǿҵŮǣһȶõҵϨr(s_t)q

ֻ˼Ƴ̳ϣȴû˽ӡ
Էһû̵̫㣬ǿһһĹͷҵʳ
磬Сȥɨr(s_t)q
֮ûʲô类ײǹģ˥¼껪
ϣڻʮʱϻСΪӰ࣬ʵһʲôûϡҲûʵڲ֦ˣһ˼סߣȻⲻߵͻȻðĴʯͷƤСײȻĳʹȴֲԸ¶ĲλȵСcase

ԶԶģͿһģӰſڵĽϡԸɣһѾѩǧ飬ǺûϢ뿴ӿӰʮ׿ʼù˼ӾϵײײұüֱΡȵڽӽʮ׵ʱҾǰӣԸÿѩԣԼҲøֻԼƬͷÿͺáǣʮ֮·90ȵɽ»r(s_t)q
ȵԶĵطˡȷǸӰ̨˯֮󣬱ƷӡƿͶ󺣵ֻħˣҾԺһҪúŰñһѩˣԽͷ֮ޣѩָ֮ḻʵĳ裡

ľֻʣ°סΣպа۾̧ţƤЬûԵСͱѡ
Nû״ĴţڲظĳˣͣЩͳء
ڿƺԼĽ֮Ѿûʹ㻵ˮҲֻܵȵշ
̾һֻĿƣļƣᣬιһѡ

part3

뻢Ц滢ûˡһݵĴӡ
ôֻˣ
ûеطȥǺܳʵ
һָڡȵĲҾɡˣҪи˵
ˣǳգ˳۽ë
˭˵Ὺ˵Լֻ˲ԣˣҪô͵ȥ
ͷǣˡ
⻹񻰣ȥԡҵϴˮźãҪԡ
ǡСΪСŮͣҲûжôԹԹأ·ǸΪûк÷˯Ϳһر˲һ㡣
Ӯ˹ʹ˾ҲûüֻеҾܰȻɳ϶һĪĿοϢ˺һŴԡгˣˮѾźˡ
ŸˮҲֱŵġҸ̡ʵϰָʹ˵үƢԣ»ԼֵĺãҪѵ򣬾͸ĸϰߺȻڵ¶֭ʵļ¼һ쵹ùװ߽ԡҡ
һ룬һûκʿȵĺ쳹С¥
ԡŭ߳ԡңˮŪôͣ
ţһ޹Լоոպð
ǸѩˡԼһ׳գˮŵٽЩˮ߽ԡңҪԡˣվſڵӵĿڣˣҪϴɣˮǺܱġr(s_t)q

Ұȵ˯һȷһǰҪʮ֮ϴΪ̫ᵼ˯ȻӰ࣬ûҲҪ칤ĥ䵽Щ뾫ˮҲɾƣĵڴϺ˯ֻһݳ޵뷨ǣ죬ԼÿһϸЩ˥եˣϴ߻ңԴһͷͽ˯״ֱ̬ߵȻѣӽǣͨ泩\(^o^)/
ԡۣ˯̫ļˣ˯¶û¥ݣôܸɾһҲûС䣬˾㡣أôһССֻϸ΢ġϸ΢ġײġȴֺԲ˵ģͬһ˫ʵЬôһϸСɳĸоվǴ󷢴ȱĳͷȥ
ȵ˯ڴϣ۽ëϻһ޵ı˪ȣֱǼ˵Ҹ˯ࡣڵЬɳԼһ׳աղŻԼ᲻Űأ
С⻷СһܵĻһȦ潡ɵĻصңһţͿһַŴ򿪵ıţԴ̽ӣҾάҿһӡ
ѵɱˣơǰȥһ׼ʱڻعͷһίˣôûѩ⣿
ûп޳ûп޳ȿ޳˵ͬġκһ迴СȻ¸п⺢ôǱǸĹΣûûε˸ŰˡҪѪ꿴һҪУȨҪŮѧˣһҪֽУˮ֮󣬷Ȼ𣬱ܵģμȥΪȨȨȻЩһǰ궼ҵõί˵ûо׼˵ıڹ͵ĳȽʯҴȥˡ
šҲͺܹԵĿű̵ǽ澤ҵҵš
ۿСݮһԼվŶû뵽аĻᶼŹղŻڵʱ˵ûȵ̣᲻ᱻײ˱˹ҵʵһأԭǵԼ̵ŶࡣСݮͶŸڱѩĿӰر΢΢ĽʵĻԲǧ¶˳ԼcosplayһŮ˹
Ǳѩ⡣˿žܸɵӣô֪ͨһ¡һְϷǵԼҴСݮһҸڹԼΪʲôСݮϵĽ㶯ӳ̶ȿȲ
ҵȻûעһˣֻ˷Ǯ㳮СݮȨɬСXΪ
ǵģһӾ࣬ƫƫˮ͸ģղܡСݮӹЩոպõĳƱ߶һȫ֪ʱר۸r(s_t)q
ˣѩˣ̳ѩȥĴ̧һֱۣͳɹСӡ
ιһΰѺóԵĶǻʲôġԼǰֻСҪƣ˲һ㳦
ĳֻڳɹħƱڻСһ硣
Ͱ͵Һж˷ȵĳһ֧СĲѩݸԼ컹˵û˼ҵֻǵĵ˵Ļr(s_t)q
ĵİѩֽҺ۾ȵʱ򣬷ֻһǹĹ¹µѩˡ
ô᣿ҲŵĶ￴һкļҲûСط̨ҪǸʲôѩӦýȥ
ߵſʱ֧ѩͲӰˡ³أǵǿļһԼԳ⵰ֻǡһΨ˵Щѩô͸˼һأ

ˣˣˣ
ڴӹϰഩ·Լҷűòš
ƿţûˣͷŷǸǸСëͷƤƤһߵڽϡ
壬󻵵
Сһʮ㡣
Լһ黹ѵõĸһΪ˳ĺͰݣôˣ
ƤƤ죬Ӹ֧ѩһһ˺ð죬ڵĿڣҵѩ͵ˡ
ѩˣŶˣɲǰСһġˡȻܻûԼҪ˥¼껪ˡ
ʲôѩˣ˵Ҳͥ´һеСëͷѩ˳øͨƵģʮǧԼϣʲô취
ߣ壬㵱Ǹ󻵵⻹ʲôɻɵĨr(s_t)q
ƤƤ۳СҲġ˵֤ݵġ
СһԼ֤ˡҼƤƤİְڸѧķϵν̡ǰ֤óɡ
ң۾գҲգӱóֻͮͮĳӡѩϵģſڷֵġ
ҵȻͨˡ
ߣ壬ô㡣һ죬żһ̨ױҵġ
ǱСߵġҿʼߡ
ƤƤѳӷŻƺŵһ޴Ŀڴڳ˫ְǰᣬСĺҶʹˡ
ڷǰݺúҰɣѩԼˡε̯̯֣ƤƤˣԼɲǶ޶ĵˡ
Ҫķ䡣ƤƤģûԵĻŹϣռȻһСֶյȣЬͷϵСϻݡ
ѩҪͻˡ
пܰѲڱﰡƤƤҵĸ첲վ˿һҲϤΡ
ȥСƨˣʼ·
ƤƤ˱¹ֵ¥µĸ̽ͷ̽ԡȥģȻһ¥ϸ˴ɨبr(s_t)q
ÿҲ֪ƤƤֻСƤѾйˡ
һҪ¥ȥ
¥ûб䡣һ߿ųӣһ͸֧ĸ͡ѾָҲǱģƤƤҲЩ£ַ¥ݵľ֣ŽŲǲðȻȥʹڱԼҪˣ峤ùֽʵģԼܴ򲻹ôˣЩģغԼǼܳѩƷغⰡr(s_t)q
ϰվŴӣհ׵ϣƤƤһ壬Ůˣ
εάӵĶعͷϿһۣһ¥
ۣ壬Ǹ󻵵ôƯĽǷͬӡҺھôö֪ҪȥҰְ֡
ƤƤߴĺУѾվˣƤƤһۣͺޱʾȥɨˣҲ˼ƤƤСɬĵı顣
ƤƤ̻ȥ

Ҵӻ԰
Ҳһ䣬ӻ԰ġ
۾ɵôģĶҿգ۽ëһ˪
ĽëƯ˪һƤƤĽСάʯļ̬һ·ƮſڣǱ˫Ҹǿġ
ţˣš
Ҵ룬һʱպãһ첻ȥҲղǸСţȥϰˡ

æһ硪۵Ĺ
緹ʱ䣬Ҹд¥ײĲԷ
Ҹ磬糿绰һֱͨ
Ŷֻˡʲô飿
ܵǸ˾˼ִ绰ˣ˵Ը⸶ԭĳͣҪܾ
ô˵
дճĻˡ΢Ǳֵ˰ɣ
극ڶ¥ĵ˲ֻ綼㶨ȥȻ˵ǰͻصȥĵȥˣû̫٣򡪡ٵԼ˵ԼԶŵᣬԼܲˣ
˵ĸ֡
ǴѧͬѧʱȷŮѹϵʱһֱûҪĹǵŮ֮ռޣһֱ˵İͼʽߡ뵽˼ȴȫ˱ˡҵǰϦҹʱֱǸĸ˵һҵҵĶѾԼ˹ס⣬˳ҲԼʱңĵߣԵĳޣҪҵгɰûᣬҵ𲽣ոǰһƯİʱ˵ճɻ顣֮ҵĿͱԼĳɹĺڡȫʦ֪ܷͼǿ˼ǵò˵˵ﾹȻһǵıͷһؾǺüꡣ
ҷµ绰˫ۣ΢΢ǣӱǿкڷˡʥʱʱѾûĶĸоˣԼнֻһŮˣǸѻŮˣҲǸԼҪӰһŮˡݳӿȴȥɣֻǺʱ̫ˡԼѡıԼŬǻ
ҵİ칫⣬Ÿո㶨ֻ൱ɵȫůҲκβ˵ǧҸȻôʵЦ̫ˣǡɴҰ칫ǰǰ̨˵ġû룬ճԹзɢŮǾͶͰˡСп˵ѵҴʦr(s_t)q
һۣͿһȺ˹ļԼſڣô˵üͬ⿪ѨһɢøʲôʲôȥˡֻʣжİŲŲȻĲֻҰ칫ǰ棬ֻŲȻĲ߳ȥ

ֻںڵɫԼr(s_t)q
ҷֻǣȻ뵽ﻹһˣҲ֪СͷһûаԼǸС¥ɱѩǱеĲͨ˼ĵ绰
ι绰ͷ˽ˣιʼòͷιǴ󻵵ңλ

part4

ϣҸϵʱƤƤﲻߡ
Ҷ֪ƤƤҵǸѩ˱ģ㲻һ˶ռҲҪСˡ
̫ˣСӷŽġĿȴΪҪϽ˶޲ЬȥЬӷŵҽ¡
㶮£һЬӣ泩һЩֻһ䵽ƤƤϣͲˬԲСƨ߳ȥʵЩ΢Ľ񱡢ƤӴ־֢ȵ֮ģǸ˿ռʶǿңϲ˶Լ֣СҲûʲôøУȻ㲻࣬ͲųԴ֮˨r(s_t)q
װƮ΢öʮһ
㣬ҪؼˡҰְֹҺϵģʮһ͵ûȥȻѼӵӣҲˡƤƤŲſڣûʲôɫʱһ䣬㣬ӦС˵ģҲᱣݵģ㲻ڵŶŶŶƤƤ˵ˣûעµ̨רr(s_t)q

ȻЬıֻе˵ӡǣΪεˣҾʮбҪȷһ˵ϣҪ׷һԷƤƤĹرǾȻ˽һСҳϵľٶ
˵ƤƤ𣿡
ûšԼġͷ
Ҷ죡˵أ
ֻһС˲ˡ̧ͷӨ͸ĺڽëų˵Ĺâ
Ҳˣ仰Լˬ

ûб˥ĥýƣ˯߻ĺʵôأҪôоӣҪôоɣҪô汩ĵϷĩҪЩرİšΪһ½ˣȻҪһЩˣ뵽龹Ȼôһ˿ϸţëȸԾԼյõֻСʱѵ˺áллˡȵȶʱСҡһεײȥģǺȤ
ĳֻ˯µСִһr(s_t)q

ҡ
ǹˮվ
ǾίԱᡣ
ҴСתƣܵΣԼС˽Ҳ̫࣬һȤζҲûСһ˿ϸţëɨɨԱߵȻһȤı飬۾ֻ70%š
Ȼˣ㶼ס𣿡
϶дr(s_t)q
ʶ֣
λǰҡ
ǰˣǱŤԼ¸ʱҲǰˣһ˿ϸţëĲڴ֮һ᲻ᵹù
ᡣʱԲˡ
ޣǾͺá컹жԶҲû뷢ʣֻǲԽСֹ
Ĵ¶ʮһ
֪ͦġ
λǰǵʦ
ŶԭһŰֽͷл֪ʶ
ǰûгʦҴŻ⡣
ֻһλɫѩġ
Ǵ￪ʼϲѩġ
ǣҵĵһǿġr(s_t)q

ʳһǰûһְҵĦʦʦ๤ԱЩΪǷʳסеĺְҵôôôô۾´˵ʱҲ֪÷֪ʶȨˡ
·СһƣһݼʮһյŮԱ̧ۿ˼롣Ҳ׽һЩһĶϷǰ֮һɣʶ
š
ѧϷˣ
һƤëӦǸҽ̵İɡ
ôݣ
Լͺá
á΢΢һôˣ꼸ˣ

ѵΪûϹѧʦ֮Ĩr(s_t)q

ƵСҵ¥ʱһɨѩͣ·ߡһɫĴͻʥҹǳѩģҵΪͻȷԼ¥ǰĻѩ
򿴵ɨѩ֮һ
ŶԭǰֻСһ󹷾⸱ӣԼ棬ֱײԼСȶ֡ǿеסСֻǰȰڵھֻͷĴƹǰʱСͻŵŤͷͰ͵ĿԼΣվ޶ֻĻˣֻСצӻȥסԼë׵СԴЩġǲã·ƵĻڻܸܵСǻĲҲеˡ
󣬸¶ͷֻεĸ˵ĺߣӦǴԼԽԽԶɫ˰ɣ˺һŸ̽ͷȻ־ķѾԼɫĽǰ˨r(s_t)q
ͷηұֽŶŵñȣʱΪ벻СĶײҵĺ󱳡ǿҸѵĵطұײĿġ
֪ʡ
šŲ죬ҾͿ·߳ѩ
С滹иĶŶеıҼĵѣԣԼҪܡ
ڰŵöڵ֮ǰķֵ뿪˿ʼꡱѩĺɫԼǺִܻģԭֻСÿζŵֻʣһȻޱ

ѩͬӵӽ졣
ѾɹΪһһϵ+˾Ҫɨ䡢Ҫ򵥵ƴʽҪЭϴ»ҽʱЬϴǰԡ׷źˮ¥ȥ˯ʱѷĵƶصǡÿ춼ѩԨr(s_t)q

㣬Ǹ󻵵̫۸ˡĳհƤƤҵķſڳѩʱ߷߲˵
ţҧһСɿƤΪþˣΪѾ֧ѩףС˴ģ
۸ȻӣƤƤһɸֵƣ㣬̫ˡ
һӦûУƤƤֻ÷ѭѭգֱ⣬㵽Ҽסÿ춼ôѩԡ˵ţִƨɺǸƺŵһ޴ڴͳ֧ѩ
ƤƤοǻ˴ǮΪ׼γʢĻ¸ְһܸǮܵһλˣְֶļ̵ƯƬ־һλ硣ԣһҪɹƤƤˣǺ㣬Ǹ󻵵Ұɡ
Ҹպ°ԶԶľƤƤǸС˵ԼĻ
еġء
ߣҳϡԸпο
۾ʱ򣬾Ϳ벻ҪҲСƤƤС춼ﵽ˱ǿϣԽһ¡
۳СȮƤƤͷһθܵ˶ǿӲŵΪ涵򣬡ԣǸ󻵵ɸˣһ͵Ҽȥˣǲǣ
ûϸΪʵѾˣʲôr(s_t)q
ƤƤＱŵõοһĲԾĲԽϿϾҪͶǸƬʱҲԼʱҲرѧѧдʱҲĻ
ƤƤԽԽճһţ㣬ǲǣǲǣǲǻǸ󻵵ң
ƤƤĿȻ̧ͷȻͿһ֮ⶢ￴һҹɫ
ˣ֮ϾͱˡЬ
ҰСȻɸһ̲ͣšǲǡƤƤš̨档ƤƤǼŲӣκޣֻ̨ܱ׸߳һСԴȻ޿κεĿ󻵵ҽȥˡ

˽첻ˣ֪ķҳĺ塣ǹ˾ò˳ɣԱʶİжҵĹ
飬˽ܳΣȻ׼ٳֵһŭһҲֻñš
벻ҪҲС仰ʱѵԼãr(s_t)q
ҳԼϴˮļ϶ߵ˸˯Ŀͷ´һǲԼһҲģͷţҲģԼͷ㣬Ŷǲһ㣿
ˣϴˡ
š߳ͷƵԡҡԡʱ򣬻ĥԼҵĺ˻ʲôҪƵĵطżһ˲뵽˸ղζֻǲԾôһ£ѡ

Ҵȥѩ⡣Լְÿ϶ȥɢóԵġ
ڱſʱ̽˿ڣˣֻܲһСѩ⣿
ţ֪׬ǮһΪ˸ĸڿִŵĶŮ֪ڼԼʱοǳǳԺһ˲Ҳô10%Ũȵ͡
С˽Ҵ˺ܶˡ
ĲŲ㱻ҵĴֱ۸ק
ǰδе׳춣СݮĶΪ֮һаʱԽԽˣСݮм¼ģԭҪʮ̣ڼݴѩ⣬ʱҲǰܾʮˣǰǾŵ㣬Ű˵ʮŰԽԽסԼ˼˰ɣҲΪһ˾׳ֵˣСݮˮĶҸڱѩӰĿ󲿷ֶۼǸԲڡСݮһЦ˼ҺзǿԽԽʵҮ
̨Ϲһ֧ѩ⣬СݮҲۣ⻨ǮʵԼҵһǲΪַʽԼǮùˣɶռĹСݮͷе

Сѩ⣬ߣӱ춼ҪһЩ
˽۸˰ɣԼһò˵أ
߽˼ţ˵˿ûкתļ󣬶Ѿ׼Ҫȥ¥Ϣˣڹˡ
ţǲҪԼϴ󰡣͵㿪ء
˿ɲ԰ҽʡµѩǮҰr(s_t)q

part5

Ǯʲôְ֪ȥʣԼǸɵѩɫжôȤƵġǡǮ׸ʲôأr(s_t)q
Ѿһˣÿ϶ֻһѩ⣬ֺʵݵѩ⡣
óͦġŸԼһԱ߱߳Եÿһڶ뷨ҧȥȻľ׽
Ҫôٶ֧ѩɡ뷨ˮļţ˾Ѿߵ˼ſڡƤƤ̨׸һȫ봦Ӱ
ƤƤһϣһЩСڵʯĽرƤƤգһֻ۾һǣзƫ̱ˡ
СڸʲôԼȥϰ࣬ƤƤССѧһ꼶˵ÿڿμ䶼ҪܻһܻȥݸֱӰѱ㵱Լԣһѧǡ
뵽һ˿ϸţëΣл֦չƮʳ˿ڣǮ׼ʲô
šе˼׼
ţʲô·ŶһֱɫȹҲûûƵĨr(s_t)q
ӿͷóһŻ̵
ⲻԼͲʱʼĹ浥
ָŹ浥һ档
ҵͷɨһۣŶԭһ¿ѩӡѩͼƬǼ֮¡
ܹ𣿡
š΢΢ߵĵͷʲôģͿԼԵö໹ǳԵr(s_t)q
ȤһС뷨򻹻Ầߣ͸ǳǳӣҺΣĵɰҶſ˼룬ǾͲԾԽ

ûзɽѩӴʦŶʱ䣬ͼŮͬһ߳Էһ졣
ɲǡҸʱſھȻ˼Ц۵С˼㲻ִ֪ıӴʦϣж꣬Ŷ
ײһ˰ɣ
϶ѵӲҪӴʦӵҵСף
ҲпܸŸɵǸĳĳСŶ
ƽʱÿŮͬ¶һӣŮͬǶΪgayبr(s_t)q
Ҷ˵ٴΣgayġиѧͬѧҵͬѧͬѧ˵ҴѧʱŮѵģҶŮغãȫУģأŮģ̫˵ұ⡣
ôûŮأ
ˡֻ˵ŮĲʶſֽˣûӴʦǸǱɡ
ۿôû۹ġӴʦǸ⵰ҲԸⰡǸ͹ˡ
ɫ˵ѧʱֻǸˮİ
ŮЦһš
˷ܵĽ˵ĲǴﰲߵС״
˶ĸӣһȺŮ֮䣬ʮۣ˿ӰɫȦ
ԣģȲʡ
šǶ֪ҸЩŶǱЩˡԭ𡪡Ƿקڰȭͷ֮ǰƵӳƣҸղ绰ǴﰲżŶԣпǽݲؽ
ۣӴʦ̫Ȼδͬˣr(s_t)q

⼸ż˼˼£ǼС¶ѡ˼뷨
ֱӸǮ̫ûˣ
ֱӴȥԡ̫ˣ
뷨ǣһɨ֧һѣȻӵĸԼȥѩAAһѡݵʱҾͰѵ绰ؼˣһɨ飬ȻץһлԼҵƤƤӵĵ绰r(s_t)q

°·ϣ鼫ãƤﶼ׼˸ǮңĩͿԴȥѩꡣ𣿲֪ûþöûʲôֻˣһξȻðõ뷨ѧʱ߻ḻʵĿһĥôĻġ
סЩ뷨ֻϸ΢С磬ﾭסɺĳû
һţȻȻһ¡
Ьʱ۾ĺε¶Ķڳ͵Ŀֻ󻵵ȴڲᡣڸ㣬Ҳõȵȥr(s_t)q
ϴ裬ɳϣʱǰȥû͵סʵԼҲ뿴õһ̵˷ϲá
ɡ˵ߴӴɳϵװڴóǮУֳֹǮİ׼õǮ㡣
ӹƱЦ̴۾ǣ͸ÿһйٵϸڻ¶ĳγģȴһֳŨĲܽҪС¥ͿϿֵɫ
ˣҪȥѩꡣ
԰ҲСʱ䡱ֻû˵ڣͻȻһҵĻһ֧һʵƤƤӴܳ˶Ѩr(s_t)q
̫󻵵廹ԶţҲԷĻؼˡƤƤһ񣬸ûѾڵòٵ
һȥƤƤ͵ſڣĵһ䡣
û˵ľǡֻǸδɵСƨûԼѴˣǣҲֻţܲԼǴԼɡԼôɣ˳ѩˣ
СëͷҲôģûԼûʣӦǾ͸һɣ֤ԼûȺڡ򡣡
ţǮͲǰתƵҪط䡣
ôȥʶ·𣿡ʶĻҰɡ
ʶҿͼˡиǰǻͼġr(s_t)q

ûеôѰ
ǰôģԣ̲İɣ뿪ѧУҲѧϰڵذϿ顣
ֻ˼ӣͼֲסˡôôУ̫˷ˣⱾûˮƽ
ϷԣϷҴ򿪵ԣ½ǸԼѪϷװȥɱ֡
ֻ˼ӣ;ܴˡôû棬ôЩֶô̬ˣѾһѪƤˣƫƫðһֻ޹Լ˨r(s_t)q
ʱ䣬ѾʮˡƤƤѾҰͷˣѩ˰ɡģôѩѡʱڿţһΣ˵ʲô־һðɣѹԼˣȥȥˡ
Ҵ·룬òסԼֻȥԶԶĿһ£湻ĻιߵĻԼҸȥţǷԼԡðԼαװС¹﷭һܾûԼʶ˵Ĵñ£Ȼǰп鿨ͨèСӡҰϣȻ֪һֻ˫ðĵĿͨռһ汳ٴһɵĺӣųȥ
Ҹϵѩʱһ£滹󣬶»ָ߸߳Σ˾Ȼ٣ֻ͸صĲɨһ鶼ûзѵּˣ
Ŵ߽ѡһεľѹ۲λһݿȡ
һѩƷɫɿݡݮĨ衢ɣȵȿζءҿһκһѩƷһֻŶرʢʫ裬ǵȴܰǻζˣ֮󣬶ζİζĵŮԱ¶Ű˿ݲĽţȴֻõһۣȻ֪ȤĴסˡŲ˵ϽתλȲȴܰҲǸζˣǶֵҳӵİɡ(>_<)
Сһȥˣ߶˰ɣҺһڿȣüͷ
ԶԶ۲ҵķԱһһҪʼë˰ɣ
һ߽һһСˣȻǣ֣СҲ±ҲмĿһͦظƤƤСӾȻһͯװr(s_t)q
ۣÿƯڵзԱ洺ӭȥһ
ƤƤᣬ̧۳ҸϽƣǰӰԼȫס
ʮ롣ԭƤƤڿӡ˵һ·ɣǲһͷ
ƤƤҫԼ·ѡҲûеĿǺס
ȻǸԶԶ۲ҵķԱͷһr(s_t)q

ĵķԱ˽һ
ʢʫҪƤƤܷԼһԼգʿȵİѲ˵ݸ
Ų˵
Աĵļ¼š
ζģζġš
Աˣɵģû뵽Сôչ⡣
ţҪݵġʵġġ
Աе࿴ˡ
Ҫɿġģݮġ
ԱЩɵˡ
ڰѲ˵һƣˡ
Աκˡ
㣬Ǯ𣿡ƤƤ½ǣõģҰҰְֵÿˡ
ոպáλǰǻʦ

µСӺƺƵİһӣƤƤʼ
Ľ
Ǿǣôʳôݡ
Ŷôݡ
ΧŮСŸ̾ƤƤӡ
˶ˣֻһռһӵģֲӰ죬û˸ȥƴӡ
㿴̫ˡˣռλòߣ˼⡣иѪŮìͷָңȻ΢ǾɼΡ
ȥؾӣֻõ˼ݸͲϲԵѩ⣬ֻסǸ˵졣֪Цõ˵ܲס۵ѹˡr(s_t)q

part6

ƤƤôණû˿ϸϸƷѸٵĳ꣬ˡ
һѩⶼûԣʣҲȥˡ֪ӷʴĿڴεһӵĽǣɹȫעҵȻǸĿ˵ļһܵ˵sorryĲӳȥˣֻǽһĸ˵Сشˮ
ղĺްҪһܹ
ŶŶǸҲζ
һǾ˫۾꣬޽㣬ɢɢתĸонʵءأͬЩʳ߼Ұһֻڹؼʱ̲ŰѾۼһ飬ŵְ
ֻһǰѳֵסĽաֻӱ飬ܷѳˡ
飬飡ŮϬĶһȴһԵλ
УϹ˵λܹݡư͵˼ֱ˾ȭͷ󺰡ʵʱŤһ壬ƺϴԸ˵ڡܵҵ
СݮߺҸǡԼ湤ҵĴȯŽƷġû뵽ĻᣬŹ
ǲǰ㰡塣
СݮӽˣУ˵ǣΪʲôǼ·أѵֻðĵĵĴˣѽôɬ

һصĪյɳϡ
Ȼǹ˰Сʱ˲ŷɿһ·ϻؼſڡ
㣬㿪𣿡ƤƤְֵഺٱ﷭ڵһطܣƼĳԼҪʵĻ
ͷԵúˬO(_)O
ƤƤҪҪҪɶģͿϰֵĳڲԶ·ϡˣüҳϹģǱϾ̫ͳˡƤƤΪ̾㣬ȥҲҪֵ̫ġ
ţ
ô󻵵ʼɡǳƤƤԲСֲڿڴﰵˬһ£ҲطдģȻΪʲôдŮĸ׻ΪŮԼֵ̫ĶʣҲӦòࡣ
ƤƤû̫е⣬һЩӸհԶĳɾ͸УȻԼҲ̫r(s_t)q
ƤƤקק¥ȥˣ㣬ݡݡݡݡ˿̨ס

һţͿɳϣſ˼ۣôô죿(o)
֪ôˡʵȥз
װϽƮȥ׼֪˽ʲôˣ
˾һ㿿ڳſϣͲģ
ʲôģڼҳԷ٣һֱ˾ԵúHAPPY𣿣λǰӢ﷭룩
ùһףԳˣ򵥵ˡҪһЩ
ֹصɳϣԺһr(s_t)q
˰죬ųȴûӡˣû
ǣ˾ȥСĲгˡ
Ĳˣϵģأһλۻ軨ظλĴһӰϽ⡣
ˣ޹𣿡
ΪԼˣɶ޹İСӣղ˵ʲô
ҲԼ˸е׵⣬һʱҲ˵Σ⻹һ־Υĸоôǰ
֧ŶǫĵһûõʲôǿλҪˡĳɰ˵ǣǸѧİɣͬѧˣƹʵˡ
ƽʱĳҲΪմװԭ򣬽ôһഺڵĴ磬쵽ѧﵹҲԵͻأ
Լ˵˴ѧеϾƤƤǸС塢㡱ҽУŪúԼƵģ
һлͣ豶ٿҸõġŮѰɣĩгСġ
Ůѣְˬһѣ͸Сʱʱɹϲһ綼ܺߡһ˿ϸţëĵƮɨһۣȻǸ۾ֻ70%ĺȤı顣Ȼֲѩꡣ
СƵĵһۣӹݹһָ߸ϵ
Ȼˣٹ䣬û뵽гͦȤҴƤƤгߡ
üΪ۵㹽ãҰԡ
һѺܲãҰԡ
ȵггʱ˫˻̵ֽ
ڼʱͶѧѧǸףܸƤƤǸСƨһ顣鲻˱һƤƤ

˼嶼ûӦʧҼſڡĩȥˣѵǽŮѳȥˣôЩ᣿뿪һתͿҺƤƤҰ
ǸŮ˭ôණôôഺȵģüͷȵ֪˷Լ֮һŵЦݡ
ô⣿ҵɫЩÿ
Ҳ𣿡΢΢ŭķһ䣬Ȼ󲻹ɫʹֱӵĶ򿴡Ůˮ飬ϴѧʱԼ˨r(s_t)q
λǣĳͷǳȥģһŮ˷˴ˣּʾλڽ⽻
ֻųĶûʲô飬һۣҲû̫ຬ壬ǵһ䡪Ҳʶ㡣
ԼĻӭԼƫҪԽԽϣǿһۣҲҽȥ
ᵽ׵ģǱϾĸأĻҲû˵ڡߣӦðſˣӦᶫ۲ǡ
ߣͲ㲻ţһ£֮ӡ
ˣȥˡʾһƮȥ
ˣ͵Сķ̡õĽȻĸиҵƽȵĹƶıԣô׸ĵġòţϸǷݲмǷĴš
ֻһ˿ţ
𣿡ĳ˿
˵ᣬˣǣʮֲĴӳƮ﷭һƮ˻ȥ
ԭŲúЦ»ûþ͸սȥûһӦľ߽˳
ʱ˾ͰڵϣȻڲһĻվڳſڣһ߳ҿһΧȹ
һĲڷϡ
վһߣԼϲЩ
ҿڵĲˣ㹽Ͳˡ϶ģǸӵмɡҰ￪һ£ֻ˳һˣζ治ĸоֱǸҪοĵطθ
ֻųһ˰ҲİɡĶһСһ㹽ݸң֪ӣͺԱʶֱӷŵҵǰֺͰĿ򣬡Ҳ԰ɡ
Ҿ׽Ķһͣô֪ˣղŷû̸İ֪̫ɣ
Сķܺ𡣡ĳɹҵעΪһڳ죬ֻ֣Ͳ˼˵ˡ
򡪡רоףʱ˵r(s_t)q
İ
ûӲ磬ЦţȷʵѵõЦţ򣬡ܺóŶ
ֺˡ(@^_^@)ǳǳӣҺδ΢΢ŵһ·͸
Ҳܿģǵ˼ҳ
ʲôϵЦͬϻ˯͵ƨɵСϻһʧӰ١ģ߸ʲô
򡪡Ϊ֮ź(@^_^@)
ҷ꣬ĲŲһ䣬ܺÿŶ

ʲô£󣬿ļû뿪Ҳû˵ʲôµļֻȿڡ
ʱֻڿڡˢr(s_t)q
ĺòá˸ԼĳнʵصʵֻǶƽѣһ鶼ûУ⼸Ҳԭͬѧ˵ҸһƵĳ˹󲿷ֶԼΡܹôİõڸҸۡľżҹ˾˸߲㣬һ˿ϸţëԲеġֻʲôԼʱҲﰡΣҪǱ䣬Ҳôðʧĸ
Ҽɳеֲ䣬ᵽһ̨˵ĿͷŴÿҹ˯žıҾr(s_t)q
ϴε鶼˰ɡ˵Ǹӣֽźܿ죬´αˡ
˭ҲóеҲһ˿ϸţëĹġ
š֪ȤӦһˡ
ô˵ţȴûվֱòվͿͣĲŴߵſڡʱСһ䣬ʱ㡣ûһظ̨ˡ
ŶһֱſƤƤ˾һͣȻӴϵr(s_t)q

ãԼеҡСõʧˣʤ»飬ڵ뿪
ҴĺȦվյ峿¥ݿڣŴɨ˼ΪԼˣΪĶˣΪʲôʹأΪʲôúû߲ûǻӰ쵽ԼأЩܲסͷϺò׳ֵ⡣
ɳժȥúϴһ¡
ѷϸ
У绹
ȥһҲԸзɨr(s_t)q
ô򵥵ãϣܵˣ
飬ôҪ𣿲ĿɬҲѾ

㡣ƤƤһ䷿ţһ߾춯صĺš
ţƤƤǰû񣬲ܸȥ档
ƤƤһۣʼʩ߼ȥŵҾʹ󷢴ȱһ
ƤƤͷҵĸôʹǶ˿󻵵ĶӹȻƤģ
ԼҲеġһСƨǶ֣ûʲô˼һиƤƤſ˿r(s_t)q
ƤƤڸһɴĵȥˡ
С㳡ǧϣƤƤһأ㣬ǸİŮˣʶ𣿡
ҡҡͷ
֪ƤƤСһƲǸ󻵵Ůˡ
ǰǸдĵĹ¡ôЩҶû㶮(o)
ǵȻƤƤĵӣҲ˰ҡ˳һָ㳡һ߲ϳǱСŮͰҺöˡ
ŶҲ濴˿
Ҳϲ̫װ崿

part7

ֻѾϾŵˡҸ磬㻹𣿡
Ҫһڿϣ̵Ļظ˰һ
Ҹע尡۵һ䡣
ҪѿȻɰҺҸȥһߴߡĵֻСŶһǻ飬һҲǻ顣
ؼǡԼҲҪżӰడr(s_t)q
ˣҸ·˯žˡķֵĶһٰĿȸҡ
Ҵ깤첲İϵĵ£һ£ϵƣ߳칫ң˫۳մһҹרòǸҪõĸӡ֮ǰ
Ҳô๤𣿱̫Ҵ󷢴ȱĹһ£߳ȥȵˡ
(>_<)

Щ֡
ƾͷЭڹڵķͥͨ
ҪЩ֤ݾͺˡ
С·ϣҵͷԻ˼Ÿոսֵһ˾
ˣͻȻ뵽һ˵֤ݣ鲻ԽĽгһЩ
̧ͷڼſڵ̨ϡ
ƵĵƹǺһӨ׵
ڵ˻ؼҵͷһůȴĸоЩҵζ
򡣡Ľһ
վ˻ˡ
šܺõȥţѵõʿһѣ֮űߡô
ˡ۾Ͱ͵ţȥѩɡr(s_t)q
ȴԼΪҪԼȥѩ⣿
ɴǮÿԼȥˡ
ǡԼ˵ʲôأ
˼šһԣҸĳѩO(_)O
㡪
ţԵĿҡ
㡪ˡûʲô

Ҹ磬ǰ̨ҡ
֪ˡ߳칫ңȴְԼ⣬˭ң
һλС㡣Ư˵
Ưˣڰ˭ƯȥŴӣŰ칫̬ȥǰ̨
ԭġ
ʲô£
ǹ¡
ǹ˾Ĺ˾Ҳӡ
ҵĹ˾أܵͣ۾Ҳ޹һҵ˾
ϴҷҲӦι˾̵憪Ҳû£ֻ򾭼ù˾˾
һ
ķһж۹˳ʲôÿ޵ġ
Ȳ衣ǰ̨˽ѽԵĴ̽һ飬û뵽ôһĻӴʦ˼ҶŪ
Ĵ緹̸Ʈǰ̨еƯƿ̫СصסôһֻըԴħֱƲסָ͵㿪ڲãõͷO(_)O

˼Ҷˣڵȵ緹ʱ䣬ѻһr(s_t)q
ۣױûлױ˻ƯƯ
Ѫ±ӣ˼Ҷôˣ㻹ƯƯ˵ʵڵģɰɡ
Ϊʲôް
֪˾мġקĺȿϲʲô鰡ֲƲ֮ġ
飿Ӵʦ
˵϶˾
רþ׵𣿡
˳뼸ֲ鷳
˵ѪǺô
޼ա
ʲô޼գ
δֻŮ뿪ʱ˵Ӵʦٿǿ֮ġ
ţǾˡӴʦôɴ࣬ʲôֵذ
˵ƵһԼżֵп϶Ľβ϶ǣӴʦĸ磬˵ʲô̼Ķأ

Ҹ磬Ĵ档
ǰĺ֮һ
ȻĴϡҿһۣͻ
ֽҲǰĺ֮һr(s_t)q
ؿڷƣԼ¹ˡ϶Լܾ𣿣д
еĹվǰ棬߲֮µĽľʵĸֽǡ
ʵڹ츳ֺңڸϣĵĿɿڴԼ˻ȶԼҪԲ뷢չϵ˾һ˿ҲԼİ绹һĴ⡣ʼߵˣֻһķ֮󣬶ò˼飬ôЩôģΪؽڵ˺٣ؽȴŽڵˣ֮١

ԼĴǰӴѧҵڣѾˡҲˡɷǸӣԼƣҲڵǿơֱˣнһ˽⡣ûʲôȱ㣬ֻԼҳ軵˶ѡ˶Էĥ˰ɡⲻɡؼǣûСż˲ˣͶһ֧жѪȲȥ֦ɢҶıڹϸɳһɼ˽ŸٿǺӵ⣬ƼˣҲͬˡǣŷ֣ЩдʩǶһ١
˲ûԸõĽᾧɷ򲻹ֻǺŻNӴһܶѣǣҲûܲľ뵽顣
ɡüëֲ֪ƲҶʲôƲסְֵĳְֵģ˾ҲְµģǼ·ǲƲôģָɲһһƽԼóȥɡ
ʱȣһй֮޲r(s_t)q
ôûвƲĵֻż˸ùϵûСҵĲȨҪּˣʲôżҵˡĵȻˣæ

ֽǰԥֻСֶ
ǲ
Щ漫пشԼڰһǰüǣǡ
֪ĴӰð˳
Ŷ
ǲʲôңһ̸塣
ûУûСеϡ
ǣķһСڸ»ˡ
ģģ˼ֺöŶ
һ롣һС˶۰Ͱ͵Ű򣬰СﱳôСҡȵȣͬѧͬѧǸӴʦͬУǸͬѧ
ͨˣͨˡͨһ绰Ƶļˣʶ𣿡
ҵĴѧŮѰϴθ˵ģСԴôأǸʱ򰡡
ģҶǵáѸٵpiaϵ绰ڲˬ¿
˵ͷ
֧Ӵʦ
Ҳǡ
Ҳǡ
Ӵʦ̣ˣһֱĴţ
д¥ֲģʮĹǶڷԷأòĿⶼͶ䵽ֿںŵһСϡ
ߵſڵңһr(s_t)q

ƣĸϻؼҡ
ҲįӡƤƤ˵ԼǶôˣô
ڵƤƤǵС˼Ȼдˡ
ţ˵ҵ
Ҫѩ⣿
֪
֪ȡҺؼңֲѩ⣬㻹𣿡
ؼһԼѩ⣿۾
ֻ֪ѩ⡣Ҹеƣּһ㣬ϵظ޴жءҾǼ裬ûѩĵȴ㻹𣿡
ᡣްĵͷ
Ϊʲô
Ϊҵ˰
ڣһĩβ΢Цϵġ

ҾܺãٳʱһϢĩҴȥ档Ҫȥɣ
ȥͷһָϨr(s_t)q
ȥֳȥ԰ȥн֣ʵͦмʶģԭҲһЩط
һɫͼס
оǼһ
ЩѩﱼܵӰӶô죡Űɫްľǲѩһ
ۣѩѩȦСѩ̫̼O(_)O
ȥѩ԰

Ҫʹ󻵵ȥѩ԰ƤƤ죬ҡ
ܴܲ㣿
ˡƤƤ̾󻵵ôأл㲻أ
УԻˡ
ǰǴ򲻹ܣʲô£ƤƤŵذ壬ҲһǺ㣬ЩҸһҪġ
ĵĵ˼롣
ƤƤôһ㣬˵һ仰ҪǹҹĻǧҪӦһġr(s_t)q
ݽˣƤƤįĳţҲһġԼÿ춼˯һ(o)

ƤƤͬѧشһ⡣һ꼶ѧʦһֱѵƤԽƤƤصȻһСѧ㣬̳һҲʮ˲ġƤƤĹƨͨѧɽУ˵һѴͳĴأԣʽĹϣѧʦʮбҪƤƤشѹ⡪õ壬С䣬ָС䣬ʣ䣿
ƤƤͬѧѧʦòٴһվȴЩĲɵƤƤ
ʦҿر𣿡
ڽǰһŵʦµĻعͷ
ƤƤΪ֮̾ҿҪʧˡr(s_t)q

ȥˣ
⡣
Һڵذϣĩĳмƻ
ѩ԰ҿ̯ڵذϵָϣԭǹػ˵ĸȻҲһЩȤ
԰ԶһСɽϡص·̾Ҫд졣ͷȻӴʦԴһ¶ûо׳ˡǣʹ򵥵һֳԤƱøӿ
µİҵͷгΣҪһҹˡ
ſڴ϶ŴƤƤˤȥr(s_t)q

мյģڶʹ绰޹˾һСȻò۾˳ϣ׼ϰϰ߾ܿˡ
СַǳԼĵ͵ǳϲرǳ׵ɫӿؼҵĵ죬Ͱѳӵһ¡ȻҺקС⻷ϵʻһȦ
ƤƤԹԼĴ̨ϿǶ硣

ϰҲȥϰ࣬Ȼ¥¸ЩߵһСԵúܺᣬӴʦݵӰһСüĺˡ
ι˺˧ŶһڵĴ
С׸˦ֻԼĺӾ
Һ˵ء󱼲ˣû˸ԼôקأСһｫôͣ졣ι㵽׻᲻˵ѵҪҶñУ
СĻӦһ¡̫ءr(s_t)q

ŵһ̣ӴʦݳϢʹ
ڲֻҵͷǺŵġ
Ali׵İ򣩣⣬Ǹ
ı磨ĻףĿ˵ܻĳ
·˼ףƯĳӣ൱ףŶ
·ңʵģǡӴʦ۾
·˱ŶŶҲ֪ľ뵽¹ʡô壬һαװô˹ĵӣ̲סȥѵ۾ԭα¶ʱ̣̫ˡ
Ali·˱ɫ
ı磺һӴʦ۾Ƭ
·˼סҡŶءأٸ㼸š
·˶ҾַʦФȨ
򣬰·˼סҡ(t_s)#(t_s)#(t_s)#(t_s)#(t_s)#
·˶ͷ񡰶һˡ
򣬰·˼סҡղǸ˭
鿴һ·˶Ϻϵͷˡ·˶ϴr(s_t)q

part8

ˡ
һ°ʱ䣬ҵһ߳һƬ̾
ڲҪһһˢҶ°ˣ°Ҳǵ̳ȥˮHAPPY
Aliһٱơ飡⣡
·˼סҡȵȣ
ֱҪҸأĻٺٵЦ
·˶ٲͰ㱬
·˼סҡȵȣ
ı磺(o)
·˼סҡȵȣ\(RQ)/װɣ¡
ı磺ӴʦĩҪУѩ԰
ֵСƣȥѩ٣

ڼſڣһС׵Ӱڹ䴦СЦһ仨
Ұѳ־׼ͣǰһ֣ѸݵɹƷO(_)O
С˹ֽýĴڳУ⣬ԭԼôãǰЩ˾ԼûˣԼԱ˺þá
ʵҲû̫ණɽӦʲôСֻǣǰУѡ

ϣʧߵһνٵϡ
ڵĸо˷
Ϊʲôǰδе
صĻϲ˵˭
µĴůѵȻ(o)

ѩ⡢˾ϡ
һ˾㲻𣿡
ĳѵһ˹ġ
һСѩͷڻЩԲ¡
ŶһЦüƶܿĶ͵⣬ֻһ£ڻһѩġ̫˷𣿡
ǰɡ
ʵԼҲ˷ܣЩȸԾȲֻһڰһ˾Եǣϲƫƫճˡ
ڳ͡׼һ£ʹˣѱѾ˼ϣȴʱˡ
ƤƤ˶ſڣƿţȴĳ
ŶҪ𣿡е㲻֪룬ŵ˫ڿڴ˼£վûзŵýȥ
šϵЩ˲䶼ӰҲĹ˾
֪ĵִٲתţƤʱһ£Ҷ֪
ǻʲô£ҺҪȥ档
ûˡĵһ˿̵ʣתߡĵȷǲ״̨̬׶ˣһžͲȿȥѸֱ۰ҪԵקסһζĶͷᷢƮһߡ
ůȵơǰһѾһ˼ǿģȴֲݻӽ¶ȡ˺ܾõѣ֧ŲסЩί㲻Ҫſң
Ľסңͷůš
ұͻӴô˲հףŻơǣԽƣȴԽʹ֮е˲䱬ը
Ҵˡ֪Լˡţ﷢ұһĻģ
۾100%ֻǲǰ顣ǲǽҪǳˣǲȥѩ԰ˣ뵽ѹˮеˮһֻʡ۾ÿգһ£ͻպϰٷ֮һֱڷԼĴһеʧȥ
򡣡ҿѹҲѹȻԼˣһֱץסԼѨˣ쵽ôˣģɣҪˡ䵭ĲǿӲֻǸûȶءһ˿ѵĳҭһ£뿪ҵ塣
ұ񾭵õһķɣȴϱĶͷôŪģ
ĵĶͷһʵ˿ڣղڶ棬ͻأĳ毺õ֮ϡĻŵĵԼ˿ڵһܳ
ˣҵĵͺг˷ŭǲǣ
ǡҡҡͷԼżҳܣɷNƵĵǡ
ݰɡҵ͵͵˵һɨϵʧ䣬ﱻһ¡ǣֱĵľٶ̫õѾһϰߡ
ϰĶϰ߾ˣҲԸ񣬾һ֡
ûȥҽҩ䡣Լȡҽ޺ƾд
ô䣬һ˿ߣȾϴ󲡾鷳ˡ
ĵϣѾڸղŵĿŵڵõһж˵ԹԵģµСӣҷڶͷϵĶһжôáѧꡣҴûкԼһμܡʱԵԼһҪͣҲ
˵һ°ɡҵûʲô飬ҲԼڴһ°ɣԸ
İԼżҽԽԽս˵һ飬żѾ·ˣ飬ֻܴ·ߣλĸƷȻҲŻƽʧȥ顪ĿЦһ£ѾûˣԼżѾûദˣرǼ˵ĴӲܳܵסġ
أ
³֣ͺ˼롣Ѿ㿪ˡ
벻飿
˵С
ĵ绰ҡ
ţ
鵱ȻҪ

ûؼˣɷȻضȼ٣ϻҲҪϡַŲӵǣֻ
Բ𣬴ˡķĵĵǸԼûбİ취ˡ̫ůԼ̫䡣
Ӧø˵Ǹ챾Ҫȥġ
Բ򡣡ת¥˵û˵ûϵֻûʲôĿһۣ͵ȥϴĨˡ
ˣ߽䣬Ժȥ档
š
ˡֻСһʱΡҾҲûж˵ıҪ

ۣ㣡ƤƤûϣֻǳϰſڴ￴һۣͷ˷ܵ
ѵúƢĸƤƤţȥһҲܻʧˡ
ۡƤƤ򣬡ȥҼңҰְֽȥԼˣ̺
߳ſڵıӰ˲һ֡
һеȻҲӰĵѾ߽ҵСԼӦף𣿿ǣǺ̰ĵġһֻһ˲״̬Լûȥࡣ
ֻһӶ𣿡
šҲһЩʲô
Ϊһ֣ϵıͱתôֻһӶأ
ܵ
šһƬѩǺαҪôأϸһѩƯһµı
𣿡
ȻûСӦûаɡȻǲԼˡ
ĶģӰһס
ô֪ûСŮӵ£ῴóôƯ¡кҾͻ׷ġ
ǲѩкأ
ôȥѧأ
ʦÿÿζԼģһڸʲôأҪҪһЩ֪ʶԺԼǰˡߣֻбǴѧµģŻ˵ϵĳĳһ󻵵塣ǿɲãԼɲԶ
Ի˱ˮǳˮסһԼ磬۶ķ

㣬ôûȥ԰ƤƤʮбҪһ˰̡
пˡ
ǸŮ˰ɡϴξ˵ǰ󻵵ģ㿴ˣֶ˰ɡߣ󻵵汿ô׾ȦףǸŮĿҲˡƤƤСΪȵļ⣬Զˡ
Զգ۾ס

ʱŲ˰
ȥгЩʳģ³˼ˡ
һ𣿡
ԼԡҲĸ˵
Ҳ̫ŰˡƲƲ죬Ҳû塣
ԵһʱҵĲдӳ˳һѹţҡҪȥöʮ֡пڸɴӷԾ
򣬹һ¡΢ЦŴк˹
С룬ԼΪƷʢһ˵ţ⡣
ԵġҼ״һĹ˵Ů˵ĽɫˣСǰ͡
ţðȺŻȵζǳС
ҪܺóԵġİס֣СĿ񶼵˿뺦ߡ
ȥѾˣС뷴ˣţֱϣ̵һ¡
û°ɣҿĵ֣ķŭ򣬡ôôСģ
ӵĿңǷݼڱһ˶жһ˵ŵһˣСߵС룬ٱ͵ϮѾûڿ־֮С˵Ƣãĵһͼʶ˵һ档ǣڵ˺İԼǲǷ̫صĴɵɵŲԼķ䣬һֱ¥ȥ˯Ҳûҳ
סڣһţҲûȥ

һǷ͸
ڴϣ޴й뷳꣬ǸصĸѹֵĸۣһǰδеаԼĵ𽺵̤š
˯š
װֻһҲеܵӡ
űӣݵıܹԼñˣźȻݺݴһܡ
תȥһ˿ϸţëҼӲĵ׵սľðӪѿڡʲôö򷢻ԣϣǷеҰҰ޵Ұȥһֱ֣֡֡һֲ
ڴѧڼɵϰߡտʼ̸ʱڴİЩʴӣֻ˵õ˶ӦúúõıԼŮѣһ˿˿˺Լϡ
ȻѾôãϰȴȻڣͬһͶĹߣͶļں󣬻һֱûж
Ϊʲôûж
Ϊʲôûж
ҴԼһȭΪʲôڸǲͲӦһ˿ǣġΪʲôÿһΣҪһЩԼסϴǴֵĶǶͷ˿ڣ´ʲôΪʲôľ壿ԼʧʹĵҩڵԼȴϣԶЩﲻڸ飬ٸӵİӶԵͷ

峿´ĵܸɾûһıꡣ
ãοֻһ˲ȴֲޣġ
͸ŽҴţһ㲵ˡĵɷ
ĵǶεĴյģֻǿԼһƣȻЩȻˡ
Ҳӷ˯߲ų
ĵ۹˼תڿԼɷ򣬿˿ڣһط
Է߳ţԸʡĻޣϣһҪף
ûʲô׵ģѾ׵ײϲˣ˵ڡ
ޣһЩٴһЩ
˯£Űӣ΢΢Ų뼡Ӳһ·ֱû˯С˻һһֱʱң
ǻڿͷתȥϴȴӿͷ̽СԹԼ
ڸղŵ˻ж̫еûġαģȡƬ̵һǰõ½Ҳ飬ڶ˿ܡɵЦţֺõĵĽһ򡱡
Ȼ󣬻ûϵɵʾۡһˡ

part9

еˣߵǰް
ֹסЩӨ͸ˮֻ˹񫣬ǾɽţȻسɱ
˲˻r(s_t)q
ްɡֲˣ˵޳ͺˣǾͲεһΰɡұ˫ۣ£ǹػ΢Ŀֱ޵СһһֱһʡѾһϰĵˡ
ҵͷԼһۣȻ⡪ļϵǳǳ̻ϸ΢ó⡢ǽ
ǴǳǳЦƣ׽ס񣬡ˣ
šͷ
Ϊʲôޣ
˸ղ˵Ѿ׵ײϲˡһ˵ֳҭ
ŶôҲϲ㰡
š˶ϲԼˣԼôסѩ⣿
ĺõĿһͷȻ֡Ҵȥѩ԰
ţ
Ǽھѩ԰˵ڱһŽţһһƯ貽ΪԼͷ⣬ڴջϲĿȴһãȻ͡ʧ
ֻбʮɷ羰˵

ʵʧߵˣҡġķ򣬻һˣǡƤƤ
󻵵˵˴ȥ԰ַڣһŮڼһԼȥĵľʱһϣƤƤ§űӿЦͣһǼΪʽĶͯװְָԼԼ׼õ֪ĳһ룬ź˷ȵҼңֺзȵ̨ϣȻ󿴵󻵵ں
ᵽżжŵıңƤƤֺзȵĴӴ̨ϵr(s_t)q

ûԼĳһλã
ûС
ֻҵİɫֲ֮µС䵱ʱʱ˵ʵﶼԼ˵һλá
գҵ͹ϵԵôһЩ仯
ȣҴɨġΣ·ϴġٴΣзġأֻһֱͷϰſ鷽һֱһ֧ѩ⣬ħһ֧ѩһ͵ģһɿģһǹģһֱĨĨr(s_t)q
ʱ֣һȥС㳡ɢԭƤƤȥģƤƤֻܳԹԼҷϣǧ浴ơ
ⲻԾľתḻĶľϡ
ԭû
а
˵Ըвڣ˺ܺõġҲġֺθеġһѶŲ˼㾦֮ʣġ˵꣬Ϳôһ˿ϸţëڴ롪
ǰЩﶼǸƣʮصĲ׽һ䣬Ȼ϶ҡҡͷ
ɵЦһ£Ȼϻùŵ˵ϡ𣿱˵ѧ˵ûİ˻ģ룬һµǰӳƣֲһ㣬ѧ
ûСֿ϶ҡҡͷ
ŶԼĵһˣ϶ļռһλõģԺٹٸ죬ٸˣӦûǵԼɡ

ĴⳡСΣڹȥˡһʱͷĵһͷһС¡һж鹦Լ踺صĸô񣬻ôѩ⡣ΪԼΣΪһ֮һССͷǿһаĴڵϷһССնԼԴУһһģעֳ֮˳ȥԼôôһСˣҲõŶ

ι
²˭
²
Ĳһ¡
ҹˡ
СӣԼˡڵ绰һͷΪѻİһµݰť
Ŷ谡ʲô°
ûʲô¡
ûҹ˰
Ȼ绰͡ཡˡվڵֻӶŲ۾ǰĻһ룬СӾȻԼĵ绰յֻһ⡣ͬ˵ݵһŮСŵһ
Ǻǣﲻ˼Цһ£źŲãǺǡ
һ̫̫Ȼ̬ȷʵһ̫̫ڸԼͣܵ𰡣ССϽҲǺǡĸŸЦ
ϣ˵еĹǲһථòЦöϲ˫۷ŹĶ˼ҹϸһСȦСҪ͸ˣ޲ҸԼҲһԼĻò
üëһʮĿˣжûأ
žڵòֺС˾һصˣôʵֱӸ
ûУˡ
ţôСԼСЦݣƽСţ׷ҵ˶ȥˡ
Ķأǲûп۵ģĲѧѧԸֶԲϣԸû
Сѻ𰡡׷Լܹһǹ˾˧ݸ㣬ԼܽѧһԼ˾ѧߣҵϧƢ̫Լ鶼ܲغ̸ޣﻹһλȣһǸԼһ˾С꣬ദúܺãǸûûҲûǮꡣƽ±С˹Ƶķŭ޲ðƤǸйߵĵ޲קڵǽϺݺݵıһ١
һֱ۲أûˣĵĿڣҪҸһ
No3QGoodbyeСһԼӢﶼ˦꣬ڵſ˲䣬ͼ˳ȥ
ŮƤ污ŵŹϵ˲䣬ϸϸĹ˾ƼסˡԼģϴСˣҲŪԼҵھǽ۸ԼȻھӵĶÿĶ󶼲һȡͻԽչһԼϱӰӶûС̫̫̫ѹˡνǣһҪѶӵ¸üĿ
֡һ££ͷ߳ȻЩԤеĻСżŵ߽ҵʦ
ǰ̨һ̧ͷ￴š
ţӹĵطȻǸ⣬һͬ¾ͳô¡
λ򿴲洺̫̫ʲôĵĹ˾ˣգҲݽ𰡣ѵԺթ
ҡΪʽЦһ¡
ʦֻӾþװġǸĽţҪұֱӾʹֻˣҴʦ̫æЩĽǰһЩ͵İӣԼΪǰ̨һҪѺĵһء
ҲǴ˾ģġ
ԾŴͣǰѵԼδŨr(s_t)q
СӻòڽӴŰչˡж𣿡
ٰţ׵ݻƤҧҧ촽ԽЦ̣˵ʲôأȥʦȥ
ѽһӴҾȭһв״㶨ȣ
ߵҰ칫ǰˣҸæأ
ˡ
һ顣ѾŽȥˣȻֺܿ˳ˡ
ôˣ⡣
˵˵ڡr(s_t)qʲôӰ

°ӴһȺˣͬ軰һµģɲԼǸһͯĵ衣Һ߽ȥڸչʾҰաƬ޷۷۵ཱུģﶼſߴ׳رǳʲô֮
һָԼǸТĶӣ￴Сʱһ˫ƤģڶǵƤˡʮϵĶҵ۾ιϡƵġ
С첲Сȵģܳô󣿡
ֶ޳ֱ˫ȣ׳ĸ첲ȻŲϣͨƽʱлôĿŵʹʦ壬ϲ̵ģЩա
᲻᣿᲻һ˵ˡڵĺˡԱߵС뵽ʲôˣ˵ڣǷƬ
߲ߣƵĿڡ
ﲻһֺۣͰɨӴһȦʱǡ

Ƴ̳ϣһֱ˼״˳һȻʵӡҡ
ţഺĸ߶кһġ
Ҳ֪ôһֱҲ󡣡
ˡü
ԭˡ㿴ǵλЩһһƵġҪг࣬Ҫòòһطϰ࣬Ҫǳж෽㡣ҿͲһڿࡣǸСףǶͷ౥һǾˡСҲаЦཱུģƵģϲС
ȻôŮģ۲ͼ˺հ
λᳫڲ̸ûĵĶӣƫȵϣ㽵̸һȦֻ֮ӳôһ䣬ɹһǻ顣
ҼиӶˡҵȵ˼ſڲһ
һվſڡձԼӴٸǰһղЩҺֶŻˡ

衣ʱݸһСʵûʲô庬塣͸ഺкӰѸĸˣӻսһ
ûʲôģһ˵裬ȻܶµЦһ¡
һжء˿϶¡ϲ̵š
˰
졪
еġش
û㣡ﲻĵ˶һۣЦĻعͷ򡣡ˣ
ǡ
裬Ů䡣
JQȷˣԸпοԼ㿪Ϩr(s_t)q
Ҵһز㳢ӰٱͳһֻС
̣Ҳϲ
ǺǡŶЦһСԴ涺˺ð죬ֹסЦ⣬ͳһֽ
ʲôŵݵԼǰֽĹذĿдһЩںɫĶ֪ʲô
š
ĵͷšȵζţζԭͲϲپǼ֮һζ롪¡
ҸմԡҳͿܽ䡣ߵͬסĸǰһţӵζǲԭţɺóԡеľ׽ţĥŽıһԼزģȴڻž޴ľϲʵĺˣ
Ȼұҭסˣr(s_t)q

part10

裬ûСͣ˾ʱľͷԴ˿̶ȼˡ͵ԽԽڡ
Ȼڱӵ֮ˣﻹýȥ˵ʲôֻССȣСӣһţԣ̸㴩·
С¶ڲŨr(s_t)q
ϲһƳĵ˲۽ǡܺ۾ˡ
ҰֲǶȥ˺öˣʮֲТķʣͷܵڽŵ͸һơ
ģʵһ൱ֵļָͥ˿ڶ࣬ҵĸĸرȤĬ̸ġҵİְȥˣ⣬֪޷εİ֢ͦʱѾǸҽѧ漣ˣǰһֻһֱڴϣԺҪź
ڷˣˣͲôˡҵ׷ʱû˵ôĵĻߵúܰ޵úܰһ̵˶֣Ϊƽʱ̸ЦĶǴɤţΪế޵ġ
ֻ֪ںͰְֿЦѣТӡҲͲˣֻǰ۲ţȫ
óһţɡӣóԵġ
߹Ȼ󿴵ιСr(s_t)q

򣬰ңɡ
ˢˣ㶼ûԷҸɾˡ
򡪡˵á
裬Һûʲôֻǲ⣬ˡҳڣӵһԣ͡
ֻǲ⣬£
šҵͷ
ﷴӦ㶮ƨбԼһۣӰԼҪ˻һ쵽ûС
ôҸ֤һ䣬λܼģԼпϵˣҲʡŲˡ
ŶԼ绰붼û˵ԼŮѣԶңصĸԼףû뵽Լһ£
ҿĵ¥ȥ˯ˡڣҾͱСӡ

ôһֱԷأ᲻ᷴӦ̫ˣ⺢ôݣΪˡ°һؼҾΪ˲һ򶼺ƤƤһ飬ֻǸ˵ѧС֡Ҳ°ˣɺڵӡ
裬ʲôʱȥ
ôŸߣ
ǿֻСסĺܲϰ߰
ڽµСֻǿҴ󻵵ŵġ
ûʲôëģղŻúõġϸ˿СǿˣأǰڼҲȥɡ
СҸôˣȻ˼ҵģ
ţҴȥ档ҵСһţ߰ɡ
Сƨɾ͸һקһקĳȥˡ
ȥӻָղŻ⵽ƤƤҿһ£СڣȥأҴС˼ţһѰ˼ȥС㳡ƤƤ

ƤƤȷʵڵƹ貵С㳡ϡиѧӵкӳ
򣿡кߵǰ
šͷ
㣿кӸղŻܱѧӣһ˵Ƶģ۾ֱҲͻȻһЩܵ㰡
ƤƤһɼ˼ˣ֪ɫˣߵߵ֮䣬кǰûƤƤôr(s_t)q
㻹ʶ𣿡кȶסʿһЦ¶Ưݡ
ϸ˿ҡҡͷ
ʶˣкЩйִܿϳԼȷʵΪˣ治ֱӵľ͸Լ˭һתкӴӵһѩŵ촽Ϸѩܿͻˮϸ
кӹӣαװСӡ
ޡ۾һգ΢΢ſ㰡
кֿ˴Цݣ˲ȥ촽ˮΣҾǳ˵Ȫ
򡪡ԣȪղŶû֨r(s_t)q
ƤƤ֮䣬ֱţ˵һע⡣
ʱ򣬾ͲôءȪԴ˱ȣʮꡣڣҶʮˣڸĴѧһ꼶
šԼǰȪҵĻ԰
û뵽㡣òһûб仯ֻǺһкӵ΢΢һ£˭֪ʮԼϲԼŮأңô꣬һֱϲšסС
šܴһ仰
һҰкӳܿ˿
ڻ԰Ǳߡ
ޡҽ̵ģһСӵ١ŵһˡ
кӷ·˵ĻЦһ侫ʢĻԽԽʢҲܿģֻкӽ飬ĸʲô֮ġ

ں˭ҸϵС㳡ʱͷһкһ顣
򡣡
ţƫͷˡ
ȪҲҿ򺰵ġˡʱԲһ¡
áȪòҵͷҺԭʶ
ŶôСҵﻮһ˿ϸţëĵִܵĵͷ͸˵ûؼˡ
һȡȪóһֽдԼĺ뽻򣬡ʱ绰ҡסĵط绰Ǽţ
ҡҵ˵Ҽûլ硣
󻵵ƭˡƤƤһ߸̡
ŶȪε±Ժ

ﶼˣϹϵԱⶼڶˡ򵽵˼ٴΣҲԻˡ
һ˼ʱ򣬿˴·һ𣿡Ϊίʡ
ࡣ͵͵ģڿȪдԼֺ֣ģÿ
е𣿡
С
е𣿡
Сڰֽڴ̧Դҳ֪񡣡ǲҼˣ
š
ѩ˶ģֻСͳ֮֡
ӡҵһôѧǵ˵ԼҲȷʵѩ˵֪٣ôһ˿ϸţëȻ𾴡ԣѾˣ
֪һҲҡҡͷ
ҾøղŵԼǸ׳ըr(s_t)q

￴ϱţӭȥһֵֻĶ¶Ҫӵͼ
ѾԹˡҶ˵
򡪡ҽûѩ(o)
һ򡣡ĿбӵԴͷᴵĻ
Сۡ
ҡͷζ㣬̫ǳġ
ţңСôû

ģôû㡣Сһֻ罻СŮ˳ɢʱҼС
ҵһҼСҡҡβͣеġ
Ŷ˵񱾵صġ
ȷʵҲбԭ򡣡ҼСдŵԸ͸¶ܵĲҴɹСȤ֮󣬲ûʲôƵ˵ʵһѪͳְ˵үүġ
ŶСһۣҵһѪʲô֣
ţСˣԼţƿһֱԼӡ
˵
ûСôҽҡСе飬رǿԶԶ߹˨r(s_t)q

ΪԼڣѶӺͶϱˣСɶ䶼ȴҪ˼ɵԼÿһ¥һ¥µġСȥˣǵһμϱ˵ʲôҲöϱ֡ǣĩʱһڳй֡
֪Ȥһǰ棬ֻǲʱŤͷʵ뿴Сڶӡ
򣬰ָҡ
ͺİֽңǱҵůһ֮Ȼˡ
ֲǺ񿼹Ƶ⻬С֡
˯ŵʱͻܱȻ֪Լ˯ŵʱ¶ֻǼѩġ
˯ʱ򡪡ҾӸԵһ¡
ﰴռƻ߽ʯ꣬Լ
ЩܳԵĶһȤҲûУһ۾Ϳһѩ״ĵ׹ָĻ
ԭ˵ĻôֲŮӶϲ໥ҧ䡣͵͵һǻķ㡣
ӵ˵ָָָ̨ѩ˵׹̣ϲ
⺢ǶԼ˼һҲԽԽϲϱ
Աӹ̨ȡѩ˵׹λʯ˵֮׵һʯػѩˡһ໥֡ͷϵԢ⡣
ѩ˲ǰ׵𡣡ԱĽһ£ͰѡػѩˡҲ˳
զһػѩˡҾúƤƤr(s_t)q
ǶҪˡáͷϡԢⶥãˬĸǮ
ڵľǰѡѩˡ˲ϡĿԱǰ顣ǰҪһɫë£򡪡

Сˣǰᣬ˵˼ѵĻСӻʵ˼أϲˣ۾Ͳб˵ġ
ԱߵƮһ˿ϸţëɬٿ򡪡һԲӦı顣

ˣֱǿġǰһסʱҲǰѷʰøɾ侮ÿϴˢͽԼˡ
ֳɹ˼ļȨңѭ򽥽ĽԼһЩµ顣
嵰ҪѧŶ糿˻
ĦҪѧŶǶѧԺఴĦ
ļҪѧŶΪ˲ڼģѼӰݴˡ
гɾ͸еĿԼ³һԽԽôˡ
ڳſϿŲ׼嵰ЦÿȥҸһҵľӼҲƷˣټһ

ƤƤҵħצȡһͬ㳡Сǧʱ⡣㣬ǲûˣ
(o)ŶŸղŵĲء
ˣ˵ڡƤƤһ֪֪Īľ飬һʮεƣڰ
ƤƤŤͷһָֻŶӣΪˡ
۽пеı飬ƤƤŤͷطϵĿƻЩõġ㣬֪ţ֯Ů𣿡
ҡҡͷ
ɽӢ̨أ
ҡҡͷ
ǣǣšˣԺʺϡļǼأǶʮֿ¡ǣҳҪǰٷ֮һ˶СƤƤطܶƤˣʦҪ󱳿ʱڱֿڵһΡԣûƤƤֶߵĶֻ֣ܳͺˡ
ŶڶСһ˼Ǿˣ𣿡
ţҸղôɿ飬ӦòֻΪôǳԭɣҲǡģɵĵԲԲԴ
ǼһҪԽԽļ񡣡Ϊ˰οƤƤļįģһӢ˴ˡ
ţ˵ĻƤƤо̫ˬˡԣǸ󻵵֮ǰҶġ
ԼĹ֮ƤƤŭĴǧϣƨվô飡ҰְȥŰˡ
õġæֹ
̫ˡƤƤڳûץס֣ͽһЦֿԣѾԽԽ벻ˣ㴺ߣ궬Ҽңͻݴġ
Ҵһ硣

ȥҸմ
ոԼҪõĴӡ꣬һתͿϴӴ
ôϴıЩЩԼͷиûдţӲקһΪеĻһ˷˵СڼԵʱʹ顣ԼúO(_)O
ϴе൱
ţ
ʦҪϴؼȥˣ
Ҳֱ֪ӷŵļóͬǲµģԼί֮¡
ϴɫֺһ㣬ʦôӰˣ
ÿ춼ܶϻؼҵġ
ҲáϴĹȻߴǵͲøӰˡ
ϴһů߿ˡ

part11

ǰһ棬ʺСС;Ͷʲô鶼ȤȱȱһʥҹڵϷСѵõһ׷Ĵѩְ˵Ҫ͸ԼһرʥʥʱԼȥ˻԰
ǿ˽һĻѩ͸ӨʥĹâһ룬ͱһո۾ƯŮ
ŮСǶȹ˶ݵĶ죬ȴ˱ʺ¸ҡȫͬѧǰʶģ༶ȥμѧУݽڽӴкӵϷôͷֲϵĲ㣬ϿˮһֱԼе벻мкӶԼĿ࿴ñͬ˻Ҫ죬һ䲻Ƶļᶨ
ֻаְ֪кܡΪʥԺСÿһѩǷʥкӶڻ԰ѩˣѹĿЩûѩһڻ
ڵ˿ѧʱѲǱкᶨѡһĴѧѧĵһѩʥһҹû˯ڶϣȫУͬѧһٳѩˡǣûҵˣһѩ˻һѶϢҲð
ģȻһҽ̵ĹУ˰ԼһĶռǸѩĹ
кȪ
ȪʦһôȪ̵ѧŮԼʦˡ
Ŷһ顣Ȫ˳ָͷںڰټŮԡ
ҵĹ̻Ǽţ¿εʱȪŮԼֻ֪ĸ׵ֻš
Ůɬдһֽ
СĺŶ⼸λͷİɣ
ӦͬѧҲסҵĺҼҾͲλ
Ŷлл´μȪֽۺãŽ
ԴϴμڣˣԼҽ̣ȴûÿζС㳡תһȦֹ֮⵽԰Ǳ߶һȦȴûջ񡪡ԽԽļr(s_t)q
ֻÿȪҪ鲻ԽȥǲģȻδʧȴŽ̲ġ
ϡĻô˷ѵȪһԴеˡС͵Ӫҵٿ飬һҪҵ
ڹϾͿʼ绰ֵĹ˵塣ѧУȪѧ¥ٵĶ㣬վڲǰŻһҳֽʼ绰
Ȫι𣿡
ɧŵСAûˣ㲦ˡ
ɧŵСBţʺٲ
ѾеСC򰡣Ҹһ¡Ȫĵһӣι£Ҽûˡж󰡨r(s_t)q
ոձŮ˦˵ĳУţǰ𣿣Ȫǣ㣡ҴˡĴˡҲøҳȥֲ˵Ǹ˵˲ǸľһΣʲôûüءģûȾϲǸɸɾġȪ˵һڹҵ

Сңʲôˣ
Ȫлһ߲ʵϣѡ
Сңʲôѣ
ȪǰסҼҡ鷳ӵ绰
СңҼû˵ģҾǰ԰ʦ̵һʡr(s_t)q

ڳľӼһ°
Ȫĵĵһʮӡ
ڵˣӼιǸõˡ
Ȫû¡
Ӽղ˵˭
ȪĵıѾɹˡ

һҳֽĺ벻ϱֻʣһšȪָͣĽŵţϣѵǸ˵ûլΣҪ״̬ٲš
ȪֻȴǸղŲĺ롪ոձѹסԸҵ̬ƾ
ιȪѾû˶еľϲʧһ㱡ɴ֮£͵ŶԷ𿪡
ôֹҵ绰Ҵ˺þöռߡĴˣҲøҳȥֲ˵Ǹ˵˲ǸľһΣʲôûüءģûȾϲǸɸɾġ
ȪŭϷıӺǿҵٲͻҿûꣿĺδ޷ݣ
ŭpiaϵ绰ڼеȪһתͿλһˮҪԼ࣬ǱȪһ̾תˡԼĿλͬѧСţڵѾôôÿк̲ݣҿϴѧꡣ

ٳȦ
ڲͨһ롣
ιҽ򡣡Ѿûһ˿
ʲôˣ
Ҿţǻþɣѣ¡
ѣʲôѣƤƤ͸ղСҵǱŮһʦµĨr(s_t)q
һŶһǸСкȪս˵绰ˡǰסҼҡ鷳ӵ绰м¡
Ҽҡ
ְֲ𣿡ȪҵƤƤİְr(s_t)q
ǡڴ󻵵ҡ
֪绰𣿡
㵽ʲô飿ҿɲĺ롣
ţиѩ˵ĻҪѩ˾ҪμӵģҲϵ򡣡
ƤƤúܿףͰѴ󻵵լŸȪ
ֲøղҲȪŸոռ¼µı룬С붼λأһΪ˶Ĵ߻绰ͰֻźլŶ˨r(s_t)q

ι˲ڼң
ġȪ
Ŷʲô飿
ֱ˵Լǲǲ̫Ŷûʲô飬Ǻþò˴绰
û飿ȹˣ¡ཡ
ȪɵˣʯͷʱĺʹҲֺ
ղλˮͼͬѧԹϰȪվڴڡ
Ů˦ˡ
ѳɣ
ǴֻܿһС죬ȥġ
ŶŶ
ôеʧ
СҲûôѪǱߵݲǿͨȥ𣿡r(s_t)q

극ǰҴĶŵź˳극ӡϴ롢˲鿴ʱ䵱ͯ˵ʱ䵱Ħʦ֮󣬴󻵵ҲŴȥ˱꣬ȻҪվſڡ
Сݮ࣬ҸڱѩʱѾ൱ܰѳֵסˣһ߸Ŷš
Сݮһֺߵ׷Ҹеѹôr(s_t)q
ӣһչûУ
СݮҲǡѩԽԽ۹ˡֻ֮䣬һֱ񽻡
Ҫͱˣ
СݮٵȵȰɡ
չҸࡣϴûȥѩ꣬źˡ
Сݮһŷȥ
ۿƷǼƷСݮȰɣֵ̫ˡ

ԣ߱߳ѩ⡣Եúܿ죬ѸȴźۼڿЩҲʱ͵飬һĽҲ㣬ʳڣ쿪𣿡
̧ͷգ۾ʲôĵģ
ҺȻеΣԴǸ𣿡
(o)Ŷѵһѩ⡣
ôãҾúܶӣ˸߸ϵ̬ĳ䵱ֽϻ˰Ħ𣿡
ԼڵĴΪѨr(s_t)q
ļܳʵ𣿡
˺ôľģƺܴŵ˵ͷ

ôܲأ
ڴϣĥոյı֡
һ˿ϸţëһɾĵ׵սľһ¡ѵ˵ֻԼĸˣ
ǣ۸򣬰ǸŹԼĸеܿİ
Ӧòô˼ɣӦòԼİɣԼڱްɣ
Խ룬ԽԽȣҲ޼¡
¥סĿͷſڣûţžʮ־ڼ龫ſˡ
üֲţΣգһɫǽô죿r(s_t)q
ķûƣ¹кڴϣ˯ܹԡ
ҵãյúܽ
ԼԽԽˡϷһڿеҸ顣΢΢ţ촽¶ˮеõ廨꣬ӯۣСɵıͷƺ΢⣬˰ȥ⻬ϸ廹˿˿ҡı
ţȥˡ
һƷζԼָĿο֪ԼһЦͬ˶ʮͷһεõǵĹﺬһЦʱĺɵ顣
۽ëôôڣô۽ǵطĽë΢΢̡İĸȥŽëĵ΢΢ĵһƿմˮ㲻ϵķӿűݣȻžžըѣƤ
ջ֣ԾİԼ촽ϣⴥһĵļһͷĵ𡪡͵͵һӦûɡС˯ôӦòᷢ
ǸղŶ¥ûд
ͬΣԽʱصĴ
ţԽԽˣֱͷǵĦһ
ñ۾촽
Ϊʲôôأ
ǣҵ͸ӯӯЦ⡣
£ǿҵЦһƤʹػˡ
ҵ۾Ϊ˸еߣűպ۽ëļĵһá
ÿһ񾭶֮衣ԼƮ뾫顣ҵͬƶûҲûʱʶĽ˯С
׷һֻʵܣܹСɽܹ̲ݵأһֱܵѩԽԽߣͯŲӿǸʵԲһƬããԽԽСͻȻžһըˡ
ҾѣŷԼ򡣺ûѡ춼ˣԼŸе䡣
һָ֪ľ촽
ǣٻԼ촽ʱŷԼ촽촽һˣ
̧촽Ϊճܵʹ
СʱƤڶȥžֺ
ץŴ࣬εĸţ촽Ŭֲ̫ק˿ôͲźèr(s_t)q
һ촽֮䶳˱㣬ڻͺˡȥŷֿһĴꡣ
ţŶٶ۵СƬĲֿĴ֮仮ȥ
ۣ
Ʈ
ʱղ¥ϵǵŵŵ¥ڿHAPPYĻǸϽҵ塣˾ְٸҵСɫһƣŰҽѡ
ҲʶַЧİ촽ֿ
رڿ졪ǰδеĸƴš
ðɡһģзִʹ˼ڳɹֿƬԹŮֻǣԼ촽˨r(s_t)q

Ҫ촽ڼȥϰɣ
ǣҴ󻵵˭һĿбӵĴ߽
Ե͵һӴʦĿȻˡٵ½ڲһС
Ӵʦ촽by򣡡
Ӵʦ촽by򣡡
Ӵʦ촽by򣡡
飬ĲƽһЩǾȻûߨr(s_t)qЩϰôô
òСײžŵ
죡ߣü۵ĳСӳһ䡣
С׵ϰ࿨ûˢֱӱԼǰ
Ȼÿһˣ˹һٶ׼ķǶԼλϣڵǰϴһſҾҵҵӣ֪JQ
һ°ȥϲף˼Ӵʦ촽by
·˼ףѵӴʦǿˣ(t_s)#
·ңӦþǰ򷢵ϢѰɡ
·˼ףӴʦ˭ǿˣ
·˱ҹĿǰܵXַ⡣
һ°ȥ򣡰򣡰
ƻ͹ˣףҪᣬֻҵ׷ȨѨr(s_t)q
·˶ʦ⺷ܱҧ촽ѵ
·˼ףѵС
·ңǾǡһֱûӴʦʵgay뷨
·˱ҪҴ绰\(^o^)/

һЭϴ»һѵõķ
Ϻʲô£ּǲˣ
졪졪
Ŷã˵װôҲϴ»淭ţr(s_t)q
졪졪
ҹʲôûǼӵȪĵ绰
ʪˣǾ˦˦ɣ˵Ŀװӽ˦ͰȻȥ绰ˡ

ѾϿΰͷˣѧ߽ҡ
̨ϵʦȻ治ɫһаͷҪĩϰѽܵĳ趼һ(t_s)#
һѧǰŽȻһ·⵽ȺۼӲǿһͷλãȻɫһͬѧһλ
ߣ̣ʦһֻ۱ʡοԾѧص㣡>O<
ͻȻһͬѧֻ
ðɣΪҽͱһΣʦȻתŭĿԲҪڣȴǿȪóֻô᣿Ȫģģ
ȪϲĿ˵ֻιιıȥʦ۾һһ;ƿݡѵôʧҽþôûȤζѵҪǸԼһֱӵXʦѧѧϿνЦ̫Լļֵˣ%>_<%

ι򡣡
šҺæûʱص绰
û¡ڸʲôأ
ϴ·
ϴ·˽ϴ·Ȫŭˡ
šŻͲƮϴ»Կһ£װѾһͰ͵ĲϨr(s_t)q
˴𣿡
ţ˼һ£ǰô᲻˼˵ãǰÿһ˶һҲ˵áѵõľˡ
㲻£һġ
ʲô
Ըˣ
áˡҹûӡḻʡԼܲ״̬ɫĿӺĶ쵽һˣһ·÷ḻʣ
ȪȻģԼҼһ棬Ҳ֪r(s_t)q
ʵԼȪǲκȥչġʱûΣ档
ŶҪʾһ˲С
ԣԼԸĶ𣿡
šԼԼȥ浱ȻO(_)O

һ°ȥϲף绰ռߣ

part12

ѩˡ
ĵڶѩ
Ǹò񻰡
ڰ칫ˣҲò񻰡ٶˮٶˮٶಹˮĤױһȥ촽ǸƤ
ƻ͹ˣܾȱ˵ٶĻƷûдǸؼһ㡣񣬾񣬸һֲｽˮ˺ܶܶ࣬ˮûϡ
ҹȥǳСѼ·˼׵־ףͬ⡣
ҹȥëë棨·ҵĸףͬ+1
һ°ȥӴʦƴա
ļϴС
ƻ͹ˣϣŲҪװ칫(t_s)#
ҹȥǳСѼϣҲǱӰ(t_s)#
ҹȥŮˣ·˱ףٻѱȥ
ļϴO(_)O
ƻ͹ˣѽš̫̫һҪ˵꣬ǣ̫
Ȼֻ͵ĵƬǰ죬ĵʮӴʦϣ΢࣬һֻ첲Ű칫΢ȭǶ򦣬ʮȽƫ̬˼󡣿ĨһůůǳɫϦĿƬֻܿһǣ΢΢ƺ۵ĵطԲ΢΢΢΢̡
ƻ͹ˣӴʦ촽ôôôҲ֪ôݣ̺һӴʦһǱ˴Ҫһ㣡һģҪƬɱֻֽû˵ʱ͵͵һҲáˣҳ׵׸˨r(s_t)q
ҹȥǳСѼ˱ÿ춼ܺǡ(@^_^@)
ҹȥëë棺+1
ֵ˱٣

С·ϡЬײѩ֨֨졣Ҫһҡ
ûӦղĵЩĪϲó԰л
û͵뷨ǽŲԾĺᣬϢҲźܻܻ۾ȴԽԽСһѩ裡
ǳһİĸ裬ȻͬȻбеķˮŵ貽죬źǿĽС裬úרģҲܡƯ
ĺڷ鶯Ʈ裬ӯ͸Ĺݴ˿̸һרעɣֱչԾӵİɫȹ˿ҲͬȹӱǼϥĳȣ貽ƤĲȵõʮֵǡôȻǴɫȹ볤ͲĦȴ˿񶯵
ࡣԼʿˮô᣿ԼҲһȵǣͺǶͣ
ѵ˨r(s_t)q
ĳ䵱ֽϻżȻ·Ⱑû˵ڣѾ˵ڣû˺ǵĻС
۾۾
ղűΧмǸѩѾһСкӡ
ˡһĺ
һ㶣ŷֲڽԼǸոձѩˡ
ˡСкԲ̵ͦĶӣֻСѸٶĒһֻڳεС꣬˲εߵߡ
ˣô˶ûзӦѵǸϣСкգŴ۾һңȵòͷˣôŰɫСѥӵĽѾڵĴƤЬͷ
Ҳˡ
ţһ۾ͿˣҪҴкӡ
УεĴ۾ղСһԱôѩˣᵥԼɣô˥
ˡеģĻиղСӡ
С۾ʱĶû뵽˻ˣ¶ʮֹȵΧߣҵװӣֵĽҡˡ
Դӵ֮ҾͶԡϡеС
ˣǶСѩˣճã̫¡ԽԽСΪǸСѩѾҵͷˡԣ˼䳤ʶģһҹͻȥġ
ǲϲСģ䲻ϲûһͣСޣҲǣͷϴӣȴһ΢磬ѸղʱµЩ˼
ģֻһҹ
š۾͵͵͵һ£һһСҵļϨr(s_t)q
ҰϵĺӶڵϣһǶʲô֣
ѩ
ѩ
Сѩ
أָָǸС㡣
ҽԶۡ
Ŷ°ֲͣƫƫԶԼ
ˣҶˡԶӣѩѩСѩҲ̱ʾͬС
ԣǶҪѩ⣿ĵɨһۣѩ˻Ƕ࣬˾ͿʼֳԵġ
ѩʲô(o)
ˣǻûԹءǳʲôԺͻϲʲôˡ΢΢Ľһ¡
ҵȻûȤȥǳʲôʹһдӼѩˡ
ǶſվšҴһѩ꣬˵꣬͵ͷ˿ѩѩСѩѵּˣôһ
춣r(s_t)q
Զ۽ȥˣҲҪȥ
Сݮеһ紵Լôһ硣ϻžӨıһСΣСݮͿ߽š
⣡Сݮεıܣһ±ӣ΢üĿһۣСݮĶˡ
ҵͺһ
СݮҪĴָ̨߳е硪ѩ׵СëͷӻƮ۵ڱԱߵΧ
ѩ⡣СëͷǶˡ
ҲҪҲҪ
ұĲͷһСݮôסˣຢӵĶô죡ԭѾмҵˣոձǿٻصǵС͵͵˳
Ұһѩŵ̨ϡ
СݮѾң˫˼òץɨǡɨ˺ü£űһͬ۹ײİǵãеһѩʱжꡣ²׷һξӨҺ
һѩ׵־ݵԼǰСݮ̧һһСëͷֱۡôСӣԼȻһֱŹǵĸףСݮѵãûޡ
š֪㲻𣿡ѩСëͷµġ
Сݮ֪ķֱǼȻе
ðɡСݮɨǣӹ־
㣬ѩ⡣ѩԲ۾Сݮ־Ų˨r(s_t)q
ѩ⣿Сݮĺ©һġ
ǣ˿йص飬Ҫ£ܵСݮû׸λ־ѵĲֻұԼ˵С

ۣóԣĸСëͷųһھķпȵٷпʱѩѾһʣˡ
ѩ˶ôθ𣿡ҿһԾִٲ¶һ˿ײ΢΢΢΢Ц
һ¿յֽϾ֡ĶˣԼûبr(s_t)q
ĸСëͷѩ⾫ˣҵĶС¥﷭Ϸ£ҺͰվһ¥
ʱĻ𣿴ԶѾ¥ʱȷʵĻĬĬעͷҿʱֻһ˲
΢΢ͷֱһõɿѩƵĳԼ۵ס
̧ͷӵȴʧЦ⡣
ˣ۾ӰһƬǿա
šҰѩŵУůĴָͷ˿Сָͷ໥Ѻõ꣬Ŀ֡
ˣ
šûϲԼˡҪлԼֻţƿ˿һѾʪյСͷԼˡ
ѩ𣿡鲻ڹ˵ﻰr(s_t)q
ҰϸĲڱֻѩó֮Ѿڳ˳һҹ̾;øղűɿĸСëͷֱҪʮƤƤͰְ̼ҹҹƤƤһܵ磩
վһ¥ҿƵĺһ䡣ĸСëͷϽ֪Ȥɳվһ
һָķ䣬ڣȥ˯
ô˯ĸСëͷʮֲԸҲηңƤƤĥĥĶķ䡣
ٿһ򣬡ȥϴˮ
ǡԱˣֻѩĵȥԡO(_)O
ԡҳʱӺܰҴԡ۵ķһһѩˣ
ƿţŶһ£԰ͷг·Ҵ⼸Сץ
߽԰һѩžͷ˹ҵҵĶͷϣѩѩ黺Ļҵĺ
ҸҪһѩַ˹ҵˡǸֻͷԿĹؼλ
һ˼r(s_t)q
ԶƮ˹ˣҰǣԼղѩʱĲ֪˰ֻ߸ѩѩСѩˡ
û׿ϣԶ۾˼ѩųѩԴȥԶƣУɹѴѩһע˹
ˣ㡣ԶʮֹȵŰһѩС
ҵѻĴѩѩŵˣ̺ۣ·ͷۣСȻһֽϻ鵯С
Ÿջĳλһεʹ֮С
˴ſ£ѩʤĴѩСѩ˿Ҳ˿˵ʵܺ۸ǡҳһѩˡԶһֱҳϵվֻ߰ÿڵͶ㵽
һѩ˳ҵͯڱˡʵͯҲк֣ܶͯҾǸСӡ
ߺߡ⼸СëͷôԼĶ֣ҰѴѩѩСѩýڽڰˣԶʱĸߺ꣡ҳɹһͯңòֺͷð
ˣаɣѩѩСѩ㵽ȹԴסˣ廹ǲϱҺԶӹѩС
ˣǰδеļ㡣ѩѩСѩͬʱеͷһ˵ȹڷķϡĶͷŴսڿʼˡ
ĺ̵ģռϷʱ򻵻롣
ǣܿģҷ֣Լ͹ˡݵѩˮ֮㡢֮࣬֫չˮµǣһֻĻˣԭزͣĵжյУ΢ͣЪһ̣ȹڻͣϥͷȥ
ȻŶĹףҳڱŷḻѩս飬֮彡ųеѪڡͶ롣΢ͣЪһ̣Զۻͣϥͷƹƹȥ
ԽԽѩ𽥱ѩڼս֮ѩһѩ
ұϾˮ࣬Խ죬ʹ˸ͷѹ¡ȻĸСëͷץȥĨr(s_t)q
Ϊ˫Һ۵˴˺İһ
ҺˮܣƤң׷տ丸ңպ࣬ᱡȴ촽ȴ뵽һЩ޵ξ
촽СɵıΪĻ̣ͦ۾е뵽ֻԼ۸СԼʱ޹ӣ͸뵽ֻСͷԼʱĸоӣҺˮ촽뵽͵ǡ
ࡱ
˶ı۾
ѩĸСëͷ˵ͷѩ
ѩƮ˲䣬ұ̧ܵͷȥ
ת۾ѩͷ˲䣬ȻˡνëգգҲΪˡ
ٵͷʱɵɵӣһʣôˣ
ÿһĻش
̲ס죬ЦôůƻҶ죬ȻᵽЩһֱ뿴ļڡ
ĸСëͷڷ˹ѩr(s_t)q
һվ·ţ
ֱЦ¥¥ºܰ
߽ķ䣬ڹԹԵ˯ĸСëͷȴˡ
ѵһΣ
ҾԼͬڴʱգðһЩõġ΢΢־ԤС
ףҳţ߽԰ѩĵҵĽӡرԼĴӡЩ໥ǵСӡиۡ
ֻܴһ졣
̫һǾڻˡ
ĻԺһ
ҷ·ۿһѩ׵ĺͬܲסµˣŤΣֱһ̲ˮ
һλͷͼЩµͷͷԣһԲˡ
Ϊʲôڻ
ǲʥҹģԲ˼죬̫ա
Ժ󻹻𣿡
ţһ
ѹ𣿡
ᣬúܿġֻһҹ֪ġ
أ
ţ
ڻ𣿡
ʱҲ֪İ
Ҳ֪飡ԼȻΪһֱԼһأ
ˣ㻹𣿡
ˡ
ɫصĿֻ߽ܺԼȹһ죬Լȴһֱ۸ҺȻԼĻ򣡲֣ۡӦĻ쵰Ļ쵰
ƮʱͿһӣˣˣ
ţһعһҪ˵ı顣
ϴǸȪѣԼҳȥ棬𣿡
дһ˿ӴеĸС԰Ҹͨ绰һ¾С
򡪡˽˵Ҫϴ·Ҳһ˵˰O(_)O

˰칫ңҲŷȥ϶˼Ҳûֻýȥá
ԸԳһؽ㣬ÿһסñӡΧܸܵΧĿ⣬ھϴ󶼳Ϊˡ
ȫ˵ϣеѹܴüͷߡ
һŵʱ򣬰ӵǼ̨ĸ߳氢һӣǰδе򶨣Ѿ㰴ˣȥأ

ҼҵʱƤƤҼ緹㵱ȥϴ˨r(s_t)q
ƽһºŰ塣
˭ѽƤƤк׷ŻͲʡ
ңңҡҲƤƤСһ̫ˣȻˡ
ʲôҴ󻵵ڼҡɡ
òϵġ
ʱ绰ˣƤƤȥӣҴ󻵵
ƤƤϣš
۾ɵԲԲģòգһ¡ôˣ
ɣӦòᣡԲᣡˮеƤһԼͷеͷǸŵĶ̫ˣӹϣΪίĿ˿ڣСѣְʲôİ
ƤƤĵɹ
ԼԼְֵôʣ
ʵƤƤֻǶԼĳνѡ
Ұְ֣Ǹ㷨ɵġƤƤİְǴѧķɽڡ
Ŷټ˿̨רr(s_t)q

һȥֱ֪׷ףҸոտӴʦĶbyΣ
ƻ͹ˣѩnb
һȥô󣡣
ڲˡ
ҴȥзȻٲҲ֡ôҶڵǰ治أ
ȥԷ𣿡
鱻һˡ
ӹŹּˡҾԼڿޡˣżǻߡĲӣŵȵȥˣ·ǰ̨ʱȻֱĶһ
ƻ͹ˣꣿӴʦǻûҵѾְѩnb
ҹȥëë棺ջعӴʦgayһۡ
ҹȥŮˣӴʦȻǿ
ҹȥǳСѼҾǿӴʦŮѨr(s_t)q

part13

Ȫһͳȥˡ
ڰ칫ܾòġ
쵽ʱˣ߰ɡϴңаҪͥ
ôˣЩŶϴʮڲԹɫѹίҲֻҪʧˡ
ġšһ۾ĿĵĿŴ⡣
ʻһխĵеչ䣬ҾͿ˳
ͣ
Ҽ󲽿絽ߡôˣ
ϣĸаСȪΧţһãһúеС쿪ҡҸû˦ֻǶһ£ͰĿȪ
Ҹղ·һ¡ǴթȪҲ첱Ӵ֣ȺС콲˰ҲûãҪǹ˼Լ͸ƴһƴˣȻԼܴ򲻹r(s_t)q
ϣײСүң˵ҶթǸпǸСͷĿʵʵͦˣⲻˣ椿ӰôͲˣX𣬹ô࣡
һعһĿԼ˵ĴУȭͷʮֻķŵеǰ
XĸסҲȭͷʹȫҵȭһ¡XӲţ漴°;ȲһֱʡӲӲȻԸԳᵽˡĺӲ
СǶˣʮ˼»ӸӰȺԱƵģ¾ͶҵȭڵϣֵĶܵˡ
ҿϳصûȶء
ɨȪʱұڵŭȴˡȪɾįѹʶı飬ҺȻԼȴˡûȶأء
СȥȪԭأöûжһ¡

ϴڸʻλϣһΪźۼͨ͵ںҺ
ֻһࡣӲϴ۵ǣӿȷԼŮѸкֱԼײˣ˭˭Ҳհ
ٿ΢΢ͷϴòҲ֪˿սľڹǿΪһȷ籩׼š
Һʲôû˵ɰһֱϷȥϴ۹Ǻr(s_t)q
ҰѰӵóֹһ飬ʮ֪ȤİԼ
ҰѿĵݸպһûģȻճƣǷĬȷɵȵĻͿپ׼
ϴ󡪡ʹݴ˱ضJQ

Ҳŵ˷ͥڹϯϣۿԷʦ֮սľȻԭġȻǰλдĹ£˾λӢսʿ塣Ȼⳡþ״žһķѣûĸ߶Ȩr(s_t)q
ղڴСʱ򣬶ûڵߴĿ˼ȥˡᶨӡרעһһͶ㣬ֵ˵ЩϹԷ绤©ϬԴǡ
ιĸѧģԱһλͲסʼһ˵͵͵ģһ
ΪܵţĿһۡûжԱôûжԱ˲ֻǣһ̣˴ŵܵĲ˲
˭Ӯûϸ۾ڲ֪εҲԭӮ
ԭ棿ϸ˿ƣǱһġ
ⳡ˾ֻӮʲô˼ǰһŮۣüĻعͷӮ岢󣬹ؼʵʵ⳥һǮҲ䣬ҵ˵ʲô壿
ô˵Ŀⷽһģ򲻽Щġǣ̨ϵѻĶԷʦûм֮˰

ҪþʱҲ
ԭʤߵʱеѹѹѹᣬѹûѩȴֳԲ¶ʱĸо⳥ʤһʧһϲûȥýȥһֿýȥһ
򣿡ԣ쳣ôˣΪȪ棬𣿡
ڸʻϴصĺһʦԭôͻݵһ棿
ǡΪղ˾ѹ
ǰɣϴˡֹ˾˭ҲȡʤίзѾĽϲˡԭӷ˶ʦҪôϵģr(s_t)q

˾ָʾȰϴͻӴãȥӷ˻ׯ԰
ڵӴõʱ򣬰ϴϸУӳһСë
˾װģƿǰĥĥļһֻԼҪõǰ̨O(_)O
ʦİ
ϵط
Ƭ̣򡢰һ˶ɵġ˳⼸״
ʵͲµӣ򿿽һЩΪʲô᲻ñأҪһԼӵС

ԴԺڷͥϱ绤ӾǳԺСϴ·ʱ򣬲Ьӵʱ򣬸˺ѩʱǸӰܻĵĴӼ
ңַ˵ĺöʵ˧ǰôû֣ܲӡͷӡļӣǣ˾ӣôÿһоһɫԱNಿӾȴһֱ²𣬵ȵڳ֮ЩɾɵӰ񶼱˳һƷζһֱϲ٣Щľ缯
ҵȻҲԼó裬ǱԼСʵĵ⡣ɨʱ򣬴Сھ̨һ¥ֻһۣ͸Ͻͷ͵ģһǽʹĴɨ
űҵ죬һƤƤСӡ
߹ȥŴһƤƤԲԴխôһ׵ķ졣
Ҵ󻵵ˣåƤƤҵСڣź󺰡áɡƤƤǰ·ǱŮŭķɱ˭ҶҼƤƤˣå
ǱŮϵȭس࣬ŵŵ̨ףһȴķɽžߵϡһű߿ˣƤƤû룬ǱŮǣֻȴСŶҵϲȹղŻɸɾСӡr(s_t)q

ŽʲôƤƤĿǱ裬ϵľɷ䲻ͨСϱһǱ̵˸ղŻĶԲա
ܵСëͷƮʧֳȥˡ
ϸСץҽʵĸ첲ץ뱻ץλееҺ˭Ҳվ
ƤƤ⵽Ҵ󻵵ߣСϻЬͷҵĴţ
ǰæɡǱ뽫
ҲãҪȥ̫׹ػˡ
ǣ˫ҵץȥģмСȿϣԼҲ֮ǳҡڶص£Ҹеһһ˼Сڲظı˭ץˡ
Ǳ辪ֵļr(s_t)q

Сϴ»ֵķš
ϴ»Եϣϸ龰
ʵʲôҲûΪع֮ѸٵİƤƤץ˹߲ǰȻ߽ԡˣȻŵ˲䣬ְƤƤ˳r(s_t)q
˺ú߰(@^_^@)
СЦԲĹĵġȵϴ»֮ͣʪյС̳ٶ˦Ͱšء
趯Сڡ̫̼ˡ
ѷɨһ飬һӲʮ㡣˻Ҫ˸Сʱܻبr(s_t)q

ǼСˡ
һǼĴˡ
ǣЩĸı䣬΢΢΢΢͡
˭һҴǣʲôҹأ
ǷݲҴȴû֮ɸң෴ʹ˵͹ϵǢһ֡
һؼҵʱԽԽ磬Ǿÿհ׵ʱԽԽ

һеĴɨһǲһ¥ǿĴȴһۺۡΪƤƤϲſr(s_t)q
Ҳ棬档ϵĨ׷ϵĨܡ˵϶ŶԳƵЦ
ƤƤվԼϣԶһо۵ףȻˡ
ǱվԼķϣõĸ߾׼Զ۲ƤƤϵÿһ˿įмһ˿˿ʤۡ˵ܵʱױ£һȥԼƤƤȥɢO(_)O
ʵһڹ۲죬ֻ򼸲ԶĵطȰͶ˳ȫȻûзȪɫ԰ף˶һֻįľ飬ĵ뿪

ʦŮӵЩɳƣٵһˡ
ȪŴ˵лعʶԼҽ̣ŮѾᡣ
öף
šŮӲ۾Һϲһǲֻ̫ѹʱóһ£Ϊһ뵽Ĺ£ԼѹԵЩ΢ˡ
ǰԼʼҲôġǣٵʱԼʹȴӵʹһڳγİ֮гȥ
ʦǲ£
šȪƵӦһǽϿΰɡ
ôϰʦҲģҲģͽвʦ˵òãһЩġ
áȪҡҡͷԼľǰĺãֱõϡǣǸǲԼʲôԼԼİְ֡
ŮȴȪһԸ񣬼Լ뷨ʦ˵ɡΪҾȰҵ˵
Ȫû˵ڡá֣ŮѾԼ顪μҰְֵĻ񡣡
ҺϲҰְ֣ӣΪðְֿģһȥ˰Ϊӳȥ
ܶ¡
ûô¡ֻǲҰְѹѡܾ뿪ҶԲңôֻġ˵˵ɡ
ϿΡ
ʦұr(s_t)q

Ϊðְֿģһȥ˰仰һճȪϡ
Ȫڻȥ·ϣҼҵʱ򣬸պôӲпҺԴ˿ˡҴ󻵵һֻ͵źüƣ򻹺֪һרעԼڿ
ΪģһǲҪŵĺáͷȪ̵Ʈҡһ£ԼһôЩֻҪҸҾ˳Ĺ
ԴϴС֮ȪΪԼûܺܺõı׵ԱûչʱȪֱعСʱǸʲôҳԵС棬ʲôҲҲκˡѾ˺٣ֻһˣù˼Լڱ˿ǷǷϡ
Ƿϵʱ򣬵ùҲ˼ҳ㷹
մʳ˷ˣһţоͿ˵ϡ
ص¥ٵһλ°̣ҪȪʾ֤ܽѧ֤˵ʲôҲҲˡʴҳȪָĶţȴĿġ
ҵ𣿡õطʡ
ˣҪѧ֤Ȫָͣڼ﷭Ҳˡ֪ȥˡȪãȻ̧ͷϰ̡֪
ȪϣתߡŲҲûо
ôô׾ͷˣͬѧҪƤһġŬŬ죬ι¥𣿡
Ȫعͷ΢΢ĵһ¡
ʲô֣
Ȫ
ȵȣҲһ¡̷޵ѧܹûмȪ֣ٿԱߵƬȪãҪϲƬϵˡƬϵСӶгôôûо񣿡
Ȫ¥ıӰ̻ڿƬ̾Ϣ϶ǵĻr(s_t)q
߽ңûпƣںڰֻϵһҹڷӨ׵Ĺâ뿪ʱ͸ԼдšƯ֡Ȫһ£Сʱ򲻿ʱϱĴȪԾı۾
绰ǰְ֡
֡
СȪƱ𣿡
û
ɣ۳Ҫˣ϶򡣡
š
ôˣġڰְȪһֱСʱı棬ɰ
һȥ
ǲǽŮˣ᲻ûְůЦ⣬һǡ
ҿˡ
ʲôְֵȫû˸ղŵϲôܣ
ҽǸСһ˼
ϳ𣿡
š
Сʱһְֵһ˿ײĵġ
šӻûб仯
Ϊʲôģ
¸ҵкӣ䲻
˵ģ
ǣԼá
һζݵĳĬְֲֿڣԼķķŬˣֻҪͲʲôɵġ

ȪѾڲٳʮȦ
ǵʮȦ
ţʮһȦİ
˵ûָָͷΧͻ⡣
ƤƤֲŶŻͷЬСѾ߱߿ܵԱ߸ǱŮ
Ǳ΢΢ͷǵһԼŶ̸ôѧr(s_t)q˵ƤƤΪѧҵO(_)O
ˣһѧòãǱ˵Լΰ뱧
˵ɡƤƤС˿ǿգòҪŮʲô֤
ȪһȦԾƤƤȪıӰʼ;еϤأ
Ǳ衪ƤƤ˼ı˧O(_)O

part14

걻װһ£ϲӭꡣСݮһװվ̨֮
һλ˾̡
ã괫ͳУĸ⣬Ҫ𣿡
ˣһû顣˾˼Ĵ֡
õģСƱպãллݹˡ
춣˿ͳȥСݮһЦָ̿ڿڴһɿ۾üͷɿӽ·ڳҩһ㡣˵ԽǣΨп·r(s_t)qǣô죿˵ֻҪһϲˣͻжΪʵġ
˵һŮ˵ĳ̶ֳϷӳİ̡ڵ绰ȵܽ˵
ô񣿡Сݮﺬɿһ˱֣˵ʧһڱݣ˵׷ˣЩ֮һôһ£
˵ͷŮ˺ŮҲǲһͺֵоŮ˻ᷴùɣûʱ򣬾Ѿʼ׼ˣȵɹ֮
ߡʧеСݮںϲ̧ܣҲô˵ĻϹˣ˵񲻸
˵һŮ˽мƷŮˣʲôʱһ˵ģԲӦ䡣
XXԱôַأ
һXXԱеĸ˦ˣ˵֮ϵİվǿסġr(s_t)q
ˣˡ͸Сݮڹ·
ɿڣ죬ǣ¶˿۾ҪĴЦ⣬۾һҪЦ⣬̬ȵص㡣
춣
á
Ф÷ͬ־񲻴
Ҳǣװ˧
𣿡ЦȻеСɬҪܽᱨ棬ҾĹһֱϸӰŵչʾϡ
õġ
Сݮ̨߳;վǰơ
춣ҽ°ֱӰѩȥһ뵽ѩʱĿı飬ĵȻһĨĿ⡣
Ұæ¿š
黹ҾͶָæ˵࣬һżܰʽĹա
ϵСݮЦЩʧΪʲôڻ涨һ˲䣬ЩѾһȥأﶼеᣬΪһʼֻԼļİ
ÿĻס̾Ф÷ͬ־㿴ЧǲǺܰ㿴ϵģһҲ
СݮǫĿһۣۿĺްԭԼյôں컹Ǵһ¡
ÿ˾ᣬ̨ݸĿˣг齱
СûôÿСݮԼҪһں·ߣһҪǫһ£ؼǾҳĵúã˭վ߶СݮԼǲ̫óǫ飬ǣ˵㿴ܸ󣬷𣬻ҪЩĵĸƸ𣿡˵꣬СݮԼȿʼЦ
ˣˣؼǲͣһһģòűһ΢͵Ļ˭˭˵ҽˣ˼￴񴿽X˨q(s^t)r
ίЩ΢΢ġɰʵ겻󣬲ˣÿѪĻͶԼĵ͵һݣ֮Բ׳׳ģر΢΢΢΢͹Сӣ뵽ӼҺɷĳƺš
û飿ôСˣ
Сӣѵ˵Ӥ𣿡
ðɣЦǵˮɱСݮò֪ںδr(s_t)q


һ˻O(_)O
ȥţȴȪ
۾ϨϲãӹеȪôˣҺЩ
Ϊ˻ˡʵ˵һ߰Ȫýӡ
˻û˵仰Ҿͻˡ
ѩ⡣
á
ȪſڴĻԻܲǷݻ죬˶ųֻ֮Ļ졣ҲȷõĻЬŷɳϵȪ
ҸոˡȪòüĿ֮İȴôҲβס
Ҳ罻ˣ˵Ȫĵطͷ·ܵȪܿԼһ˵Ͳһô䵭
ȪͻϼңԼȥ
԰ҵͷԼҲÿ춼ڼֻǣעⰲȫ
šȪ˵Ҫ˵Ļ׼ˣһ򡣡Ȼ󣬾ͳŶȥ
򡪡Ϊʲôûҵ(o)

㲻ܺһӡѩˣǲͬࡣʮ꣬ʱʮ꣬ܶʲôаǰҲк֣ܶÿһֶҪһһ𡣶ңһ죬ͻصȥһֱȪܵİְ֣Ȫ˺ܾáְ֪Ȫϲǰһֵ֧̬ȣٴγȴ
ҶǣǾֿûнƺҪ飬Ҿôˣһ񲻾ʮֱ꣬ӵʮʴӡڰְֵĲ֮£ȪҲԼ뷨ú峺
˵ĻԸɡǼңԵֻ֣ҪԵֻͷͺˣеһҪһЩʽϵĳһιȥһ
һЩʽϵĳһιȥȪͰ̸ְյ㡣Ŀͷľһյ㣿ȪһְֵĻҲǣǣϣܼ򵥵ľͰհְ˵ȥ٣ٸԼһλᣡ
ֻλᣬ⾫İŵһ죬ȴĳΪһʽҲϴγʱô˷ˣ۾ֻ70%
ǲǾģ
ţʵԼҲ֪ôһ£ӦǺܺóԵģӰҲӦǺܾʵģǴѵָоģڼµʱҲָо
û˵ʲôȪҲˣһждϡؼҰɡ
ǣܲɵǡȻϴС졣
ţǾȻϽ֣Сүˡǿôһ̣зſɶ۾ǿǺܳʱˡʵǸһˣȷϴǸȭڣŴŵǰr(s_t)q
Ȫ˿һ۵

СҲû뵽ô
Ȫĸ첲ˣеĸ첲ҲˡصǣСȪؽһСֿ
㲻ǰܰС׺ܵģһη
ɣûҪǮСҲûٵֻҰοǳСõϴ󣬸ϴëˡСӽXĲһӵСҲˡ˵ţǹػĿһԱ߿ǽ˸첲С
С⺢ӵСûˣСôһ˵е˿ڵʹּ˼˵˿ڣ᲻˷磿Խ˵۾ԽȣԼϴ죬ɶôûţȴ˵ӡӣᡪӢŰɣȻ󣬾صĿ
ᡣȣСһеʵҲδr(s_t)q
кͻСڻˡ
ϴϴ󣡡ұϽӭ
ϴղûޣûޣСϴǰ棬סĻ֣ûޣûޣִһ·϶ڶСϵвͷԵСе˨r(s_t)q
һŽŸСͷ˼ƣǣһֻ첲ϵ˴۵졣
Ӻ档ȪڵϣϻŽ
ŽҲ̫ˡССֹţһȪͰͷŤһȥѪȱСŮһ
Ȫϵֻһ
㣬ȥֻùһСס
ΪʲôңС׸ŭҴԣϸôĥĥߵȪߣֻ֣ȴֻƵİ
ˣСֻһĲˣĲҮϴĲҮҾͲˣ÷ˣ

Ӷֻ΢΢üͷȪȥأԼ°࣬ʱӣڶߵˡȻֻһͷΪвоȴһļֻء
һΡᲦε绰ңȻûԼЩˡ

ôˣСһдֻ࣬ҲѶסˣ˺üţŲȪȪŭĿע£óֻɽһݵϴǰ
ӣ
СױǽǣСķֻɤι
˭Ȫ
ȪСظһȪԼֱζ壬ӹĹĵģȴ
ⲻȪֻ𣿡
ڲӵ绰С׿˿ȪĿ⿴˿С
󷢴ȱĽӹ˵绰ιĸ
ҵļܺãһͼǸԼ˵ԭŵ绰ƺȻʹ˺ܴ֪˭
ţк֪ҲеϤ漴ϴܵʹڼŻ˹д˸սʶİѵ绰ˣַ˨r(s_t)qϽֽͨˣղŵˡ
ţŶԲߡ
ǵˣ²ֿźŲá
²ֿ⣿ͷһȴ¶һ˿ڵ²ֿ⣿ɣǰ˹棬ǷǷҪ̵ġ
и˳Ҹ̨˵ǡʱסˣһתϴεĿɲܰ׳ԣû˭ֻ˳ֻҪ˳˳ͳɡ̵XX㳡ĵ²ֿ
ߣղӦ˵ĺưɣйŲӿ˿ΧļұȻѵ绰ӵȪĻ
ǵĽˣҲˡ´ûзǷǿpartyǺúÿ˲ɣ

ڱ⿪Ȫ˲Լ˿ڣȰο
û¡һֱܹԣûܵʲô˺
һܴȥȪ۾ȼС磬СʱҴ𣿡
ţ˴ǰôͯȪġ
˵ǶθҴĻǶλԼҪһҪ˵Źá
ˡǰְֽҽġ
ʲʲôҰ̵ְֽģǡ˯ǰĹأ
Ҳְֽҽġ
Ȫ۾еĻϨˡ뿪ʱûһޣ
ãȻҡҡͷ
ȫ·𶼱ȥˣȪ̱ǽϣ׵ֽ
ҿʲôûҸûСµεĴr(s_t)q

Ƴ̳ٸXX㳡³֮ܵ²ֿ⣬ƽʱܲϢû̫ңվСǰʱ΢΢ŵŭĺ⣬ûʲôŵı֣˵ʮǿҡ
СͰںƣдСҺСվǰ棬С׺СںʧȥȪ
ΪԼռƵλĴУһң̾еˣԸһСסǰѵ椲ϰɡ
ôҰСױİذ׷ŵϣԼԽԽ˨r(s_t)q
ұûѴзŵǰûһǰߣֱɢذ׼ڲϵǰ
ǰذײͣĲɨһʧȪĿȪ첲ϵ˿ʱͣһ¡ЩСң˿һЩǡβܳһ˿ڣ
ûзǷĻεǰֻһpartyһpartyѡأͽбǰӡֻҪҰϴܵȭͷһǾͰӷˡ
˿Ȼţǰذ϶ͣذ׵С׿һ񲡻ߡȭͷӦûʲôϵɣһС˵ľ飬ݺ˵ľأ
ðɡұ˼⣬ɡ
Ƶ̽ͱһȭ
治ۣ㿴˵ûɡǸСʱİҡ
ɡеĶҲһС飬ݺһٰɣϴܵĶֻ
ʹҵһȭνôߣ󣬴жҵСһš
˲Ťһ£ӰϱӼĻ棬ͬú֮ȪۣҺ壬سɱĽ
졪찡С׾еذף֣֣
С׵ļУ˶ҳûȰѴߵڵأȻ·һͿ絽ǰץס˫§Լؿϣܸܵˮ͸ϻ͸Ȼ󶳽ᡣ
СѾŵû˻꣬ܡûǣҲ׷ظղܵ裬ֻıסؿڵˡ

ˡһ߸˿ĺڷһ˵Ƿݱܵᣬʥҹ·ϵ֮ĻﻹҪᡢůֱĵĸȾ
̧ӨȻһ⡣ˣ
š
ûң㲻ᵹù㻹𣿡
ᡣȻ
ĻΪЦ

part15

ǣ֣Ȫĸ첲Ȫһεˣ˫Ŀ񣬽ŸƯ
Ȫ˵̾Ȫ֢ᣬʲôҲû˵
Щ飬ҪԼһȥеһյƣԽ˵ԼڰķǽվҪǣñ˵ȼԼĵо
ڶ죬Ȫµг

ˣûң㲻ᵹùҲ᲻Ҫң
š첻Ƕʹˡ
ûң㲻ᵹù㻹ǻѩ⣿
š
ûңҲҲ᲻ң
š֮ǰҶԼԼġ
ûңҲ㻹ǻѩ⣿
ţɳϵ΢΢šüëԱһڴ򣬡No
ûѩ⣿˵ҪѩԨr(s_t)q
Ҫ˵ûĻŲ
ѩأ
֪ѩ⣡Ϊʲô̸նҪ䵽ôǳһϡѵղûеĸҵһЩڣ㲻񣿡
ǡԼҲ֪Ϊʲôȷһ£ѡ򻶿ȥҪõȥO(_)O

ײף򰢱ϴϴĵһͷ
ǳϦֻа࣬ͻ١࣬ҲûʲôݣһǴǾӭӭ¡ΪÿĹݶ˾մߣڴͳЩáǣİ򣬷һùСʽÿһ˾ýСTAͷˮٶTA˵Ѿˡ㡣
ܹϴ˸оֻӴʦûˡȻõϴ׿ϣǰֻЩСӴʦɲǰЦԼ̫˨r(s_t)q
ˡſİţϽվſڲϣһ׼СĸζҪˡȵӴʦǺͷһ֣һˮӿ
ˣŶˡ򿴵Լʩ¶մӴʦķ˿ֵСҵԲO(_)O
һ֮ͣ΢΢㶣Ϳһ״ǰ˷ܹͷ
ô˵Ҳ̩ɽѸٵĹ˫ȣ쿪׳˫ۣס˰
Աһɿֵ˶ɵˡˮ߸Ь˫ŻϣϰȴӴʦĻңǸǽȼûʱλãͷĳؼλĸŵ㷢б٣ϴȾ΢ķ˿ҵҲӴʦ͸ĳʪͷڵƹ⡣
̫ˡһֻ񱨱ŰµСƣˡ
һȥɡĵԵֻƬ̣ӴʦѾ޹ǵİŵɿˡ
Ůͬ¶һָסüëӺСֻһССķ϶Ȼ󣬽żС
ѭͬĿԼһѵõĶˡʪһƬr(s_t)q
лߣ֪롣ȥ˲һǣ²ȥ֡ӴʦѾˡˡ

ðԼӣǾȻǮˣڴǮֻСڨr(s_t)q
ӣҸ硣ƽҰ칫ңδµʹҪãɩһɡ
ɩӣ
Ůѡ
ŮѣҵȻָ֪򣬱һ£һҲûǸҪ
԰ͷһ°ɡ
ҲáҲͨ˼ĵ绰ȻѾżٳճԼƤƤӵĨr(s_t)q
ҽļôʱƤƤԱһĳšҲҪȥҲҪȥ
ԣpiaֻ֮ǰҲһ䣬ƤƤ
ԱСֹʵ

ͬеѻļеԵӲ̣еڲΰʣе죬ǣҶһͬ㡪һڵȴӴʦŮѳ֡ȻѾˣֻԶԶ͸͵ôۣʵġ
󣨰ˮ֮ףû˭ҿһǰ̨
һǾ꣨ףһ
һޣͬ¼׵־ףС¡
Ҿе顣
һļޣͬҵ¸ףɶ
һӴʦŮѾˣңոôԲ¡
һޣֻ˵̫˨r(s_t)q

˵ɫĿӡ¹ǰˡ˵㶼ǺɫġʵڹⶺO(_)O
һͺƤƤ¹һУҴ󻵵ʲôһ
Ҳԡ˴ʲôô˧ĵһ۵÷Žųȥˡ
ȥءƤƤԹſڡҵ㵽꣬㻹ĻҾֻٹµһˡû뵽մطϱĻͱԼO(_)O

ϴֻӴõǰҪȥ˹¥㡣ȻЩ˷ܣĲʲôأԼ˵ĸо
һúһˣһûˡڵİˡ
ڿϵĲ㣬ֻðȥӿӡȵÿӳʱѾûŮ֮ͬˡ
Ƥôİ
ȹҲƯ
ʵôôǳ̾Ϣؼ⻹ÿԼھ򣬡ǸĸųͲʲôӵİr(s_t)qʵʡӴʦô׷㰡ܵӴʦڽıֻʱʡ
ĵŮͬ¶̾ĵĿģЩ⡣װβǣڳһĮģȥ
ˡϴǰ̨߳ղŰﰢվ̨ţۻᣬ˼ҲԴҾġ
ҾãԼ񲢲ų⡣ֻǣ𣿡
ϴΪԼˣţӦаɡ
Ǿͺá˿ûΪԼʶеһ˿ߣԾһβǵӣһ򣬡ȵҰ칫ɡ
ô˳ϴԲԴѵҵı绤ˡ
ϴ˼ʱɲԲҪҵԴr(s_t)q

˵ĿⶼٹϵӴʦİ칫ľŸסˡ
ľšڽһ챻õһΡſ˨q(s3t)r
ġ
׻һ֣JQ
һǾ꣺զһأ
֣ܿͬñģ¡
һޣһǾ仰̫ˡ
һļޣҺ뿴һӴʦ칫(o)
һޣǾǡյľţô˼˼
һǾ꣺ΪɶҾ̸¼r(s_t)q

űϵ˲䣬պʱ΢˵졣
ǷĲˣЩ£ȴƺЩڴ⵽ôأᷢʲô
߽΢ͷվŰǰ
ſƤЬļ·ʱʱĪôԼģǷݲûԽǿҡ
򣿡ղŵĴβǲˣҵᣬʲôأ
ʲôأãȻ̧˫ۣҺЦȴֱ
һ˵İ칫ңôúõĿһ΢΢תںɫİ칫ϣƺڸҰĿǸװɵ伮ǸɫĵĻ϶ѷŵһЩϣȻĿƻصϡ
ЦһЩǳϦأһҪһ鶼С
ţʲô飿еĲͬ̽ڷ˵һ
ǡ¶ÿݣ˵İ칫Ҵɨɾ
Ŷһԭô¡

ڲϣһ÷ḻ²Ӵʦİ칫ʲôĻɫ㣬ʵУҺֺʰŲ̫İ칫ҡ
һСʱĿⱻһƯֽƬסˡ
ʲôˣһƲߣʱ͵͵ķ㡣
Ŀ䵽ֽƬϵʱҺȻеΪ飬ӵɡһĳĴ档ԼռЩģǣȴ˳

칫ǸĹߵʮˣ
һpiaϸԵĵԣװ
ӴʦľŴ򿪵˲䣬ܻڵİ칫Ȼúܾܾûϴβ׵Ǿ䱾Ϊûӻеġ𣿡һӳ˵ڵϵǸûĥơ
̫ˡ컹һ˹ͦõСںȻԼиҨr(s_t)q

󣬲ʣòܵһγ̾úʡǱᴵŷ磬Ȼ˵ıӰѾʮ˫װ˵۾۵Ķš
ҾӴʦôŶһҪԼ˵仰ԼĲػУЩʲôĸûϡ
ǾǡȻһڣһѩףһͦӲʣһϸ磻һԸоɣһ崿ˡǣ;úôŶͬСϲűȾ䣨űȵĻ
⡰һһһĸġܴûйϵ𡣰򸹷̨r(s_t)q
һ̸ɫı˸λð£ԼȥһЩ˳ԵģȲˡ
һ˳û۵ߡ
ֻۣСŴ᳤ֵͣġƽʱԡ
ҲݰûΪղŵġܴͻ
˵ķԸ˵΢Ц
ɣԼһҪֵú飬ʾԼûЩϣϺʡ
Ҩһ龧ӨϺʣȻĲϺòҵһ϶Сķ˽ȥػļһ䡰ҵӻûùŶ
ʢȴֱӾܾ̫òˣͳǿϺʡȵζ۾ֿ˵Ĳпţţʳʼ鶯ˡǡһϽȥ䡣
ѽԼ׵ӣˣѵѵС̥ˣr(s_t)q

Ӵʦ޵ĵˣԼϻΪӢ۾ĶԲˣϺʵԼӺݺݴٷŵѵȥ
濴ôϸΣôϸļôµݣô񡣾ȻҪڶ̥ˣС˦һΪɵĺűȵľ䣬ž˳

ϲ̵ڲƷתһȦռ伸ѧ޵ʢ˾ζȻּڲƷľͷ˿ϴͬĲ̣ϴĴоԵúܾ£رֻصķ˼ССء
վۻʽĸΪİԭ䷹ġȻʢô࣬ԲλˣرϴĿ֮£ļһ˸ǵԼȫĲϡ
ϴԼֽʰô࣬ôô
ֻСӦѧͦݵġ
㿴Ϊʲôô׳ϴڳ񼸴ĹĵͰǴŵɿ
Ϊʲôǫ̧Сr(s_t)q
Ϊٵĳһ֮󣬺ȥʢڶϴӳ𰸡Լ˰Ц
ǣ鴤ˡһ·ӵߣ뵽УһҨr(s_t)q

۲ȻҪһЩСϷ֮ģһʱ䣬ͬ¶Ҫվ̨ϣλ˱ݴϳұǲȥģϴ顰еȴûûУֵ֮͵͵ңԼˡ㲻е𣿡ȻҾʵж֤ԼʵǸеġ΢΢ͷ̨ڵķ˿ǡôĵƹ񱦹ĿعתӰͬǴӵĹʱ߽Ƥ̨³رĽշļУ̨ҲеΪ֮һĴڡ
򡪡ԭôƵġǰֻǸԱųO(_)O
ұˮĺߺͼӦ¡ǴƫƫѶһмλø˳һ̧ͷͶڴˣȥˡ
ŶԭӴʦĸôԭ̬
Ŷص㣬ÿڶܡء
˵ĸЦˡ
֪Լúص㣬ǿǷ峺ĵЦ⣬ҺȻúҸ;ĳ䵱ֽϻһֱǻԲĳȥ
ų˼׸裬ͬ˼СϷȻ̨µļָЦССĲƯڳϦϲֺ֮С

䵥ʱ򣬶žƱ˹ȥһҪΰĸһƣԼΪԼһһϵĳ¼ûһɵĹɵһġǵ̫˨r(s_t)q
ûȹƣӹݹʢǳǳѾƵĸ߽űһ£ѩ⡣
һ㣬ûġ˶ǰ̨Ц൱иȾ
ĺһڣţζĲӿζģ񿧷ȿζģһɬƻζ档๾࣬Ѹ߽űѾƶȹˣ͸ӣҺΡԲ뿪ˡ

̨ʿݻڳɹɵ˾۲͵һ״ҵĿ񻶡һصߣҸղųô
˵ֻǿɵɵЦ
ǷЦ̫̫ˣҿҲ΢΢ָ̲סȥСӣЦʲôأ
ǲ˵ҵָͷ˲䣬գһ´۾Ȼ©һġڡԼ
Կҵָգ˶ǿļȴЦøˡ
һԣǵıӽ֪ͨСɫġˣûռˡ

part16

ʹЦݣֹĳһ·Ҳķë
˾ͨӿںҺﶼеơǸŮˣǸ߷µģr(s_t)q
ǣС֣ǣһСС·ϡ
ſʱƤƤ·̨ԣ㣬Ҫȥ̼ҹˣ⼸첻ˡ
šʵʲôû塣
ƤƤ͸ԼЦĵ÷ˣŶҮŶҮĽܻؼҡ

ŵϡ
ܹԣƷܺáӦһͻ˯ŵġ
ǣһڴߣŲߡ
СŶһСƦӾ飬һСå顣
СĶҵСɫģʧһ£һֲỳУ
ŶҲһνO(_)O
ҸӣĿ˵ôе飬¹ײأ
ŶΪһֱƯԼЦк¶һ˿ڴ
һ˿ڴɣҶ۾СɫһֱԱ߸šǵģǵģڴ
ҵֵ˼֡
۾ĺţë˳
ȥѵڡ٣СŰСƦӺСå߷ˣСɫһơ
ҪСɫίʣ밡
С顪Ǻô˼Ұ۾˰ɨr(s_t)q
򣬰۾Ϻòã
šӦţֻ۾ֻԵСͷ˵ʲôǲϨq(s^t)r
ĺ֣ůƣȥֻͷҵ̫󣬼סˣƵıԵ촽Ϸ
Ӧһʧ˰ɣԼķ֪ʶ֪һ£ûʲô֤˨r(s_t)q
ҵѾʼһһ㿿ĴꡣϾҪóѣǳˣСɫͨͨ
ͻȻʪĴƵıԵܵĴۣСͷԼԵ
ҵ塪ˡ
ҪִˣССɫĻƴŤһ𣬹˼СڰСɫ¡
˦˦ͷֱӲӣŽӲĲӣ߳ķ䡣
СɫҪˣСϽָҰķŽpiaϡ

ԡһͷ
СȷСɫѱʱѹסˣƣ߳
ԡɿĹϣˮλűֱڻĻҹԡҵţһ̧ͷͿվԼǰ
СڱСɫϸ۲죬õСɫġĲˡı֤֮󣬲ŰָȨԡ
ôˣ
šЦˡ
ţ
ÿСȥԡ۵Ĵӣʲô
ţסӣûʲôȥ˯
ر󣬸˵Ļһȥ˵Ĵ֣ҿ
ܿԣȥ˯
š˫ȾóڵϵС˵ʲôҲŲһ֡
ˡ
֪ԼˣҸ̡
ͷһ㡣
ߡq(s^t)rȻָʹ£Ҽ̡
һ㣬ԣٵһ㡣
ţûеͰ
ˣҪٵһ㡣
е
һţһţҵͷĥĥһһĵ
΢΢ţС죬ҵ°Ĳһ¡ңֽˡ
ˡתȥ˯ˡ
ǣС°ͣĵëëġ
СɫĳǾʧĿյһ䡪Ϊʲô촽أr(s_t)q
ҸղٵһͺˡҰյ˦˦ͷתԡңѹСɫġ

굽
ͳ緹ٶˮȡ
ÿһҧȥ뷨ӰƮƮȥ
㻹ǵ𣿡
ʲô飿
֮¡
ţһ֮˾Ͱͻطˡ
Ȼأ
Ȼ󣬡룬͸ߵĺΣȻ˾Ͱס۾ˡ
ȻأòǺ𣡾ȻᱡˣһĻװû˨q(s^t)r
Ȼ󣿡գգ۾촽
ԣ촽ҹĿ򣬡Ȼʲôˣ
ŶҾ˯ˡȻӣĵȥ¥ݷˡ
ʲôҡĿɿڴˡԼͲˣӣһֻˮȷŵУ뷨ľ׽е񡪡ίСϱ

ô᣿ôǲأ
һ磬кüͨ绰ҽӵĶĲɣǸӰȥѵ绰£һǲģ룬˯֮ǰǲԼ´ţ
㣬ֵĿˣǵа
q(s^t)rȻˣץŵϥϺúõһ١ˣҪ֤ݡûΣ
Σ
԰пܰʵĵΡ˵жΡұ˫ۣӸµĿһСһСŬ롰ξ
ʾʾǲ˴˷ţ
ţ˳ҵգʾľʵļа
ƺһЩʲôҵıһαװСģСå߳䣬һı飬һָԣվŵĵط
Ͱ˵ָҴһԼĵԱ߾ԡңԽԽƺһŴƴͼѾ˴󲿷ֻ֣ؼĲλûд룬˵˵֮ġ
ʱվԡſڣš뵽ԼԡΣԴսԼôԡ۶ûʵͳ˨r(s_t)q
˵ͻȻͣסǫѧС˿
̫̫ɶˣȻԼٽһĿָһɫʡڷ״µ񡣺ðɣҪ޳ܣҪ¸ؿ˵ĵһ͹ԡ۵Ĵӡ
(o)Ŷԡ۵Ĵӣãˣ黭ԡ۴ӵƴͼ˵ʲôҲҲ
Ҫ˨q(s^t)rԡ۵Ĵӡҿҽ
ѵõüͷˣ˽ƴͼҲûۣ
ˡԽԽӵĿˣˣҡ
˵ɡԼſɳϣPPٰ㽻ϣ
Ҿãƴ䣬һλǰоεѧҡ
죿
šҾУѧǣڰ˴۸ļǲ鰡
۸ģʦڱԩһء

磬Ҵ״̬˫ĿСǶżˣüëСҲϢˣСҲˣСåСƦӡСɫġβǵȵһɾ飬ŶС飬ˡ
ææɨɳ̣Ҫҵʱ⽻ĺѼ䣬ڼСʱַһЩǹٲȻһ๦ܴСˣ¥¥·ǰݺͣıš
ڱЩζˣСΧȹӾĸѧżͻݵ££ͷȻĿƮɳϡ
ȻΪһ֮ǰδеĸǡ쵽׷ʲôôήңһҪ
ѧ˵оۣصҪظ龰ƮԼСͷԹԵɵϣʮֵλı۾һӣȻվϣţԡߡ
ǡʲôûr(s_t)q
ĸڳأ΢΢С˼ͻȻ򡪡кѾƣ
䣬һƿ˵ѾơˣҺһ㡣
ɵеҲ
ӦĬʾԣ˵ķļгֹO(_)OĵİľڱﵹǳǳһϺҺ壬Ȼ๾඼ȵˡ
ĺһۡȻݾͱɵɵЦ
ѱpiaϣееѹ˨r(s_t)q
Ҫ
Ҫ
ڿ￪ĵƮƮȥΪٶȺܿ죬ʵھƾ£ܻܻ
ڣĿ⻺Ʈϡ
ףôr(s_t)q

ˣ
š
ˣ
š
ңһһմһһӦźʵݵĶԻ
ҾãܿġߵƵĻһС첲ģ˵ررġ
ңš
ҲôãЦøɵˣӣˣСֲֵȥץҵ½
½ϽŸԼҪõһƤѣҲΪԡ۴ӵƤϽԼҪõ񾭣١
ţ½󴫵ݸҵźŸպõҴԵָӲʱΪ֮һ漴˵ˣÿ

һģһŶԼֶŵ򣬻ػҸ̡
½Ѿקܸˡһ֣½ֵ˻ȥ档漴ְ½קȸղŸߡ
һԣҶͷ
Ȼûɿ֣ǡͷ̽˹ȥҪͨҵȥľɫ
СԴҵ°͵²ȥͷʱҵĺᣬССֹšôʲôҲ壿зƤҵУײ½ҪõǿƤr(s_t)q
ҶҲҶһˡ
СڵĻһʹ˶СåǨ͡ǣȴصڵǸСɫĵС顣
򡣡ҵЩ΢΢΢΢սɳơѾӲСɫ˵Ĵ֣סˡ죬סǣҲܽӲ
򡣡ɳƸһ֡СɫľͿɹˣָͷѾʼ΢΢΢΢
ţڳԼСԴĿҽӲ١
롪СɫʹڰѴˡ
ȴͻȻɿ֣½ɶĵ˻ȥСɫĴôֲ
档ɵɵЦˣúڰ
ҶͷҪˣ
ִսҵˣÿ
䣡ϣ¼Ժ󷴸ζ֤֮á
ˣͷһ㡣
ҲŲϵ
ԣٵһ㡣
ûСҷӦáûеͷɡת飬ƺ֤Լֻһᶯ۾ĵѡ
˺ðٵһ㣬һ㡣򻶿С֣۵СѾʼƺһͻᷢ
ԣ°Ѿ׹ˣҪҵ촽ҵͷһӵ͵һˮƽϡ
ͻĶŵһͣˣŵСپҪʢһˡ
ȲҪʢҷΪСɫվҵ˵촽ϣСһӡ尡Եأ
Ŀӡ
ǼĦ
꼷ѹ
Сոջعҵ壬ͿһĻ̫ˡ

ˣã
Ҳʮӡ
Ҳһӡ
Ҳ˵ŵʱһֱǵ硣
˯Լϵ
ľһ糿ҺȻÿϸǰδеС
Ҹղţǲǣ˵ǡr(s_t)q

part17

򱧵ķŵϡҸϽŴϡ
ССɫѾΪһˡҸе̫˨r(s_t)q
ǣһ磬ҰԼ
ů͸ӣڵذϣҵϡ
̯ڵذϵöûжһ¡
棬Щ̵СѿƤǳǳá

һ˯
ҵͷ¥Ŀһ䵽ϣĵѵ𣿨r(s_t)q
ǵ˯֮ǰ𣿡
˯֮ǰ
šǵ˰ɣ
񣬼ǲˡ
ǾͺáǿԼǲˡ

ƤƤڰְֵСŤŤȥһңҵС¥
죡Ҫٿһ㣡ֻٶɢĵĿϲǷЦ⣬һǶԼİʾ
һƤƤʹҵĸ첲˽ȥֱǰ
ֶ˫ȷֿͬҪƫʾ֡ؼҪص㡣ƤƤطջصָѾ̼ҵС԰ϰ˺öףվǰֻһ͸㶨һĶﵽһʺ̬
ҺתͿƤƤƵվǰҲЩ״
ڣƤƤлһ㣬Ŀˣֻһ֡򡣡
ԣֻܽһ֣Ϊ֮ľѾ̫С̫Сݲ̫ࡣģƤƤĴСֱֱĶ򡣡ҲԽңƤ
Ƥѵõć
ôģ˴ƤƤͷԼһﵼξ׼ıײɿ飿>O<
ôͻȻҪһ֣ǲѧУڸʲôٻ
ƤƤһûʲôط˵ײɹ͵Ц´ټҸܿε״ȣƤƤתһȦӲõúܵλĿ䵽ʱͻȻһǰ㣬ʾԷθĹġ
û·𣿡
·򿴿Լİɫװ
꣬һҪµġƤƤɨҴ󻵵һۣǰҼʱ򣬹ҲһҪµģһ˫ӡ

һ̶ݼˡϵ˷ǻؼҵġ
ڣһҹ㳡С
վԶϣ߻ϲ´֡Сͷճʾ滹һС֡ϲơ
һ¥ĿȺʶǣסС֣ɨһۣȻеߡߣʱĵӶȥˣҸ̡һ뵽ƣĿ˽һ촽ܡСpiaСɫһƣһͷȥ
ͷֻ֣ܶȴֲ˫ֽϴǷǳǳ仹΢΢ƺһصͷĵĽǣŵĴСϣĶǱѡ
ֵһ״̬
ôָһ޳ţжȹûO(_)O
ǿ԰ɡԱİȥ¼䡣
ҲһգۣԱֻˣŮѰɣ·졣
ӷԱ¶ҶپҰһˮɫ޳ٰõΰǡôӴ׺һֻƤĺᣬžѣľȴԡֻа˷ֳ˴һǳǳ˿Ө׵˿¶Ȼȴ־ޡƣȻɵţжȹϡˮɫʽ˿̴ȴһɵζرͻأߣӳڳ°ڵ֮£ɰ֮
ʲôҲû˵ֻǰп˷ԱС
ллݹˡչϲͲñѡһԱѿУְ˴ԱߵƷ̨
ۡĺһöӰ۵ľ̾
ԭôϲӣҲĿһۣмƲƲ죬ЩϸϸĶʲôÿģ
ϲصıӣߡ
㿴ǸеģиŮѵֱָָۣ㣬ģ͸Ů˫ӡ
ɲǡҶðɣ۰Ѽҵˡ
ţϹáŮϲ˲䡣Ĵ߹
ҿ˿򣬡ľ·أ
ⰡװӵĴӸҿ
ȥ򼸼ұҲôġ
ûO(_)O

ͳմͳΡϲĴŴĳؿĽӹ㲥дԴ۵Ů˷ݵ¥F
˳ݿӿ˽С
㲥˵ʲô𣿡
ûСֻ˵Ů뵽¥أ
Ҳǡ

ѵõĸһ˳ǣ¥
÷ȥFģֻҪѰϲĽͿˡ
ϲԱԣеۣĺõĶҴԡ
߹˿ͼףе¶
ϲԱСͳΣ¶Ҳڵ¹淶֮µ¶ȥ¼ԣоӡ
ҺϲԱĵƼһɫۡ
Ҳ⣬ǿһɫģ
һԣо˺þáǼۣʲôأԽ뿴Խ˳ҷԼȻЩļ
ڣӺ̽߳дǳǳĲڣȴʫг˵Ůӣŷҷ﹡İϻļڣ΢΢н̧ףͷֻСС
۹⣬ʽţɫȴޣŮϣԸĹˡڽѾλר˿͵ϲԱǽӰרٳ޲ڷԱԱ
û˵ֻǲΧתһܣѵһδ·򿴵ֲȻһ֣ѩ¶ǣзۺͣͷ
ǹһ˿ȵЦݣǷС飬ǷСɫģֹΪһˡ
޲ڷԱĿ䵽ұ촽ϣ˲ʧ񣬸˵ʲôŷλ˧еûдԼҵʲô·ǣĺĵľ̾r(s_t)q
ԱʲôֻǰпŵǰĿȴִ̰󵨵ͣϡ
ȥʾˡ
ţ΢΢̧ü۾˻׽ŪһСȴܾ۹ȵߡҲˬĳɨɨЩ̶ɡؼٻϣȻꡣ

ŵӿڴֹ˹ġҹ飬ȴ޼
תЬչ̨һΣȴûʲôʺ򴩵ģȻ˿̴ڽϵֻһ˫ͨСѥ
ڵЬӶôûд𣿡Ҳֹһ䣬ǰ׼ƷķԱ̱ͨ졪۹̫˨r(s_t)q
λС㿴źúƣǺദԱЦߵߣС㣬˫ʲôģ
ů͵ġϿȴϲůСЬO(_)O
ԱλСλһ
ȱķԱһңЬӶͦů͵ģô
ƼһʱֵķƤЬص߱֯ѥͲ⣬ůΧ
Ķˡ
ԱĵңѰŮѻЬǾӡ
Լˡ
Աȱˡ
ʵԼѥO(_)O
ôƯСšԱԱ߰æ﷢̾ã͵һۡôĽţ˰ɣ
ߨq(s^t)rһ˫ʲôÿġҿԱԼߵ۹ʱϴ΢΢Ĳмǵתڱ˿СʱСɫĺȻռߵˡ
˿಻͸ģǣǰ״ȥ븧һ£ǾɵĽ׸һա
С쵽ѥʱҵĺ΢΢һ£ֻԼõʿˮ
ڵϲ˼ţҪһЬǲǳԿһ
ů𣿡ԱԹ˿ķʣǾеģ᲻ҪҼӰࣿ
šĵͷҪпĿO(_)O
ӿڴֹ˹ġ
һװӵĴôװôණǺΨġ
Ѿѥӣûм߳Ьһ˫ƥЬӡ
ˡܹԵҡҡͷС⡪˫ЬˣʡǮѩO(_)O
ǣЬӹ޵ûһ˫߸Ьʺ򴩡
ҹŴ۾ˡٲ˫Ьͺá
Ŵۣ΢תղǿɰĽͺͽ׶ͷл˻ΡǸ⡣СɫĶռߵء

ݿڵָʾʾһмҿꡣ
пɣÿݿڣͻῴһָʾƣ͵͵һˡӦпɣ
Ѿһ¥ˡ˶û顣Щˣ۾ָͣʾ֮͵͵ת
һĵģʵźۼ֮С
Ҫ˵󻵵Ҫ뿪ݿˡ
ˣһˡؼҺˡʹżѵĲӸſߡ
ˣĹСеڿˡ
һмҿꡣO(_)Oڶǻ˺üȦĻѿڶ
ţ𣿡һĿ򣬡Ҷ֪
ָʾдС΢΢

ɫѩ졣ɫѩ⡣
Űϵɫѩ⣬Щ΢΢ʧ
ôˣ
иǰҲɫѩ⡣
˵
ղˡ
һ˵ĳһˣҾúʣ΢΢ᣬǴ⣬ǺȻ뵽ԼԺֻļˡ
һѩŵС
ʹǰζһ𣿡
ҡҡͷÿζͬ
ٳѩҲһѩŵУϵ

ҲҪ١
ϵƹ財
òһ⳵
λԿһ㳵𣿡
΢΢ͷʱĺ鶼ɫѩ죬ʱŵеİˡȻϻġ
ڻ¶ԱһЩ綼˴ζڵĹ㲥Ʈ
ǰҲڡ
ͻߡ
Ĵ¶ʮһ
֪ġ
ǰ˵Ļ۾⡣ķ羰ϣֻ뿴ڳϵĵӰѡӿڴӶ㡣ǡ

³ʱùıڴŰɡ
򲻽ĿңȴѾ˲ӡ
ĸں棬ʱ͵һ£˽ôôȰͶ(o)
֮Ҵ򿪵ƣȻһͷʲôԸ𣿡
ţûǹ⡣
һ룬˰ʵ֡

ʲôԸ
ʲôԸأ
תתȥȻԼûʲôر˰ʵֵġ
Ҵԡ߳û뵽
šĿ䵽ԡ۵ϣϵûʲôҪġ
Ǿڵһжܺã
š
ҷ²ͷë˫ĿҲͬˮϴһһĹ׷Ѱ񡣡ԣ룬ڵ״̬һֱȥ
ţ仰е¡
˵һֱȥ𣿡ҵ۾Ȼúһ˿ϸţëļ
ǵġ˵ڣеΪ顣
ѵõʧˡ
ϻ۵ˮԡϣµµĹɢά֮С
ƮƮƮƮһۣȻƮƮƮƮôһ˴پͻأңЩڴ

һҪȥˣҲТķֵĲĵ绰Ҳڵ绰Աߣһ˵̹á
ڵ绰Եȴѣλһ𣬴յĻ֮ϣǶӡ
СΪڽԼr(s_t)q
ԣû㡣ͷοһС绰ˡ
˼ӣŽ绰Ҷ˯ֱ㳳ѡ
̣á
򰡡ΪԼĿڿһ£áãҲðɣ
裬áҲТѾӹ˵绰᣿
ߣһͽ衣
ʱˡ
Ŷôôģϲ̵ģҶϱأ
ϱһ㻵Цǣҿһ񳯷ߵһȣϱˡO(_)O
϶̫ˣô¸Լ꣬ΪοƽʱҪûϱӻź㡣
Ҫ˯ˡ
һ˵˲ϵҲ˵ˣҲˣеµ㡣
ţ
ҽԤ˵ůӰߣԼעٴ·r(s_t)q

ϵ绰׼¥ȥˣ绰ȴˡ
ι
𣿡
š
Ȫİְ֣򡣡
ʲô£
ʵֱӸ̸ԷƺһҪȸ˵Ȫְֶһ£Ȫ˵ЩټĻһҪһ˵ȻţĲܵȵ졣ЩʽȪְΪֻȪ
ҶȪְͬиҵĴͳȪҲŴʽᣬĻЧҲʧΪһ£ҲȥһĸԵһ£ȥ򡣡
ķ䣬˵㲻Ը⣬Ҿȥѵ绰ˡ
ԸⰡȪסһġ

Ȫְڵ绰һһĽ˵ʲôͷÿһ䶼ظһ顣
ӣĵײһĸоЩѧʱ⣬էֵķ˼·ǻҪϸһ²꣬ƷɫѩζһƵĳͷС죬ȥľһЩϵһӰͨһУֵĺߡĴˡԽԽһˣ
ѩ˶ģֻСͳ֮֡
ѵڳꣿ
ǲΪԼһأ
ڵ绰ｲʲôȻˣȻԶͬһ͸У뷨
С鵽ҴмޣһİС飬˭
İСקקģѩС顣
ôû㡣
ֻ·˴һѡ
ڣ˾͹ˡңһһЩ֡
ԼˣվһţСɫġСƦӡСåдβǨr(s_t)q

Ȫ򡣡
ţҡȪӰְнӹ绰һֻޡ
ûбкߡ
š
ûбк׳
š
Ҳûбкڡ
š
ǣΪʲôҪкһأ
Ҳ֪
ĵ𣿡
ﾺ档
أ
־첻Ͻ
԰Ķȥһġԣ㣬ԱģΪûܳΪһ㣡
һɵĻȴȪְȪСʱƵģԶӵŨŨ֮⣬ͨͯȪ㹻
һζô˵ˡ
šҲˡ
ǣټ
ټ

Ȫְ֣Щ𣿡
ȪšСʱһO(_)O
Ȫְ֣СӰ̻Ϊʲôġ
ȪҵҲ˵ҵĳơ
Ȫְ֣뵽ڻһֻޡ
ȪǰǣΪʲôҪһأ
µһڰʱĽ١

part18

ȷȳһáһһıů
ҳ֮󣬴ȥѩ⡣
ſڣϰߵ̵ǽϹԹԵվš
죬һȥɡҳ֣ԺҲǡ
춣
ͷСݮһ̧źڿ۾Сһڿܵζʮĺһãӭ١
ҲĿ䵽СݮϡֻµԱr(s_t)q
СݮĿȴ۽Һϡ
ԭϲ崿͵ģ֪Ҿװˣ·ˡôô޷˵ȥԸûǰ;·أǴ˵崿ǶݵģΨԸв㡣ԼͣСݮĻ˻ԴûԼôֳеɫ֮ˣòȡǱʷ֤ĴοѾмҵˣԼƻƽǼͥ
Сݮ̰еġʧ᫵ɽ·ƪڵСϣһͦأׯϵпʼĬ
İף
ԽԽ
Ϊʧߣ
ҪľǸˣ
ӹı麺ʤö٣
ɱ
ɫʤ
һɰ¡
Ǯ仵
ʼԪ𡣬

ʾߵĻбXͶǸˣ
XʤX¡XXԪ𡣬XX
ʾٵһ飬Ҫߣص㣩

Һɲ֪Сݮ̨Ϊˣ˵˼ڴ򿪵ıO(_)O
Լᴵŷ硣
šûO(_)O
ǣһֱʲôԼͳʲôںȻȨգȻѹôr(s_t)q
ţǰǸ˴ǰһӣеֻһֻС
ɿôҲòһ½顣
ţáĵץһ֧ŵ
ǣĨأ
ţá
ݮ¿ζŶ
İ
ǸǾŶ
ţ˺ð
̵Ķá
ţ
СĽţ»ѡ¥һr(s_t)q

ҰǮ򣬡ȥˡӻǫ̃ϡ
ģɷΪӰСݮ漴Сݮĵ־Сpia˻СһƣСϽĬһ
СԼƤУǸһĨǳǳЦ
ˣСݮһ飬ʤr(s_t)q
еᡪ¡
еƺÿ񡪡
еļȽԪ𡣡
Ŷпŵƺټˣr(s_t)q

ãҿԼǿûбɫաСݮغһͷ鷭µСڡʧñʧǸ
硣
ιڵࡣ
֪þöûϵˡ
šڿŬʧ
뷨𰡣˭ģ
Լ
Ǻǣ١ӳһ֮󣬡֪ࡣ
ûµĻҹˣҪء
飿ˡʲô飿
šСݮףһۡϼܽ飬ġ
ŶСݮ㳯֪ԸԿ£ʵ㻹ʺԸ
𣿡ͦġ
šмҹĿҵ·ȫһһһֱ壬ȥȥ̫ѵˣƽֶǴ۵ģ⻮㡣
һһŻݲǴ𣿶㻹ܴۡr(s_t)q
𣿰Ǻǣ١ӳһʣ㵽ȥȥأСܲɶȥ
ȥˣʱ䲻ӼӰࡣ˵ĵһԶԽкѡ
ʲôŭˣСȻôң˭ģ(t_s)#
ʱҵġ
·ŻкˮʱСһ㣡
ţˡ
ཡµ绰ըˣһͷСܲӵһ󣬡ôˣ
СݮҪ췴
ҡСܲͬ
ֱֿڣû¶࿴飬Сݮڶѧѧ
ʱҲ޹ڴ⺷Ŀע£ԽԽС̬ѧ
ˡСݮҪڣҲҪ֧վȴҶһ£ͷڶء
ת󣬡СܲⷢôģһжûС
תСܲ󣬡Ⱦ𣿰СܲӽȾҲˣ
Сܲһҿԡ
󣺡꿪𣿹ֵ£ѧͷǼΪԸоô𣿡

СĿŵıζ϶ƤƤǰСëͷɵġ
ҺС·ϡ
Լѩ𣿡
šĵĵͷ
Ƕĺţʮ죬Ժÿ춼һѩ⡣
ʮ죿
šֻʮ졣
˵־ͲҪ𣿡͸Ž
Ҳѹ㲻ڴ־Ҫȥ𣿡
ǰ
ǣѵõ¶Ծı飬֮ǰǸ˵
ʱҴʲôʱҾ˵ǡ
ûĻ˵ңҾͼץס˫磬ǣʲôʱߣ
Ҳ֪۾͸ãȻ
Ҳ֪ ǿԲ
֪һ죬˯֮󣬾뿪ˡ

֪һ죬˯֮󣬾뿪ˡ
ȻʴȣǹɵĲ̣
ҽڻ۵׽ʪ
ˣҵĻ﷢
š
ôˣ
ûʲô
Ŷ͹ԹԵİ
һһĴŴϢ
ҵֱһһսεȦؿڣ㿪Щ˿˿ơ
°͵ͷϣȵƤķ˿ỺĦ
һУ۾ҵؿڣҵζ˫ĵ̧§ס˾׳һǱܣһȴոܵڤڤ֮еָ

˭еıѩСϣһİС顣
İСһμôƯС飬ңС顣
ôҼأ
ΪһѩС鵽ҵˣطõġ
ŶҴתתѩСǣСȺС֡
Сõۣγ̫O(_)O

һ
ûнϢĳ̶ȡ
ǣ޷ĳ̶ȡ
Ҫһģһ
ȥЩӣ໥Ѱѡ
õ֣ľֺаöųζʱٻ룬ŷֵһ۵ʱһоע
ҺͷĬһλãֱĴڶԷļϣٻ࿿һ÷˿໥ࡣ

վڲԶƤƤеֺСϵĺỹʺôļڷɪɪõƨɺǸƺڵľ޴ڴﰧ˵ĵڶα׾ôʧ
ƤƤС绰ιƤƤҶȡ˰ɡr(s_t)q

ǻҪãƤƤյǰͲԶʱɿ˸Եĸ첲Ȼǣߣûע⵽ԱߵƤƤr(s_t)q
վԼҷϣſڸ߾׼ԶǰǱϲǲΰĻλСӡ˵öԣе˶äĿġ

Ժÿһ춼Ҫÿģа¶ġ
ͻȻеҵյøһЩĵ̧ͷһҵĲС͸ǳǳӣҺΡ
ҪѩСָСŻʵСϣĵĽڰֽϻƯĺΡСȺõһӶﰡO(_)O

ɨء

ϴ·
Ҫ͡ǣԭԼúúõ飬ôãȻЩˡ
ˡ󣬡ɡ
ţҿԵġҴסһǽʹɨ
ҡգգ۾Ҹ£ܣġ
ţҼվͷȴײǽǵĹϡ˵ʲô
Ҹ£ܿġٵظһ飬Ͻڵϵ㣬ŴסǸǽʹɨ
ѩСһ࣬λ̫ˣѶ䶼ˡССԴϴ̬á

ڿ绰ߣ㿪ͨѶ¼ҵһزȥ
ι
ҡ
ң绰žĹҵˡ
üͷٲһΡ
ι
Ϊʲôҵ绰
Ѿ͹ȥˣٺκιϵȪ˵
ְ֡
Եһ¡Ȫȥаְ֡
ιȪְ֡
ǵõʲôʱ뿪ҵ𣿡Ǻܾûùǫ
Եһ¡Ȫְְѵ绰ŵһߣһԵȪһرܡơ
ôͻȻ
ǵ𣿡Ҳơ
šӦǶŵ糿ɡ
ţǲ죿ץŵ绰ֺȻʧȥǣô뿪ġ
ʱһ𴲣ȥķ䣬ĴϾֻʣһˣȻҲûˡȪʱ˺þá
ҴͲٴʲôȴ˲š
ι𣿡Ȫְ˼ڰѵ绰£΢΢̾
ôˣȪſ߳վְǰ
͵
š֪ôˣ
Ҫ˰ɡ
ҪˣȪıſ޵ǰףҪѧУҪ𡣡
ʲô͸𣿡ȪְץסȪ󣬻˻Ρ
ԺĿݡ
ȪְɿȪߣһġʧ᫵ɽ·棩ĵ˨r(s_t)q

Ȫ±Ƚϸߣ뿪ıȽ硣ӦɣĴ컹Զأ򻹻һӰɣеĲǿ־壬ȴôױѹסġ
㿪ϣһЩѩ˵Ϣ£һбѩ˰ɣһдֻƬɣѪЩص壬ܻ˼¼һЩʲôɣҵĻˡٹĶο飬ǶûʱĻҡǵķΧԼĿûаôĹʱߡ
ѩ˵ҳ࣬ȥЩѩ˵ɰߵģȥЩѩ˴ʱģȥЩѩ׷˼ͯģһƪǽѩģǣȥȴ̽ռһӴ֡嫵ҳȴƫƫҲѩ˵Ϣ
һԼһصжãûһͬĹСҾԼЩʧˡ

˯𣿡
û
ҿԽ𣿡
š
߽ţվڵϡ
ڸʲô
׼˯
Ŷһ˯˵˵ðɡţϰ˸ʹ䣬ʵһġ
ûġ
Ա𣿡˴ϣһԱߵλãԹԵ˹ȥ
߿һңе䡣
ˡ̽˵ҿȥ䡣
ţҵķӦʱЩ֣İȰʱ䡣
ѩ˵һ仯ҪһЩ
ʵҲǡֻûôԡңǲҲ¶Ȳһ
ҵֱҸܲ
šӦһƺѾԼԼµģɢһƬˮãã
֪ôֻоߣȴ֪ʱѾƶҵϣͬһ˫ֳڶٵĦС

ʱ⣬
Ϸ
ҲҲòã
㲻˵Ҿ͵ˡ
ǣΪʲôһۣҹȴ˰졣


۵˲䣬ͱʧĿ־ռ

ѹŷֱϵ

ҵֱϰȵ˯ţëϸһ㱡İ˪
ҵڵõƬ̵ɳڡоſʼ񡪡ĺ䰡r(s_t)q

part19

ǽȥ档ŸһЦݣõҫۡȥ
ﶼá
ȥаɡ
˽Цر࣬ΪʲôԼܾʵЩ˵أ
ˣҸ㽲һЦòã
ŶȻáһ˲Ȥڳ嵭һеĲã
ǰһѩˣվڱϣֱҲһѩˡǣÿ춼ȥǸѩˡһ죬ֱµѩ˱˱ˣҲȥˡ
ˣЦ
ˡգգۣЦ𣿡
һҲЦСƦƵƲƲ죬ٽһ
һ룬һЦǰһѩˣվѩɽϣɽҲһѩˡǣߣ˺ܾúܾãԼɽΪʲô
ˣˡ
šѵЦ
ЩЦǴģ
ӱѩ
㣬˼뿪֮󣬾ͻصѩ֮ȥ𣿡
š
ʲôģ
Ҳ˵塣˼˯֮󣬾ͻһΣƮƶһЩѩ˼һ𣬴Ҷ޻һģȻ󣬾һֱһֱٽ䵽˼䡣
ѩ֮䲻̸˴˵𣿡
ʱļͲˣֻżһѩ飬磬ղĴͳЦ
䲻ˣ
šҶԺģͬ˯һ
Ժǻļǰ㲻Ǽǵǰ𣿡
šһЩ֪ʶǻǵõģǹ˾ͼǵòˣǸһֱûб仯
磬ǸԱ
š
Ȫ
ҵʱûϳһЩʾ
ѩ̫Ѫˡͬһ˵СƦһֻǵñ˽֪ʶŶǵɫѩζǲǵˡ
ίĵͷ֪ͲЩ˨r(s_t)q

Ĵϣ˺ܶࡣ
СƦҴżѹ׵һʱ·ţȹǣվڽֱߣվڷԶԶȻЩµζr(s_t)q
ۣĵӾ𣿡һرϲżŮѵ½ǡ
пŶҲرϲż磬Ů־ͬϵO(_)O
ǴԱߣ˵ڽľ缯
š˵缯ȣ
п⼯ߵĴ֡
ȻǾͻᱻע⵽
̽ע⵽
˵ȷ־ͬϵĨr(s_t)q
ǣҺ˵ļһҲòע⵽һһŮǣѾأڼ䣬ٹȥɿ100·
ΪʲôһҪ˫ʿ100·Ҳֲ԰͵һڷаſƵˣ̰еpiaɢѡһģ
һҡҡλε˫ʿ
ǣ֣ȥ
͵ò񻰣ҹߵ¥ݴֱһֱȻ϶¥ҵλ
ǲǺܲҿϣҲǵһء
(o)ŶѵĵͷȻԩ˸ղŵɨr(s_t)q
Һڳͷ˾ҿŲ
ˣǰ᲻򵽲ϣֲѺۣ˵ѡ
ᡣקקģĸ߶ȣ·Ŀȣĸ߶ȣǾϸ淶ġг֦ᱻԱ
Ŷ֪ʶԨĵλá
һ֮һ죬һϡ
͵ˡ
ûϵ⳵϶רƵģᱻһҾĳ䵱ֽϻ

ῴ̵𣿡Ҿ˵һһ·ƵҪ
šɫʱ򣬳Ҫͣ¡ɫʱ򣬳Ҫߡ
ǻɫʱأ
ǻĻ𣿡ǫѧ
Ȼ֪塱Ҹ̡
ҵһμʱͳ̵һ
Ŷ
ƤƤѵѩ˰۾㽶͡
ɫʲô
ţҾȣûɫ
ȣ
̫ܲ¡
иǰҡ
ðɡһԼԺ̵һ̫˸ˡ

ֲ԰ů
𣿡
šͷ
ʱǰС԰͸ࡣ
˷ܣһֱ뿴ʲôġ
ڻݵĲ֮С
ʲôָ֣һźصֲ
ǵѧơСƦקקġ
ŶԨ
ƶˡСƦһ֮ģr(s_t)q
㡣
򡣡
򣿡ţ֡
š򻨣һ֧ء
裿̲һͷھۻϳܺ
ǵȻСƦұģţҸ㳪򻨡
ڲġ򻨡ڽС
ȴȻ鵽ıǩ㡣
׸òøأ˨r(s_t)q

õ԰¡
չϹõ밮ĻƣΪʲôõأ
икһ͵͵ĲҽС
ʹ𣿡Ůлһ֧Żֽõ塣
ʹ
õд̵ģ԰鲻ңͻ̵ܵĳͷõΪ˸ûĴܲһΡ
ҲһÿΡԭõΪд̰ǫѧ̲һκ˼⡪ղѧƵ̸ֲడ(o)
һصߣһԻ״һֻǰ΢΢˻Ρ
ţ
ʲôأ
õͰء
ŶµĿ򣬡Щʲô
õΪд̣ƵĴ̲Ǹ𣿡
һԼҪҪѲһõ廻һبr(s_t)q
õ飬Ϊõ廨ֻܱߵԢ⡣
ŶˡǣҾˣڰõóһնû˨r(s_t)q

ֲ԰±һչһɫƶ䣬׺ܱӰ¶ĵ˽
һƬɫСر뾪ֻ֮ĿڶࡢεΪ׳ۣ˺һ֮䡢豹ĸоƮŵ滺֣治˻ǻϲֲ㽻ǻ䡣
ûеĵĸоϸĵкֱߺߵŮѣž벻ᣬʹġĺǣҲǵ㡣οֻ뿪һѡ
Ѿµֻ֣ĿΧչһɨȻ롣
չǰ¡
߰ɡҵͷ˵
ţһû(o)
ҽ
áԽԽ˵ر
㿴ˣ
ժ֮ݵ
ţȻ󣬡ҿʼ£Ҫһ壬ˮûϥǡȡؿڡǡѡ
򡪡˽ºϸ£þգ
ǣϻǶ仨߸ߵľšˮѾû°͡͡ӡҺ۾һһݣֱͷûˮ֮С
ᷢĵĸ̾(o)
ʱŮ˹ɨһרע򣬡ŮһλʿС㡣
򡪡(o)Ŷ
µĿͷûǾڽעûȻСǾɣ¼˵ͷȻѾʧУŻֱҲˮпˣǶ仨¶һǡʿСƾǶ仨жϳкλá
򡪡þհǶ仨
Ůĸ첲˵ĻҪţڹµĽעҲнܡϣĸ첲Ѱǵɫץסк֣ǣͰ
صĺһкӦû°ɡ
Ѿʧȥ˺

ǴʿС㲻촽ϵͣ͵ͷסȻԼĺˡ
˹
ţ㶮úܶడСƦӴƭ΢΢Ŀһۣ˹˳ˣ촽϶ͣ᲻
ᣡȵɵ˵ڣβź֪ĸѩС̵С飺ǵ´Ҫùϵܡr(s_t)q
ƭקקЦԣֻͱ˵
ѧʲô
ҳ˼룬ô˵˵֣ǣαضһЩأforget me not
forget me notظһ飬ǲҪҵ˼
ţ㶮Ӣ
иǰӢ﷭롣ѧĵͷ(@^_^@)
Ҫҡ⻨ֺأ͵͵͵˼顣
ҲᡣҲװr(s_t)q

չֲҲһƬֲͬľۼһɫơչ˽ɫƶֱҪƯڻֵС
ֲΣ϶ˡҶڷͷŲݣʲô
ˣпλŶֵСһ˱ش֡
Ǹλ뿪˴ʧı飬ͬ̽ʧߣ߹ʱϷͬĿһۣǶǷ׳ʲôҲûҵ㻹ȥ
һƬ˲ֲʵʵӦýҶֲ۹οͶϣƬҶݴҵһŴ˵Ҷݡ
ҲĽ飬׵鷽λϨr(s_t)q
ЩݶˡСֹһŲõıǰ˼޹ҶݣСϴһϧ
ֲܰŶҴβʦһν
̽ͷ̽ԵĿֲмЩڽСǣߵȥͺˣ涼Ǻúõġ
ҾԼղžһ׳ըr(s_t)q
֪ΪʲôһҪĸҶĲŽ˲𣿡βҸոտܱչϵĽܣѧ
ΪҶҡܾʵ
еʵġβװǻơ
ԱߵǸվˣڳأҡҡ׹ۿžҪԵϣڡҲûü˾İݶŵƳ㵹Ȼ󡪡Ե˵ϡ
ûע⵽ߵΣգֻǧһ֮ʣͻȻһҶݣ˷ܵĸȥ˵ĶԱѡ֡
β°ͣضӵĵʻЩŶȵġ
ĵǸҶݣһƬʺ۵עĿ£תǰ
ǿ鶼˷˺üˡ
ǣǸճİɣ
ղҾ˵վǣƫ˵ãҹⱲӸ׼ûǰ;
Щ۵Уǰδеĵ⣬֣ƵߣһŮǲǺܰקק顣
ˣǽȥģ
ȥҸղǼǵˡr(s_t)q
ԱǰҶôǺ׻ġܷԺʱ˴ߡ

ҺϢϣʦ˲ݷ
Ժ˲ݣͲùˡСƦָĿһۡ
ԲѵĵͷСֹʱ˾Բˣù˵ġ
ҺһǰδصϢǾͺá
úȻЩѹ
λ˵οͣѾˡʦ۹Һɨ˼飬հѾµܷ˲ݽһһ֮ӵУףλһ

˲ݣСڸչѰղǼǵ˨r(s_t)q
һΣӸǶȶǼλ̴
뿪˻ǼȺд
ˣǻ˭ƮŲۣǰеĺġ
żһΡСҳ־˵
˶ȻҲ˵˨r(s_t)q

ЩǾջҪŻῪšҴƲӲӵľջ̨
죿ڿļˡ
죬Ҷˣڵϡ
һƯ
Ҳɡգգ̧ۣͷӣҵһļоͶ㣬ļֻҲ
ǵã
ǵá
ң
ʱѹ
ֻж㣬
ʵ⣬ļ

˫ʿѾȥˣڹվʧң̽Ľһ
ˣ
ţһع񣬡𣿡
򡪡һ
һ100·Сͷңұ˫ʿҲҪҪԣ
˫𣿡
롣ʱҲûк(o)
100·ɡ

ÿ𣿡
ţÿġ
ȥģ
ѩ԰ܰʾǰվׯ԰
ȥѩ԰
ͬ
Աߵװѩ԰ӪܼࡪԭվЧôðO(_)O

part20

Ƭʡԣˮǹؼ˵ζص㡣
ƤƤվױ֮ǰĻӻŰְֵĭˮ
ϰְֵβ䵽ϥϡֻ
ϵϰְֵƤյһۣһȦֻڿ϶ë
һʰ׵ƤƤҡ硣ܸϾŵʮһļʱͰ㡣
õ壬ƤƤ󻵵Ҽ׼еαףǵһαסɹı׼
˼ţûӦƤƤֻԼҪõȴһֽ
ƤƤҺȥѩ԰ˣO(_)O
򡣡
ҹĴͣƤƤȥȻƨɺǸƺڵ˶ڴżʵʵڵĴ̵õ壬ңЩͶƤƤƨϽ۵Ƥèr(s_t)q

ƤƤĲʹͨ߾׼ԶݵԼҷϵǱ۾ǱĿתĿƤƤǵĳ鶯ԼҲų鶯
ôһϷˡǱӸ¥̽ԴҲֻ³Ǳ衣
ңҹ۲ࡣңҰġ
ģʲôǣ
̫ڣġr(s_t)q

ڴڼ𱬣⳵ѾûгˣҺγǳ;˯ṳ́ܲʵںλϲߡǰһŵʱͷ͵顣
糿͸бҵϣΪǷǳǳһ͵ɫԡ˯״̬ңܵëţЩԡ
򽥽˯΢΢һƣСɫռߵصһ
ǲˣ򷢾ҵ΢΢š
дҳѩϡһůıҵƺһ֡
һ˯úܺãʱֳ;Ѿͣϲˡ
˶ˣϵĻѩѾʮˡ
ţԼͷɡ
ôң
˯ú㡣ѩҪO(_)O
գ۾һһŮѹ֮ȴŵС

³ǿʽģһϵĴӡԽɽϣҰԽ
ȥһҺ֮ǰһ³ϵŮ̾
ܿ쵽СɽϣһĻѩѾһЩ滬С
ҴһɫĻѩűɫ־лһƬĿ⡣ɫĻѩվԱߡ˿ȻЩɾµζ
Ȼ𣿡ͱͱҵ
Ҳᡣ
㲻᣿ѩ˾ȻỬѩ
һҲţҵһ
վţȿ˻קקڻѩˣһС׼һ֣ͻĻȥˡ
Ϊ˸򿴵һò죬úܵλϸΧĳѧһǸ֡չѧߵľн硣
ԭ򣬻ѩұȽֱһֱԽԽСӰ
СǲỬ𣿡λѩĵĹڨ
ͷĿȴû뿪ڻѩ׶һĻ״еҡ
Ҫҽ𣿡
˿Ѿͣˣڵ³תͷ˿Ửѩ
š̵ĵͷһһθ㿴
ûͬ⣬ڱԼʵĽͻȥĿ׷һʧȥȤ΢΢֮Ǹͷֱϲһ˨r(s_t)q

ô³ˡ
˺ðһ±аС񡣡
С񣿡ˡ
ŵõ϶ĻѩҲѾˣߵô
û˻úáпϵ˵
ΣҲˣСһεƨһĨr(s_t)q

ðƽҿʼҪ졣ؼϥתص㡣
ȻҾڿĳʾŻ˼£𣿡
λ˧簮߶Ա߲ſΣϽ׷ðƽĽ࣬ҲѩŤŤõòȣ󲿷ֶŹϣû뿪ԭأһżԵ˨r(s_t)q
Իѩ֣ðƽָ¾Ȼٳˡ
Һгɾ͸УԺ󲻵ʦ˿ѩ
ƽԱ߸̡СһװģôôѧҪ죬һƨһĨq(s^t)r
ɫгһԡС
һֻɫģһֻɫģڷѩĭ֮С

ڻѩѩ˸ࡣ
ż˴Ǻ«һڳѩϣ̾һ¡
һƯ˫ѩռһϵĻǰ棬ں档
ҡҳûҪӵҵĺ󱳡
׼ãҪɆǰһѩͻУԽԽ졣
úܰȫҪοжټԦڱžҵȻûмĻУϸʱ۵ССɣ󼤶Ļ󣬻ҪקקĻԶľϲͣ
ѩߣͲô̼ˡһߵúΪѩr(s_t)q
ѩĽٵˣһĻӼᶨˡСһƨжϡ

ڶλУǰ棬Լ棬ҡڰȫǡ򣬼ǰȦ˻
ԱдŮкӼһĻ̸ѧϰO(_)O
һս࣬û׼áѩѾȥˣΪȻİԼ˾ҳûǰ˴ա

ʱ򣬵Ȼѩr(s_t)q
ղǸѧϰкӼһĻҲŮѸѧϰȻ󰤵ȭ
λУ»Լſѩϡ
ϡ
ͺܹԵ֮
ǰһ㣬ϲŰȫ
ᣬСĽŲҵϣ̷֡ҲO(_)O
ǫѧǸкӸϽ硣
Ůûз
װģһ𣬱ҵĲϡ%>_<%
׼ãҪɡһڷɳ֮ŰǸҪġӳڡ
ЩѣЩΣգܶΣ΢̣ȣ˫绹̣ܵڻй簿С񣬰Ǻսһ
ǣڳѧ˵ǸΣϵߵơǫѧкӻ׶˵ʱ϶ѩĭЬǹεأҲڻмһֻ
ǣǫѧк͹һֻżѵѩϣҪѩ%>_<%

湻˻ѩҺѩȦ
ʵ˳ӦõΪѩȦס
νѩḀ̇˸O(_)O
棬ոպãرǣϸ߸߼Ĺ㲥ڲе衣Ʈڲã֮䣬Զ鳤
ҵ̫ˣʹţЬ⻹ǵѩȦڲֻ࣬ҵЬϣСžֻ÷ŵҵĻ
βһܲԸӣΪ˲ˣһѩȦҪŶ
ʵڵĵͷ
Ϊ򵥣ȴΪɣͷǳգĶĴȻ
ѩȦءϣܲ
Ҵ󻵵ĵԭ򣬡ߵһµİɡ
ʵϷǲԼѩȦġعĬĬᡪΪ˿룬׼㹻ѩȦǿ׳ѩȦ˹ˡ
󻵵ѩȦҲҪڣǲһɲ䡣
̥ϣһߡ
̫Σաع߸ϽȰ裬һ
̫ΣˡѾԵѩȦͬԵǵԼĳ˵

һ磬һĹˡ
СƮĲ㡣
ҰһԶȺλϣԼȥǰ̨͡
ûѩ⡣ԱݸһЦ
̵𣿡
ҾͲ֪ˣȥ
ҳ˲߽Աߵ̵ꡣ
˭ѩⰡӲأ
Ե򵽣
Ҫɽ°ɣҲȷС
ʲôͨ߳ģ
硣
ģ
ѾЪˡ
ִķԱͷ²嵽ѩ
һĦгŵġ
һ𣿡
ֲʶ㣬Ȼ㳤ú˧Ҳ̫ˡ
ҰֻпѺ㡣
ֲ
вУ¶һ档
ǺðɡԱѸٵֻһҵ沿ƬֻǣԱһ
õĦгԿףصԹԵţɽˡ
þûô̼ĽͨO(_)O

ۡŷϵһɫѩ⣬õĻ
͸֧ҵһѷˡǣһȺĿУԵһѩ⣬ҳԵһʳr(s_t)q
ʣһЩѩ⡣
ﵱȻ鷳
ҴСһúñϵСѩԵѩ
ҪμסŶҪˡ˵
ϸһÿεСɡ
Сɺˡ(@^_^@)

ɽϵҹֶࡣ
ڳܿСǾ;ú󣬶ɽϣͷͬ鱦ĳһҪƯ鱦Ưһȳ
ʶ𣿡βƦƦ
ʵڵҡҡͷ
һҲʶ
ҡҡͷڲһ㡣
ߣͺԼ˵ƭٺٵЦ
ǱǡңңһָǱ档
򡪡˺Ԩ
ɴһĴӡָָӡ

ǣܴһƬģɴ
ŶСһ𣿡ͿһСӣӡ̡
ţСȷʵɴıҪһŶһ˿ϸţë
ǽӡ
򡪡֪ࡣ
βקקģţ֯ŮûϳҲ֪ܲҵţֻתһñϵϣָȥϷա
Ի
һţ
һţŹɵһ״һţһţС
򡪡һҲԻԻʲô(o)
ڰԼеһ֪ʶӵ˼£ʵڵݵء
Һгɾ͸УҲáO(_)O
ˣ
ţЦһİ
롣
ţʲôԣôʱ̡
ȥѩ⡣r(s_t)q

ǿ貣κӰڼž֮С
ôҲأСֹ
󻵵ҾվǿÿεСǰ棬סСɼ¶ĿͶ֮ࡣ
ҼǵһáƭָָһСɺССɡ
СɡұһҲ׳һĺò%>_<%
мĿһۣĵȥССѩء
ôûУ
᲻Ǳʲô͵ˣƭ̧ͷ
ĳֻ˯ŵһ硣
ԡƶСѩء
͹٣ڰѩק˳
ѩǿתƵˡƭҼǿա

򣿡̽ͱͱֱϵ塣Ѿ˯ˡ
ɽϵĿԽԽ䡣
һӦûܰȫɡ̾õߡ
иŮңĽðѷȭǶ°͵ףСڵ˯ѣȥ˯
Ժվð߳˼ԶŷԼŮѻվԭءһ֣ô߰
ۡ
Ъһҵš
Ҫ㱧Ǳ˵(t_s)#
áѻλƵ߻
ǣŮ̫ߣк˼ҲûŮġС
ұɡкὨ飬õŮ׿֮󣬾ȡ𱳡ŮӲһеƱȥ§סкĲӡ
Σ¥߰ٳߣֿժǳŮڻλƵıϣижһ룬˾Ե˽ŹµµĹƵ͵ĵطȥ
ᡣŮվкӣͲһÿζᴦ롣
ûУҾˡкӺί%>_<%
ˣ˵ûᴦ롣Ůš
Сѥµѩغˡ(@^_^@)

ǲɽϵ峿úأ
ңİĴϣûб顣
ɽϵ峿ǴɫʱҲݵߡ
´ޱˢϴ
ΪʲôоһƬãã֮УǰҶǿãȴ޳·
šվȵУůͷֺһһ飬ȡ¶ȡ

ˣǽʲô
ȥԷ
ȥѩ⡣O(_)O
ˡҴӱóѩѾڳˣѹѩ类̫ɹˡ
СƦһŮϵĴβǡ(@^_^@)
ѩѩȦɽɢɢ
ɢɢصץסŴ

ɽһƬƽѩϡôõĵطȻû
ѩ׵ĵֻҺµ㼣
ΪɽĴѩЩطѾӲӲѩӣ󻵵վ涼ȥֻŴĶ˶⣬ʹЬӡСȶ˽ȥѩûϥϷѩϣͬһ
ҲԱȥ塣ѩֿĶ־˵Ĵڣʼֶ塣
һԵңһŵѩԱߣȫ˷׷ѩĭr(s_t)qС쵰ץŵϣȻἸ¡
üͷվߵԱߣսһڵϲȳһĿȻĺ֣źɫףŵС

̫Ʋð˳ȻλεĹȦ
̫ܵ𣿡
šͷǶ̫
ֹ̫һȻӣ̫󺰣㣡
һֱôкɣǺˣҾôʱѹؿڵƺļһЩ
Ҳ
(o)Ŷ
Ҳ㣡
Ҳ㣡ܸƤƤˣȻʮ㡣
ǣɽϳ˴С񾭲
㣡
Ҳ㣡
㣡
㣡
Ե㣡
㣡
ӵɽ£
׵ߣ

ѩ԰ıھ״̬

ڣһֱ̫ҲܲסôؽƲ


С
С
ڣ̫ܲסί%>_<%
ƮϸСġƤѩ
һžӨѩ䵽ŵϣ۸̫Ĵ󻵵
һ䵽СϣѧõС
һһһƮæ
󻵵Сڱ۾Ȼɵѩ
Ҫ𣿡ҵ黳ˡ
š
ǣڲ桶ѩƮɽϣ˷ܡ
ɽһϷ򸾣ипǧ
ˡ
ǰں˻ˡ
ţǳһЩr(s_t)q
׹صĵһǧѩڱǴòҽO(_)O

һʱ⣬ɽš
;⣬ѩ
һƬһƬѩƮײϣ֪ûˡ
룬
ÿһֱ뿪
ǣ
һ뵽㽫뿪
ֵͲ١
뿪
뿪
ȴڸ֮УԸ

part21

ˮƿһ֧õ壬Ρ
û뵽ֻǿֲ԰֮ҾͰֲⶼȥˣĻ껹ӯ
ѵĵĸˮ

ϰһ졣ҶܿܡͻȻװҲסϲ죬һԲ뱥Ĺÿ˵ϡ
ϴƸһİԴΪʦгءO(_)O
ϴĹŸĽŲ뿪ˡҲһģţиŶá
ǣҶڰæ֮Уͷһġ
ѵҶûҴһ˫³ĺЬ%>_<%

Ԥ˵ůѾϷȥˡ
ǣһǾôӲ͸ȷʵǰůࡣֻðѰҶϣЩԼܰĹ
ϵӰӦֻǵĹ⣬ɡ

һǧף׷Ӵʦдա
ƬϴO(_)O
ɫķ豹ףӴʦԽԽζˡ
һޣͬ⡣
һǧӦøĳɡһޡⶼոµһˡ
һޣģѹ̫r(s_t)q
ɫȻãǷ档Ӵʦ°ɣ
һޣϰʱЩŶƺһЩ紵ɹ
һļޣҾñȾµ˸ζŶҪļޣ͵ġ
ɫļࡣ

磬ûʲôθڣ˷СĲɵľ׽ܾ֮ǰÿҲûθڣΪ̻ʡ֪ΪʲôںȻûǷȥαװĶˡ
ڿĳӾ磬һ˶ԴУСò֣γĶԱȡ
ҿʼʱ͵ͷԷңģµĻ⣬ҲͰҵɱ˨r(s_t)q
򣺡ʲôʱ㶼û֪ͨң֪ҺҪõģȸס
СףǴһ졣õģľѾһЩˡ
򣺡˭ˣڶԲ·ˡ
СףǾͱˣǼ죬Ҽûʲôˣ˭ѹóԲȥ
Сң˵ģ㡣ѧʱֻèжȻͣ͡ϿεʱһһߵᡣҼҲûè
СףҲ˵ˣԺҲˡû˹һӿ˺ܶ࣬ȥܲҼǷ100ƽˡ
򣺡ǵʲôģ
СףҲ̫ܵģҽֻ˵ûˣӦ̫˰ɡǰü֪ˣһؼҾҪôһλ壬ߵʱһܲˡ
ҵĶ佥뵽֮СһȴƮ֪δ
ص칫ңʹŰҶҲ޷רĹĻͷðʲôʱôӰˣ
ĪķƣңЩõԤС漴ֻ
壬εˡ

ǽڡšݡֵʻĳ⳵ϤСţһж͸ġƵġնġͬһƬ֮ϣĩսʱΨһĴڡ

ΪѾľ
ԣƿš
룬ٴһ
ҿܾͻԥ
Ѿʧȥ㡣

壬塭
ƤƤĽȴҲҵһ˿˿ע⡣
ɳϵѾҵȫռݡ
̫ȣҰ򱧵ͷŵϡ
ȣҪڻ
ҰѱеĶóŵߣڱʢˮ
ñˮʪëŵĶͷϣñֱű
ÿӣҾҪм֣̽̽ĺ

һС
һξһѲɫݰԼһƮ
һըѣһˮСֻƯֶԱһܣ̳ƯΡ
Σİһڣһ߳ΣһԽΡ
ǣһûи˸𡣡ֻħ֡
ø֪뿪ֻĦգģƺҪϵļڻ
һ뿪ˡ
Ѿ˴ٲ뿪Ҫһڻÿһ춼һܳܳ󡣡
ж󣿡
СԴһ£ͷˣüëëˣ۾ʧȥ촽ʼѣ嶼һһһã̽еò
һҲ𣿡
Ѿһ졣
ǡ
ٿҪڻֻʣһֻ𣿡
ֻʣһֻҲûϵͺƯ
(o)

ҵĶС¥캮ض
˱顣ߣŴѾһ顣
ҵ·ˮ뺹ʪ͸ٶᣬһªס
ʱͬԶҩڴֻǴɵұʯһ۾ͱʧȥ⣬˵촽Ƭֽ
ˡۣԼʧңĻڡ
ΪʲôϤĳƺһʧõϲáЦ΢Ĵǡ
ҺȻޡ
ˣ㿴㲡
ûϵ˯˺ܾãҪҪһߡ
šվǰܺڣеƹ⣬˾ӡϸĿžңĿȥĦʧȥ۾ʧȥˮֵ촽
ԼҲ𣿿ǣ˻ôÿ
ЦʲôҲڹ۲첣
˺ܺÿ
𣿡ҵ˵һһ縵Ķȥѩ⡣
šҲԼˣǣǲҲܶأԼûʹǲǶ쿪ʼһҪŬѧϰǱO(_)O
ߣ˳ڵһŵƤƤr(s_t)q

վڵſڣҿ˿Լֿͣ˿ͣȻǮƤƤȥѩ⡣
ƤƤԼͣСˣ㣬ҺҴ󻵵ͬŬģŶһҪŬŶԣ㿴ҵҲǱ
û˵㲻ŬֻһСƨںѡҵ˵ۣһһͲɶˡ
ƤƤ֪Ȥ˱ꡣ
Һű̵ǽվţȻ⾳־ĴƬһ
һȺȥѧѧӱſھĿⶼҺϡ
ҾͺаŶ
˵أҪǵ˽ھۻͲʪòã
ʪ½̵β
㱿־ͺбˮô
ιǵۣѧ˵ֵĶ
ûǻë˽ڰ
˽Ǹ˵ģǵ֮䣬顢飬·֮ĸ飬ԡ
ŶҡǾͽˮӢô˼Ŷ
Ⱥѧڱǰš
ƤƤӵһѩˡһһƤƤһȻӻƤƤШr(s_t)q
šһڡҧѩԶˡѧǵʤˡ
г
ǣӶôˣô
ҺȻ̫ĽˡO(_)O

˼١ǣڴϣ˯¡
ѾˡԼҪһЩʱߡͬ¹ڳִͷðǣԼȴûаܿЩ뿪ʱԼʹ
ʲôʱôˣ
Եֱʼյģ
ԼӦƯƯȹһʱ⣬ñ˴˶õĻŶԡ

ԺѾǴˡ¥ϣõƹƹҵ
ߵſڣһΧȹվڰУ񷭹ڷˮеˮȣŮ·һ
ڸ
ǲҪֱ˵ĺãϾⳡ濴Щֺ̫˨r(s_t)q
ſϱڻ˯״̬ңȻ˹
վˮ顣
ôȣôܵˣë
ûµģСcase
ֻҪˮһоˣӽ嵰ҲҪˡ
ˣûϵġĴëڸ֮Уֻ¶һС죬ҲҪģѧ塣
Сҵûκء
ĲˡëҲס˨r(s_t)q
г̻ȵģôôҴ򿪴ӣóؼգͬʱǽǵİƤƤҲ¶

ҧ˾ַС켫͵ľš
ˣ
Ĭϡ
˾ѾԹ˾ˡ
򡪡ǣû̫༼Өr(s_t)q

Ҹ׷弦
ƤƤϴܲƹϡ
вˡˡ׷ĿľҪְ
ĺܲͻƹȻڲгʱ򣬾ڰĴԣû뵽Ȼô̵Բ˵˵һ͸ʹɣ
ڴϸǴµܲƹŵ˴Уʳɰ״أҲ֮󣬰Ŀŵٳɷ
֪ζôԼ᳢ֲͼһƹ׸˴ˡ
ҡң㣬ңƤƤڵϱȥҪԲ˹
ɡҴ󷢴ȱбƤƤһۡ
ǣǸĻƹ͵ƤƤĻ
ζô
еĹػĿ£ƤƤеѹܴr(s_t)q

ھһ˾壬ҼѵһƬ
ҡң㣬ңƤƤڵϱȥȡϴνѵ
õɺѵһƷַŵƤƤСУʦӽȥһֲ֮ȴķӦ
ƤƤˡˣ·Ҫը
ˣŲסˣⲻǶԳŵĿ飬ҲǶļ尾ƤƤǻͱǻġǿ˾Ϻʵڵմ˺ܶĩr(s_t)q

ҡң㣬ңƤƤʤ̫ǿˣһԲ˻ʱȻֻ
ʵҲûȸƤƤԺ˨r(s_t)q
ھֵζ֮ƤƤؿеĶ̫óO(_)O
ǣһ˾мг룬ѾûҺƤƤһڣŵ

лʲôˣ
۾һѩ⣬ҿܼ򵥣ɨͰҪ춼סˡ
˸ѩ⣡ʮִβǵ˵
˻ѩ⣿
ţҪʲôζģƻѼ棿
ȿζ𣿡
ţֻˮϵеġˮ𣿡
˵ȿ˲˯һ˯ȥͿ%>_<%
ҲڱĲ׵˵ϡ
ƤƤСݲڣʱһҪ

ƤƤСҼֱװƮʮ
㣬ҪȥˡȻҰְ˵һ꣬ҳ࣬Ѿٶʱˡǣ춼ûȥӦûȥһϾҼҽ̻Ǻϵġr(s_t)q
֮ǰƤƤȵȥһ¡õ嶼ˣٸһ֧µ

ͨ߾׼ԶҼſڳƤƤӳڷϵǱ۾
㻹˯˯һ쵽֪ǡǱӸ¥̽ͷ
ǱѾû˵ˡƤƤڱ˼ҹҹҲôų
裬Ѻˡ
ʲô
߽г졣
ܿǱӸ¥߳ĴŮнӹԶ

part22

һѡС΢
򣺡С΢ǰҵ֮һȹȹҪһ㳦֮һȹǡһ㳦
СףҴС΢ǰҵֶ֮һˮ
ϴ󣺡С΢ǰĶ
С΢ģǡ΢¯r(s_t)q
һֱЦСڽͺһѣд¥ܿǰ
ʦҲϴĿҴӱóһƯķ
˾
¥²𣿡
Լġקק˵

ɫӴʦЩҲ֪ô
һǧӼҷ
һޣϢ
һļޣмҵˡüжһ뵣ǣһθеĹ˼ҵĺһ
ɫȷǷ뵣ǲΪ㲣
һǧӴʦÿһҪҴ绰
һļޣѵǶ̥Ѿ(o)
ɫӦûôɨr(s_t)q
һļޣӴʦǿԳ˱׼
ɫǣҪϱ°ɡ
ױǰһ硣ͷĶһЩѾˣͻ¶˰ɣͰѶ䶼¶ͻᡪƤƤ˰ɨr(s_t)q
ѵõױǰ˺ܾáԼߴأëƺҲˣҲһЩΪڻͷڻطģû뵽Ҷһڻ̾˻Ҫò°أ
칫ǰҴһ硣°ˡͷһθоڰ칫Լƺĥ󹤡
ѩС飺ȥɡ
С飺ţ㲻˵ֻ·ôҵ˵
ѩС飺˼ˣѵӦȥ𣿡
С飺ߣ֪ģҲһ
ѩС飺ΪעʧȥͲȥӵУС
С飺ֻǰǰ¶ѡ
ѩС飺˵ˣڵġq(s^t)r

ڰ°࣬Ҿôʱȹۣһ㾫ҲûС߽֮ǰһõЦݡ
˻
šôͷˣ
һO(_)O
ȥҹ档
ððСƤƤɳr(s_t)q

ҹȤĵطϰʱͼƻõģú÷һ󣬿ǣֵȺУʵ᲻㹻ľЩʧǡ
ƤƤ縺εĹСϷǣעҵϣʱȻһûС
һ¿ĵۣͺܶأǽڼǲģǶīɫɽˮ
һꡣƤƤܡ
ʲô
ϻһӶʵ
һŮߣҪҪƶԷ֡
ϸһЩһӲǡ֮Ļ
㣿ƤƤ½ǣʲô𣿡
šһǺȻһͷǣİĶֲײΪȺҲ˷ôֶĳ벻
ˣҵ˻ƵһΪСëͷִʲôϷԼأ͵ͷ
ȥɡ֪˭˵һȻ˾Ͷվ˵һλͷЦܿɰкк
㣬ҪƵҵϡƤƤĵ˵
Сѣ̫СŶʺƢƤƤһЦ㳤ˣƺòã
áƤƤСһӣھҪƣҪ֣ˮ򣬾ڣţҵϡ
Щʹ㻹Ҫһܡ
ϵɽ͹ҶܡƤƤˣС
иСŮڵѾԥþˣƤƤĴ󺰴֮תͷ뿪˨r(s_t)q
λ˹ܹ⺢ӰɡкεȥҺȻƺֻСëͷûߡ
˭ڽУӹ߳һλźɫëţпСӣȻ̫ḻļ⣬ǹǼǿ׳ͣòһҰްҰ޼ߵƤƤǰڽʲô
ҪƤƤΣͦͦظϡ
Ǿƺˣиƨ
ǣơƤƤλҰ޸ǺԼһɵģ̴ίָ磬Ұ޸һϾ͡
ƤƤ˰һƤİ崲ϣŵѵף꿶´֮󣬾Ӣ¶ˡ
ڻعȻ󣬺ĿƤƤr(s_t)q
Ұ޸óһ֧һԴ룬˺һװͷӣƤƤǰ˻ΣһԵŶ
ƤƤͷĿ־壬ࡱһһڿˮ
һҲ̫ʹ͸ࡣҰ޸࣬ñָ˴ƤƤСšǿСžʮûϢĶ
ȡңںˡͷӦܺһЩɡ
԰Ұ޸ñץסƤƤ͸˹ȥȻƺһסƤƤļƺ˵Ѿˣ
ӲӲָƤƤĺţȷλָ֮һ£ڴһ
ƤƤһս㣬ܹץ𣿡
Ұ޸ɲƤƤҪˣһմЩ͸ҺƤƤĺϲ
һģһƤƤˣȻҲ治ûȥˣץ֣ʱĴһ
ո뿪ŮͲ˻ȻƤƤĲҽ¼Աٷʱá
򡪡ôҺ(o)
Ұ޸绹ʱļϼ䡰ѾȥˡڸƤŶдŶʵֻӹһɡֽˮʪ֮ƤƤĺϡ
˾ĶǵʮӣƤƤѾŵˡ
ߣq(s^t)rԺٸ۸磡Ұ޸繴һЦݣֱĿ硣
ֻ˵̫ˣ
ƤƤĵߵ֮ǰȥԼʱѾˣǰһĻΪƤƤġֱҰ޸翪ʼ̽ľҪ㡣
Ҫ
š˲ǶӦ
Сס־Ҫߡ
Ұ޸סһֻ֡
ǣƤƤ翴ֻҰr(s_t)q
ֱһ仰ȴĻصڷ
ס㡣
ס㡣
ס㡣
̫ڼסԣǣһ򵥵صû̫鲨ĻȴĪĵһǰδеȡ޸ɿ֡
Ҳɿ֡۾ȴ۾Ŀӣ˭ﶼһˮ
ðɡҵͷ

أ޸ںԻȴƲҵһ¡
ҾȻЩڣƺеܱԼûȷ׼ǷܵݡͬԼһбˣԼȴ֪ĶʲôԲ
С޸·͸۵˫ĿСһ£ЩĿһ˫Сֿ˵°ڡ漴˫СļͷҲ
Ҫô޸Ŀֹһ£׿
Уҵͺһ޸ƴƺҪһȵ˭åǵ˭
ϣҪôϰɡֻħơ
СֻһӦ
漴뵽ҲǡǺܿ챻ˣΪһ׵ݸޡ
޲û򿪰װ̯һŻֽͼ

д˼ġҡ
޸д˼ġҡ
ƤƤд˼ġҡ
ҿȥˣǩһڵġҡ
˫۾򣬵ȴҪִһӵ
϶ѡԼġ
ƤƤƤڵϱȥҡң㣬ң
޺硪ڲ롣
ȴ΢΢ҡҡͷ˵
ҵСɫ˲ռߵO(_)O
̫ϸˣĻЧáר緢רҵ
һ룬һ֣ӡΪ̣޸ѭѭա
СɫѾСpiaˣһȴҵδüȥϸĶ˵١
Ѿ񰡣ϲüëСϲ۾СǣҲϲСϲ촽ģ
ҵüë۾ӡ촽
䡪ôˣôˣ˵Ķиࡣ

۾ɡĸ˶ȵҪ˯֮³ĸ˶øĿڵһʯͷڷһ
ȵȣƺҪԣ뿪˫۾ŭӣ
ɲ԰ֻ۾ϣr(s_t)q

ҪҽȻͼ
Уʱܡϡҵ¶һ޵Ŀ㾡ɣЧϾһ¡
ǣȵóƷ֮ҲˡʵҲ׷r(s_t)q
ѾǵˡͷһζԼĻɡʵĻܺã
ٲУǾʹˡ޸翴һƣ磬ͨ뺡
ڣʼ̣ǿʼ̡

ҹˡ
ӡˡ
罻һע
ҡƤƤͻؼˡ
˫۾ңҳһ
šϲ޺Ʒһϵ˫۾ں˶һǲϵξԺĶμס˵أ

һ·ķ紵ƤƤصԼСʱѾʱû˯⣬ѵµǰȥԼ
žӣŤŲӿƤƤúϵֺƯO(_)O
ȻһͶԶԽǱҲƤƤϵ֡
򣿡ⲻǸŮ˵Ǳ谧ˡû뵽Լİ·ôĿ뾣
򣿡ǱһӸ¥Ů֣ʲôǷµ𣿡

part23

ˢ롣
ΪմˮO(_)O
Ҳ֣һԵ򣬡ҿ
ȴ󣬱ܹ졣
ôˣҾȻǷݹ쵽ڲʲôֻǲ˵ƺһӳһ̵ˡ
绰ʱ뿪ߡ
šֱȥͻɣǼϴг̡
ˡҴ·֮ǰϸĿ˿ֲڱ򣬡ʲôһҪ˵Ŷ
ˡ
ţҸһ
ܲܡ
绰ˡûǼĿϣġ
ܲܡʱ򣬸֧ѩ⣿
ȻܡЩɻĿ򣬲Ҳǿ˵ﻰҪʲôζģ
ȵİɡ
һЩġҳȥһ߽绰

ӱó֣ŵԼǰ󿴡
һһҲСˮɣǽһ·ϴŹЩһǻǽһ𣬾񻹶Һʱ޽٣ֱһӨı飬ڵϣȴɼꡣ
컹úõһѾǳֻʣµۼǲ㼡Ѿڻ𣿻Щֻܷ
Ȼ£ˣҲˡ
ȥͷԸеֶһЩǲأ
վӣҲڻʵ
⸱ڻãʱĳȡʵԼղʣܲܲȥϰࣿ
С飺Ҫ˵
ѩС飺˵ҡ
С飺ûʲôҡڸϺܱģҲ׷Ҫ
ѩС飺
С飺ûʲôȰѰȷȥ⣬˲裬ҲԲơ
ѩС顪СԭôMAN

ҺϴһͻæݺͬݹǿԴصȥģ֮ҵһЩӦĳɷ棬˫Ҫһչϵ
ʦΪ˾ľþ׷ɹʡԷʱҵĲŸ롣
ҲóⷽӦꡣͷ罻֩ϴæǫһ£ʦᣬ˾ôҵǡ
Ÿ˺Цڰ֣ǧǫ顣ҲǾ˼ǵġرǽռʦֻʦΪأһǺθеˡǺܷġ
·ϣϴϲ̵ݲ½ӵĺðɣΪԱǲҲҪϽһЩأҳԶٷɣҲϻ˵
ҿ˿ϴûκġ
ôˣװǲʲôˣϴͷҵĲԾ
ҿ˿ϴ󣬼Ĭ
ԵϴҲŭʲô鶼档㱻ķɫأҵθС˵ǲǣʯͷҲзģʲôѶƽ֮пȡ֮
ұϴ˵ĻˣƺҪе˵һ㡣
ϴӽԮ˵
ʲôл㣿䰡Ҽɸ֮ǰǸǵĹʱһϴазصİϴ֮°С˵˳Ȼ绹ûȥȫ֪˰ʧϸڨr(s_t)q
ʯͷҲзģʲôѶƽ֮пȡ֮仰Ǳҵȥˡ

Ϊ㣬ҳҹĶ
Һܲ
Ϊܾҹ̫̣糤޷ܡ

ſѩصңһţͿ򡪡ֻǱ(o)
Ҵ󻵵ŻҪųȥ档
ȥһǲ̫ϰֻǱ衣
ȥͷĹ԰ƤƤק֣ͷˣһ˾Ͷˡ
ͷ԰մһƣڰչʣرǽ˽ڣ˴ǰ͡
ôˣƤƤҺ֮䣬춼ơ
һ仰ҲûƤƤʵж֤ȷʵƵʵ
԰ھһƵĹСȣȻ׳ۣʮ־£ƽͬȻһ壬ɫʴĲʵǶз˹ԡУƤƤȥͣķпϣ˵ע⣬Ǩr(s_t)q
ۣǱŶ
ۣҲǱŶ
ۣ㶼ǱŶ
ƤƤÿһͽһ䣬ͬɨһͲһȲ죬ע

򣿡ȴƳĬ
ţ
ǲл˵
΢ԥһ¡ʵѾĬл˵ʵ
˵𣿡ҳ򿿽һ㣬ƤƤѾԶˡr(s_t)q
ܲܡ
˵ҳڴĿ
ܲܡͷôһܲܡϰࣿ
ΪʲôҲȥϰࣿ
ΪʱҲһʱʱ̶̿һ𡣡һȥˡԱ˵ԽǻẦߵʱ򣬾ԽҪνȻ̧ͷ۾ͬҫĻ档
ҵ۾ҲȼС硣
øɲҶǡҪԷȼһ¡ǲԼĸ磬ʹСƦƵľȥеζҲվСƦһԶ۸ϲõŮ˵һԡǸŮҲһʮıߣǣֻСƦװνЦṲ̊ԶȥѧгȻһӵζǣŮO(_)O
ҵŮѰɡ
Ҽ֮Ҫˡ
мŮѡ
ûO(_)O
Сǣ

򡪡򡪡㡪㣡
ҺѭƤƤĺһŷ֣ƤƤǸĽڽǡ
ʦСѣˣһҪͨˡ
ƤƤҡҲ벻ˣǡճˡ
һ֪ô£ԼҲоr(s_t)qɲӲˣϴ˺ü졣
Űɣһ¾ͻöĵطġǧļȥŶֻǹػƤƤһק߽Ⱥ
ʦҴôôΰСѣɣҪͨˡ
ƤƤˣͨͨɣֲ絽ҡ
ʦǣ֮ͨ˹͵ġr(s_t)q
˼ƤƤ̿ʼŬϵĹͷҿֻ촽вסˣһӣƤƤԾȳɹ촽µ߿ˡ
ʦʼͨ磬
οСŮ裬㿴Ǹ̰촽ȥˣźúͰO(_)O

ҺһǱ֮ǰǱСʵǸֱ׵Բٸһⶥѣԣÿֻ˽ȥһӣԻ˽ڳǱ
ȥ˶²֮һδҾҲʼŶӡ
ƤƤȺѰңתǱǡҺŵšǱߵǽһԲεı̨֮ϣ߳ƽһ̨࣬ȥʱЩĿЧӦ
ҡң㣬ңƤƤƴȺＷǼ̨ǰĿųǱŹˡ
Сѣÿֻ˽ȥġԱԴ֮ȵסƤƤ
ңҡƤƤһתҰְ衣
ţԱϸƤƤˣ
ꡣ
ˡԱüǾ͸ҪҪְһЩʱŶO(_)O

ǱȻǱɵģǶʵĵƣȴ͸Ĺ˵ӰӶ
ʲôÿģڲǽһȦȴһЩڴĻ˵Щʲô͵͵һһֱûңĿȴɵƿ
ЦáƯü䣬ǹӲӡߵǺܶԳƵ̣¶ݷ͸Ĺ󣬰ݵ촽ꡣ
Ŀڴ촽ƿŲҿͦıϡıͬһӨ׵ɫȥһ¡
һ˶˼ƫߣȴѾůĴ쵽Ķ£סС漴ëһƮĦĴꡣ
˫ۡ
ѩһʼֻƮƮļѩǵ˿ĵȻѩ˱۾֮ѩƬЩƤҵϣѩ˾ͱŻ׷ֱͨ泩
ѩ죬Ȼֻӡ

ƤƤС֮䣬㣬ǸƳǱûиɻ°ɣ
ֻ߶
˵֣ƤƤǽ߾ȫļ֮ģǻĪı䶯ȻƤƤͱ˱Ϩr(s_t)q
չľͷһͱͼǶС¥С԰רοӰ
ҺվС¥֮ǰƤƤȻǼ֮䡣
ҰַƤƤϷӰʦȻһ¡ӰʦְҵʵˡֻİO(_)O
˼ţƤƤҲŲ൫ˮĻ˺üͨr(s_t)q
򡣡
ţ
ҽԼƫͷȻͱҵ촽׽סӰʦͺϬİ˿š
Ƭȿȡ
ҪƤƤҵֱת
ٿɡҴ󻵵װƬŷŵƤƤĿڴr(s_t)q

һš
뿪ʱϤͷһվڱǰĺɷ
ĺɷʲôͣûзҡ
Ҳźۼתͷǣ֣ϡǹ⽥ʢĵطȥ

part24

СݮҪڿӭĵڶʮ˽ڡ
µΪСʧ᫵ɽ·顣
Сݮжºڱ۾һ۱٣һ衣ҳˣסϾҪˡƺϲ书׶ΣҪƽľչҪѵŹϣУҪץ
ģСݮ˫ֺʮϵı۾߶һ֮ҵǩҳ
ǣСŮܽ׶ܵһСݮҲΪؼʱܵΪϾĿ顪춣һҽ
СݮС˳ܷɹڴһ٣һҪ˽ڵ֮ǰصɸɾĴ״̬

ã˽ںõҪ𣿡Сݮ̧ͦͷЦһ߽ӹùѩ⡣
ע⵽̨Ա߰һϸĵһõ廨
ȤĹ˿ͣҪԽܡĻСݮͷС
һõСݮ˶٣ֻλС㡣õֻСλС㡣
ֻҾԢⶥãرվһСƤƤʱ

ʮٵǰһӣСݮһõ壬Ҳһҳ
һŵһصˡ
ͻȻöö
رǿЩڽϵ˶˫ԡŨ⣬ӼСݮ԰
ǸʲôʱĶⱾʱ򣬾ˡ
Сݮһҳϵһ仰˺þãڰᷭȻ۾˲˾桪ķдһɵС֡Ұ㣬񡣡
ǾĴ
춣һõ壬ͨ죬ŶˣִһЩ
СݮЦĿžӹõŵһ£ʮ䡪İվ
СݮȻЦᡪôɬr(s_t)q

ԣʥҹ꣬Ϊң
š
ѩŻȯҲԱ
š
ܽʱգҲһͷ
š
ôⱾҲǡ
Ŀĵġ
ǣ㰮ʲô
ܶࡣ
жࣿ
ÿ춼Ҫһ㡣ȵľڿʼ״̬ңΪ㣬ڼСӡ
Ŷ
ȻЧǺܺár(s_t)q
üص㡣
ϲ𣿡
ɸֵСݮò̫ˣô˽ڣôʱ򡣻ߣǿһ
Ф÷ͬ־㻹ǵһ仰𣿡
һ䣿
ʧñʧһr(s_t)q

װ˽ڵ
ƤƤįԼҵС¥
˽ڿ֡վ¥ſڵǱתһõ塣
ллƤƤʿĽӹǰδе̬ȿǱһۣʵķͺܲ
𣿡ǱɬˣȴըһͰ׻ԶƵļ֣ʵŴ
û׼ʲô͸һӵɡʧƤƤں˱
ɬǱͷŤһߣ첲ȴɬĽƤƤһ¡
ôůĸоƤƤе
ǣϲ𣿡ǱͷŤ
ƤƤ̧ͷ죬ܵ䣬ǵ顣
ǱȻʧǣƤƤθУһġ
ƤƤûüȰ裬Ǳɬܿˡ
Ӷ¥̨įĵһ̣ӻСʱҡr(s_t)q

ڴϣЩƬһһ飬ֱˮڹ
Ƭϵ򡪡ѩģ۾㽶ͣÿնüį
ס㣬ȴһƬҲס𣿣
һ죬ȴòһӦ
ЩЪ˹ﵽܣֻѪѡ

ˣôˣ
ûʲôҲô˯
˯֮󣬾Ѳˡ
ᰡұס򣬡ڲǺúõ𣿡
˿ҵڻͷڱ̣üëҲڱ̡
ûϵġ
ôûϵ𣿡
š
Ҳûϵ𣿡ģ

򣡡ҽԭجΣ
ǣج̫ʵ
¥ƿķţȴûӰӣϻмϸСı
򣿡ûŲĴߣ𴲵ϵһССıŵ촽ϡ˲ڻҲ֪룬ȻԼԼһơ
㲻ҪߡҴſǻڴǰմʪݡ
ô
ԼӦһֱġ
ҾԼɵϣĻ쵰ͷ֪룬ʼС
ˣ
ţҾϲĻͷվſڣܹȥıסûߣûߣҪ뿪ҡ
ҲجΣһŵȥվӣԼüëѾˣӺܳªܿ¡˿ԼӣͶ㵽ȥһ˵ܽ

üë
šÿԼ۾Ϸ֣¶ͺͺƤ
ָȥûϵҲܺÿ
˲ƭҡ֪ܳ󡣡
һ¡Ұױǰ棬ܣһҪһ¡
Ƭ֮Ҵһֻüʡ
Զűʸϸϸüë𣿡
ױеԼ˾úãСеԼ㲣촽ģҲһƤ΢΢ͷ
ٵһ¡صҡ
ܻػױƷ꣬󴽸࣬Ǯߡ
ױƷϰ塪Ҷλ˿͵Ļ̫O(_)O

ȵָߣÿ֮Өĺۼ
ſһ㡣󴽸࣬ΪͿĨʵҲܺÿ
𣿡һ˵󴽸ͲУǸɰСݶŪþɰ
šпϵĵͷʲôӣҶϲ
á
ҡ
ӡҡŽгڣǵȵ죬ҵͷҲô죿
һǻϲ㡣ңҲǻ⡣Ϊϵ绰֮󣬾ʹ乫˾
һһǾͳ
ȥ
ȥҵļ磬Ҫࡣ
ǣǻᡪȻҲ뾡ܵӳദʱ䣬ǲԸΪ˶ҵңرԼս뿪
ȴû˵ֻ꣬ǿʼѸٵʰװ

һΡ
ĳ水ȣڿﲻ֪롣
дȥ֣߰ɡ
ǡ
ûпǡҼᶨĿ۾Ѿûˡ

ҵļڲԭıԵϣҹĳ600ҡ
ҪߵӲܵҿܹߡ˾ʦһ߽̣ܳҫһ£˽ڣŮ˵ʲôҲҳǲô˵ģ
˾˵ʲôֻһֱЦſȵĿ򿴵ͨ죬һֱ쵽⡣
ǲһô˵ģ˾ʼ
ţڷɢע뷨Ӧһ
ǣǡ˾ʼ˼ˣ˵Ҹһͽ顣
飿䵱
԰Ҫˣ˾ĵĴڣһ·̵ƾ˸O(_)O

ǲԽԽ䣿ҹĵ
š
ҰѴٴһ㡣
͵˾Իˡ
ɣ
š
ҰѴٿһ㡣
˾һԻ˨r(s_t)q
Сһ·򱱣Ӵļڴذѩĺ
ǣܰѴһ𣿡˾ֱ£ڲԭϿɱǵ¡䰡ҪǸðˣŮѻᵣĵġ

̵룬СϢ
ȡ˾߽ϤСꡣ
Ա˾죬ŶֳڼҸŮѹڣ
ⲻҸ˵һͽ飬űɽ𡣡O(_)O
ҺҲ߽С꣬˼֧ѩͳȥˡ
˾Ա۾̫ˣ

ˡ
ҡ
ӡҡЩϰߣҾҺֹͣڻˡ
𣿡Ѿˣûпƣҳ˴գȻȥ
򲻸Ҷһ£ҲҷǰǸ˾һֱŶ䡣
ƫƫʹǵúܴ뵽ĿȥŪСͷ
оԼҪʼ˨r(s_t)q
Сåڴ󷢴ȱļȣ뿪ʱĴ꣬ţֳӯˡO(_)O

part25(happy ending)

ҼҵĴš
ŵӣﳪŸԶһſڵĶӺͶϱ̰ѻһõ廨r(s_t)q
ˡТڨڨ˵
ģۣġȿȣôˣ
ݸꡣ
ܿģֻĿһ䵽ϣ̽ôˣ
µ̫ˡҸﴵĻ
ס֣֪ࡣ
򡪡(o)Ŷ

һҪԺӣͿھӴС߸ϡ
һӣʲô϶ҪЩ統Ƕͬһ飬ȻͬһҽԺӣȺ󶼱˹Ѹ㡣ҲϾģõ壬ƵĿûԱ߻Сڡ
ϣ׷ͷ𣿡ȷˡ
ߨq(s^t)rͷСҵġ
û̧ܣͷ죬ƺӳһ䡰̫ˡ
ܽҲ㣬ע⵽ڣʲôʱģ
ոաͷ
ҰġקקģҴզûأסֲԶ
һغϣתעϣλǣ
ҶŮѣôҶҲҵŮˡ˰˶ȡ
ҵһ㡣

ˣ
ʲô
ҡ
˵ɡ
ŮѺ𣿡(o)
ţûаб۾ܺҡ
ȻҵĻŲɣԼվԺ˴ǰҪ˼˵ҶӵŮѡȻ·˾ͻһ䡰ҵŮǣԻ˨r(s_t)q

ھӴĶӴڵɳŶ˿ղ飬˳㿴ĸס
ںСʱ飬һɹ໵£ͯȤ¡˵ҡڽŮˡ²Ĺ
ĵ˰Ӳ黧ɵĿͷ
ġҺࡣ
ţôûϸ˿򣬡ᰡƯŮǰôƯôӹġ
ҵļУڡʱɲסˣЦҾǱһ򳤵̫Ưˣ˼ѡ
򣬵СɢʱˡҰԱŶ͵СȻ󣬸ڽ̸£ûлں֤һס
ʲôŴͣǺˣ
ʲôˣҲˣ˿ղ鲻Ϊ˾˿Ϣǰζˣ಻㡣
ģ
š
ˣ
ʮ塣СԼոպá
Ա𣿰ϡ
ӡ
򣿡ھˡ
˰ѱҪϢ죬ǵ֮Ĵڵļ򣬡찡֤һʱ첻Ҫ֤ʲôģҼá
ǸǺܸεĹߣܰһһݣҲңǡĲ͵ɹ𣿡
ȭ

СǶȼ٣Ǳ߸ܴĶᡣ
קΪȥμӵ˾ԾȻͨˣõŵһҵͿ
ԺҵˡϴҲڣˡ
С˵֮ϴ֮¡
ΪʲôϴļܲĹʦҪˡ
ʲô
ҪһҵŶȥ˹̸ͬߡڰ
Ҹôûǩ֤
Ǹҵǿˣ˻˵ӷ˸һȥء
ֲǲˡġ
ϴƤЦⲻЦĺߺ߼㼸쿼ʱ䡣

Сס˽һݡ
䣬컹ûӰӡƺһЩüëҲһ㣬ֻǵһ㣬ÿ糿һǻü
鷳ɣЩ˼
ʵϲҲܣȴô˵Ȼ鷳ԣҪ촽O(_)O
δҪѾ˴㣬򽲣Ϊ״ϧ״ÿ춼һͻӦࡰٲûлᡱr(s_t)q

ɹʵҵ밢˹ӵһʯҵиĿȥ˹̸ͬڼ
ԣҪˡұܲᣬܲܡ
Ŀҡ
ܲܡһ˯
ԥˣʵֻ¶ҡ
ұܰˣԺûлˡ
ǣ̵ĵͷ
͵ʮֿġСڰסO(_)O
ҺȻɤźһ䣬裬㲻˽Ŷ
ţ֪
֪͵
ûУֻ·Тӵĵ˨r(s_t)q

ȡͻԽչһҹʵֻʽϡ˶ֵַɣм仹ϸţëǴţһ϶
ָ֪µı仯ԽԽ˯֮ǰ̫O(_)O

ڶһ磬ҴȥһУɻ֤ͻƱѾݵҵУ˹ٺ͹˾̸Ŷӻϡ
Ҳȥ𣿡ȻԼϵˡ
Ȼ
첻ǡ
ֻ˵Ҫˣû˵㲻߰Ҿĳ䵱ֽϻ

˹ӵ䡣رǣǼʯ͹˾ڱȦ
ż¸ҵİ˹Ħˡ˵ǵȾйıˣһǰǨȥġ
̸неú˳˫ˬİˣҪһ֮󣬸ԶڴҪĵطһòͬǩˡ
֮мĹ۹ʱ䡣Һ̸ŶȥO(_)O

һ۲ⱱλþѵСҺСԵһСľݡСľΧԶûн·
ҹĻ٣ѩĭƮ裬֮һƬãҴŹĹĵ޴£Դë׵ñϴźףȻĵȹڷԶԶЩŮ⾳
һ仰ҰñӿקקԼ¶͡
֪һ䡣
ŶҾˣԭз۾һӵɵúܴ
קקģѧž缯Ů˦˦ͷķ˿ĿԶԼɽͻƶҵϣֻ
䣡ҾԼһԲˣСˣĿļϡ

վһ
üëСӲˡ
СǶ㵽۱
еСҲʧȥ˵ԡ
ǡβǡСƦӡСåһС鶼ů͵ĵطϳ
ҵһյıֻСɫľ黹ظλ߶µͷԵͷӭƮ
򡪡ڷҪЩ˵ҡáܡҡͷҲѾӲ˨r(s_t)q
ѾӲףǽȥһɡ
ûϸҲܡʵǡû£Ҳ䡣

СľһֵֻͲ
мªǸȡůʩСľﻹů˼Ӳȥһë̺
ɫë̺ľϣȥק򣬡Ҳë̺СɫĻռߵبr(s_t)q

˺ö飬ҹգʱҲոùʮһ£õǰס
Һɫë̺˷ܶĿʴɫĻûϷ·һյ˶Ĳʵһա
ڣһͬŵĻӯƮʡıԵʱʱ̣һ㡣֮󣬹㣬Ϸתļⶥ
Ưҵı﷢ֱ̾۲Խ̧
ҪҰֱȦסָģߡ
ţ
ûϵĴ˵𣿡ҰѶĴβС˳Ļ棬˵ߣԣҪָҳûֱսһ֡

𣿡ҺСľϣɵɵʡ
šʹĵͷ
벻һ
𣿡(o)
Ͳڻˡ
ԸڻҲһ˴
ôգգ۾ΣҲֻˡ
ǴʡԺš
ӴˣҺҸֵһ
ǵһǺĶ죬һǸġ족й뱱֮תÿһΪݱ𱱼򶼻ڻɬһ䣺ǵѩأҡҾͻ޷˵ķļ

ãصС˵ www.27txt.com ӭȥŶ
վԴתԻ˲κθ,֧棬ȨУ
ȨУغ24Сʱɾñ鲻빺鼮лߵ֧֣