AlphaGoè¿ä¸ªç³»ç»ä¸»è¦ç±å 个é¨åç»æï¼
èµ°æ£ç½ç»(Policy Network)ï¼ç»å®å½åå±é¢ï¼é¢æµ/éæ ·ä¸ä¸æ¥çèµ°æ£ã
å¿«éèµ°å(Fast rollout)ï¼ç®æ å1ä¸æ ·ï¼ä½å¨éå½çºç²èµ°æ£è´¨éçæ¡ä»¶ä¸ï¼é度è¦æ¯1å¿«1000åã
ä¼°å¼ç½ç»(Value Network)ï¼ç»å®å½åå±é¢ï¼ä¼°è®¡æ¯ç½èè¿æ¯é»èã
èç¹å¡ç½æ æç´¢(Monte Carlo Tree Searchï¼MCTS)ï¼æ以ä¸è¿ä¸ä¸ªé¨åè¿èµ·æ¥ï¼å½¢æä¸ä¸ªå®æ´çç³»ç»ã
æ们çDarkForeståAlphaGoåæ ·æ¯ç¨4æ建çç³»ç»ãDarkForestè¾AlphaGoèè¨ï¼å¨è®ç»æ¶å 强äº1ï¼èå°äº2å3ï¼ç¶å以å¼æºè½¯ä»¶Pachiç缺ççç¥ (default policy)é¨åæ¿ä»£äº2çåè½ã以ä¸ä»ç»ä¸åé¨åã
1ãèµ°æ£ç½ç»
èµ°æ£ç½ç»æå½åå±é¢ä½ä¸ºè¾å
¥ï¼é¢æµ/éæ ·ä¸ä¸æ¥çèµ°æ£ãå®çé¢æµä¸åªç»åºæ强çä¸æï¼èæ¯å¯¹æ£çä¸ææå¯è½çä¸ä¸çç»ä¸ä¸ªåæ°ãæ£çä¸æ361个ç¹ï¼å®å°±ç»åº361个æ°ï¼å¥½æçåæ°æ¯åæè¦é«ã
DarkForestå¨è¿é¨åæåæ°ï¼éè¿å¨è®ç»æ¶é¢æµä¸æ¥èéä¸æ¥ï¼æé«äºçç¥è¾åºçè´¨éï¼åä»ä»¬å¨ä½¿ç¨å¢å¼ºå¦ä¹ è¿è¡èªæ对å±åå¾å°çèµ°æ£ç½ç»(RL network)çææç¸å½ãå½ç¶ï¼ä»ä»¬å¹¶æ²¡æå¨æåçç³»ç»ä¸ä½¿ç¨å¢å¼ºå¦ä¹ åçç½ç»ï¼èæ¯ç¨äºç´æ¥éè¿è®ç»å¦ä¹ å°çç½ç»(SL network)ï¼çç±æ¯RL networkè¾åºçèµ°æ£ç¼ºä¹ååï¼å¯¹æç´¢ä¸å©ã
æææçæ¯å¨AlphaGo为äºé度ä¸çèèï¼åªç¨äºå®½åº¦ä¸º192çç½ç»ï¼è并没æ使ç¨æ好ç宽度为384çç½ç»(è§å¾2(a))ï¼æ以è¦æ¯GPUæ´å¿«ä¸ç¹(æè
æ´å¤ä¸ç¹)ï¼AlphaGoè¯å®æ¯ä¼åå¾æ´å¼ºçã
æè°ç0.1ç§èµ°ä¸æ¥ï¼å°±æ¯çº¯ç²¹ç¨è¿æ ·çç½ç»ï¼ä¸åºææé«ç½®ä¿¡åº¦çåæ³çæ³ãè¿ç§åæ³ä¸ç¹ä¹æ²¡æåæç´¢ï¼ä½æ¯å¤§å±è§é常强ï¼ä¸ä¼é·å
¥å±é¨ææä¸ï¼è¯´å®å»ºæ¨¡äºâæ£æâä¸ç¹ä¹æ²¡æéãæ们æDarkForestçèµ°æ£ç½ç»ç´æ¥æ¾ä¸KGSå°±æ3dçæ°´å¹³ï¼è®©ææ人é½æå¹äºä¸ã
å¯ä»¥è¯´ï¼è¿ä¸æ³¢å´æ£AIççªç ´ï¼ä¸»è¦å¾çäºèµ°æ£ç½ç»ççªç ´ãè¿ä¸ªå¨ä»¥åæ¯ä¸å¯æ³åçï¼ä»¥åç¨çæ¯åºäºè§åï¼æè
åºäºå±é¨å½¢ç¶åå ä¸ç®å线æ§åç±»å¨è®ç»çèµ°åçææ³ï¼éè¦æ
¢æ
¢è°åæ°å¹´ï¼ææè¿æ¥ã
å½ç¶ï¼åªç¨èµ°æ£ç½ç»é®é¢ä¹å¾å¤ï¼å°±æ们å¨DarkForestä¸çå°çæ¥è¯´ï¼ä¼ä¸é¡¾å¤§å°æ è°äºå«ï¼ä¼æ è°è±å
ï¼ä¸é¡¾å±é¨æ»æ´»ï¼å¯¹æåºéï¼ççãæç¹åé«æä¸ç»è®¤çæèçéææ£ãå 为走æ£ç½ç»æ²¡æä»·å¼å¤æåè½ï¼åªæ¯åâç´è§âå¨ä¸æ£ï¼åªæå¨å äºæç´¢ä¹åï¼çµèææä»·å¼å¤æçè½åã
2ãå¿«éèµ°å
é£æäºèµ°æ£ç½ç»ï¼ä¸ºä»ä¹è¿è¦åå¿«éèµ°åå¢ï¼æ两个åå ï¼é¦å
èµ°æ£ç½ç»çè¿è¡é度æ¯æ¯è¾æ
¢çï¼AlphaGo说æ¯3毫ç§ï¼æ们è¿éä¹å·®ä¸å¤ï¼èå¿«éèµ°åè½åå°å å¾®ç§çº§å«ï¼å·®äº1000åãæ以å¨èµ°æ£ç½ç»æ²¡æè¿åçæ¶å让CPUä¸é²çå
æ索起æ¥æ¯å¾éè¦çï¼çå°ç½ç»è¿åæ´å¥½ççæ³åï¼åæ´æ°å¯¹åºççæ³ä¿¡æ¯ã
å
¶æ¬¡ï¼å¿«éèµ°åå¯ä»¥ç¨æ¥è¯ä¼°çé¢ãç±äºå¤©ææ°åè¬çå¯è½å±é¢æ°ï¼å´æ£çæç´¢æ¯æ¯«æ å¸æèµ°å°åºçï¼æç´¢å°ä¸å®ç¨åº¦å°±è¦å¯¹ç°æå±é¢å个估åãå¨æ²¡æä¼°å¼ç½ç»çæ¶åï¼ä¸åå½è±¡å¯ä»¥éè¿ç®æ£åçåæ°æ¥å¯¹çé¢åæ¯è¾ç²¾ç¡®çä¼°å¼ï¼å´æ£çé¢ç估计å¾è¦éè¿æ¨¡æèµ°åæ¥è¿è¡ï¼ä»å½åçé¢ä¸è·¯èµ°å°åºï¼ä¸èèå²è·¯å°ç®åºèè´ï¼ç¶åæèè´å¼ä½ä¸ºå½åçé¢ä»·å¼çä¸ä¸ªä¼°è®¡ã
è¿éæ个éè¦æè¡¡çå°æ¹ï¼å¨åçæ¶é´ä¸ï¼æ¨¡æèµ°åçè´¨éé«ï¼å次估å¼ç²¾åº¦é«ä½èµ°åé度æ
¢ï¼æ¨¡æèµ°åé度快ä¹è³ä½¿ç¨éæºèµ°åï¼è½ç¶å次估å¼ç²¾åº¦ä½ï¼ä½å¯ä»¥å¤æ¨¡æå 次ç®å¹³åå¼ï¼æææªå¿
ä¸å¥½ãæ以说ï¼å¦ææä¸ä¸ªè´¨éé«åé度快çèµ°åçç¥ï¼é£å¯¹äºæ£åçæé«æ¯é常æ帮å©çã
为äºè¾¾å°è¿ä¸ªç®æ ï¼ç¥ç»ç½ç»ç模åå°±æ¾å¾å¤ªæ
¢ï¼è¿æ¯è¦ç¨ä¼ ç»çå±é¨ç¹å¾å¹é
(local pattern matching)å 线æ§åå½(logistic regression)çæ¹æ³ï¼è¿åæ³è½ç¶ä¸æ°ä½é常好使ï¼å ä¹ææç广åæ¨èï¼ç«ä»·æåï¼æ°é»æåºï¼é½æ¯ç¨çå®ã
ä¸æ´ä¸ºä¼ ç»çåºäºè§åçæ¹æ¡ç¸æ¯ï¼å®å¨å¸çº³äºä¼å¤é«æ对å±ä¹åå°±å
·å¤äºç¨æ¢¯åº¦ä¸éæ³èªå¨è°åçè½åï¼æ以æ§è½æé«èµ·æ¥ä¼æ´å¿«æ´çå¿ãAlphaGoç¨è¿ä¸ªåæ³è¾¾å°äº2å¾®ç§çèµ°åé度å24.2%çèµ°ååç¡®çã24.2%çæææ¯è¯´å®çæ好é¢æµåå´æ£é«æçä¸åæ0.242çæ¦çæ¯éåçï¼ç¸æ¯ä¹ä¸ï¼èµ°æ£ç½ç»å¨GPUä¸ç¨2毫ç§è½è¾¾å°57%çåç¡®çãè¿éï¼æ们就çå°äºèµ°åé度å精度çæè¡¡ã
åè®ç»æ·±åº¦å¦ä¹ 模åä¸åï¼å¿«éèµ°åç¨å°äºå±é¨ç¹å¾å¹é
ï¼èªç¶éè¦ä¸äºå´æ£çé¢åç¥è¯æ¥éæ©å±é¨ç¹å¾ã对æ¤AlphaGoåªæä¾äºå±é¨ç¹å¾çæ°ç®(è§Extended Table 4)ï¼è没æ说æç¹å¾çå
·ä½ç»èãææè¿ä¹å®éªäºä»ä»¬çåæ³ï¼è¾¾å°äº25.1%çåç¡®çå4-5å¾®ç§çèµ°åé度ï¼ç¶èå
¨ç³»ç»æ´åä¸æ¥å¹¶æ²¡æå¤ç°ä»ä»¬çæ°´å¹³ã
ææè§ä¸24.2%并ä¸è½å®å
¨æ¦æ¬ä»ä»¬å¿«éèµ°åçæ£åï¼å 为åªè¦èµ°éå
³é®çä¸æ¥ï¼å±é¢å¤æå°±å®å
¨é误äºï¼èå¾2(b)æ´è½ä½ç°ä»ä»¬å¿«éèµ°å对çé¢å½¢å¿ä¼°è®¡ç精确度ï¼è¦è½è¾¾å°ä»ä»¬å¾2(b)è¿æ ·çæ°´åï¼æ¯ç®åå°å¹é
24.2%è¦åæ´å¤çå·¥ä½ï¼èä»ä»¬å¹¶æªå¨æç« ä¸å¼ºè°è¿ä¸ç¹ã
å¨AlphaGoæäºå¿«éèµ°åä¹åï¼ä¸éè¦èµ°æ£ç½ç»åä¼°å¼ç½ç»ï¼ä¸åå©ä»»ä½æ·±åº¦å¦ä¹ åGPUç帮å©ï¼ä¸ä½¿ç¨å¢å¼ºå¦ä¹ ï¼å¨åæºä¸å°±å·²ç»è¾¾å°äº3dçæ°´å¹³(è§Extended Table 7åæ°ç¬¬äºè¡)ï¼è¿æ¯ç¸å½å害çäºãä»»ä½ä½¿ç¨ä¼ ç»æ¹æ³å¨åæºä¸è¾¾å°è¿ä¸ªæ°´å¹³çå´æ£ç¨åºï¼é½éè¦è±è´¹æ°å¹´çæ¶é´ãå¨AlphaGoä¹åï¼Aja Huangæ¾ç»èªå·±åè¿é常ä¸éçå´æ£ç¨åºï¼å¨è¿æ¹é¢ç¸ä¿¡æ¯æå¾å¤ç积累çã
3ãä¼°å¼ç½ç»
AlphaGoçä¼°å¼ç½ç»å¯ä»¥è¯´æ¯é¦ä¸æ·»è±çé¨åï¼ä»Fig 2(b)åExtended Table 7æ¥çï¼æ²¡æå®AlphaGoä¹ä¸ä¼åå¾å¤ªå¼±ï¼è³å°è¿æ¯ä¼å¨7d-8dçæ°´å¹³ãå°äºä¼°å¼ç½ç»ï¼ç级åå°äº480åï¼ä½æ¯å°äºèµ°æ£ç½ç»ï¼ç级åå°±ä¼å°æ800è³1000åãç¹å«æææçæ¯ï¼å¦æåªç¨ä¼°å¼ç½ç»æ¥è¯ä¼°å±é¢(2177)ï¼é£å
¶ææè¿ä¸ååªç¨å¿«éèµ°å(2416)ï¼åªæå°ä¸¤ä¸ªåèµ·æ¥æææ´å¤§çæé«ã
æççæµæ¯ï¼ä¼°å¼ç½ç»åå¿«éèµ°å对çé¢ä¼°è®¡æ¯äºè¡¥çï¼å¨æ£å±ä¸å¼å§æ¶ï¼å¤§å®¶ä¸å¾æ¯è¾åæ°ï¼ä¼°å¼ç½ç»ä¼æ¯è¾éè¦ï¼ä½å¨æå¤æçæ»æ´»ææ¯å¯¹ææ¶ï¼éè¿å¿«éèµ°åæ¥ä¼°è®¡çé¢å°±åå¾æ´éè¦äºãèèå°ä¼°å¼ç½ç»æ¯æ´ä¸ªç³»ç»ä¸æé¾è®ç»çé¨å(éè¦ä¸åä¸å±èªæ对å±)ï¼æçæµå®æ¯ææååºæ¥å¹¶ä¸ææå¯è½è½è¿ä¸æ¥æé«çã
å
³äºä¼°å¼ç½ç»è®ç»æ°æ®ççæï¼å¼å¾æ³¨æçæ¯æç« ä¸çéå½å°åé¨åãä¸èµ°æ£ç½ç»ä¸åï¼æ¯ä¸çæ£åªåä¸ä¸ªæ ·æ¬æ¥è®ç»ä»¥é¿å
è¿æåï¼ä¸ç¶å¯¹åä¸å¯¹å±èè¨è¾å
¥ç¨æä¸åèè¾åºé½ç¸åï¼å¯¹è®ç»æ¯é常ä¸å©çãè¿å°±æ¯ä¸ºä»ä¹éè¦ä¸åä¸å±ï¼èéä¸åä¸ä¸ªçé¢çåå ã对äºæ¯å±èªæ对å±ï¼åæ ·æ¬æ¯å¾æ讲究çï¼å
ç¨SL networkä¿è¯èµ°æ£çå¤æ ·æ§ï¼ç¶åéæºèµ°åï¼åçé¢ï¼ç¶åç¨æ´ç²¾ç¡®çRL networkèµ°å°åºä»¥å¾å°ææ£ç¡®çèè´ä¼°è®¡ãå½ç¶è¿æ ·åçæææ¯ç¨åä¸ç½ç»ç¸æ¯å¥½å¤å°ï¼æä¸å¥½è¯´ã
ä¸ä¸ªè®©æåæçå°æ¹æ¯ï¼ä»ä»¬å®å
¨æ²¡æåä»»ä½å±é¨æ»æ´»/对æåæï¼çº¯ç²¹æ¯ç¨æ´åè®ç»æ³è®ç»åºä¸ä¸ªç¸å½ä¸éçä¼°å¼ç½ç»ãè¿å¨ä¸å®ç¨åº¦ä¸è¯´æ深度å·ç§¯ç½ç»(DCNN)æèªå¨å°é®é¢å解æåé®é¢ï¼å¹¶åå«è§£å³çè½åã
å¦å¤ï¼æçæµä»ä»¬å¨åè®ç»æ ·æ¬æ¶ï¼å¤å®æç»èè´ç¨çæ¯ä¸å½è§åãæ以说ä¸æåæä¸ç³å¯¹å±çæ¶åä¹è¦æ±ç¨ä¸å½è§åï¼ä¸ç¶å¦ææ¢æå«çè§åï¼å°±éè¦éæ°è®ç»ä¼°å¼ç½ç»(è½ç¶æ估计ç»æå·®è·ä¸ä¼å¤ªå¤§)ãè³äºä¸ºä»ä¹ä¸å¼å§å°±ç¨çä¸å½è§åï¼æççæµæ¯ç¼ç¨é常æ¹ä¾¿(æå¨åDarkForestçæ¶åä¹æ¯è¿æ ·è§å¾ç)ã
4ãèç¹å¡ç½æ æç´¢
è¿é¨ååºæ¬ç¨çæ¯ä¼ ç»æ¹æ³ï¼æ²¡æ太å¤å¯ä»¥è¯è®ºçï¼ä»ä»¬ç¨çæ¯å¸¦å
éªçUCTï¼å³å
èèDCNN认为æ¯è¾å¥½ççæ³ï¼ç¶åçå°æ¯ä¸ªçæ³æ¢ç´¢æ¬¡æ°å¤äºï¼éæ©æ´ç¸ä¿¡æ¢ç´¢å¾æ¥çèçå¼ãèDarkForeståç´æ¥éäºDCNNæ¨èçå3ææ¯å5ççæ³è¿è¡æç´¢ãæåæ¥è¯éªä¸æ¥ææå·®ä¸å¤ï¼å½ç¶ä»ä»¬çåæ³æ´çµæ´»äºï¼å¨å
许使ç¨å¤§éæ索次æ°çæ
åµä¸ï¼ä»ä»¬çåæ³å¯ä»¥æ¾å°ä¸äºDCNN认为ä¸å¥½ä½å´å¯¹å±é¢è³å
³éè¦ççæ³ã
ä¸ä¸ªæ趣çå°æ¹æ¯å¨æ¯æ¬¡æç´¢å°å¶åèç¹æ¶ï¼æ²¡æç«å³å±å¼å¶åèç¹ï¼èæ¯çå°è®¿é®æ¬¡æ°å°è¾¾ä¸å®æ°ç®(40)æå±å¼ï¼è¿æ ·é¿å
产ç太å¤çåæ¯ï¼åæ£æç´¢ç注æåï¼ä¹è½èçGPUçå®è´µèµæºï¼åæ¶å¨å±å¼æ¶ï¼å¯¹å¶èç¹ççé¢ä¼°å¼ä¼æ´åç¡®äºãé¤æ¤ä¹å¤ï¼ä»ä»¬ä¹ç¨äºä¸äºæå·§ï¼ä»¥å¨æç´¢ä¸å¼å§æ¶ï¼é¿å
å¤ä¸ªçº¿ç¨åæ¶æç´¢ä¸è·¯ååï¼è¿é¨åæ们å¨DarkForestä¸ä¹æ³¨æå°äºï¼å¹¶ä¸åäºæ¹è¿ã
5ãæ»ç»
æ»çæ¥è¯´ï¼è¿æ´ç¯æç« æ¯ä¸ä¸ªç³»ç»æ§çå·¥ä½ï¼èä¸æ¯ä¸ä¸¤ä¸ªå°ç¹æäºçªç ´å°±è½è¾¾å°çèå©ãå¨æåèåï¼æ¯ä½è
们ï¼ç¹å«æ¯ä¸¤ä½ç¬¬ä¸ä½è
David SilveråAja Huangï¼å¨å士é¶æ®µåæ¯ä¸ä»¥åäºå¹´ä»¥ä¸ç积累ï¼éä¸æä¸å¤æè½å®æçãä»ä»¬è½ååºAlphaGo并享æç°å¨çè£èªï¼æ¯å®è³åå½çã
ä»ä»¥ä¸åæä¹å¯ä»¥çåºï¼ä¸ä¹åçå´æ£ç³»ç»ç¸æ¯ï¼AlphaGoè¾å°ä¾èµå´æ£çé¢åç¥è¯ï¼ä½è¿è¿æªè¾¾å°éç¨ç³»ç»çç¨åº¦ãèä¸æ£æå¯ä»¥å¨çè¿äºå¯¥å¯¥å å±ä¹åæç½å¯¹æçé£æ ¼å¹¶éåç¸åºçç¥ï¼ä¸ä½èµæ·±æ¸¸æç©å®¶ä¹å¯ä»¥å¨ç©ä¸ä¸ªæ°æ¸¸æå 次åå¾å¿«ä¸æï¼ä½å°ç®å为æ¢ï¼äººå·¥æºè½ç³»ç»è¦è¾¾å°äººç±»æ°´å¹³ï¼è¿æ¯éè¦å¤§éæ ·æ¬çè®ç»çãå¯ä»¥è¯´ï¼æ²¡æåå¹´æ¥ä¼å¤æ£æå¨å´æ£ä¸ç积累ï¼å°±æ²¡æå´æ£AIçä»å¤©ã
温馨提示:答案为网友推荐,仅供参考