• The TAO team
• The Artelys company
• National University of Tainan (Taiwan)

• O. Teytaud (coordinator)
• A. Renaud + engineers from Artelys
• A. Couetoux (funded by the Iomca project)
• Related people from other projects:
  • N. Sokolovska (funded by the Mash project)
  • J.-B. Hoock (funded by the Mash project)
  • A. Takahashi

Since the pioneering work by R. Coulom (2006), Monte-Carlo Tree Search is an important tool for discrete time control. Nonetheless, there are clear weaknesses to be corrected by further work.

Goals:

• Improve MCTS methods, in particular by the use of ontologies/machine learning.
• Investigate MCTS methods, in particular for one-player optimizing decisions in front of random processes.
• Experiment MCTS methods, in particular in the field of energy.

Schedule T0: January 2010 T0+6 report T0+10 months: simulators ok (in particular: realistic energy management simulators, mainly Artelys) one of them can be seen here. others are available on the git archive of Mash or can be requested by email. Interfacing with the Risk Manager simulator done. UCT in continuous domains T0+19 months: optimizers ok (Tao + Artelys) MCTS developped by A. Couetoux and O. Teytaud (can be requested by email or found in the Mash git) Q-learning developped by N. Sokolovska (October 2010) Decision stump developped by J.-B. Hoock (November 2010) Interfacing with the OpenDP platform (recompilation: October 2010; interfacing in progress) T0+30 months: introducing ontologies/machine learning into MCTS Continuous Rapid-Action-Value-Estimates Application of MCTS for hydroelectric stocks January 15th: the Inria associate team "indema" is passed Franco-Taiwanese forum on Energy Management probably in May 2012 T0+36 months: publications T0+42 months: valorization Already done: Generalities random processes (A. Takahashi) introduction into the Mash git repository ( mash-project.eu ) Interfacing the Mash problems with our solvers (J.-B. Hoock, October 2010) Solvers preliminary MCTS solver (A. Couetoux, O. Teytaud). various simple solvers for comparison (J.-B. Hoock, N. Sokolovska, O. Teytaud). a decision stump solver (learning by direct policy search; J.-B. Hoock). preliminary Q-learning implementation (N. Sokolovska) OpenDP solvers are under integration (O. Teytaud; first integration November 2010). Testbeds Swing problem (A. Takahashi, O. Teytaud) Stock management problem (Artelys + A. Couetoux + O. Teytaud) Energy management problem (but: not yet interfaced with the rest of the platform) OpenDP problems are under integration (O. Teytaud). Coming soon: Introduction of the energy management problem into the platform Introducing parts of the OpenDP projects Events: Project description Franco-Taiwanese days of artificial intelligence (10th - 13th of August, 2010) (followed by the computer-Go event at WCCI 2010 in which MoGo won the first 13x13 game with H2 against a 6D and was the only computer winning against the top level human player Chun-Hsun Chou 9P - however, mogo lost his two games against Ping-Chiang Chou (4P) :-( ) Signature of the consortium agreement as shown here Talk given at London, 2010, for the MCTS workshop ( slides ) October 2010: MCTS for one player games operational (main author: A. Couetoux). October 2010: Experiments of the MCTS on the stock problem. Comparison with Q-learning (from N. Sokolovska; uses also the baseline solvers by J.-B. Hoock). November 2010: The Franco-Taiwanese team wins the 9x9 Gold medal, the 13x13 Gold medal, and the 19x19 Gold medal in the TAAI Computer-Go competition. Technically speaking, the collaboration is based on introducing expert knowledge through ontologies both Automatically learnt on clusters; Through human advices. November 2010: acceptance of the LION paper on UCT in continuous domains. January 2011: Paper on MCTS for power systems accepted in LION2011. minutes of the meeting (january the 6th) March 2011: Blind value techniques in progress (inspired by the work by S. Bubeck and coauthors). Should start soon: long horizons. April 2011: Fitted Q-iteration looks like a great algorithm. Maybe we should consider integrating it in the plateforme ? Usefull also for the Mash project. April 2011: the ``Systematic'' presentation paper. Minutes of the May 2011 meeting (May 2nd) Minutes of the June 2011 meeting (June 8th) Minutes of the July 2011 meeting (July 5th) Minutes of August 25 meeting, part 1 Minutes of August 25 meeting, part_2 Minutes of September 6th meeting Publications UCT in continuous domains January 2010: paper on MCTS for power systems accepted in Roadef 2011. February 2010: the Roadef paper on MCTS for stock management is preselected for student award.

---

---

---

More informations ? email me

Photos ? see here.