The system of emotional appraisal based on reinforcement learning and bio-inspired methods

Article Sidebar

PDF (Русский)
Published: 22.07.2016
UDC 004.81 004.853 004.855 159.942
Issue
Vol. 19 No. 3 (2016)

Main Article Content

Евгения Юрьевна Майорова
Kazan (Volga Region) Federal University
Максим Олегович Таланов
Kazan (Volga Region) Federal University
Роберт Лоу
University of Gothenburg

Abstract

I research and lecture in Cognitive Science where my particular interest is in emotions – neural networks modeling and applications – and animal and human learning.

Article Details

Author Biographies

Евгения Юрьевна Майорова

Bachelor of Higher School Information technologies and systems of Kazan Federal University.

Максим Олегович Таланов

PhD, head of Machine Cognition Lab in Higher Institute of Information Technology in Kazan (Volga Region) Federal University.

Роберт Лоу

The docent and research leader at the ICE Lab and at the Interaction Lab at the University of Gothenburg.

References

1. Максим Таланов. Эмоциональные вычисления. URL: http://postnauka. ru/video/45297.
2. Lowenstein G., Lerner J.S. The role of affect in decision-making // In R. Davidson, K. Scherer, H. Goldsmith (Eds.) Handbook of Affective Science. New York: Oxford University Press, 2003. P. 619–642.
3. Максим Таланов. Эмоциональный искусственный интеллект. URL: http://postnauka.ru/video/45296.
4. Tom Ziemke, Robert Lowe. On the Role of Emotion in Embodied Cognitive Architectures: From Organisms to Robots. Springer Science+Business Media, LLC 2009. P. 71–73.
5. David Sander, Didier Grandjean, Klaus R. Scherer. A systems approach to appraisal mechanisms in emotion. Geneva Emotion Research Group, Department of Psychology, University of Geneva, 2005. P. 140–148.
6. Petta P. The role of emotion in a tractable architecture for situated cognizers // In: Trappl R., Petta P., Payr S. Eds. Emotions in Humans and Artifacts. Cambridge, MA: MIT Press, 2003. P. 87–88.
7. Minsky Marvin. The Emotion Machine: Commonsense Thinking, Artifiial Intelligence, and the Future of the Human Mind. Simon and Schuster, 2007. P. 256–258.
8. Wörgötter F., Porr B. Temporal Sequence Learning, Prediction, and Control – a Review of different models and their relation to biological mechanisms. Department of Psychology, University of Stirling, 2005. P. 45.
9. Ortony A., Norman D., Revelle W. Affect and proto-affect in effective functioning // In: Fellous J-M, Arbib M.A., Eds. Who need emotions? New York: Oxford University Press, 2005.
10. Damasio A.R. The feeling of what happens: body, emotion and the making of consciousness. Heinemann: London, 1999. 400 p.
11. Rolls E. Emotion explained. Oxford: Oxford University Press, 2005.
12. Phelps E. Emotion and cognition: Insights from studies of the human amygdala // Annu. Rev. Psychol. 2006. V. 57. P. 27–53.
13. Scherer K.R., Ekman P. On the nature and function of emotion: a component process approach // In: Approaches to Emotion. Hillsdale, N.J.: Lawrence Erlbaum, 1984. P. 293–317.
14. Paulus Martin P., Angela J.Yu. Emotion and decision-making: affect-driven belief systems in anxiety and depression // Trends in Cognitive Sciences. September 2012. V. 16, No 9. P. 476–483.
15. Kahneman D., Tversky A. Prospect theory: an analysis of decision under risk // Econometrica. 1979. V. 47. P. 263–291.
16. Mukherjee K. A dual system model of preferences under risk // Psychol. Rev. 2010. V. 117. P. 243–255.
17. Hsee C.K., Rottenstreich Y. Music, pandas, and muggers: on the affective psychology of value // J. Exp. Psychol. Gen. 2004. V. 133. P. 23–30.
18. Kusev P., van Schaik P. Preferences under risk: content-dependent behavior and psychological processing //Front. Psychol. 2011. V. 2. P. 269–271.
19. Breazeal C. Designing sociable robots. Cambridge, MA: MIT Press, 2002. 244 p.
20. Kelley A.E. Neurochemical networks encoding emotion and motivation: An evolutionary perspective // In: Fellous J-M., Arbib M.A., Eds. Who needs emotions? The brain meets the robot. New York: Oxford University Press, 2005.
21. Max Talanov, Jordi Vallverdu, Salvatore Distefano, Manuel Mazzara, Radhakrishnan Delhibabu. neuromodulating cognitive architecture: towards biomimetic emotional AI // Advanced Information Networking and Applications (AINA), 2015 IEEE 29th International Conference. P. 587–592.
22. Аллахвердов В.М., Богданова С.И. и др. Психология: учеб. / отв. ред. А.А. Крылов. М.: Проспект, 2005. С. 214–217.
23. Vernon David. Artificial Cognitive Systems: a Primer. The MIT Press Cambridge, Massachusetts London, England, 2014. 288 p.
24. McCarthy J., Hayes P.J. Some philosophical problems from the standpoint of artificial intelligence at the Wayback Machine //In: Meltzer B., Michie D., Eds. Machine Intelligence. Edinburgh: Edinburgh University Press, 1969. No 4. P. 463–502 (archived August 25, 2013).
25. Таланов Максим. Марвин Минский и эмоциональные машины. URL: https://postnauka.ru/faq/58727.
26. Lövheim H. A new three-dimensional model for emotions and monoamine neurotransmitters // Med Hypotheses. 2012. V. 8. P. 341–348.
27. Tomkins S. Affect theory // In: P. Ekman, W. Friesen, P. Ellsworth, Eds. Emotions in the Human Face. Cambridge: Cambridge University Press, 1982. P. 355–395.
28. Smith Craig A., Lazarus Richard S. Emotion and Adaptation // In: L.A. Pervin, Ed. Handbook of Personality: Theory and Research. New York: Guilford, 1990. P. 609–637.
29. Lazarus Richard S. Progress on a cognitive-motivational-relational theory of emotion // American Psychologist. 1991. V. 46, No 8. P. 819–834.
30. Niv Yael. Reinforcement learning in the brain // Psychology Department & Princeton Neuroscience Institute, Princeton University, 2009.
31. Barto A.G. Adaptive critic and the basal ganglia // In J.C. Houk, J.L. Davis, D.G. Beiser, Eds. Models of information processing in the basal ganglia. Cambridge: MIT Press, 1995. P. 215–232.
32. Schultz W., Dayan P., Montague P.R. A neural substrate of prediction and reward // Science. 1997. No 275. P. 1593–1599.
33. Wickens J.R., Kotter R. Cellular models of reinforcement // In: J.C. Houk, J.L. Davis, D.G. Beiser, Eds. Models of information processing in the basal ganglia. MIT Press, 1995. P. 187–214.
34. Barto A.G., Sutton R.S., Watkins C.J.C.H. Learning and sequential decision making // In: M. Gabriel, J. Moore, Eds. Learning and computational neuroscience: Foundations of adaptive networks. Cambridge, MA: MIT Press, 1990. P. 593–602.
35. Bertsekas D.P., Tsitsiklis J.N. Neuro-dynamic programming. Athena Sc., Scientific, 1996. 512 p.
36. Sutton R.S., Barto A.G. Reinforcement Learning. An Introduction. Bradford Books, MIT Press, Cambridge, MA, 2002 edition, 1998. 320 p.
37. Bellman R.E. Dynamic Programming. Princeton: Princeton University Press, 1957. 392 p.
38. Sutton R.S. Learning to predict by the methods of temporal differences // Machine Learning. August 1988. V. 3, Issue 1. P. 9–44.
39. Sutton R.S. Generalization in reinforcement learning: successful examples using sparse coarse coding // In: D.S. Touretzky, M.C. Mozer, M.E. Hasselmo, Eds. Advances in Neural Information Processing Systems: Proceedings of the 1995 Conference. Cambridge, MA, 1996. P. 1038–1044.
40. Rummery G.A. Problem solving with reinforcement learning. PhD thesis. Cambridge University, Cambridge, 1995. 52 p.
41. Watkins C.J.C.H. Learning from delayed rewards. PhD thesis. University of Cambridge, Cambridge, England, 1989. 234 p. URL: https://www.cs.rhul.ac.uk/home/ chrisw/new_thesis.pdf.
42. Watkins C.J.C.H., Dayan P. Technical note: Q-Learning // Machine Learning. 1992. V. 7, Issue 8. P. 279–292. URL: http://www.gatsby.ucl.ac.uk/~dayan/papers/ cjch.pdf.
43. Pavlov I.P. Conditioned reflexes. London: Oxford University Press, 1927. URL: http://s-f-walker.org.uk/pubsebooks/pdfs/Conditioned-Reflexes-Pavlov.pdf.
44. Воронцов К.В. Обучение с подкреплением (Reinforcement Learning) URL: http://www.machinelearning.ru/wiki/images/archive/3/35/20140621071329! Voron-ML-RL-slides.pdf.
45. Bellman R. A Markovian decision process // Journal of Mathematics and Mechanics. 1957. No 6. P. 716–719.
46. Rescorla R.A., Wagner A.R. A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement // In: A.H. Black, W.F. Prokasy, Eds. Classical conditioning II: Current research and theory. New York, NY: Appleton-Century-Crofts, 1972. P. 64–99.
47. Gewaltig Marc-Oliver, Diesmann Markus. NEST (NEural Simulation Tool) // Scholarpedia. 2007. V. 2, No 4. P. 1430. URL: http://www.scholarpedia.org/article/ NEST_(NEural_Simulation_Tool).
48. Supercomputers Ready for Use as Discovery Machines for Neuroscience // Frontiers in Neuroinformatics. November 2012. V. 6. P. 1–12.
49. Diesmann M., Gewaltig M. NEST: an environment for neural systems simulations // Forschung und wisschenschaftliches Rechnen, Beiträge zum Heinz-Billing-Preis. 2001. Bd. 58. S. 43–70.
50. Picard R.W. Affective Computing. MIT Press, 1997.


Presenting an article for publication in the Russian Digital Libraries Journal (RDLJ), the authors automatically give consent to grant a limited license to use the materials of the Kazan (Volga) Federal University (KFU) (of course, only if the article is accepted for publication). This means that KFU has the right to publish an article in the next issue of the journal (on the website or in printed form), as well as to reprint this article in the archives of RDLJ CDs or to include in a particular information system or database, produced by KFU.

All copyrighted materials are placed in RDLJ with the consent of the authors. In the event that any of the authors have objected to its publication of materials on this site, the material can be removed, subject to notification to the Editor in writing.

Documents published in RDLJ are protected by copyright and all rights are reserved by the authors. Authors independently monitor compliance with their rights to reproduce or translate their papers published in the journal. If the material is published in RDLJ, reprinted with permission by another publisher or translated into another language, a reference to the original publication.

By submitting an article for publication in RDLJ, authors should take into account that the publication on the Internet, on the one hand, provide unique opportunities for access to their content, but on the other hand, are a new form of information exchange in the global information society where authors and publishers is not always provided with protection against unauthorized copying or other use of materials protected by copyright.

RDLJ is copyrighted. When using materials from the log must indicate the URL: index.phtml page = elbib / rus / journal?. Any change, addition or editing of the author's text are not allowed. Copying individual fragments of articles from the journal is allowed for distribute, remix, adapt, and build upon article, even commercially, as long as they credit that article for the original creation.

Request for the right to reproduce or use any of the materials published in RDLJ should be addressed to the Editor-in-Chief A.M. Elizarov at the following address: amelizarov@gmail.com.

The publishers of RDLJ is not responsible for the view, set out in the published opinion articles.

We suggest the authors of articles downloaded from this page, sign it and send it to the journal publisher's address by e-mail scan copyright agreements on the transfer of non-exclusive rights to use the work.

Most read articles by the same author(s)