Öz
Deneysel psikoloji ve davranışsal sinirbilimde robot kullanımı gittikçe yaygınlaşmakta ve hayvan davranış deneylerinin temel yöntemlerinden birine dönüşmektedir. Robotların deney hayvanlarıyla beraber kullanımı, hayvanlara belirgin çevresel uyaranları seçici bir biçimde aktarmak, farklı ve görece zor davranışlar öğretmek ve hayvanların grup davranışını manipüle etmek için önemli avantajlar sağlamaktadır. Robotlar ve hayvanların davranış düzeneklerinde beraber kullanımının yanı sıra, çeşitli robotik sistemlerin deney hayvanlarına entegre edilmesiyle biyohibrit canlılar oluşturulmaktadır. Hayvanların dışarıdan verilen komutlara uyarak robotik bir biçimde yönetilmesini sağlayan yöntemler ile robotların hayvanlar tarafından yönetilmesini sağlayan uygulamalar, deneysel hayvan çalışmalarına yeni yöntemler kazandırmaktadır. Bununla birlikte, hayvan davranışını ve sinir sisteminin çalışma ilkelerini temel alarak geliştirilen yapay zekâ ve robotik öğrenme modelleri ortaya çıkmakta ve bu teknolojiler hayvan deneylerinde kullanılmaktadır. Bu derleme, deneysel psikoloji ve davranışsal sinirbilimde robotlar ve deney hayvanlarının birlikte kullanımını üç ana başlık altında incelemektedir. Deney hayvanlarının ve robotların etkileşimine dayanan davranış testleri (1) ele alınmakta, robotik sistemlerin hayvanlara entegre edilmesiyle oluşturulan biyohibrit canlılar (2) ve biyoloji temelli robotik sistemler (3) açıklanmaktadır. Ortaya çıkan bu yenilikçi uygulamaların deneysel hayvan çalışmalarına sunduğu teorik ve teknik katkılar değerlendirilmektedir.
Referanslar
Abbeel, P. ve Ng, A. Y. (2004). Apprenticeship learning via inverse reinforcement learning. Twenty-first international conference on Machine learning - ICML ’04 içinde . doi:10.1145/1015330.1015430
Aghajan, Z. M., Acharya, L., Moore, J. J., Cushman, J. D., Vuong, C. ve Mehta, M. R. (2015). Impaired spatial selectivity and intact phase precession in two-dimensional virtual reality. Nature Neuroscience, 18(1), 121–128. doi:10.1038/nn.3884
Alapan, Y., Yasa, O., Yigit, B., Yasa, I. C., Erkoc, P. ve Sitti, M. (2019). Microrobotics and Microorganisms: Biohybrid Autonomous Cellular Robots. Annual Review of Control, Robotics, and Autonomous Systems, 2(1), 205–230. doi:10.1146/annurev-control-053018-023803
Ando, N. ve Kanzaki, R. (2017, Eylül). Using insects to drive mobile robots — hybrid robots bridge the gap between biological and artificial systems. Arthropod Structure and Development. Elsevier Ltd. doi:10.1016/j.asd.2017.02.003
Balleine, B. W., Liljeholm, M. ve Ostlund, S. B. (2009, Nisan). The integrative function of the basal ganglia in instrumental conditioning. Behavioural Brain Research. Elsevier. doi:10.1016/j.bbr.2008.10.034
Bartal, I. B. A., Decety, J. ve Mason, P. (2011). Empathy and pro-social behavior in rats. Science, 334(6061), 1427–1430. doi:10.1126/science.1210789
Beer, R. D., Quinn, R. D., Chiel, H. J. ve Ritzmann, R. E. (1997). Biologically Inspired Approaches to Robotics. Communications of the ACM, 40(3), 30–38. doi:10.1145/245108.245118
Benabid, A. L. (2003, Aralık). Deep brain stimulation for Parkinson’s disease. Current Opinion in Neurobiology. Elsevier Ltd. doi:10.1016/j.conb.2003.11.001
Bozkurt, A., Lobaton, E. ve Sichitiu, M. (2016). A Biobotic Distributed Sensor Network for Under-Rubble Search and Rescue. Computer, 49(5), 38–46. doi:10.1109/MC.2016.136
Brown, M. F. ve Brown, A. A. (2017). The promise of cyborg intelligence. Learning and Behavior, 45(1), 5– 6. doi:10.3758/s13420-016-0249-7
Butler, S. R. ve Fernández-Juricic, E. (2014). European starlings recognize the location of robotic conspecific attention. Biology Letters, 10(10). doi:10.1098/rsbl.2014.0665
Buzsáki, G. (1989). Two-stage model of memory trace formation: A role for “noisy” brain states. Neuroscience, 31(3), 551–570. doi:10.1016/0306-4522(89)90423-5
Buzsaki, G. ve Tingley, D. (2018). Space and Time: The Hippocampus as a Sequence Generator. Trends in cognitive sciences, 22(10), 853–869. doi:10.1016/j.tics.2018.07.006
Dabney, W., Kurth-Nelson, Z., Uchida, N., Starkweather, C. K., Hassabis, D., Munos, R. ve Botvinick, M. (2020). A distributional code for value in dopamine-based reinforcement learning. Nature, 577(7792), 671–675. doi:10.1038/s41586-019-1924-6
Dabney, W., Rowland, M., Bellemare, M. G. ve Munos, R. (2017). Distributional Reinforcement Learning with Quantile Regression. 32nd AAAI Conference on Artificial Intelligence, AAAI 2018, 2892– 2901.
de Margerie, E., Peris, A., Pittet, F., Houdelier, C., Lumineau, S. ve Richard-Yris, M. A. (2013). Effect of mothering on the spatial exploratory behavior of quail chicks. Developmental Psychobiology, 55(3), 256–264. doi:10.1002/dev.21019
Diorio, C. ve Mavoori, J. (2003). Computer electornics meet animal brains. Computer, 36(1), 69–75. doi:10.1109/MC.2003.1160058
Dirafzoon, A., Bozkurt, A. ve Lobaton, E. (2017). A framework for mapping with biobotic insect networks: From local to global maps. Robotics and Autonomous Systems, 88, 79–96. doi:10.1016/j.robot.2016.11.004
Erickson, J. C., Herrera, M., Bustamante, M., Shingiro, A., Bowen, T. ve Lazzari, C. R. (2015). Effective stimulus parameters for directed locomotion in Madagascar hissing cockroach biobot. PLoS ONE, 10(8). doi:10.1371/journal.pone.0134348
Eshel, N., Bukwich, M., Rao, V., Hemmelder, V., Tian, J. ve Uchida, N. (2015). Arithmetic and local circuitry underlying dopamine prediction errors. Nature, 525(7568), 243–246. doi:10.1038/nature14855
Eshel, N., Tian, J., Bukwich, M. ve Uchida, N. (2016). Dopamine neurons share common response function for reward prediction error. Nature Neuroscience, 19(3), 479–486. doi:10.1038/nn.4239
Faria, J. J., Dyer, J. R. G., Clément, R. O., Couzin, I. D., Holt, N., Ward, A. J. W., … Krause, J. (2010). A novel method for investigating the collective behaviour of fish: Introducing “Robofish”. Behavioral Ecology and Sociobiology, 64(8), 1211–1218. doi:10.1007/s00265-010-0988-y
Flynn, J. P. (1969). NEURAL ASPECTS OF ATTACK BEHAVIOR IN CATS. Annals of the New York Academy of Sciences, 159(3 Experimental), 1008–1012. doi:10.1111/j.1749-6632.1969.tb12993.x
Folkertsma, G. A., Straatman, W., Nijenhuis, N., Venner, C. H. ve Stramigioli, S. (2017). Robird: A Robotic Bird of Prey. IEEE Robotics and Automation Magazine, 24(3), 22–29. doi:10.1109/MRA.2016.2636368
Foster, T. C., Castro, C. A. ve McNaughton, B. L. (1989). Spatial selectivity of rat hippocampal neurons: Dependence on preparedness for movement. Science, 244(4912), 1580–1582. doi:10.1126/science.2740902
Fritsch, G. ve Hitzig, E. (1870). “Ueber die elektrische Erregbarkeit des Grosshirns”. Archiv fur Anatomie und Physiologie Wissenschaftliche Medicin, 37(300), 300–332.
Frohnwieser, A., Murray, J. C., Pike, T. W. ve Wilkinson, A. (2016, Ocak). Using robots to understand animal cognition. Journal of the Experimental Analysis of Behavior. Wiley-Blackwell Publishing Ltd. doi:10.1002/jeab.193
Galef, B. G. ve Laland, K. N. (2005, Haziran). Social learning in animals: Empirical studies and theoretical models. BioScience. American Institute of Biological Sciences. doi:10.1641/0006-3568(2005)055[0489:SLIAES]2.0.CO;2
Gianelli, S., Harland, B. ve Fellous, J. M. (2018). A new rat-compatible robotic framework for spatial navigation behavioral experiments. Journal of Neuroscience Methods, 294, 40–50. doi:10.1016/j.jneumeth.2017.10.021
Glimcher, P. W. (2011). Understanding dopamine and reinforcement learning: The dopamine reward prediction error hypothesis. Proceedings of the National Academy of Sciences of the United States of America, 108(SUPPL. 3), 15647–15654. doi:10.1073/pnas.1014269108
Göth, A. ve Evans, C. S. (2004). Social responses without early experience: Australian brush-turkey chicks use specific visual cues to aggregate with conspecifics. Journal of Experimental Biology, 207(13), 2199–2208. doi:10.1242/jeb.01008
Gribovskiy, A., Halloy, J. I., Deneubourg, J.-L., Bleuler, H. ve Mondada, F. (y.y.). Towards Mixed Societies of Chickens and Robots. Proceedings of the IEEE/RSJ 2010 International Conference on Intelligent Robots and Systems (IROS 2010). IEEE Press. doi:10.1109/IROS.2010.5649542
Guthrie, E. R. ve Horton, G. P. (1946). Cats in a puzzle box. Cats in a puzzle box. Oxford, England: Rinehart.
Hachiga, Y., Schwartz, L. P., Silberberg, A., Kearns, D. N., Gomez, M. ve Slotnick, B. (2018). Does a rat free a trapped rat due to empathy or for sociality? Journal of the Experimental Analysis of Behavior, 110(2), 267–274. doi:10.1002/jeab.464
Halloy, J., Sempo, G., Caprari, G., Rivault, C., Asadpour, M., Tâche, F., … Deneubourg, J. L. (2007). Social integration of robots into groups of cockroaches to control self-organized choices. Science, 318(5853), 1155–1158. doi:10.1126/science.1144259
Hassabis, D., Kumaran, D., Summerfield, C. ve Botvinick, M. Neuroscience-Inspired Artificial Intelligence. 95 Neuron 245–258 (2017). doi:10.1016/j.neuron.2017.06.011
Henderson, J. V., Nicol, C. J., Lines, J. A., White, R. P. ve Wathes, C. M. (2001). Behaviour of domestic ducks exposed to mobile predator stimuli. 1. Flock responses. British Poultry Science, 42(4), 433– 438. doi:10.1080/00071660120070668
Hepper, P. G. (1986). Kin recognition: functions and mechanisms. A review. Biological Reviews - Cambridge Philosophical Society, 61(1), 63–93. doi:10.1111/j.1469-185X.1986.tb00427.x
Heyes, C. M. (1994). Social learning in animals: Categories and mechanisms. Biological Reviews of the Cambridge Philosophical Society. doi:10.1111/j.1469-185x.1994.tb01506.x
Huai, R., Yang, J., Wang, H. ve Su, X. (2009). A new robo-animals navigation method guided by the remote control. Proceedings of the 2009 2nd International Conference on Biomedical Engineering and Informatics, BMEI 2009 içinde . doi:10.1109/BMEI.2009.5305809
Huang, J. V., Wei, Y. ve Krapp, H. G. (2019). A biohybrid fly-robot interface system that performs active collision avoidance. Bioinspiration and Biomimetics, 14(6), 065001. doi:10.1088/1748-3190/ab3b23
Ikemoto, S., Yang, C. ve Tan, A. (2015, Eylül). Basal ganglia circuit loops, dopamine and motivation: A review and enquiry. Behavioural Brain Research. Elsevier B.V. doi:10.1016/j.bbr.2015.04.018
Ishii, H., Aoki, T., Nakasuji, M., Miwa, H. ve Takanishi, A. (2004). Experimental study on interaction between a rat and a rat-robot based on animal psychology - analysis of basic factors necessary for a symbiosis between the rat and the robot. doi:10.1109/robot.2004.1307478
Jolly, L., Pittet, F., Caudal, J. P., Mouret, J. B., Houdelier, C., Lumineau, S. ve De Margerie, E. (2016). Animal-to-robot social attachment: Initial requisites in a gallinaceous bird. Bioinspiration and Biomimetics, 11(1). doi:10.1088/1748-3190/11/1/016007
Koo, B., Koh, C. S., Park, H.-Y., Lee, H.-G., Chang, J. W., Choi, S. ve Shin, H.-C. (2017). Manipulation of Rat Movement via Nigrostriatal Stimulation Controlled by Human Visually Evoked Potentials. Scientific Reports, 7(1), 2340. doi:10.1038/s41598-017-02521-6
Kuwana, Y., Nagasawa, S., Shimoyama, I. ve Kanzaki, R. (1999). Synthesis of the pheromone-oriented behaviour of silkworm moths by a mobile robot with moth antennae as pheromone sensors. Biosensors and Bioelectronics, 14(2), 195–202. doi:10.1016/S0956-5663(98)00106-7
Ladu, F., Bartolini, T., Panitz, S. G., Chiarotti, F., Butail, S., Macrì, S. ve Porfiri, M. (2015). Live predators, robots, and computer-animated images elicit differential avoidance responses in zebrafish. Zebrafish, 12(3), 205–214. doi:10.1089/zeb.2014.1041
Laschi, C., Mazzolai, B., Patanè, F., Mattoli, V., Dario, P., Ishii, H., … Takanishi, A. (2006). Design and development of a Legged rat robot for studying animal-robot interaction. Proceedings of the First IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics, 2006, BioRob 2006 içinde . doi:10.1109/BIOROB.2006.1639160
Latif, T., Whitmire, E., Novak, T. ve Bozkurt, A. (2016). Sound Localization Sensors for Search and Rescue Biobots. IEEE Sensors Journal, 16(10), 3444–3453. doi:10.1109/JSEN.2015.2477443
Levison, P. K. ve Flynn, J. P. (1965). The objects attacked by cats during stimulation of the hypothalamus. Animal Behaviour, 13(2–3), 217–220. doi:10.1016/0003-3472(65)90037-0
Lorenz, K. Z. (1937). The Companion in the Bird’s World. The Auk, 54(3), 245–273. doi:10.2307/4078077
Milford, M. J., Wyeth, G. F. ve Prasser, D. (2004). RatSLAM: A hippocampal model for simultaneous localization and mapping. Proceedings - IEEE International Conference on Robotics and Automation içinde (C. 2004, ss. 403–408). doi:10.1109/robot.2004.1307183
Milford, M., Jacobson, A., Chen, Z. ve Wyeth, G. (2016). RatSLAM: Using models of rodent hippocampus for robot navigation and beyond. Springer Tracts in Advanced Robotics içinde (C. 114, ss. 467–485). Springer Verlag. doi:10.1007/978-3-319-28872-7_27
Min, K., Kim, H. ve Huh, K. (2019). Deep Distributional Reinforcement Learning Based High-Level Driving Policy Determination. IEEE Transactions on Intelligent Vehicles içinde (C. 4, ss. 416–424). Institute of Electrical and Electronics Engineers Inc. doi:10.1109/TIV.2019.2919467
Misawa, N., Fujii, S., Kamiya, K., Osaki, T., Takaku, T., Takahashi, Y. ve Takeuchi, S. (2019). Construction of a Biohybrid Odorant Sensor Using Biological Olfactory Receptors Embedded into Bilayer Lipid Membrane on a Chip. ACS Sensors, 4(3), 711–716. doi:10.1021/acssensors.8b01615
Mitri, S., Wischmann, S., Floreano, D. ve Keller, L. (2013). Using robots to understand social behaviour. Biological Reviews, 88(1), 31–39. doi:10.1111/j.1469-185X.2012.00236.x
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., … Hassabis, D. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540), 529–533. doi:10.1038/nature14236
Morimura, T., Sugiyama, M., Kashima, H., Hachiya, H. ve Tanaka, T. (2010). Parametric Return Density Estimation for Reinforcement Learning. Proceedings of the Twenty-Sixth Conference on Uncertainty in Artificial Intelligence içinde , UAI’10 (ss. 368–375). Arlington, Virginia, USA: AUAI Press.
Myrick, A. J. ve Baker, T. C. (2011). Locating a compact odor source using a four-channel insect electroantennogram sensor. Bioinspiration and Biomimetics, 6(1). doi:10.1088/1748-3182/6/1/016002
O’Neill, J., Pleydell-Bouverie, B., Dupret, D. ve Csicsvari, J. (2010). Play it again: reactivation of waking experience and memory. Trends in neurosciences, 33(5), 220–9. doi:10.1016/j.tins.2010.01.006
Oyama, K., Hernádi, I., Iijima, T. ve Tsutsui, K. I. (2010). Reward prediction error coding in dorsal striatal neurons. Journal of Neuroscience, 30(34), 11447–11457. doi:10.1523/JNEUROSCI.1719-10.2010
Patricelli, G. L., Uy, J. A. C., Walsh, G. ve Borgia, G. (2002). Male displays adjusted to female’s response. Nature, 415(6869), 279–280. doi:10.1038/415279a
Quinn, L. K., Schuster, L. P., Aguilar-Rivera, M., Arnold, J., Ball, D., Gygi, E., … Chiba, A. A. (2018). When rats rescue robots. Animal Behavior and Cognition, 5(4), 368–379. doi:10.26451/abc.05.04.04.2018
Romano, D., Donati, E., Benelli, G. ve Stefanini, C. (2019). A review on animal–robot interaction: from bio- hybrid organisms to mixed societies. Biological Cybernetics, 113(3), 201–225. doi:10.1007/s00422-018-0787-5
Rundus, A. S., Owings, D. H., Joshi, S. S., Chinn, E. ve Giannini, N. (2007). Ground squirrels use an infrared signal to deter rattlesnake predation. Proceedings of the National Academy of Sciences of the United States of America, 104(36), 14372–14376. doi:10.1073/pnas.0702599104
Schaul, T., Quan, J., Antonoglou, I. ve Silver, D. (2016). Prioritized Experience Replay. 4th International Conference on Learning Representations, ICLR 2016 - Conference Track Proceedings.
Schrock, L. E., Mink, J. W., Woods, D. W., Porta, M., Servello, D., Visser-Vandewalle, V., … Okun, M. S. (2015, Nisan). Tourette syndrome deep brain stimulation: A review and updated recommendations. Movement Disorders. John Wiley and Sons Inc. doi:10.1002/mds.26094
Sesack, S. R. ve Grace, A. A. (2010, Ocak). Cortico-basal ganglia reward network: Microcircuitry. Neuropsychopharmacology. Nature Publishing Group. doi:10.1038/npp.2009.93
Shah, A. (2012). Psychological and neuroscientific connections with reinforcement learning. Adaptation, Learning, and Optimization içinde (C. 12, ss. 507–537). Springer Verlag. doi:10.1007/978-3-642-27645-3_16
Shi, Q., Ishii, H., Kinoshita, S., Konno, S., Takanishi, A., Okabayashi, S., … Kimura, H. (2013). A rat-like robot for interacting with real rats. Robotica. doi:10.1017/S0263574713000568
Shi, Q., Ishii, H., Kinoshita, S., Takanishi, A., Okabayashi, S., Iida, N., … Shibata, S. (2013). Modulation of rat behaviour by using a rat-like robot. Bioinspiration & Biomimetics, 8(4), 46002. doi:10.1088/1748-3182/8/4/046002
Shi, Q., Ishii, H., Tanaka, K., Sugahara, Y., Takanishi, A., Okabayashi, S., … Fukuda, T. (2015). Behavior modulation of rats to a robotic rat in multi-rat interaction. Bioinspiration and Biomimetics, 10(5). doi:10.1088/1748-3190/10/5/056011
Skinner, B. F. (1932). On the rate of formation of a conditioned reflex. Journal of General Psychology, 7(2), 274–286. doi:10.1080/00221309.1932.9918467
Skinner, B. F. (1948). “Superstition” in the pigeon. Journal of Experimental Psychology, 38(2), 168–172. doi:10.1037/h0055873
Skinner, B. F. (1959). Cumulative record. Cumulative record. East Norwalk, CT, US: Appleton-Century- Crofts.
Staddon, J. E. R. ve Cerutti, D. T. (2003). Operant Conditioning. Annual Review of Psychology, 54(1), 115– 144. doi:10.1146/annurev.psych.54.101601.145124
Steager, E. B., Wong, D., Mishra, D., Weiss, R. ve Kumar, V. (2014). Sensors for micro bio robots via synthetic biology. Proceedings - IEEE International Conference on Robotics and Automation içinde (ss. 3783–3788). Institute of Electrical and Electronics Engineers Inc. doi:10.1109/ICRA.2014.6907407
Takanishi, A., Aoki, T., Ito, M., Ohkawa, Y. ve Yamaguchi, J. (1998). Interaction between creature and robot - development of an experiment system for rat and rat robot interaction. IEEE International Conference on Intelligent Robots and Systems içinde (C. 3, ss. 1975–1980). doi:10.1109/iros.1998.724896
Talwar, S. K., Xu, S., Hawley, E. S., Weiss, S. A., Moxon, K. A. ve Chapin, J. K. (2002). Rat navigation guided by remote control. Nature, 417(6884), 37–38. doi:10.1038/417037a
Tan, X., Lee, Y., Chng, C. B., Lim, K. Bin ve Chui, C. K. (2020). Robot-assisted flexible needle insertion using universal distributional deep reinforcement learning. International Journal of Computer Assisted Radiology and Surgery, 15(2), 341–349. doi:10.1007/s11548-019-02098-7
Thorndike, E. L. (1913). The psychology of learning (C. 2). Teachers College, Columbia University.
Ünal, G. (2019). The Cortico-hippocampal Circuit: The Brain’s Center for Mapping and Declarative Memory. J Ankara Univ Fac Med, 72, 13–23.
Unal, G. ve Canbeyli, R. (2019). Psychomotor retardation in depression: A critical measure of the forced swim test. Behavioural brain research, 372, 112047. doi:10.1016/j.bbr.2019.112047
Wang, Z. ve Qiao, H. (2017). Behavior classification and image processing for biorobot-rat interaction. Proceedings of 2017 IEEE 7th International Conference on Electronics Information and Emergency Communication, ICEIEC 2017 içinde (ss. 572–575). doi:10.1109/ICEIEC.2017.8076631
Yamaguchi, S., Naoki, H., Ikeda, M., Tsukada, Y., Nakano, S., Mori, I. ve Ishii, S. (2018). Identification of animal behavioral strategies by inverse reinforcement learning. PLoS Computational Biology. doi:10.1371/journal.pcbi.1006122
Yu, Y., Pan, G., Gong, Y., Xu, K., Zheng, N., Hua, W., … Wu, Z. (2016). Intelligence-augmented rat cyborgs in maze solving. PLoS ONE, 11(2). doi:10.1371/journal.pone.0147754

Bu çalışma Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License ile lisanslanmıştır.
Telif Hakkı (c) 2021 Nesne Psikoloji Dergisi
