Auteurs : Aakanksha Chowdhery, Tianhe Yu, Jeffrey Zhao, Dian Yu, Yuan Cao, Alexander Pritzel, David Silver, Fan Yang, Basil Mustafa, Neil Houlsby, Sholto Douglas, Jacob Devlin, James Bradbury, Kefan Xiao, Shivani Agrawal, Zizhao Zhang, Mary Phuong, Timothy Lillicrap, Tao Wang, David Reitter, Behnam Neyshabur, Shuo-yiin Chang, Zhifeng Chen, Yuanzhong Xu, Ed Chi, Mostafa Dehghani, Elena Buchatskaya, Chih-Kuan Yeh, Benjamin Lee, Mohammad Saleh, George Tucker, Julian Schrittwieser, Lev Proleev, Diana Mincu, Subhrajit Roy, Le Hou, Nenad Tomasev, Yifeng Lu, Sebastian Borgeaud, Arthur Mensch, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Jack W. Rae, Laurent Sifre, Pranav Shyam, Pengcheng Yin, Henryk Michalewski, Jacob Austin, Hexiang Hu, Ming-Wei Chang, Andrew M. Dai, Justin Gilmer, Mehran Kazemi, Quan Yuan, Jean-Baptiste Lespiau, Roman Ring, Loren Maggiore, Albin Cassirer, Michela Paganini, Swaroop Mishra, Chen Zhu, Siamak Shakeri, Kelvin Guu, Tianqi Liu, Dmitry Lepikhin, Maxim Krikun, Orhan Firat, Marie Pellat, Kevin Robinson, Xi Chen, Chen Zhou, Josip Djolonga, Mario Lučić, Sebastian Riedel, Cheng Li, Yingjie Miao, Srivatsan Srinivasan, Tom Le Paine, Richard Powell, Petko Georgiev, Xuehan Xiong, Yifan He, Zhen Yang, Shaobo Hou, Anirudh Goyal, Jian Li, Jakub Sygnowski, Maja Trebacz, Maribeth Rauh, Will Hawkins, Machel Reid, Disha Shrivastava, Zihang Dai, Elliot Catt, Matthew Aitchison, Kate Baumli, Natalie Clay, Sarah York, Denis Teplyashin, Junhyuk Oh, Tao Zhu, Amelia Glaese, Fangyu Liu, Xinying Song, Kalpesh Krishna, Rohan Anil, Melvin Johnson, Emanuel Taropa, Paige Bailey, Laurent El Shafey, Gaurav Mishra, Erica Moreira, Yujing Zhang, Junwhan Ahn, Siddhartha Brahma, Kevin Brooks, Yong Cheng, Ethan Dyer, Fangxiaoyu Feng, Xavier Garcia, Lucas Gonzalez, Steven Hand, Michael Isard, Abe Ittycheriah, Wenhao Jia, Chang Lan, Music Li, YaGuang Li, Hyeontaek Lim, Hanzhao Lin, Marcello Maggioni, Joshua Maynez, Vedant Misra, Zachary Nado, Eric Ni, Martin Polacek, Siyuan Qiao, Aurko Roy, Rajkumar Samuel, Ambrose Slone, Daniel Sohn, Simon Tokumine, Dasha Valter, Vijay Vasudevan, Kiran Vodrahalli, Pidong Wang, Yunhan Xu, Jiahui Yu, Qiao Zhang, Steven Zheng, Ce Zheng, Heng-Tze Cheng, Sanjay Ghemawat, Thanumalayan Sankaranarayana Pillai, Albert Webson, Nora Kassner, Dan Garrette, Allan Dafoe, Xiang Zhou, Jilin Chen, Qi Li, James Qin, Ankur Bapna, Jason Riesa, Tara Sainath, Johan Schalkwyk, Arthur Guez, Elena Gribovskaya, Angeliki Lazaridou, Thibault Sottiaux, Mantas Pajarskas, Zhitao Gong, Yujia Li, Kareem Ayoub, Ioannis Antonoglou, Boyi Liu, Aditya Siddhant, Chenjie Gu, Andrew Brock, Ankesh Anand, Piyush Patil, Laura Culp, Yunxuan Li, Antoine Miech, Anja Hauth, Rahul Goel, Shashi Narayan, Thang Luong, Léonard Hussenot, Ewa Andrejczuk, Oran Lang, Johan Ferret, Gemini Team, Jean-Baptiste Alayrac, Radu Soricut, Emily Pitler, James Molloy, Paul R. Barham, Fabio Viola, Malcolm Reynolds, Ryan Doherty, Eli Collins, Clemens Meyer, Megha Goel, Enrique Piqueras, Iain Barr, Nikolay Savinov, Ivo Danihelka, Becca Roelofs, Anaïs White, Anders Andreassen, Tamara von Glehn, Lakshman Yagati, Misha Khalman, Alexandre Frechette, Charlotte Smith, Yi Luan, James Lottes, Nathan Schucher, Federico Lebron, Alban Rrustemi, Phil Crone, Tomas Kocisky, Bartek Perz, Heidi Howard, Adam Bloniarz, Han Lu, Fred Alcober, Megan Barnes, Shantanu Thakoor, Gabriel Barth-Maron, William Wong, Rishabh Joshi, Rahma Chaabouni, Deeni Fatiha, Arun Ahuja, Ruibo Liu, Sarah Cogan, Jeremy Chen, Chao Jia, Jordan Grimstad, Ale Jakse Hartman, Martin Chadwick, Gaurav Singh Tomar, Evan Senter, Michael Laskin, Connie Tao, Lorenzo Blanco, Adrià Puigdomènech Badia, Mianna Chen, Jenny Brennan, Clara Rivera, Sergey Brin, Shariq Iqbal, Gabriela Surita, Jane Labanowski, Abhi Rao, Stephanie Winkler, Emilio Parisotto, Yiming Gu, Kate Olszewska, Ravi Addanki, Annie Louis, Geoff Brown, Nithya Attaluri, Jan Balaguer, Jackie Xiang, Zoe Ashwood, Anton Briukhov, Sanjay Ganapathy, Smit Sanghavi, Ajay Kannan, Axel Stjerngren, Yuting Sun, Pedram Pejman, Cindy Wang, Juliette Love, Dawn Bloxwich, Kehang Han, Peter Humphreys, Thibault Sellam, Varun Godbole, Sina Samangooei, Alex Kaskasoli, Sébastien M. R. Arnold, Shubham Agrawal, Richard Tanburn, Sarah Hodkinson, Ankush Garg, Minh Giang, Alexander Neitz, Zaheer Abbas, Elizabeth Cole, Dipanjan Das, Dominika Rogozińska, Pablo Sprechmann, Lukas Zilka, Flavien Prost, Luheng He, Marianne Monteiro, Chris Welty, Josh Newlan, Dawei Jia, Miltiadis Allamanis, Clara Huiyi Hu, Raoul de Liedekerke, Carl Saroufim, Shruti Rijhwani, Anirudh Baddepudi, Alex Goldin, Adnan Ozturel, Devendra Sachan, Reinald Kim Amplayo, Craig Swanson, Dessie Petrova, Jessica Landon, Miteyan Patel, Ruizhe Zhao, Kevin Villela, Luyu Wang, Matthew Rahtz, Mai Giménez, Legg Yeung, James Keeling, Boxi Wu, Salem Haykal, Rachel Saputro, Zeynep Cankara, Abhanshu Sharma, Nick Fernando, Solomon Kim, Adrian Hutter, Priyanka Agrawal, Alex Castro-Ros, Paul Komarek, Ross McIlroy, Guodong Zhang, Wael Farhan, Michael Sharman, Paul Natsev, Paul Michel, Yamini Bansal, Kris Cao, Christina Butterfield, Justin Chung, Paul Kishan Rubenstein, Kedar Soparkar, Karel Lenc, Timothy Chung, Aedan Pope, Jackie Kay, Priya Jhakra, Shibo Wang, Taylor Tobin, Andrea Tacchetti, Yash Katariya, Nimesh Ghelani, Lora Aroyo, Jonas Adler, Mateo Wirth, Lisa Lee, Thais Kagohara, Jay Pavagadhi, Sophie Bridgers, Anna Bortsova, Zafarali Ahmed, Vijay Bolina, Mariko Iinuma, Polina Zablotskaia, James Besley, Da-Woon Chung, Timothy Dozat, Ramona Comanescu, Xiance Si, Jeremy Greer, Guolong Su, Raphaël Lopez Kaufman, Mohamed Elhawaty, Jinwei Xing, Christina Greer, Helen Miller, Shereen Ashraf, Ada Ma, Angelos Filos, Milos Besta, Rory Blevins, Ted Klimenko, Soravit Changpinyo, Jiaqi Mu, Oscar Chang, Carrie Muir, Vered Cohen, Charline Le Lan, Krishna Haridasan, Amit Marathe, Steven Hansen, Mingqiu Wang, Sophia Austin, Jiepu Jiang, Justin Chiu, Jaime Alonso Lorenzo, Lars Lowe Sjösund, Sébastien Cevey, Zach Gleicher, Thi Avrahami, Anudhyan Boral, Hansa Srinivasan, Vittorio Selo, Rhys May, Konstantinos Aisopos, Livio Baldini Soares, Michael B. Chang, Adrià Recasens, Ben Caine, Filip Pavetic, Fabio Pardo, Anita Gergely, Justin Frye, Vinay Ramasesh, Dan Horgan, Kartikeya Badola, Alex Tomala, Yunhao Tang, Dalia El Badawy, Elspeth White, Abhishek Jindal, Sharad Vikram, Sergi Caelles, Ross Hemsley, Gregory Thornton, Wojciech Stokowiec, Phoebe Thacker, Çağlar Ünlü, Zhishuai Zhang, James Svensson, Max Bileschi, Katerina Tsihlas, Arpi Vezer, Marco Selvi, Toby Shevlane, Mikel Rodriguez, Tom Kwiatkowski, Samira Daruki, Keran Rong, Nicholas FitzGerald, Keren Gu-Lemberg, Mina Khan, Vladimir Feinberg, James Cobon-Kerr, Sayed Hadi Hashemi, Richard Ives, Yana Hasson, Nathan Byrd, Qingze Wang, Alexandre Moufarek, Samer Hassan, Kaushik Shivakumar, Joost van Amersfoort, Amol Mandhane, Pratik Joshi, Matthew Tung, Hannah Sheahan, Nemanja Rakićević, Sid Mittal, Seb Noury, Eren Sezener, Fantine Huot, Matthew Lamm, Nicola De Cao, Charlie Chen, Mahdis Mahdieh, Patrick Kane, Dominik Rabiej, Gautam Vasudevan, Betty Chan, Pam G Rabinovitch, Manaal Faruqui, Lam Nguyen Thiet, Sebastian Krause, Tejasi Latkar, Sid Lall, Egor Filonov, Vikas Yadav, Hao Zhou, Jeremiah Liu, Taylor Bos, Anhad Mohananey, Sidharth Mudgal, Sumit Bagri, Aliaksei Severyn, Abhimanyu Goyal, Cosmo Du, Dustin Tran, Norbert Kalb, Balaji Lakshminarayanan, Shyam Upadhyay, Summer Yue, Daniil Mirylenka, Francesco Pongetti, Pei Sun, Adam Kurzrok, Mia Chen, Siddharth Gopal, Rakesh Shivanna, Jack Krawczyk, Purvi Shah, Andras Orban, Fabian Güra, Aurelien Boffy, Harish Ganapathy, HyunJeong Choe, Ágoston Weisz, Jarrod Kahn, Maciej Kula, Jeff Pitman, Rushin Shah, Majd Al Merey, Martin Baeuml, Olcan Sercinoglu, Ilya Kornakov, Iñaki Iturrate, Vitaliy Nikolaev, Víctor Campos Campos, Romina Stella, Chenxi Liu, Mainak Chain, Nivedita Melinkeri, Aaron Cohen, Venus Wang, Kristie Seymore, Sergey Zubkov, Sai Krishnakumaran, Brian Albert, Nate Hurley, Motoki Sano, Jonah Joughin, Tomasz Kępa, Yomna Eldawy, Jiawern Lim, Rahul Rishi, Shirin Badiezadegan, Jerry Chang, Sanil Jain, Sri Gayatri Sundara Padmanabhan, Subha Puttagunta, Leslie Baker, Vamsi Bedapudi, Shuntong Lei, Anthony Yu, Oren Litvin, Zhichun Wu, Sam Sobell, Andrea Siciliano, Alan Papir, Robby Neale, Jonas Bragagnolo, Tej Toor, Tina Chen, Valentin Anklin, Feiran Wang, Richie Feng, Milad Gholami, Kevin Ling, Lijuan Liu, Jules Walter, Hamid Moghaddam, Arun Kishore, Jakub Adamek, Tyler Mercado, Jonathan Mallinson, Siddhinita Wandekar, Stephen Cagle, Eran Ofek, Guillermo Garrido, Clemens Lombriser, Maksim Mukha, Botu Sun, Hafeezul Rahman Mohammad, Josip Matak, Yadi Qian, Vikas Peswani, Pawel Janus, Leif Schelin, Oana David, Ankur Garg, Oleksii Duzhyi, Anton Älgmyr, Timothée Lottaz, Luyao Xu, Alex Chinien, Aleksandr Chuklin, Josie Li, Carrie Spadine, Travis Wolfe, Kareem Mohamed, Subhabrata Das, Kyle He, Daniel von Dincklage, Akanksha Maurya, Luyan Chi, Khalid Salama, Pavan Kumar Reddy M, Aarush Selvan, Mikhail Dektiarev, Golnaz Ghiasi, Erdem Guven, Himanshu Gupta, Deepak Sharma, Idan Heimlich Shtacher, Shachi Paul, Oscar Akerlund, François-Xavier Aubet, Terry Huang, Eric Zhu, Elico Teixeira, Matthew Fritze, Francesco Bertolini, Liana-Eleonora Marinescu, Martin Bölle, Dominik Paulus, Khyatti Gupta, Max Chang, Jason Sanders, Roopa Wilson, Xuewei Wu, Yi-Xuan Tan, Tulsee Doshi, Wanming Chen, Seth Benjamin, Jasmine Lee, Vipul Ranjan, Krzysztof Styrc, Jon Simon, Malcolm Rose Harriott, Mudit Bansal, Alexei Robsky, Geoff Bacon, David Greene, Obaid Sarvana, Samuel Andermatt, Patrick Siegler, Ben Horn, Assaf Israel, Chih-Wei "Louis" Chen, Marco Selvatici, Pedro Silva, Kathie Wang, Jackson Tolins, Roey Yogev, Xiaochen Cai, Alessandro Agostini, Maulik Shah, Hung Nguyen, Noah Ó Donnaile, Sébastien Pereira, Linda Friso, Adam Stambler