Large-Scale Optical Neural Networks based on Photoelectric Multiplication

Authors: Ryan Hamerly, Liane Bernstein, Alexander Sludds, Marin Soljačić, Dirk Englund

Phys. Rev. X 9, 021032 (2019)
Submitted to arXiv on 12 Nov. 2018
Abstract

Recent success in deep neural networks has generated strong interest in hardware accelerators to improve speed and energy consumption. This paper presents a new type of photonic accelerator based on coherent detection that is scalable to large ($N \gtrsim 10^6$) networks and can be operated at high (GHz) speeds and very low (sub-aJ) energies per multiply-and-accumulate (MAC), using the massive spatial multiplexing enabled by standard free-space optical components. In contrast to previous approaches, both weights and inputs are optically encoded so that the network can be reprogrammed and trained on the fly. Simulations of the network using models for digit- and image-classification reveal a "standard quantum limit" for optical neural networks, set by photodetector shot noise. This bound, which can be as low as 50 zJ/MAC, suggests performance below the thermodynamic (Landauer) limit for digital irreversible computation is theoretically possible in this device. The proposed accelerator can implement both fully-connected and convolutional networks. We also present a scheme for back-propagation and training that can be performed in the same hardware. This architecture will enable a new class of ultra-low-energy processors for deep learning.

The papers around it, by date and closeness

Higher is closer to this paper; left is older. Lines join papers close to each other: groups of lines are sub-topics. Select a point to see why it is there. Back to the summary of this paper

How to read this map? Illustrated guide

Closest papers Prior work it builds on Papers that cite it

Papers around this one

Ranked by how close their title and abstract are to this paper’s. Select some to read them together.

Closest papers

  1. Very close Cites this paper
    An optical neural network using less than 1 photon per multiplication
    Tian-Yu Wang, Shi-Fan Ma, L. Wright et al. · 2021 · Nature Communications · cited 294 times · Not summarized yet
    Abstract

    Deep learning has become a widespread tool in both science and industry. However, continued progress is hampered by the rapid growth in energy costs of ever-larger deep neural networks. Optical neural networks provide a potential means to solve the energy-cost problem faced by deep learning. Here, we experimentally demonstrate an optical neural network based on optical dot products that achieves 99% accuracy on handwritten-digit classification using ~3.1 detected photons per weight multiplication and ~90% accuracy using ~0.66 photons (~2.5 × 10−19 J of optical energy) per weight multiplication. The fundamental principle enabling our sub-photon-per-multiplication demonstration—noise reduction…

  2. Very close
    Progress on photonic tensor processors based on time multiplexing and photoelectric multiplication
    R. Hamerly, Alexander Sludds, Liane Bernstein et al. · 2021 · OPTO · Not summarized yet
    Abstract

    Optical approaches to machine learning rely heavily on programmable linear photonic circuits. Since the performance and energy efficiency scale with size, a major challenge is overcoming scaling roadblocks to the photonic technology. Recently, we proposed an optical neural network architecture based on coherent detection. This architecture has several scaling advantages over competing approaches, including linear (rather than quadratic) chip-area scaling and constant circuit depth. We review the fundamental and technological limits to the energy consumption in this architecture, which shed light on the quantum limits to analog computing, which are distinct from the thermodynamic (e.g. Landau…

  3. Very close Cites this paper
    Spatially varying nanophotonic neural networks
    Kaixuan Wei, Xiao Li, Johannes Froech et al. · 2023 · Science Advances · cited 46 times · Not summarized yet
    Abstract

    The explosive growth in computation and energy cost of artificial intelligence has spurred interest in alternative computing modalities to conventional electronic processors. Photonic processors, which use photons instead of electrons, promise optical neural networks with ultralow latency and power consumption. However, existing optical neural networks, limited by their designs, have not achieved the recognition accuracy of modern electronic neural networks. In this work, we bridge this gap by embedding parallelized optical computation into flat camera optics that perform neural network computations during capture, before recording on the sensor. We leverage large kernels and propose a spati…

  4. Very close Cites this paper
    Single-chip photonic deep neural network with forward-only training
    Saumil Bandyopadhyay, Alexander Sludds, Stefan Krastanov et al. · 2022 · Nature Photonics · cited 291 times · Not summarized yet
    Abstract

    As deep neural networks revolutionize machine learning, energy consumption and throughput are emerging as fundamental limitations of complementary metal–oxide–semiconductor (CMOS) electronics. This has motivated a search for new hardware architectures optimized for artificial intelligence, such as electronic systolic arrays, memristor crossbar arrays and optical accelerators. Optical systems can perform linear matrix operations at an exceptionally high rate and efficiency, motivating recent demonstrations of low-latency matrix accelerators and optoelectronic image classifiers. However, demonstrating coherent, ultralow-latency optical processing of deep neural networks has remained an outstan…

  5. Very close
    Coherent photonic neuromorphic computing for high-speed deep learning applications
    M. Moralis-Pegios, G. Mourgias-Alexandris, A. Tsakyridis et al. · 2022 · OPTO · Not summarized yet
    Abstract

    The emergence of demanding machine learning and AI workloads in modern computational systems and Data Centers (DC) has fueled a drive towards custom hardware, designed to accelerate Multiply-Accumulate (MAC) operations. In this context, neuromorphic photonics have recently attracted attention as a promising technological candidate, that can transfer photonics low-power, high bandwidth credentials in neuromorphic hardware implementations. However, the deployment of such systems necessitates progress in both the underlying constituent building blocks as well as the development of deep learning training models that can take into account the physical properties of the employed photonic component…

  6. Very close
    Photonic Multiply-Accumulate Operations for Neural Networks
    M. Nahmias, T. F. de Lima, A. Tait et al. · 2020 · IEEE Journal of Selected Topics in Quantum Electronics · cited 258 times · Not summarized yet
    Abstract

    It has long been known that photonic communication can alleviate the data movement bottlenecks that plague conventional microelectronic processors. More recently, there has also been interest in its capabilities to implement low precision linear operations, such as matrix multiplications, fast and efficiently. We characterize the performance of photonic and electronic hardware underlying neural network models using multiply-accumulate operations. First, we investigate the limits of analog electronic crossbar arrays and on-chip photonic linear computing systems. Photonic processors are shown to have advantages in the limit of large processor sizes (<inline-formula><tex-math notation="LaTeX">$…

  7. Very close
    Photonic perceptron based on a Kerr microcomb for high-speed, scalable, optical neural networks
    Xing-Yuan Xu, M. Tan, B. Corcoran et al. · 2020 · International Topical Meeting on Microwave Photonics · cited 128 times · Not summarized yet
    Abstract

    Optical artificial neural networks (ONNs) have significant potential for ultra-high computing speed and energy efficiency. We report a new approach to ONNs based on integrated Kerr micro-combs that is programmable, highly scalable and capable of reaching ultra-high speeds, demonstrating the building block of the ONN — a single neuron perceptron — by mapping synapses onto 49 wavelengths to achieve a single-unit throughput of 11.9 Giga-OPS at 8 bits per OP, or 95.2 Gbps. We test the perceptron on handwritten-digit recognition and cancer-cell detection — achieving over 90% and 85% accuracy, respectively. By scaling the perceptron to a deep learning network using off-the-shelf telecom technology…

  8. Very close Cites this paper
    Low-power scalable multilayer optoelectronic neural networks enabled with incoherent light
    Alexander Song, Sai Nikhilesh Murty Kottapalli, Rahul Goyal et al. · 2024 · Nature Communications · cited 38 times · Not summarized yet
    Abstract

    Optical approaches have made great strides towards the goal of high-speed, energy-efficient computing necessary for modern deep learning and AI applications. Read-in and read-out of data, however, limit the overall performance of existing approaches. This study introduces a multilayer optoelectronic computing framework that alternates between optical and optoelectronic layers to implement matrix-vector multiplications and rectified linear functions, respectively. Our framework is designed for real-time, parallelized operations, leveraging 2D arrays of LEDs and photodetectors connected via independent analog electronics. We experimentally demonstrate this approach using a system with a three-…

Prior work it builds on

  1. Very close Used in depth by this paper A method it uses
    Deep learning with coherent nanophotonic circuits
    Yichen Shen, N. Harris, S. Skirlo et al. · 2016 · Nature Photonics · cited 3,146 times · Not summarized yet
    Abstract

    Artificial neural networks are computational network models inspired by signal processing in the brain. These models have dramatically improved performance for many machine-learning tasks, including speech and image recognition. However, today's computing hardware is inefficient at implementing neural networks, in large part because much of it was designed for von Neumann computing schemes. Significant effort has been made towards developing electronic architectures tuned to implement artificial neural networks that exhibit improved computational speed and accuracy. Here, we propose a new architecture for a fully optical neural network that, in principle, could offer an enhancement in comput…

  2. Close Used in depth by this paper A method it uses
    Neuromorphic photonic networks using silicon photonic weight banks
    A. Tait, T. F. de Lima, E. Zhou et al. · 2016 · Scientific Reports · cited 821 times · Not summarized yet
    Abstract

    Photonic systems for high-performance information processing have attracted renewed interest. Neuromorphic silicon photonics has the potential to integrate processing functions that vastly exceed the capabilities of electronics. We report first observations of a recurrent silicon photonic neural network, in which connections are configured by microring weight banks. A mathematical isomorphism between the silicon photonic circuit and a continuous neural network model is demonstrated through dynamical bifurcation analysis. Exploiting this isomorphism, a simulated 24-node silicon photonic neural network is programmed using “neural compiler” to solve a differential system emulation task. A 294-f…

  3. Close* A method it uses
    All-optical neuromorphic computing in optical networks of semiconductor lasers
    D. Brunner, S. Reitzenstein, Ingo Fischer · 2016 · International Conference on Rebooting Computing · cited 9 times · Not summarized yet
  4. Close A method it uses
    All-optical machine learning using diffractive deep neural networks
    Xing Lin, Y. Rivenson, N. Yardimci et al. · 2018 · Science · cited 2,500 times · Not summarized yet
    Abstract

    All-optical deep learning Deep learning uses multilayered artificial neural networks to learn digitally from large datasets. It then performs advanced identification and classification tasks. To date, these multilayered neural networks have been implemented on a computer. Lin et al. demonstrate all-optical machine learning that uses passive optical components that can be patterned and fabricated with 3D-printing. Their hardware approach comprises stacked layers of diffractive optical elements analogous to an artificial neural network that can be trained to execute complex functions at the speed of light. Science, this issue p. 1004 All-optical deep learning can be implemented with 3D-printed…

  5. Close A method it uses
    Photonic information processing beyond Turing: an optoelectronic implementation of reservoir computing.
    L. Larger, M. Soriano, D. Brunner et al. · 2012 · Optics Express · cited 732 times · Not summarized yet
    Abstract

    Many information processing challenges are difficult to solve with traditional Turing or von Neumann approaches. Implementing unconventional computational methods is therefore essential and optics provides promising opportunities. Here we experimentally demonstrate optical information processing using a nonlinear optoelectronic oscillator subject to delayed feedback. We implement a neuro-inspired concept, called Reservoir Computing, proven to possess universal computational capabilities. We particularly exploit the transient response of a complex dynamical system to an input data stream. We employ spoken digit recognition and time series prediction tasks as benchmarks, achieving competitive …

Papers that cite it

  1. Very close Uses this paper in depth
    Single-shot matrix-matrix photonic processor based on spatial-spectral hypermultiplexed parallel diffraction
    C. Luan, R. Davis, Zai-Jun Chen et al. · 2025 · Nature Communications · cited 18 times · Not summarized yet
    Abstract

    The ever-increasing data demand craves advancements in high-speed and energy-efficient computing hardware. Analog optical neural network (ONN) processors have emerged as a promising solution, offering benefits in bandwidth and energy consumption. However, existing ONN processors exhibit limited computational parallelism, and while certain architectures achieve high parallelism, they encounter serious scaling up roadblocks for large-scale implementation. Here, we introduce a spatial-wavelength-temporal hyper-multiplexed ONN processor, which is based on parallel diffractive beam routing. The architecture supports high three-dimensional data, high O(N3) computing parallelism, and is feasible fo…

  2. Very close
    Space-efficient optical computing with an integrated chip diffractive neural network
    H.-H. Zhu, J. Zou, H. Zhang et al. · 2022 · Nature Communications · cited 365 times · Not summarized yet
    Abstract

    Large-scale, highly integrated and low-power-consuming hardware is becoming progressively more important for realizing optical neural networks (ONNs) capable of advanced optical computing. Traditional experimental implementations need N2 units such as Mach-Zehnder interferometers (MZIs) for an input dimension N to realize typical computing operations (convolutions and matrix multiplication), resulting in limited scalability and consuming excessive power. Here, we propose the integrated diffractive optical network for implementing parallel Fourier transforms, convolution operations and application-specific optical computing using two ultracompact diffractive cells (Fourier transform operation…

  3. Very close
    Photonic multiplexing techniques for neuromorphic computing
    Yunping Bai, Xingyuan Xu, Mengxi Tan et al. · 2023 · Nanophotonics · cited 287 times · Not summarized yet
    Abstract

    The simultaneous advances in artificial neural networks and photonic integration technologies have spurred extensive research in optical computing and optical neural networks (ONNs). The potential to simultaneously exploit multiple physical dimensions of time, wavelength and space give ONNs the ability to achieve computing operations with high parallelism and large-data throughput. Different photonic multiplexing techniques based on these multiple degrees of freedom have enabled ONNs with large-scale interconnectivity and linear computing functions. Here, we review the recent advances of ONNs based on different approaches to photonic multiplexing, and present our outlook on key technologies …

  4. Very close
    Neuromorphic Computing Based on Wavelength-Division Multiplexing
    Xingyuan Xu, Weiwei Han, Mengxi Tan et al. · 2022 · IEEE Journal of Selected Topics in Quantum Electronics · cited 236 times · Not summarized yet
    Abstract

    Optical neural networks (ONNs), or optical neuromorphic hardware accelerators, have the potential to dramatically enhance the computing power and energy efficiency of mainstream electronic processors, due to their ultralarge bandwidths of up to 10s of terahertz together with their analog architecture that avoids the need for reading and writing data back and forth. Different multiplexing techniques have been employed to demonstrate ONNs, amongst which wavelength division multiplexing (WDM) techniques make sufficient use of the unique advantages of optics in terms of broad bandwidths. Here, we review recent advances in WDM based ONNs, focusing on methods that use integrated microcombs to impl…

  5. Very close
    Photonic neural networks and optics-informed deep learning fundamentals
    A. Tsakyridis, M. Moralis-Pegios, G. Giamougiannis et al. · 2024 · APL Photonics · cited 50 times · Not summarized yet
    Abstract

    The recent explosive compute growth, mainly fueled by the boost of artificial intelligence (AI) and deep neural networks (DNNs), is currently instigating the demand for a novel computing paradigm that can overcome the insurmountable barriers imposed by conventional electronic computing architectures. Photonic neural networks (PNNs) implemented on silicon integration platforms stand out as a promising candidate to endow neural network (NN) hardware, offering the potential for energy efficient and ultra-fast computations through the utilization of the unique primitives of photonics, i.e., energy efficiency, THz bandwidth, and low-latency. Thus far, several demonstrations have revealed the huge…

How are these papers found?

We compare the title and abstract of this paper with those of every paper summarized on this site, and with the papers Semantic Scholar recommends, those it cites and those citing it (for a much-cited paper, its most recent citations and, from OpenAlex, the most cited ones). Among the prior work and the follow-ups, the papers that use each other in depth, a method it uses and the much-cited follow-ups come a little higher. The levels (very close, close, related) come from the similarity of the texts, not from a judgment of quality; * marks a paper without an abstract, compared by its title. The sentence under a paper is written by an AI from the two abstracts: check it in the papers. A paper marked not relevant by three readers leaves the list.

Updated on 5 November 2026.

The illustrated guide to the papers around a paper