From BrainScales to Human Brain Project: Neuromorphic Computing Coming of Age



you I would like to thank demanda in particular he was a good friend of mine and I have followed the synapse project pretty much from the beginning and and I was really impressed by the professionalism where it was which this project was was performed and I'm extremely glad to see all these wonderful devices here and I think what we see these days is that there is a community of your morphic computing that really starts to bring your more computing to those who can use it so far it was mostly the PhD students that made the chips that we're working with the systems now for the first time we are ready to give the systems to other people like neuroscience for example but also outside neuroscience busyness finances and and other things and I hope we are pretty much where these two guys have been more than 60 years ago you know them it's shown for Norman and Robert Oppenheimer in front of the machine that was built at the Princeton lab and I quoted a role model for the for the following reason they used state-of-the-art devices vacuum tubes cathode ray tubes mercury delay lines and of course all those devices were replaced a couple of years later and in a way we do the same thing we are still using CMOS for most of our systems it's clear that Siemens is likely to be replaced in particular when it comes to these cognitive architectures but for the time being it's good to use it because it's available but what we concentrate on is the architecture like phone women concentrated on the memory processor architecture we are concentrating on the massively parallel architecture we concentrate on universality we do not want special chips but we want systems that are configurable by the users to solve a multitude of different tasks and I think we are seeing that size matters not in terms of cubic meters but in terms of computational resources is in a very naive way I try to categorize neuromorphic computing systems I would say there are four approaches one is to build on commodity micro processors like ARM cores for example this is what the spinnaker group and also now the human brain project does and that is a soft binary model of new so kids there is the of course what people do here so successfully the custom fully digital approach where you have hard binary models of neural cells there is the custom mixed signal approach that is what we do in our Heidelberg group in brain scales also in HPP where we have physical models of neural circuits and I make this little comment of being accelerated which I will explain in my talk there are custom sub-threshold analog cells so the purely analog approach which is very close to biology in terms of pulse sight and pulse which so that is typically something you would use if you want to interface the real biological substrates and there is also a new development which are called custom hybrid Qualcomm is an example where you really merge neuromorphic cores with conventional computing architectures do they have anything in common this seems to be totally different well I think what the has in common is that they are all definitely massively parallel there or use asynchronous communication and very very very very important they are all highly configurable because we do not really know exactly how to do brains back computing we better make this system very flexible so there's a very important statement I want to make here is that the complementarity of the approaches is extremely important I think it's too early to just focus on one it's great that we have this complementarity and this is why we need meetings like this so that we see what our colleagues do in Europe there has been a lot of activity in neuromorphic computing one branch now focuses on the human brain project in short HBP there are three input projects that form the basis of the current HPP that is of course the Blue Brain Project which is not brain inspired computing but integrating neuroscience data into very detailed biological models in simulating them on supercomputers based on those simulations and based on other inputs we now start to build computing architectures and there were three projects in Europe that contributed to the basis of HPP that is the facets project the brain scales project and the spinnaker project in consequence in HPP I will not describe the entire project here because I have only 20 minutes I will focus on the neuromorphic computing in HBP have to again complementary concepts of your amorphic computing the many call processor system and the physical model system the many core processor system is run by Steve Ferber and Manchester you know him well he uses many clock rather simplified ARM processors in address based small packet fast asynchronous low-density communication protocol and is effectively running in real time so this is really a neural simulator that is very well suited for example to run with robots or with physical sensors and actuators the system that we build in Heidelberg is what we call a physical model system so we do local analog computation but binary again asynchronous and continuous time not clock continuous time high-density communication and the essential feature here there is is an accelerated system which I will explain a bit more in detail as I have only 20 minutes I will now focus on the physical model approach that we do in the human brain project and that's based on the previous brain scales project there are ten rationals that are behind our work and that I consider extremely important first of all it is a mixed signal approach we do local analog computing and binary spike communication that's just like our brain neurons are analog devices action potentials are binary they all look the same but they have a continuous time scale they are not synchronized of course our approach is driven by architecture we really want to find interesting architecture we do not work on devices we use the devices that are available we build on what we call very high neuron input count more than ten or fifteen thousand and this is to mimic cortical neurons which we think is very important if you want to do cortical information processing we very much build in configurability I said that already that concerns the cells but in particular also the connections so basically what we build is a sea of neurons and synapses pretty much like an FPGA in the digital world and you as a user can decide how you want to wire them up and what kind of parameters we want to set so this is universality which is an important aspect scalability of course is the key we always start with chips like everybody else but then we want to scale up to wafers and large systems we want to have an accelerated system factor 10,000 faster than biological real time for reasons that I will explain plasticity is important upgrade ability was unchanged system architecture we spend a lot of time building systems like that so if you have the next chip generation you do not want to redevelop all the infrastructure so to be able to upgrade the system with latest neuroscience inputs is also very important hybrid operation interfacing with conventional computers non-expert user access really very important we want Eddy neuroscientist student compliant issed whoever wants to do neuromorphic computing must be able to use those systems and the objectives are record circuit exploration and – very structure and to study in particular temporal dynamics the chips we build they look like all other chips there is nothing special but if you look to this chip you maybe see some kind of structure looks like a tennis court ok so there are the two fields of the Tennis Court Oath are the synaptic arrays in between a very thin strip of red color which you see those are the neurons and the message you can take home from this talk a very important one is if you build these systems connectivity is the challenge the neurons are very fancy circuits but they don't use in your real estate you can make them very complicated this is not a lot of problem or occupying the space in this kind of architecture the real challenge is to connect those neurons in particular with the high input count that we have in mind so all the gray stuff there the tennis courts those are the synapses and the red stuff around the chip are the connections now what you see is scalability is a problem here if you want to go off chip and connect to the next one you need these silly little bond wires which you see here on this on this photo so what we did right from the beginning is that we said well we don't want bond wires we want to do wafer scale integration we want to leave the wafers intact and have these little elementary chips the high-end chips on the network and of course all the synapses and the connections that requires a lot of technology development but having a post-processing layer to build networks which are as large as the wafer so we have solved all these problems and today we have wafer based systems that carry 50 million neurons and and about 200,000 over 50 million synapses and about 200,000 neurons with the plasticity mechanism short term and long term plasticity highly configurable neurons and synaptic parameters and an acceleration factor of 10,000 beyond biologic the real time if you build a chip or a wafer then you're not done you have to build a system as you see very nicely here this is a computer drawing of our system you see that the waiver is connected from any printed circuit boards so at the bottom you see piles of FPGAs that be used to input an output data you have to bring power in data in and data out and a lot of effort went into building this system this is a computer drawing so in real life the system is ready now we are having these systems and for the human brain project we are now building 20 of them this is one of the board which is about that size and under this cooling copper cooling element is one of these 20 centimeter or 8 inch wafers so the system that's currently now under construction in Heidelberg as a physical model system is the one you see here so this looks rather boring it's a set of racks like any other computer but it carries this this is 20 wafers and and another thing that looks boring and the middle are conventional computers there is a pile of really state-of-the-art off-the-shelf computers so why I am i referring to them the reason is that they are very low latency very high bandwidth local computers that we can use to run closed-loop experiments those are the systems that provide the data that the system is going to analyze I have to show you this picture so we build a little container at bag of our Institute at the back yard where we have the control room and seminar room and workshops and in the blue corner that you see here we build what we call the HP pnp m1 which is a very complicated abbreviation means neuromorphic physical model phase 1 system and I will show you the upgrade plans and one of my last slides so what is the performance of the system let me refer to energy and time and then give you some examples on experiments that we run with the still single chip systems I have to say the waiver are now being commissioned and what is the energy often people talk about power I think energy is the better thing because energy is what you pay to your electricity company and so energy you have to normalize of course to some kind of time or better to an elementary operation and an elementary operation that is very useful I think in your morphic computing is for example a spike or a synaptic transmission if you look for the price you pay for synaptic transmission in our and your brain it's 10:00 a.m. to Jule if you go to high performance computers like Henry Markram is very detailed model it's a Joule so there are 14 orders of magnitude the systems of HPP the spinnaker and the brain scale system they sit at 10 to the minus 8 and tender – tendril this is these are real numbers including all the overheads and inefficiencies of power supplies cables FPGAs to read or in control of the system this is not a measurement done on a single synapse is it really using the entire system and dividing by the number of elements time scales I consider even more important because simulations on supercomputers which are very useful for neuroscience because they are so configurable have a big problem that if you want to simulate a day for example of learning and plasticity it takes typically on the supercomputer effect a thousand more and a thousand days is three years that's the time of a PhD thesis so if you want to repeat experiments in a gross loop for example by reconfiguring the system it's not a Stan hopeless enterprise if you ever want to attack development which i think is a key i mean to attack develop many of these systems is really one of the major scientific issues also for applications later then that is totally unacceptable it would take thousands of years to do this this is why we build this X accelerated system which at the same time can run synaptic coincidences at the level a couple of nanoseconds and then running system for tens of seconds of thousands of seconds or hours you can even address developmental timescales usability is another thing as I said you have to build a system that is useful by non-experts so we did a lot of work in the previous facets and brains guides project to develop configuration languages very much like people do here in IBM so there is a very nice parallelism which you see we develop the pine language which is a bit like very locked for neuromorphic computing it's there to describe networks then we have a place in route system we can simulate our system and we have all the infrastructure for remote access to this neuromorphic system at the last human brain project summit in Heidelberg which was a month ago or so for the first time we demonstrated this remote access from our lecture hall where Andrew Davison from chief Suhr Evette gave the talk and he submitted jobs to the neuromorphic system sitting in Manchester and in Heidelberg and as you say that as you see the laptop is somewhere in Portugal so you can sit on the beach and hopefully then at some point run your experiments on this large scale neuromorphic systems applications I will spend now rest of my time with applications there are many many applications now we have published quite a few papers let me just go through three or four examples one is reverse engineering biological systems so typically what you can do is you know from biological data that there are some concepts which are realized like for example winner-take-all circuits D correlation units lateral inhibition and things like that this is an example where we copied and that is work not done by our group but by a neuroscientist mukesh ma current martin a verb with that time we're at the branchline centre in berlin and they had a model for the olfactory system of insects which seems to be very special but it it's an interesting one because what it does it does multivariate data analysis so you have these the sensors that react to certain chemicals and then the combination of those chemicals that is then assigned to certain flower types for example roses and tulips and whatever this is what the insect does so there is this circuit which we can even for a neuroscientist is possible to implement that on our system using this pine language this is a typical experiment which we do where you see spike patterns you will see more spike patterns in real life here on the IBM system we do the same thing the only thing I would love would like to point to is at the bottom of this raster plot the little dots are the spikes you see the input data okay this is this doesn't really work very doesn't show so at the bottom there this is sort of very bizarre and very randomly looking dots those are the input data and you have to find the correlations and the associations in this input data and of course you have to go through a training step left is before training right is after training and what you have to look at is at the upper corner of the right plot there are these two rate spikes arising and this is a demonstration that the learning is actually broke why is that interesting well of course you can do that on a laptop there is nothing special about doing multivariate data classification but it has interesting features one is the low energy the other is there the high speed but there is one other thing which is also these systems are able to work with imperfect circuits these are analog circuits and this is a typical characteristic of neurons its input rate versus output rate and it should be linear and you see it is more or less linear but it's a disaster because the slope is different from neuron to neuron and there are also huge offsets so the variability is typically 20 to 30 percent of course by using the configurability of circuits we might calibrate them you make them all equal that's imprinted we could do but it's a huge effort but what we have shown in this papers if we don't do that if you use uncalibrated circuits and and population coding we can solve the problem just by using more neurons okay two more applications closed loop experiments we are now running closed loop experiments integrating the wafer into a into this computer system we'll use all the high bandwidths that we have to run the experiments and the typical experiments that we now do is that we have a simulated environment on the computer simulated Actros and sent and and census and and then we have in the environment for example a force field which here happens to be what the velocity is proportional to the force K and the distance and the task is to track a point or to move it to a certain position in a closed loop experiment and that now starts to work very well using plasticity is another thing typical example that I can shortly demonstrate here or show you is the localization of sound sources you know that Alex for example bullets are able to localize sound sources by measuring the phase difference between the left and the right ear there is a neural model for that we can implement that on our system using stdp we can run experiments here you see the learning process the color plot there at the left corner is the learning process the change of the synaptic weights and the all the bird that's echolocation with the precision of roughly 100 milliseconds because of the acceleration we can do that with typically 10 nanosecond precision that means we have a face detector with imperfect synapses and neurons with big variability that works to the precision of nanoseconds which for example has a very interesting application in automotive for example you can use this to locate the ultrasound sources stochastic computing is a very very interesting thing driven not so much by biology but by theory ok you have all seen this picture if you look at it you see there are two solutions to the to the riddling here you can see either a duck or a rabbit but never Bo's at the same time and there is a theory behind this which says what we do in our brain is what we call stochastic sampling so somewhere in your brain and you learn that by looking at such animals before you develop a stationary probability distribution and what we have shown together with others is that you can implement such probability distribution in your amorphic circuits and that you can even understand the flipping time between the two images this is the gamma distribution the one you have s you see there on the right corner so take some time to flip between the dark and the rabbit and we have shown that there is kind of you can actually demonstrate that you understand this process and you can use it for computing using these systems that we have as a short remark how can you do stochastic inference with deterministic neurons because our neurons are deterministic systems they are not quantum mechanical or anything like that well what we do is we use the background noise in the circuit to provide the stochasticity and there is a paper out there which you can look at this is an example where you where you let the network explore the space of three numbers for example and like you always sitting either at the duck or the rabbit here the system sits either at the 0 the 4 or the 3 let me shortly comment on the road map of the future HPP has been reviewed recently in particular for the next plan which reaches out to 2023 this is a book which I I have only one copy here but of course you can look at it on on the web and so we have several upgrades one is to provide what we call algorithmic synaptic processing so we want to put RISC processors on each of our little chips to be able to implement plasticity mechanisms the other is implementing multi compartment neurons prototypes are available now that means we have dendritic branching a signal dispersion which are passive features but also dendritic spikes and back propagating action potentials this is a very important thing for temporal sequence learning there is a plan in this book I know that you cannot read it now but you can look to the book on the web and what you will find is our roadmap it reaches out of September 2022 here and what we want to build is a 5,000 wafer system with all the features in particular the much improved plasticity mechanism implementation by these plasticity processors for which we have a prototype now let me close by making one last comment we are also going public again this is what I see IBM is doing now which is really important we have these big systems which are in a way the supercomputers of neuromorphic computing but it's important that people start to play with is also outside the big projects so what we do is we have these little chips individual chips no not wave for chips both groups have that the spinnaker group enter Heidelberg crew we are putting them on little USB boards and we are going to sell them as of next year we are now setting up a Kickstarter activity to get this financed and it comes with a complete development set with software and this little hardware thing which you can plug into your laptop it has analog input analog output digital input output even it's Bluetooth and wireless LAN and both on board so this is my conclusion your remote in computing I think is a very consistent concept for non fun moment and also non touring computing architecture it's accessible to available technologies that's the important thing for the time being like phenomen used to vacuum tubes we are using CMOS so once there are better components available like resistive memory for example we are ready to implement it but at the moment we do not care about it we watch the market and once it's there we will use it the key features are universality scalability for polar ones I've shown that power efficiency speed and most importantly the ability to learn the accelerated operation of the system that I showed to my knowledge is the only known approach to bridge all Prime skills relevant for circuit dynamics in the very last slide this one I like very much HDPE has been criticized a lot some of you may know that and so I'm glad to show you that the consortium is actually a very lively and active one and it's a big consortium this picture has been taken in Heidelberg at the the HPP 2014 summit so you see this is not even all people use part you see a good fraction of the project and the amazing thing and this is a mystery to me if you look to the shape of this group okay does it remind you of something it looks a lot like Switzerland for those of you who know the European map but that's a completely a complete coincidence of course thank you very much you

7 Comments

  1. James Elger said:

    If you want your models to be accurate, figure out the electrochemistry to the mole-e.

    June 29, 2019
    Reply
  2. Amir said:

    Somewhere at the end mentioned, there will be NM chips on boards with USB port and …, according to the date of this video, I guess this should be out by now. Any news out there about it (link to the kickstarter page maybe ?) ? is the board available on the market ?

    June 29, 2019
    Reply
  3. Joseph Leftwich said:

    Good thought

    June 29, 2019
    Reply
  4. Christopher Harris said:

    Great talk. Hope to hear more. Some sound problems but nothing too bad

    June 29, 2019
    Reply
  5. Ivan.joan Roijals-Miras said:

    Very interesting in computational terms. But I fail to see how this is any close neuromorphic

    June 29, 2019
    Reply
  6. electrodacus said:

    Neuromorphic Computing is an interesting new technology that I will be sure to get involve with in the near future.

    June 29, 2019
    Reply
  7. Paleologic said:

    fascinating!

    June 29, 2019
    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *