Intro to Ubiquitous Computing



my graduate work was primarily in a field called ubiquitous computing this is a field of computing that's an academic research discipline and it's something that's been very formative in a certain way of looking at user experience and user interaction in modern computer systems and it is also you know maybe a transitional stage as we were getting to cloud computing or maybe cloud computing could be thought of as supporting ubiquitous computing as well and so in this lecture I'd like to give just a little bit of background on ubiquitous cube iqua tiss computing I think it's a useful lens through which you can look at user experience there's also a lot of other reading that can be done to fill in some of the pretty sophisticated thought around this way of thinking about computing so for starters what is ubiquitous computing now we could talk about it just in terms of the words themselves that are involved ubiquitous being a word that means something that's present or appearing everywhere are things that are found all around us we might say that the air is ubiquitous or in this dictionary entry we might say cowboy hats are ubiquitous among the male singers we might say that what we mean by it though when we say ubiquitous computing is we mean a way in which computing becomes imbued in the world around us so that everywhere you go computing is part of the physical mental kind of experiential landscape in which you are participating so in addition to ubiquitous just having this meaning ubiquitous computing itself is an academic discipline and the origins of it as an academic discipline or an academic field of study come from a paper authored by Mark Weiser and Scientific American called the computer for the 21st century this article came out in 1991 which is quite remarkable as it predates you know the iPhone by almost 25 years right yes I'd be right and and that's phenomenal given the state of the world then because the things that Weiser and his through Weiser and through the work that he was doing with his team at Xerox PARC they were envisioning largely big parts of the world that we experience today and that vision that computer for the 21st century came to be the field of ubiquitous computing generally marked by a number of different ways of thinking about competing part of it is a technological trend so Weiser was looking forward and seeing the shrinking of the transistors the Moore's law the increase in speed increasing transistor counts and you know understanding that this was going to bring more power but beyond looking at Moore's law Weiser was also looking into thinking about how this would change the way in which we interacted with computers and he made some predictions about how computers would come to be part of our lives and how we would start to work with them as a result so part of the context that he put computing in was in three ways and I'm going to add a zero wave on top of that because I'm a proper computer scientist and we need to start Lucey zeroes as our accounting for some of the historical context for ubiquitous computing come from viewing computing as a series of waves the zero wave one which Weiser didn't talk about but I'm going to add in here it's basically what I would call computer computing this is a kind of computing that was going on maybe in the 1930s or 1940s just rough timeframes this was a time period in which computers were really theoretic technology things that didn't exist yet but people kind of saw coming in the research labs and the mathematician people who are previously were mathematicians we're starting to become computer scientists without realizing it by thinking about the ways in which these new machines were going to be able to compute what sort of limits were there on the things that were possible to be computed so people like church and touring we're arguing questions about theoretical concepts like does P equal NP or not this question remains one of the great unsolved theoretical computing questions in computer science today it's asking whether polynomial time algorithms and non non polynomial time algorithms whether they're the same class of algorithm so before there were computers there were people working on the fundamental limits of computing in many ways this is analogous to where we are today with quantum computing we have some fits and starts with quantum computing that are happening in research labs but they're all very small computers very limited in their ability to compute and yet there are theoretical computer scientists who are doing an enormous amount of work saying well let's imagine what the world would be like if we had these new quantum computers what kinds of things would be possible to be computed so again these is a theoretical approach to quantum computing because we don't have those computers made maybe we won't but we certainly don't have them yet but again these two ways of thinking about the limits of traditional computing there's a limit of quantum computing are different than what Weiser is talking about because Weiser was really talking about the humans relationship with computing in addition to the fundamental mathematical properties of computing so his proper first wave was mainframe computing the computing that became predominant in the 1960s and 1970s there's an image of a mainframe computer with a printer that was one of the primary ways of getting output from the mainframe very limited in their network capabilities there were huge computers that were in special room with raised floors and air conditioning special power requirements very massive computers that did very simple data processing maybe military context they would be used to calculate ballistic cable business context they might be used to do accounting or to do basic spreadsheet operations that we would think you know our watch could do now our Smart Watch could do now nonetheless there was significant amounts of time and savings in being able to do these things on these large computers and also probably a little bit of an investment in the future and being able to understand computing where is going nonetheless in this first wave there were very few computers in the world only the largest corporations and maybe some military scientific data labs were able to support them in terms of the infrastructure and the personnel and the training that was needed and only those those organizations actually needed the computing example that's the first wave the second wave then could be called desktop computing this was the wave in which Microsoft rate rose to power happened maybe around the 1980s and 1990s again rough boundaries around this time frame and in this environment business applications really drove usage notwithstanding the fact that this was a picture of a computer mostly designed for playing games in this environment the business applications were causing people to have one computer per desk at least in the office space gradually this paradigm of desktop computing began to colonize homes as well and it became typical for homes to have a computer that they would do maybe bring home work from home or do communications spreadsheet kind of town basically they were all wired there were wired in intranet meaning that they had limited networks to an office would be connected through one network a lab would be connected through a network college would be connected through a network but it really wasn't the case that most places had been linked globally to the Internet as we know it today and all these computers were definitely wired the idea of having wireless computing really hadn't arrived yet gradually a new paradigm took over this was a third wave and this is what we would properly call ubiquitous computing this wave started again maybe 2000 lasting to the present I don't think we've really gone past anything that like ubiquitous computing yet and in this stage a different kind of demand was driving the new computing deployments and the new purchases of computers and they were you know driven by information creation by access to the Internet and access to resources online and by communication needing to communicate with other people through instant messaging social media networks things like that so in this picture you see someone with a early phone spell a feature phone but taking a picture of a lion perhaps to share it with someone certainly to record it for their own memory but this was a first you know bow shot across the bow of what you Baker's competing would look like small handheld computer available not for business use not for a spreadsheet not for any economic advantage necessarily but for just what we start to see as part of a lifestyle of using digital technology and it's starting to become something that is not very obvious that we're doing using computing anymore in this in this stage we started to hear terms like lands and lands wide area networks local area networks hands personal area networks largely supported by Bluetooth things like ad hoc networking and definitely this stage was a stage of wireless computing now an important thing that Weiser predicted in this third in this third wave the few Victor's competing ways was that computers would begin to be would begin disappearing now when you first hear this you think oh well they're getting smaller they must be hard to see we don't see them anymore and certainly that was part of what he was saying but a more important part of what he was saying is that the fact that we were using them were starting to fade into the background the idea that these were becoming tools that were working so well that we stopped thinking though of them as being something that we were working with and working on and we started to work through them and started to work with them as if they were an extension of ourselves maybe an analogy to this is something like the way we use a hammer when a hammer is functioning properly and when you are hammering and nailed you don't think much about the hammer itself you use it to drive nails and until the hammer starts to get loose until you miss the nail and hit your thumb you don't really think much about the hammer or another analogy might be driving if you are someone who has a driver's license and you drive regularly you know that the process of driving once you pass the beginner stage becomes one in which you stop thinking about the controls in a way that's a little bit odd if you step back and think about it for a little bit the car itself becomes an extension of your body and when you want to go faster you hit the accelerator and when you want to slow down you hit the brake and you turn you turn the wheel and look in the mirrors and after that initial learning stage these behaviors become second nature and the car becomes disappear of course not functional like not literally it doesn't become invisible it's not Wonder Woman's invisible jet but it becomes like a tool like a hammer such that we begin to work through it we stop thinking about it there's something that we need to manipulate we stop thinking about the steps we need to manipulate that so wise you're anticipated that computers would begin to operate like this we would stop thinking about the fact that we were using a computer and we would begin to work through them and they would become an extension of ourselves I think in many cases we maybe haven't quite achieved that yet and the things that we think of as computers you know using snapchat using instant messaging maybe looking up a location on a map taking photos I think that all still feels very present like we understand we don't forget we're working with computers but that's exactly the point there are plenty of places where we are working with computers where we completely don't notice it anymore the ways in which our cars themselves have become imbued with computing the ways in which our homes and some degrees have become imbued with computing primarily through appliances perhaps the ways in which certain work gets done for us in databases in finance you know in informing us about different conditions through notifications in those places we are starting to stop noticing that computing is present and it is starting to fade from view not because it's invisible physically but because functionally becomes invisible so in the first wave of computing the relationship between people and computers was that there were many many people and one computer this was the mainframe so several people would log on to one computer in order to use it as the second wave came along there was a one-to-one relation there was a desktop computer you would sit down to use it one person one computer in it now a new a third way the navicular ubiquitous computing stage suddenly there are many many more computers per person than there have ever been before and that one person now has music players and cars homes office spaces phones making multiple phones all interacting with their smart house and all these different visions that we have of the way in which computing will be part of the fabric of our lives means that per person there are many many more computers than there ever were before well that another point that much wiser wanted to differentiate his vision of ubiquitous computing from other visions was to draw out how ubiquitous computing was very different than another technology that even back then was hyped up very much which is virtual reality now again we're in a resurgence of the excitement around virtual reality but the idea of ubiquitous computing it you become for short the idea of virtual reality are really very different kinds of visions of the world so ubiquitous computing is mostly about computers leaving the domain of the computer and coming and starting to enter into the human world and so we see in this picture what I wanted to draw out was the fact that the display changes as you turn the device this is something that we've become very customed to but if you think about it that means that the computer is reacting to our world as we turn the device physically the computer responds to that gives us the result that is appropriate for its physical embodiment in the world likewise in the content of this video is in the content of this image as well the map markers are being placed will save the map marker is being placed at where the user currently certainly that's not something that's the price and to some degree that's also the phone responding to the physical location of the device a proper ubiquitous computing vision would say that as the computer went to a certain location it would do work for you on your behalf without you really having to compel it to on the maps not quite doing that but certainly turning the device it's doing work for you without you really thinking about it now augmented reality somewhere in between let me come back to that let me touch base on virtual reality first neck the virtual reality is different than ubiquitous computing because in virtual reality what we're really asking our human to do is to enter into the computer's world so in this image we see the user donning a headset putting it on a haptic position track tracking glove or maybe two and they've entered into a world in which the computer gets to dictate the senses and the views and the interaction as you move your head the computer gives you a new view to make it seem as if you've entered into this virtual world and as you move your hands and the devices different interactions happen in that virtual world which has very little to do with the real world and in fact it's so disconnected from the real world that it's easy for you to pull on a wire and not expect to it's easy to get fatigued in different ways because of the different weights that are being placed on your head and on your neck it's also possible if you're in a more mobile virtual reality setting to run into wall has the relationship between the digital world that you've entered into and the physical world that you're actually in become unsynchronized or just they're just not related to one another necessarily also in virtual reality is the idea of the virtual world itself so this is a picture taken from virtual world system called second life as a new been resurrected of open sim professor crystal Lopes is one of the main developers on this second Second Life this is just a virtual world like you might see in any game it's a user created pathway to a house but this is something this is an example of what you would step into as you're entered virtual reality and then as we move towards talking about augmented reality we can also talk about something which has been come to be known as a mirror world and a mirror world is a virtual reality world that you step into but that is designed to have a relationship to the real world that's kind of one-to-one so this is an example of Google Earth and Google Earth attempts to keep a very synchronized representation of the real world and yet it is a virtual world so you can go into Google Earth you can move around using virtual reality metaphors and interaction techniques but you're not moving through the real world in any sense you're moving through a map just an elaborate three-dimensional now finally in the middle we do have mended reality and augmented reality doesn't neatly fit into your victus computing or virtual reality because with augmented reality typically the way it's envisioned the way it's built out is that you look at the world through a camera on your device and so what you're seeing on your device is an image that's taken from the camera on the other side of the device so you're looking at a screen that's showing you what's beyond be on the other side of the phone the reason why you do this is because as the image is being shown to you on the screen it can be overlaid with virtual information that somehow tied to your location or your viewpoint so this is an example of a virtual reality app that you can hold up and pan around the world that's near you and the application will overlay onto that world little icons and information about what you're seeing on the screen and there are various technologies that support this why I wanted to say that ubiquitous computing was different than virtual reality because the computer enter your world and in virtual reality you entered the computer world in augmented reality it's something like a hybrid the computer has come is acknowledging that it's existing in your world but you're also entering into the computers world maybe this is the boundary between ubiquitous computing a virtual reality a little bit of wool so this summarize computers enter the human's world and ubiquitous computing and humans enter the computer's world in virtual reality with augmented reality beam well a great question is with all of these devices that are out there with computers entering the world that doesn't mean all the sudden that there's no work associated with building the infrastructure of these computing system it has to be maintained they have to be fixed when they're broken software has to be patched and upgraded and you know trainee you have to learn how to use them all these kinds of things have to update them and load them all the things that we've come to know and love about computers Wow what are we going to do to manage all of these computers this third wave of computing that wise you're called ubiquitous computing where does that management going to come from especially the infrastructure on top of the devices themselves well turns out one thing that wiser didn't expect is that this was going to be managed by what I'll call the professional but basically what we've come to know now is cloud computing and cloud computing will be the topic of the next lecture so in summary ubiquitous computing as a paradigm that envisions a particular way that people would relate to computers it was a prediction about scale and the way that the relationship from computers would be many to one one to one and one to many and what that would mean about how we would need to react how we need to interact with computers in order to manage all the many computers that were around us Weiser suggested would have to become very natural for us to work with computers as we used voice and motion and physical layout in order to tell the computer what our intentions are he also brought up a lot of philosophical background about the invisibility of computers and how that becomes an important part of his vision of what ubiquitous computing would look like and all the things that result from those changes as well so that's a nutshell is a very quick introduction to ubiquitous computing encourage you to read the original paper and a lot of the work that's been done around this concept since then thank you for your time you

2 Comments

  1. Saurabh Sharma said:

    Great summary. Thanks for sharing. It seems that we can also map various tech companies in one bucket or other, one bucket being "making computing bend where we bend" (iPhone orientation example) vs "getting inside the virtual world" (VR). Companies like Apple (may be?) are trying to make computers more human. Companies like Facebook seem to be doing just the opposite. Creating virtual worlds, gamifying human behavior and turning us into a different version of ourselves. Your video summary got me thinking. This lecture has been deeply thought-provoking. I loved it!

    May 22, 2019
    Reply
  2. Shankar V said:

    Well laid out. The historical perspective helps

    May 22, 2019
    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *